A user behavior sequence recommendation method, system and storage medium based on the combination of large model and graph neural network

Abstract

The present invention relates to a method, system and storage medium for recommending user behavior sequences based on a combination of a large model and a graph neural network, and relates to the technical field of behavior sequence recommendation. The method aims to solve the problem that existing methods are difficult to deeply integrate the text feature extraction capability of a large language model with the dynamic modeling capability of sequence recommendation, and are difficult to achieve accurate and comprehensive recommendations. The large-model-based item text representation learning module of the LLM‑SR model of the present invention extracts high-quality semantic embeddings of item texts through a large language model, combines linear layer mapping and a multi-layer perceptron to generate item representations adapted to recommendation tasks; the graph neural network-based sequence representation learning module uses a session graph and a global graph to capture the local sequential relationship and cross-session co-occurrence relationship of items, and generates a comprehensive representation of user behavior sequences; the predicted score calculation module calculates the item score by taking the dot product of the user sequence representation and the candidate item embedding and sorts them to generate the final recommendation result.

Description

Technical Field

[0001] The present invention relates to the technical field of behavior sequence recommendation, and in particular to a method, system and storage medium for recommending user behavior sequences based on a combination of a large model and a graph neural network. Background Art

[0002] Recommendation systems play an important role in personalized services and information filtering. Sequence recommendation, as a key technology, analyzes users' historical behavior sequences, explores users' dynamic preferences, and provides users with more accurate recommendations. However, traditional sequence recommendation methods often only focus on the temporal order and co-occurrence relationship of user behavior sequences, and lack full utilization of the text semantic features of items, resulting in insufficient performance of the model in long-tail item recommendations and cold start scenarios. In recent years, large language models have demonstrated their application potential in recommendation systems with their excellent semantic understanding capabilities in the field of natural language processing. The high-quality text embeddings generated by pre-training in large models can effectively extract the semantic features of items and provide a new source of information for recommendation systems. However, how to deeply integrate the text feature extraction capabilities of large language models with the dynamic modeling capabilities of sequence recommendations to give full play to the advantages of both is an important challenge in current research. Summary of the Invention

[0003] The technical problems to be solved by the present invention are:

[0004] Existing methods find it difficult to deeply integrate the text feature extraction capabilities of large language models with the dynamic modeling capabilities of sequence recommendations, making it difficult to achieve accurate and comprehensive recommendations.

[0005] The present invention is to solve the above technical problems using the following technical solutions:

[0006] The present invention provides a user behavior sequence recommendation method based on a combination of a large model and a graph neural network, comprising the following steps:

[0007] Collecting user-item interaction information data, preprocessing the user-item interaction information data to obtain a sequence set, collecting item text information data, and preprocessing the text information data to obtain a text information set;

[0008] The sequence set and the text information set are input into the LLM-SR model, which includes a large-model-based item text representation learning module, a graph neural network-based sequence representation learning module, and a prediction score calculation module; the large-model-based item text representation learning module extracts high-quality semantic embeddings of item texts through a large language model, and combines linear layer mapping and multi-layer perceptron to generate item representations adapted to recommendation tasks; the graph neural network-based sequence representation learning module uses session graphs and global graphs to capture the local sequential relationships and cross-session co-occurrence relationships of items, and generates a comprehensive representation of user behavior sequences; the prediction score calculation module calculates and ranks item scores through the dot product of user sequence representations and candidate item embeddings to generate the final recommendation results.

[0009] Furthermore, the multi-layer perceptron in the large-model-based item text representation learning module is a three-layer multi-layer perceptron, each layer uses a ReLU activation function to introduce nonlinearity, and a Dropout mechanism is added after the first and second layers.

[0010] Furthermore, the linear layer in the large-model-based item text representation learning module linearly maps the input features through a weighted matrix and a bias vector based on a linear transformation method to achieve dimensionality adjustment and feature conversion of the embedding space.

[0011] Furthermore, the functional implementation process of the sequence representation learning module based on graph neural network is as follows:

[0012] The session graph is constructed as follows: Each session s is constructed as a directed session graph The point set The vertices in are the vertices in session s, and the edge set Any edge in Represents the order of items in the sequence, Indicates that item i is interacted before item j;

[0013] For node v i Representation Indicated by the upper layer and neighbor nodes Features Weighted aggregation is obtained, namely:

[0014]

[0015] The weight α ij is the attention weight calculated based on the correlation between nodes, which is used to dynamically adjust the influence of neighbor nodes on the update of the target node. ij For node v i and v jThe relevance score is combined with the feature representation of the item embedding representation and Through nonlinear changes of the learnable parameters W4 and the attention vector a, we can obtain:

[0016]

[0017] Global Picture The construction method is: each node v in the graph i Represents an item. If the item v i and v j co-occurs in at least one user session, then i and v j Create an edge (v i ,v j ), the edge weight is defined by the co-occurrence frequency between items, that is:

[0018]

[0019] where freq(v i ,v j ) is v i and v j The number of co-occurrences, freq(v i ) and freq(v j ) are items v i and v j Number of occurrences;

[0020] Information propagation aggregates information of neighbor nodes through weighted sum based on attention mechanism; neighbor features The weighted aggregation formula is:

[0021]

[0022] The importance of neighbor nodes is determined by the conversation context-sensitive weight function π(v i ,v j ) calculation, which is used to distinguish the importance of different neighbors to the current item, and the weight is:

[0023]

[0024] in is the feature vector obtained by averaging the pooled items in the current session, namely:

[0025]

[0026] Among them, w ij is the weight between items, ⊙ represents element-wise multiplication, || represents concatenation, W5 and q1 are learnable parameters;

[0027] The information aggregation step represents the feature of the current item h v and features propagated from neighbors Perform fusion and generate new item representations through nonlinear transformation, namely:

[0028]

[0029] Where W6 is the linear transformation weight matrix and ReLU is the activation function;

[0030] The stacked multi-layer propagation mechanism captures high-order neighbor information. Each layer is generated by calculating the item representation and neighbor representation of the previous layer. The representation of the k-th layer is defined as:

[0031]

[0032] First, we learn high-level feature representations of items through the global graph and combine them with session features to generate a comprehensive representation of the items. The final representation of each item is:

[0033]

[0034] Representing items and reverse position indicates p l-i+1 Perform splicing and generate a new representation through nonlinear changes, namely:

[0035]

[0036] Where W7 and b4 are learnable parameters;

[0037] Finally, the weighted representations of the items are combined into the final session representation, i.e.:

[0038]

[0039] Weight β i Obtained by dynamic calculation of the soft attention mechanism:

[0040]

[0041] W8, W9, q2, and b5 are learnable parameters;

[0042] The preliminary session feature s′ is calculated using the average representation of all items in the session, namely:

[0043]

[0044] Where l is the sequence length.

[0045] Furthermore, the function implementation process of the score prediction of the prediction score calculation module is as follows:

[0046] The initial embedding vector of the session representation S and the candidate item The dot product of the candidate item v is calculated i The probability of becoming the next click recommendation is normalized by SoftMax, that is:

[0047]

[0048] m is the size of the candidate item set.

[0049] Furthermore, the cross-loss entropy function used by the LLM-SR model is:

[0050]

[0051] where y i ∈{0,1} is the true label, is the model prediction item v i probability of click.

[0052] Furthermore, step one includes the following steps:

[0053] Collect user-item interaction data, determine whether items and users interact and score them, filter out samples where users and items have been rated less than five times, sort items chronologically, and then filter out user tags to form a sequence file. Then filter out sequences with a length of 1 in the sequence file and split the sequence into multiple subsequences. The last item in the sequence is used as the next interaction item in the sequence, and all the interaction items before the item are treated as a sequence to obtain a sequence set.

[0054] Collecting item text information data and preprocessing the text information data, including text cleaning, splicing text information of different tags, and generating a text information set;

[0055] Finally, the items in the sequence set that do not contain text information are filtered out.

[0056] The present invention provides a user behavior sequence recommendation system based on the combination of a large model and a graph neural network. The system has a program module corresponding to the steps of the method described in any of the above technical solutions, and executes the steps in the above-mentioned user behavior sequence recommendation method based on the combination of a large model and a graph neural network during runtime.

[0057] The present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to implement the steps of the user behavior sequence recommendation method based on the combination of a large model and a graph neural network described in any one of the above technical solutions when called by a processor.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] The present invention aims to explore and solve key issues in the deep integration of large language models and sequential recommendations, give full play to the advantages of large language models in semantic understanding and text feature extraction, and combine the ability of sequential recommendations to accurately model the dynamic changes in user behavior to build an efficient and accurate recommendation framework. By designing a high-quality method for generating item text embeddings and a sequence representation learning mechanism, the present invention achieves the collaborative modeling of semantic features and sequence features, improving the performance of the recommendation system in long-tail items and cold start scenarios. At the same time, by optimizing the adaptation method of the large model and reducing the computational cost, the accuracy, diversity of the recommendation results and the generalization ability of the system are further enhanced, providing users with more accurate and comprehensive personalized services. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of the LLM-SR model architecture in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of an item prompt template in an embodiment of the present invention;

[0062] Figure 3 Graph showing the text length ratios of five datasets in an embodiment of the present invention;

[0063] Figure 4 Graph showing the results of a learning rate sensitivity analysis experiment in an embodiment of the present invention;

[0064] Figure 5 1 is a graph showing the experimental results of sensitivity analysis of embedding representation dimensions in an embodiment of the present invention;

[0065] Figure 6 Parameter sensitivity heat map in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to enable those skilled in the art to better understand the present invention, exemplary embodiments or examples of the present invention will be described below with reference to the accompanying drawings. Obviously, the described embodiments or examples are only some of the embodiments or examples of the present invention, and not all of them. Based on the embodiments or examples of the present invention, all other embodiments or examples obtained by those skilled in the art without creative work should fall within the scope of protection of the present invention.

[0067] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0068] Specific implementation plan 1: Figure 1As shown, the present invention provides a user behavior sequence recommendation method based on a combination of a large model and a graph neural network, comprising the following steps:

[0069] Collecting item text information data, preprocessing the collected data, and generating a sequence set;

[0070] The sequence set is input into the LLM-SR model, which includes a large-model-based item text representation learning module, a graph neural network-based sequence representation learning module, and a prediction score calculation module; the large-model-based item text representation learning module extracts high-quality semantic embeddings of item texts through a large language model, and combines linear layer mapping and multi-layer perceptron to generate item representations adapted to the recommendation task; the graph neural network-based sequence representation learning module uses session graphs and global graphs to capture the local sequential relationships and cross-session co-occurrence relationships of items, and generates a comprehensive representation of user behavior sequences; the prediction score calculation module calculates and ranks item scores through the dot product of user sequence representations and candidate item embeddings to generate the final recommendation results.

[0071] The sequence recommendation model based on large model and graph neural network proposed in this paper is Large Language Model-based Sequential Recommendation (LLM-SR), and its framework is as follows Figure 1 As shown. In the sequence recommendation module based on the large model, the text information of the item (such as title, category and brand) is the key input data. In order to make full use of the large model's ability to understand and represent the text information of the item, the present invention designs a text generation and embedding extraction process based on the prompt template (Prompt). Specifically, first, the attributes such as the title, category and brand of the item are constructed into a standardized input through the predefined Prompt template, and input into the large model to generate normalized text information. Then, the high-quality semantic embedding representation is directly obtained through the embedding extraction layer of the large model, and mapped to the target text space through the linear layer. Subsequently, the mapped text representation is spliced ​​with the randomly initialized item ID representation, and input into the multi-layer perceptron for nonlinear feature learning, and finally an item embedding representation suitable for the recommendation task is generated. The advantage of the method of the present invention is that the large model is responsible for text generation and embedding extraction at the same time, avoiding the information loss that may be caused by cross-model transmission, while making full use of the embedding extraction capability of the large model, without the need to additionally design a complex embedding learning module, and ensuring the semantic quality of the embedding.

[0072] The Prompt template is designed to convert the text information of the item (such as title, category, brand) into structured input, clarify the task objectives, and guide the large model to generate standardized output. Figure 2 shown.

[0073] Specifically, the item text information is generated in the format of {[item ID] Title, Category, Brand}, and then inserted into the predefined template to obtain Pm i The constructed Prompt is input into the large language model, and the large model generates standardized output according to the Prompt prompt. Then, the high-quality semantic embedding representation F is directly obtained through the embedding extraction layer of the large model. i , the whole process is formalized as:

[0074] F i =LLM(Pm i ).#(3-1)

[0075] The text semantic representations generated by large models are usually high-dimensional vectors, which may not be consistent with the embedding space of the recommendation system when used directly. Therefore, they need to be projected into the specific text embedding space of the item through linear mapping.

[0076] The graph neural network-based sequence representation learning module of the present invention can not only comprehensively capture the global and local information in user behavior, but also enhance the saliency modeling capability of item positions in the sequence, providing more accurate prediction results for recommendation tasks.

[0077] Specific implementation plan 2: This implementation plan is based on the multi-layer perceptron layer in the item text representation learning module of the large model, and generates high-quality item embedding representation through nonlinear learning to ensure that it contains both semantic features and has the adaptability to recommendation tasks. Specifically, the item text representation after linear mapping and randomly initialized item IDs represent X i After splicing, the nonlinear features are further learned through the multi-layer perceptron to generate the final item embedding representation h that is suitable for the recommendation task. i This implementation implements a three-layer multilayer perceptron. Each layer uses the ReLU activation function to introduce nonlinearity. To prevent overfitting, a Dropout mechanism is added after the first and second layers:

[0078]

[0079] h i =W3·ReLU(W2·(ReLU(W1·FX i +b1))+b2)+b3.#(3-4)

[0080] Where W1, W2 and W3 are weight matrices, b1, b2, b3 are biases, and ReLU is the activation function.

[0081] The rest of this embodiment is the same as the specific embodiment 1.

[0082] Specific implementation plan three: The linear layer in the large-scale model-based item text representation learning module uses a linear transformation method to linearly map the input features through a weighted matrix and a bias vector to achieve dimensionality adjustment and feature conversion in the embedding space. The mapping formula is:

[0083]

[0084] Where W is the linear transformation matrix and b is the bias vector.

[0085] The rest of this implementation plan is the same as the second specific implementation plan.

[0086] Specific implementation plan 4: The functional implementation process of the sequence representation learning module based on graph neural network is as follows:

[0087] The session graph is constructed as follows: Each session s is constructed as a directed session graph The point set The vertices in are the vertices in session s, and the edge set Any edge in Represents the order of items in the sequence, Indicates that item i is interacted before item j;

[0088] The feature aggregation process of the conversation graph aims to iteratively update the feature representation of the node through the graph neural network to capture the local relationship and sequential dependency between items in the conversation. Specifically, for node v i Representation Indicated by the upper layer and neighbor nodes Features Weighted aggregation is obtained, namely:

[0089]

[0090] The weight α ij is the attention weight calculated based on the correlation between nodes, which is used to dynamically adjust the influence of neighbor nodes on the update of the target node. ij For node v i and v j The relevance score is combined with the feature representation of the item embedding representation and Through nonlinear changes of the learnable parameters W4 and the attention vector a, we can obtain:

[0091]

[0092] The Global Graph is the opposite of the Session Graph. It is an undirected weighted graph constructed based on the co-occurrence relationship between items and is used to capture the global interaction patterns of items across sessions. The construction method is: each node v in the graph i Represents an item. If the item v i and v j co-occurs in at least one user session, then i and v j Create an edge (v i ,v j ), the edge weight is defined by the co-occurrence frequency between items, that is:

[0093]

[0094] Global item representation learning aims to propagate features through the global graph structure, capturing cross-session item transition information and thus improving the performance of recommendation systems. This approach uses a graph neural network framework and incorporates a graph attention mechanism to model the importance of neighbor information. It consists of two steps: information propagation and information aggregation.

[0095] where freq(v i ,v j ) is v i and v j The number of co-occurrences, freq(v i ) and freq(v j ) are items v i and v j Number of occurrences;

[0096] Information propagation aggregates information of neighbor nodes through weighted sum based on attention mechanism; neighbor features The weighted aggregation formula is:

[0097]

[0098] The importance of neighbor nodes is determined by the conversation context-sensitive weight function π(v i ,v j ) calculation, which is used to distinguish the importance of different neighbors to the current item, and the weight is:

[0099]

[0100] in is the feature vector obtained by averaging the pooled items in the current session, namely:

[0101]

[0102] Among them, w ij is the weight between items, ⊙ represents element-wise multiplication, || represents concatenation, W5 and q1 are learnable parameters;

[0103] The information aggregation step represents the feature of the current item h v and features propagated from neighbors Perform fusion and generate new item representations through nonlinear transformation, namely:

[0104]

[0105] W6 is the linear transformation weight matrix used to integrate different features, and ReLU is the activation function used to increase the nonlinear expression capability of the model.

[0106] In order to capture higher-order neighbor information, a multi-layer propagation mechanism is used to capture high-order neighbor information. Each layer is generated by calculating the item representation and neighbor representation of the previous layer. The representation of the kth layer is defined as:

[0107]

[0108] Sequence representation learning generates high-quality conversational representations by combining global and conversational context. Core methods include the fusion of global and conversational features, reverse position embedding, and the attention mechanism to model item importance.

[0109] First, we learn high-level feature representations of items through the global graph and combine them with session features to generate a comprehensive representation of the items. The final representation of each item is obtained by combining the global features and session features using an addition operation:

[0110]

[0111] Then, since the length of the conversation sequence is not fixed, the reverse position embedding can capture the distance information between the predicted item and the current item. Compared with the forward position, the reverse position embedding is more effective. This paper uses the reverse position embedding to capture the relative importance of items in the conversation, and calculates the contribution of each item to the conversation representation through the attention mechanism. Specifically, the item representation and reverse position indicates P l-i+1 Perform splicing and generate a new representation through nonlinear changes, namely:

[0112]

[0113] Where W7 and b4 are learnable parameters.

[0114] Finally, the weighted representations of the items are combined into the final session representation, i.e.:

[0115]

[0116] Weight β i Obtained by dynamic calculation of the soft attention mechanism:

[0117]

[0118] W8, W9, q2, and b5 are learnable parameters;

[0119] The preliminary session feature s′ is calculated using the average representation of all items in the session, namely:

[0120]

[0121] Where l is the sequence length.

[0122] The rest of this embodiment is the same as the specific embodiment 1.

[0123] Specific implementation scheme 5: The function implementation process of the score prediction of the prediction score calculation module is as follows:

[0124] The prediction score calculation module uses the session representation S and the initial embedding vector of the candidate item Generate recommendation probability. Through the session representation S and the initial embedding vector of the candidate item The dot product of the candidate item v is calculated i The probability of becoming the next click recommendation is normalized by SoftMax, that is:

[0125]

[0126] m is the size of the candidate item set.

[0127] The rest of this implementation plan is the same as the specific implementation plan four.

[0128] Specific implementation plan six:

[0129] The LLM-SR model uses the cross-loss entropy function to optimize the prediction results, namely:

[0130]

[0131] where y i ∈{0,1} is the true label, is the model prediction item v i The click probability of . This loss function measures the predicted probability distribution The difference from the target distribution Y.

[0132] The rest of this implementation plan is the same as the specific implementation plan five.

[0133] The pseudo code for the LLM-SR algorithm is shown below.

[0134]

[0135]

[0136] The input of the sequence recommendation algorithm based on the large model is the item information T = {T1, T2, ..., T n}, sequence set S=[S1,S2,…,S n ], the maximum number of model training times n_epochs, batch size B1>0, learning rate α>0 and learnable parameters Θ. The output is the predicted score of the candidate item set

[0137] Lines 1-7 of the algorithm describe the item text representation learning module based on the large model, whose main goal is to generate high-quality initial representations of items. This process first converts the item's text information (title, category, brand) into a standardized input form through a predefined Prompt template, and then inputs it into the large model to generate a semantic embedding representation. Next, the generated semantic embedding is linearly transformed to map it to the target specific space and concatenated with the randomly initialized item ID embedding. Finally, a multi-layer perceptron is used to further learn nonlinear features to generate an item target representation h suitable for subsequent recommendation tasks. i .

[0138] Lines 8-12 of the algorithm describe the process of constructing the session graph and the global graph. In this process, the session graph is first constructed based on the item interaction sequence of each session. This graph represents the local relationship between items in a conversation. Then, by calculating the co-occurrence frequency and relationship of items, the co-occurrence relationship set cr for each item is updated. i Finally, the co-occurrence relationships of all items are aggregated to construct a global graph across sessions.

[0139] Lines 13-24 of the algorithm describe the implementation details of the graph neural network sequence representation learning module and the rating prediction module. The specific process includes: randomly sampling small batches of data from the session data, and learning the session representation h of each item through the session graph. s , capture high-order cross-session relationships through the global graph and generate a global representation h of the item g,(k) Next, the session representation and the global representation are combined and the sequence representation s is obtained through attention mechanism learning. Then, the sequence representation is used to predict the rating of the candidate items and calculate the loss function Finally, the Adam optimizer is used to update the model parameters, iteratively train until convergence, save the final model parameters and generate a prediction score.

[0140] The user behavior sequence recommendation method (algorithm) proposed in the present invention, which is based on the combination of a large model and a graph neural network, is the underlying technical core of the present invention. Various products can be derived based on the algorithm.

[0141] Based on the method proposed in the present invention, a user behavior sequence recommendation system based on the combination of a large model and a graph neural network is developed using a programming language. The system has program modules corresponding to the steps of the above-mentioned technical solution, and executes the steps in the above-mentioned user behavior sequence recommendation method based on the combination of a large model and a graph neural network during runtime.

[0142] The developed system (software) computer program is stored on a computer-readable storage medium. When called by a processor, the computer program is configured to implement the steps of the aforementioned user behavior sequence recommendation method based on combining a large model with a graph neural network. This materializes the present invention on a carrier, becoming a computer program product.

[0143] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] The computer programs (also referred to as programs, software, software applications, or code) of the present invention include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used in this embodiment, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., a disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0145] The beneficial effects of the present invention are verified by the following examples.

[0146] Example 1

[0147] The model in this paper primarily targets implicit user-item interactions and also requires textual descriptions of items. Based on this requirement, this example conducted experiments on five different datasets from Amazon Review Data 2018: Industrial and Scientific (Scientific), Arts Crafts and Sewing (Arts), Office Products (Office), Musical Instruments (Instruments), and Prime Pantry (Pantry).

[0148] Table 1 shows the statistical information for the five datasets selected for this example. With the exception of the Scientific dataset, all other datasets have reached millions of interactions. The text information in all five datasets is also very rich, with an average text length exceeding 100 characters. The number of users also reached tens of thousands. The Office dataset boasts 87,436 users, 25,986 items, and 684,837 interactions, the highest number of records among the five datasets.

[0149] Table 1 Statistics of the dataset

[0150]

[0151] All data are initially in two files, a rating only file and a metadata file. In the metadata file, each item consists of multiple data tags and corresponding information. For this reason, this embodiment selects items in five data sets with the common tags "Title", "Brand" and "category", because the information of these three types of tags are all important text description information for the items. This embodiment first performs text cleaning, and then splices the text information of three different tags together to generate a new data file in the form of (item, text). Finally, the items in the sequence file that do not contain text information are filtered out. This embodiment statistically analyzes the text length of the items in the five data sets, and the results are as follows: Figure 3 As shown in the figure, Scientific and Panty are lightweight datasets, with text length accounting for less than 10% of the total dataset. However, Office has a much longer text length than other datasets, accounting for 45% of the total dataset text length.

[0152] For the rating-only file, the data content is in the form of (user, item, rating, timestamp). This embodiment uses the rating as the interaction criterion to modify the rating value, filter out samples where the user and item appear less than 5 times, and sort the interaction behaviors of the same user according to the timestamp label to generate data pairs of (user, item1, item2, item3). Then, the user label is filtered out to form a sequence file. Sequences with a sequence length of 1 in the sequence file are filtered out, and sequence cutting is performed at the same time. A sequence is cut into multiple sequences using a sliding window method, and the last item in the sequence is used as the next interaction item result of the sequence. All the interaction items before the item are regarded as a sequence, such as S = [v s,1 ,v s,2 ,…,v s,n ], and then cut into multiple subsequences according to the sliding window [(v s,1 ),v s,2 ],[(v s,1 ,v s,2 ),v s,3 ],…[(v s,1 ,v s,2 ,v s,n-1 ),v s,n ].

[0153] This example divides the training and test sets into groups based on session timestamps. First, the latest session time in the dataset is determined as a benchmark, and a 7-day time window is subtracted from this threshold to determine the time threshold for division. Sessions with timestamps earlier than this threshold are classified as the training set, while sessions with timestamps later than this threshold are classified as the test set. During this process, items appearing in the training set are renumbered, and a dictionary of uniquely identified items is constructed. Only items appearing in the training set are retained in the test set, ensuring that the test data has the same distribution characteristics as the training data. Finally, the session data is converted into a sequence format to meet the requirements of subsequent model training and evaluation.

[0154] To comprehensively evaluate the performance of the LLM-SR model of the present invention, this example selects different types of recommendation models for comparison, including sequential recommendation models (ZESRec and SASRec), graph neural network-based recommendation models (SR-GNN and SPARE), and pre-trained model-based recommendation models (UniSRec and VQ-Rec). The specific methods are described as follows:

[0155] We conducted comparative experiments on five datasets comparing our LLM-SR model with six baseline methods. The training and datasets were divided by timestamp. The experiment was repeated five times, and the Recall@10, Recall@50, NDCG@10, and NDCG@50 metrics were calculated. The average of these metrics was used as the final result. The experimental results are shown in Table 2.

[0156] Table 2 Comparison results of baseline models

[0157]

[0158] As can be seen, the LLM-SR model of this invention demonstrates excellent performance in scenarios with rich text information and complex user behavior by combining the text representation capabilities of a large language model, the sequence modeling capabilities of a graph neural network, and the sorting capabilities of an attention mechanism. On the Scientific and Instruments datasets, LLM-SR outperforms baseline models in all metrics, particularly in leveraging long text information and capturing diverse interest patterns. LLM-SR also demonstrates good adaptability in scenarios with a large number of users and intensive interactions, maintaining high recommendation performance despite limited text information.

[0159] In order to verify the necessity of each module in the large-model-based sequence recommendation model of the present invention, the following example is used to conduct an ablation experiment to evaluate the independence and effectiveness of the algorithm of the present invention.

[0160] Example 2

[0161] This example designs three model variants: one without large-scale model embedding generation, one without local session feature modeling, and one without global feature aggregation, each designed to analyze the contribution of each module to overall performance. The first model without large-scale model representation generation removes the text representation module generated by the large model and directly uses randomly initialized item representations as the input for sequence learning, validating the role of the large model in item representation learning. The second model without local session feature modeling masks the session graph module and directly uses global feature representations as the output of the item graph convolution to evaluate the impact of session-level feature modeling on model performance. The third model without global feature aggregation masks the global graph module and uses only local session features as the output of the item graph convolution, validating the contribution of global features to overall performance.

[0162] Through the three ablation experiments described above, we deeply analyze the independent role and collaborative contribution of each module in the large-scale model-based sequential recommendation model. The experimental results are shown in Tables 3 and 4. In the table, "w / o LLM" represents a model without large-scale model embedding generation, "w / o global" represents a model without global feature aggregation, and "w / o session" represents a model without local session feature modeling.

[0163] Table 3 Ablation experiment Recall@10 results

[0164]

[0165] Table 4 Ablation experiment NDCG@10 results

[0166]

[0167] The results show that the complete LLM-SR model performs best in both indicators, which proves the effectiveness of the large model-based text generation module, global graph module, and sequence graph module.

[0168] Example 3

[0169] In order to tune the parameters of the LLM-SR model, this embodiment selects four parameters: learning rate lr, item representation dimension d, sequence graph neuron drop rate drop_s rate, and global graph neuron drop rate drop_g for comparative experiments.

[0170] In order to study the influence of learning rate lr on the experimental results, experiments were conducted on five data sets, setting lr = {1e -1 ,1e -2 ,1e -3 ,1e -4}, keep other parameters unchanged, modify the learning rate value, and observe the changes in the evaluation indicators Recall@10, Recall@50, NDCG@10, and NDCG@50. The experimental results are as follows Figure 4 shown.

[0171] Sensitivity analysis of the representation dimension d was conducted on five datasets. Experiments were conducted on d = {32, 64, 128, 256} with only the representation dimension changed, keeping other experimental parameters unchanged. The experimental results were recorded. Figure 5 shown.

[0172] To study the impact of the sequence graph neuron dropout rate dr_s and the global graph neuron dropout rate dr_g on model performance, a cross-sensitivity analysis of these two parameters was performed. The dr_s value range was set to [0-0.9] with a step size of 0.1; the dr_g value range was set to [0-0.9] with a step size of 0.1. Other parameters were kept unchanged. Experiments were conducted on the Arts and Office datasets and the corresponding Recall@10 and NDCG@50 evaluation indicators were recorded. Thermal images were drawn. The experimental results are shown in the figure below. Figure 6 shown.

[0173] Experimental results show that it is more appropriate to set the learning rate lr to 0.001, the item representation dimension d to 128, the sequence graph neuron drop rate drop_s to 0.4-0.6, and the global graph neuron drop rate drop_g to 0.4-0.6.

[0174] Although the present invention is disclosed as above, the scope of protection disclosed by the present invention is not limited thereto. Those skilled in the art of the present invention may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A user behavior sequence recommendation method based on a combination of a large model and a graph neural network, characterized in that: The steps include: Collecting user-item interaction information data, preprocessing the user-item interaction information data to obtain a sequence set, collecting item text information data, and preprocessing the text information data to obtain a text information set; The sequence set and the text information set are input into the LLM-SR model, which includes a large-model-based item text representation learning module, a graph neural network-based sequence representation learning module, and a prediction score calculation module; the large-model-based item text representation learning module extracts high-quality semantic embeddings of item texts through a large language model, and combines linear layer mapping and multi-layer perceptron to generate item representations adapted to recommendation tasks; the graph neural network-based sequence representation learning module uses session graphs and global graphs to capture the local sequential relationships and cross-session co-occurrence relationships of items, and generates a comprehensive representation of user behavior sequences; the prediction score calculation module calculates and ranks item scores through the dot product of user sequence representations and candidate item embeddings to generate the final recommendation results.

2. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 1 is characterized in that: The multi-layer perceptron described in the large-model-based item text representation learning module is a three-layer multi-layer perceptron. Each layer uses the ReLU activation function to introduce nonlinearity, and a Dropout mechanism is added after the first and second layers.

3. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 2 is characterized in that: The linear layer in the large-model-based item text representation learning module uses a linear transformation method to linearly map the input features through a weighted matrix and a bias vector to achieve dimensionality adjustment and feature conversion of the embedding space.

4. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 1 is characterized in that: The functional implementation process of the sequence representation learning module based on graph neural network is as follows: The session graph is constructed as follows: Each session s is constructed as a directed session graph The point set The vertices in are the vertices in session s, and the edge set Any edge in Represents the order of items in the sequence, Indicates that item i is interacted before item j; For node v i Representation Indicated by the upper layer and neighbor nodes Features Weighted aggregation is obtained, namely: The weight α ij is the attention weight calculated based on the correlation between nodes, which is used to dynamically adjust the influence of neighbor nodes on the update of the target node. ij For node v i and v j The relevance score is combined with the feature representation of the item embedding representation and Through nonlinear changes of the learnable parameters W4 and the attention vector a, we can obtain: Global Picture The construction method is: each node v in the graph i Represents an item. If the item v i and v j co-occurs in at least one user session, then i and v j Create an edge (v i ,v j ), the edge weight is defined by the co-occurrence frequency between items, that is: where freq(v i ,v j ) is v i and v j The number of co-occurrences, freq(v i ) and freq(v j ) are items v i and v j Number of occurrences; Information propagation aggregates information from neighboring nodes through a weighted sum based on an attention mechanism; Neighborhood characteristics The weighted aggregation formula is: The importance of neighbor nodes is determined by the conversation context-sensitive weight function π(v i ,v j ) calculation, which is used to distinguish the importance of different neighbors to the current item, and the weight is: in is the feature vector obtained by averaging the pooled items in the current session, namely: Among them, w ij is the weight between items, ⊙ represents element-wise multiplication, || represents concatenation, W5 and q1 are learnable parameters; The information aggregation step represents the feature of the current item h v and features propagated from neighbors Perform fusion and generate new item representations through nonlinear transformation, namely: Where W6 is the linear transformation weight matrix and ReLU is the activation function; The stacked multi-layer propagation mechanism captures high-order neighbor information. Each layer is generated by calculating the item representation and neighbor representation of the previous layer. The representation of the k-th layer is defined as: First, we learn high-level feature representations of items through the global graph and combine them with session features to generate a comprehensive representation of the items. The final representation of each item is: Representing items and reverse position indicates p l-i+1 Perform splicing and generate a new representation through nonlinear changes, namely: Where W7 and b4 are learnable parameters; Finally, the weighted representations of the items are combined into the final session representation, i.e.: Weight β β Obtained by dynamic calculation of the soft attention mechanism: W8, W9, q2, and b5 are learnable parameters; The preliminary session feature s′ is calculated using the average representation of all items in the session, namely: Where l is the sequence length.

5. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 4 is characterized in that: The function implementation process of the score prediction of the prediction score calculation module is as follows: The initial embedding vector of the session representation S and the candidate item The dot product of the candidate item v is calculated i The probability of becoming the next click recommendation is normalized by SoftMax, that is: m is the size of the candidate item set.

6. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 5 is characterized in that: The cross-loss entropy function used by the LLM-SR model is: where y i ∈{0,1} is the true label, is the model prediction item v i probability of click.

7. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 6 is characterized in that: Step 1 includes the following steps: Collect user-item interaction data, determine whether items and users interact and score them, filter out samples where users and items have been rated less than five times, sort items chronologically, and then filter out user tags to form a sequence file. Then filter out sequences with a length of 1 in the sequence file and split the sequence into multiple subsequences. The last item in the sequence is used as the next interaction item in the sequence, and all the interaction items before the item are treated as a sequence to obtain a sequence set. Collecting item text information data and preprocessing the text information data, including text cleaning, splicing text information of different tags, and generating a text information set; Finally, the items in the sequence set that do not contain text information are filtered out.

8. A user behavior sequence recommendation system based on a combination of a large model and a graph neural network, characterized by: The system has a program module corresponding to the steps of the method described in any one of claims 1 to 7 above, and executes the steps of the user behavior sequence recommendation method based on the combination of a large model and a graph neural network during operation.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps of the user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to any one of claims 1 to 7 when called by a processor.

Images