User behavior sequence recommendation method and system based on combination of large model and graph neural network, and storage medium

Abstract

The invention discloses a user behavior sequence recommendation method and system based on the combination of a large model and a graph neural network, and a storage medium, relates to the technical field of behavior sequence recommendation, and aims to solve the problem that the text feature extraction capability of a large language model and the dynamic modeling capability of sequence recommendation are difficult to deeply fuse in an existing method. And accurate and comprehensive recommendation is difficult to realize. A large-model-based article text representation learning module of an LLM-SR model extracts high-quality semantic embedding of an article text through a large language model, and article representation adaptive to a recommendation task is generated in combination with linear layer mapping and a multi-layer perceptron; capturing a local sequence relation and a cross-session co-occurrence relation of the articles by utilizing the session graph and the global graph through a sequence representation learning module based on a graph neural network, and generating comprehensive representation of a user behavior sequence; and article scores are calculated according to the dot product of the user sequence representation and the candidate article embedding through a prediction score calculation module, and a final recommendation result is generated.

Description

Technical Field

[0001] The present invention relates to the technical field of behavior sequence recommendation, and in particular to a method, system and storage medium for recommending user behavior sequences based on a combination of a large model and a graph neural network. Background Art

[0002] Recommendation systems play an important role in personalized services and information filtering. Sequence recommendation, as a key technology, analyzes users' historical behavior sequences, explores users' dynamic preferences, and provides users with more accurate recommendations. However, traditional sequence recommendation methods often only focus on the time order and co-occurrence relationship of user behavior sequences, and lack full utilization of the text semantic features of items, resulting in insufficient performance of the model in long-tail item recommendations and cold start scenarios. In recent years, large language models have demonstrated their application potential in recommendation systems with their excellent semantic understanding capabilities in the field of natural language processing. The high-quality text embeddings generated by pre-training in large models can effectively extract the semantic features of items and provide a new source of information for recommendation systems. However, how to deeply integrate the text feature extraction capabilities of large language models with the dynamic modeling capabilities of sequence recommendations to give full play to the advantages of both is an important challenge in current research. Summary of the invention

[0003] The technical problems to be solved by the present invention are:

[0004] Existing methods find it difficult to deeply integrate the text feature extraction capabilities of large language models with the dynamic modeling capabilities of sequence recommendations, making it difficult to achieve accurate and comprehensive recommendations.

[0005] The present invention adopts the following technical solutions to solve the above technical problems:

[0006] The present invention provides a user behavior sequence recommendation method based on a combination of a large model and a graph neural network, comprising the following steps:

[0007] Collecting user-item interaction information data, preprocessing the user-item interaction information data to obtain a sequence set, collecting item text information data, and preprocessing the text information data to obtain a text information set;

[0008] The sequence set and the text information set are input into the LLM-SR model, which includes an item text representation learning module based on a large model, a sequence representation learning module based on a graph neural network, and a prediction score calculation module; the item text representation learning module based on the large model extracts high-quality semantic embedding of item text through a large language model, and combines linear layer mapping and multi-layer perceptron to generate item representations suitable for recommendation tasks; the sequence representation learning module based on the graph neural network uses the session graph and the global graph to capture the local sequential relationship and cross-session co-occurrence relationship of items, and generates a comprehensive representation of the user behavior sequence; the prediction score calculation module calculates and sorts the item scores through the dot product of the user sequence representation and the candidate item embedding, and generates the final recommendation result.

[0009] Furthermore, the multi-layer perceptron in the large-model-based item text representation learning module is a three-layer multi-layer perceptron, each layer uses a ReLU activation function to introduce nonlinearity, and a Dropout mechanism is added after the first and second layers.

[0010] Furthermore, the linear layer in the large-model-based object text representation learning module linearly maps the input features through a weighted matrix and a bias vector based on a linear transformation method to achieve dimensionality adjustment and feature conversion of the embedding space.

[0011] Furthermore, the functional implementation process of the sequence representation learning module based on graph neural network is as follows:

[0012] The session graph is constructed in the following way: Each session s is constructed as a directed session graph The point set The vertices in are the vertices in session s, and the edge set Any edge in Represents the order of items in the sequence. Indicates that item i is interacted before item j;

[0013] For node v i Representation Represented by the upper layer and neighbor nodes Features Weighted aggregation is obtained, namely:

[0014]

[0015] The weight α ij is the attention weight calculated based on the correlation between nodes, which is used to dynamically adjust the influence of neighbor nodes on the target node update. ij For node v i and v jThe relevance score is combined with the feature representation of the item embedding representation and Through the learnable parameter W 4 And the attention vector a is nonlinearly changed to obtain:

[0016]

[0017] Global picture The construction method is as follows: each node v in the graph i Represents an item. If the item v i and v j co-occur in at least one user session, then in v i and v j Create an edge between i ,v j ), the weight of the edge is defined by the co-occurrence frequency between items, that is:

[0018]

[0019] where freq(v i ,v j ) is v i and v j The number of co-occurrences, freq(v i ) and freq(v j ) are items v i and v j Number of occurrences;

[0020] Information propagation aggregates information of neighbor nodes through weighted sum based on attention mechanism; neighbor features The weighted aggregation formula is:

[0021]

[0022] The importance of neighbor nodes is determined by the conversation context-sensitive weight function π(v i ,v j ) calculation, which is used to distinguish the importance of different neighbors to the current item, and the weight is:

[0023]

[0024] in is the feature vector obtained by averaging the items in the current session, that is:

[0025]

[0026] Among them, w ij is the weight between items, ⊙ represents element-by-element multiplication, || represents concatenation, W5 and q 1 is a learnable parameter;

[0027] The information aggregation step represents the feature of the current item h v and features propagated from neighbors Fusion is performed to generate new item representations through nonlinear transformation, namely:

[0028]

[0029] Where W 6 is the linear transformation weight matrix, ReLU is the activation function;

[0030] The stacked multi-layer propagation mechanism captures high-order neighbor information. Each layer is generated by calculating the item representation and neighbor representation of the previous layer. The representation of the kth layer is defined as:

[0031]

[0032] First, we learn high-level feature representations of items through the global graph and generate comprehensive representations of items by combining session features. The final representation of each item is:

[0033]

[0034] Representing items and reverse position indicates p l-i+1 Splicing is performed and a new representation is generated through nonlinear changes, namely:

[0035]

[0036] Where W 7 and b 4 is a learnable parameter;

[0037] Finally, the weighted representations of the items are combined into the final session representation, namely:

[0038]

[0039] Weight β i It is obtained by dynamic calculation of the soft attention mechanism:

[0040]

[0041] W 8 ,W 9 ,q 2 ,b 5 is a learnable parameter;

[0042] The preliminary session feature s′ is calculated using the average representation of all items in the session, namely:

[0043]

[0044] Where l is the sequence length.

[0045] Furthermore, the function implementation process of the score prediction of the prediction score calculation module is as follows:

[0046] The initial embedding vectors of the session representation S and candidate items The dot product of the candidate item v is calculated i The probability of becoming the next click recommendation is normalized by SoftMax, that is:

[0047]

[0048] m is the size of the candidate item set.

[0049] Furthermore, the cross loss entropy function adopted by the LLM-SR model is:

[0050]

[0051] where y i ∈{0,1} is the true label, is the model prediction item v i The probability of clicking.

[0052] Furthermore, step one includes the following steps:

[0053] Collect user-item interaction information data, determine whether items and users interact and score, filter out samples where users and items have been scored less than 5 times, sort items in chronological order, and then filter out user labels to form a sequence file; then filter out sequences with a sequence length of 1 in the sequence file, and cut the sequence into multiple subsequences, and take the last item in the sequence as the next interaction item result of the sequence, and take all the interaction items before the item as a sequence to obtain a sequence set;

[0054] Collecting the text information data of items and preprocessing the text information data, including text cleaning, splicing the text information of different labels, and generating a text information set;

[0055] Finally, the items in the sequence set that do not contain text information are filtered out.

[0056] The present invention provides a user behavior sequence recommendation system based on the combination of a large model and a graph neural network. The system has a program module corresponding to the steps of the method described in any one of the above technical solutions, and executes the steps in the above-mentioned user behavior sequence recommendation method based on the combination of a large model and a graph neural network during runtime.

[0057] The present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to implement the steps of the user behavior sequence recommendation method based on the combination of a large model and a graph neural network described in any one of the above-mentioned technical solutions when called by a processor.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] The present invention aims to explore and solve key issues in the deep integration of large language models and sequence recommendations, give full play to the advantages of large language models in semantic understanding and text feature extraction, and combine the ability of sequence recommendations to accurately model dynamic changes in user behavior to build an efficient and accurate recommendation framework. By designing a high-quality method for generating item text embeddings and a sequence representation learning mechanism, the present invention achieves collaborative modeling of semantic features and sequence features, and improves the performance of the recommendation system in long-tail items and cold start scenarios. At the same time, by optimizing the adaptation method of the large model and reducing the computational cost, the accuracy, diversity and generalization ability of the recommendation results are further enhanced, providing users with more accurate and comprehensive personalized services. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of the LLM-SR model architecture in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of an item prompt template in an embodiment of the present invention;

[0062] Figure 3 Graph showing the text length ratios of five data sets in an embodiment of the present invention;

[0063] Figure 4 Result diagram of the learning rate sensitivity analysis experiment in the embodiment of the present invention;

[0064] Figure 5 This is a diagram showing the results of an experiment on sensitivity analysis of embedding representation dimensions in an embodiment of the present invention;

[0065] Figure 6 Parameter sensitivity heat map in the embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to enable those skilled in the art to better understand the scheme of the present invention, exemplary implementations or embodiments of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described implementations or embodiments are only implementations or embodiments of a part of the present invention, not all of them. Based on the implementations or embodiments of the present invention, all other implementations or embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0067] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0068] Specific implementation plan 1: Figure 1 As shown, the present invention provides a user behavior sequence recommendation method based on a combination of a large model and a graph neural network, comprising the following steps:

[0069] Collecting the text information data of items, preprocessing the collected data, and generating a sequence set;

[0070] The sequence set is input into the LLM-SR model, which includes an item text representation learning module based on a large model, a sequence representation learning module based on a graph neural network, and a prediction score calculation module; the item text representation learning module based on the large model extracts high-quality semantic embedding of item text through a large language model, and combines linear layer mapping and multi-layer perceptron to generate item representations adapted to recommendation tasks; the sequence representation learning module based on the graph neural network uses session graphs and global graphs to capture local sequential relationships and cross-session co-occurrence relationships of items, and generates a comprehensive representation of user behavior sequences; the prediction score calculation module calculates and sorts item scores through the dot product of user sequence representation and candidate item embedding, and generates the final recommendation result.

[0071] The sequence recommendation model based on large model and graph neural network proposed in this paper is Large Language Model-based Sequential Recommendation (LLM-SR), and its framework is as follows Figure 1As shown. In the sequence recommendation module based on the large model, the text information of the item (such as title, category and brand) is the key input data. In order to make full use of the large model's ability to understand and represent the text information of the item, the present invention designs a text generation and embedding extraction process based on the prompt template (Prompt). Specifically, first, the attributes such as the title, category and brand of the item are constructed into a standardized input through a predefined Prompt template, and input into the large model to generate standardized text information. Then, a high-quality semantic embedding representation is directly obtained through the embedding extraction layer of the large model, and it is mapped to the target text space through a linear layer. Subsequently, the mapped text representation is spliced ​​with the randomly initialized item ID representation, input into a multi-layer perceptron for nonlinear feature learning, and finally generates an item embedding representation suitable for the recommendation task. The advantage of the method of the present invention is that the large model is responsible for text generation and embedding extraction at the same time, avoiding the information loss that may be caused by cross-model transmission, while making full use of the embedding extraction capability of the large model, without the need to design an additional complex embedding learning module, to ensure the semantic quality of the embedding.

[0072] The prompt template is designed to convert the text information of an item (such as title, category, brand) into structured input, clarify the task objectives, and guide the large model to generate standardized output. Figure 2 shown.

[0073] Specifically, the item text information is generated into a unified text in the format of {[item ID] Title, Category, Brand}, and then inserted into the predefined template to obtain Pm i The constructed prompt is input into the large language model, and the large model generates standardized output according to the prompt. Then, the high-quality semantic embedding representation F is directly obtained through the embedding extraction layer of the large model. i , the whole process is formalized as:

[0074] F i =LLM(Pm i ).#(3-1)

[0075] The text semantic representation generated by a large model is usually a high-dimensional vector, which may not be consistent with the embedding space of the recommendation system when used directly. Therefore, it is necessary to project it into the specific text embedding space of the item through linear mapping.

[0076] The graph neural network-based sequence representation learning module of the present invention can not only comprehensively capture the global and local information in user behavior, but also enhance the saliency modeling capability of item positions in the sequence, providing more accurate prediction results for recommendation tasks.

[0077] Specific implementation plan 2: This implementation plan is based on the multi-layer perceptron layer in the object text representation learning module of the large model, and generates high-quality object embedding representation through nonlinear learning to ensure that it contains both semantic features and has the adaptability to recommendation tasks. Specifically, the object text representation after linear mapping and randomly initialized item IDs to represent X i After concatenation, the nonlinear features are further learned through a multi-layer perceptron to generate the final item embedding representation h suitable for the recommendation task. i This implementation implements a three-layer multilayer perceptron. Each layer uses the ReLU activation function to introduce nonlinearity. At the same time, in order to prevent overfitting, a Dropout mechanism is added after the first and second layers:

[0078]

[0079] h i =W 3 ReLU(W 2 (ReLU(W 1 FX i +b 1 ))+b 2 )+b 3 .#(3-4)

[0080] Where W 1 , W 2 and W 3 is the weight matrix, b 1 , b 2 , b 3 is the bias and ReLU is the activation function.

[0081] The rest of this embodiment is the same as the first embodiment.

[0082] Specific implementation plan three: The linear layer in the large model-based object text representation learning module is based on the linear transformation method, and the input features are linearly mapped through the weighted matrix and the bias vector to achieve dimensional adjustment and feature conversion of the embedding space. The mapping formula is:

[0083]

[0084] Where W is the linear transformation matrix and b is the bias vector.

[0085] The rest of this implementation plan is the same as the second specific implementation plan.

[0086] Specific implementation scheme 4: The functional implementation process of the sequence representation learning module based on graph neural network is as follows:

[0087] The session graph is constructed in the following way: Each session s is constructed as a directed session graph The point set The vertices in are the vertices in session s, and the edge set Any edge in Represents the order of items in the sequence. Indicates that item i is interacted before item j;

[0088] The feature aggregation process of the conversation graph aims to iteratively update the feature representation of the node through the graph neural network to capture the local relationship and sequential dependency between items in the conversation. Specifically, for node v i Representation Represented by the upper layer and neighbor nodes Features Weighted aggregation is obtained, namely:

[0089]

[0090] The weight α ij is the attention weight calculated based on the correlation between nodes, which is used to dynamically adjust the influence of neighbor nodes on the target node update. ij For node v i and v j The relevance score is combined with the feature representation of the item embedding representation and Through the learnable parameter W 4 And the attention vector a is nonlinearly changed to obtain:

[0091]

[0092] The Global Graph is the opposite of the Session Graph. It is an undirected weighted graph constructed based on the co-occurrence relationship between items and is used to capture the global interaction patterns of items across sessions. The construction method is as follows: each node v in the graph i Represents an item. If the item v i and v j co-occur in at least one user session, then in v i and v j Create an edge between i ,v j ), the weight of the edge is defined by the co-occurrence frequency between items, that is:

[0093]

[0094] Global item representation learning aims to propagate features through the global graph structure and capture item transition information across sessions, thereby improving the performance of the recommendation system. This method uses graph neural networks as a framework and combines graph attention mechanisms to model the importance of neighbor information. It consists of two steps: information propagation and information aggregation.

[0095] where freq(v i ,v j ) is v i and v j The number of co-occurrences, freq(v i ) and freq(v j ) are items v i and v j Number of occurrences;

[0096] Information propagation aggregates information of neighbor nodes through weighted sum based on attention mechanism; neighbor features The weighted aggregation formula is:

[0097]

[0098] The importance of neighbor nodes is determined by the conversation context-sensitive weight function π(v i ,v j ) calculation, which is used to distinguish the importance of different neighbors to the current item, and the weight is:

[0099]

[0100] in is the feature vector obtained by averaging the items in the current session, that is:

[0101]

[0102] Among them, w ij is the weight between items, ⊙ represents element-by-element multiplication, || represents concatenation, W 5 and q 1 is a learnable parameter;

[0103] The information aggregation step represents the feature of the current item h v and features propagated from neighbors Fusion is performed to generate new item representations through nonlinear transformation, namely:

[0104]

[0105] Where W 6 It is a linear transformation weight matrix used to integrate different features, and ReLU is an activation function used to increase the nonlinear expression ability of the model.

[0106] In order to capture higher-order neighbor information, a multi-layer propagation mechanism is used to capture high-order neighbor information. Each layer is generated by calculating the item representation and neighbor representation of the previous layer. The representation of the kth layer is defined as:

[0107]

[0108] Sequence representation learning generates high-quality conversational representations by combining global context and conversational context. The core methods include the fusion of global features and conversational features, reverse position embedding, and attention mechanism modeling the importance of items.

[0109] First, we learn high-level feature representations of items through the global graph and generate comprehensive representations of items by combining session features. The final representation of each item is obtained by combining global features and session features using an addition operation:

[0110]

[0111] Then, since the length of the session sequence is not fixed, the reverse position embedding can capture the distance information between the predicted item and the current item. Compared with the forward position, the reverse position embedding is more effective. This paper uses the reverse position embedding to capture the relative importance of items in the session, and calculates the contribution of each item to the session representation through the attention mechanism. Specifically, the item representation and reverse position indicates P l-i+1 Splicing is performed and a new representation is generated through nonlinear changes, namely:

[0112]

[0113] Where W 7 and b 4 is a learnable parameter.

[0114] Finally, the weighted representations of the items are combined into the final session representation, namely:

[0115]

[0116] Weight β i It is obtained by dynamic calculation of the soft attention mechanism:

[0117]

[0118] W 8 ,W 9 ,q 2 ,b 5 is a learnable parameter;

[0119] The preliminary session feature s′ is calculated using the average representation of all items in the session, namely:

[0120]

[0121] Where l is the sequence length.

[0122] The rest of this embodiment is the same as the first embodiment.

[0123] Specific implementation scheme 5: The function realization process of the score prediction of the prediction score calculation module is as follows:

[0124] The prediction score calculation module uses the session representation S and the initial embedding vector of the candidate item Generate recommendation probability. Through the session representation S and the initial embedding vector of the candidate item The dot product of the candidate item v is calculated i The probability of becoming the next click recommendation is normalized by SoftMax, that is:

[0125]

[0126] m is the size of the candidate item set.

[0127] The rest of this implementation plan is the same as the specific implementation plan four.

[0128] Specific implementation plan six:

[0129] The LLM-SR model uses the cross-loss entropy function to optimize the prediction results, namely:

[0130]

[0131] where y i ∈{0,1} is the true label, is the model prediction item v i The click probability of . This loss function measures the predicted probability distribution The difference from the target distribution Y.

[0132] The rest of this implementation plan is the same as the specific implementation plan five.

[0133] The pseudo code for LLM-SR algorithm implementation is shown below.

[0134]

[0135]

[0136] The input of the sequence recommendation algorithm based on the large model is the item information T = {T 1 ,T 2 ,…,T n}, sequence set S = [S 1 ,S 2 ,…,Sn ], maximum number of model training n_epochs, batch size B 1 >0, learning rate α>0 and learnable parameters Θ. The output is the predicted score of the candidate item set

[0137] Lines 1-7 of the algorithm describe the module for learning the text representation of items based on the large model, whose main goal is to generate high-quality initial representations of items. This process first converts the text information of the item (title, category, brand) into a standardized input form through a predefined prompt template, and then inputs it into the large model to generate a semantic embedding representation. Next, the generated semantic embedding is linearly transformed to map it to the target specific space and concatenated with the randomly initialized item ID embedding. Finally, a multi-layer perceptron is used to further learn nonlinear features to generate an item target representation h suitable for subsequent recommendation tasks. i .

[0138] Lines 8-12 of the algorithm describe the process of constructing the session graph and the global graph. In this process, the session graph is first constructed based on the order of item interactions in each session. This graph represents the local relationship between items in a session. Next, by calculating the co-occurrence frequency and relationship of items, the co-occurrence relationship set cr for each item is updated. i Finally, the co-occurrence relationships of all items are aggregated to construct a global graph across sessions.

[0139] Lines 13-24 of the algorithm describe the specific implementation details of the graph neural network sequence representation learning module and the rating prediction module. The specific process includes: randomly sampling small batches of data from the session data, and learning the session representation h of each item through the session graph. s , capturing high-order cross-session relationships through the global graph and generating a global representation h of the item g,(k) Next, the session representation and the global representation are combined and the sequence representation s is obtained through attention learning. Then, the sequence representation is used to predict the ratings of the candidate items and calculate the loss function Finally, the Adam optimizer is used to update the model parameters, iteratively train until convergence, save the final model parameters and generate a prediction score.

[0140] The user behavior sequence recommendation method (algorithm) based on the combination of a large model and a graph neural network proposed in the present invention is the underlying technical core of the present invention, and various products can be derived based on the algorithm.

[0141] Based on the method proposed in the present invention, a user behavior sequence recommendation system based on the combination of a large model and a graph neural network is developed using a programming language. The system has program modules corresponding to the steps of the above-mentioned technical solution, and executes the steps in the above-mentioned user behavior sequence recommendation method based on the combination of a large model and a graph neural network during runtime.

[0142] The computer program of the developed system (software) is stored on a computer-readable storage medium, and the computer program is configured to implement the steps of the above-mentioned user behavior sequence recommendation method based on the combination of a large model and a graph neural network when called by a processor. That is, the present invention is materialized on a carrier to become a computer program product.

[0143] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] The computer programs (also referred to as programs, software, software applications, or codes) of the present invention include machine instructions for programmable processors, and these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used in this embodiment, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (e.g., disk, optical disk, memory, programmable logic device PLD) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0145] The beneficial effects of the present invention are verified by the following examples.

[0146] Example 1

[0147] The model of the present invention mainly targets the implicit interaction between users and items, and also requires text information description of items. Based on this requirement, this embodiment selects five different datasets in Amazon Review Data 2018 for experiments. They are Industrial and Scientific (Scientific for short), Arts Crafts and Sewing (Arts for short), Office Products (Office for short), Musical Instruments (Instruments for short), and Prime Pantry (Pantry for short).

[0148] The statistical information of five data sets selected in this embodiment is shown in Table 1. Except for the Scientific data set, the number of interactions in other data sets has reached the million level. The text information of the five data sets is also very rich, and the average length of the text exceeds 100. The number of users has also reached the tens of thousands level. Among them, the number of users in the Office data set is 87,436, the number of items is 25,986, and the number of interactions is 684,837, which has the largest number of records among the five data sets.

[0149] Table 1 Statistics of the dataset

[0150]

[0151] All data are initially in two files, a rating only file and a metadata file. In the metadata file, each item consists of multiple data tags and corresponding information. For this reason, this embodiment selects items in five data sets that have the common tags "Title", "Brand" and "category", because the information of these three types of tags are all important text description information for the items. This embodiment first cleans the text, and then splices the text information of the three different tags together to generate a new data file in the form of (item, text). Finally, the items in the sequence file that do not contain text information are filtered out. This embodiment statistically analyzes the text length of the items in the five data sets, and the results are as follows: Figure 3 As shown in the figure, Scientific and Panty are lightweight datasets, and the text length of the dataset accounts for less than 10%. The text length of Office is much higher than that of other datasets, accounting for 45% of the total dataset text length.

[0152] For the rating only file, the data content contained is in the form of (user, item, rating, timestamp). This embodiment uses whether to rate as the interaction standard to modify the rating value, filters out samples where the number of times the user and item appear is less than 5 times, and sorts the interaction behaviors of the same user according to the timestamp label to generate (user, item 1 , item 2 , item 3 ) data pairs, and then filter out the user tags to form a sequence file. Then filter the sequences with a sequence length of 1 in the sequence file, and cut the sequence at the same time. A sequence is cut into multiple sequences by sliding the window, and the last item in the sequence is used as the next interaction item result of the sequence. All the interaction items before the item are taken as a sequence, such as S = [v s,1 ,v s,2 ,…,v s,n ], and then cut into multiple subsequences according to the sliding window [(v s,1 ),v s,2 ],[(v s,1 ,v s,2 ),v s,3 ],…[(v s,1 ,v s,2 ,v s,n-1 ),v s,n ].

[0153] This embodiment divides the training set and the test set according to the session timestamp. First, the latest time of the session in the data set is determined as the reference point, and a 7-day time window is subtracted from it to determine the division time threshold. Sessions with timestamps earlier than the threshold are divided into the training set, and sessions with timestamps later than the threshold are classified as the test set. During the processing, the items appearing in the training set are renumbered, and a uniquely identified item dictionary is constructed; only the items that appear in the training set are retained in the test set, thereby ensuring that the test data has the same distribution characteristics as the training data. Finally, the session data is converted into a sequence form to meet the needs of subsequent model training and evaluation.

[0154] In order to comprehensively evaluate the performance of the LLM-SR model of the present invention, this embodiment selects different types of recommendation models for comparison, including sequence recommendation models (ZESRec and SASRec), recommendation models based on graph neural networks (SR-GNN and SPARE), and recommendation models based on pre-trained models (UniSRec and VQ-Rec). The specific methods are described as follows:

[0155] The LLM-SR model of the present invention and six baseline methods are compared in five data sets. The training set and the data set are divided according to the timestamp, and the experiment is repeated five times to calculate the four indicators of Recall@10, Recall@50, NDCG@10 and NDCG@50, and the average value is taken as the final result. The experimental results are shown in Table 2.

[0156] Table 2 Comparison results of baseline models

[0157]

[0158] It can be seen that the LLM-SR model of the present invention has demonstrated excellent performance in scenarios with rich text information and complex user behavior by combining the text representation ability of the large language model, the sequence modeling ability of the graph neural network, and the sorting ability of the attention mechanism. On the Scientific and Instruments datasets, LLM-SR is ahead of the baseline model in all indicators, especially in utilizing long text information and capturing diverse interest patterns. In scenarios with a large number of users and intensive interactions, LLM-SR also shows good adaptability, and can still guarantee high recommendation performance despite the limited text information.

[0159] In order to verify the necessity of each module in the sequence recommendation model based on a large model of the present invention, an ablation experiment is performed using the following embodiment to evaluate the independence and effectiveness of the algorithm of the present invention.

[0160] Example 2

[0161] This embodiment designs three model variants: embedding without large model generation, local session feature modeling without modeling, and global feature aggregation without modeling, which are used to analyze the contribution of each module to the overall performance. The model without large model generation means removing the text representation module generated by the large model, and directly using the randomly initialized item representation as the input of sequence learning to verify the role of the large model in item representation learning; the model without local session feature modeling means shielding the session graph module, and directly using the global feature representation as the graph convolution output of the item to evaluate the impact of session-level feature modeling on model performance; the model without global feature aggregation means shielding the global graph module, and only using the local session features as the graph convolution output of the item to verify the contribution of global features in the overall performance.

[0162] Through the above three ablation experiments, we deeply analyze the independent role and collaborative contribution of each module in the sequence recommendation model based on the large model. The experimental results are shown in Tables 3 and 4. In the table, w / o LLM represents the embedding model without large model generation, w / oglobal represents the model without global feature aggregation, and w / o session represents the model without local session feature modeling.

[0163] Table 3 Ablation experiment Recall@10 results

[0164]

[0165] Table 4 Ablation experiment NDCG@10 results

[0166]

[0167] The results show that the complete LLM-SR model performs best in both indicators, which proves the effectiveness of the large model-based text generation module, global graph module, and sequence graph module.

[0168] Example 3

[0169] In order to tune the parameters of the LLM-SR model, this embodiment selects four parameters, namely, the learning rate lr, the item representation dimension d, the sequence graph neuron drop rate drop_s rate, and the global graph neuron drop rate drop_g, for comparative experiments.

[0170] In order to study the influence of learning rate lr on the experimental results, experiments were conducted on five data sets, setting lr = {1e -1 ,1e -2 ,1e -3 ,1e -4}, keep other parameters unchanged, modify the learning rate value, and observe the changes in the evaluation indicators Recall@10, Recall@50, NDCG@10, and NDCG@50. The experimental results are as follows Figure 4 shown.

[0171] Sensitivity analysis of the representation dimension d was performed on five data sets, with other experimental parameters kept unchanged and only the value of the representation dimension was changed. Experiments were conducted with d = {32, 64, 128, 256} and the experimental results were recorded. Figure 5 shown.

[0172] In order to study the impact of the sequence graph neuron dropout rate dr_s and the global graph neuron dropout rate dr_g on the model performance, a cross-sensitivity analysis of these two parameters was performed. The dr_s value range is set to [0-0.9], with a step size of 0.1; the dr_g value range is set to [0-0.9], with a step size of 0.1, and other parameters are kept unchanged. Experiments are conducted on the Arts and Office datasets and the corresponding Recall@10 and NDCG@50 evaluation indicators are recorded. Thermal images are drawn. The experimental results are shown in the figure. Figure 6 shown.

[0173] Experimental results show that it is more appropriate to set the learning rate lr to 0.001, the item representation dimension d to 128, the sequence graph neuron drop rate drop_s to 0.4-0.6, and the global graph neuron drop rate drop_g to 0.4-0.6.

[0174] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A user behavior sequence recommendation method based on a combination of a large model and a graph neural network, characterized in that: The steps include: Collecting user-item interaction information data, preprocessing the user-item interaction information data to obtain a sequence set, collecting item text information data, and preprocessing the text information data to obtain a text information set; The sequence set and the text information set are input into the LLM-SR model, which includes an item text representation learning module based on a large model, a sequence representation learning module based on a graph neural network, and a prediction score calculation module; the item text representation learning module based on the large model extracts high-quality semantic embedding of item text through a large language model, and combines linear layer mapping and multi-layer perceptron to generate item representations suitable for recommendation tasks; the sequence representation learning module based on the graph neural network uses the session graph and the global graph to capture the local sequential relationship and cross-session co-occurrence relationship of items, and generates a comprehensive representation of the user behavior sequence; the prediction score calculation module calculates and sorts the item scores through the dot product of the user sequence representation and the candidate item embedding, and generates the final recommendation result.

2. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 1 is characterized in that: The multilayer perceptron described in the large model-based item text representation learning module is a three-layer multilayer perceptron. Each layer uses the ReLU activation function to introduce nonlinearity, and a Dropout mechanism is added after the first and second layers.

3. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 2 is characterized in that: The linear layer in the large-model-based object text representation learning module linearly maps the input features through a weighted matrix and a bias vector based on a linear transformation method to achieve dimensionality adjustment and feature conversion of the embedding space.

4. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 1 is characterized in that: The functional implementation process of the sequence representation learning module based on graph neural network is as follows: The session graph is constructed in the following way: Each session s is constructed as a directed session graph The point set The vertices in are the vertices in session s, and the edge set Any edge in Represents the order of items in the sequence. Indicates that item i is interacted before item j; For node v i Representation Represented by the upper layer and neighbor nodes Features Weighted aggregation is obtained, namely: The weight α ij is the attention weight calculated based on the correlation between nodes, which is used to dynamically adjust the influence of neighbor nodes on the target node update. ij For node v i and v j The relevance score is combined with the feature representation of the item embedding representation and Through nonlinear changes of the learnable parameters W4 and the attention vector a, we get: Global picture The construction method is as follows: each node v in the graph i Represents an item. If the item v i and v j co-occur in at least one user session, then in v i and v j Create an edge between i ,v j ), the weight of the edge is defined by the co-occurrence frequency between items, that is: where freq(v i ,v j ) is v i and v j The number of co-occurrences, freq(v i ) and freq(v j ) are items v i and v j Number of occurrences; Information propagation aggregates information from neighboring nodes through a weighted sum based on an attention mechanism; Neighborhood characteristics The weighted aggregation formula is: The importance of neighbor nodes is determined by the conversation context-sensitive weight function π(v i ,v j ) calculation, which is used to distinguish the importance of different neighbors to the current item, and the weight is: in is the feature vector obtained by averaging the items in the current session, that is: Among them, w ij is the weight between items, ⊙ represents element-by-element multiplication, || represents concatenation, W5 and q1 are learnable parameters; The information aggregation step represents the feature of the current item h v and features propagated from neighbors Fusion is performed to generate new item representations through nonlinear transformation, namely: Where W6 is the linear transformation weight matrix and ReLU is the activation function; The stacked multi-layer propagation mechanism captures high-order neighbor information. Each layer is generated by calculating the item representation and neighbor representation of the previous layer. The representation of the kth layer is defined as: First, we learn high-level feature representations of items through the global graph and generate comprehensive representations of items by combining session features. The final representation of each item is: Representing items and reverse position indicates p l-i+1 Splicing is performed and a new representation is generated through nonlinear changes, namely: Among them, W7 and b4 are learnable parameters; Finally, the weighted representations of the items are combined into the final session representation, namely: Weight β β It is obtained by dynamic calculation of the soft attention mechanism: W8, W9, q2, b5 are learnable parameters; The preliminary session feature s′ is calculated using the average representation of all items in the session, namely: Where l is the sequence length.

5. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 4 is characterized in that: The function implementation process of the score prediction of the prediction score calculation module is as follows: The initial embedding vectors of the session representation S and candidate items The dot product of the candidate item v is calculated i The probability of becoming the next click recommendation is normalized by SoftMax, that is: m is the size of the candidate item set.

6. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 5 is characterized in that: The cross loss entropy function used by the LLM-SR model is: where y i ∈{0,1} is the true label, is the model prediction item v i The probability of clicking.

7. The user behavior sequence recommendation method based on the combination of a large model and a graph neural network according to claim 6 is characterized in that: Step 1 includes the following steps: Collect user-item interaction information data, determine whether items and users interact and score, filter out samples where users and items have been scored less than 5 times, sort items in chronological order, and then filter out user labels to form a sequence file; then filter out sequences with a sequence length of 1 in the sequence file, and cut the sequence into multiple subsequences, and take the last item in the sequence as the next interaction item result of the sequence, and take all the interaction items before the item as a sequence to obtain a sequence set; Collecting the text information data of items and preprocessing the text information data, including text cleaning, splicing the text information of different labels, and generating a text information set; Finally, the items in the sequence set that do not contain text information are filtered out.

8. A user behavior sequence recommendation system based on a combination of a large model and a graph neural network, characterized in that: The system has a program module corresponding to the steps of the method described in any one of claims 1 to 7 above, and executes the steps in the above-mentioned user behavior sequence recommendation method based on the combination of a large model and a graph neural network when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps in the user behavior sequence recommendation method based on the combination of a large model and a graph neural network as described in any one of claims 1 to 7 when called by a processor.

Images