Intelligent news content recommendation method based on large language model and comparative learning

Abstract

The invention discloses a news content intelligent recommendation method based on a big language model and comparative learning. The method comprises the following steps: constructing a fine-tuning training data set, and performing fine-tuning training on the big language model; constructing a training set, inputting the training set into the fine-tuned large language model to obtain confidence prediction of graph structure enhancement, and cleaning to obtain a graph structure enhancement view; clustering information is extracted; the method comprises the following steps of: obtaining a collaborative embedding vector of a user and news contents through combined training of a graph neural network and graph contrast learning, obtaining a final embedding vector for recommendation through a multi-layer perceptron after the collaborative embedding vector is combined with a clustering embedding vector, and calculating an interaction score of the user and different news contents according to the final embedding vector. And recommending the news content to the user according to the interaction score. According to the method, the uncertainty in graph structure enhancement can be reduced, and key information is reserved as far as possible for joint training, so that a better recommendation effect is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of content recommendation, and in particular to a news content intelligent recommendation method based on a large language model and contrastive learning. Background Art

[0002] With the development of the Internet, the relationship between users and items has become more and more complex, and traditional recommendation algorithms have difficulty in fully capturing these relationships. The combination of graph neural networks (GNNs) and contrastive learning in recommendation systems is becoming an important method to improve the effect of personalized recommendations. In graph contrastive learning, graph structure enhancement is a common technique that aims to improve the representation ability of the model by generating different graph views. However, one of the most critical issues in this process is information loss. When performing graph structure enhancement, such as randomly deleting edges, adding noise, or discarding nodes, key information may be inadvertently removed, resulting in the model being unable to correctly understand the essential structure of the graph.

[0003] There are several main reasons for information loss:

[0004] First, in a graph, there are often complex relationships and semantic information between nodes and edges. When edges or nodes are randomly deleted, connections that are critical to the node representation may be removed. For example, in a social network, a user's connection with a friend may indicate a strong relationship between them, and deleting this edge may cause the model to ignore this important social information, thus affecting the recommendation results.

[0005] Secondly, although adding and deleting edges can increase the diversity of the graph, if the operation is not done properly, it may introduce false relationships and cause the model to learn wrong information. This noise will not only reduce the learning efficiency of the model, but may also cause overfitting, causing the model to perform well on the training data but poorly in actual applications.

[0006] Furthermore, information loss is also reflected at the feature level. When node features are randomly perturbed, excessive perturbation may cause the node to lose its original feature information, which to some extent weakens the model's understanding of the node itself. Node features largely determine its position and role in the graph. If these features are destroyed, the model will not be able to accurately capture the true meaning of the node.

[0007] In addition, subgraph sampling methods may also lose global information when extracting local structures. The global characteristics of the graph structure have an important impact on the relationship between nodes. The local view may not fully reflect the global context, resulting in a lack of sufficient information support when the model is learning. Therefore, it is urgent to design a new graph structure enhancement technology to mine the structural information of the graph as much as possible while retaining key information. Summary of the invention

[0008] In order to overcome the defects and shortcomings of the prior art, the present invention provides a news content intelligent recommendation method based on a large language model and contrastive learning. The present invention can reduce the uncertainty in graph structure enhancement and retain key information as much as possible for joint training, thereby obtaining better recommendation effects.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] The present invention provides a news content intelligent recommendation method based on a large language model and contrastive learning, comprising the following steps:

[0011] Build a fine-tuning training dataset based on user attributes and interaction data, perform fine-tuning training on the large language model, and obtain a fine-tuned large language model;

[0012] A training set is built based on historical interaction data between users and news, and input into a fine-tuned large language model to obtain confidence predictions enhanced by graph structures.

[0013] Clean the confidence predictions of graph structure enhancement to obtain graph structure enhancement views;

[0014] Construct the embedding vector E of the user node and news content node of the input graph neural network, and obtain the clustering embedding vector of the user and news content through clustering operation;

[0015] Obtain different enhanced views of the graph structure, perform graph convolution on the embedding vector E through the graph neural network, obtain the embedding vector of the corresponding enhanced view, and construct the graph contrast learning loss function;

[0016] Based on the interaction records of the training set, the interaction matrix is ​​obtained, and the generalization matrix is ​​constructed. According to the generalization matrix, information aggregation is performed on users and news content nodes;

[0017] Construct a Bayesian personalized ranking loss function and construct a joint training loss with the graph comparison learning loss function;

[0018] Based on joint training, the collaborative embedding vector of users and news content is obtained, which is then combined with the clustering embedding vector and passed through a multi-layer perceptron to obtain the final embedding vector for recommendation.

[0019] The interaction scores between users and different news contents are calculated based on the final embedding vector, and news contents are recommended to users according to the interaction scores.

[0020] As a preferred technical solution, the fine-tuning training of the large language model is specifically performed by fine-tuning the large language model through an adaptive low-rank matrix training method;

[0021] The original parameters of the large language model are recorded as PO , the adaptive parameter of the large language model after adaptive low-rank matrix training is P l , the original parameter P O Merge with the adaptive parameter Pl, and the overall parameter of the large language model after merging is P u , expressed as:

[0022] P u =P O +P l

[0023] The optimization goal of fine-tuning training is:

[0024]

[0025] Among them, x represents the given task, y t Represents the output of the large language model, y ＜t Indicates the output y t The previous output content, Z represents all the input corpus content.

[0026] As a preferred technical solution, the fine-tuned large language model is input to obtain the confidence prediction with graph structure enhancement, which is specifically expressed as:

[0027] v i =L t (F i , Prompt)

[0028] T i，j =v i，j

[0029] in, It is a vector that records the possibility of retaining each edge of user i, indicating whether all edges of user i should be discarded by the fine-tuned large language model. M represents the amount of news content, and v i，j represents the importance of news content j to user i, L t is the parameter of the large language model, F i is the original feature of user i, T i，j Represents the value of the element in the i-th row and j-th column of T.

[0030] As a preferred technical solution, the embedding vector E of the user node and the news content node of the input graph neural network is constructed, which is specifically expressed as:

[0031] E={u1,u2,…,u N , i1, i2, …, i M}

[0032] in, O is the embedding vector dimension of each node, u xis the embedding vector of user x, x = 1, 2, 3…, N, i y is the embedding vector of news content y, y = 1, 2, 3…, M;

[0033] Set the number of cluster centers K for users and news content u and K i , based on the Kmeans algorithm, the clustering embedding vectors of users and news content are obtained, expressed as:

[0034]

[0035] in, E M The first N embedding vectors of represent the user’s cluster embedding vector, E M The last M embedding vectors represent the clustered embedding vectors of the news content.

[0036] As a preferred technical solution, different graph structure enhanced views are obtained, and the embedded vector E is convolved through the graph neural network to obtain the embedded vector of the corresponding enhanced view, which specifically includes:

[0037] Obtain the graph structure enhanced view S1 and the graph structure enhanced view S2, perform graph convolution on the embedding vector E through the graph neural network, and obtain the embedding vector E1 and the embedding vector E2 of the corresponding enhanced view, which are specifically expressed as:

[0038] (E1, E2) = Encoder (S1, S2, E)

[0039] Among them, Encode r Represents a graph neural network;

[0040] After obtaining the embedding vector of the corresponding enhanced view, the InfoNCE function is used as the loss function of contrastive learning, and the contrastive learning loss function on the user side is constructed as:

[0041]

[0042] Among them, sim and τ represent cosine similarity and temperature coefficient respectively, U represents the parent set of user data, ssl represents unsupervised learning, and user represents that the loss (L) is the loss on the user side. It represents the embedding vector of user u in the embedding vector E1. It represents the embedding vector of user u in the embedding vector E2;

[0043] The contrastive learning loss function for building news content is:

[0044]

[0045] in, represents the embedding vector of news content i in the embedding vector E1, represents the embedding vector of news content i in the embedding vector E2, and I represents the parent set of news content data;

[0046] The graph contrast learning loss function is constructed as:

[0047]

[0048] Among them, L ssl Represents the graph contrastive learning loss function.

[0049] As a preferred technical solution, an interaction matrix is ​​obtained based on the interaction records of the training set, and a generalization matrix is ​​constructed, which specifically includes:

[0050] Let the interaction matrix be R. If user i and news content j have interaction records, set the i-th row and j-th column element in the interaction matrix R to 1, and the adjacency matrix is:

[0051]

[0052] Construct the generalization matrix as:

[0053]

[0054] Where D is the degree matrix of the adjacency matrix A;

[0055] According to the generalization matrix, information aggregation is performed on users and news content nodes, which is specifically expressed as follows:

[0056]

[0057] in, represents the generalization matrix.

[0058] As an optimal technical solution, a loss function of Bayesian personalized sorting is constructed, which is specifically expressed as:

[0059]

[0060] Among them, the news content i that user u has interacted with and the news content j that user has not interacted with are screened out, and the set of tuples (u, i, j) is recorded as P, y u，i and u，j are the interactive scores of user u on news content i and news content j, respectively. The corresponding embedding vectors are obtained by dot multiplication after information aggregation, which is specifically expressed as:

[0061]

[0062] Among them, u u is the embedding vector of user u, is the transpose of the embedding vector of news content i, is the transpose of the embedding vector of the news content.

[0063] As a preferred technical solution, the collaborative embedding vectors of users and news content are obtained based on joint training, and then merged with the clustering embedding vectors to obtain the final embedding vector for recommendation through a multi-layer perceptron, which is specifically expressed as:

[0064] E L =MLP(Merge(E G , E M ))

[0065] Among them, MLP represents multi-layer perceptron, E L represents the final embedding vector for recommendation, E G The collaborative embedding vector E is obtained by jointly training the loss function of graph contrastive learning and Bayesian personalized ranking. M Represents the cluster embedding vector obtained by the clustering algorithm, and Merge represents the merging function.

[0066] As a preferred technical solution, the interaction scores between users and different news contents are calculated according to the final embedding vector. Specifically, the interaction scores between users and different news contents are obtained by calculating the cosine similarity. The calculation method of the cosine similarity is as follows:

[0067]

[0068] In the formula, A represents the user embedding vector, B represents the news content embedding vector, and A i represents the i-th dimension of the user embedding vector in the final embedding vector, n represents the dimension of the embedding vector, and B i Represents the i-th dimension of the news content embedding vector in the final embedding vector.

[0069] The present invention also provides a news content intelligent recommendation system based on a large language model and contrastive learning, comprising: a fine-tuning training data set construction module, a large language model fine-tuning training module, a training set construction module, a confidence prediction output module, a graph structure enhanced view output module, an embedding vector construction module, a clustering module, a graph contrast learning module, an information aggregation module, a joint training module, a final embedding vector output module, an interactive score calculation module, and a product recommendation result output module;

[0070] The fine-tuning training data set construction module is used to construct a fine-tuning training data set based on user attributes and interaction data;

[0071] The large language model fine-tuning training module is used to perform fine-tuning training on the large language model to obtain a fine-tuned large language model;

[0072] The training set construction module is used to construct a training set based on historical interaction data between users and news;

[0073] The confidence prediction output module is used to input the training set into the fine-tuned large language model to obtain a confidence prediction with graph structure enhancement;

[0074] The graph structure enhanced view output module is used to clean the confidence prediction of graph structure enhancement to obtain a graph structure enhanced view;

[0075] The embedding vector construction module is used to construct the embedding vector E of the user node and the news content node of the input graph neural network;

[0076] The clustering module is used to obtain cluster embedding vectors of users and news content through clustering operations;

[0077] The graph contrast learning module is used to obtain different graph structure enhanced views, perform graph convolution on the embedding vector E through the graph neural network, obtain the embedding vector of the corresponding enhanced view, and construct a graph contrast learning loss function;

[0078] The information aggregation module is used to obtain an interaction matrix based on the interaction records of the training set, and to construct a generalization matrix, and to aggregate information on users and news content nodes according to the generalization matrix;

[0079] The joint training module is used to construct a loss function for Bayesian personalized sorting, and to construct a joint training loss with a graph comparison learning loss function;

[0080] The final embedding vector output module is used to obtain the collaborative embedding vector of the user and the news content based on joint training, and after merging with the clustering embedding vector, obtain the final embedding vector for recommendation through a multi-layer perceptron;

[0081] The interaction score calculation module is used to calculate the interaction score between the user and different news contents according to the final embedding vector;

[0082] The product recommendation result output module is used to recommend news content to users according to the interaction score.

[0083] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0084] (1) The present invention solves the limitations of traditional news content recommendation methods by introducing graph neural network models and graph comparative learning technology, so as to achieve more accurate and efficient news content recommendation and improve user experience and platform competitiveness.

[0085] (2) The present invention solves the limitation of graph contrast learning method that key information is easily lost by introducing a large language model, so as to achieve more accurate graph structure learning and bring users a more comprehensive and in-depth news content recommendation experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 It is a flow chart of the intelligent recommendation method of news content based on large language model and contrastive learning of the present invention;

[0087] Figure 2 This is a schematic diagram of the fine-tuning training process of the large language model of the present invention;

[0088] Figure 3 Schematic diagram of the implementation architecture of the multi-layer perceptron of the present invention;

[0089] Figure 4 It is a schematic diagram of the structure of the graph neural network of the present invention;

[0090] Figure 5 This is a schematic diagram of the implementation architecture of graph comparison learning of the present invention;

[0091] Figure 6 Schematic diagram of the implementation architecture of the clustering algorithm of the present invention. DETAILED DESCRIPTION

[0092] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0093] Example 1

[0094] like Figure 1 As shown, this embodiment provides a news content intelligent recommendation method based on a large language model and contrastive learning, comprising the following steps:

[0095] S1: Construct a fine-tuning training dataset and perform fine-tuning training on the large language model to obtain a fine-tuned large language model.

[0096] In this embodiment, each piece of data in the fine-tuning training data set includes task instructions, task input and task output. The task instruction is a natural language description of the specific task that the large language model needs to complete, which clearly indicates the goals and requirements of the model; the task input is information that can be processed by the large language model to ensure that the model can obtain sufficient context data to perform the task; the task output is the target output that needs to be fitted by the large language model. The model needs to match the given task output in format and content to facilitate subsequent processing and evaluation. By effectively combining these components, a high-quality fine-tuning training data set can be constructed, thereby significantly improving the performance and adaptability of the large language model on specific tasks.

[0097] Specifically, content data and interaction data are obtained to form task input, where content data is part of the user's attributes, and interaction data is the historical interaction data between the user and the news. Specifically, Douban's dataset can be used to construct a fine-tuning dataset to stimulate the corresponding capabilities of the large language model.

[0098] In this embodiment, the large language model can be selected into two types: paid models and open source models. The paid model can be trained by calling the interface, such as OpenAI's ChatGPT model, which provides a high-performance solution for training by calling the interface, while the open source model allows local deployment and reasoning, thereby significantly reducing related costs and expenses. This embodiment selects the MetaAI open source Llama3-8B model as the original pre-trained model and marks it as L O When selecting this model, we fully considered the flexibility and scalability brought by its open source characteristics. By combining the needs of fine-tuning large language models, this choice not only helps to achieve efficient model training, but also provides an economically feasible solution, making the model more adaptable and effective in practical applications. This selection strategy aims to make full use of the advantages of open source models to achieve the best training results and resource allocation, laying a solid foundation for subsequent fine-tuning and application.

[0099] like Figure 2 As shown, the large language model is fine-tuned through the training method of adaptive low-rank matrix (AdaLoRA). This method is improved on the basis of LoRA. LoRA ignores the difference in importance of different modules in the model. AdaLoRA introduces an adaptive low-rank decomposition method, which can dynamically adjust the low-rank matrix according to the changes in tasks and data. The method is to update it through SVD (singular value decomposition). During training, AdaLoRA will assign a larger rank to more important features, so as to more effectively capture the detailed information of the features, so that the large language model can activate the recommendation ability, understand the recommendation task, and standardize the output format of the large model;

[0100] For the original large model L O , it is a time-consuming and labor-intensive task to fully train its parameters. In order to facilitate fine-tuning training, this embodiment uses AdaLoRA (Adaptive Low-Rank Adaptation of Large Language Models) adaptive low-rank matrix as a supplement to the fine-tuning parameters of the large language model. The low-rank matrix obtained by AdaLoRA training is applied to the weight matrix of the large language model, and some layers of the large language model are matrix decomposed, and replaced or modified with the low-rank matrix of AdaLoRA, and finally saved. This method dynamically adjusts the model parameters during the fine-tuning process to better adapt to the needs of specific tasks. In this process, the fine-tuning parameters generated by AdaLoRA are merged with the original parameters of the large language model to form a model that integrates the original knowledge and new task information. The essence of the LoRA method is to use rank decomposition to approximate the effect of originally training all parameters with less training parameters. On this basis, AdaLoRA uses the triplets in the singular value decomposition (SVD) to replace the binary in the traditional LoRA, making full use of the orthogonality of the SVD singular matrix and the correlation between the absolute value of the singular vector and the feature importance.

[0101] In this embodiment, the original parameters of the large language model are recorded as P O , the adaptive parameter of the large language model after adaptive low-rank matrix training is P l , the original parameter P O With the adaptive parameter P l After merging, the overall parameters of the large language model are expressed as P u , which can be expressed as follows:

[0102] P u =P O +P l ;

[0103] The overall task is to make the trained large language model L t The output y t Be able to do as much as possible in the given task x and in the output y t Previous output content y ＜t , which can maximize the output y t , and its overall optimization goal is as follows:

[0104]

[0105] Among them, P u represents the overall parameters of the large language model after merging, x represents the given task, and y t Represents the output of the large language model, y ＜tIndicates the output y t The previous output content, Z represents all the input corpus content;

[0106] S2: Obtain the historical interaction dataset between users and news, and divide the dataset into a training set and a test set. Input the training set into the fine-tuned large language model to obtain the model output.

[0107] In this embodiment, 20% of the data is selected as a test set for evaluating the performance of the model; and the remaining 80% of the data is used to construct a training set. This division method ensures a reasonable distribution between the training set and the test set, so that the model can be evaluated on an independent test set during fine-tuning training. This not only helps to effectively monitor the generalization ability of the model, but also verifies its performance in practical applications. Through this data partitioning strategy, the accuracy and reliability of large language models on specific tasks can be improved, so as to better meet user needs and adapt to changing application scenarios. Specifically, an open source news content platform data set can be selected as a training data set, such as TaoBao's data set for embedding representation training.

[0108] In this embodiment, a prompt is designed to ask questions about the content information in the training set to obtain more features.

[0109] For users, their original feature information includes: user ID, reading time, evaluation, overall score, historical interaction records, and news content information corresponding to each interaction record. The original features are represented as F. Through the original features of different users, we hope to determine whether each news content interaction record is critical. The model prompt is constructed as follows:

[0110] "Your task is to determine whether each news content interaction record is critical based on the original features of the given user. The original features of user i are [original features]. Please determine the criticality of each interaction record. The criticality value range is (0, 1). The output format is [the output format you want to process]."

[0111] The content in [] in the above prompt is filled according to the user features in different data sets, and then the model output is obtained by querying the large language model, that is, the confidence prediction T enhanced by the graph structure is obtained. The process can be expressed as follows:

[0112] v i =L t (F i , PrOmpt)

[0113] T i，j =v i，j

[0114] in, It is a vector that records the possibility of retaining each edge of user i, indicating whether all edges of user i should be discarded by the fine-tuned large language model. i，j represents the importance of news content j to user i, L t is the parameter of the large language model, F i is the original feature of user i, T i，j Represents the value of the element in the i-th row and j-th column of T.

[0115] S3: Clean the graph structure enhanced confidence prediction T to obtain the cleaned graph structure enhanced confidence prediction S, i.e., the graph structure enhanced view;

[0116] In this embodiment, data with low confidence after prediction are deleted by using methods such as numerical truncation. Specifically, 0.5 can be preferably used as the confidence threshold, and data below 0.5 is regarded as 0, and data above 0.5 is retained as 1;

[0117] The present invention introduces a large language model, asks questions about the content information in the training data, and designs confidence prediction for graph structure enhancement. The data with low confidence is screened and deleted using numerical truncation to obtain an enhanced graph structure. The existing graph structure enhancement is completed by randomly discarding the edges in the graph, while the present invention allows the large language model to serve as a referee of whether an edge should be discarded, outputs the confidence of whether an edge should be discarded, and then sets a threshold for the confidence based on prior knowledge to complete the real edge discarding.

[0118] S4: extract clustering information;

[0119] S41: In the process of obtaining interaction records, embedding vectors of N user nodes and M news content nodes of the input graph neural network are constructed, as follows:

[0120] E={u1,u2,…,u N , i1, i2, …, i M}

[0121] in, O is the embedding vector dimension of each node, u x is the embedding vector of user x, x = 1, 2, 3…, N, i y is the embedding vector of news content y, y=1, 2, 3…, M, E is the embedding vector matrix.

[0122] S42: Figure 6 As shown, set the number of cluster centers K for users and news content u and K i, Kmeans algorithm is used as the clustering algorithm to obtain the cluster center. Through the clustering operation, the clustering embedding vectors of users and news content are obtained, which is expressed as:

[0123]

[0124] in, E M The first N embedding vectors of represent the user’s cluster embedding vector, E M The last M embedding vectors represent the clustered embedding vectors of the news content;

[0125] S5: Joint training of graph neural networks and graph contrastive learning;

[0126] S51: The confidence prediction S based on the cleaned graph structure enhancement, i.e., the graph structure enhancement view, is repeated multiple times to obtain the graph structure enhancement view S1 and graph structure enhancement view S2. In each training epoch, S1 and S2 are re-output by the fine-tuned large language model to encode the user and news content nodes. The process can be expressed as follows:

[0127] (E1, E2) = Encoder (S1, S2, E)

[0128] Among them, Encoder represents a graph neural network, which uses different enhanced views to perform graph convolution on E through the graph neural network to obtain the embedding vectors E1 and E2 of the corresponding enhanced views;

[0129] After obtaining the embedding vector of the corresponding enhanced view, the InfoNCE function is used as the loss function of contrastive learning. The formula on the user side is as follows:

[0130]

[0131] Among them, sim and τ represent cosine similarity and temperature coefficient respectively, U represents the parent set of user data, ssl represents unsupervised learning, and user represents that the loss (L) is the loss on the user side. It represents the embedding vector of user u in E1. represents the embedding vector of user u in E2;

[0132] Similarly, the contrastive learning loss function of the news content side can be obtained:

[0133]

[0134] in, It represents the embedding vector of news content i in E1. It represents the embedding vector of news content i in E2, and I represents the parent set of news content data.

[0135] like Figure 5 As shown in the figure, graph contrast learning is performed based on graph structure enhanced view S1 and graph structure enhanced view S2. The contrast learning loss functions on the user side and the news content side are added together to obtain the complete graph contrast learning task, which makes the embedded vector have richer graph structure information and enhances the robustness of the embedded vector:

[0136]

[0137] S52: Constructing the propagation method of graph neural network;

[0138] Based on the interaction records of the training set (e.g., the interaction records between user i and news content), the interaction matrix is ​​obtained and recorded as If user i and news content j have interaction records, the i-th row and column elements in R can be set to 1, and then the adjacency matrix A can be obtained. The formula is as follows:

[0139]

[0140] Its generalization matrix It can be calculated by the following formula:

[0141]

[0142] Where D is the degree matrix of A.

[0143] According to the generalization matrix, the node information is aggregated, which is specifically expressed as:

[0144]

[0145] S53: Joint training;

[0146] In each training epoch, we filter out news content i that user u has interacted with and news content j that user u has not interacted with. The set of tuples (u, i, j) is denoted as P. The loss function of Bayesian personalized ranking is:

[0147]

[0148] Among them, y u，i and u，j are the interactive scores of user u on news content i and news content j, respectively. The corresponding embedding vectors are obtained by dot multiplication after information aggregation through graph neural network, such as:

[0149]

[0150]

[0151] Among them, uu is the embedding vector of user u, is the transpose of the embedding vector of news content i, is the transpose of the embedding vector of the news content;

[0152] like Figure 4 As shown, LightGCN is used as a collaborative embedding vector trainer to obtain the embedding vector corresponding to the interaction data. The loss function of Bayesian personalized sorting and the graph contrast learning loss function are combined to obtain the joint training loss function of the model for joint training. The joint training loss function is expressed as:

[0153] L=L ssl +L bpr

[0154] Through graph neural network and graph contrast learning, the collaborative embedding vector E of users and news content is obtained G ;

[0155] Finally, the collaborative embedding vector and clustering embedding vector obtained through training are merged, such as Figure 3 As shown in the figure, the nonlinear relationship of the merging results is enriched by a multi-layer perceptron to obtain the final embedding vector for recommendation, namely:

[0156] E L =MLP(Merge(E G , E M ))

[0157] Among them, MLP represents multi-layer perceptron, E L represents the final embedding vector for recommendation, E G The collaborative embedding vector E is obtained by jointly training the loss function of graph contrastive learning and Bayesian personalized ranking. M represents the cluster embedding vector obtained by the clustering algorithm, and Merge represents the merging function;

[0158] S6: Calculate the interaction scores between the user and different news contents according to the final embedding vector used for recommendation, and recommend news contents to the user according to the interaction scores;

[0159] In this embodiment, according to the final embedding vector E obtained for recommendation L , that is, the embedding vector E L The first N are the embedding vectors of the user, and the last M are the embedding vectors of the news content. The embedding vectors of the user and the news content are wrapped in the container E. When the corresponding user embedding vector and news content vector are needed, they are added to the final embedding vector E. L The corresponding position is taken out and calculated;

[0160] The interaction scores between users and different news contents are obtained by calculating the cosine similarity, where the cosine similarity is calculated as follows:

[0161]

[0162] In the formula, A represents the user embedding vector, B represents the news content embedding vector, and A i Represents the final embedding vector E L The i-th dimension of the user embedding vector in B, n represents the dimension of the embedding vector, i Represents the final embedding vector E L The i-th dimension of the news content embedding vector in .

[0163] The present invention constructs a fine-tuning training data set to fine-tune a large language model, performs graph structure enhancement, extracts clustering information, jointly trains graph neural network and graph contrast learning, and performs recommendations based on embedded vectors. The graph contrast learning method can be improved by introducing a large language model to solve the problem of key information being easily lost in previous graph structure enhancement. At the same time, the graph neural network model is introduced for joint training with contrast learning, which solves the limitations of traditional news content recommendation methods to achieve more accurate and efficient news content recommendations, improve user experience and platform competitiveness, and the present invention uses an adaptive low-rank matrix with singular value decomposition, so that the effect of originally training all parameters can be approximated with fewer trained parameters, while more effectively identifying and utilizing those features that contribute most to model performance.

[0164] Example 2

[0165] This embodiment provides a news content intelligent recommendation system based on a large language model and contrastive learning, which is used to implement the news content intelligent recommendation method based on a large language model and contrastive learning in the above-mentioned embodiment 1. The system includes: a fine-tuning training data set construction module, a large language model fine-tuning training module, a training set construction module, a confidence prediction output module, a graph structure enhanced view output module, an embedding vector construction module, a clustering module, a graph contrast learning module, an information aggregation module, a joint training module, a final embedding vector output module, an interactive score calculation module, and a product recommendation result output module;

[0166] In this embodiment, the fine-tuning training data set construction module is used to construct a fine-tuning training data set based on user attributes and interaction data;

[0167] In this embodiment, the large language model fine-tuning training module is used to perform fine-tuning training on the large language model to obtain a fine-tuned large language model;

[0168] In this embodiment, the training set construction module is used to construct a training set based on historical interaction data between users and news;

[0169] In this embodiment, the confidence prediction output module is used to input the training set into the fine-tuned large language model to obtain a confidence prediction with graph structure enhancement;

[0170] In this embodiment, the graph structure enhanced view output module is used to clean the confidence prediction of the graph structure enhancement to obtain the graph structure enhanced view;

[0171] In this embodiment, the embedding vector construction module is used to construct the embedding vector E of the user node and the news content node of the input graph neural network;

[0172] In this embodiment, the clustering module is used to obtain cluster embedding vectors of users and news content through clustering operations;

[0173] In this embodiment, the graph contrast learning module is used to obtain different graph structure enhanced views, perform graph convolution on the embedding vector E through the graph neural network, obtain the embedding vector of the corresponding enhanced view, and construct the graph contrast learning loss function;

[0174] In this embodiment, the information aggregation module is used to obtain an interaction matrix based on the interaction records of the training set, and to construct a generalization matrix, and to aggregate information of users and news content nodes according to the generalization matrix;

[0175] In this embodiment, the joint training module is used to construct the loss function of Bayesian personalized sorting, and to construct the joint training loss with the graph comparison learning loss function;

[0176] In this embodiment, the final embedding vector output module is used to obtain the collaborative embedding vector of the user and the news content based on joint training, and after merging with the clustering embedding vector, obtain the final embedding vector for recommendation through a multi-layer perceptron;

[0177] In this embodiment, the interaction score calculation module is used to calculate the interaction score between the user and different news contents according to the final embedding vector;

[0178] In this embodiment, the product recommendation result output module is used to recommend news content to users according to the interaction score.

[0179] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A news content intelligent recommendation method based on a large language model and contrastive learning, characterized in that: The steps include: Build a fine-tuning training dataset based on user attributes and interaction data, perform fine-tuning training on the large language model, and obtain a fine-tuned large language model; A training set is built based on historical interaction data between users and news, and input into a fine-tuned large language model to obtain confidence predictions enhanced by graph structures. Clean the confidence predictions of graph structure enhancement to obtain graph structure enhancement views; Construct the embedding vector E of the user node and news content node of the input graph neural network, and obtain the clustering embedding vector of the user and news content through clustering operation; Obtain different enhanced views of the graph structure, perform graph convolution on the embedding vector E through the graph neural network, obtain the embedding vector of the corresponding enhanced view, and construct the graph contrast learning loss function; Based on the interaction records of the training set, the interaction matrix is ​​obtained, and the generalization matrix is ​​constructed. According to the generalization matrix, information aggregation is performed on users and news content nodes; Construct a Bayesian personalized ranking loss function and construct a joint training loss with the graph comparison learning loss function; Based on joint training, the collaborative embedding vector of users and news content is obtained, which is then combined with the clustering embedding vector and passed through a multi-layer perceptron to obtain the final embedding vector for recommendation. The interaction scores between users and different news contents are calculated based on the final embedding vector, and news contents are recommended to users according to the interaction scores.

2. The news content intelligent recommendation method based on large language model and contrastive learning according to claim 1 is characterized in that: The fine-tuning training of the large language model is specifically performed by fine-tuning the large language model through an adaptive low-rank matrix training method; The original parameters of the large language model are recorded as P O , the adaptive parameter of the large language model after adaptive low-rank matrix training is P l , the original parameter P O With the adaptive parameter P l After merging, the overall parameter of the large language model is P u , expressed as: P u =P O +P l The optimization goal of fine-tuning training is: Among them, x represents the given task, y t Represents the output of the large language model, y ＜t Indicates the output y t The previous output content, Z represents all the input corpus content.

3. The news content intelligent recommendation method based on large language model and contrastive learning according to claim 1 is characterized in that: The fine-tuned large language model is input to obtain the confidence prediction with graph structure enhancement, which is specifically expressed as: in, It is a vector that records the possibility of retaining each edge of user i, indicating whether all edges of user i should be discarded by the fine-tuned large language model. M represents the amount of news content, and v i,j represents the importance of news content j to user i, L t is the parameter of the large language model, F i is the original feature of user i, T i,j Represents the value of the element in the i-th row and j-th column of T.

4. The news content intelligent recommendation method based on large language model and contrastive learning according to claim 1 is characterized in that: Construct the embedding vector E of the user node and news content node of the input graph neural network, which is specifically expressed as: E={u1,u2,…,u N ,i1,i2,…,i M } in, O is the embedding vector dimension of each node, u x is the embedding vector of user x, x=1, 2, 3..., N, i y is the embedding vector of news content y, y = 1, 2, 3..., M; Set the number of cluster centers K for users and news content u and K i , based on the Kmeans algorithm, the clustering embedding vectors of users and news content are obtained, expressed as: in, E M The first N embedding vectors of represent the user’s cluster embedding vector, E M The last M embedding vectors represent the clustered embedding vectors of the news content.

5. The news content intelligent recommendation method based on large language model and contrastive learning according to claim 1 is characterized in that: Obtain different enhanced views of the graph structure, perform graph convolution on the embedding vector E through the graph neural network, and obtain the embedding vector of the corresponding enhanced view, including: Obtain the graph structure enhanced view S1 and the graph structure enhanced view S2, perform graph convolution on the embedding vector E through the graph neural network, and obtain the embedding vector E1 and the embedding vector E2 of the corresponding enhanced view, which are specifically expressed as: (E1, E2) = Encoder(S1, S2, E) Among them, Encoder represents a graph neural network; After obtaining the embedding vector of the corresponding enhanced view, the InfoNCE function is used as the loss function of contrastive learning, and the contrastive learning loss function on the user side is constructed as: Among them, sim and τ represent cosine similarity and temperature coefficient respectively, U represents the parent set of user data, ssl represents unsupervised learning, and user represents that the loss (L) is the loss on the user side. It represents the embedding vector of user u in the embedding vector E1. It represents the embedding vector of user u in the embedding vector E2; The contrastive learning loss function for building news content is: in, represents the embedding vector of news content i in the embedding vector E1, represents the embedding vector of news content i in the embedding vector E2, and I represents the parent set of news content data; The graph contrast learning loss function is constructed as: Among them, L ssl Represents the graph contrastive learning loss function.

6. The news content intelligent recommendation method based on large language model and contrastive learning according to claim 1 is characterized in that: Based on the interaction records of the training set, the interaction matrix is ​​obtained and the generalization matrix is ​​constructed, which includes: Let the interaction matrix be R. If user i and news content j have interaction records, set the i-th row and j-th column element in the interaction matrix R to 1, and the adjacency matrix is: Construct the generalization matrix as: Where D is the degree matrix of the adjacency matrix A; According to the generalization matrix, information aggregation is performed on users and news content nodes, which is specifically expressed as follows: in, represents the generalization matrix.

7. The news content intelligent recommendation method based on large language model and contrastive learning according to claim 1 is characterized in that: Construct the loss function of Bayesian personalized sorting, which is specifically expressed as: Among them, we filter out the news content i that user u has interacted with and the news content j that user has not interacted with, and record the set of tuples (u, i, j) as P, y u,i and u , j are the interactive scores of user u on news content i and news content j, respectively. The corresponding embedding vectors are obtained by dot multiplication after information aggregation, which is specifically expressed as: Among them, u u is the embedding vector of user u, is the transpose of the embedding vector of news content i, is the transpose of the embedding vector of news content j.

8. The news content intelligent recommendation method based on large language model and contrastive learning according to claim 1 is characterized in that: Based on joint training, the collaborative embedding vector of users and news content is obtained, which is combined with the clustering embedding vector and then passed through a multi-layer perceptron to obtain the final embedding vector for recommendation, which is specifically expressed as: It is L =MLP(Go(E G ,It M )) Among them, MLP represents multi-layer perceptron, E L represents the final embedding vector for recommendation, E G The collaborative embedding vector E is obtained by jointly training the loss function of graph contrastive learning and Bayesian personalized ranking. M Represents the cluster embedding vector obtained by the clustering algorithm, and Merge represents the merging function.

9. The news content intelligent recommendation method based on large language model and contrastive learning according to claim 1 is characterized in that: The interaction scores between users and different news contents are calculated based on the final embedding vector. Specifically, the interaction scores between users and different news contents are obtained by calculating the cosine similarity. The calculation method of cosine similarity is as follows: In the formula, A represents the user embedding vector, B represents the news content embedding vector, and A i represents the i-th dimension of the user embedding vector in the final embedding vector, n represents the dimension of the embedding vector, and B i Represents the i-th dimension of the news content embedding vector in the final embedding vector.

10. A news content intelligent recommendation system based on large language model and contrastive learning, characterized in that: include: Fine-tuning training dataset construction module, large language model fine-tuning training module, training set construction module, confidence prediction output module, graph structure enhanced view output module, embedding vector construction module, clustering module, graph comparison learning module, information aggregation module, joint training module, final embedding vector output module, interactive score calculation module, product recommendation result output module; The fine-tuning training data set construction module is used to construct a fine-tuning training data set based on user attributes and interaction data; The large language model fine-tuning training module is used to perform fine-tuning training on the large language model to obtain a fine-tuned large language model; The training set construction module is used to construct a training set based on historical interaction data between users and news; The confidence prediction output module is used to input the training set into the fine-tuned large language model to obtain a confidence prediction with graph structure enhancement; The graph structure enhanced view output module is used to clean the confidence prediction of graph structure enhancement to obtain a graph structure enhanced view; The embedding vector construction module is used to construct the embedding vector E of the user node and the news content node of the input graph neural network; The clustering module is used to obtain cluster embedding vectors of users and news content through clustering operations; The graph contrast learning module is used to obtain different graph structure enhanced views, perform graph convolution on the embedding vector E through the graph neural network, obtain the embedding vector of the corresponding enhanced view, and construct a graph contrast learning loss function; The information aggregation module is used to obtain an interaction matrix based on the interaction records of the training set, and to construct a generalization matrix, and to aggregate information on users and news content nodes according to the generalization matrix; The joint training module is used to construct a loss function for Bayesian personalized sorting, and to construct a joint training loss with a graph comparison learning loss function; The final embedding vector output module is used to obtain the collaborative embedding vector of the user and the news content based on joint training, and after merging with the clustering embedding vector, obtain the final embedding vector for recommendation through a multi-layer perceptron; The interaction score calculation module is used to calculate the interaction score between the user and different news contents according to the final embedding vector; The product recommendation result output module is used to recommend news content to users according to the interaction score.

Images