A cold start recommendation method and system based on large language model

Abstract

The present invention discloses a cold start recommendation method and system based on a large language model, which belongs to the field of recommendation systems, including: constructing a fine-tuning training data set; selecting a pre-trained large language model, and fine-tuning the model using a low-rank matrix training method, and then iteratively training the fine-tuning training data set, using a semi-supervised method to judge the data quality in each iteration process until the loss function drops to a design threshold, and generating a target large language model; based on the content data and interaction data of the fine-tuning training data set, obtaining a content embedding vector corresponding to the content data and a collaborative embedding vector corresponding to the interaction data through the target large language model; aligning the content embedding vector and the collaborative embedding vector through a feedforward neural network, and generating an item cold start embedding vector generator, which is used to obtain a recommended cold start item embedding vector according to the content embedding vector of the cold item. The present invention effectively improves the recommendation performance in the cold start item recommendation scenario.

Description

Technical Field

[0001] The present invention belongs to the field of recommendation systems, and in particular, relates to a cold start recommendation method and system based on a large language model. Background Art

[0002] In recent years, computer technology, the Internet and its affiliated industries have developed rapidly. Countless text-based data are generated on the Internet every day, showing the characteristics of big data. At the same time, with the vigorous development of artificial intelligence content generation technology, data information is generated continuously every day. Users are faced with the problem of information overload, and the cost of obtaining high-value information is getting higher and higher. How to quickly retrieve the data that everyone is really interested in from massive data has promoted the research of recommendation systems. Recommendation system is a technology that uses user behavior data and item information to provide users with personalized recommendations; it analyzes users' interests and preferences through algorithm models based on users' historical behavior and item characteristics, predicts items that users may like, and recommends them to them. Recommendation systems are widely used in various application scenarios such as e-commerce, social media, and video streaming, aiming to improve user experience, increase user stickiness, and promote sales and content dissemination. The development of recommendation systems is not only of great significance to users and enterprises, but also provides rich practical scenarios for promoting research and application in the fields of information technology and artificial intelligence.

[0003] When a new recommendation system or new users or items appear, the user's historical behavior data is missing, making it impossible for the algorithm to provide users with relevant recommendation information. As an important scenario of the recommendation system, cold start has been widely studied. In the recommendation system, the interaction data between users and items is very important. For newly generated items, there is no interaction data. Effective recommendation is directly related to the subsequent exposure of the item, which ultimately affects the recommendation effect of the item. The key to solving the cold start problem is to infer user preferences or content characteristics from limited information through a series of technical means, so as to make personalized recommendations or searches.

[0004] The existing recommendation system startup method adopts random sampling combined with content filtering and behavior filtering to provide recommendations to users. There are problems such as small reference sample size, insufficient use of content information, and inaccurate recommendations caused by misalignment of existing collaborative information and content information recommendation space. In addition, some recommendation system cold start methods require pre-training of massive data, which is extremely costly. Therefore, it is urgent to design a new cold start recommendation technology to reduce the resource investment and startup time of the recommendation system while retaining accuracy. Summary of the invention

[0005] In order to solve the above problems, the purpose of the present invention is to propose a cold start recommendation technology based on a large language model, overcome the insufficient utilization of item content information in the cold start scenario and design a comparative Loss function to align different recommendation spaces, so as to obtain better recommendation effects.

[0006] In order to achieve the above technical objectives, the present application provides a cold start recommendation method based on a large language model, comprising the following steps:

[0007] Construct a fine-tuning training dataset, where each piece of data includes task instructions, task input, and task output;

[0008] Select a pre-trained large language model, use the low-rank matrix training method to fine-tune the model, and then perform iterative training by fine-tuning the training data set. Use a semi-supervised method to judge the data quality in each iteration until the loss function drops to the designed threshold, generating the target large language model.

[0009] Based on the content data and interaction data of the fine-tuned training dataset, the content embedding vector corresponding to the content data and the collaborative embedding vector corresponding to the interaction data are obtained through the target large language model;

[0010] Through the feedforward neural network, the content embedding vector and the collaborative embedding vector are aligned to generate an item cold start embedding vector generator, which is used to obtain the recommended cold start item embedding vector based on the content embedding vector of the cold item.

[0011] Preferably, during the process of model fine-tuning training, the LoRA low-rank matrix is ​​used as a supplement to the fine-tuning parameters of the large model, and is merged with the original parameters of the large language model to complete the model fine-tuning training.

[0012] Preferably, during the process of model fine-tuning training, the optimization goal of the model fine-tuning training is:

[0013]

[0014] Where P u represents the overall parameters of the large language after merging, x represents the given task, y t Represents the output of the large language model, y ＜t Indicates the output y t The previous output content, Z represents the expected content of all inputs.

[0015] Preferably, in the process of constructing the fine-tuning training data set, data augmentation is achieved by asking questions about the content information in the training data, and the data that does not conform to the format after augmentation is screened and deleted using a regular matching method, and the features after augmentation are merged into the original data set, wherein the merged data D u It is expressed as:

[0016] D u =Merge(f clean (F u ),D o )

[0017] Where D o represents the original data, f clean Represents the cleaning and filtering function, and Merge represents the merging function.

[0018] Preferably, in the process of obtaining the content embedding vector, the vector in the last layer of the large language model encoding layer is bitwise averaged as the content embedding vector of the input content, where the content embedding vector E c It is expressed as:

[0019]

[0020] In the formula, n is the number of content information converted into tokens, e t Represents the vector in the last layer of the encoding layer of the large language model.

[0021] Preferably, in the process of obtaining the collaborative embedding vector, LightGCN is used as a collaborative embedding vector trainer to obtain the collaborative embedding vector corresponding to the interaction data.

[0022] Preferably, in the process of aligning the content embedding vector and the collaborative embedding vector, a loss function is set as an objective function to constrain the alignment process of the content embedding vector and the collaborative embedding vector, wherein the loss function is expressed as:

[0023] L o =L align +L cc +L bc

[0024]

[0025] In the formula, E o represents the vector obtained by training the content embedding vector through a feedforward neural network, E b represents the collaborative embedding vector, e cai Indicates training items, e cpi represents items that have been interacted with by the same user as the training items, e cni represents items that have not interacted with the same user, α1 and α2 are hyperparameters used to control the boundary and interval of the loss, L cc Indicates content comparison loss, L bc represents the collaborative comparison loss, L align Indicates alignment loss.

[0026] Preferably, in the process of training the content embedding vector through a feedforward neural network, the content embedding vector is trained as the input of the feedforward neural network, and the dimension of the output layer is the same as the collaborative embedding vector, wherein the nonlinear layer of the feedforward neural network selects the ReLU function, and a dropout operation is performed to prevent the model from overfitting.

[0027] The present invention discloses a cold start recommendation system based on a large language model, comprising:

[0028] A data construction module is used to construct a fine-tuning training data set, where each piece of data includes task instructions, task input, and task output;

[0029] The training module is used to select a pre-trained large language model, fine-tune the model using the low-rank matrix training method, and then iteratively train the model by fine-tuning the training data set. The semi-supervised method is used to judge the data quality in each iteration until the loss function drops to the designed threshold, generating the target large language model.

[0030] A vector extraction module is used to obtain a content embedding vector corresponding to the content data and a collaborative embedding vector corresponding to the interaction data through a target large language model based on the content data and interaction data of the fine-tuning training data set;

[0031] The cold start recommendation module is used to align the content embedding vector and the collaborative embedding vector through a feedforward neural network to generate an item cold start embedding vector generator, which is used to obtain the recommended cold start item embedding vector based on the content embedding vector of the cold item.

[0032] Preferably, the training module is also used to use the LoRA low-rank matrix as a supplement to the fine-tuning parameters of the large model, and after merging it with the original parameters of the large language model, complete the model fine-tuning training.

[0033] The present invention discloses the following technical effects:

[0034] (1) The present invention uses a large language model to generate a content embedding vector for the content information of an object. Thanks to the sufficient pre-training of the large language model, a more accurate embedding vector representation can be obtained compared to the content feature extractors such as Bert used in the past.

[0035] (2) Considering the spatial mismatch between the content recommendation space and the collaborative recommendation space, the two spaces are aligned by designing a Loss function. Compared with previous methods, the collaborative information and content information can be learned more accurately, thereby achieving better recommendation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a flow chart of the steps of the method described in the present invention.

[0038] Figure 2 It is a schematic diagram of the large language model fine-tuning training process described in the present invention.

[0039] Figure 3 It is a schematic diagram of generating a content embedding vector according to the present invention.

[0040] Figure 4 2. It is a schematic diagram of training the cold start item embedding vector generator described in the present invention.

[0041] Figure 5 It is a schematic diagram of generating the embedding vector of the cold start item described in the present invention. DETAILED DESCRIPTION

[0042] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0043] like Figure 1-5As shown, the present invention provides a cold start recommendation technology for items based on a large language model, including: 1. Large model pre-training stage: by constructing a fine-tuning data training set based on a supervisory signal, the large language model is trained for recommendation tasks, so that the large language model can reason and output according to the expected tasks. 2: Data augmentation stage: limited by the problem that there is no interactive data between items and users in the cold start scenario of the recommendation system, the non-interactive data in the cold start scenario - content data is extremely important. In this stage, by utilizing the sufficient pre-training knowledge of the large language model, the content data is expanded to obtain richer content features. 3: Vector alignment stage: based on the content feature, the content feature embedding vector of the item is obtained through the large language model, and the interactive feature embedding vector is obtained through the recommendation model training based on the interaction of the existing data, and the vector is aligned by designing a neural network. 4: Cold start scenario embedding vector generation stage: the content embedding vector of the cold start item is passed through the large language model to obtain the content embedding vector, and the embedding vector of the cold start item for final recommendation is generated based on the network generated by the 3-stage training. The present invention can fully supplement the features in the cold start scenario by introducing a large language model and obtain a more accurate content embedding vector through the rich pre-training corpus of the large language model, thereby solving the problems of insufficient content features and inaccurate content feature modeling in previous cold start recommendation algorithms; at the same time, by designing a neural network to align the content embedding vector with the behavior embedding vector to learn the collaborative signal in the behavior embedding vector, the problem of poor recommendation effect caused by mismatch of recommendation space based on existing large language model recommendation is solved, and the recommendation performance in the cold start item recommendation scenario is effectively improved.

[0044] The main steps of the cold start recommendation technology based on a large language model proposed in the present invention are: constructing a fine-tuning training data set, fine-tuning the large language model, augmenting the training data, generating content embedding vectors and collaborative embedding vectors, training a cold start vector generator based on the generated collaborative embedding vectors and content embedding vectors, and generating embedding vectors for cold start items based on the cold start vector generator.

[0045] Each step is described in detail below.

[0046] S1. Construct fine-tuning training data:

[0047] Constructing fine-tuning training data includes constructing a data set for fine-tuning a large language model based on recommended existing data, and each piece of data in the data set includes task instructions, task input, and task output.

[0048] S1.1. Preprocessing recommended data:

[0049] The collected training data is divided into hot and cold data according to the industry's commonly used classification methods. 20% of the existing item data is divided into cold item data. The interaction data of the hot item data is saved for easy training. The interaction data of the cold data is manually deleted to simulate the scenario in which cold items have no interaction data in real scenarios.

[0050] S1.2, fine-tuning dataset construction:

[0051] Based on the above constructed cold and hot data, a large model fine-tuning data set is constructed. First, a small number of data samples that need to be augmented are manually collected to construct three parts of data: task instructions, task input, and task output. The task instructions are described as the tasks that the large language model needs to complete in natural language; the task input is the information that can be provided to the large language model; the task output is the output information that needs to be fitted by the large model. The large language model needs to approximate the given task output in both format and content to facilitate subsequent processing.

[0052] S2. Fine-tune the large language model:

[0053] like Figure 2 As shown, for the fine-tuning training dataset obtained in step S1, low-rank matrix training and other methods are used to adjust the large language model based on the existing pre-training so that the large language model activates the recommendation ability, understands the recommendation task, and standardizes the output format of the large model.

[0054] S2.1. Obtaining the original model

[0055] For the selection of large language models, there are usually two types: paid and open source. For paid models, you can choose a model similar to OpenAI's ChatGPT and call the interface for training. For open source models, you can deploy and reason locally to reduce overhead. Referring to the models used in the industry, select MetaAI's open source Llama2-7B model as the original pre-trained model and record it as L o .

[0056] S2.2. Training low-rank matrix parameters

[0057] For the original large model L o , it is a time-consuming and labor-intensive task to fully train its parameters. In order to facilitate fine-tuning training, LoRA (Low-Rank Adaptation of Large Language Models) low-rank matrix is ​​used as a supplement to the fine-tuning parameters of the large model. The essence of the LoRA method is to use rank decomposition to approximate the effect of training all parameters with fewer parameters.

[0058] S2.3, parameter merging:

[0059] Let the original parameters of the large language model be Po , the parameters of the low-rank matrix after training in step S2.2 are P l . Change the original parameter P o and the parameter P of the low-rank matrix l After merging, the overall parameter of the large language is expressed as P u , which can be expressed as follows:

[0060] P u =P o +P l .

[0061] The overall task of step S2 is to make the trained large language model L t The output y t Be able to do as much as possible in the given task x and in the output y t Previous output content y ＜t , which can maximize the output y t The overall optimization goal is as follows:

[0062]

[0063] S3. Augment existing training data:

[0064] The content data in the training data is expanded to make the features as rich as possible when constructing the content embedding vector.

[0065] S3.1、Designing a large language model for questions Tips:

[0066] Based on the task prompts and task inputs in the fine-tuned dataset in step S1, a prompt is designed for the content information in the training data to obtain more features. Taking the commonly used MovieLens-1m in the industry as an example, for an item, its original feature information only has title and type features, and its original features are represented as F o Based on the original title and genre features, we hope to add the film director, film country, and film language features and record them as F a , the prompt for constructing the model is as follows:

[0067] Your task is to give [supplementary features] based on the given item content features. The original features of [item name] are [original features]. Please give [supplementary features], and the output format is [desired output format].

[0068] The content in [] in the above prompt is filled according to the features of the items in different data sets. Then, the model output is obtained by querying the large language model, that is, the augmented data feature F u , the process can be expressed as follows:

[0069] F a =L t (F o , Prompt)

[0070] F u =F o +F a .

[0071] S3.2. Clean data and integrate features:

[0072] For the augmented data feature F obtained in step S4.1 u , use regular matching and other methods to filter and delete the data that does not conform to the format after augmentation, and merge the augmented features into the original data set. The process can be expressed as follows, where the original data is represented by D o , the cleaning and screening function is represented by f clean , the merge function is Merge, and the merged data is D u :

[0073] D u =Merge(f clean (F u ), D o ).

[0074] S4. Generate content embedding vector and collaborative embedding vector:

[0075] For content data, the content embedding vector obtained after pre-training of the large language model is obtained through the large language model; for interaction data, the collaborative embedding vector based on the interaction data is obtained through the existing recommendation model.

[0076] S4.1 Content embedding vector generation:

[0077] like Figure 3 As shown, the content data D finally generated based on step 4 u , and input it into the trained large language model L as content t , the vector e in the last layer of the large language model encoding layer t Take the bitwise mean as the content embedding vector E of the input content c The process can be expressed as follows, where n is the number of content information converted into tokens:

[0078]

[0079] S4.2. Collaborative embedding vector generation:

[0080] For the hot item data in the preprocessed data of step S1.1, since it has interactive feature data, a collaborative embedding vector with collaborative information can be trained based on its interactive data. The commonly used recommendation model LightGCN in the industry is selected as the collaborative embedding vector trainer. The generation process of the collaborative embedding vector can be expressed as follows:

[0081] E b =LightGCN(D bw )

[0082] Among them, D bw Represents the interaction data between hot items and users.

[0083] S5. Generate item cold start embedding vector generator:

[0084] For the content embedding vector and the collaborative embedding vector obtained in step S4, a neural network is designed to align the content embedding vector and the collaborative embedding vector, so that the collaborative signal in the existing collaborative embedding vector can be learned through the vector generator based on the content embedding vector. Finally, the neural network after training is obtained as an item cold start embedding vector generator.

[0085] S5.1. Embedding vector alignment:

[0086] like Figure 4 As shown, construct a feedforward neural network M c As the cold start embedding vector generator of the item, it consists of a linear layer and a nonlinear layer. The nonlinear layer uses the ReLU function and performs a dropout operation to prevent the model from overfitting. c As model input through M c Training, the dimension of the output layer is the same as the co-embedding vector E obtained in step S4.2 b The same; the vector output after training is E o . It can be specifically expressed as the following formula:

[0087]

[0088] Among them, d represents the output dimension and E b same.

[0089] In order to align the content embedding vector with the collaborative embedding vector, the alignment Loss is designed to be L alian , by bitwise calculation through M c After training, E o and the collaborative embedding vector E b The distance between the vectors is the distance between them. The goal is to make the distance as small as possible, so L align The goal can be expressed as follows:

[0090]

[0091] n represents the number of training samples.

[0092] S5.2. Contrastive learning loss function construction:

[0093] In order to better capture similar information, we construct contrast loss functions in different spaces for training. The spaces are mainly content contrast loss and collaborative contrast loss:

[0094]

[0095] (1) Content comparison loss:

[0096] For items, interactions with the same user u can be considered a cluster, indicating that these items have certain similarities. By designing the Loss function L cc To shorten the distance between similar items. On the contrary, all training items that have no interaction with the user are called negative samples. For each positive sample, k negative samples are randomly selected from the global items to construct the comparison LossL cc As shown in the following formula,

[0097]

[0098]

[0099] where e cai Indicates training items, e cpi Indicates items that have been interacted with by the same user, e cni represents items that have not interacted with the same user, and α1 is a hyperparameter used to control L cc boundaries and intervals.

[0100] (2) Collaborative comparison loss:

[0101] For the embedding vector of the collaborative space, The vectors in should also have the above characteristics, so we design a collaborative comparison LossL bc As shown below,

[0102]

[0103] in The meaning is the same as (1).

[0104] Overall optimization of the embedding vector alignment stage LossL oIt can be expressed as the sum of the above three losses: L o =L align +L cc +L bc .

[0105] S6. Generate cold start item embedding vector:

[0106] Step S6 is to generate the cold start item embedding vector. After training in S5, the item cold start embedding vector generator M is obtained. c . When there is a cold start item I c When the content features are passed through the large language model L t Then the content embedding vector Set the positive sample content embedding vector and negative sample content embedding vector required for input in the training phase to 0, such as Figure 5 As shown, after cold start embedding content embedding vector generator M c Then we get the embedding vector e of the user's final recommendation r The process can be described by the following formula: r =M c (L t (I c )).

[0107] The present invention can fully supplement the features in the cold start scenario by introducing a large language model and obtain a more accurate content embedding vector through the rich pre-training corpus of the large language model, solving the problems of insufficient content features and inaccurate modeling of content features in previous cold start recommendation algorithms; at the same time, by designing a neural network to align the content embedding vector with the behavior embedding vector to learn the collaborative signal in the behavior embedding vector, the problem of poor recommendation effect caused by the mismatch of the recommendation space based on the existing large language model recommendation is solved, and the recommendation performance in the cold start item recommendation scenario is effectively improved. The present invention uses a low-rank matrix to use rank decomposition so that the effect of training all parameters can be approximated with parameters with fewer training parameters; and uses a large language model to generate a content embedding vector for the content information of the item. Thanks to the sufficient pre-training of the large language model, a more accurate embedding vector representation can be obtained compared to the content feature extractors such as Bert used in the past. The present invention aligns the content recommendation space and the collaborative recommendation space by designing a better Loss function, improves the matching accuracy, and can more accurately learn the collaborative information and content information compared to the previous methods, thereby obtaining better recommendation performance.

[0108] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0109] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0110] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A cold start recommendation method based on a large language model, characterized in that: The following steps are involved: Construct a fine-tuning training dataset, where each piece of data includes task instructions, task input, and task output; Select a pre-trained large language model, use a low-rank matrix training method to perform model fine-tuning training, and then perform iterative training using the fine-tuning training data set. Use a semi-supervised method to judge data quality in each iteration process until the loss function drops to a designed threshold, thereby generating a target large language model. Based on the content data and interaction data of the fine-tuning training data set, obtaining a content embedding vector corresponding to the content data and a collaborative embedding vector corresponding to the interaction data through the target large language model; The content embedding vector and the collaborative embedding vector are aligned through a feedforward neural network to generate an item cold start embedding vector generator, which is used to obtain a recommended cold start item embedding vector based on the content embedding vector of the cold item; In the process of aligning the content embedding vector and the collaborative embedding vector, a loss function is set as an objective function to constrain the alignment process of the content embedding vector and the collaborative embedding vector, wherein the loss function is expressed as: L o =L align +L cc +L bc In the formula, E o represents the vector obtained by training the content embedding vector through a feedforward neural network, E b represents the collaborative embedding vector, e cai Indicates training items, e cpi represents items that have been interacted with by the same user as the training items, e cni represents items that have not interacted with the same user, α1 and α2 are hyperparameters used to control the boundary and interval of the loss, L cc Indicates content comparison loss, L bc represents the collaborative comparison loss, L align Indicates alignment loss.

2. The cold start recommendation method based on a large language model according to claim 1, characterized in that: During the model fine-tuning training process, the LoRA low-rank matrix is ​​used as a supplement to the large model fine-tuning parameters, and is merged with the original parameters of the large language model to complete the model fine-tuning training.

3. The cold start recommendation method based on a large language model according to claim 2, characterized in that: During the model fine-tuning training process, the optimization goal of the model fine-tuning training is: Where P u represents the overall parameters of the large language after merging, x represents the given task, yt represents the output of the large language model, and y<t represents the output y t The previous output content, Z represents the expected content of all inputs.

4. The cold start recommendation method based on a large language model according to claim 3, characterized in that: In the process of constructing the fine-tuning training data set, data augmentation is achieved by asking questions about the content information in the training data. The data that does not conform to the format after augmentation is filtered and deleted using regular matching, and the features after augmentation are merged into the original data set. u It is expressed as: D u =Merge(f clean (F u ),D o ) Where D o represents the original data, f clean Represents the cleaning and filtering function, and Merge represents the merging function.

5. The cold start recommendation method based on a large language model according to claim 4, characterized in that: In the process of obtaining the content embedding vector, the vector in the last layer of the large language model encoding layer is bitwise averaged as the content embedding vector of the input content, where the content embedding vector E c It is expressed as: In the formula, n is the number of content information converted into tokens, e t Represents the vector in the last layer of the encoding layer of the large language model.

6. The cold start recommendation method based on a large language model according to claim 5, characterized in that: In the process of obtaining the collaborative embedding vector, LightGCN is used as a collaborative embedding vector trainer to obtain the collaborative embedding vector corresponding to the interaction data.

7. The cold start recommendation method based on a large language model according to claim 1, characterized in that: In the process of training the content embedding vector through the feedforward neural network, the content embedding vector is used as the input of the feedforward neural network for training, and the dimension of the output layer is the same as the collaborative embedding vector, wherein the nonlinear layer of the feedforward neural network selects the ReLU function, and a dropout operation is performed to prevent the model from overfitting.

8. A cold start recommendation system based on a large language model, characterized in that: include: A data construction module is used to construct a fine-tuning training data set, where each piece of data includes task instructions, task input, and task output; A training module is used to select a pre-trained large language model, perform model fine-tuning training using a low-rank matrix training method, and then perform iterative training using the fine-tuning training data set. A semi-supervised method is used to judge data quality in each iteration process until the loss function drops to a designed threshold, thereby generating a target large language model. A vector extraction module, configured to obtain, based on the content data and interaction data of the fine-tuning training data set, a content embedding vector corresponding to the content data and a collaborative embedding vector corresponding to the interaction data through the target large language model; A cold start recommendation module, used to align the content embedding vector and the collaborative embedding vector through a feedforward neural network to generate an item cold start embedding vector generator, used to obtain a recommended cold start item embedding vector based on the content embedding vector of the cold item; In the process of aligning the content embedding vector and the collaborative embedding vector, a loss function is set as an objective function to constrain the alignment process of the content embedding vector and the collaborative embedding vector, wherein the loss function is expressed as: L o =L align +L cc +L bc In the formula, E o represents the vector obtained by training the content embedding vector through a feedforward neural network, E b represents the collaborative embedding vector, e cai Indicates training items, e cpi represents items that have been interacted with by the same user as the training items, e cni represents items that have not interacted with the same user, α1 and α2 are hyperparameters used to control the boundary and interval of the loss, L cc Indicates content comparison loss, L bc represents the collaborative comparison loss, L align Indicates alignment loss.

9. The cold start recommendation system based on a large language model according to claim 8, characterized in that: The training module is also used to use the LoRA low-rank matrix as a supplement to the fine-tuning parameters of the large model, and after merging it with the original parameters of the large language model, complete the model fine-tuning training.

Images