Recommendation method, system, device and medium based on large language model

Abstract

The present application discloses a recommendation method, system, device and medium based on a large language model, which relates to the field of computer technology. The method includes: compressing first input information through a small language model and then outputting a first semantic coding embedding; extracting a first project embedding representation and a first user embedding representation from the first input information through a collaborative filtering module; fusing the semantic feature representation and the project feature representation through a dual-view feature fusion module to obtain a first fused feature representation, wherein the semantic feature representation includes the first semantic coding embedding, and the project feature representation includes the first project embedding representation; replacing the corresponding mark in the prompt template through the user feature representation and the first fused feature representation to obtain a first prompt, wherein the user feature representation includes the first user embedding representation; and selecting the next recommended item from a list of candidate items according to the first prompt through the large language model. The present application improves the recommendation effect of the recommendation system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a recommendation method, system, device, and medium based on a large language model. Background Art

[0002] With the rapid development of internet technology and the explosive growth of information, users often struggle to quickly find content of real interest amidst the vast amount of information available. To address this issue, recommendation systems have emerged and are widely used in a variety of fields, including e-commerce, social media, news, and video platforms. They analyze historical user behavior data or item features to predict user interests and preferences, thereby enabling personalized recommendations. However, traditional recommendation systems suffer from limited understanding and reasoning capabilities and suffer from the cold start problem.

[0003] To address the issues of traditional recommendation systems, existing technologies integrate large language models (LLMs) into recommendation systems to form LLM-based recommendation systems. These systems then feed user interaction history and candidate text information (titles, descriptions, etc.) into the LLM to perform recommendations. This improves the recommendation system's understanding and reasoning capabilities and alleviates the cold start problem. However, due to the context length limitations of LLMs and the potential for irrelevant information (noise) in user interaction history and candidate text information, feeding the entire LLM into the system can degrade the evaluation metrics of the recommendation system, resulting in poor recommendation performance. Summary of the Invention

[0004] The purpose of this application is to provide a recommendation method, system, device and medium based on a large language model to solve the problem of poor recommendation effect of existing recommendation systems based on large language models.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a recommendation method based on a large language model, comprising:

[0007] Compressing first input information using a small language model and outputting a first semantic encoding embedding; wherein the first input information is an input sequence formed by text information of a target user's interaction history and a candidate list, the text information including a title and a text description, and the small language model is aligned with the large language model;

[0008] extracting a first item embedding representation and a first user embedding representation from the first input information through a collaborative filtering module;

[0009] Performing feature fusion on the semantic feature representation and the project feature representation through a dual-view feature fusion module to obtain a first fused feature representation; wherein the semantic feature representation includes a first semantic code embedding, and the project feature representation includes a first project embedding representation;

[0010] Replacing corresponding tags in the prompt template with the user feature representation and the first fused feature representation to obtain a first prompt; wherein the user feature representation includes a first user embedding representation;

[0011] The next recommended item is selected from the candidate item list according to the first prompt using the large language model.

[0012] Optionally, before fusing the semantic feature representation and the project feature representation by the dual-view feature fusion module, the method further includes:

[0013] Projecting the first semantic code embedding, the first item embedding representation, and the first user embedding representation into a latent feature space respectively through a projection module to obtain a first semantic feature representation, a first item embedding feature representation, and a first user embedding feature representation;

[0014] The semantic feature representation includes a first semantic feature representation, the item feature representation includes a first item embedding feature representation, and the user feature representation includes a first user embedding feature representation.

[0015] Optionally, the small language model adopts a BERT model;

[0016] The step of compressing the first input information using the small language model and outputting the first semantic encoding embedding specifically includes:

[0017] The first input information is input into a BERT model. After encoding the semantic information of the first input information through the BERT model, the output of the [CLS] tag position of the BERT model is embedded as the first semantic code.

[0018] Optionally, the large language model-based recommendation method further includes:

[0019] Before outputting the first semantic encoding embedding after compressing the first input information by the small language model, pre-training the small language model, the projection module, and the large language model is performed according to the following steps to align the small language model with the large language model:

[0020] Generate corresponding instruction prompts based on the text information of each interaction history item in the original dataset , sample input and sample output; wherein the text information of the sample input and the sample output are consistent; wherein the original data set includes text information of each interaction history item of multiple sample users;

[0021] Compressing each sample input through the small language model and outputting a second semantic encoding embedding;

[0022] The second semantic code embedding is projected into the latent feature space through the projection module to obtain the second semantic feature representation;

[0023] Using the second semantic feature to represent the replacement instruction prompt The characteristic mark in gets the second prompt;

[0024] The second prompt is input into the large language model, and the LoRA fine-tuning method is used to fine-tune the small language model, the projection module and the large language model in combination with the sample output corresponding to each sample input.

[0025] Optionally, the original data set further includes a candidate list of the next interaction and the next interaction item for each interaction history item of each sample user;

[0026] The recommendation method based on the large language model further includes:

[0027] After compressing the first input information through the small language model and outputting the first semantic encoding embedding, and before extracting the first item embedding representation and the first user embedding representation from the first input information through the collaborative filtering module, the collaborative filtering module, the dual-view feature fusion module, the prompt template, and the pre-trained small language model, the projection module, and the large language model are fine-tuned using the LoRA fine-tuning method using the text information of the interaction history of each sample user in the original data set, the text information of the candidate item list for the next interaction, and the text information of the item for the next interaction.

[0028] Optionally, fine-tuning the prompt template includes:

[0029] The disturbance level of each soft instruction prompt block in the prompt template is increased according to the following formula:

[0030] ;

[0031] in, Indicates the disturbance degree of each soft instruction prompt block in the prompt template. The initial value of the disturbance degree of each soft instruction prompt block in the prompt template is 0. Indicates the current training round, Indicates the preset total number of training rounds, Indicates the maximum disturbance level.

[0032] Optionally, the prompt template includes a first marker, a first marker block, a second marker block, and an output prompt text, wherein the first marker and the first marker block are separated by a first soft instruction prompt block, the first marker block and the second marker block are separated by a second soft instruction prompt block, and the second marker block and the output prompt text are separated by a third soft instruction prompt block;

[0033] The step of replacing corresponding marks in the prompt template with the user feature representation and the first fused feature representation to obtain the first prompt specifically includes:

[0034] Replacing the first tag in the prompt template with the user feature representation;

[0035] replacing the corresponding second mark in the first mark block of the prompt template by the fused feature representation of each interaction history item in the first fused feature representation;

[0036] The first prompt is obtained by replacing the corresponding third mark in the second mark block of the prompt template with the fused feature representation of each candidate item in the first fused feature representation.

[0037] In a second aspect, the present application provides a recommendation system based on a large language model, comprising:

[0038] a semantic encoding module, configured to compress first input information using a small language model and output a first semantic encoding embedding; wherein the first input information is an input sequence formed by text information of a target user's interaction history and a candidate list, the text information including a title and a text description, and the small language model is aligned with the large language model;

[0039] a collaborative filtering module, configured to extract a first item embedding representation and a first user embedding representation from the first input information;

[0040] A dual-view feature fusion module, configured to fuse the semantic feature representation and the project feature representation to obtain a first fused feature representation; wherein the semantic feature representation includes a first semantic code embedding, and the project feature representation includes a first project embedding representation;

[0041] a prompt design module, configured to obtain a first prompt by replacing corresponding tags in a prompt template with a user feature representation and a first fused feature representation; wherein the user feature representation includes a first user embedding representation;

[0042] An item recommendation module is used to select a next recommended item from the candidate item list according to the first prompt using the large language model.

[0043] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned large language model-based recommendation methods.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-mentioned recommendation methods based on a large language model.

[0045] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0046] This application provides a recommendation method, system, device, and medium based on a large language model. By aligning a small language model with a large language model, the small language model is ensured to accurately compress the first input information (the input sequence formed by the target user's interaction history and the text information of the candidate list), avoiding information loss during the compression process. This ensures that the first semantic encoding embedding output by the small language model can retain the semantics of the first input information to the greatest extent possible.

[0047] After aligning the small language model with the large language model, the small language model compresses the first input information (an input sequence formed by the target user's interaction history and the text information of the candidate list, including the title and text description) and outputs the first semantic code embedding. This allows the small language model to compress the first input information into a single vector, significantly reducing the input length of the large language model and avoiding directly inputting the first input information into the large language model. This avoids the problem of the length of the first input information or the combined length of the first input information and the output of the large language model exceeding the length limit of the large language model's context, as well as the problem of noise in the text information of the interaction history and candidate list affecting the recommendation effect of the large language model, thus preliminarily improving the recommendation effect of the large language model.

[0048] A collaborative filtering module extracts a first item embedding representation from the first input information, and a dual-view feature fusion module fuses the semantic feature representation (including the first semantic code embedding) and the item feature representation (including the first item embedding representation) to obtain a first fused feature representation. This integrates the semantic features and item features of the first input information while achieving a comprehensive item representation, further improving the recommendation effect of the large language model.

[0049] The collaborative filtering module extracts the first user embedding representation (interaction information of the first input information) and replaces the corresponding tags in the prompt template with the user feature representation (including the first user embedding representation) and the first fused feature representation. This integrates the collaborative representation information of the first input information (item feature representation and user feature representation) into the first prompt, making the first prompt more accurate in its contribution to the large language model and further improving the recommendation effect of the large language model.

[0050] Through the large language model, according to the first prompt, the next recommended item is selected from the candidate list to achieve item recommendation;

[0051] In summary, this application comprehensively improves the recommendation effect of the recommendation system based on the large language model, thereby meeting diverse recommendation needs and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 This is a diagram of an application environment of a recommendation method based on a large language model in one embodiment of the present application;

[0054] Figure 2 A flowchart of a recommendation method based on a large language model provided in one embodiment of the present application;

[0055] Figure 3 for Figure 1 A detailed flowchart of the steps for aligning the small language model with the large language model during the training phase.

[0056] Figure 4 A schematic diagram of a fine-tuning process of a prompt template provided in another embodiment of the present application;

[0057] Figure 5 A schematic diagram of the functional modules of a recommendation system based on a large language model provided in one embodiment of the present application;

[0058] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0061] The recommendation method based on the large language model provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the user's interaction history and candidate list to the server 104. After the server 104 receives the interaction history and candidate list, for the received interaction history and candidate list, the server 104 aligns the small language model with the large language model in the training phase, compresses the first input information using the small language model, and outputs a first semantic encoding embedding. The collaborative filtering module extracts the item embedding representation of the first input information and the user embedding representation of the interaction history in the first input information. The dual-view feature fusion module performs feature fusion on the semantic feature representation and the item embedding feature representation to obtain a first fused feature representation. The corresponding tag in the prompt template is replaced by the user feature representation and the first fused feature representation to obtain a first prompt. The first prompt is input into the recommender based on the large language model, and the recommender based on the large language model selects the next recommended item from the candidate list. The server 104 can feedback the obtained recommendation result to the terminal 102. In addition, in some embodiments, the recommendation method based on the large language model can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly give the next recommended item based on the interaction history and the candidate list, or the server 104 can obtain the interaction history and the candidate list from the data storage system and give the next recommended item based on the interaction history and the candidate list.

[0062] Terminal 102 may include, but is not limited to, various desktop computers, laptops, smartphones, tablet computers, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers, or may be a cloud server.

[0063] In an exemplary embodiment, Figure 2 As shown, a recommendation method based on a large language model is provided. The method is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the process, including the following steps 201 to 205.

[0064] Step 201 : compressing first input information through a small language model and outputting a first semantic encoding embedding; wherein the first input information is an input sequence formed by the interaction history of the target user and the text information of the candidate list.

[0065] In this embodiment of the present application, text information includes titles and text descriptions. A small language model refers to a small-scale language model (SLM), which has a relatively small number of parameters and requires low computational resources. The target user is the specific target group for recommending items (such as information, products, videos, etc.).

[0066] The interaction history refers to the titles and text descriptions of the target user's interaction history items (items they have viewed), as well as the target user's favorites, likes, and / or comments on the interaction history items, when recommending the target user the next interaction item. The candidate list is a list of items that may be displayed to the target user next time.

[0067] Aligning the small language model with the large language model enables the large language model to decode the textual information input to the small language model based on the semantic encoding embeddings (including the first and second semantic encoding embeddings) output by the small language model. Aligning the small and large language models ensures that the small language model accurately compresses the first input information (the input sequence formed by the target user's interaction history and the textual information in the candidate list), avoiding information loss during the compression process. This ensures that the first semantic encoding embedding output by the small language model retains the semantic meaning of the first input information to the greatest extent possible.

[0068] Step 202: extract a first item embedding representation and a first user embedding representation from the first input information through a collaborative filtering module.

[0069] In an embodiment of the present application, the first item embedding representation includes the item embedding representation of each interaction history item in the first input information and the item embedding representation of each candidate item in the candidate item list. The item embedding representation is the item vector representation obtained by mapping the title and text description of the interaction history item or candidate item through the corresponding embedding layer, or the item vector representation output by the neural network after the item vector representation output by the embedding layer is further input into the constructed neural network. The first user embedding representation includes the user embedding representation of each interaction history item in the first input information. The user embedding representation is the first user vector representation obtained by mapping the target user's ID and discrete features (such as the city where the user is located, the title of the item that the target user is interested in in the interaction history, etc.) through the corresponding embedding layer, normalizing the target user's continuous features (such as the target user's age, activity level, etc.) to obtain a second user vector representation, and then splicing the first user vector representation and the second user vector representation to obtain a third user vector representation, or the user vector representation output by the neural network after the third user vector representation is further input into the constructed neural network.

[0070] Step 203 , performing feature fusion on the semantic feature representation and the project feature representation through a dual-view feature fusion module to obtain a first fused feature representation; wherein the semantic feature representation includes a first semantic code embedding, and the project feature representation includes a first project embedding representation.

[0071] In the embodiment of the present application, weighted feature fusion is performed on the semantic feature representation and the project feature representation.

[0072] Step 204 : Replace the corresponding mark in the prompt template with the user feature representation and the first fused feature representation to obtain a first prompt; wherein the user feature representation includes the first user embedded representation.

[0073] In the embodiment of the present application, the prompt template is a pre-designed prompt template that includes an instruction prompt and a mark. The corresponding mark in the prompt template is replaced by the first user embedded feature representation and the first fused feature representation, i.e., the mark corresponding to the first user embedded feature representation is replaced by the first user embedded feature representation, and the mark corresponding to the first fused feature representation is replaced by the first fused feature representation, thereby obtaining a complete prompt.

[0074] Step 205 : Using the large language model and according to the first prompt, select the next recommended item from the candidate list.

[0075] By implementing steps 201 to 205 above, the small language model is aligned with the large language model to ensure that the small language model accurately compresses the first input information (the input sequence formed by the target user's interaction history and the text information in the candidate list), avoiding information loss during the compression process. This ensures that the first semantic encoding embedding output by the small language model retains the semantics of the first input information to the greatest extent possible. After aligning the small language model with the large language model, the small language model compresses the first input information (the input sequence formed by the target user's interaction history and the text information in the candidate list, including the title and description) and then outputs the first semantic encoding embedding. This compresses the first input information into a single vector using the small language model, significantly reducing the input length of the large language model and avoiding the need to directly input the first input information into the large language model. This avoids the problem of the length of the first input information or the combined length of the first input information and the output of the large language model exceeding the context length limit of the large language model, as well as the problem of noise in the interaction history and the text information in the candidate list affecting the recommendation performance of the large language model, thereby initially improving the recommendation performance of the large language model. The first project embedding representation is extracted from the first input information through the collaborative filtering module, and the semantic feature representation (including the first semantic coding embedding) and the project feature representation (including the first project embedding representation) are fused through the dual-view feature fusion module to obtain the first fused feature representation. While integrating the semantic features and project features of the first input information, a comprehensive project representation is achieved, further improving the recommendation effect of the large language model. The first user embedding representation (interaction information of the first input information) is extracted through the collaborative filtering module, and the corresponding tags in the prompt template are replaced by the user feature representation (including the first user embedding representation) and the first fused feature representation to achieve the integration of the collaborative representation information (project feature representation and user feature representation) of the first input information into the first prompt, making the first prompt more accurate in its prompting effect on the large language model, further improving the recommendation effect of the large language model. Through the large language model, according to the first prompt, the next recommended item is selected from the candidate list to achieve item recommendation. In summary, the present application comprehensively improves the recommendation effect of the recommendation system based on the large language model, thereby meeting diverse recommendation needs and improving user experience.

[0076] In another exemplary embodiment of the present application, to ensure that the dimensions of semantic feature representation, item feature representation, and user feature representation are consistent, the above-mentioned recommendation method based on a large language model, before step 203, further includes:

[0077] In step 301, the first semantic code embedding, the first item embedding representation, and the first user embedding representation are projected into a latent feature space through a projection module to obtain a first semantic feature representation, a first item embedding feature representation, and a first user embedding feature representation.

[0078] In the embodiment of the present application, the latent feature space is the feature space where the first semantic feature representation, the first item embedding feature representation and / or the first user embedding representation are located, and its dimension is A B, A = m1 + m2, where m1 is the number of interaction history items in the target user's interaction history, m2 is the number of candidate items in the candidate list, and B is the input embedding dimension of the large language model.

[0079] Accordingly, at this time, the semantic feature representation includes the first semantic feature representation, and the item feature representation includes the first item embedded feature representation, i.e., step 203 performs feature fusion on the first semantic feature representation and the first item embedded feature representation via the dual-view feature fusion module. The user feature representation includes the first user embedded feature representation, i.e., step 204 replaces the corresponding mark in the prompt template with the first user embedded feature representation and the first fused feature representation.

[0080] The above-mentioned first semantic coding embedding, first item embedding representation, and first project embedding representation are projected into the latent feature space through the projection module in order to make the dimensions of the semantic feature representation and the item feature representation input to the dual-view feature fusion module consistent, so as to perform feature fusion on the two, and to ensure that the first prompt obtained by replacing the corresponding tags in the prompt template by the user feature representation and the first fused feature representation meets the input embedding dimension requirements of the large language model.

[0081] In another exemplary embodiment of the present application, the above-mentioned projection module adopts a multi-layer perceptron (MLP). By training the multi-layer perceptron, the multi-layer perceptron can convert the input first semantic coding embedding, first item embedding representation, and first user embedding representation into a first semantic feature representation (vector representation), a first item embedding feature representation (vector representation), and a first user embedding feature representation (vector representation) as output.

[0082] In another exemplary embodiment of the present application, the above-mentioned small language model adopts the BERT model.

[0083] Accordingly, in the above step 201, compressing the first input information by using the small language model and outputting the first semantic code embedding includes:

[0084] The first input information is input into the BERT model. After the semantic information of the first input information is encoded by the BERT model, the output of the [CLS] tag position of the BERT model is embedded as the first semantic code.

[0085] In the embodiment of the present application, after the entire input sequence is encoded by the BERT model, the vector output at the [CLS] tag position is the representation of the entire input text information (first input information).

[0086] In another exemplary embodiment of the present application, the above-mentioned recommendation method based on a large language model further includes:

[0087] Before step 201, the small language model, the projection module, and the large language model are pre-trained according to the following steps (1) to (5) to align the small language model with the large language model.

[0088] Step (1) generates corresponding instruction prompts based on the text information of each interaction history item in the original dataset. , sample input and sample output ; Among them, the sample input and sample output The text information is consistent, such as Figure 3 shown.

[0089] Step (2), compression through small language model And output the second semantic encoding embedding ( semantic encoding embedding).

[0090] Step (3), embed the second semantic code through the projection module ( Figure 3 The projection layer) is projected into the latent feature space to obtain the second semantic feature representation ( semantic feature representation).

[0091] Step (4): Use the second semantic feature to represent the replacement instruction prompt The feature mark in the second prompt .

[0092] In the embodiment of this application, There is no specific limit, you can set it according to your actual needs. For example, Figure 3 As shown, set If "[Item Emb]'s title and text description are", then [Item Emb] is feature markers.

[0093] Step (5), the second prompt Input to the large language model, use LoRA (low rank adaptation) fine-tuning method and combine each Corresponding Fine-tune the small language model, projection module, and large language model.

[0094] In the embodiment of the present application, the above-mentioned pre-training ensures that the first semantic coding embedding and the second semantic coding embedding generated by the small language model can restore the original text information.

[0095] In the pre-training phase, the pre-training weights of the large language model are kept unchanged, while integrating the trainable low-rank decomposition matrix. The optimization objective of the LoRA fine-tuning method is defined as the following formula:

[0096] ;

[0097] in, Represents the learnable parameters of the overall framework consisting of the small language model, projection module and large language model in the pre-training phase, Indicates the parameters of the large language model itself that are frozen during the pre-training phase, Represents sample output No. j Tokens, Represents sample output Before j -1 word, Represents sample output The total number of words, Indicates that the parameter and Next, given and Output under the condition The probability of Represents the logarithmic function.

[0098] In another exemplary embodiment of the present application, the above-mentioned original data set also includes a list of candidate items for the next interaction and the item for the next interaction for each interaction history item of each sample user.

[0099] At this point, the steps for pre-training the small language model, projection module, and large language model also include:

[0100] Before step (1), an item dataset is derived from the original dataset, where the items include the interaction history items and candidate items of each sample user.

[0101] Accordingly, the above step (1) includes:

[0102] Generate corresponding instruction prompts based on the text information of each project in the project dataset , sample input and sample output .

[0103] In another exemplary embodiment of the present application, the above-mentioned prompt template includes a first mark, a first mark block, a second mark block and an output prompt text, wherein the first mark and the first mark block are separated by a first soft instruction prompt block, the first mark block and the second mark block are separated by a second soft instruction prompt block, and the second mark block and the output prompt text are separated by a third soft instruction prompt block.

[0104] For example, Figure 4 As shown, the green soft instruction prompt block is the first soft instruction prompt block, the blue soft instruction prompt block is the second soft instruction prompt block, and the orange soft instruction prompt block is the third soft instruction prompt block.

[0105] In an embodiment of the present application, the output prompt text (such as "The recommended next item is") is used to prompt the large language model to output the recommended next item based on the first prompt. Specifically, the large language model outputs the title of the recommended next item.

[0106] The first soft instruction prompt block is used to separate the first tag and the first tag block, the second soft instruction prompt block is used to separate the first tag block and the second tag block, and the third soft instruction prompt block is used to separate the second tag block and the output prompt text, thereby effectively avoiding confusion among the first tag, the second tag, the third tag and the output prompt text. This enables the large language model to accurately identify the user feature representation in the first prompt, the fused feature representation of the interaction history item in the first fused feature representation, the fused feature representation of the candidate item in the first fused feature representation, and the output prompt text, thereby accurately understanding the semantics of the first prompt and giving the next recommended item.

[0107] In another exemplary embodiment of the present application, the above step 204 includes:

[0108] The first tag in the prompt template is replaced by the user feature representation, the corresponding second tag in the prompt template is replaced by the fused feature representation of each interaction history item in the first fused feature representation, and the corresponding third tag in the prompt template is replaced by the fused feature representation of each candidate item in the first fused feature representation.

[0109] In this embodiment of the present application, the number of second tags is consistent with the number of interaction history items in the interaction history, and the fused feature representation of each interaction history item in the first fused feature representation replaces the corresponding second tag in the prompt template. The number of third tags is consistent with the number of candidate items in the candidate list, and the fused feature representation of each candidate item in the first fused feature representation replaces the corresponding third tag in the prompt template.

[0110] In another exemplary embodiment of the present application, the first tag uses the [User Rep] tag, and the second and third tags are the same, both using the [Titel, Emb] tag.

[0111] In another exemplary embodiment of the present application, the first mark, the first soft instruction prompt block, the first mark block, the second soft instruction prompt block, the second mark block, the third soft instruction prompt block and the output prompt text in the prompt template are arranged in sequence from front to back.

[0112] In the embodiment of the present application, placing the first tag at the beginning of the prompt template helps the large language model to better understand the preferences of the target user based on the generated first prompt.

[0113] In another exemplary embodiment of the present application, the above-mentioned recommendation method based on a large language model further includes:

[0114] Step 401, before steps 201 and 202, the collaborative filtering module, the dual-view feature fusion module, the prompt template, and the pre-trained small language model, the projection module, and the large language model are fine-tuned using the LoRA fine-tuning method based on the text information of the interaction history of each sample user in the original data set, the text information of the candidate list for the next interaction, and the text information of the item for the next interaction.

[0115] In an embodiment of the present application, step 201 uses the small language model obtained by fine-tuning in step 401 to compress the first input information and then output the first semantic coding embedding. Step 202 uses the collaborative filtering module obtained by fine-tuning in step 401 to extract the first project embedding representation and the first user embedding representation from the first input information. Step 203 uses the dual-view feature fusion module obtained by fine-tuning in step 401 to perform feature fusion on the semantic feature representation and the project feature representation. Step 204 replaces the corresponding mark in the prompt template obtained by fine-tuning in step 401 with the user feature representation and the first fused feature representation to obtain the first prompt. Step 205 uses the large language model obtained by fine-tuning in step 401 to select the next recommended item from the candidate list according to the first prompt. Step 301 uses the projection module obtained by fine-tuning in step 401 to project the first semantic coding embedding, the first project embedding representation, and the first user embedding representation to the latent feature space.

[0116] In another exemplary embodiment of the present application, the above step 401 includes the following steps 501 to 507. Among them:

[0117] Step 501: compress the second input information through the pre-trained small language model and output a third semantic coding embedding; wherein the second input information is an input sequence formed by the interaction history information of each sample user in the original data set, and the interaction history information of each sample user includes the text information of each interaction history item, the list of candidate items for the next interaction, and the text information of the item for the next interaction.

[0118] Step 502 : Projecting the third semantic code embedding into the latent feature space through the pre-trained projection module to obtain a third semantic feature representation.

[0119] Step 503: Input the second input information into the collaborative filtering module, and extract and output the second item embedding representation and the second user embedding representation through the collaborative filtering module.

[0120] In step 504, the second item embedding representation and the second user embedding representation are projected into the latent feature space by the pre-trained projection module to obtain the second item embedding feature representation and the second user embedding feature representation.

[0121] Step 505 : Perform feature fusion on the third semantic code embedding and the second item embedding representation through a dual-view feature fusion module to obtain a second fused feature representation.

[0122] In the embodiment of the present application, different weights of the dual-view feature fusion module are trained according to the following formula to obtain the first fused feature representation and the second fused feature representation:

[0123] ;

[0124] in, Indicates the k a first fused feature representation (a first historical interaction fused feature representation or a first candidate item fused feature representation) or a second fused feature representation (a second historical interaction fused feature representation or a second candidate item fused feature representation), represents the second item embedding feature representation (the second interaction history item embedding feature representation or the second candidate item embedding feature representation), represents the third semantic feature representation (the third interaction history semantic feature representation or the third candidate list semantic feature representation), represents the semantic feature weight, Represents the item feature weight.

[0125] Step 506: Replace the corresponding mark in the prompt template with the second user embedded representation and the second fused feature representation to obtain a third prompt.

[0126] In an embodiment of the present application, the first tag in the prompt template is replaced by the second user embedding representation, the corresponding second tag in the first tag block of the prompt template is replaced by the fused feature representation of each interaction history item in the second fused feature representation, and the corresponding third tag in the second tag block of the prompt template is replaced by the fused feature representation of each candidate item in the second fused feature representation.

[0127] Step 507: Input the third prompt into the pre-trained large language model, adopt the LoRA fine-tuning method and combine the next interaction item of each sample user to fine-tune the collaborative filtering module, dual-view feature fusion module, prompt template, and the pre-trained small language model, projection module, and large language model.

[0128] In the embodiment of the present application, the pre-trained weights of the large language model are kept unchanged, and a trainable low-rank decomposition matrix is ​​integrated. The optimization objective of the LoRA fine-tuning method is defined as the following formula:

[0129] ;

[0130] in, Represents the learnable parameters of the entire framework consisting of the collaborative filtering module, dual-view feature fusion module, prompt template, pre-trained small language model, projection module and recommender based on the large language model in the fine-tuning stage, represents the frozen parameters of the large language model and collaborative filtering model itself during the fine-tuning phase, Indicates the next interaction item that the user actually clicks on in the candidate list in a training sample. express The jth token of express The first j-1 tokens of represents the third prompt for input to the recommender based on the large language model, express The total number of words, Indicates that the parameter and Next, give the third prompt and Output under the condition The probability of Represents the logarithmic function.

[0131] In another exemplary embodiment of the present application, in order to add interference in the fine-tuning stage, minimize the manual work in preparing the prompt template, and increase the robustness of the third prompt, in step 507, fine-tuning the prompt template includes:

[0132] Increase the perturbation level of each soft instruction prompt block in the prompt template according to the following formula:

[0133] ;

[0134] in, Indicates the perturbation degree of each soft instruction prompt block in the prompt template. The initial value of the perturbation degree of each soft instruction prompt block in the prompt template is 0. Indicates the current training round, Indicates the preset total number of training rounds, Indicates the maximum disturbance level.

[0135] In the embodiment of this application, Figure 4 As shown in the green, blue and orange blocks, the prompt template includes multiple soft instruction prompt blocks, each of which can be expressed as , Indicates the t The mth vector of the soft instruction prompt block, the perturbation degree of each soft instruction prompt block is the ratio of the perturbation vector in each soft instruction prompt block, that is, the ratio of the number of perturbation vectors in each soft instruction prompt block to the total number of vectors m. The perturbation vector is Figure 4 The vectors framed by red dots in the figure have different positions in the two training rounds.

[0136] Before fine-tuning the prompt template, the first soft instruction prompt block, the second soft instruction prompt block, and the third soft instruction prompt block in the prompt template are obtained by random initialization, and the dimensions of the first soft instruction prompt block, the second soft instruction prompt block, and the third soft instruction prompt block meet the input embedding dimension requirements of the large language model.

[0137] By using the progressive learning strategy of the embodiment of the present application that transitions from simple tasks to challenging tasks, the generalization ability and convergence speed of the model can be improved while adopting soft instruction prompts.

[0138] Based on the same inventive concept, embodiments of the present application also provide a large language model-based recommendation system for implementing the large language model-based recommendation method described above. The implementation solution provided by this system is similar to the implementation solution described in the above-mentioned method. Therefore, the specific limitations of one or more of the following large language model-based recommendation system embodiments can be found in the limitations of the large language model-based recommendation method described above and will not be repeated here.

[0139] In an exemplary embodiment, Figure 5 As shown, a recommendation system 60 based on a large language model is provided, comprising:

[0140] Semantic encoding module 601, configured to compress first input information using a small language model and output a first semantic encoding embedding; wherein the first input information is an input sequence formed by the target user's interaction history and text information of a candidate list, the text information including a title and a text description, and the small language model is aligned with the large language model;

[0141] Collaborative filtering module 602, configured to extract a first item embedding representation and a first user embedding representation from the first input information; the first item embedding representation and the first user embedding representation are described in detail in the above method embodiment and will not be repeated here;

[0142] A dual-view feature fusion module 603 is configured to fuse the semantic feature representation and the project feature representation to obtain a first fused feature representation; wherein the semantic feature representation includes a first semantic code embedding, and the project feature representation includes a first project embedding representation;

[0143] a prompt design module 604 configured to replace corresponding tags in the prompt template using the user feature representation and the first fused feature representation to obtain a first prompt; wherein the user feature representation includes the first user embedding representation;

[0144] The item recommendation module 605 is configured to select the next recommended item from the candidate item list according to the first prompt using the large language model.

[0145] In another exemplary embodiment of the present application, the aforementioned large language model-based recommendation system 60 further includes:

[0146] The projection module is used to project the above-mentioned first semantic coding embedding, first project embedding representation, and first user embedding representation into the latent feature space before the semantic feature representation and the project feature representation are fused through the dual-view feature fusion module to obtain the first semantic feature representation, the first project embedding feature representation, and the first user embedding feature representation.

[0147] In the embodiment of the present application, the latent feature space is the feature space where the first semantic feature representation, the first item embedding feature representation and / or the first user embedding representation are located, and its dimension is A B, A = m1 + m2, where m1 is the number of interaction history items in the target user's interaction history, m2 is the number of candidate items in the candidate list, and B is the input embedding dimension of the large language model.

[0148] In another exemplary embodiment of the present application, the above-mentioned projection module adopts a multi-layer perceptron (MLP). By training the multi-layer perceptron, the multi-layer perceptron can convert the input first semantic coding embedding, first item embedding representation, and first user embedding representation into a first semantic feature representation (vector representation), a first item embedding feature representation (vector representation), and a first user embedding feature representation (vector representation) as output.

[0149] In another exemplary embodiment of the present application, the above-mentioned small language model adopts the BERT model.

[0150] Accordingly, the semantic encoding module 601 is further configured to:

[0151] The first input information is input into the BERT model. After the semantic information of the first input information is encoded by the BERT model, the output of the [CLS] tag position of the BERT model is embedded as the first semantic code.

[0152] In the embodiment of the present application, after the entire input sequence is encoded by the BERT model, the vector output at the [CLS] tag position is the representation of the entire input text information (first input information).

[0153] In another exemplary embodiment of the present application, the aforementioned large language model-based recommendation system 60 further includes:

[0154] The training module is configured to pre-train the small language model, the projection module, and the large language model according to the following steps before outputting the first semantic encoding embedding after compressing the first input information through the small language model, so as to align the small language model with the large language model:

[0155] Compression through small language models And output the second semantic encoding embedding ( semantic encoding embedding of );

[0156] The second semantic code is embedded through the projection module ( Figure 3 The projection layer) is projected into the latent feature space to obtain the second semantic feature representation ( semantic feature representation of );

[0157] Using the second semantic feature to represent the replacement instruction prompt The feature mark in the second prompt ;

[0158] The second tip Input to the large language model, use LoRA (low rank adaptation) fine-tuning method and combine each Corresponding Fine-tune the small language model, projection module, and large language model.

[0159] In the embodiment of the present application, the above-mentioned pre-training ensures that the first semantic coding embedding and the second semantic coding embedding generated by the small language model can restore the original text information.

[0160] In the pre-training phase, the pre-training weights of the large language model are kept unchanged, while integrating the trainable low-rank decomposition matrix. The optimization objective of the LoRA fine-tuning method is defined as the following formula:

[0161] ;

[0162] in, Represents the learnable parameters of the overall framework consisting of the small language model, projection module and large language model in the pre-training phase, Indicates the parameters of the large language model itself that are frozen during the pre-training phase, Represents sample output No. j Tokens, Represents sample output Before j -1 word, Represents sample output The total number of words, Indicates that the parameter and Next, given and Output under the condition The probability of Represents the logarithmic function.

[0163] In another exemplary embodiment of the present application, the above-mentioned original data set also includes a list of candidate items for the next interaction and the item for the next interaction for each interaction history item of each sample user.

[0164] Accordingly, the above training module is also used to:

[0165] An item dataset is derived from the original dataset, where the items include interaction history items and candidate items of each sample user;

[0166] Generate corresponding instruction prompts based on the text information of each project in the project dataset , sample input and sample output .

[0167] In another exemplary embodiment of the present application, the above-mentioned prompt template includes a first mark, a first mark block, a second mark block and an output prompt text, wherein the first mark and the first mark block are separated by a first soft instruction prompt block, the first mark block and the second mark block are separated by a second soft instruction prompt block, and the second mark block and the output prompt text are separated by a third soft instruction prompt block.

[0168] For example, Figure 4 As shown, the green soft instruction prompt block is the first soft instruction prompt block, the blue soft instruction prompt block is the second soft instruction prompt block, and the orange soft instruction prompt block is the third soft instruction prompt block.

[0169] The output prompt text (such as “The recommended next item is”) is used to prompt the large language model to output the recommended next item according to the first prompt. Specifically, the large language model outputs the title of the recommended next item.

[0170] Accordingly, the above-mentioned prompt design module 604 is further used to:

[0171] The first tag in the prompt template is replaced by the user feature representation, the corresponding second tag in the prompt template is replaced by the fused feature representation of each interaction history item in the first fused feature representation, and the corresponding third tag in the prompt template is replaced by the fused feature representation of each candidate item in the first fused feature representation.

[0172] In this embodiment of the present application, the number of second tags is consistent with the number of interaction history items in the interaction history, and the fused feature representation of each interaction history item in the first fused feature representation replaces the corresponding second tag in the prompt template. The number of third tags is consistent with the number of candidate items in the candidate list, and the fused feature representation of each candidate item in the first fused feature representation replaces the corresponding third tag in the prompt template.

[0173] In another exemplary embodiment of the present application, the first tag uses the [User Rep] tag, and the second and third tags are the same, both using the [Titel, Emb] tag.

[0174] In another exemplary embodiment of the present application, the first mark, the first soft instruction prompt block, the first mark block, the second soft instruction prompt block, the second mark block, the third soft instruction prompt block and the output prompt text in the prompt template are arranged in sequence from front to back.

[0175] In the embodiment of the present application, placing the first tag at the beginning of the prompt template helps the large language model to better understand the preferences of the target user based on the generated first prompt.

[0176] In another exemplary embodiment of the present application, the above-mentioned training module is further used to:

[0177] After compressing the first input information through the small language model, outputting the first semantic encoding embedding and before extracting the first item embedding representation and the first user embedding representation from the first input information through the collaborative filtering module, the LoRA fine-tuning method is used to fine-tune the collaborative filtering module, the dual-view feature fusion module, the prompt template, and the pre-trained small language model, the projection module, and the large language model using the text information of the interaction history of each sample user in the original dataset, the text information of the candidate item list for the next interaction, and the text information of the item for the next interaction.

[0178] In another exemplary embodiment of the present application, the above-mentioned training module is further used to:

[0179] The second input information is compressed using a pre-trained small language model and then outputted as a third semantic encoding embedding; wherein the second input information is an input sequence formed by the interaction history information of each sample user in the original dataset, and the interaction history information of each sample user includes the text information of each interaction history item, a list of candidate items for the next interaction, and the text information of the item for the next interaction;

[0180] The third semantic encoding embedding is projected into the latent feature space through the pre-trained projection module to obtain the third semantic feature representation;

[0181] Inputting the second input information into the collaborative filtering module, extracting and outputting a second item embedding representation and a second user embedding representation through the collaborative filtering module;

[0182] The second item embedding representation and the second user embedding representation are respectively projected into the latent feature space by the pre-trained projection module to obtain the second item embedding feature representation and the second user embedding feature representation;

[0183] The third semantic code embedding and the second item embedding representation are fused through a dual-view feature fusion module to obtain a second fused feature representation;

[0184] Replacing the corresponding mark in the prompt template by the second user embedding representation and the second fused feature representation to obtain a third prompt;

[0185] The third prompt is input into the pre-trained large language model. The LoRA fine-tuning method is used and combined with the next interaction item of each sample user, the collaborative filtering module, dual-view feature fusion module, prompt template, and the pre-trained small language model, projection module and large language model are fine-tuned.

[0186] In the embodiment of the present application, in the fine-tuning stage, different weights of the dual-view feature fusion module are trained according to the following formula to obtain the first fused feature representation and the second fused feature representation:

[0187] ;

[0188] in, Indicates the k a first fused feature representation (a first historical interaction fused feature representation or a first candidate item fused feature representation) or a second fused feature representation (a second historical interaction fused feature representation or a second candidate item fused feature representation), represents the second item embedding feature representation (the second interaction history item embedding feature representation or the second candidate item embedding feature representation), represents the third semantic feature representation (the third interaction history semantic feature representation or the third candidate list semantic feature representation), represents the semantic feature weight, Represents the item feature weight.

[0189] The first tag in the prompt template is replaced by the second user embedding representation, the corresponding second tag in the first tag block of the prompt template is replaced by the fused feature representation of each interaction history item in the second fused feature representation, and the corresponding third tag in the second tag block of the prompt template is replaced by the fused feature representation of each candidate item in the second fused feature representation.

[0190] Keeping the pre-trained weights of the large language model unchanged and integrating the trainable low-rank decomposition matrix, the optimization objective of the LoRA fine-tuning method is defined as the following formula:

[0191] ;

[0192] in, Represents the learnable parameters of the entire framework consisting of the collaborative filtering module, dual-view feature fusion module, prompt template, pre-trained small language model, projection module and recommender based on the large language model in the fine-tuning stage, represents the frozen parameters of the large language model and collaborative filtering model itself during the fine-tuning phase, Indicates the next interaction item that the user actually clicks on in the candidate list in a training sample. express The jth token of express The first j-1 tokens of represents the third prompt for input to the recommender based on the large language model, express The total number of words, Indicates that the parameter and Next, give the third prompt and Output under the condition The probability of Represents the logarithmic function.

[0193] In another exemplary embodiment of the present application, the above-mentioned training module is further used to:

[0194] Increase the perturbation level of each soft instruction prompt block in the prompt template according to the following formula:

[0195] ;

[0196] in, Indicates the perturbation degree of each soft instruction prompt block in the prompt template. The initial value of the perturbation degree of each soft instruction prompt block in the prompt template is 0. Indicates the current training round, Indicates the preset total number of training rounds, Indicates the maximum disturbance level.

[0197] In the embodiment of this application, Figure 4 As shown in the green, blue and orange blocks, the prompt template includes multiple soft instruction prompt blocks, each of which can be expressed as , Indicates the t The mth vector of the soft instruction prompt block, the perturbation degree of each soft instruction prompt block is the ratio of the perturbation vector in each soft instruction prompt block, that is, the ratio of the number of perturbation vectors in each soft instruction prompt block to the total number of vectors m. The perturbation vector is Figure 4 The vectors framed by red dots in the figure have different positions in the two training rounds.

[0198] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store first input information data and an original data set. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a recommendation method based on a large language model is implemented.

[0199] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0200] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0201] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0202] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0203] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0204] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0205] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0206] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0207] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A recommendation method based on a large language model, characterized in that: The recommendation method based on the large language model includes: Outputting a first semantic encoding embedding after compressing first input information using a small language model; wherein the first input information is an input sequence formed by the target user's interaction history and text information of a candidate list, and the small language model is aligned with the large language model; extracting a first item embedding representation and a first user embedding representation from the first input information through a collaborative filtering module; Performing feature fusion on the semantic feature representation and the project feature representation through a dual-view feature fusion module to obtain a first fused feature representation; wherein the semantic feature representation includes a first semantic code embedding, and the project feature representation includes a first project embedding representation; Replacing corresponding tags in the prompt template with the user feature representation and the first fused feature representation to obtain a first prompt; wherein the user feature representation includes a first user embedding representation; The next recommended item is selected from the candidate item list according to the first prompt using the large language model.

2. The recommendation method based on a large language model according to claim 1, characterized in that Before the feature fusion of the semantic feature representation and the project feature representation is performed by the dual-view feature fusion module, the method further includes: Projecting the first semantic code embedding, the first item embedding representation, and the first user embedding representation into a latent feature space respectively through a projection module to obtain a first semantic feature representation, a first item embedding feature representation, and a first user embedding feature representation; The semantic feature representation includes a first semantic feature representation, the item feature representation includes a first item embedding feature representation, and the user feature representation includes a first user embedding feature representation.

3. The recommendation method based on a large language model according to claim 1, characterized in that The small language model adopts the BERT model; The step of compressing the first input information using the small language model and outputting the first semantic encoding embedding specifically includes: The first input information is input into a BERT model. After encoding the semantic information of the first input information through the BERT model, the output of the [CLS] tag position of the BERT model is embedded as the first semantic code.

4. The recommendation method based on a large language model according to claim 2, characterized in that Also includes: Before outputting the first semantic encoding embedding after compressing the first input information by the small language model, pre-training the small language model, the projection module, and the large language model is performed according to the following steps to align the small language model with the large language model: Generate corresponding instruction prompts based on the text information of each interaction history item in the original dataset , sample input and sample output; wherein the text information of the sample input and the sample output are consistent; wherein the original data set includes text information of each interaction history item of multiple sample users; Compressing each sample input through the small language model and outputting a second semantic encoding embedding; The second semantic code embedding is projected into the latent feature space through the projection module to obtain the second semantic feature representation; Using the second semantic feature to represent the replacement instruction prompt The characteristic mark in gets the second prompt; The second prompt is input into the large language model, and the LoRA fine-tuning method is used to fine-tune the small language model, the projection module and the large language model in combination with the sample output corresponding to each sample input.

5. The recommendation method based on a large language model according to claim 4, characterized in that The original data set also includes a candidate list of the next interaction and the next interaction item for each interaction history item of each sample user; The recommendation method based on the large language model further includes: After compressing the first input information through the small language model and outputting the first semantic encoding embedding, and before extracting the first item embedding representation and the first user embedding representation from the first input information through the collaborative filtering module, the collaborative filtering module, the dual-view feature fusion module, the prompt template, and the pre-trained small language model, the projection module, and the large language model are fine-tuned using the LoRA fine-tuning method using the text information of the interaction history of each sample user in the original data set, the text information of the candidate item list for the next interaction, and the text information of the item for the next interaction.

6. The recommendation method based on a large language model according to claim 5, characterized in that Fine-tuning the prompt template includes: The disturbance level of each soft instruction prompt block in the prompt template is increased according to the following formula: ； in, Indicates the disturbance degree of each soft instruction prompt block in the prompt template. The initial value of the disturbance degree of each soft instruction prompt block in the prompt template is 0. Indicates the current training round, Indicates the preset total number of training rounds, Indicates the maximum disturbance level.

7. The recommendation method based on a large language model according to claim 1, characterized in that The prompt template includes a first marker, a first marker block, a second marker block, and an output prompt text, wherein the first marker and the first marker block are separated by a first soft instruction prompt block, the first marker block and the second marker block are separated by a second soft instruction prompt block, and the second marker block and the output prompt text are separated by a third soft instruction prompt block; The step of replacing corresponding marks in the prompt template with the user feature representation and the first fused feature representation to obtain the first prompt specifically includes: Replacing the first tag in the prompt template with the user feature representation; replacing the corresponding second mark in the first mark block of the prompt template by the fused feature representation of each interaction history item in the first fused feature representation; The first prompt is obtained by replacing the corresponding third mark in the second mark block of the prompt template with the fused feature representation of each candidate item in the first fused feature representation.

8. A recommendation system based on a large language model, characterized in that: The recommendation system based on the large language model includes: a semantic encoding module, configured to compress first input information using a small language model and output a first semantic encoding embedding; wherein the first input information is an input sequence formed by text information of a target user's interaction history and a candidate list, the text information including a title and a text description, and the small language model is aligned with the large language model; a collaborative filtering module, configured to extract a first item embedding representation and a first user embedding representation from the first input information; A dual-view feature fusion module, configured to fuse the semantic feature representation and the project feature representation to obtain a first fused feature representation; wherein the semantic feature representation includes a first semantic code embedding, and the project feature representation includes a first project embedding representation; a prompt design module, configured to obtain a first prompt by replacing corresponding tags in a prompt template with a user feature representation and a first fused feature representation; wherein the user feature representation includes a first user embedding representation; An item recommendation module is used to select a next recommended item from the candidate item list according to the first prompt using the large language model.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the large language model-based recommendation method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the recommendation method based on a large language model according to any one of claims 1 to 7 are implemented.

Images