A text sequence recommendation method and system based on large language model

Abstract

The present invention discloses a text sequence recommendation method and system based on a large language model, belonging to the technical field of recommendation algorithms, and comprising: a data preprocessing stage, a large language model pre-training stage, a sequence model fine-tuning stage, and a matching stage. By introducing a large language model into the text sequence recommendation task, the present invention can utilize the rich pre-training corpus of the large language model to better model the text; at the same time, by modeling the text in sequence, the ability of the large model for sequence recommendation modeling is activated, breaking away from the ID-based recommendation paradigm in traditional recommendation algorithms, and better performing recommendation task learning and processing in cold start scenarios and knowledge transfer scenarios; finally, through the final optimization of the recommendation results by the sequence model, the problem of hallucinations and inability to accurately match the output results of the previous large language model can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of recommendation algorithms, and in particular to a text sequence recommendation method and system based on a large language model. Background Art

[0002] In recent years, with the rapid development of computer technology, the internet, and related industries, massive amounts of text-based data are generated daily on the internet, presenting the characteristics of big data. Simultaneously, AI-generated content generation technologies are also rapidly developing, further accelerating the surge of data and information. The question of how to quickly filter content of genuine interest to individuals from this vast amount of data has driven research in recommender systems. Recommender systems leverage user behavior data and text information to provide personalized recommendations. They analyze user history and text features, using algorithmic models to predict content that users may be interested in and then recommend it. These systems are widely used in text-based fields, such as news push, e-book recommendations, and academic article retrieval, aiming to enhance user experience, strengthen user engagement, and promote the effective dissemination of content. The development of recommender systems not only has a significant impact on users and businesses but also provides rich practical opportunities for research and application in information technology and artificial intelligence.

[0003] Sequential recommendation, as a key scenario in recommendation systems, has long garnered widespread attention. In recommendation systems, the temporal information of user behavior is crucial. Effectively leveraging past behavior sequences to predict a user's next behavior directly impacts recommendation accuracy and user experience. The key to solving the sequential recommendation problem lies in inferring a user's preferences from their historical behavior sequences through a series of technical means, and then performing personalized recommendations or searches based on the order of their behaviors. Existing methods for sequential recommendation scenarios suffer from ID-based encoding logic, which underutilizes the textual information in historical behavior information, misaligns behavior sequence patterns with the recommendation algorithm space, and uses large language models, leading to inaccurate recommendations due to large language hallucinations. Summary of the Invention

[0004] In order to solve the above problems, the purpose of the present invention is to provide a text sequence recommendation technology based on a large language model, aiming to overcome the insufficient utilization of historical text content information in the text sequence recommendation scenario. By activating the sequence recommendation capability of the large language model and optimizing it in the semantic space and vector space at the same time, the impact of the hallucination problem caused by the large model in recommendation tasks is alleviated, thereby achieving better recommendation results.

[0005] To achieve the above technical objectives, this application provides a text sequence recommendation method based on a large language model, which includes the following stages:

[0006] Data preprocessing stage: Convert the historical text sequence data clicked by the user into a set of attribute key-value pairs to construct a dataset for large model training;

[0007] Pre-training stage of the large language model: Construct the dataset into a prompt training set for the large language model, convert the sequential recommendation task into a natural language task and then perform model training to obtain the best text sequence recommendation results at the semantic space level;

[0008] Fine-tuning stage of the sequence model: Optimize the results based on the large language model in the sequential recommendation task by fine-tuning the sequence model;

[0009] Matching stage: Based on the processing results of the large language model stage and the sequence model stage, match the text content in the dimension of the embedding vector space and output the results of sequential recommendation.

[0010] Preferably, in the data preprocessing stage, the historical text sequence data is processed by removing punctuation marks, removing HTML tags, removing stop words, removing extra spaces, and word form reduction;

[0011] Construct the processed data into a format including attribute keys and values, sort it in reverse order according to the interaction time, and select the interaction sequences with more than 5 interactions as the dataset.

[0012] Preferably, in the pre-training stage of the large language model, construct a missing corpus prompt dataset by obscuring the value in a certain sequence in the interaction sequence;

[0013] Construct a contrastive learning prompt dataset according to the user interaction text data and the un-interacted text data;

[0014] According to the missing corpus dataset and the contrastive learning prompt dataset, divide them according to the ratio of training set: validation set: test set of 8:1:1 to construct a prompt training set.

[0015] Preferably, in the pre-training stage of the large language model, use a low-rank matrix as a supplement to the large model fine-tuning parameters, and merge the original parameters with the parameters of the low-rank matrix to form the overall parameters of the large language model for model training.

[0016] Preferably, in the pre-training stage of the large language model, the goal of training the large model is:

[0017]

[0018] In the formula, x represents the given task, y

[0018] is the output of the large model, y<t represents the output content between the large model output y t , P u represents the overall parameters of the large language model.

[0019] Preferably, during the sequence model fine-tuning stage, Langformer is used as the sequence model to prevent excessively long interaction sequences from affecting the calculation results and to speed up the calculation efficiency of the attention mechanism.

[0020] Preferably, during the sequence model fine-tuning phase, a reference vector is selected, and a vector obtained after training with the large language model is used as a positive sample of the reference vector, and a vector obtained without training with the large language model is used as a negative sample of the reference vector;

[0021] Based on the benchmark vector, according to the positive and negative samples, by setting the perturbation factor, a contrastive learning loss function is constructed for fine-tuning the downstream task. The contrastive learning loss function is expressed as:

[0022]

[0023] Where, E final represents the base vector, represents a positive sample, represents negative samples, α1 represents the perturbation factor, L c Represents the contrastive learning Loss function.

[0024] Preferably, in the process of fine-tuning the downstream task, the multilayer perceptron is used as the downstream task and is transferred using cross entropy Loss, where the cross entropy Loss is expressed as:

[0025]

[0026] Where n is the number of samples, y i For real click behavior, if clicked, it is 1, if not clicked, it is 0. is the predicted probability output by the model;

[0027] For the sequence model, the fine-tuning overall Loss function is: L = L b +L c .

[0028] Preferably, in the matching stage, the text data to be recommended is obtained, reasoning is performed through the large model, the output result of the large model is obtained, and the embedding vector set of the text data to be recommended is reasoned with using the downstream task model, and the final text recommendation is performed based on the cosine similarity between the reasoning result and the vector in the embedding vector set to be recommended.

[0029] The present invention discloses a text sequence recommendation system based on a large language model, which is used to implement the above-mentioned text sequence recommendation method based on a large language model. The system includes:

[0030] The data preprocessing module is used to convert the historical text sequence data of user clicks into a set of attribute key-value pairs to construct a dataset for large model training;

[0031] The large language model pre-training module is used to construct the dataset into a prompt word training set for the large language model, convert the sequence recommendation task into a natural language task, and then perform model training to obtain the best text sequence recommendation results at the semantic space level;

[0032] The sequence model fine-tuning module is used to fine-tune the sequence model to optimize the results based on the large language model in the sequence recommendation task;

[0033] The matching stage module is used to match text content in the embedding vector space dimension based on the processing results of the large language model stage and the sequence model stage, and output the results of sequence recommendation.

[0034] The present invention discloses the following technical effects:

[0035] This invention optimizes text sequence recommendation by introducing a large language model. Leveraging the model's more comprehensive training corpus, it enhances the utilization of textual information contained in historical behavioral data. Furthermore, based on contrastive learning and sequence construction prompts, it stimulates the large language model's sequence recommendation capabilities. Addressing existing issues such as misalignment between sequence patterns and the recommendation algorithm space, as well as large language model hallucinations, this invention introduces a sequence model for fine-tuning and optimization at the vector level, effectively matching different spaces. By fitting vectors, this alleviates the large language model hallucinations and effectively improves the performance of text sequence recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 is a flowchart of the steps of the method described in the present invention;

[0038] Figure 2 is a schematic diagram of training data processing according to the present invention;

[0039] Figure 3 Schematic diagram of the large language model training process of the present invention;

[0040] Figure 4 Schematic diagram of fine-tuning the sequence model according to the present invention;

[0041] Figure 5 This is a schematic diagram of the sequence recommendation result matching described in the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of this application.

[0043] like Figure 1-5 As shown, the main steps of the text sequence recommendation algorithm technology based on the large language model provided by the present invention are: training data preprocessing, training the large language model, fine-tuning the sequence model, and matching the sequence recommendation results; specifically, it includes the following contents:

[0044] S1. Training data preprocessing:

[0045] Based on the text content clicked by the user, the historical text sequence data of the user's click is constructed, and the recommended sequence text dataset required in the subsequent steps is constructed by converting it into a set of attribute key-value pairs. Each text is represented by an attribute dictionary, which includes different attribute feature information of the text. The processing process is finally as follows Figure 2 shown.

[0046] S1.1 Training data preprocessing:

[0047] Processing the collected training data includes removing punctuation: removing punctuation from the text to reduce noise. Removing HTML tags: If the captured text contains HTML tags, use regular expressions or HTML parsers (such as BeautifulSoup) to clean it up. Removing stop words: Stop words (such as "the", "is", etc.) may not be helpful for the task, so delete these words to reduce the input dimension. Removing extra spaces: There may be extra spaces or line breaks in the text, which are cleaned up uniformly. At the same time, perform lemmatization: restore the words to the dictionary form, for example, "running" becomes "run". You can use the lemmatization tool in spaCy or NLTK.

[0048] S1.2 Text sequence key-value pair dataset construction:

[0049] Based on the data processed in step S1.1, a text sequence key-value pair dataset is constructed according to the task. For the text data clicked by the user, its data is constructed as a dictionary value of {attribute key 1: value 1; attribute 2: value 2; ...; attribute key n: value n}, represented as D. For the attribute key and value of the i-th position, use Keyi and Value i For a user, the text sequence he has interacted with is composed of D sequences and is recorded as Seq (D∈Seq). The overall data processing flow is as follows: Figure 1 As shown in Figure 2, the length of interaction sequences for each user is not necessarily the same. To reduce the impact of insufficient interactions on model training, interaction sequences with more than 5 interactions are selected as the final dataset.

[0050] S2. Training large language model:

[0051] like Figure 3 As shown, based on the data set obtained in step S1, a prompt word training set of the large language model is constructed, the sequence recommendation task is converted into a natural language task, and the large language model is pulled for training at the same time to obtain the best text sequence recommendation results at the semantic space level.

[0052] S2.1 Large language model acquisition:

[0053] Large language models are usually divided into open source and paid. Open source models are usually model weights, such as Meta's L1ama series and Alibaba's Qwen series; paid models are usually called through API interfaces, such as OpenAI's ChatGPT series. This article takes the L1ama series as an example, and the other large language models can be slightly adjusted. The obtained original large model parameters are represented as L0.

[0054] S2.2 Construction of a large language model training prompt word dataset

[0055] Use the dataset constructed in step S1.2 to construct the large model prompt word, and construct the missing corpus prompt dataset by blocking the value in a sequence in the interaction sequence, denoted as C q Blocking certain content in the sequence and letting the big language model complete it can significantly improve the big language model's ability to learn the sequence content. mask ∈C q , the characteristics of its construction are as follows:

[0056] Q: Users have interacted with the following text content. The content sorted in reverse order of interaction time is [Seq mask ], please predict the missing content;

[0057] A: The complete sequence data of user interaction is [Seq];

[0058] In the above content, Seqmas k It is a text interaction sequence to be supplemented after masking certain positions, and Seq is a complete text sequence of user interaction.

[0059] Construct a comparative learning prompt dataset based on user interaction text data and non-interaction text data, denoted as C c By constructing contrastive learning prompts at the text level, the core idea of ​​contrastive learning, “bringing similarities closer and dissimilarities further apart”, is instilled into the large language model through natural language. Cc ∈C c , the characteristics of its construction are as follows:

[0060] The user clicks on the item [D i ], and clicked the item [D j ], but it does not click on the item [D k ];

[0061] The missing corpus dataset and the comparative learning prompt dataset are divided into training set: validation set: test set ratio of 8:1:1. The training set, validation set, and test set are represented by T cq , T cc 、V cq , V cc 、T cq , T cc To express.

[0062] S2.2 Train a large language model based on the prompt dataset:

[0063] For the original large model L o , it is a time-consuming and labor-intensive task to fully train its parameters. In order to facilitate fine-tuning training, a low-rank matrix is ​​used as a supplement to the fine-tuning parameters of the large model. The principle of the LoRA method is to use rank decomposition, and then merge the trained parameters with the original parameters of the large model. Since the parameters of the original model are not trained, only a few parameters are needed to approach the effect of the original training of all parameters. The original parameters of the large language model are recorded as P o , the parameters of the low-rank matrix after training are P l . The original parameter P o and the parameter P of the low-rank matrix l After merging, the overall parameter of the large language is expressed as P u , which can be expressed by the following formula: P u =P o +P l .

[0064] The overall task of step S2 is to make the trained large language model L t The output y t Can be as close as possible to the given task x and the output y t Previous output content y <t , which can maximize the output yt. Its overall optimization goal is shown in the following formula:

[0065]

[0066] S2.3 Get the original text interaction embedding vector:

[0067] Based on the text attribute key-value pair dictionary D constructed in step S1.2, it is input into the large language model L trained in step S2.2 t In the example, since the original structure of the large language model is a multi-layer Transformer structure, we refer to the structure of the Transformer model and take the vector output by its embedding layer as D in the model L. t Attribute key value embedding vector E under model encoding d , the process can be expressed as follows:

[0068] E d =L t (D);

[0069] This is convenient for subsequent fine-tuning of the sequence model. Based on the data sequence Seq in step S1.2 and the attribute key-value pair dictionary D, based on the trained large language model L t To sequence training embedding vector E seq , its training process can be expressed as follows:

[0070] E seq =L t (Seq(D1, D2, ..., D n ));

[0071] S3. Fine-tuning the sequence model:

[0072] In order to align the sequence recommendation results at a more fine-grained level, the sequence model for the sequence prediction task is fine-tuned based on the dataset constructed in step S1 using the text embedding vector output by the large language model trained in step S2, thereby enhancing the sequence prediction information at the embedding vector level and reducing the impact of the hallucination output of the large language model on the final result. The overall process is as follows: Figure 4 shown.

[0073] S3.1 Initialize the sequence model tensor:

[0074] For the sequence model, to prevent the influence of overly long interaction sequences on the calculation results and to accelerate the calculation efficiency of the attention mechanism, Langformer is selected as the sequence model in this invention. Its structure is similar to the traditional Transformer model, but the attention layer mechanism is optimized, converting the traditional self-attention module into a sliding window attention module. The sliding window attention module stipulates that each token can only see a set number w of tokens in the window. Therefore, its complexity is reduced from O(n 2 ) to O(n*w) where w << n. For the sequence model Langformer, the sequence model trained in step S2.3 is used as its initial embedding vector. Before the initial embedding vector, the initial embedding vector of the sequence identifier [CLS] is added where n is the size of each training batch, and m is the vector dimension when initializing E cls , and its value is randomly initialized. After the final initialization, the model tensor E init can be expressed as:

[0075] E init = E cls || E seq ;

[0076] After the initialized vector is trained and fitted by Langformer, the Langformer output vector E final is obtained.

[0077] S3.2 Fine-tuning the downstream task based on the sequence model:

[0078] For the sequence model, based on the initialized tensor in step S3.1, the downstream task is fine-tuned. The downstream task is mainly the matching score Score output of the embedding vector E final output by the sequence model and the attribute key-value embedding vector obtained from the training in the previous step S2.3. For each user U i The fine-tuning training steps are as follows:

[0079] - Take the embedding vector E final after the training in step S3.1 as the reference vector

[0080] - Take the dictionary D i of the interaction text attribute key-value pairs of user U k after the training sequence, and the vector t obtained after the training of the large language model L as the positive sample comparison of the reference vector E final

[0081] - Take user U i ​The dictionary D of interactive text attribute key-value pairs that are not performed in the entire text data k , in the large language model L t The vector obtained after training As the reference vector E final Negative sample comparison

[0082] -Select 1 positive sample and h negative samples, construct a comparative learning Loss function for learning, and compare the learning Loss function L c As shown in the following formula, where α1 is the disturbance factor:

[0083]

[0084] The downstream task model here can use a multi-layer perceptron. In the present invention, the downstream task model adopts a four-layer structure. Specifically, the first layer is a linear layer, and the dimensional space is Among them, m and n are the input feature dimension size and output feature dimension size respectively. The second layer is a nonlinear layer, and the activation function is the tanh function. The third layer is a linear layer, and the dimension space is Where k is the attribute key-value pair dictionary vector E output from the large language model D The dimensions are the same; the fourth layer calculates E final With E D The score and cross entropy Loss transfer can be expressed as follows:

[0085]

[0086] Where n is the number of samples, y i For real click behavior, if clicked, it is 1, if not clicked, it is 0. is the predicted probability output by the model.

[0087] For the sequence model, the fine-tuning overall Loss function is: L = L b +L c .

[0088] S4.Sequence recommendation result matching:

[0089] For users who need to be predicted, their historical interaction sequences are generated according to step S1, and prompt words are constructed at the same time. The prompt word interaction is performed on the model in S2 after training to obtain the output recommendation results; the recommendation results are then input into the sequence model to generate the embedded text vector for final retrieval, and matching is performed in the vector database based on the embedded text vector, and the K bits with the highest matching scores are output as the recommendation results.

[0090] S4.1 Recommended data collation:

[0091] For the text data to be recommended, organize it into the attribute key-value pair form required for model input according to step S1.1, and construct the prompt use case required by the large model according to step S2.2.

[0092] S4.2 Large Model Reasoning:

[0093] Construct a prompt using the prompt use case organized in step S4.1 final as follows:

[0094] The following is a sequence of text content data [Seq] that the user has interacted with. Please output the text content that the user may have interacted with in the format of {attribute key 1: attribute value 1, ..., attribute key n: attribute value n}, taking into account the content that may appear at the same time as the content that the user has interacted with, and avoid content that is unlikely to appear at the same time.

[0095] Take the final output content {attribute key 1: attribute value 1, ..., attribute key n: attribute value n} as the text recommendation result output by the large language model, and use rep fianl to represent it and input it into the large language model L as input data t , and get its sequence embedding vector representation At the same time, all the text data attribute key-value pairs dictionary D to be matched are input into the large language model L t , and its embedding vector is represented as E D and store it in the vector database.

[0096] S4.2 Sequence model downstream task matching:

[0097] For the large model output obtained in step S4.2 And the embedding vector set C to be recommended emd , use the downstream task model to perform reasoning and obtain its final representation embedding vector E ult Calculate the cosine similarity between it and the vector in the embedding vector set to be recommended, and select the k with the highest similarity for the final text recommendation. The process can be expressed as follows:

[0098]

[0099] Final rep =TopK(Sim)

[0100] Final in the above formula rep Represents the final recommendation result.

[0101] By introducing a large language model into the text sequence recommendation task, the present invention can utilize the rich pre-trained corpus of the large language model to better model the text; at the same time, by performing sequence modeling on the text, the ability of the large model for sequence recommendation modeling is activated, breaking away from the ID-based recommendation paradigm in traditional recommendation algorithms, and better performing recommendation task learning and processing in cold start scenarios and knowledge transfer scenarios; finally, through the final optimization of the recommendation results by the sequence model, the problem of hallucinations and inability to accurately match the output results of the previous large language model can be solved.

[0102] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0104] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A text sequence recommendation method based on a large language model, characterized in that: The following stages are included: Data preprocessing stage: Convert historical text sequence data of user clicks into a set of attribute key-value pairs to construct a dataset for large model training; Large language model pre-training stage: The dataset is constructed as a prompt word training set for the large language model, and the sequence recommendation task is converted into a natural language task before model training to obtain the best text sequence recommendation results at the semantic space level; Sequence model fine-tuning stage: By fine-tuning the sequence model, the results based on the large language model are optimized for the sequence recommendation task; Matching stage: Based on the processing results of the large language model stage and the sequence model stage, the text content is matched in the embedding vector space dimension and the sequence recommendation results are output; During the sequence model fine-tuning phase, Langformer is used as the sequence model to prevent excessively long interaction sequences from affecting the computational results and to improve the computational efficiency of the attention mechanism. During the sequence model fine-tuning phase, a reference vector is selected, and the vector obtained after training with the large language model is used as a positive sample of the reference vector, and the vector obtained without training with the large language model is used as a negative sample of the reference vector; Based on the reference vector, according to the positive sample and the negative sample, by setting the perturbation factor, a contrastive learning Loss function is constructed for fine-tuning the downstream task, wherein the contrastive learning Loss function is expressed as: Where, E final represents the base vector, represents a positive sample, represents negative samples, α1 represents the perturbation factor, L c Represents the contrastive learning Loss function.

2. The text sequence recommendation method based on a large language model according to claim 1, characterized in that: In the data preprocessing stage, the historical text sequence data is processed by removing punctuation marks, removing HTML tags, removing stop words, removing redundant spaces, and restoring the word form; The processed data is constructed into a format including attribute keys and values, and is sorted in reverse order of interaction time, and interaction sequences with a number of interactions greater than 5 are selected as the data set.

3. The text sequence recommendation method based on a large language model according to claim 2, characterized in that: During the pre-training phase of the large language model, a missing corpus prompt dataset is constructed by masking the values ​​in a certain sequence in the interaction sequence. Construct a comparative learning prompt dataset based on user interaction text data and non-interaction text data; The missing corpus dataset and the comparative learning prompt dataset are divided into a training set: a validation set: a test set ratio of 8:1:1 to construct the prompt word training set.

4. The text sequence recommendation method based on a large language model according to claim 3, characterized in that: During the pre-training stage of the large language model, a low-rank matrix is ​​used as a supplement to the fine-tuning parameters of the large model. The original parameters are merged with the parameters of the low-rank matrix to form the overall parameters of the large language model for model training.

5. The text sequence recommendation method based on a large language model according to claim 4, characterized in that: During the large language model pre-training phase, the goals of training the large model are: In the formula, x represents the given task, y t is the output of the large model, y ＜t Represents the large model output y t Output content between P u Represents the overall parameters of the large language model.

6. The text sequence recommendation method based on a large language model according to claim 1, characterized in that: In the process of fine-tuning the downstream task, the multi-layer perceptron is used as the downstream task and transferred using cross entropy Loss, where the cross entropy Loss is expressed as: Where n is the number of samples, y i For real click behavior, if clicked, it is 1, if not clicked, it is 0. is the predicted probability output by the model; For the sequence model, the fine-tuning overall Loss function is: L = L b +L c .

7. The text sequence recommendation method based on a large language model according to claim 6, characterized in that: In the matching stage, the text data to be recommended is obtained, and reasoning is performed through the large model to obtain the output result of the large model. The output result and the embedding vector set of the text data to be recommended are reasoned using the downstream task model, and the final text recommendation is performed based on the cosine similarity between the reasoning result and the vector in the embedding vector set to be recommended.

8. A text sequence recommendation system based on a large language model, characterized in that: The system is used to execute the text sequence recommendation method based on a large language model as described in claim 1, and the system includes: The data preprocessing module is used to convert the historical text sequence data of user clicks into a set of attribute key-value pairs to construct a dataset for large model training; A large language model pre-training module is used to construct the dataset into a prompt word training set for the large language model, convert the sequence recommendation task into a natural language task, and then perform model training to obtain the best text sequence recommendation results at the semantic space level; The sequence model fine-tuning module is used to fine-tune the sequence model to optimize the results based on the large language model in the sequence recommendation task; The matching stage module is used to match text content in the embedding vector space dimension based on the processing results of the large language model stage and the sequence model stage, and output the results of sequence recommendation.

Images