Recommended method, model training method, device, equipment and readable storage medium

Abstract

The application discloses a recommendation method and model training method and device, equipment and readable storage medium in the technical field of computer application, and the method comprises the steps of obtaining historical text information of an accessed text and candidate text information of a candidate text; inputting the historical text information and the candidate text information into a prompt template to obtain an answer word; selecting a mark from the answer word and converting the historical text information, the candidate text information and the mark into a natural language sentence; inputting the natural language sentence into a natural language processing model, predicting the mark by using an answer space, and obtaining a mark probability; and selecting a recommended text from a plurality of candidate texts based on the mark probability. The technical effect of the application is that the language information of a real world large-scale corpus can be fully utilized, and an individualized text intelligent recommendation task is converted into a mask prediction task based on multi-template prompt learning, so that content interested by a user is recommended.

Description

Technical Field

[0001] This invention relates to the field of computer application technology, and in particular to a recommendation method, model training method, apparatus, device, and readable storage medium. Background Technology

[0002] Personalized text intelligent recommendation systems, as a knowledge transfer assistant, an infrastructure for generating user stickiness, and an increase in user click-through rates, can not only serve as a core backend technology module for large internet companies, but also be integrated with search engines or used as a standalone system to effectively help users find the descriptions they are most interested in from a large number of documents, thereby alleviating the problem of information overload and broadening their horizons.

[0003] Most existing neural network recommendation methods focus on designing various ingenious neural networks to encode text and user representations, with core modules including a text encoder, a user encoder, and a similarity metric. While these neural network recommendation methods can achieve personalized text recommendations, the limitations of the neural network model itself mean that the recommendation results still cannot fully meet user needs.

[0004] In conclusion, how to effectively solve problems such as personalized text recommendation is a technical issue that urgently needs to be addressed by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a recommendation method, apparatus, device, and readable storage medium that transforms the task of personalized text intelligent recommendation into a mask prediction task based on multi-template prompt learning, thereby recommending content that users are interested in.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] One recommended approach includes:

[0008] Retrieve historical text information of visited texts and candidate text information of candidate texts;

[0009] Input the historical text information and the candidate text information into the prompt template to obtain the answer words;

[0010] Select tags from the answer words, and convert the historical text information, the candidate text information, and the tags into natural language sentences;

[0011] The natural language statement is input into a natural language processing model, and the label is predicted using the answer space to obtain the label probability.

[0012] Based on the labeled probabilities, a recommended text is selected from the multiple candidate texts.

[0013] Preferably, the historical text information of the visited text and the candidate text information of the candidate text are obtained, including:

[0014] Read the historical titles of each of the accessed texts;

[0015] By connecting the various historical titles, the historical text information is obtained;

[0016] Read the candidate titles of the candidate text and determine the candidate titles as the candidate text information.

[0017] Preferably, the historical text information is obtained by concatenating the various historical titles, including:

[0018] The historical text information is obtained by connecting the word sequences of each historical title using delimiters.

[0019] Preferably, the historical text information and the candidate text information are input into the prompt template to obtain the answer words, including:

[0020] The historical text information and the candidate text information are input into the discrete prompt template to perform a cloze test and obtain the answer words.

[0021] Preferably, the historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including:

[0022] The historical text information and the candidate text information are input into various discrete prompt models to perform cloze tests, resulting in multiple answer words.

[0023] Preferably, the historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including:

[0024] The historical text information and the candidate text information are input into the semantic relevance prompting model to predict semantic relevance and obtain semantically relevant answer words.

[0025] Preferably, the historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including:

[0026] The historical text information and the candidate text information are input into the sentiment prompting model to predict user sentiment and obtain user sentiment-related response words.

[0027] Preferably, the historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including:

[0028] The historical text information and the candidate text information are input into the user behavior prompting model to predict user behavior and obtain user behavior-related answer words.

[0029] Preferably, the historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including:

[0030] The historical text information and the candidate text information are input into the recommendation effect prompting model to predict the recommendation effect and obtain the answer words related to the recommendation effect.

[0031] Preferably, the historical text information and the candidate text information are input into the prompt template to obtain the answer words, including:

[0032] The historical text information and the candidate text information are input into a continuous prompt template with random virtual words to obtain the answer word.

[0033] Preferably, the historical text information and the candidate text information are input into the prompt template to obtain the answer words, including:

[0034] The historical text information and the candidate text information are input into a mixed prompt template with random virtual words to obtain the answer word.

[0035] Preferably, selecting markers from the answer words includes:

[0036] The affirmative answer words in the answer words are identified as the markers.

[0037] Preferably, the natural language statement is input into a natural language processing model, and the label is predicted using the answer space to obtain the label probability, including:

[0038] Using the natural language processing model, the labels in the natural sentences are mapped to answer words in the pre-trained language model vocabulary, and the probability of the answer words is predicted;

[0039] The probability of the answer word is determined as the tag probability of the corresponding tag.

[0040] Preferably, based on the labeled probabilities, a recommended text is selected from multiple candidate texts, including:

[0041] The ranking score of a candidate text is obtained by summing the tag probabilities of the candidate texts.

[0042] Based on the ranking score, the recommended text is selected from the multiple candidate texts.

[0043] Preferably, the ranking score of the candidate text is obtained by summing the tag probabilities of the candidate texts, including:

[0044] The ranking score is obtained by summing the tag probabilities of the candidate text similar prompt template output.

[0045] Preferably, the ranking score of the candidate text is obtained by summing the tag probabilities of the candidate texts, including:

[0046] The ranking score is obtained by summing the label probabilities of the different types of prompt templates output by the candidate text.

[0047] Preferably, selecting the recommended text from the plurality of candidate texts based on the ranking score includes:

[0048] The candidate texts are sorted based on the ranking score, and the candidate text with the highest ranking score is determined as the recommended text.

[0049] Preferably, it further includes:

[0050] The recommended text is sent to the client.

[0051] A model training method, comprising:

[0052] Obtain historical text information of visited texts and candidate text information of candidate texts, as well as the real tags of whether the candidate texts were visited after being recommended;

[0053] Based on the prompt template, the historical text information and the candidate text information are converted into tagged natural sentences;

[0054] The natural language statement is input into the natural language processing model to be trained, and the label is predicted using the answer space to obtain the label probability.

[0055] Based on the labeled probability and the true label, the loss value of the current natural language processing model is calculated, and the current natural language processing model is tuned based on the loss value until training is completed, so that the trained natural language processing model can be used as described above.

[0056] A recommended device includes:

[0057] The information acquisition module is used to acquire historical text information of accessed text and candidate text information of candidate text;

[0058] The prompt conversion module is used to input the historical text information and the candidate text information into the prompt template to obtain the answer words; select a marker from the answer words, and convert the historical text information, the candidate text information, and the marker into natural language sentences;

[0059] The label prediction module is used to input the natural sentence into the natural language processing model, predict the label using the answer space, and obtain the label probability;

[0060] The recommended selection module is used to select recommended text from multiple candidate texts based on the labeled probabilities.

[0061] An electronic device, comprising:

[0062] Memory, used to store computer programs;

[0063] A processor is configured to implement the steps of the recommended method described above when executing the computer program, or to implement the steps of the model training method described above when executing the computer program.

[0064] A readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the recommended method described above, or, when executed by a processor, implements the steps of the model training method described above.

[0065] The method provided in this embodiment of the invention is used to obtain historical text information of visited text and candidate text information of candidate text; input the historical text information and candidate text information into a prompt template to obtain answer words; select tags from the answer words and convert the historical text information, candidate text information and tags into natural language sentences; input the natural language sentences into a natural language processing model, use the answer space to predict tags, and obtain tag probabilities; and select recommended text from multiple candidate texts based on the tag probabilities.

[0066] In this invention, after acquiring historical text information and candidate text information, the historical text information and candidate text information can be input into a prompt template to obtain answer words. Then, tags are selected from the answer words, and the historical text information, candidate text information, and tags are converted into natural language sentences. These natural language sentences are then input into a natural language processing model to predict tags using the answer space and obtain tag probabilities. Finally, based on the tag probabilities, recommended text is selected from multiple candidate texts.

[0067] As can be seen, in this invention, historical text information and candidate text information can be converted into labeled natural sentences through the prompting model, which can make full use of the language information of the real-world large-scale corpus, and transform the personalized text intelligent recommendation task into a mask prediction task based on multi-template prompting learning, thereby recommending content that users are interested in.

[0068] Accordingly, embodiments of the present invention also provide model training methods, recommendation devices, equipment, and readable storage media corresponding to the above-described recommendation methods, which have the aforementioned technical effects, and will not be elaborated further here. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart illustrating the implementation of a recommended method in an embodiment of the present invention;

[0071] Figure 2 This is a schematic diagram of the structure of a natural language processing model in an embodiment of the present invention;

[0072] Figure 3 This is a schematic diagram of the structure of a recommended device in an embodiment of the present invention;

[0073] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention;

[0074] Figure 5 This is a schematic diagram of the specific structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0075] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0076] Please refer to Figure 1 , Figure 1 This is a flowchart of a recommended method in an embodiment of the present invention, which includes the following steps:

[0077] S101. Obtain historical text information of visited texts and candidate text information of candidate texts.

[0078] Historical text information refers to the text information of texts that users have clicked or accessed, such as the characteristics of the texts themselves (e.g., keywords) or the historical titles of the texts clicked. Candidate text information can also be related to candidate texts, such as their titles. In this article, "history" only indicates that the corresponding text has been accessed, and "candidate" only indicates that it is a candidate text.

[0079] Specifically, a text library can be created to store a number of texts. Based on whether a user has accessed the text, it can be determined which texts have been accessed. For candidate texts, generally, texts that the current user has not yet accessed can be used as candidate texts. Of course, all texts in the entire text library can also be used as candidate texts. Candidate texts are selected for recommendation. For example, newly generated or newly received texts can be used as candidate texts.

[0080] In this invention, text can specifically refer to information, news, terminology content, dialogue content, etc. That is, the method provided by this invention can be applied to internet applications such as information feed recommendations and news content recommendations. It can also be extended to other text-based recommendation tasks according to the needs of actual applications, such as search, sentiment classification, dialogue recommendations, and marketing script recommendations.

[0081] In one specific embodiment of the present invention, the historical text information of accessed text and the candidate text information of candidate text are obtained, including:

[0082] Step 1: Read the historical titles of each visited text file;

[0083] Step 2: Connect the various historical titles to obtain historical text information;

[0084] Step 3: Read the candidate titles of the candidate text and determine the candidate titles as candidate text information.

[0085] For ease of description, the three steps mentioned above will be explained together below.

[0086] By monitoring user clicks, we can identify the historical texts that the user has clicked / accessed, and then retrieve the historical titles of each text. Finally, we can organize the historical titles of each text.

[0087] Then, the various historical titles are connected to obtain the historical text information. Correspondingly, the candidate titles of the candidate texts are also determined as candidate text information.

[0088] It should be noted that in this invention, both historical titles and candidate titles are titles of text. The prefixes "historical" and "candidate" are only used to distinguish whether the corresponding text has been accessed or is being considered for recommendation.

[0089] In this invention, to avoid confusion, connecting various historical titles to obtain historical text information can specifically include: using delimiters to connect the word sequences of various historical titles to obtain historical text information. That is, using delimiters to connect the word sequences of different historical titles can connect various different historical titles, and can also distinguish different titles based on the delimiters.

[0090] For example, let U and T represent the user set and text set, respectively. Each t∈T mainly includes its title Title={w1, w2, …, wM}, where Title is a sequence of words and M is the number of words. For a given user u's click history Hu={T1, T2, …, TL} containing L clicked texts and candidate texts Tc, these can be converted into natural language sentences to adapt to the subsequent cue learning paradigm, represented as follows: <user>and <candidate>.for <user>This connects the document titles in the user's history (Hu), with a special token [NCLS] added at the beginning of each title to separate each clicked title. Here, NCLS represents N CLS (categories), i.e., separators, such as commas. For <candidate>This invention uses the title Tc of the candidate documents. Therefore, this invention formally represents them by the following formula:

[0091] <user>←Title1,…,Title L ;

[0092] <candidate>←Title C ;

[0093] Where {Title1, ..., TitleL} corresponds to the document titles in Hu = {T1, T2, ..., TL}. Intuitively, <user>This can be seen as a summary of the user's areas of interest. <candidate>This can be seen as a core semantic summary of the candidate text. Both of these serve as input data for subsequent prompt templates.

[0094] S102. Based on the prompt template, convert historical text information and candidate text information into tagged natural sentences.

[0095] For personalized text intelligent recommendation tasks, this invention utilizes cue learning for model training, where appropriate cue templates are beneficial to improving recommendation performance.

[0096] In this embodiment of the invention, the prompt template T(·) is used, i.e., x prompt =T( <user> ， <candidate>[MASK]), where MASK stands for marker, and is based on the input data ( <user> ， <candidate>The personalized text intelligent recommendation task is transformed into a cloze test task to predict [MASK], thereby obtaining a natural sentence with tags converted from historical text information and candidate text information.

[0097] In one specific embodiment of the present invention, based on a prompt template, historical text information and candidate text information are converted into tagged natural sentences, including:

[0098] Step 1: Input historical text information and candidate text information into the prompt template to obtain the answer words;

[0099] Step 2: Select tags from the response words and convert historical text information, candidate text information and tags into natural language.

[0100] For ease of description, the two steps above will be explained together below.

[0101] In this invention, historical text information and candidate text information can be input into a prompt template to obtain answer words. Then, tags are selected from the answer words, and the historical text information, candidate text information, and tags are converted into natural language.

[0102] In the process of selecting markers from the response words, affirmative response words can be identified as markers.

[0103] In one specific embodiment of the present invention, the prompt template can be a discrete prompt template, a continuous discrete template, or a hybrid prompt module. That is, step one above, inputting historical text information and candidate text information into the prompt template to obtain the answer word, includes:

[0104] Method 1: Input historical text information and candidate text information into the discrete prompt template to complete the cloze test and obtain the answer words.

[0105] Discrete cue templates, as the most common template type in cue learning, are usually based on some prior knowledge combined with user-interpretable natural language to form input data.

[0106] Specifically, historical text information and candidate text information are input into discrete prompt templates for cloze tests to obtain answer words. This includes inputting historical text information and candidate text information into multiple discrete prompt models for cloze tests to obtain multiple answer words. That is, this invention can design multiple discrete templates from various considerations, where each template corresponds to a method for measuring the matching signal between user interests and candidate texts. In other words, the embodiments of this invention aim to explore suitable [MASK] cloze tests as a similarity measure to solve personalized text intelligent recommendation tasks.

[0107] The following lists the discrete templates corresponding to different considerations and provides a detailed explanation of how to obtain the answer words.

[0108] For semantically relevant factors, that is, inputting historical text information and candidate text information into a discrete prompt template to complete the cloze test and obtain the answer words, including: inputting historical text information and candidate text information into a semantically relevant prompt model to predict semantic relevance and obtain semantically relevant answer words.

[0109] Semantic relevance involves examining whether the relevant text content aligns with the core motivation behind the user's interest. In other words, does the user have a sustained interest in certain specific topics and content? To this end, this invention transforms text recommendation requests into deterministic... <candidate>and <user>The correlation between them was analyzed, and the answer words were selected as relevant or irrelevant.

[0110] For user sentiment-related factors, historical text information and candidate text information are input into a discrete prompt template to complete a cloze test and obtain answer words. This includes inputting historical text information and candidate text information into a sentiment prompt model to predict user sentiment and obtain user sentiment-related answer words.

[0111] Among these, user sentiment: This study investigates whether users' emotional response to text is the most influential factor. That is, users choose to read a text as if that text best satisfies their emotional needs. Therefore, this invention uses the sentiment words "interesting" and "boring" as answers to estimate users' emotional response to text. <candidate>Emotional response.

[0112] For user behavior-related factors, historical text information and candidate text information are input into a discrete prompt template to complete the cloze test and obtain the answer words. This includes inputting historical text information and candidate text information into a user behavior prompt model to predict user behavior and obtain user behavior-related answer words.

[0113] Among these, user behavior was studied to determine whether MLM could directly serve as a click predictor. In other words, interest guides action, and action reflects interest. <user>After informing the MLM with CANDIDATE>, this invention allows the MLM to directly predict whether the user will click on this text, with the answer being either yes or no.

[0114] For factors related to recommendation effectiveness, historical text information and candidate text information are input into a discrete prompt template to complete a cloze test and obtain answer words. This includes inputting historical text information and candidate text information into a recommendation effectiveness prompt model to predict recommendation effectiveness and obtain answer words related to recommendation effectiveness.

[0115] The recommendation effect involves exploring whether the MLM (Multi-Level Model) judges the merits of recommending a candidate text, i.e., the utility of such a recommendation. To this end, this invention uses a prompting question to guide the MLM, with the answer being either "good" or "bad" as a prediction of recommendation utility.

[0116] The aforementioned prompt templates, while appearing to differ only slightly in the prompt template sentences and answer words, hold promise for representing semantic and linguistic knowledge in a PLM (Combining a pretrained language model) through predefined natural sentences and pre-selected answer words. Therefore, on the one hand, this invention recognizes that this is the core concept of prompt learning: predicting the probability of answer words from the PLM vocabulary is equivalent to inserting task-specific input sentences into a large corpus to guide the PLM model. On the other hand, while these manually designed prompt templates contain carefully crafted natural sentences, they clearly cannot exhaust all possible cases. Therefore, embodiments of this invention can also use some special tokens (words) to search for more cases within a template to achieve prompt learning. That is, the next method is to use consecutive prompt templates to obtain answer words.

[0117] Method 2: Input historical text information and candidate text information into a continuous prompt template with random virtual words to obtain the answer word.

[0118] In this embodiment of the invention, multiple consecutive prompt templates can also be set, with each prompt template corresponding to a discrete prompt template. Specifically, the invention can... <user> 、 <candidate>The [MASK] is preceded by several virtual learnable tokens, denoted as [[P1][P2]......[Pn]][[Q1][Q2]......[Qn]] and [[M1][M2]……[Mn]], where n is the number of virtual tokens. Furthermore, regarding the setting of the answer word and token positions, this embodiment of the invention can refer to the previous discrete hint template.

[0119] While continuous cue templates offer greater freedom to the model, the embedding of these virtual tokens is randomly initialized, which may introduce ambiguity and lead to insufficient utilization of knowledge by the PLM. Therefore, this invention also designs a hybrid template to attempt to combine the advantages of discrete and continuous cue templates.

[0120] Method 3: Input historical text information and candidate text information into a mixed prompt template with random virtual words to obtain the answer word.

[0121] Hybrid Prompt Template. In the hybrid prompt template, embodiments of the present invention retain the virtual tokens [Pi] and [Qj] in... <user>and <candidate>The preceding position aims to automatically search for an appropriate format to present information to the PLM model. Based on this, the present invention replaces the virtual token [Mk] with a natural language sentence for answer prediction; then, the present invention can design four representative natural sentences, each corresponding to a design template. Therefore, the hybrid prompt template consists of a continuous prompt template, positional tokens, and natural language sentences. Compared to the aforementioned continuous prompt template, the hybrid prompt template in the present invention can utilize the above prompt template to select a suitable virtual token, and then use natural language sentences to guide the answer direction of the discrete template.

[0122] In practical applications, at least one of the three methods mentioned above can be selected based on actual needs.

[0123] S103. Input the natural language statement into the natural language processing model, use the answer space to predict the label, and obtain the label probability.

[0124] Once the natural language statement is obtained, it can be input into the natural language processing model to predict the label using the answer space and obtain the label probability.

[0125] Natural language processing (NLP) models aim to enable computers to understand, process, and generate natural language. In this embodiment of the invention, any specific NLP model capable of predicting tags and thus obtaining tag probabilities can be used.

[0126] In one specific embodiment of this invention, a pre-trained language model from natural language processing models can be employed. For example, the bidirectional language model architecture BERT can be used. BERT uses a masking prediction mechanism during pre-training, hence it is often referred to as a masked language model. Its basic building block is the Transformer (encoder-decoder based on self-attention), where the multi-head self-attention mechanism enables the model to obtain context-sensitive representations. Figure 2 This is a schematic diagram of the model. The calculation formulas for the Transformer module are described below:

[0127] This formula describes scaled dot product attention, where Q, K, and V are the query, key, and value matrices, respectively. The input consists of a d-dimensional matrix. k The query and key and the dimension d v The values ​​are composed of .

[0128] Among them W O It is a parameter matrix.

[0129] h norm =LayerNorm(V+SubLayer1(V)), this formula is the residual connection around the sublayer, followed by layer normalization. SubLayer1(·) is the multi-head self-attention function.

[0130] h ffn =max(0, h) norm W1+b1)W2+b2, where W1 and W2 are linear transformations and b2 is the bias. The linear transformations are the same at different locations, and they use different parameters between layers.

[0131] h out =LayerNorm(h norm +SubLayer2(h norm This formula represents another layer of normalization and residual connections. SubLayer2(·) is a position-feedforward network. After constructing the language model, the model is adapted to downstream tasks.

[0132] In one specific embodiment of the present invention, a natural language processing model is input into a natural language sentence, and the label is predicted using the answer space to obtain the label probability, including:

[0133] Step 1: Using a natural language processing model, map the labels in the natural language sentence to the answer words in the vocabulary of the pre-trained language model, and predict the probability of the answer words;

[0134] Step 2: Determine the probability of the answer word as the probability of the corresponding tag.

[0135] For example: Given the click history text Hu and the candidate text Tc, corresponding to a true label y∈{0,1}, this invention first determines whether the user clicked the candidate text (y=1) or (y=0). Therefore, this invention designs a Verbalizer v(·) to map the label to two answer words in the PLM vocabulary, as shown in the following formula:

[0136]

[0137] in, It is the answer word space, which can capture specific textual context features based on the prompt template. Therefore, the personalized text intelligent recommendation task is transformed into a cloze test task, and then the probability of the answer word is predicted by a pre-trained MLM.

[0138] P(y|H u T c ) = P M ([MASK]=v(y)|x prompt );

[0139] Among them, P M ([MASK]) = v(y = 1)|x prompt This can be viewed as the confidence level for recommending the current candidate text. This invention subsequently uses it as a ranking score to form the final recommendation list.

[0140] The embodiments of the present invention can also construct two extended answer space vocabularies to analyze more complex answer space features and thus capture more words.

[0141] Compared to pre-training and fine-tuning paradigms, the model proposed in this invention only requires optimizing a small number of PLM parameters to complete training. Therefore, the proposed model can be trained using loss functions such as the cross-entropy loss function. The cross-entropy loss function is as follows:

[0142]

[0143] Among them, y i and P i These are the true label and predicted probability of the i-th training instance, respectively. Furthermore, this invention uses an Adam optimizer (a first-order optimization algorithm) with L2 regularization (i.e., adding the sum of squares of all parameters across all layers to the original loss function) for model training.

[0144] S104. Based on the label probability, select the recommended text from multiple candidate texts.

[0145] As discussed above, the label probability indicates the probability that a user is interested in a text based on historical clicks and unsubscriptions. Therefore, recommended text can be selected from multiple candidate texts based on this label probability. For example, a threshold can be set, and candidate texts with a label probability greater than the threshold can be used as recommended texts. Alternatively, the label probabilities can be sorted, and the candidate text with the highest label probability can be selected as the recommended text.

[0146] In one specific embodiment of the present invention, a recommended text is selected from multiple candidate texts based on the label probability, including:

[0147] Step 1: Accumulate the label probabilities of candidate texts to obtain the ranking score of the candidate text;

[0148] Step 2: Select recommended texts from multiple candidate texts based on the ranking scores.

[0149] The ranking score of a candidate text is obtained by accumulating the label probabilities of the candidate texts, including: accumulating the label probabilities of the candidate texts output by similar prompt templates, or accumulating the label probabilities of the candidate texts output by different prompt templates.

[0150] Then, based on the ranking scores, the texts are sorted to select the recommended texts.

[0151] Specifically, based on the ranking score, a recommended text is selected from multiple candidate texts, including: ranking multiple candidate texts based on the ranking score, and determining the candidate text with the highest ranking score as the recommended text.

[0152] Because different prompt templates focus on different specific designs and utilize linguistic and semantic knowledge differently in PLM, they may have different advantages. Therefore, in this invention, multi-prompt integration can be employed to combine the advantages of various prompt templates to facilitate the final decision. Thus, this invention sums the probabilities of affirmative answer words in each prompt as the final ranking score.

[0153] Among them, P e It is template e to w pos The output probability is ε, where ε is the fusion template set. This invention provides two types of cue template fusion: one fuses predictions from templates of the same type, and the other fuses predictions from templates of different types, called cross-type fusion. Finally, based on the results of the above fusion, the model generates the final prediction.

[0154] Once the recommendation text is obtained, it can be sent to the user's bound client.

[0155] Applying the method provided in the embodiments of the present invention,

[0156] Obtain historical text information of visited texts and candidate text information of candidate texts; input historical text information and candidate text information into a prompt template to obtain answer words; select tags from answer words and convert historical text information, candidate text information and tags into natural language sentences; input natural language sentences into a natural language processing model, use the answer space to predict tags, and obtain tag probabilities; based on tag probabilities, select recommended texts from multiple candidate texts.

[0157] In this invention, after acquiring historical text information and candidate text information, the historical text information and candidate text information can be input into a prompt template to obtain answer words. Then, tags are selected from the answer words, and the historical text information, candidate text information, and tags are converted into natural language sentences. These natural language sentences are then input into a natural language processing model to predict tags using the answer space and obtain tag probabilities. Finally, based on the tag probabilities, recommended text is selected from multiple candidate texts.

[0158] As can be seen, in this invention, historical text information and candidate text information can be converted into labeled natural sentences through the prompting model, which can make full use of the language information of the real-world large-scale corpus, and transform the personalized text intelligent recommendation task into a mask prediction task based on multi-template prompting learning, thereby recommending content that users are interested in.

[0159] To facilitate a better understanding of the recommended methods provided in the embodiments of the present invention by those skilled in the art, the recommended methods will be described in detail below in conjunction with related technologies.

[0160] With the explosive growth of internet data, various internet applications such as information flow recommendation and intelligent customer service dialogue recommendation have emerged. Users' needs for acquiring knowledge, news, gossip, current events, etc., have become more diversified, personalized, and frequent. Information flow recommendation applications, both domestically and internationally, have attracted a large user base and achieved significant commercial value, all relying on the support of excellent personalized text recommendation systems. Today, user experience standards are increasingly high, and competition among apps is intensifying, making the development of high-quality user profile knowledge graphs and intelligent recommendation algorithms even more challenging. Furthermore, the modeling of user profile knowledge graphs is itself a heavyweight engineering module, posing considerable challenges to system development investment. Personalized text intelligent recommendation systems, as an assistant for knowledge transfer, an infrastructure for generating user stickiness, and an increase in user click-through rates, can not only serve as a core backend technology module for large internet companies, but also be integrated with search engines or operate as a standalone system to effectively help users find the descriptions they are most interested in from a large number of documents, thereby alleviating information overload and broadening their horizons.

[0161] Most neural network recommendation methods focus on designing various clever neural networks to encode representations of text and users, with core modules being text encoders, user encoders, and similarity measures. For example, multi-head self-attention networks are used as both text and user encoders, or stacked dilated convolutional networks are used to learn a hierarchical, multi-level text representation for fine-grained matching. In these neural network models, static word embedding models are mostly used to initialize the training model and then mine intra-domain information from the text dataset, but this often ignores the rich semantics of the text and the linguistic information from large-scale real-world corpora.

[0162] Specifically, there are three main recommendation methods: Neural Network-based methods and Ensemble Learning methods. Neural Network-based methods extract content features through multi-layered nonlinear transformations and then use functions such as Softmax (normalized exponential function) to map the text content to a feature space to achieve recommendation results. Traditional rule-based recommendation methods use the simplest text content classification methods, manually setting a series of rules for content classification and recommendation, such as predicting results based on keyword frequency and text length. Ensemble learning-based recommendation methods integrate multiple content classifiers into a unified joint framework for classification, and then use voting mechanisms or weighted averages to achieve better predictive performance. While these methods can fully utilize the differences between different classifiers, they require designing specific ensemble structures, a time-consuming and labor-intensive process.

[0163] The related technologies have several shortcomings: First, text feature extraction is difficult. Previous recommendation systems required extracting text content features, but in practical applications, improving the accuracy and quality of feature extraction is challenging when dealing with large-scale text. Second, the model has weak generalization ability. Neural network-based methods are prone to overfitting, performing well on the training set but poorly on the test set. This is because the recommendation algorithm only considers text content features and ignores users' historical behavior. Third, the cold start problem cannot be solved. When new users or new text are added, accurate recommendation results cannot be provided because the model can only analyze the relevant attributes of existing users or text, and cannot obtain information about new users or new text.

[0164] Furthermore, pre-trained language models (PLMs) are introduced to learn text representations, primarily utilizing a pre-training and fine-tuning process to adapt for downstream text recommendation tasks. In this process, the PLM acts only as a text encoder, while another neural network design module encodes the user, and a text recommendation-specific objective function is used to train the entire model. Although these methods show promising performance improvements, they do not fully leverage the rich semantic knowledge within large-scale PLMs due to the inconsistency between the downstream text recommendation objective and the PLM training objective.

[0165] A novel pre-training-hint-prediction paradigm based on cue learning has achieved significant success in many intelligent applications. This new paradigm is based on re-representing downstream tasks as PLM training tasks by designing task-related cue templates and answer word spaces. Due to its enormous predictive potential, this invention transforms text recommendation tasks into cloze-style mask prediction tasks, and then uses the cue learning paradigm to improve the prediction accuracy of personalized intelligent text recommendations.

[0166] Specifically, given a user's click history Hu = {C1, D2, ..., CL} and candidate text Tc, this invention first converts Hu into a sentence, represented as: <user>In order to encode user interests from history; then Tc is converted into a sentence, represented as <candidate>Candidate text; secondly, this invention designs a prompt template T( <user> ， <candidate>The two sentences are concatenated into another sentence with the [MASK] tag. After passing through a natural language processing model (Masked Language Model, MLM), an answer space corresponding to the template is designed to predict the [MASK]. Finally, the probability of the predicted token is converted into a ranking score to determine whether the candidate text should be recommended to the user.

[0167] Therefore, to address the above problems, this invention first analyzes how cue learning affects recommendation performance, including:

[0168] 1. What kind of template is more suitable for integrating text data and user behavior in the field of text recommendation?

[0169] 2. How to map recommended tags to the predicted answer words of [MASK].

[0170] 3. Can the advantages of different templates be integrated to improve prediction performance?

[0171] To address the aforementioned problems, this invention provides three types of templates, including discrete templates, continuous templates, and hybrid templates, for integration. <user>and <candidate>Content tagging; Based on the above template, this invention explores the similarity evaluation problem of personalized text recommendation tasks from four aspects, including semantic relevance, user sentiment, user behavior, and recommendation effect; then, a binary answer space is constructed according to the template, containing two answer words with opposite meanings, corresponding to the true tags of whether the user clicked on the candidate text; finally, multi-cue ensemble is used to perform decision fusion on predictions from different templates. In addition, extensive experimental results verify that the method proposed in this invention has better recommendation performance than state-of-the-art baseline models.

[0172] Specifically, the technical solutions provided in the embodiments of the present invention have the following technical effects:

[0173] 1. This invention proposes to transform the task of personalized text intelligent recommendation into a mask prediction task based on cue learning, thereby recommending content that users are interested in.

[0174] 2. This invention designs a series of prompt templates, including discrete templates, continuous templates and mixed templates, and constructs a corresponding answer feature space to calculate recommendation scores and recommendations.

[0175] 3. This invention evaluates text recommendation tasks from four aspects: semantic relevance, user sentiment, user behavior, and recommendation effect, and uses multi-cue integration to perform decision fusion on predictions from different templates.

[0176] 4. This invention constructs a personalized content recommendation system based on prompt learning, which makes personalized recommendations based on the user's browsing history and click behavior.

[0177] 5. The personalized intelligent recommendation method of the present invention can be applied to the recommendation algorithms of various information flow recommendation platforms. It is not limited to specific tasks, but can also be inserted into the recommendation process of short video platforms.

[0178] 6. The solution proposed in this invention integrates three forms of prompting and learning from a novel perspective, solving the problems of user profile knowledge graph construction, cold start of recommendation system, and performance improvement of user behavior modeling in recommendation.

[0179] 7. The solution proposed in this invention is loosely coupled, can quickly adapt to and access new business data, and can be trained based on historical user click logs without the need for manual data labeling.

[0180] 8. The solution of the present invention can be applied to dialogue systems, marketing script recommendation systems, and other scenarios involving the need for dialogue robots in vertical fields.

[0181] 9. The multi-template prompt learning personalized recommendation training and application system of the present invention can be applied to optimize the data center's support requirements for recommendation systems and optimize the recommendation effects of content recommendation, product recommendation, intelligent customer service, etc.

[0182] 10. This invention supports general CPU and GPU solutions when in use, without the need to adjust the server hardware in the data center, which greatly reduces the cost of model training and the cycle of new business development and maintenance.

[0183] Corresponding to the above recommended method embodiments, this embodiment of the invention also provides a model training method, which can train the natural language processing model required by the above recommended method. The training process is the same as the above usage process. The following explanation focuses only on the specificity of the training process. For parts similar to or the same as those in the usage process, please refer to the above description. They will not be listed one by one below.

[0184] Specifically, model training methods include:

[0185] Step 1: Obtain historical text information of visited texts and candidate text information of candidate texts, as well as the real tags of whether candidate texts were visited after being recommended.

[0186] In other words, compared to the recommendation method, this method requires additional information on whether the candidate text was accessed after being recommended. Specifically, this actual label can be represented by a label y∈{0,1}, for example, y=1 indicates that the candidate text was accessed by the user after being recommended, and y=0 indicates that the candidate text was not accessed by the user after being recommended.

[0187] Step 2: Based on the prompt template, convert historical text information and candidate text information into tagged natural sentences.

[0188] The natural language processing model is fed into the natural language processing model to be trained, and the label is predicted using the answer space to obtain the label probability.

[0189] Step 3: Calculate the loss value of the current natural language processing model based on the labeled probability and the true label, and fine-tune the current natural language processing model based on the loss value until training is complete, so that the trained natural language processing model can be used as recommended above.

[0190] As we can see from the above, the higher the label probability, the more likely it is to be selected as the recommended text and recommended to the user. The real label indicates whether the candidate text is accessed after it is recommended to the user. Therefore, based on the label probability and the real label, the loss value of the current natural language processing model can be calculated.

[0191] Compared to pre-training and fine-tuning paradigms, the model proposed in this invention only requires optimizing a small number of PLM parameters to complete training. Therefore, this invention can use the cross-entropy loss function to train the natural language processing model. The loss function calculation formula is as follows: Among them, y i and P i These are the true label and predicted probability of the i-th training instance, respectively.

[0192] Then, based on the current loss value, the parameters in the natural language processing model are tuned until training is complete. During the optimization process, the model can be trained using an Adam optimizer with L2 regularization.

[0193] Thus, a usable natural language processing model can be obtained, specifically one that can be used with the recommended methods described above.

[0194] Corresponding to the above method embodiments, this invention also provides a recommendation device, which can be referred to in correspondence with the recommendation method described above.

[0195] See Figure 3 As shown, the device includes the following modules:

[0196] The information acquisition module 101 is used to acquire historical text information of accessed text and candidate text information of candidate text;

[0197] The prompt conversion module 102 is used to input historical text information and candidate text information into the prompt template to obtain the answer words; select the marker from the answer words, and convert the historical text information, candidate text information and marker into natural language;

[0198] The label prediction module 103 is used to input natural sentences into the natural language processing model, predict labels using the answer space, and obtain label probabilities;

[0199] The recommended selection module 104 is used to select recommended text from multiple candidate texts based on the label probability.

[0200] Using the apparatus provided in this embodiment of the invention, historical text information of visited text and candidate text information of candidate text are obtained; the historical text information and candidate text information are input into a prompt template to obtain answer words; a tag is selected from the answer words, and the historical text information, candidate text information and tag are converted into natural language sentences; the natural language sentences are input into a natural language processing model, and the tag is predicted using the answer space to obtain the tag probability; based on the tag probability, a recommended text is selected from multiple candidate texts.

[0201] In this invention, after acquiring historical text information and candidate text information, the historical text information and candidate text information can be input into a prompt template to obtain answer words. Then, tags are selected from the answer words, and the historical text information, candidate text information, and tags are converted into natural language sentences. These natural language sentences are then input into a natural language processing model to predict tags using the answer space and obtain tag probabilities. Finally, based on the tag probabilities, recommended text is selected from multiple candidate texts.

[0202] As can be seen, in this invention, historical text information and candidate text information can be converted into labeled natural sentences through the prompting model, which can make full use of the language information of the real-world large-scale corpus, and transform the personalized text intelligent recommendation task into a mask prediction task based on multi-template prompting learning, thereby recommending content that users are interested in.

[0203] In one specific embodiment of the present invention, the information acquisition module is specifically used to read the historical titles of each accessed text.

[0204] Connect the various historical titles to obtain historical text information;

[0205] Read the candidate titles from the candidate text and identify them as candidate text information.

[0206] In one specific embodiment of the present invention, the information acquisition module is specifically used to connect the word sequences of each historical title using delimiters to obtain historical text information.

[0207] In one specific embodiment of the present invention, the prompt conversion module is specifically used to input historical text information and candidate text information into a discrete prompt template for cloze test to obtain the answer words.

[0208] In one specific embodiment of the present invention, the prompt conversion module is specifically used to input historical text information and candidate text information into multiple discrete prompt models to perform cloze tests and obtain multiple answer words.

[0209] In one specific embodiment of the present invention, the prompt conversion module is specifically used to input historical text information and candidate text information into a semantically relevant prompt model to predict semantic relevance and obtain semantically relevant answer words.

[0210] In one specific embodiment of the present invention, the prompt conversion module is specifically used to input historical text information and candidate text information into the sentiment prompt model to predict user sentiment and obtain user sentiment-related answer words.

[0211] In one specific embodiment of the present invention, the prompt conversion module is specifically used to input historical text information and candidate text information into the user behavior prompt model to predict user behavior and obtain answer words related to user behavior.

[0212] In one specific embodiment of the present invention, the prompt conversion module is specifically used to input historical text information and candidate text information into the recommendation effect prompt model to predict the recommendation effect and obtain answer words related to the recommendation effect.

[0213] In one specific embodiment of the present invention, the prompt conversion module is specifically used to input historical text information and candidate text information into a continuous prompt template with random virtual words to obtain the answer word.

[0214] In one specific embodiment of the present invention, the prompt conversion module is specifically used to input historical text information and candidate text information into a mixed prompt template with random virtual words to obtain the answer word.

[0215] In one specific embodiment of the present invention, the prompt conversion module is specifically used to identify affirmative answer words in the answer words as markers.

[0216] In one specific embodiment of the present invention, the label prediction module is specifically used to map the labels in the natural language statement to the answer words in the pre-trained language model vocabulary using a natural language processing model, and predict the probability of the answer words.

[0217] The probability of the answer word is determined as the probability of the corresponding tag.

[0218] In one specific embodiment of the present invention, it is recommended to select a module specifically used to accumulate the label probability of candidate texts to obtain the ranking score of the candidate texts;

[0219] Recommended texts are selected from multiple candidate texts based on ranking scores.

[0220] In one specific embodiment of the present invention, it is recommended to select a module, specifically used to accumulate the labeling probabilities of the output of similar prompt templates for candidate text, to obtain a ranking score.

[0221] In one specific embodiment of the present invention, it is recommended to select a module specifically for accumulating the label probabilities of different types of prompt template outputs of candidate text to obtain a ranking score.

[0222] In one specific embodiment of the present invention, a recommendation selection module is specifically used to sort multiple candidate texts based on ranking scores, and to determine the candidate text with the highest ranking score as the recommended text.

[0223] In one specific embodiment of the present invention, it further includes:

[0224] The recommendation module is used to send recommendation text to the client.

[0225] Corresponding to the above method embodiments, this invention also provides an electronic device. The electronic device described below can be referred to in conjunction with the recommendation method and model training method described above.

[0226] See Figure 4 As shown, the electronic device includes:

[0227] Memory 332 is used to store computer programs;

[0228] The processor 322 is configured to implement the steps of the recommended method of the above method embodiments when executing a computer program, or to implement the steps of the model training method of the above method embodiments when executing a computer program.

[0229] For details, please refer to Figure 5 , Figure 5 This is a schematic diagram of a specific structure of an electronic device provided in this embodiment. The electronic device can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) 322 (e.g., one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. The memory 332 can be temporary or permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332 and execute the series of instruction operations stored in the memory 332 on the electronic device 301.

[0230] Electronic device 301 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0231] The steps in the recommendation method or model training method described above can be implemented by the structure of an electronic device.

[0232] Corresponding to the above method embodiments, this invention also provides a readable storage medium. The readable storage medium described below can be referred to in conjunction with the recommendation method and model recommendation method described above.

[0233] A readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the recommended method of the above method embodiments, or when the computer program is executed by a processor, it implements the steps of the model training method of the above method embodiments.

[0234] Specifically, the readable storage medium can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other readable storage medium capable of storing program code.

[0235] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0236] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the invention.

[0237] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0238] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "include," "contain," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0239] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.< / candidate> < / user> < / candidate> < / user> < / candidate> < / user> < / candidate> < / user> < / candidate> < / user> < / user> < / candidate> < / user> < / candidate> < / candidate> < / user> < / candidate> < / user> < / candidate> < / user> < / candidate> < / user> < / candidate> < / user> < / candidate> < / user>

Claims

1. A recommendation method, characterized in that, include: Obtain historical text information of visited texts and candidate text information of candidate texts; wherein, the historical text information is the text information of texts that the user has clicked / visited; and the candidate text information is the text information of candidate texts. Input the historical text information and the candidate text information into the prompt template to obtain the answer words; Select tags from the answer words, and convert the historical text information, the candidate text information, and the tags into natural language sentences; The natural language statement is input into a natural language processing model, and the label is predicted using the answer space to obtain the label probability. Based on the labeled probabilities, a recommended text is selected from the plurality of candidate texts; Specifically, the natural language statement is input into a natural language processing model, and the label is predicted using the answer space to obtain the label probability, including: Using the natural language processing model, the labels in the natural sentences are mapped to answer words in the pre-trained language model vocabulary, and the probability of the answer words is predicted; The probability of the answer word is determined as the tag probability of the corresponding tag.

2. The recommended method according to claim 1, characterized in that, Retrieve historical text information of visited texts and candidate text information of candidate texts, including: Read the historical titles of each of the accessed texts; By connecting the various historical titles, the historical text information is obtained; Read the candidate titles of the candidate text and determine the candidate titles as the candidate text information.

3. The recommended method according to claim 2, characterized in that, By concatenating the various historical titles, the historical text information is obtained, including: The historical text information is obtained by connecting the word sequences of each historical title using delimiters.

4. The recommended method according to claim 1, characterized in that, Input the historical text information and the candidate text information into the prompt template to obtain the answer words, including: The historical text information and the candidate text information are input into the discrete prompt template to perform a cloze test and obtain the answer words.

5. The recommended method according to claim 4, characterized in that, The historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including: The historical text information and the candidate text information are input into various discrete prompt templates to perform a cloze test, resulting in multiple answer words.

6. The recommended method according to claim 4, characterized in that, The historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including: The historical text information and the candidate text information are input into the semantic relevance prompting model to predict semantic relevance and obtain semantically relevant answer words.

7. The recommended method according to claim 4, characterized in that, The historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including: The historical text information and the candidate text information are input into the sentiment prompting model to predict user sentiment and obtain user sentiment-related response words.

8. The recommended method according to claim 4, characterized in that, The historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including: The historical text information and the candidate text information are input into the user behavior prompting model to predict user behavior and obtain user behavior-related answer words.

9. The recommended method according to claim 4, characterized in that, The historical text information and the candidate text information are input into a discrete prompt template for cloze test to obtain the answer words, including: The historical text information and the candidate text information are input into the recommendation effect prompting model to predict the recommendation effect and obtain the answer words related to the recommendation effect.

10. The recommendation method according to claim 1, characterized in that, Inputting the historical text information and the candidate text information into the prompt template yields the answer words, including: The historical text information and the candidate text information are input into a continuous prompt template with random virtual words to obtain the answer word.

11. The recommendation method according to claim 1, characterized in that, Inputting the historical text information and the candidate text information into the prompt template yields the answer words, including: The historical text information and the candidate text information are input into a mixed prompt template with random virtual words to obtain the answer word.

12. The recommendation method according to claim 1, characterized in that, Select markers from the response words, including: The affirmative answer words in the answer words are identified as the markers.

13. The recommended method according to any one of claims 1 to 12, characterized in that, Based on the labeled probabilities, recommended texts are selected from multiple candidate texts, including: The ranking score of a candidate text is obtained by summing the tag probabilities of the candidate texts. Based on the ranking score, the recommended text is selected from the multiple candidate texts.

14. The recommended method according to claim 13, characterized in that, The ranking score of the candidate text is obtained by summing the tag probabilities of the candidate texts, including: The ranking score is obtained by summing the tag probabilities of the candidate text similar prompt template output.

15. The recommended method according to claim 13, characterized in that, The ranking score of the candidate text is obtained by summing the tag probabilities of the candidate texts, including: The ranking score is obtained by summing the label probabilities of the different types of prompt templates output by the candidate text.

16. The recommended method according to claim 13, characterized in that, Based on the ranking score, the recommended text is selected from the plurality of candidate texts, including: The candidate texts are sorted based on the ranking score, and the candidate text with the highest ranking score is determined as the recommended text.

17. The recommendation method according to claim 1, characterized in that, Also includes: The recommended text is sent to the client.

18. A model training method, characterized in that, include: Obtain historical text information of visited texts and candidate text information of candidate texts, as well as the real tags of whether the candidate texts were visited after being recommended; Based on the prompt template, the historical text information and the candidate text information are converted into tagged natural sentences; The natural language statement is input into the natural language processing model to be trained, and the label is predicted using the answer space to obtain the label probability. Based on the labeled probability and the true label, calculate the loss value of the current natural language processing model, and fine-tune the current natural language processing model based on the loss value until training is completed, so that the trained natural language processing model can be used in the recommendation method as described in any one of claims 1 to 17.

19. A recommendation device, characterized in that, include: The information acquisition module is used to acquire historical text information of accessed text and candidate text information of candidate text; wherein, the historical text information is the text information of text that the user has clicked / accessed; and the candidate text information is the text information of candidate text. The prompt conversion module is used to input the historical text information and the candidate text information into the prompt template to obtain the answer words; select a marker from the answer words, and convert the historical text information, the candidate text information, and the marker into natural language sentences; The label prediction module is used to input the natural sentence into the natural language processing model, predict the label using the answer space, and obtain the label probability; The recommended selection module is used to select recommended text from multiple candidate texts based on the labeled probabilities; Specifically, the label prediction module is used to map the labels in the natural language statement to answer words in the pre-trained language model vocabulary using the natural language processing model, and predict the probability of the answer words; the probability of the answer words is determined as the label probability of the corresponding label.

20. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the recommended method as claimed in any one of claims 1 to 17 when executing the computer program, or to implement the steps of the model training method as claimed in claim 18 when executing the computer program.

21. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the steps of the recommended method as described in any one of claims 1 to 17, or, when executed by a processor, implements the steps of the model training method as described in claim 18.

Images