Prompt word generation method of large language model and question answering method of large language model

Abstract

The invention relates to a cue word generation method of a large language model and a question answering method of the large language model, and relates to the technical field of natural languages. The cue word generation method of the large language model comprises the following steps: obtaining an initial cue word, and carrying out semantic classification processing on the initial cue word to obtain classification information of the initial cue word; judging that the dialogue intention of the initial cue word is a preset dialogue intention needing cue word generation; obtaining a cue word template bound with the dialogue intention needing to generate the cue word in advance; initial entity words in the initial cue words are extracted, and target entity words are determined in the initial entity words according to a first entity word type in the cue word template; and replacing the vacant cue word in the cue word template with the target entity word, and taking the replaced cue word template as the target cue word. According to the method and the device, knowledge enhancement can be performed on the initial cue word related to the preset dialogue intention needing to generate the cue word, so that the use of computing resources is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a prompt word generation method for a large language model and a question-answering method for a large language model. Background Art

[0002] Using a large model for conversation is the latest approach in the field of conversation, but this approach works poorly when the pre-training data lacks domain knowledge (perhaps because the target domain is relatively niche or lacks real-time knowledge). Fine-tuning is a common technique in natural language processing (NLP) and deep learning. It allows further training on a specific task or dataset based on a pre-trained large model to improve the model's performance on the task. Existing solutions often use fine-tuning methods to fine-tune large models to enhance the effect, but large models are often Few-Shot Learning, so the knowledge that can be enhanced is limited. Summary of the invention

[0003] The prompt word generation method of a large language model and the question-answering method of a large language model provided in the embodiments of the present application can at least improve the knowledge enhancement effect of the prompt words of the large language model.

[0004] According to one aspect of the present application, a method for generating prompt words for a large language model is provided, comprising: obtaining an initial prompt word, performing semantic classification processing on the initial prompt word, and obtaining classification information of the initial prompt word; wherein the classification information at least includes the conversation intention of the initial prompt word; determining that the conversation intention of the initial prompt word is a preset conversation intention for which a prompt word needs to be generated; obtaining a prompt word template that is pre-bound to the conversation intention for which a prompt word needs to be generated; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word; extracting the initial entity word in the initial prompt word, and determining a target entity word in the initial entity word according to the first entity word type in the prompt word template; replacing the vacancy prompt word in the prompt word template with the target entity word, and using the prompt word template that has completed the replacement as the target prompt word.

[0005] According to another aspect of the present application, a question-answering method for a large language model is also provided, including: obtaining a target prompt word; wherein the target prompt word is generated according to the above method; and inputting the target prompt word into a pre-trained large language model to obtain a question-answering result.

[0006] According to another aspect of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor implements the above method when executing the computer program.

[0007] According to another aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program implements the above method when executed by a processor.

[0008] The method for generating prompt words of a large language model provided by the embodiment of the present application first obtains an initial prompt word, performs semantic classification processing on the initial prompt word, and obtains classification information of the initial prompt word; wherein the classification information at least includes the conversation intention of the initial prompt word, and then determines that the conversation intention of the initial prompt word is a preset conversation intention that requires prompt words to be generated, and obtains a prompt word template that is pre-bound to the conversation intention that requires prompt words to be generated; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word; extracts the initial entity word in the initial prompt word, according to the first entity word type in the prompt word template, and finally, determines the target entity word in the initial entity word, replaces the vacancy prompt word in the prompt word template with the target entity word, and uses the prompt word template that completes the replacement as the target prompt word. The present application can perform knowledge enhancement on the initial prompt word related to the preset conversation intention that requires prompt words to be generated, and the knowledge enhancement process does not require a large number of samples, reducing the use of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0010] Figure 1 is a flow chart of a method for generating prompt words of a large language model according to an embodiment of the present application;

[0011] Figure 2 It is a flowchart of the prompt word construction of the existing scheme;

[0012] Figure 3 It is a flowchart of the prompt word construction of this application;

[0013] Figure 4 is a flow chart of the question-answering method of the large language model of the present application;

[0014] Figure 5 It is a schematic diagram of the structure of the electronic device of the present application. DETAILED DESCRIPTION

[0015] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0016] The present application provides a method for generating prompt words for a large language model and a method for question-answering of a large language model. The method can integrate the large model and the classification method of traditional deep learning, can perform knowledge enhancement on the prompt words, enrich the information included in the prompt words, and then obtain higher-quality question-answering results based on the optimized prompt words. The present application can alleviate the problem of poor results due to less pre-training data in some fields, solve the problem of needing to finetune a large number of samples, avoid the need for large computing resources, and may also cause the model to overfit, and solve the problem of lack of real-time performance of the basic large model. Since the NLU model has a low training cost, it can achieve faster iteration for the latest data, thereby alleviating the real-time performance problem of the large model.

[0017] In an embodiment of the present application, a method for generating prompt words of a large language model is provided, wherein the large language model

[0018] Large Language Model (LLM) refers to natural language processing (NLP) models with huge parameters and huge amounts of training data. These models are usually built using deep learning technology and trained on large datasets in order to grasp the complexity and nuances of language so that they can perform various natural language processing tasks. Large language models can be GPT (Generative Pre-trained Transformer) series, such as GPT-3, GPT-4 and other models, BERT (Bidirectional Encoder Representations from Transformers) series models, ERNIE (a language model optimized for Chinese and other Asian languages) models, or T5 (a unified text-to-text pre-trained model that can perform a variety of NLP tasks) models, etc.

[0019] Figure 1is a flowchart of a method for generating prompt words of a large language model according to an embodiment of the present application. Figure 1 The method steps involved are described.

[0020] Step S101, obtaining an initial prompt word, performing semantic classification processing on the initial prompt word, and obtaining classification information of the initial prompt word; wherein the classification information at least includes the dialogue intention of the initial prompt word.

[0021] In the embodiment of the present application, the initial prompt words are question and answer information provided by the user. The initial prompt words can be data in text form, or data in voice form, picture form or video form. For initial prompt words in non-text form, they can be converted into text form before processing.

[0022] Semantic classification is a natural language processing technology that involves understanding and classifying the meaning of text or sentences. In an embodiment of the present application, the initial prompt word is subjected to semantic classification processing, which refers to assigning the text to different categories according to its meaning or content. This classification can be based on many different criteria, such as but not limited to: theme, emotion, intention, etc. Among them, the classification results based on the theme include but are not limited to: technology, sports, entertainment, and finance, etc. The classification results based on emotion include but are not limited to: positive, negative, and neutral. The classification results based on intention include but are not limited to: query information (users seek specific information or answers), reservation services (users want to make appointments for services or purchase goods), technical support (users encounter technical problems and seek help), account management (users want to change account settings or passwords), complaints and suggestions (users express dissatisfaction or make suggestions for improvement) and navigation guidance (users need to obtain route or location information).

[0023] In an embodiment of the present application, the classification information obtained through semantic classification processing includes at least the conversation intention of the initial prompt word. The conversation intention is used to determine the intention or purpose of the user providing the initial prompt word. For example, in a customer service chat, identify whether the user is seeking technical support, making complaints, or asking about prices, etc. When implementing this step specifically, the conversation intention may include domain information and question and answer content information, wherein the domain information includes but is not limited to: music field, film field, literature field, and navigation field. For example, the conversation intention is: asking for the name of the singer in the music field, wherein the domain information is the music field, and the question and answer content information is asking for the name of the singer.

[0024] Step S102, determining whether the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of prompt words.

[0025] In the embodiment of the present application, the conversation intention for which prompt words need to be generated is a conversation intention determined in advance after testing. Based on the conversation intention for which prompt words need to be generated, the question-and-answer results obtained are usually not good enough and cannot obtain the information required by the user. For example, the initial prompt word is "Please introduce Ningxia", and the large model provides relevant information about Ningxia based on the initial prompt word, while the information required by the user is information about the song Ningxia.

[0026] In the embodiment of the present application, by comparing the conversation intention of the initial prompt word with the preset conversation intention that needs to generate prompt words, it can be determined whether the conversation intention of the initial prompt word is a conversation intention that needs to generate prompt words. For example, the conversation intention of the initial prompt word "Please introduce Ningxia" is "Introduction to songs in the field of music". If the preset conversation intention that needs to generate prompt words includes "Introduction to songs in the field of music", then it can be determined that the conversation intention of the initial prompt word is a preset conversation intention that needs to generate prompt words.

[0027] In the embodiment of the present application, the preset conversation intentions for which prompt words need to be generated may be one or more. By setting the preset conversation intentions for which prompt words need to be generated for the field to which the question and answer results that the user is dissatisfied belong, the prompt words for the field are enhanced.

[0028] For example, the conversation intention of the initial prompt word is: asking about the name of the singer of a song in the field of music. If the preset conversation intention that requires the generation of prompt words also includes asking about the name of the singer of a song in the field of music, then it can be determined that the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of prompt words.

[0029] Step S103, obtaining a prompt word template pre-bound to the dialogue intention for which a prompt word needs to be generated; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word.

[0030] In an embodiment of the present application, a dialog intent for which prompt words need to be generated is bound to a prompt word template, and the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word. Among them, the prompt word sentence pattern refers to a specific text pattern or structure designed to guide the model to generate a specific type of output. The vacancy prompt word is the information that needs to be replaced in the prompt word template. The first entity word type can be set in combination with the field and experience to which the prompt word sentence pattern belongs. The first entity word type is used to characterize the meaning of the vacancy entity word in the prompt word template, for example, including but not limited to: song title, singer name, place name, person name, organization name, date, currency value, etc.

[0031] For example, in a possible implementation, the prompt word sentence is: "If "XXX" is a song, then who is the singer? If you don't know, you can answer "I don't know", where "XXX" is the vacancy prompt word. The first entity word type is: song title.

[0032] In this step, the information of the prompt words can be enriched based on the prompt word template that is pre-bound to the dialogue intention for which the prompt words need to be generated.

[0033] Step S104: extracting initial entity words from the initial prompt words, and determining target entity words from the initial entity words according to the first entity word type in the prompt word template.

[0034] In the embodiment of the present application, the initial entity words in the initial prompt words can be extracted by a named entity recognition (NER) operation, which can identify and classify specific types of entity words from the text.

[0035] Content words refer to words that carry specific meaning in a sentence. These words usually include nouns, verbs, adjectives, and adverbs, which constitute the core content of a sentence and provide the main information of the sentence. Content words are in contrast to function words, which include prepositions, articles, conjunctions, etc., and are mainly used to construct sentence structures without directly conveying specific meanings. For example, content words can be names of people, etc.

[0036] According to the first entity word type in the prompt word template, the target entity word is determined in the initial entity word, that is, the initial entity word that meets the first entity word type is used as the target entity word. For example, the first entity word type is: song title, and the target entity word includes entity words of the song title and singer name, then the entity word of the song title type can be used as the target entity word.

[0037] Step S105: using the target entity word to replace the missing prompt word in the prompt word template, and using the prompt word template that has completed the replacement as the target prompt word.

[0038] In an embodiment of the present application, the target entity word is used to replace the missing prompt word in the prompt word template, so that the entity word that matches the missing prompt word after screening in the initial prompt word is replaced in the prompt word template, and the prompt word template that completes the replacement is used as the target prompt word, so that the initial prompt word is optimized to the target prompt word, the prompt word is enriched and enhanced, and the question and answer results are optimized.

[0039] For example, the initial prompt word is: Please introduce Ningxia. The initial entity words extracted based on the initial prompt word include: "Ningxia" and "introduction", among which "Ningxia" is the target entity word. The prompt word sentence pattern of the prompt word template is: "If "XXX" is a song, then who is the singer? If you don't know, you can answer that you don't know." The first entity word type of the prompt word template is: song title. By replacing the blank prompt word in the prompt word template with the target entity word, the target prompt word can be obtained: If "Ningxia" is a song, then who is the singer? If you don't know, you can answer that you don't know.

[0040] The method for generating prompt words of a large language model provided by the embodiment of the present application first obtains an initial prompt word, performs semantic classification processing on the initial prompt word, and obtains classification information of the initial prompt word; wherein the classification information at least includes the conversation intention of the initial prompt word, and then determines that the conversation intention of the initial prompt word is a preset conversation intention that requires prompt words to be generated, and obtains a prompt word template that is pre-bound to the conversation intention that requires prompt words to be generated; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word; extracts the initial entity word in the initial prompt word, according to the first entity word type in the prompt word template, and finally, determines the target entity word in the initial entity word, replaces the vacancy prompt word in the prompt word template with the target entity word, and uses the prompt word template that completes the replacement as the target prompt word. The present application can perform knowledge enhancement on the initial prompt word related to the preset conversation intention that requires prompt words to be generated, and the knowledge enhancement process does not require a large number of samples, reducing the use of computing resources.

[0041] In a possible implementation, semantic classification processing of the initial prompt word can be performed according to the following steps: inputting the initial prompt word into a pre-trained first natural language processing model; extracting the initial entity words in the initial prompt word can be performed according to the following steps: inputting the initial prompt word into a pre-trained second natural language processing model to obtain an entity word extraction result; wherein the entity word extraction result includes one or more initial entity words and a second entity word type corresponding to the initial entity word.

[0042] In this possible implementation, a first natural language processing model can be used to perform semantic classification processing on the initial prompt word. A second natural language processing model can be used to perform entity word extraction processing on the initial prompt word. The first natural language processing model can be a model such as BERT or GPT. The second natural language processing model can be a Transformer model, such as a pre-trained model such as BERT, or a deep learning model such as a convolutional neural network (CNN), a recurrent neural network (RNN), and a long short-term memory network (LSTM).

[0043] In this possible implementation, the entity word extraction result includes one or more initial entity words and a second entity word type corresponding to the initial entity word. The second entity word type is used to characterize the meaning of the initial entity word in the entity word extraction result, for example, including but not limited to: song title, singer name, place name, person name, organization name, date, currency value, etc.

[0044] In a possible implementation, according to the first entity word type in the prompt word template, the target entity word is determined in the initial entity words, which can be performed according to the following steps: determining the initial entity word to be processed in the one or more initial entity words; if the second entity word type of the initial entity word to be processed is the same as the first entity word type, then taking the initial entity word to be processed as the target entity word.

[0045] In this possible implementation, one or more initial entity words are processed one by one, and the initial entity words currently to be processed are used as the initial entity words to be processed. The second entity word type of the initial entity word to be processed is compared with the first entity word type required by the vacancy prompt word. If the two are the same, the initial entity word to be processed is used as the target entity word. For example, the second entity word type is the singer's name, and the first entity word type is the song title. The two are different. If the second entity word type and the first entity word type are both song titles, they are the same, and the initial entity word to be processed whose second entity word type is the song title is used as the target entity word.

[0046] In order to improve the richness of prompt word templates, in a possible implementation, multiple prompt word templates are pre-bound to the dialogue intention for which prompt words need to be generated; obtaining the prompt word templates pre-bound to the dialogue intention for which prompt words need to be generated can be performed according to the following steps: respectively calculating a first matching degree based on the number of missing prompt words and a second matching degree based on the entity word type of the missing prompt words between the initial prompt word and the multiple prompt word templates; obtaining a total matching degree based on the first matching degree and the second matching degree; and obtaining the prompt word template with the highest total matching degree among the multiple prompt word templates.

[0047] In this possible implementation, the entity word type of the initial entity word in the initial prompt word and the number of the initial prompt words can be determined based on the initial prompt word. The number of vacancy prompt words and the first entity word type required for each vacancy prompt word can be determined based on the prompt word template. By comparing the initial prompt word with the prompt word template, the first matching degree between the initial prompt word and the number of vacancy prompt words and the second matching degree between the initial prompt word and the entity word type of the vacancy prompt word can be determined.

[0048] For example, the initial prompt words are: Please play song B by singer A covered by singer C. The first prompt word template includes two missing prompt words, and the second prompt word template includes three missing prompt words. The second prompt word template has a higher first matching degree with the initial prompt words. The missing entity word types of the second prompt word template are singer name, cover singer name and song name, while the missing entity word types of the third prompt word template are singer name, author name and song name. The second prompt word template has a higher second matching degree with the initial prompt words. The first matching degree and the second matching degree can be expressed in the form of scores, respectively. Based on the sum of the first matching degree and the second matching degree or the weighted summation, the total matching degree between the initial prompt word and the multiple prompt word templates is obtained. Afterwards, the prompt word template with the highest total matching degree is obtained from the multiple prompt word templates to obtain a prompt word template with a higher degree of fit with the initial prompt word.

[0049] In a possible implementation, determining whether the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of prompt words can be performed according to the following steps: calculating the similarity between the conversation intention of the initial prompt word and the preset conversation intention that requires the generation of prompt words; if the similarity exceeds a set similarity threshold, determining that the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of prompt words.

[0050] In this possible implementation, the similarity between the conversation intention of the initial prompt word and the preset conversation intention of the prompt word that needs to be generated is calculated, including: generating a first word vector based on the conversation intention of the initial prompt word, the first word vector is used to characterize the conversation intention of the initial prompt word, generating a second word vector based on the preset conversation intention of the prompt word that needs to be generated, the second word vector is used to characterize the preset conversation intention of the prompt word that needs to be generated; calculating the cosine similarity between the first word vector and the second word vector, and using the cosine similarity as the similarity between the conversation intention of the initial prompt word and the preset conversation intention of the prompt word that needs to be generated.

[0051] It should be noted that the first word vector and the second word vector can be obtained by word embedding. Word embedding is a technique for mapping words to a continuous vector space. These vectors capture the semantic and grammatical relationships between words. The word embedding model learns the vector representation of words by training on a large-scale corpus.

[0052] In the embodiment of the present application, the similarity threshold can be set according to actual needs or experience, and the embodiment of the present application does not specifically limit this. If the similarity exceeds the set similarity threshold, it is determined that the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of prompt words. Among them, the conversation intention that requires the generation of prompt words is pre-set information.

[0053] In order to achieve knowledge enhancement for the field of concern or the conversation intention of concern, in a possible implementation, the method may further perform the following steps: obtaining a test prompt word corresponding to the target conversation intention, inputting the test prompt word into a pre-trained large language model, and obtaining a test result; calculating a vector distance between the test result and a set standard result; if the vector distance is greater than a preset vector distance threshold, taking the target conversation intention as a preset conversation intention that requires the generation of prompt words.

[0054] In this possible implementation, the dialogue intent of the currently processed test prompt word is used as the target dialogue intent. The test prompt word is input into a pre-trained large language model, which is a deep learning model with a large number of parameters. These models are usually pre-trained on large-scale text datasets and can handle a wide range of natural language processing tasks, including but not limited to the following types: BERT (Bidirectional Encoder Representations from Transformers) model, GPT (Generative Pre-trained Transformer) model, etc.

[0055] Calculating the vector distance between the test result and the set standard result includes calculating one or more of the following distances: cosine similarity, Euclidean distance, and Chebyshev distance. The vector distance threshold can be set according to actual needs or experience, and the embodiment of the present application does not specifically limit this.

[0056] It should be noted that the vectors used to represent the test results and the set standard results can be obtained by sentence embedding. Sentence embedding is a technology that converts the entire sentence into a vector representation of a fixed length. Unlike word embedding, sentence embedding not only considers the meaning of a single word, but also the order and context of the words in the sentence. The goal of sentence embedding is to capture the meaning of the entire sentence and is usually used for tasks such as text classification, sentiment analysis, and semantic search. Word Embedding and Sentence Embedding are very important concepts in the field of Natural Language Processing (NLP). They are both methods of converting text into numerical representations so that machine learning models can process and understand natural language data.

[0057] In this step, the dialogue intentions that require knowledge enhancement can be screened out through a preset vector distance threshold, that is, the preset dialogue intentions that require the generation of prompt words are obtained.

[0058] In a possible implementation, the natural language processing model includes at least one of the following models: a BERT model, a RoBERTa model, an ALBERT model, and a DistilBERT model; the first natural language processing model and the second natural language processing model may be the same model or different models.

[0059] In this possible implementation, BERT, RoBERTa (Robustly Optimized BERT Pretraining Approach), ALBERT (A Lite BERT), and DistilBERT (Distilled BERT) are several popular pre-trained language models based on the Transformer architecture. Although they all belong to the same class of models, they have their own unique features in design and training strategies.

[0060] Among them, the BERT model is based on the Transformer architecture and is the first widely successful bidirectional pre-trained model that can capture the complex relationships of words in context. RoBERTa, an optimization of BERT, uses a larger batch size and longer training time to improve model performance. ALBERT (ALite BERT), a simplified version of BERT, aims to reduce the number of parameters while maintaining similar performance. DistilBERT, compresses BERT through knowledge distillation technology to reduce computational costs.

[0061] See also Figure 3 The prompt word construction flow chart shown in the figure is used, and the implementation process of the method is described below in conjunction with a specific embodiment.

[0062] First, input the initial prompt word, use BERT to perform semantic analysis on the initial prompt word, obtain the classification result, and judge whether the prompt word needs to be reconstructed based on the classification result. If not, input the initial prompt word into the large language model to obtain the output result. If yes, use the output classification result to reconstruct the prompt word, and input the reconstructed prompt word into the large language model to obtain the output result.

[0063] Taking the dialogue in the music field as an example, the user inputs "Who sings Ningxia?"

[0064] First, use the traditional method to train an NLU model, which contains intent information in vertical domains such as music and navigation. For example, when the input is "Who sings Ningxia", the NLU model outputs the NER result "Ning_Song Xia_Song is_OWho_O sings_O_O", and the classification result is: Music_Ask_Singer (music field - ask the name of the singer of the song).

[0065] The second step is to determine whether it is necessary to construct a prompt (big model input prompt) with the help of NLU results. The basis for judgment is whether the NLU model hits the target field (assuming that the big model is far less effective in the target field than the NLU model after analysis in advance). For example, if the big model does not work well in music question-answering, the prompt can be: "If "Ningxia" is a song, then who is the singer? If you don't know, you can answer that you don't know." This design prevents NLU from making mistakes in recognition.

[0066] The third step is to input the prompt into the large model to get the final result.

[0067] The present application provides a method for generating prompt words for a large language model, which can be applied to domain knowledge enhancement of the large model. The method uses the traditional NLU (Natural Language Understanding) method to add useful information in the prompt construction process, thereby enhancing the final effect and greatly improving the effect of the large model in dialogue in the field under the condition of lack of knowledge in some fields.

[0068] In an embodiment of the present application, a question-answering method using a large language model is provided. Figure 4 is a flowchart of a question-answering method for a large language model according to an embodiment of the present application. Figure 4 The method steps involved are described.

[0069] Step S401, obtaining a target prompt word; wherein the target prompt word is generated according to the prompt word generation method of the large language model.

[0070] Step S402: input the target prompt word into a pre-trained large language model to obtain a question-answering result.

[0071] In the embodiment of the present application, the target prompt word is a prompt word generated according to the prompt word generation method of the large language model described above. The prompt word is input into the pre-trained large language model to obtain the question-answering result, which can improve the accuracy of the question-answering result.

[0072] Figure 2 The flowchart of the prompt word construction of the existing scheme is shown. Figure 2 "Input" refers to the data or information provided by the user in a computer program or system. A prompt is constructed based on the information input by the user. After that, the prompt is input into the LLM (large language model) to obtain the output result. In some fields, the output results obtained through this process are usually poor. For example, if there is no data for the new song "Ningxia" (Ningxia is the same as the name of a province) in the pre-training stage, and the user says "Introduce Ningxia", the question and answer result may be: This is the name of a province.

[0073] For the above example, based on the solution provided in this application, BERT will identify two categories: music and location, and then optimize the prompt words. Finally, based on the optimized prompt words, the question and answer results may also include music-related information, such as composition, singing, etc.

[0074] In an embodiment of the present application, the process of generating a target prompt word includes: first obtaining an initial prompt word, performing semantic classification processing on the initial prompt word, and obtaining classification information of the initial prompt word; wherein the classification information at least includes the conversation intention of the initial prompt word, then determining that the conversation intention of the initial prompt word is a preset conversation intention that requires prompt words to be generated, and obtaining a prompt word template that is pre-bound to the conversation intention that requires prompt words to be generated; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word; extracting the initial entity word in the initial prompt word, according to the first entity word type in the prompt word template, and finally, determining the target entity word in the initial entity word, using the target entity word to replace the vacancy prompt word in the prompt word template, and using the prompt word template that completes the replacement as the target prompt word. The present application can perform knowledge enhancement on the initial prompt word related to the preset conversation intention that requires prompt words to be generated, and the knowledge enhancement process does not require a large number of samples, reducing the use of computing resources.

[0075] In an embodiment of the present application, a device for generating prompt words of a large language model is provided, and the device includes:

[0076] An acquisition module is used to acquire an initial prompt word, perform semantic classification processing on the initial prompt word, and obtain classification information of the initial prompt word; wherein the classification information at least includes the dialogue intention of the initial prompt word; a determination module is used to determine that the dialogue intention of the initial prompt word is a preset dialogue intention that requires the generation of prompt words; a template module is used to acquire a prompt word template that is pre-bound to the dialogue intention that requires the generation of prompt words; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word; an extraction module is used to extract the initial entity word in the initial prompt word, and determine the target entity word in the initial entity word according to the first entity word type in the prompt word template; a replacement module is used to replace the vacancy prompt word in the prompt word template with the target entity word, and use the prompt word template that completes the replacement as the target prompt word.

[0077] In an embodiment of the present application, the device first obtains an initial prompt word, performs semantic classification processing on the initial prompt word, and obtains classification information of the initial prompt word; wherein the classification information at least includes the conversation intention of the initial prompt word, and then determines that the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of prompt words, and obtains a prompt word template that is pre-bound to the conversation intention that requires the generation of prompt words; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word; extracts the initial entity word in the initial prompt word, according to the first entity word type in the prompt word template, and finally, determines the target entity word in the initial entity word, and uses the target entity word to replace the vacancy prompt word in the prompt word template, and uses the prompt word template that completes the replacement as the target prompt word. The present application can perform knowledge enhancement on the initial prompt word related to the preset conversation intention that requires the generation of prompt words, and the knowledge enhancement process does not require a large number of samples, reducing the use of computing resources.

[0078] In a possible implementation, the initial prompt word is semantically classified, including: inputting the initial prompt word into a pre-trained first natural language processing model; extracting the initial entity word in the initial prompt word, which can be performed according to the following steps: inputting the initial prompt word into a pre-trained second natural language processing model to obtain an entity word extraction result; wherein the entity word extraction result includes one or more initial entity words and a second entity word type corresponding to the initial entity word.

[0079] In a possible implementation, according to the first entity word type in the prompt word template, a target entity word is determined among the initial entity words, including: determining an initial entity word to be processed among the one or more initial entity words; if the second entity word type of the initial entity word to be processed is the same as the first entity word type, then taking the initial entity word to be processed as the target entity word.

[0080] In a possible implementation, a plurality of prompt word templates are pre-bound to the dialog intent for which prompt words need to be generated; obtaining the prompt word templates pre-bound to the dialog intent for which prompt words need to be generated includes: respectively calculating a first matching degree based on the number of vacant prompt words and a second matching degree based on the entity word type of the vacant prompt words between the initial prompt word and the plurality of prompt word templates; obtaining a total matching degree based on the first matching degree and the second matching degree; and obtaining the prompt word template with the highest total matching degree among the plurality of prompt word templates.

[0081] In one possible implementation, determining whether the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of prompt words includes: calculating the similarity between the conversation intention of the initial prompt word and the preset conversation intention that requires the generation of prompt words; if the similarity exceeds a set similarity threshold, determining that the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of prompt words.

[0082] In a possible implementation, the method further includes: obtaining a test prompt word corresponding to a target dialogue intent, inputting the test prompt word into a pre-trained large language model, and obtaining a test result; calculating a vector distance between the test result and a set standard result; if the vector distance is greater than a preset vector distance threshold, taking the target dialogue intent as a preset dialogue intent that requires generating a prompt word.

[0083] In a possible implementation, the natural language processing model includes at least one of the following models: a BERT model, a RoBERTa model, an ALBERT model, and a DistilBERT model; the first natural language processing model and the second natural language processing model may be the same model or different models.

[0084] In an embodiment of the present application, a question-answering device for a large language model is provided, comprising:

[0085] A prompt word module is used to obtain a target prompt word; wherein the target prompt word is generated according to the above method; and a question-answering module is used to input the target prompt word into a pre-trained large language model to obtain a question-answering result.

[0086] In an embodiment of the present application, the process of generating a target prompt word includes: first obtaining an initial prompt word, performing semantic classification processing on the initial prompt word, and obtaining classification information of the initial prompt word; wherein the classification information at least includes the conversation intention of the initial prompt word, then determining that the conversation intention of the initial prompt word is a preset conversation intention that requires prompt words to be generated, and obtaining a prompt word template that is pre-bound to the conversation intention that requires prompt words to be generated; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word; extracting the initial entity word in the initial prompt word, according to the first entity word type in the prompt word template, and finally, determining the target entity word in the initial entity word, using the target entity word to replace the vacancy prompt word in the prompt word template, and using the prompt word template that completes the replacement as the target prompt word. The present application can perform knowledge enhancement on the initial prompt word related to the preset conversation intention that requires prompt words to be generated, and the knowledge enhancement process does not require a large number of samples, reducing the use of computing resources.

[0087] An embodiment of the present application further provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0088] An embodiment of the present application further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiment of the present application.

[0089] The embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program executable by the at least one processor, and the computer program is used to enable the electronic device to perform the method of the embodiment of the present application when executed by the at least one processor.

[0090] refer to Figure 5 , the structural block diagram of the electronic device that can be used as the server or client of the embodiment of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0091] like Figure 5 As shown, the electronic device includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0092] Multiple components in the electronic device are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 can be any type of device that can input information to the electronic device, and the input unit 506 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 507 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include but is not limited to a disk, an optical disk. The communication unit 509 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, and / or a wireless communication transceiver, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0093] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing units, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present application may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 502 and / or a communication unit 509. In some embodiments, the computing unit 501 may be configured to perform the above method in any other appropriate manner (e.g., by means of firmware).

[0094] The computer program for implementing the method of the embodiment of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of the present application embodiment, machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable signal medium can include but is not limited to electronic, magnetic, optical, electromagnetic or infrared systems, devices or equipment, or any suitable combination of the above. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0096] It should be noted that the term "including" and its variations used in the embodiments of the present application are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present application are illustrative and not restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0097] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0098] The various steps described in the method implementation scheme provided by the embodiments of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation scheme may include additional steps and / or omit the steps shown. The scope of protection of the present application is not limited in this respect.

[0099] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0100] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of protection. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A method for generating prompt words for a large language model, including: Acquire an initial prompt word, perform semantic classification processing on the initial prompt word, and obtain classification information of the initial prompt word; wherein the classification information at least includes the dialogue intention of the initial prompt word; Determining that the conversation intention of the initial prompt word is a preset conversation intention that requires generating a prompt word; Acquire a prompt word template that is pre-bound to the conversation intention for which a prompt word needs to be generated; wherein the prompt word template includes a prompt word sentence pattern, a vacancy prompt word, and a first entity word type required by the vacancy prompt word; Extracting initial entity words from the initial prompt words, and determining target entity words from the initial entity words according to the first entity word type in the prompt word template; The target entity word is used to replace the missing prompt word in the prompt word template, and the prompt word template that completes the replacement is used as the target prompt word.

2. The method according to claim 1, wherein: The initial prompt word is subjected to semantic classification processing, including: Inputting the initial prompt word into a pre-trained first natural language processing model to obtain classification information of the initial prompt word; Extracting the initial entity words from the initial prompt words includes: The initial prompt word is input into a pre-trained second natural language processing model to obtain an entity word extraction result; wherein the entity word extraction result includes one or more initial entity words and a second entity word type corresponding to the initial entity word.

3. The method according to claim 2, wherein: Determining a target entity word from the initial entity words according to the first entity word type in the prompt word template includes: Determining an initial entity word to be processed among the one or more initial entity words; If the second entity word type of the initial entity word to be processed is the same as the first entity word type, the initial entity word to be processed is used as the target entity word.

4. The method according to claim 2, wherein: A plurality of prompt word templates are pre-bound to the conversation intention for which prompt words need to be generated; Obtaining a prompt word template pre-bound to the conversation intention for which a prompt word needs to be generated, including: respectively calculating a first matching degree based on the number of vacancy prompt words and a second matching degree based on the entity word type of the vacancy prompt words between the initial prompt word and the plurality of prompt word templates; Obtaining a total matching degree based on the first matching degree and the second matching degree; The prompt word template with the highest total matching degree is obtained from the multiple prompt word templates.

5. The method according to claim 1, wherein: Determining that the conversation intention of the initial prompt word is a preset conversation intention that requires generating a prompt word includes: Calculating the similarity between the conversation intention of the initial prompt word and the preset conversation intention for which the prompt word needs to be generated; If the similarity exceeds a set similarity threshold, it is determined that the conversation intention of the initial prompt word is a preset conversation intention that requires the generation of a prompt word.

6. The method according to any one of claims 1 to 5, wherein: Also includes: Obtain a test prompt word corresponding to the target dialogue intent, input the test prompt word into a pre-trained large language model, and obtain a test result; Calculating the vector distance between the test result and the set standard result; If the vector distance is greater than a preset vector distance threshold, the target dialogue intention is used as a preset dialogue intention that requires generating a prompt word.

7. The method according to claim 2, wherein: The natural language processing model includes at least one of the following models: BERT model, RoBERTa model, ALBERT model and DistilBERT model; the first natural language processing model and the second natural language processing model can be the same model or different models.

8. Question answering methods for large language models, including: Obtaining a target prompt word; wherein the target prompt word is generated according to the method according to any one of claims 1-7; The target prompt word is input into a pre-trained large language model to obtain a question-answering result.

9. Electronic equipment, including: A memory, a processor, and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium having a computer program stored therein, wherein the computer program implements the method according to any one of claims 1 to 8 when executed by a processor.

Images