Language model cue word generation method, question and answer method and electronic equipment

Abstract

The embodiment of the invention discloses a cue word generation method of a language model, a question and answer method and electronic equipment, and the cue word generation method comprises the steps: receiving first information related to a target business of the language model, and determining business identification information corresponding to the first information, at least one cue word sample is obtained from a plurality of cue word databases corresponding to the target service according to the service identification information, the types of the cue word samples stored in the cue word databases are different, and a target cue word used for inputting the language model is determined according to the obtained cue word samples. Therefore, the obtained target cue word more accurately reflects the key information of the first information related to the target service, a relatively good cue word corresponding to the first information input by the user can be obtained, the accuracy of language model output is improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the technical field of language models, and in particular to a prompt word generation method, a question-answering method, and an electronic device for a language model. Background Art

[0002] In the field of artificial intelligence (AI) and machine learning (ML), large language models (LLM) such as the Generative Pre-Trained Transformer (GPT) model and the Bidirectional Encoder Representations from Transformers model have made significant progress. These language models can learn the deep structure and rules of language through massive text data training, and then generate and understand natural language.

[0003] In the process of natural language generation and understanding by language models, prompts are an important bridge for the interaction between text and language models. Appropriate prompts can accurately convey user needs and guide the language model to generate high-quality, qualified outputs. Inappropriate prompts will cause the model output to fail to meet user requirements. Therefore, how to obtain more accurate prompts is very important for the training of language models and the use of model classification. Summary of the invention

[0004] The embodiments of the present application provide a prompt word generation method, a question-answering method, and an electronic device for a language model, which are used to solve the problem in the prior art of how to obtain more accurate prompt words for application in language model training and model classification.

[0005] To solve the above technical problems, in the first aspect, an implementation method of an embodiment of the present application discloses a method for generating prompt words of a language model, which is applied to a first electronic device, and the method includes: receiving first information, the first information being business inquiry information related to a target business of the language model; determining business identification information corresponding to the first information; obtaining at least one prompt word sample from each of multiple prompt word databases corresponding to the target business according to the business identification information, the prompt word samples stored in each prompt word database being of a different type, and the type corresponding to the application scenario of the target business; determining a target prompt word based on the acquired prompt word sample, the target prompt word being a prompt word used to input the language model.

[0006] By adopting the above technical solution, a prompt word database storing prompt word samples related to multiple target businesses is preset in advance in the first electronic device. When receiving business inquiry information related to the target business of the language model (for example, the first information can be input by the user through the user terminal and sent to the first electronic device, the first electronic device can be, for example, a computer, a server, a cloud server, etc., and the user terminal can be a car machine, a mobile phone, a tablet, etc.), at least one prompt word sample is obtained from each of the multiple prompt word databases. Since the types of prompt word samples stored in each prompt word database are different, and the type corresponds to the application scenario of the target business (for example, what application scenarios of the target business are included, the type of prompt word sample can correspond to these application scenarios respectively), the scope of the obtained prompt word samples is relatively wide, and the relatively accurate (for example, relatively better) target prompt word corresponding to the first information can be determined according to the obtained prompt word samples. As a result, the obtained target prompt word more accurately reflects the key information of the first information related to the target business, and can obtain a relatively better prompt word corresponding to the first information input by the user, thereby improving the accuracy of the output of the language model (such as the LLM model) and increasing the user experience.

[0007] According to another specific implementation method of the embodiment of the present application, the implementation method of the embodiment of the present application discloses a method for generating prompt words of a language model, which method also includes: inputting a target prompt word into the language model for classification processing to obtain a classification result; when it is determined according to the classification result that the target prompt word does not meet the preset requirements, generating feedback information for the user to update the prompt word database according to the feedback information.

[0008] By adopting the above technical solution, when it is determined according to the classification result that the target prompt word does not meet the preset requirements, feedback information is generated, and the user can update the prompt word database according to the feedback information. As a result, the prompt word database can be updated in real time and dynamically according to the classification results, effectively increasing the number and type of prompt word samples in the prompt word database, thereby improving the accuracy of the corresponding target prompt word obtained according to the user information, improving the accuracy of the model output, and enhancing the user experience.

[0009] According to another specific implementation method of the embodiment of the present application, the implementation method of the embodiment of the present application discloses a method for generating prompt words of a language model, which determines the business identification information corresponding to the first information, including: determining a first keyword related to the business identification included in the first information; and determining the business identification information based on the first keyword.

[0010] By adopting the above technical solution, the service identification information determined according to the first keyword related to the service identification included in the first information is more accurate.

[0011] According to another specific implementation method of the embodiment of the present application, the implementation method of the embodiment of the present application discloses a prompt word generation method of a language model, which obtains at least one prompt word sample from each of multiple prompt word databases corresponding to the target business according to the business identification information, including: determining the target business according to the business identification information; determining the matching score between the prompt word samples included in the prompt word database corresponding to the target business and the first information; selecting the prompt word samples whose matching scores meet the target conditions to obtain at least one prompt word sample.

[0012] By adopting the above technical solution, by calculating the matching score between the prompt word sample and the first information, and selecting the prompt word sample that meets the target condition according to the matching score, the selection process is more intuitive and the accuracy of the corresponding target prompt word obtained according to the user information is further improved.

[0013] According to another specific implementation method of the embodiment of the present application, the implementation method of the embodiment of the present application discloses a prompt word generation method of a language model, which determines the matching score between the prompt word sample included in the prompt word database corresponding to the target business and the first information, including: determining the second keyword corresponding to the prompt word sample included in the prompt word database, and determining the third keyword included in the first information; determining the matching score based on the prompt word sample and the third keyword.

[0014] By adopting the above technical solution, the matching score determined according to the second keyword corresponding to the prompt word sample and the third keyword included in the first information is more accurate.

[0015] According to another specific implementation method of the embodiment of the present application, the implementation method of the embodiment of the present application discloses a prompt word generation method of a language model, which selects prompt word samples whose matching scores meet the target conditions, including: selecting a preset number of prompt word samples corresponding to the matching scores in order from high to low as prompt word samples that meet the target conditions.

[0016] By adopting the above technical solution, a preset number of prompt word samples corresponding to matching scores are selected in order from high to low matching scores, so that the selected prompt word samples can have a higher matching degree with the first information, thereby improving the accuracy of the corresponding target prompt word obtained according to the user information.

[0017] According to another specific implementation of the embodiment of the present application, the implementation of the embodiment of the present application discloses a prompt word generation method for a language model, wherein the multiple prompt word databases include multiple positive prompt word databases and multiple negative prompt word databases, the positive prompt word database stores positive prompt word samples, and the negative prompt word database stores negative prompt word samples, and at least one prompt word sample is obtained from each of the multiple prompt word databases corresponding to the target business according to the business identification information, including: obtaining at least one positive prompt word sample from each positive prompt word database according to the business identification information, and obtaining at least one negative prompt word sample from each negative prompt word database.

[0018] By adopting the above technical scheme, multiple prompt word databases are divided into positive prompt word databases and negative prompt word databases, and at least one positive prompt word sample is obtained from each positive prompt word database according to the business identification information, and at least one negative prompt word sample is obtained from each negative prompt word database, thereby increasing the comprehensiveness of the prompt word sample coverage and thus improving the accuracy of the corresponding target prompt word obtained according to the user information.

[0019] According to another specific implementation of the embodiment of the present application, the implementation of the embodiment of the present application discloses a method for generating prompt words of a language model, and the preset numbers corresponding to each prompt word database are the same.

[0020] By adopting the above technical solution, the number of prompt word samples selected from each prompt word database is the same, that is, the proportions of prompt word samples of various types are the same, thereby obtaining a relatively accurate target prompt word.

[0021] According to another specific implementation of the embodiment of the present application, the implementation of the embodiment of the present application discloses a method for generating prompt words of a language model, which determines a target prompt word based on an acquired prompt word sample, including: concatenating the acquired prompt word samples to obtain a target prompt word.

[0022] By adopting the above technical solution, the prompt word samples are spliced ​​and processed, and the key information corresponding to the first information included in the obtained target prompt word is more comprehensive, which can make the output of the model more accurate.

[0023] According to another specific implementation of the embodiment of the present application, the implementation of the embodiment of the present application discloses a prompt word generation method of a language model, and the prompt word database is a vector database.

[0024] According to another specific implementation of the embodiment of the present application, the implementation of the embodiment of the present application discloses a method for generating prompt words of a language model, wherein the first information is text information or voice information.

[0025] In the second aspect, the implementation method of the embodiment of the present application also discloses a question-and-answer method, which includes: a second electronic device receives second information and sends the second information to a first electronic device, where the second information is business inquiry information related to a target business; the first electronic device obtains a target prompt word based on the received second information, and inputs the target prompt word into a language model to obtain an output result, and sends the output result to the second electronic device; wherein the output result is answer information corresponding to the second information, and the target prompt word is obtained based on the prompt word generation method of any language model of the first aspect.

[0026] On the third aspect, the implementation method of the embodiment of the present application also discloses a prompt word generation device for a language model, including: a first processing module, used to receive first information, the first information is business inquiry information related to the target business of the language model; a second processing module, used to determine the business identification information corresponding to the first information; a third processing module, used to obtain at least one prompt word sample from each of multiple prompt word databases corresponding to the target business according to the business identification information, the prompt word samples stored in each prompt word database are of different types, and the type corresponds to the application scenario of the target business; a fourth processing module, used to determine the target prompt word based on the acquired prompt word sample, the target prompt word is the prompt word used to input the language model.

[0027] In a fourth aspect, an implementation method of an embodiment of the present application further discloses an electronic device, comprising: a processor, and a memory connected to the processor in communication; the memory stores a computer program; the processor executes the computer program stored in the memory, so that the electronic device implements the prompt word generation method of the language model provided in any one of the implementation methods of the first aspect above, or the question and answer method provided in the implementation method of the second aspect.

[0028] In a fifth aspect, an implementation method of an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement a method for generating prompt words for a language model provided in any one of the implementation methods of the first aspect above, or a question-and-answer method provided in the implementation method of the second aspect.

[0029] In a sixth aspect, an implementation method of an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements a method for generating prompt words for a language model provided in any one of the implementation methods of the first aspect above, or a question-and-answer method provided in the implementation method of the second aspect.

[0030] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can also be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flowchart of a method for generating prompt words of a language model provided in an embodiment of the present application;

[0032] Figure 2 is a schematic diagram of a flow chart of obtaining at least one prompt word sample provided by an embodiment of the present application;

[0033] Figure 3 It is a flowchart of another method for generating prompt words of a language model provided in an embodiment of the present application;

[0034] Figure 4 It is a flowchart of another method for generating prompt words of a language model provided in an embodiment of the present application;

[0035] Figure 5 It is a flowchart of another method for generating prompt words of a language model provided in an embodiment of the present application;

[0036] Figure 6 It is a flowchart of another method for generating prompt words of a language model provided in an embodiment of the present application;

[0037] Figure 7 is a structural diagram of a question-answering system provided in an embodiment of the present application;

[0038] Figure 8 It is a flowchart of a question-and-answer method provided in an embodiment of the present application;

[0039] Fig. 9 It is a structural schematic diagram of a prompt word generating device of a language model provided in an embodiment of the present application;

[0040] Fig.10 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0041] Specific implementation method

[0042] The embodiment of the present application provides a method for generating prompt words of a language model, wherein a prompt word database storing prompt word samples related to multiple target businesses is preset in advance on a first electronic device, and upon receiving business inquiry information (such as first information) related to the target business of the language model input by a user, at least one prompt word sample is obtained from each of the multiple prompt word databases. Since the types of prompt word samples stored in each prompt word database are different, the scope covered by the obtained prompt word samples is relatively wide, and a relatively accurate (such as relatively better) target prompt word corresponding to the first information can be determined based on the obtained prompt word samples. Thus, the obtained target prompt word more accurately reflects the key information of the first information related to the target business, and a relatively better prompt word corresponding to the first information input by the user can be obtained, thereby improving the accuracy of the output of the language model (such as the LLM model) and increasing the user experience.

[0043] Next, in conjunction with the accompanying drawings, the specific steps of the method for generating prompt words of the language model provided in the embodiment of the present application are described in detail.

[0044] First, the terms used in the embodiments of the present application are explained.

[0045] Language model: For example, it can be a large language model (LLM for short), which refers to a deep learning model trained with large amounts of text data, which can generate natural language text or understand the meaning of language text. Language models can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc., and are an important path to artificial intelligence.

[0046] Prompt: Also known as prompt, it refers to a short text or instruction used to guide the model to generate a specific type of response or output. Through carefully designed prompts, the model's internal knowledge, reasoning ability, and creativity can be effectively stimulated to generate diverse content that meets the needs (such as output that meets user requirements).

[0047] Vector database: A vector database is a database specifically used to store and query vectors. The vectors stored in it come from the vectorization of text, speech, images, videos, etc. Compared with traditional databases, vector databases can process more unstructured data (such as images and audio). In machine learning and deep learning, data is usually represented in vector form.

[0048] In an implementation of the embodiment of the present application, the prompt word samples in the prompt word database are represented in the form of vectors.

[0049] In one implementation of the embodiment of the present application, the prompt word generation method of the language model provided in the embodiment of the present application is applied to a first electronic device, such as a computer, a server, a cloud server, a cluster, etc. The following takes the first electronic device as an example of a server to illustrate the prompt word generation method provided in the embodiment of the present application.

[0050] When a user needs to use a language model to query a question, the user can ask a question through a user terminal (such as a car computer, mobile phone, tablet, etc.), and the user terminal sends the query information to the server. The server can determine the target prompt word for the language model to classify based on the query information. The server then inputs the target prompt word into the language model so that the language model outputs the answer information corresponding to the query information based on the target prompt word. The server sends the answer information to the user terminal for the user to view the query results and perform corresponding processing.

[0051] In one implementation of the embodiment of the present application, Figure 1 As shown, the method for generating prompt words of the language model provided in the embodiment of the present application is applied to the server, and includes the following steps:

[0052] S100: Receive first information.

[0053] The first information is query information related to the target business of the language model. The query information may be information input by a user to a server through a user terminal, such as "CPU temperature is too high".

[0054] S200: Determine the service identification information corresponding to the first information.

[0055] In an implementation manner of the embodiment of the present application, determining the service identification information corresponding to the first information includes: determining a first keyword related to the service identification included in the first information, and determining the service identification information according to the first keyword.

[0056] The first keyword is, for example, a field included in the first information that can be used to identify a service type. The service type corresponding to the current first information can be accurately located according to the first keyword.

[0057] S300: Obtain at least one prompt word sample from each of a plurality of prompt word databases corresponding to the target business according to the business identification information.

[0058] In an implementation of the embodiment of the present application, at least one prompt word sample is obtained from each of multiple prompt word databases corresponding to the target business according to the business identification information, such as Figure 2 As shown, the following steps are included:

[0059] S310: Determine the target service according to the service identification information.

[0060] S320: Determine a matching score between the prompt word sample included in the prompt word database corresponding to the target business and the first information.

[0061] In one implementation of the embodiment of the present application, determining a matching score between a prompt word sample included in a prompt word database corresponding to a target business and the first information includes: determining a second keyword included in the first information that identifies the first information; and determining a matching score based on the prompt word sample and the second keyword.

[0062] In one implementation of the embodiment of the present application, selecting prompt word samples whose matching scores meet the target condition includes: selecting a preset number of prompt word samples corresponding to the matching scores in descending order as prompt word samples that meet the target condition.

[0063] In other implementations of the embodiments of the present application, the prompt word samples whose matching scores meet the target conditions may be selected, and the prompt word samples whose matching scores are greater than a preset score threshold may be selected as the prompt word samples that meet the target conditions.

[0064] S330: Select prompt word samples whose matching scores meet the target condition to obtain at least one prompt word sample.

[0065] In other implementations of the embodiments of the present application, the multiple prompt word databases include multiple positive prompt word databases and multiple negative prompt word databases, the positive prompt word database stores positive prompt word samples, the negative prompt word database stores negative prompt word samples, and at least one prompt word sample is obtained from each of the multiple prompt word databases corresponding to the target business according to the business identification information, including: obtaining at least one positive prompt word sample from each positive prompt word database according to the business identification information, and obtaining at least one negative prompt word sample from each negative prompt word database.

[0066] The specific method of obtaining at least one positive prompt word sample from the positive prompt word database and at least one negative prompt word sample from the negative prompt word database is similar to the specific method of obtaining at least one prompt word sample from each of the multiple prompt word databases corresponding to the target business.

[0067] It should be noted that positive samples refer to samples that match the real target category, such as samples whose predicted results are consistent with the actual results, while negative samples refer to samples that do not match the real target category. Therefore, the positive prompt word samples stored in the positive prompt word database can be understood as samples that match the application scenario of the target business, and the negative prompt word samples stored in the negative prompt word database can be understood as samples that do not match the application scenario of the target business.

[0068] S400: determining a target prompt word according to the acquired prompt word sample, where the target prompt word is a prompt word used to input a language model.

[0069] In an implementation of the embodiment of the present application, determining a target prompt word according to the acquired prompt word samples includes: performing splicing processing on the acquired prompt word samples to obtain the target prompt word.

[0070] In one implementation of the embodiment of the present application, Figure 3 As shown, the method for generating prompt words of the language model provided in the embodiment of the present application also includes the following steps:

[0071] S500: Input the target prompt word into the language model for classification processing to obtain a classification result.

[0072] S600: When it is determined according to the classification result that the target prompt word does not meet the preset requirement, feedback information is generated so that the user can update the prompt word database according to the feedback information.

[0073] In an implementation of the embodiment of the present application, the preset numbers corresponding to the respective prompt word databases are the same.

[0074] The prompt word generation method of the language model provided in the embodiment of the present application is that a prompt word database storing prompt word samples related to multiple target businesses is preset in advance in the first electronic device. When receiving business inquiry information related to the target business of the language model (for example, the first information can be input by the user through the user terminal and sent to the first electronic device, the first electronic device can be a computer, a server, a cloud server, etc., and the user terminal can be a car machine, a mobile phone, a tablet, etc.), at least one prompt word sample is obtained from each of the multiple prompt word databases. Since the types of prompt word samples stored in each prompt word database are different, and the type corresponds to the application scenario of the target business (for example, what application scenarios of the target business include, the type of prompt word sample can correspond to these application scenarios respectively), the scope of the obtained prompt word sample is relatively wide, and the relatively accurate (for example, relatively better) target prompt word corresponding to the first information can be determined according to the obtained prompt word sample. As a result, the obtained target prompt word more accurately reflects the key information of the first information related to the target business, and can obtain a relatively better prompt word corresponding to the first information input by the user, thereby improving the accuracy of the output of the language model (such as the LLM model) and increasing the user experience.

[0075] Next, refer to Figure 4 and Figure 5 , the prompt word generation method of the language model provided in the embodiment of the present application is further explained.

[0076] For example, Figure 4As shown, in one implementation of an embodiment of the present application, the first information received in the above step S100 can be a text message "CPU temperature is too high" (for example, the first information) input by the user through a user terminal with a screen (such as a car computer, a mobile phone, a tablet, etc.) received by the server, or it can be a voice message "CPU temperature is too high" input by the user through the user terminal and then converted into a text message.

[0077] The business identification information included in the first information determined in step S200 can be that the server locates the target business of the language model currently required as a model classification question based on the information "CPU temperature is too high" input by the user. Among them, CPU is also the first keyword included in the first information. According to the first keyword, it can be determined that the CPU in the question is a kind of business identification information. After obtaining the CPU word, the electronic device will determine that the current target business is a model classification business. The business identification information of the business can also be determined by other means, such as active input by the user, or the electronic device asks again, to obtain the corresponding business identification information related content.

[0078] Step S300, obtaining at least one prompt word sample from each of multiple prompt word databases corresponding to the target business according to the business identification information, for example, may include:

[0079] After determining that the target business is the model classification business, multiple prompt word databases of the model classification business can be obtained according to pre-stored data, namely, a required product model vector library and a unnecessary product model vector library.

[0080] Among them, the application scenarios corresponding to the model classification business include scenarios requiring product models and scenarios not requiring product models. According to these application scenarios, it can be determined that the model classification business corresponds to scenarios requiring product model vector libraries and scenarios not requiring product model vector libraries.

[0081] Among them, there are multiple prompt word databases corresponding to the model classification business that require a product model vector library and those that do not require a product model vector library. The product model vector library needs to store relevant prompt word samples for product models that need to be queried, and the product model vector library does not need to store relevant prompt word samples for product models that do not need to be queried.

[0082] After determining the required product model vector library and the unnecessary product model vector library, determine the second keyword corresponding to the user's question, the second keyword is such as CPU, temperature, high, and then use the second keyword to search the two pre-stored vector libraries, namely the required product model vector library and the unnecessary product model vector library, and respectively take out two prompt word samples (such as prompt word samples with matching scores ranked first and second) that are most relevant to the user's question (such as CPU temperature is too high) after judgment (such as calculating the matching score between the second keyword and the prompt word sample). Specifically, the two prompt word samples taken out from the required product model vector library are: the CPU's P and T energy-saving settings are set to cause frequency reduction, corresponding to the answer to the product model class that needs to be asked, and the CPU temperature is too high abnormal alarm, corresponding to the answer to the product model class that needs to be asked; the two prompt word samples taken out from the unnecessary product model vector library are: CPU failure causes memory error Configuration error, corresponding to the answer to the product model class that does not need to be asked, and CPU disassembly and assembly steps, corresponding to the answer to the product model class that does not need to be asked.

[0083] Step S400, determining a target prompt word according to the acquired prompt word samples, for example, by splicing the prompt word samples, a target prompt word {need to inquire about product model category, no need to inquire about product model category} is obtained.

[0084] like Figure 4 As shown, the server now plays the role of a question classification master, receiving user questions from user terminals and classifying them, mainly including:

[0085] The server receives the inquiry information input by the user through the user terminal, and determines to which category (eg, a specific target business) the question input by the user belongs.

[0086] The server determines the target prompt word based on the corresponding prompt word database of the target business and the query information, and then inputs the target prompt word into the language model for judgment. The language model determines similar cases based on the question and the corresponding target prompt word, and obtains the processing result (such as answer information) as "product model category required", and sends the processing result to the user terminal.

[0087] The user terminal receives the processing result so that the user can perform corresponding operations according to the processing result. If the user judges that the processing result of the language model is wrong after the language model outputs the result, the user feedback can be provided to the first electronic device through the user terminal so that the first point device updates the prompt word database (such as the vector library).

[0088] In one implementation of the embodiment of the present application, the aforementioned search through the user question (CPU temperature is too high) to the two pre-stored vector libraries, namely the required product model vector library and the unnecessary product model vector library, to obtain four prompt word samples can be performed by respectively calculating the matching scores of the user question and the prompt word samples in the required product model vector library, and selecting the prompt word samples corresponding to the first two (for example, a preset number) matching scores in order from high to low, and calculating the matching scores of the user question and the prompt word samples in the unnecessary product model vector library, and selecting the prompt word samples corresponding to the first two matching scores in order from high to low, to obtain the aforementioned prompt word samples.

[0089] In another implementation method of the embodiment of the present application, the matching scores of the user questions and all the prompt word samples in the required product model vector library and the unnecessary product model vector library can be calculated, and the matching scores can be comprehensively sorted. The prompt word samples corresponding to the first four matching scores can be selected in order from high to low to obtain the aforementioned prompt word samples.

[0090] In another implementation of the embodiment of the present application, in the process of calculating the matching scores between the user question and all the prompt word samples in the required product model vector library and the unnecessary product model vector library, and selecting a preset number of prompt word samples according to the matching scores, different selection ratios can also be set for different vector libraries, for example, a total of four are selected, the proportion selected from the required product model vector library accounts for one, and the proportion selected from the unnecessary product model vector library accounts for three, that is, for the required product model vector library, the prompt word sample corresponding to the previous (as an example of the preset number) matching score is selected in the order of matching scores from high to low, and for the unnecessary product model vector library, the prompt word samples corresponding to the previous three (as an example of the preset number) matching scores are selected in the order of matching scores from high to low. The specific setting ratio can be set as needed.

[0091] In another implementation of the embodiment of the present application, if the user question (i.e., Question) and the cases of the prompt word samples (i.e., Examples) in the prompt word database are very similar when ignoring capitalization, the target prompt word obtained has the same prompt word content as the prompt word sample.

[0092] In one implementation of the embodiment of the present application, the aforementioned required product model vector library may be, for example, a positive prompt word database, and the non-required product model vector library may be, for example, a negative prompt word database.

[0093] For example, Figure 5 As shown, in another implementation of the embodiment of the present application, step S100, receiving the first information, may be that the server receives a user question input by the user through a user terminal.

[0094] The business identification information included in the first information determined in step S200 may be determined according to the first keyword of the user's question, and the target business and the prompt word database corresponding to the target business are determined as a type A question vector library, a type B question vector library, and a type C question vector library.

[0095] Step S300, at least one prompt word sample is obtained from each of the multiple prompt word databases corresponding to the target business according to the business identification information, which may be a search and search from a category A question vector library, a category B question vector library, and a category C question vector library (as an example of multiple prompt word databases), through a search and matching method similar to that of the first embodiment, for example, calculating a matching score, and obtaining n (as an example of a preset number) prompt word samples corresponding to each vector library according to the matching score, for example, the question is XXXX, corresponding to the answer category A, ..., the question is XXXX, corresponding to the answer category B, ..., the question is XXXX, corresponding to the answer category C, ...

[0096] Among them, the A-type question vector library, the B-type question vector library, and the C-type question vector library store three types of prompt word samples: A, B, and C. That is, the application scenarios corresponding to the target business include A, B, and C.

[0097] In other implementations, the A-type question vector library, the B-type question vector library, and the C-type question vector library may include a positive prompt word database and a negative prompt word database.

[0098] Step S400, determining a target prompt word according to the acquired prompt word samples, for example, the 3n prompt word samples may be concatenated to obtain corresponding target prompt words {category A, category B, category C}.

[0099] That is, the server now plays the role of a question classification master, and its tasks include:

[0100] The server receives the inquiry information input by the user through the user terminal, and determines to which category (eg, a specific target business) the question input by the user belongs.

[0101] The server determines the target prompt word based on the corresponding prompt word database of the target business and the query information, and then inputs the target prompt word into the language model for judgment. The language model determines similar cases based on the question and the corresponding target prompt word, and obtains the processing result (such as answer information) as XXXX, and sends the processing result to the user terminal.

[0102] The user terminal receives the processing result so that the user can perform corresponding operations according to the processing result. If the user judges that the processing result of the language model is wrong after the language model outputs the result, the user feedback can be provided to the first electronic device through the user terminal so that the first point device updates the prompt word database (such as the vector library).

[0103] The execution steps of other contents are similar to the above contents and will not be repeated here.

[0104] In one implementation of the embodiment of the present application, Figure 6 As shown, the method for generating prompt words of the language model provided in the embodiment of the present application is applied to the server, and includes the following steps:

[0105] S10: Obtain the user question sent by the user through the user terminal.

[0106] S20: According to the user's question, search and find in the A-type vector library, the B-type vector library, ..., the N-type vector library (i.e., N prompt word databases), wherein the A-type vector library, the B-type vector library, ..., the N-type vector library stores N types of different prompt word samples.

[0107] The server uses a retrieval matching method similar to that in the first embodiment, for example, to calculate a matching score, and obtains m (as an example of a preset number) prompt word samples corresponding to each vector library according to the matching score.

[0108] S30: The result (eg, prompt word sample) searched out from the vector library is spliced ​​into prompt as a case to obtain the target prompt word.

[0109] S40: Input the target prompt word into the LLM language model (as an example of a language model) to do the classification problem. If the classification result is correct, go to step S50; if the classification result is wrong, go to step S60.

[0110] S50: The method is then executed to completion.

[0111] S60: If the classification result is wrong, the user provides feedback and updates the incorrectly classified prompt word samples to the relevant vector library.

[0112] The execution steps of other contents are similar to those in Example 1 and will not be repeated here.

[0113] In one implementation of the embodiment of the present application, multiple prompt word databases can be obtained in the following manner:

[0114] The user makes up a small number of cases and stores them in multiple categories (such as A, B, and C) of text files. After that, these multiple categories of data are stored in the vector library. When receiving user questions, the user questions are first used as search terms to search the vector library. Here, the vector library storing each type of question is searched, and then the top n data in each vector library with the highest search results are taken out and spliced ​​into the prompt word (Prompt) as examples. In the subsequent testing process, many cases are generated. The cases in the testing process are updated to the vector library to obtain the basic case library (such as multiple prompt word databases). In this way, there will be a large number of cases to support the large model to make judgments.

[0115] After the subsequent launch, a user feedback channel will be built. When users ask questions and feel that the classification of the questions is not accurate, they can give feedback to the backend. If it is the user's own system, it can be directly updated to the vector library, so that the next time there is a similar question, it can be correctly classified; if it is a cloud product used by the user, after feedback and review, the user's feedback case can also be updated to the relevant vector library. In this way, each user can dynamically adjust the classification effect to solve the problem that the case library cannot be updated in real time.

[0116] For language model classification scenarios, there are simple dual-classification problems, such as classifying user questions into {need..., don't need...} categories, and there are multi-classification scenarios, such as classifying user questions into {chat, product,..., privacy} questions. There are also other classification scenarios, which can be set according to needs.

[0117] In different classification scenarios, the corresponding target businesses are different. For different target businesses, the number of corresponding prompt word databases and the included prompt word samples are also different.

[0118] The prompt word generation method of the language model provided in the embodiment of the present application is based on the existing prompt word optimization means, and is improved on the basis of few samples and dynamic samples to solve some disadvantages of zero samples, few samples and many samples, and construct a method flow for prompt word optimization (such as prompt word generation), so that the prompt words have enough cases to support model judgment, the cases can change dynamically according to the user's questions, the cases can be updated in real time, and the cases can provide positive and negative samples.

[0119] The prompt word generation method of the language model provided in the embodiment of the present application constructs a positive and negative sample library (such as a prompt word database), which can combine user feedback results with a vector library, combine prompt cases with a vector library, and combine existing solutions.

[0120] The prompt word generation method of the language model provided in the embodiment of the present application provides some samples to help the model better classify when using the language model in the language model classification scenario to classify user questions. The step of providing samples is improved to obtain a vector library storing a large number of prompt word samples. In the subsequent use process, when the language model classification result is wrong, the case library is dynamically updated and adjusted through user feedback.

[0121] The core part of the prompt word generation method of the language model provided in the embodiment of the present application includes: storing the cases in a plurality of vector libraries of different classifications (such as a prompt word database), and after the user question is input, taking the user question as a search term, and taking out the top m data with high scores from each vector library of different classifications, so that each prompt word case is dynamically adjusted according to the user question, which enables the model to make better and more accurate judgments. The proportion of cases taken out from each vector library can be controlled to solve the problem of uneven proportion of positive and negative samples. The vector library stores a large number of cases, and the type coverage of user questions is also relatively comprehensive, so there can be a large number of cases to support model classification.

[0122] The core part of the prompt word generation method of the language model provided in the embodiment of the present application also includes: building a user feedback channel. When the user determines that the model has classified his or her user problem incorrectly, the user can provide feedback to the background to provide his or her case. After that, the user's case can be updated to the relevant vector library. The next time a similar problem is encountered, the problem can be accurately classified, thereby realizing real-time updating and dynamic adjustment of the case library.

[0123] The prompt word generation method of the language model provided in the embodiment of the present application solves the problem that the ratio of positive and negative cases (such as positive and negative prompt word samples) cannot be controlled when the prompt word case (such as prompt word sample) is written (such as generated). By storing the categories in different vector libraries, the number and ratio of cases in each category can be controlled during retrieval. It also solves the problem that the case cannot be updated in real time according to user feedback, and a user feedback channel is constructed to update the case library in real time. It also solves the problem that the case must change dynamically according to the user's question. By searching the vector library, more similar and more suitable cases can be retrieved for each question and spliced ​​into the prompt. It also solves the problem that the number of prompt samples is too small and the case coverage is not comprehensive. All cases are stored in the vector library. A large number of cases can be stored in the vector library. When used, m most suitable cases are taken out, and the cases are fully covered, and those spliced ​​into the prompt word do not appear redundant.

[0124] The prompt word generation method of the language model provided in the embodiment of the present application constructs a database containing positive and negative samples (such as a prompt word database), constructs a vector database of multiple classification results, and selects the most relevant n samples in each vector library for reference by the language model when performing retrieval. Since these vector libraries must contain positive samples and a certain number of negative samples, the problem of not being able to provide positive and negative samples is solved.

[0125] We further combine user feedback results with the vector library, provide an entry for feedback on each judgment during the user testing and production stages, and write the feedback results into the vector database in real time to ensure that user problems can be corrected in a timely manner if classification errors occur. This builds a feedback channel and solves the problem of the case library not being able to be updated in real time.

[0126] We further combined the prompt cases with the vector library, and stored some classified cases in the vector library after classification. After going online, the user's questions will first be searched in each vector library, and the top n cases from each case library will be spliced ​​into the prompt. We built a rich set of cases by using product test cases + model generation cases + vector database, and stored the cases according to semantics, solving the problem of insufficient cases.

[0127] Further combine the existing solutions, adopt thinking chains, apply frameworks, use separators, labels, and dynamic samples to write prompt words, and use dynamic sample solutions to solve the problem that cases must change dynamically according to user questions.

[0128] In another implementation of the embodiment of the present application, the Figure 7 The question-answering system shown includes a first electronic device and a second electronic device. The first electronic device may be the aforementioned server, or a computer, a cloud server, etc. The second electronic device may be the aforementioned user terminal, such as a car computer, a mobile phone, a tablet, etc. Figure 8 As shown, the method comprises the steps of:

[0129] S10': The second electronic device receives the second information and sends the second information to the first electronic device, where the second information is business inquiry information related to the target business.

[0130] S20': the first electronic device obtains a target prompt word according to the received second information, inputs the target prompt word into a language model, obtains an output result, and sends the output result to the second electronic device.

[0131] The output result is answer information corresponding to the second information, and the target prompt word is obtained based on the prompt word generation method of the aforementioned language model.

[0132] In another implementation of the embodiment of the present application, Fig. 9 As shown, a prompt word generating device of a language model is provided, comprising:

[0133] The first processing module is used to receive first information, where the first information is business inquiry information related to the target business of the language model.

[0134] The second processing module is used to determine the service identification information corresponding to the first information.

[0135] The third processing module is used to obtain at least one prompt word sample from each of the multiple prompt word databases corresponding to the target business according to the business identification information. The prompt word samples stored in each prompt word database are of different types, and the types correspond to the application scenarios of the target business.

[0136] The fourth processing module is used to determine a target prompt word according to the acquired prompt word sample, where the target prompt word is a prompt word used to input the language model.

[0137] The prompt word generating device of the language model may be the aforementioned first electronic device, or may be a module included in the first electronic device.

[0138] The prompt word generation device of the language model may also include more processing modules to execute the steps of the prompt word generation method of the language model.

[0139] See also Fig.10 , Fig.10 The figure shows a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.10 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0140] The processor 122 executes the computer-executable instructions stored in the memory, so that the processor 122 executes part of the technical solution of the prompt word generation method of the language model in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital data processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components.

[0141] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0142] By way of example and not limitation, the memory 123 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 123 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 123 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 123 is a non-volatile solid-state memory. In a particular embodiment, the memory 123 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0143] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0144] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0145] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the method for generating prompt words of the language model in the above embodiment.

[0146] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a processor of an electronic device, the processor of the electronic device executes the technical solution of the method for generating prompt words of the language model of the above embodiment.

[0147] In some possible implementations, various aspects of the method provided in the embodiments of the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a processor of an electronic device, the program code is used to enable the processor of the electronic device to execute the steps of the method described above in this specification according to various exemplary implementations of the embodiments of the present application. For example, the electronic device can execute the prompt word generation method of the language model recorded in the embodiments of the present application.

[0148] The program product may employ any combination of one or more readable media. The readable medium may be a readable data medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0149] The implementation method of the embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the prompt word generation method of the language model in the above embodiment.

[0150] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products provided according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable information processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable information processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0151] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable information processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0152] These computer program instructions can also be loaded onto a computer or other programmable information processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0153] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0154] It should be noted that, in addition to the implementation of the present application described in the above-mentioned specific embodiments, those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Although the description of the present application is introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to the implementation. On the contrary, the purpose of introducing the invention in conjunction with the implementation is to cover other options or modifications that may extend based on the claims of the present application. In order to provide an in-depth understanding of the present application, the above description contains many specific details, and the present application may also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present application, some specific details will be omitted in the description. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0155] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0156] Although the present application has been illustrated and described by referring to certain preferred implementations of the present application, it should be understood by those skilled in the art that the above contents are further detailed descriptions of the present application in combination with specific implementations, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. Those skilled in the art may make various changes in form and details, including making several simple deductions or substitutions, without departing from the spirit and scope of the present application.

Claims

1. A method for generating prompt words of a language model, applied to a first electronic device, characterized in that: The method comprises: receiving first information, where the first information is business inquiry information related to a target business of the language model; Determining service identification information corresponding to the first information; Acquire at least one prompt word sample from each of a plurality of prompt word databases corresponding to the target business according to the business identification information, wherein the prompt word samples stored in each of the prompt word databases are of different types, and the types correspond to application scenarios of the target business; A target prompt word is determined according to the acquired prompt word sample, where the target prompt word is a prompt word used to input the language model.

2. The method for generating prompt words of a language model according to claim 1, wherein: The method further comprises: Inputting the target prompt word into the language model for classification processing to obtain a classification result; When it is determined according to the classification result that the target prompt word does not meet the preset requirement, feedback information is generated so that the user can update the prompt word database according to the feedback information.

3. The method for generating prompt words of a language model according to claim 1 or 2, characterized in that: Determining the service identification information corresponding to the first information includes: Determining a first keyword related to the service identifier included in the first information; The service identification information is determined according to the first keyword.

4. The method for generating prompt words of a language model according to any one of claims 1 to 3, characterized in that: Acquiring at least one prompt word sample from each of a plurality of prompt word databases corresponding to the target business according to the business identification information, including: Determine the target service according to the service identification information; Determine a matching score between the prompt word sample included in the prompt word database corresponding to the target business and the first information; The prompt word sample whose matching score meets the target condition is selected to obtain the at least one prompt word sample.

5. The method for generating prompt words of a language model according to claim 4, wherein: Determining a matching score between the prompt word sample included in the prompt word database corresponding to the target business and the first information includes: determining a second keyword included in the first information and identifying the first information; The matching score is determined according to the prompt word sample and the second keyword.

6. The method for generating prompt words of a language model according to claim 5, characterized in that: Selecting the prompt word sample whose matching score meets the target condition includes: A preset number of the prompt word samples corresponding to the matching scores are selected in descending order of the matching scores as the prompt word samples that meet the target condition.

7. The method for generating prompt words of a language model according to any one of claims 1 to 6, characterized in that: The multiple prompt word databases include multiple positive prompt word databases and multiple negative prompt word databases, the positive prompt word database stores positive prompt word samples, and the negative prompt word database stores negative prompt word samples. At least one prompt word sample is obtained from each of the multiple prompt word databases corresponding to the target business according to the business identification information, including: At least one positive prompt word sample is acquired from each of the positive prompt word databases according to the service identification information, and at least one negative prompt word sample is acquired from each of the negative prompt word databases.

8. The method for generating prompt words of a language model according to any one of claims 1 to 7, characterized in that: Determining a target prompt word according to the acquired prompt word sample includes: The acquired prompt word samples are spliced ​​to obtain the target prompt word.

9. A question-answering method, characterized in that: The method comprises: The second electronic device receives the second information and sends the second information to the first electronic device, where the second information is business inquiry information related to the target business; The first electronic device obtains a target prompt word based on the received second information, inputs the target prompt word into a language model, obtains an output result, and sends the output result to the second electronic device; wherein the output result is answer information corresponding to the second information, and the target prompt word is obtained based on the prompt word generation method of the language model described in any one of claims 1-8.

10. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores a computer program; The processor executes the computer program stored in the memory so that the electronic device implements the method for generating prompt words of the language model as described in any one of claims 1-8.

Images