Information determination method and device, model training method and device, equipment, storage medium and computer program product

Abstract

The embodiment of the invention discloses an information determination method. The method comprises the steps of obtaining background information corresponding to information to be processed and first prompt information of a user for the information to be processed; processing the first prompt information and the background information by adopting a first target large language model to obtain a plurality of pieces of second prompt information corresponding to the to-be-processed information; and inputting the plurality of pieces of second prompt information into a second target large language model to obtain a plurality of pieces of response information, and determining target prompt information corresponding to the to-be-processed information based on the plurality of pieces of response information. The embodiment of the invention further discloses a model training method and device, equipment, a storage medium and a computer program product.

Description

Technical Field

[0001] The present application relates to the field of computer technologies, and in particular, to a method, apparatus, device, storage medium, and computer program product for information determination and model training. Background Art

[0002] With the development of technology, intelligent question-and-answer systems based on large language models (LLMs) have been increasingly widely used in daily life. Currently, it is usually necessary to input a pre-constructed prompt into the intelligent question-and-answer system first to help the LLM better understand the user's question and provide accurate answers. However, in related technologies, the prompt is usually determined manually by developers according to business scenarios and input into the intelligent question-and-answer system. This way of determining the prompt completely relying on human experience not only has low accuracy but also poor efficiency. Summary of the Invention

[0003] To solve the above technical problems, embodiments of the present application are expected to provide a method, apparatus, device, storage medium, and computer program product for information determination and model training, which solves the problems of low efficiency and poor accuracy in determining the prompt in related technologies.

[0004] To achieve the above object, the technical solution of the present application is implemented as follows:

[0005] An information determination method, the method includes:

[0006] Obtain background information corresponding to the information to be processed and first prompt information of a user for the information to be processed;

[0007] Process the first prompt information and the background information by using a first target large language model to obtain multiple second prompt information corresponding to the information to be processed;

[0008] Input the multiple second prompt information into a second target large language model to obtain multiple response information, and determine target prompt information corresponding to the information to be processed based on the multiple response information.

[0009] In the above solution, determining the target prompt information corresponding to the information to be processed based on the multiple response information includes:

[0010] Determine candidate prompt information based on the first prompt information and the multiple second prompt information;

[0011] Determine the target prompt information based on the multiple response information, a target information evaluation model, and the candidate prompt information.

[0012] In the above solution, determining the candidate prompt information based on the first prompt information and the multiple second prompt information includes:

[0013] Determining the similarity between the first prompt information and each second prompt information;

[0014] Determining the candidate prompt information from the multiple second prompt information based on the multiple similarities.

[0015] In the above solution, determining the candidate prompt information from the multiple second prompt information based on the multiple similarities includes:

[0016] Determining the candidate prompt information from the multiple second prompt information in the target database based on the multiple similarities.

[0017] In the above solution, determining the target prompt information based on the multiple response information, the target information evaluation model, and the candidate prompt information includes:

[0018] Determining the response information to be evaluated corresponding to the candidate prompt information from the multiple response information;

[0019] Inputting the response information to be evaluated into the target information evaluation model to obtain an evaluation result, and determining the target prompt information from the candidate prompt information based on the evaluation result.

[0020] A model training method, the method includes:

[0021] Obtaining the sample background information corresponding to the sample information and the sample prompt information of the user for the sample information;

[0022] Training a first initial large language model based on the sample background information and the sample prompt information to obtain a first target large language model, to determine multiple second prompt words based on the first target large language model and the information to be processed, and to determine the target prompt information corresponding to the information to be processed based on the multiple second prompt words.

[0023] An information determination device, the device includes:

[0024] A first acquisition unit, configured to acquire the background information corresponding to the information to be processed and the first prompt information of the user for the information to be processed;

[0025] A processing unit, configured to process the first prompt information and the background information by using a first target large language model to obtain multiple second prompt information corresponding to the information to be processed;

[0026] A determination unit, configured to input the multiple second prompt messages into a second target large language model to obtain multiple response messages, and determine a target prompt message corresponding to the information to be processed based on the multiple response messages.

[0027] A model training device, the device includes:

[0028] A second acquisition unit, configured to acquire sample background information corresponding to sample information and a sample prompt message of a user for the sample information;

[0029] A training unit, configured to train a first initial large language model based on the sample background information and the sample prompt message to obtain a first target large language model, to determine multiple second prompt words based on the first target large language model and the information to be processed, and determine a target prompt message corresponding to the information to be processed based on the multiple second prompt words.

[0030] An electronic device, the device includes: a processor, a memory, and a communication bus;

[0031] The communication bus is configured to implement a communication connection between the processor and the memory;

[0032] The processor is configured to execute a program in the memory to implement the steps of the above information determination method or model training method.

[0033] A computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the above information determination method or model training method.

[0034] A computer program product, the computer program product includes a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0035] The information determination method, model training method, device, equipment, computer storage medium, and computer program product provided by the embodiments of the present application can obtain the background information corresponding to the information to be processed and the first prompt information of the user for the information to be processed, and use the first target large language model to process the first prompt information and the background information to obtain multiple second prompt information corresponding to the information to be processed. Then, the multiple second prompt information is input into the second target large language model to obtain multiple response information, and the target prompt information corresponding to the information to be processed is determined based on the multiple response information. In this way, multiple second prompt information corresponding to the information to be processed can be generated by the first target large language model, and the final target prompt information can be determined according to the multiple response information obtained after processing the multiple second prompt information by the second target large language model. That is, the target prompt information can be automatically generated by the first target large language model, rather than being manually generated as in the related art, which solves the problems of low determination efficiency and poor accuracy of the prompt words in the process of determining the prompt words in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic flowchart of an information determination method provided by an embodiment of the present application;

[0037] Figure 2 It is a schematic flowchart of a model training method provided by an embodiment of the present application;

[0038] Figure 3 It is a schematic flowchart of another information determination method provided by an embodiment of the present application;

[0039] Figure 4 It is a schematic flowchart of determining the target response information in an information determination method provided by an embodiment of the present application;

[0040] Figure 5 It is a schematic structural diagram of an information determination device provided by an embodiment of the present application;

[0041] Figure 6 It is a schematic structural diagram of a model training device provided by an embodiment of the present application;

[0042] Figure 7 It is a schematic structural diagram of an information determination device provided by an embodiment of the present application;

[0043] Figure 8 It is a schematic structural diagram of a model training device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application.

[0045] It should be noted that before further elaborating on the embodiments of the present application, the following explanations are provided for the nouns and terms involved in the embodiments of the present application:

[0046] 1) Large language model (hereinafter referred to as: LLM), which belongs to a pre-trained model and can be simply referred to as a large model. It is an artificial intelligence model composed of a neural network with hundreds of millions or billions of parameters. It is trained on a large amount of text data, learns complex patterns in the language data, and can perform a wide range of natural language understanding and generation tasks.

[0047] 2) Prompt: Functionally, it is also considered an "instruction". It is a form or template designed to enable the large model to perform well in downstream tasks. As a clue or hint, it is added as additional text to the large model input to help the large model "recall" what it "learned" during training, so that it can better "understand" the user's question and give an answer that better matches the human intention.

[0048] 3) Prompt Engineering: It refers to a method of guiding the large model to generate answers with target effects by designing and developing prompts without updating the model parameters.

[0049] It should be understood that the "embodiments of the present application" or "the foregoing embodiments" mentioned throughout the specification mean that specific features, structures, or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, the "in the embodiments of the present application" or "in the foregoing embodiments" that appear throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics can be combined in one or more embodiments in any suitable manner. In various embodiments of the present application, the magnitudes of the serial numbers of the above processes do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments.

[0050] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] The embodiments of the present application provide an information determination method. Referring to Figure 1 as shown, the method includes the following steps:

[0052] Step 101, obtain the background information corresponding to the information to be processed and the first prompt information of the user for the information to be processed.

[0053] In the embodiments of the present application, the information to be processed may refer to the information that is waiting to be processed, that is, the information that needs to obtain a response. Exemplarily, in an intelligent question-answering system, the information to be processed may refer to the information input by the user and waiting for the system to reply. Then, the information to be processed may be "judging an alarm event", or "generating a piece of text", or "generating a piece of code for detecting the startup time of a device"; correspondingly, the background information of the information to be processed may include the specific content of the information to be processed and the information related to the event corresponding to the information to be processed, etc. For example, if the information to be processed is "judging an alarm event", the background information of the information to be processed may include the time and location when the alarm event occurred and the relevant associated information of the alarm event.

[0054] In the embodiments of the present application, the first prompt information may also be referred to as the first prompt word, and may be determined by the user manually according to the background information corresponding to the information to be processed and the prompt word design methodology.

[0055] Step 102: Process the first prompt information and the background information using a first target large language model to obtain multiple second prompt information corresponding to the information to be processed.

[0056] In the embodiments of the present application, the first target large language model (hereinafter referred to as: the first LLM) may be the Qianwen large model, the Spark large model, the Wenxin Yiyan large model, or other large models, and specific limitations are not made here; the second prompt information may also be referred to as the second prompt word. Specifically, the first prompt information and the background information may be used as input parameters, that is, the first prompt information and the background information are input into the first LLM. Then, the first LLM will process the input first prompt information and background information to output multiple second prompt information corresponding to the information to be processed, that is, multiple second prompt words.

[0057] It should be noted that the first target large language model may be obtained by training a first initial large language model according to the background information of the sample information and the sample prompt information of the user for the sample information.

[0058] Step 103: Input the multiple second prompt information into a second target large language model to obtain multiple response information, and determine the target prompt information corresponding to the information to be processed based on the multiple response information.

[0059] In the embodiments of the present application, the second target large language model (hereinafter referred to as: the second LLM) may also be the Qianwen large model, the Spark large model, and the Wenxin Yiyan large model; the target prompt information may refer to the prompt word that is finally input into the second LLM to obtain the corresponding response information.

[0060] In an implementable manner, each response message can be directly scored, and the target prompt message can be filtered out from multiple second prompt messages according to the score of each response message.

[0061] In another implementable manner, candidate prompt messages can be first filtered out from multiple second prompt messages according to the first prompt message, and then the final target prompt message, that is, the target prompt word (target Prompt), can be determined according to the second target large language model and the candidate prompt messages.

[0062] It should be noted that the first target large language model and the second target large language model can be exactly the same model, that is, they can both be the Qianwen large model; correspondingly, the first target large language model and the second target large language model can also be different models, that is, the first target large language model can be the Qianwen large model, and the second target large language model can be the Spark large model. Among them, which specific model the first target large language model and the second target large language model are is set according to actual needs.

[0063] The information determination method provided by the embodiments of the present application can generate multiple second prompt messages corresponding to the information to be processed through the first target large language model, and then determine the final target prompt message according to the multiple response messages obtained after processing the multiple second prompt messages by the second target large language model, that is, the target prompt message can be automatically generated through the first target large language model, rather than being manually generated as in the related art, solving the problems of low determination efficiency and poor accuracy of the prompt word in the process of determining the prompt word in the related art.

[0064] Based on the foregoing embodiments, an embodiment of the present application provides a model training method, as shown in Figure 2 The method may include the following steps:

[0065] Step 201, obtain the sample background information corresponding to the sample information and the sample prompt information of the user for the sample information.

[0066] In the embodiments of the present application, the sample information may refer to the information determined for model training that requires a response. For example: the sample information may be "generate an article about the weather".

[0067] In the embodiments of the present application, the sample prompt information may also be referred to as the sample prompt word, which is considered by the user to be determined according to the background information corresponding to the sample information and the prompt word design theory. For example, the sample prompt word may be "please generate a 200-word article about the weather according to the xx format".

[0068] Step 202: Based on the sample background information and sample prompt information, train the first initial large language model to obtain the first target large language model, so as to determine multiple second prompt words based on the first target large language model and the information to be processed, and determine the target prompt information corresponding to the information to be processed based on the multiple second prompt words.

[0069] In the embodiment of the present application, the sample background information and sample prompt information corresponding to the sample information can be used as input parameters, that is, the sample background information and sample prompt information can be input into the first initial large language model to implement the training of the first initial large language model, so as to obtain the first target large language model. Among them, the first initial large language model can be a large language model that is initially created and has not been trained yet.

[0070] The model training method provided by the embodiment of the present application can perform model training on the first initial large language model according to the sample background information and sample prompt information to obtain the first target large language model, and then automatically generate the target prompt information corresponding to the information to be processed through the first target large language model, rather than generating prompt information manually as in the related art, which solves the problems of low determination efficiency and poor accuracy of prompt words in the process of determining prompt words in the related art.

[0071] Based on the foregoing embodiments, the embodiment of the present application provides an information determination method. Referring to Figure 3 as shown, this method can be applied to an information determination device. Among them, the electronic device includes an information determination device, and this method includes the following steps:

[0072] Step 301: The information determination device obtains the background information corresponding to the information to be processed and the first prompt information of the user for the information to be processed.

[0073] In the embodiment of the present application, the background information and the first prompt information, that is, the first prompt word, can be obtained from the target database. It should be noted that, preferably, there is only one first prompt word. In another implementable manner, there can also be multiple first prompt words, which are not specifically limited here.

[0074] For example, if the information to be processed is to judge an alarm event, the first prompt information can be "Please judge whether there is a security vulnerability at xx according to the existing xx information?"

[0075] Step 302: The information determination device processes the first prompt information and the background information by using the first target large language model to obtain multiple second prompt words corresponding to the information to be processed.

[0076] In the embodiment of the present application, as Figure 4As shown, the second prompt message, that is, the second prompt word, may include the background information corresponding to the information to be processed for which the user wants to obtain feedback, the evidence information corresponding to the information to be processed, and the instruction information for the LLM. Among them, the instruction information is used to instruct the LLM to analyze and answer the information to be processed.

[0077] Exemplarily: If the information to be processed is "Analyze the alarm event", the background information of the information to be processed may include the time and location of the alarm event and the evidence information related to the alarm event. The first prompt message may be "Please judge whether there is a security vulnerability at xx based on this information?" Then, after inputting the background information and the first prompt message into the first LLM, the second prompt message obtained may be "You now know xx information. Please briefly introduce the level, components, attack type, etc. of the current vulnerability in the following format."

[0078] In an implementable manner, as Figure 4 shown, there are two ways to obtain multiple second prompt messages: offline or online. Among them, the offline method is usually applied to scenarios where the network of the device location is unstable or there are strict restrictions on the generation duration of the target prompt message. After generating the second prompt message in the offline method, it can be formed into a question set and stored in the target database. In this way, when generating the target prompt word subsequently, multiple second prompt words can be directly obtained from the target database.

[0079] It should be noted that after storing the question set including multiple second prompt words in the target database, it is necessary to regularly maintain the question set according to business requirements, that is, perform addition, deletion, or modification.

[0080] Correspondingly, the online method is usually applied to scenarios where the network of the device location is relatively stable and there are no strict restrictions on the generation duration of the target prompt message. The second prompt message generated by the online method does not need to be stored in the target database. Furthermore, the target prompt message can be determined directly based on the obtained multiple second prompt messages and the target large language model, thus saving the storage resources of the server.

[0081] Step 303, the information determination device inputs multiple second prompt messages into the second target large language model to obtain multiple response messages.

[0082] In the embodiment of the present application, if the information to be processed is "Determine the system vulnerability of the device" and the candidate prompt message is "You now know xx information. Please briefly introduce the level, components, attack type, etc. of the current system vulnerability in the following format.", then the response message needs to include the level, components, attack type, etc. of the alarm event.

[0083] Specifically, each second prompt message can be used as an input parameter, that is, multiple second prompt messages are input into the first target large language model, so that the first target large language model processes the information to be processed, and then multiple response messages for the information to be processed can be obtained.

[0084] It should be noted that one second prompt message can correspond to multiple response messages.

[0085] Step 304: The information determination device determines candidate prompt messages based on the first prompt message and multiple second prompt messages.

[0086] In an embodiment of the present application, the first prompt message can be matched with each second prompt message to obtain the similarity between the first prompt message and each second prompt message, and candidate prompt messages are determined from multiple second prompt messages according to the obtained multiple similarities.

[0087] In an embodiment of the present application, step 304 can be implemented through steps 304a to 304c.

[0088] Step 304a: The information determination device determines the similarity between the first prompt message and each second prompt message.

[0089] In a feasible manner, the Euclidean distance between the first prompt word and each second prompt word can be calculated, and the similarity between the first prompt word and each second prompt word is determined according to the obtained Euclidean distance.

[0090] In another feasible manner, the bag-of-words model or neural network model can also be used to process the first prompt word and multiple second prompt words to obtain multiple similarities. Specifically, the first prompt word and multiple second prompt words can be used as input parameters, that is, the first prompt word and multiple second prompt words are input into the bag-of-words model or neural network model. Then, the bag-of-words model or neural network model processes the input information to obtain the similarity between the first prompt word and multiple second prompt words.

[0091] In another feasible manner, the first prompt word and each second prompt word can be segmented respectively to obtain the word vectors corresponding to the first prompt word and each second prompt word, and multiple similarities are determined by calculating the cosine value between the word vector corresponding to the first prompt word and the word vector corresponding to each second prompt word.

[0092] It should be noted that after step 304a, step 304b or step 304c can be executed.

[0093] Step 304b: The information determination device determines candidate prompt messages from multiple second prompt messages based on multiple similarities.

[0094] In an embodiment of the present application, each similarity can be directly compared with a first threshold. If the similarity is greater than or equal to the first threshold, the second prompt information corresponding to the similarity is determined as candidate prompt information, and the second prompt information with a similarity less than the first threshold is directly discarded.

[0095] It should be noted that the first threshold can be set according to historical data and actual requirements.

[0096] Step 304c: The information determination device determines candidate prompt information from multiple second prompt information in the target database based on multiple similarities.

[0097] In an embodiment of the present application, if the present application is applied to a scenario where the network of the device location is poor or there are strict restrictions on the generation duration of the target prompt word, the question set including multiple second prompt information is stored in the target database. At this time, it is necessary to obtain candidate prompt information (i.e., candidate prompt words) from the question set in the target database according to multiple similarities, rather than directly obtaining candidate prompt information from the generated multiple second prompt information.

[0098] It should be noted that there can be one or more candidate prompt information.

[0099] Step 305: The information determination device determines the target prompt information based on multiple response information, the target information evaluation model, and the candidate prompt information.

[0100] In an embodiment of the present application, the target information evaluation model can refer to a model used to evaluate the accuracy of the response information corresponding to the information to be processed. In one implementable manner, the target information evaluation model can be an Extreme Gradient Boosting (XGBoost) model, or a target large language model, i.e., LLM. Of course, the target information evaluation model can also be other deep learning models, which are not specifically limited here.

[0101] It should be noted that if the target information evaluation model is a target large language model, it can be the same model as the first target large language model, or the same model as the second target large language model, or a model that is different from both the first target large language model and the second target large language model.

[0102] Specifically, the response information to be evaluated corresponding to the candidate prompt information (i.e., candidate prompt words) can be first determined from multiple response information, and the target prompt words can be determined from the candidate prompt words according to the target information evaluation model and the response information to be evaluated.

[0103] In an embodiment of the present application, step 305 can be implemented through steps 305a to 305b.

[0104] Step 305a: The information determination device determines the response information to be evaluated corresponding to the candidate prompt information from multiple response messages.

[0105] In an embodiment of the present application, the response information corresponding to each second prompt information can be screened to screen out the response information to be evaluated corresponding to the candidate prompt information therefrom.

[0106] Wherein, a candidate prompt information can correspond to one evaluation response information or multiple response information to be evaluated, and no specific limitation is made here.

[0107] Step 305b: The information determination device inputs the response information to be evaluated into the target information evaluation model to obtain an evaluation result, and determines the target prompt information from the candidate prompt information based on the evaluation result.

[0108] In an embodiment of the present application, the target information evaluation model can be used to process each response information to be evaluated first to obtain the score (i.e., the evaluation result) corresponding to each response information to be evaluated. Then, the target prompt word can be determined from multiple candidate prompt words according to the score corresponding to each response information to be evaluated.

[0109] In an embodiment of the present application, step 305b can be implemented in the following manner.

[0110] a1: The information determination device uses the target information evaluation model to process the response information to be evaluated based on the target evaluation strategy to obtain the target score corresponding to the response information to be evaluated.

[0111] Wherein, the target score represents the accuracy of the first response information.

[0112] In an embodiment of the present application, the target evaluation strategy can refer to the strategy for evaluating the accuracy of the response information to be evaluated. For example, the target evaluation strategy can be a scoring rule for the response information, that is, how to score the response information is specified in the target evaluation strategy; the target score can refer to the score corresponding to the response information to be evaluated, and the higher the score, the higher the accuracy of the response information to be evaluated.

[0113] In a feasible manner, the target score can be represented by a numerical value. For example, the target score can be represented by 1 to 10. The larger the numerical value, the higher the score, and the higher the accuracy of the corresponding response information to be evaluated. It should be noted that the target score can also be represented in other forms (letters, etc.), and no specific limitation is made here.

[0114] Exemplarily, if the information to be processed is "judging the alarm event", the target evaluation strategy can be as shown in Table 1 below:

[0115] Response information to be evaluated Score Level information with alarm events 2 points Component information with alarm events 2 points Attack type with alarm events 2 points Level and component information with alarm events 5 points … Level, component and attack type with alarm events 10 points

[0116] Table 1

[0117] In the embodiments of the present application, multiple response information to be evaluated can be used as input parameters, that is, each response information to be evaluated is input into the target information evaluation model, and the target evaluation strategy is used to process each response information to be evaluated to obtain the score corresponding to each response information to be evaluated.

[0118] It should be noted that the target evaluation strategy can be preset according to actual needs, and different information to be processed corresponds to different evaluation strategies.

[0119] a2. The information determination device determines the target prompt information from the candidate prompt information based on the target score.

[0120] In the embodiments of the present application, there is only one target prompt information, that is, the target prompt word. Specifically, after obtaining the target score of each response information to be evaluated, the multiple response information to be evaluated can be sorted in descending order according to the target score. Then, the candidate prompt information corresponding to the response information to be evaluated ranked first is determined as the target prompt information, that is, the target prompt word.

[0121] It should be noted that the target information evaluation model can be trained in the following way:

[0122] a21. Obtain the target sample prompt information corresponding to the sample information.

[0123] In the embodiments of the present application, the target sample prompt information can be the determined prompt word for model training. Specifically, the background information of the sample information and the sample prompt information of the user for the sample information can be obtained first, and then the first target large language model is used to process the background information of the sample information and the sample prompt information of the user for the sample information to obtain the target sample prompt information corresponding to the sample information.

[0124] It should be noted that there are multiple target sample prompt information.

[0125] a22. Input the target sample prompt information into the second target large language model to obtain the response information corresponding to the sample information.

[0126] In the embodiments of the present application, the target sample prompt information can be used as an input parameter, that is, the target sample prompt information is input into the second target large language model. After that, the second target large language model processes the sample information according to each target sample prompt information to obtain multiple response information corresponding to the sample information.

[0127] a23. Train the initial information evaluation model based on the response information corresponding to the sample information and the sample evaluation strategy to obtain the target information evaluation model.

[0128] Among them, the sample evaluation strategy is used to evaluate the accuracy of the response information corresponding to the sample information.

[0129] In the embodiments of the present application, the response information corresponding to the sample information can be used as an input parameter of the initial information evaluation model, that is, the response information corresponding to the sample information can be input into the initial information evaluation model. After that, the initial information evaluation model processes the response information according to the sample evaluation strategy to implement the model training of the initial information evaluation model, so as to obtain the target information evaluation model.

[0130] It should be noted that the initial information evaluation model can be an initially created model that has not been trained and is used to evaluate the accuracy of the response information corresponding to the sample information.

[0131] In other embodiments of the present application, step 306 can be executed after step 305.

[0132] Step 306: The information determination device inputs the target prompt information into the second target large language model to obtain the target response information for the information to be processed.

[0133] In the embodiments of the present application, after determining the target prompt information corresponding to the information to be processed, that is, the target prompt word, the target prompt word can be used as an input parameter, that is, the target prompt word is input into the second target large language model. After that, the second target large language model uses the target prompt word to process the information to be processed to obtain the optimal response information (i.e., the target response information) for the information to be processed.

[0134] In the embodiments of the present application, it can be realized that the design of the prompt word for each user's question by humans before is changed to let the target large language model think, speculate and generate appropriate prompt words by itself. That is to say, the large language model can generate the prompt word instead of human design and input it into the large language model to obtain the final high-quality answer. For example, in a dialogue system, compared with letting the large language model answer the user's question freely, the scheme of generating prompt words in the present application can generate a high-quality answer that is more relevant to the user's intention and can better reflect the existing capabilities of the large language model.

[0135] The information determination method provided by the embodiments of the present application can generate multiple second prompt messages corresponding to the information to be processed through a first target large language model, and then determine the final target prompt message based on the multiple response messages obtained by processing the multiple second prompt messages through a second target large language model. That is, the target prompt message can be automatically generated through the first target large language model, rather than being manually generated as in the related art, solving the problems of low determination efficiency and poor accuracy of the prompt words in the process of determining the prompt words in the related art.

[0136] Based on the foregoing embodiments, the embodiments of the present application provide an information determination device, which can be applied to Figure 1 and 3 the information determination method provided in the corresponding embodiments, referring to Figure 5 As shown, the information determination device 4 may include: a first acquisition unit 41, a processing unit 42, and a determination unit 43, where:

[0137] The first acquisition unit 41 is configured to acquire the background information corresponding to the information to be processed and the first prompt information of the user for the information to be processed;

[0138] The processing unit 42 is configured to process the first prompt information and the background information by using a first target large language model to obtain multiple second prompt messages corresponding to the information to be processed;

[0139] The determination unit 43 is configured to input the multiple second prompt messages into a second target large language model to obtain multiple response messages, and determine the target prompt message corresponding to the information to be processed based on the multiple response messages.

[0140] In other embodiments of the present application, the determination unit 43 is further configured to perform the following steps:

[0141] Determine candidate prompt messages based on the first prompt information and the multiple second prompt messages;

[0142] Determine the target prompt message based on the multiple response messages, the target information evaluation model, and the candidate prompt messages.

[0143] In other embodiments of the present application, the determination unit 43 is further configured to perform the following steps:

[0144] Determine the similarity between the first prompt information and each second prompt information;

[0145] Determine candidate prompt messages from the multiple second prompt messages based on the multiple similarities.

[0146] In other embodiments of the present application, the determination unit 43 is further configured to perform the following steps:

[0147] Determine candidate prompt information from multiple second prompt messages in the target database based on multiple similarities.

[0148] In other embodiments of the present application, the determining unit 43 is further configured to perform the following steps:

[0149] Determine the response information to be evaluated corresponding to the candidate prompt information from multiple response messages;

[0150] Input the response information to be evaluated into the target information evaluation model to obtain an evaluation result, and determine the target prompt information from the candidate prompt information based on the evaluation result.

[0151] It should be noted that for the specific implementation process of the steps executed by each unit in the embodiments of the present application, reference can be made to Figure 1 and 3 the implementation process in the information determination method provided in the corresponding embodiments, which will not be elaborated here.

[0152] The information determination device provided in the embodiments of the present application can generate multiple second prompt messages corresponding to the information to be processed through the first target large language model, and then determine the final target prompt information based on the multiple response messages obtained after processing the multiple second prompt messages by the second target large language model, that is, the target prompt information can be automatically generated through the first target large language model, solving the problems of low determination efficiency and poor accuracy of the prompt words in the related art during the process of determining the prompt words.

[0153] Based on the foregoing embodiments, the embodiments of the present application provide a model training device, and the model training device 5 can be applied to Figure 2 the model training method provided in the corresponding embodiments, referring to Figure 6 as shown, the model training device 5 may include: a second acquisition unit 51 and a training unit 52, where:

[0154] The second acquisition unit 51 is configured to acquire the sample background information corresponding to the sample information and the sample prompt information of the user for the sample information;

[0155] The training unit 52 is configured to train the first initial large language model based on the sample background information and the sample prompt information to obtain a first target large language model, so as to determine multiple second prompt words based on the first target large language model and the information to be processed, and determine the target prompt information corresponding to the information to be processed based on the multiple second prompt words.

[0156] It should be noted that for the specific implementation process of the steps executed by each unit in the embodiments of the present application, reference can be made to Figure 2 the implementation process in the model training method provided in the corresponding embodiments, which will not be elaborated here.

[0157] The model training device provided by the embodiments of the present application can train a first initial large language model according to sample background information and sample prompt information to obtain a first target large language model, and then automatically generate target prompt information corresponding to the information to be processed through the first target large language model, rather than generating prompt information manually as in the related art, which solves the problems of low determination efficiency and poor accuracy of prompt words in the process of determining prompt words in the related art.

[0158] Based on the foregoing embodiments, an embodiment of the present application provides an electronic device, which may include a processor, a memory, and a communication bus, and the electronic device may include an information determination device 6. The processor may include a first processor 61, the memory may include a first memory 62, and the communication bus may include a first communication bus 63. Refer to Figure 7 As shown, the information determination device 6 may be applied to Figure 1 and 3 the information determination method provided by the corresponding embodiments, where:

[0159] The first communication bus 63 is used to implement the communication connection between the first processor 61 and the first memory 62;

[0160] The first processor 61 is configured to execute the information determination program in the first memory 62 to implement the following steps:

[0161] Obtain the background information corresponding to the information to be processed and the first prompt information of the user for the information to be processed;

[0162] Use the target large language model to process the first prompt information and the background information to obtain multiple second prompt information corresponding to the information to be processed;

[0163] Input the multiple second prompt information into the second target large language model to obtain multiple response information, and determine the target prompt information corresponding to the information to be processed based on the multiple response information.

[0164] In other embodiments of the present application, the first processor 61 is configured to execute the information determination program in the first memory 62 to determine the target prompt information corresponding to the information to be processed based on the multiple response information, so as to implement the following steps:

[0165] Based on the first prompt information and the multiple second prompt information, determine candidate prompt information;

[0166] Based on the multiple response information, the target information evaluation model, and the candidate prompt information, determine the target prompt information.

[0167] In other embodiments of the present application, the first processor 61 is configured to execute the information determination program in the first memory 62 to determine candidate prompt information based on the first prompt information and multiple second prompt information, so as to implement the following steps:

[0168] Determine the similarity between the first prompt information and each second prompt information;

[0169] Based on multiple similarities, determine candidate prompt information from multiple second prompt information.

[0170] In other embodiments of the present application, the first processor 61 is configured to execute the information determination program in the first memory 62 to determine candidate prompt information from multiple second prompt information based on multiple similarities, so as to implement the following steps:

[0171] Based on multiple similarities, determine candidate prompt information from multiple second prompt information in the target database.

[0172] In other embodiments of the present application, the first processor 61 is configured to execute the information determination program in the first memory 62 to determine target prompt information based on multiple response information, a target information evaluation model, and candidate prompt information, so as to implement the following steps:

[0173] Determine the response information to be evaluated corresponding to the candidate prompt information from multiple response information;

[0174] Input the response information to be evaluated into the target information evaluation model to obtain an evaluation result, and determine the target prompt information from the candidate prompt information based on the evaluation result.

[0175] It should be noted that the specific description of the steps executed by the first processor 61 can be referred to Figure 1 and 3 In the information determination method provided in the corresponding embodiments, it will not be elaborated here.

[0176] The information determination device provided in the embodiments of the present application can generate multiple second prompt information corresponding to the information to be processed through the first target large language model, and then determine the final target prompt information based on the multiple response information obtained after processing the multiple second prompt information through the second target large language model, that is, it can automatically generate the target prompt information through the first target large language model, rather than manually generating the prompt information as in the related art, solving the problems of low determination efficiency and poor accuracy of the prompt words in the process of determining the prompt words in the related art.

[0177] Based on the foregoing embodiments, an embodiment of the present application provides another electronic device. The electronic device may include a processor, a memory, and a communication bus, and the electronic device may further include a model training device 7. The processor may further include a second processor 71, the memory may further include a second memory 72, and the communication bus may further include a second communication bus 73. Referring to Figure 8 as shown, the model training device 8 may be applied to Figure 2 the model training method provided in the corresponding embodiment, where:

[0178] The second communication bus 73 is used to implement a communication connection between the second processor 71 and the second memory 72;

[0179] The second processor 71 is used to execute the model training program in the second memory 72 to implement the following steps:

[0180] Obtain the sample background information corresponding to the sample information and the sample prompt information of the user for the sample information;

[0181] Based on the sample background information and the sample prompt information, train the first initial large language model to obtain a first target large language model, and based on the first target large language model and the information to be processed, determine multiple second prompt words, and based on the multiple second prompt words, determine the target prompt information corresponding to the information to be processed.

[0182] It should be noted that the specific description of the steps executed by the second processor 71 may be referred to Figure 2 in the model training method provided in the corresponding embodiment, which will not be elaborated here.

[0183] The model training device provided by the embodiment of the present application can train the first initial large language model according to the sample background information and the sample prompt information to obtain a first target large language model, and then automatically generate the target prompt information corresponding to the information to be processed through the first target large language model, rather than generating the prompt information manually as in the related art, solving the problems of low determination efficiency and poor accuracy of the prompt words existing in the process of determining the prompt words in the related art.

[0184] Based on the foregoing embodiments, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement Figure 1 and 3 the information determination method provided in the corresponding embodiment, or Figure 2 the steps of the model training method provided in the corresponding embodiment.

[0185] Based on the foregoing embodiments, an embodiment of the present application provides a computer program product, which includes a computer program that, when executed by a processor, implements Figure 1 and 3 the information determination method provided by the corresponding embodiment, or Figure 2 the steps of the model training method provided by the corresponding embodiment.

[0186] It should be noted that the above computer-readable storage medium may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it may also be various electronic devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0187] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or device including that element.

[0188] The serial numbers of the embodiments of the present application above are only for description and do not represent the superiority or inferiority of the embodiments.

[0189] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described example methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0190] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, as well as the combination of flows and / or blocks in the flowchart and / or block diagram. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one or more of the processes and / or Figure 1 blocks or multiple blocks.

[0191] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one or more of the processes and / or Figure 1 blocks or multiple blocks.

[0192] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more of the processes and / or Figure 1 blocks or multiple blocks.

Claims

1. An information determination method, characterized in that, The method includes: Obtaining background information corresponding to the information to be processed and first prompt information of the user for the information to be processed; Processing the first prompt information and the background information by using a first target large language model to obtain multiple second prompt information corresponding to the information to be processed; Inputting the multiple second prompt information into a second target large language model to obtain multiple response information, and determining target prompt information corresponding to the information to be processed based on the multiple response information.

2. The method according to claim 1, characterized in that, Determining target prompt information corresponding to the information to be processed based on the multiple response information includes: Determining candidate prompt information based on the first prompt information and the multiple second prompt information; Determining the target prompt information based on the multiple response information, a target information evaluation model, and the candidate prompt information.

3. The method according to claim 2, characterized in that, The determining candidate prompt information based on the first prompt information and the multiple second prompt information includes: Determining the similarity between the first prompt information and each second prompt information; Determining the candidate prompt information from the multiple second prompt information based on multiple such similarities.

4. The method according to claim 3, wherein The determining the candidate prompt information from the multiple second prompt information based on multiple such similarities includes: Determining the candidate prompt information from the multiple second prompt information in a target database based on multiple such similarities.

5. The method according to claim 2, wherein The determining the target prompt information based on the multiple response information, a target information evaluation model, and the candidate prompt information includes: Determining response information to be evaluated corresponding to the candidate prompt information from the multiple response information; Inputting the response information to be evaluated into the target information evaluation model to obtain an evaluation result, and determining the target prompt information from the candidate prompt information based on the evaluation result.

6. A model training method, characterized in that, The method includes: Obtaining sample background information corresponding to sample information and sample prompt information of the user for the sample information; Training a first initial large language model based on the sample background information and the sample prompt information to obtain a first target large language model, so as to determine multiple second prompt words based on the first target large language model and the information to be processed, and determining target prompt information corresponding to the information to be processed based on the multiple second prompt words.

7. An information determination device, characterized in that, The apparatus includes: A first obtaining unit, configured to obtain background information corresponding to the information to be processed and first prompt information of the user for the information to be processed; A processing unit, configured to process the first prompt information and the background information by using a first target large language model to obtain multiple second prompt information corresponding to the information to be processed; A determining unit, configured to input the multiple second prompt information into a second target large language model to obtain multiple response information, and determine target prompt information corresponding to the information to be processed based on the multiple response information.

8. An electronic device, characterized in that, The device includes: a processor, a memory, and a communication bus; The communication bus is used to implement a communication connection between the processor and the memory; The processor is configured to execute a program in the memory to implement the information determination method according to any one of claims 1 to 5, or the steps of the model training method according to claim 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the information determination method according to any one of claims 1 to 5, or the steps of the model training method according to claim 6.

10. A computer program product, the computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method according to any one of claims 1 to 5 or 6.

Images