Problem processing method and device, equipment, storage medium and program product

Abstract

The invention provides a problem processing method and device, equipment, a storage medium and a program product, and relates to the technical field of natural language processing. The method comprises the steps of obtaining a first instruction, wherein the first instruction is an instruction for a first problem; inputting the first instruction into the large language model to obtain lexical elements output by the illusion recognition block of the large language model; under the condition that the lexical elements output by the illusion recognition block indicate that the first question has no answer, changing the lexical elements output by the last layer of the large language model into the lexical elements output by the illusion recognition block, and outputting the lexical elements as the answer of the first question; and under the condition that the lexical elements output by the illusion recognition block indicate that the first question has the answer, outputting the lexical elements output by the last layer of the large language model as the answer of the first question. According to the method, the illusion problem of the large language model is solved through the inherent capability of the large language model by activating the block capability of whether the large language model can answer the problem or not, and the problem solving accuracy of the large language model is improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a problem processing method, device, equipment, storage medium and program product. Background Art

[0002] At present, the hallucination phenomenon of large language models is quite serious. Hallucination refers to the fact that the results generated by large language models based on input instructions contain unfounded or wrong content, which is simply fabricated and serious nonsense. The hallucination phenomenon of large language models will reduce the accuracy of large language models in solving problems. Summary of the invention

[0003] The object of the present invention is to provide a problem processing method, apparatus, device, storage medium and program product to solve the problem that the accuracy of large language models in solving problems is low due to the hallucination phenomenon of large language models.

[0004] In order to achieve the above object, in a first aspect, the present invention provides a problem solving method, the method comprising:

[0005] Obtaining a first instruction, where the first instruction is an instruction for a first question;

[0006] Inputting the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is used to identify whether a result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language model;

[0007] When the word-gram output by the hallucination recognition block indicates that there is no answer to the first question, the word-gram output by the last layer of the large language model is changed to the word-gram output by the hallucination recognition block, and is output as the answer to the first question;

[0008] When the word-gram output by the hallucination recognition block indicates that the first question has an answer, the word-gram output by the last layer of the large language model is output as the answer to the first question.

[0009] In some embodiments, inputting the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model includes:

[0010] Inputting the first instruction into the large language model to obtain a latent vector of the hallucination recognition block of the large language model when the last layer of the large language model outputs the first word unit;

[0011] The latent vector of the hallucination recognition block is decoded to obtain the word unit output by the hallucination recognition block.

[0012] In some embodiments, the method further comprises:

[0013] When the word unit output by the hallucination recognition block indicates that the first question has an answer, obtaining other word units except the first word unit output by the last layer of the large language model;

[0014] Outputting the word unit output by the last layer of the large language model as the answer to the first question includes:

[0015] The first word-gram and the other word-grams output by the last layer of the large language model are output as answers to the first question.

[0016] In some embodiments, the method further comprises:

[0017] Obtaining a set of hallucination instructions, wherein the questions corresponding to the instructions in the hallucination instruction set have no answers;

[0018] For each instruction of the hallucination instruction set, input the instruction into the large language model to obtain the word unit and confidence corresponding to the target word unit output at each layer of the large language model, wherein the target word unit is the first word unit outputted by the last layer of the large language model;

[0019] The hallucination recognition block of the large language model is obtained by using the word unit and confidence level of the target word unit corresponding to each instruction and outputted at each layer of the large language model.

[0020] In some embodiments, the method of obtaining a hallucination recognition block of the large language model by using the word unit and confidence level of the target word unit corresponding to each instruction and outputted at each layer of the large language model includes:

[0021] For each of the instructions, based on the word-grams outputted at each layer of the large language model corresponding to the target word-gram, a first block corresponding to a first word-gram is obtained, wherein the first word-gram includes at least one word-gram indicating that there is no answer to the question corresponding to the instruction;

[0022] Obtaining the confidence corresponding to the first block based on the confidence corresponding to the target word output at each layer of the large language model;

[0023] A first block whose confidence level is greater than a preset threshold is selected from the obtained multiple first blocks as a hallucination recognition block of the large language model.

[0024] In a second aspect, the present invention further provides a problem handling device, the device comprising:

[0025] A first acquisition module, used to acquire a first instruction, where the first instruction is an instruction for a first question;

[0026] a second acquisition module, configured to input the first instruction into a large language model, and obtain a word unit output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is configured to identify whether a result generated by the large language model based on the input instruction contains content representing that there is no answer, and the hallucination recognition block includes at least one layer of the large language model;

[0027] A first processing module is configured to change the word-gram output by the last layer of the large language model to the word-gram output by the hallucination recognition block, and output the word-gram as the answer to the first question when the word-gram output by the hallucination recognition block indicates that the first question has no answer;

[0028] The second processing module is used to output the word-gram output by the last layer of the large language model as the answer to the first question when the word-gram output by the hallucination recognition block indicates that the first question has an answer.

[0029] In a third aspect, the present invention further provides a problem handling device, comprising a processor and a transceiver, wherein the transceiver receives and sends data under the control of the processor, and the processor is used to perform the following operations:

[0030] Obtaining a first instruction, where the first instruction is an instruction for a first question;

[0031] Inputting the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is used to identify whether a result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language model;

[0032] When the word-gram output by the hallucination recognition block indicates that there is no answer to the first question, the word-gram output by the last layer of the large language model is changed to the word-gram output by the hallucination recognition block, and is output as the answer to the first question;

[0033] When the word-gram output by the hallucination recognition block indicates that the first question has an answer, the word-gram output by the last layer of the large language model is output as the answer to the first question.

[0034] In a fourth aspect, the present invention further provides a problem handling device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; when the processor executes the program, the problem handling method as described in the first aspect above is implemented.

[0035] In a fifth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the problem handling method as described in the first aspect above.

[0036] In a sixth aspect, the present invention further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps in the problem handling method as described in the first aspect above.

[0037] The above technical solution of the present invention has at least the following beneficial effects:

[0038] In an embodiment of the present invention, a first instruction is obtained, where the first instruction is an instruction for a first question; then, the first instruction is input into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, where the hallucination recognition block is used to identify whether the result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language module; finally, when the word unit output by the hallucination recognition block indicates that there is no answer to the first question, the word unit output by the last layer of the large language model is changed to the word unit output by the hallucination recognition block, and is output as the answer to the first question; when the word unit output by the hallucination recognition block indicates that there is an answer to the first question, the word unit output by the last layer of the large language model is output as the answer to the first question. In this way, by activating the ability of the large language model to answer the question (i.e., the hallucination recognition block), the hallucination problem of the large language model is solved with the inherent ability of the large language model, thereby improving the accuracy of the large language model in solving the problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram showing a flow chart of a problem handling method provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram showing a module of a problem handling device provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram showing the hardware structure of a problem handling device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0043] When the command prompt is input to the large language model, the large language model will convert the input into a vector, and the vector of the non-output layer becomes the latent vector. Under normal circumstances, the output is the word token output after decoding the vector of the last layer of the large language model, while the output of the large language model is the result of the last layer after the latent vector is continuously changed. The disadvantage is that the capabilities or functions of each layer of the large language model are different, and the results of the last layer may be ambiguous in other layers, that is, the results of the large language model in other layers may be "sorry", "excuse me" and other fields, and the implicit meaning is that this question cannot be answered. At the last layer, the large language model pays too much attention to the output of the result and ignores the information injected into the latent vector by other layers, which makes the large language model unable to coordinate the use of the hallucination recognition capabilities of different layers, and ultimately reduces the accuracy of the large language model in solving problems.

[0044] In order to solve the above technical problems, a problem processing method, device, equipment, storage medium and program product are provided. The method and device are based on the same application concept. Since the method and device solve the problem in a similar principle, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0045] like Figure 1 FIG. 1 is a flow chart of a problem handling method provided by an embodiment of the present invention, and the method may include:

[0046] Step 101: Obtain a first instruction, where the first instruction is an instruction for a first question.

[0047] Step 102: input the first instruction into a large language model to obtain word units output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is used to identify whether a result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language model.

[0048] It should be noted that the hallucination recognition block is a block in a pre-acquired large language model. For specific implementation, please refer to the subsequent embodiments.

[0049] Step 103, when the word-gram output by the hallucination recognition block indicates that there is no answer to the first question, the word-gram output by the last layer of the large language model is changed to the word-gram output by the hallucination recognition block and output as the answer to the first question.

[0050] For example, the instruction prompt for the first question is: "Please answer the following question. \nQuestion: List the five cities under Weihai?" It is known that the hallucination recognition block of the large language model is the 28th to 33rd layer of the large language model. The instruction is input into the large language model, and the output token of the hallucination recognition block before the output of the first token is "sorry", that is, the word unit output by the hallucination recognition block indicates that there is no answer to the first question, that is, the hallucination recognition block recognizes that the first question cannot be answered; then the word unit output by the last layer of the large language model, in this example, the first word unit token is changed to the word unit output by the hallucination recognition block, that is, sorry, and finally the "sorry" token is output as the answer to the first question. In this way, the problem of sealing the ability of the low-level layer due to the strong answer attribute of the high-level layer can be solved, that is, the hallucination problem of the large language model is avoided, and the accuracy of the large language model in solving problems is improved.

[0051] Step 104: When the word-gram output by the hallucination recognition block indicates that the first question has an answer, the word-gram output by the last layer of the large language model is output as the answer to the first question.

[0052] Here, the word unit output by the hallucination recognition block indicates that the first question has an answer, that is, there are no semantic tokens such as "sorry" and "excuse me" in the token output by the hallucination recognition block, which means that the hallucination recognition block recognizes that the first question can be answered. Finally, the word unit output by the last layer of the large language model is output as the answer to the first question.

[0053] As mentioned above, the existing large language model has the problem that the strong response attribute of the high-level layer seals the ability of the low-level layer. The following example illustrates this.

[0054] Example 1

[0055] The prompt is "Please answer the following question. Question: List the five cities under Weihai?" The output of the large language model is "The five cities under Weihai are Rongcheng, Wendeng, Rushan, Huancui, and Gaoqu". Because Weihai only has Huancui District, Wendeng District, Rongcheng City, and Rushan City under the jurisdiction of Weihai City, that is, 2 districts and 2 cities, and there is no fifth city, the fifth city "Gaoqu" output by the large language model is an illusion question. In order to meet the five requirements in the user prompt, the large language model fabricated the fifth one, but the researchers found that the output token of the 28th to 33rd layers before the output of the first token is "Sorry", as shown in Table 1 below, indicating that from the 28th to 33rd layers, the large language model recognized that the question could not be answered. After the 34th layer, the large language model forcibly answered the user's question, resulting in the block ability (illusion recognition ability) of the large language model being sealed. The method of the present invention can just solve this problem.

[0056] Table 1

[0057]

[0058]

[0059] In an optional embodiment, the above step 102, inputting the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, includes:

[0060] Step 1021: input the first instruction into the large language model to obtain the latent vector of the hallucination recognition block of the large language model when the last layer of the large language model outputs the first word.

[0061] That is, after the first instruction is input into the large language model, the large language model vectorizes the first instruction into a latent vector, and when the last layer of the large language model outputs the first word unit, the latent vector of the hallucination recognition block of the large language model is obtained.

[0062] Step 1022: decode the latent vector of the hallucination recognition block to obtain the word unit output by the hallucination recognition block.

[0063] Specifically, the latent vector of each layer of the hallucination recognition block is decoded into a word unit token, that is, the word unit output by the hallucination recognition block is obtained.

[0064] Furthermore, the method of the present invention further comprises:

[0065] When the word-gram output by the hallucination recognition block indicates that the first question has an answer, other word-grams except the first word-gram output by the last layer of the large language model are obtained.

[0066] Here, when the first word-gram is outputted by the last layer of the large language model, the word-gram outputted by the hallucination recognition block is obtained; thereafter, when the word-gram outputted by the hallucination recognition block indicates that the first question has an answer, that is, when there are no semantic tokens such as "sorry" or "excuse me" in the token outputted by the hallucination recognition block, the normal large language model reasoning process is continued to output other word-grams.

[0067] Accordingly, the above step 104, outputting the word unit output by the last layer of the large language model as the answer to the first question, includes:

[0068] The first word-gram and the other word-grams output by the last layer of the large language model are output as answers to the first question.

[0069] In an optional embodiment, the method of the present invention further comprises:

[0070] a1, obtaining a set of hallucination instructions, wherein the questions corresponding to the instructions in the hallucination instruction set have no answers.

[0071] Here, the questions corresponding to the instructions include but are not limited to manually constructed unanswered questions and hallucination questions collected from the Internet. Afterwards, the manually constructed unanswered questions and the hallucination questions collected from the Internet are used to design the best prompt according to the input preference of the large language model to obtain a set of hallucination instructions.

[0072] a2. For each instruction of the hallucination instruction set, input the instruction into the large language model to obtain the word unit and confidence corresponding to the target word unit output at each layer of the large language model, wherein the target word unit is the first word unit outputted by the last layer of the large language model.

[0073] Specifically, for each instruction of the hallucination instruction set, the instruction is input into the large language model to obtain the latent vector of the target word corresponding to each layer of the large language model and the confidence of the target word corresponding to each layer of the large language model.

[0074] Afterwards, the latent vector of the target word corresponding to each layer of the large language model is decoded to obtain the word corresponding to the target word output at each layer of the large language model.

[0075] Here, for each instruction in the hallucination instruction set, after the instruction is input into the large language model, the latent vector of each layer of the first token is taken out. Then, the obtained latent vector is decoded into a readable token by lm-head (a layer in the large language model).

[0076] a3, using the word units and confidences of the target word units corresponding to each instruction and outputted at each layer of the large language model, to obtain the hallucination recognition block of the large language model.

[0077] Optionally, this step a3 specifically includes:

[0078] a31, for each of the instructions, based on the word-grams outputted at each layer of the large language model corresponding to the target word-gram, a first block corresponding to the first word-gram is obtained, wherein the first word-gram includes at least one word-gram indicating that the question corresponding to the instruction has no answer.

[0079] That is, based on the word unit outputted at each layer of the large language model corresponding to the first token, a statistical analysis is performed to find out which layers of the large language model output the semantic tokens (i.e., the first word unit) of "sorry" and "excuse me", and these layers are determined as the first block.

[0080] a32, based on the confidence of the target word corresponding to the output of each layer of the large language model, obtain the confidence corresponding to the first block.

[0081] a33, selecting a first block whose confidence is greater than a preset threshold from the obtained multiple first blocks as a hallucination recognition block of the large language model.

[0082] Here, each instruction corresponds to a first block. Multiple first blocks are obtained through the above processing, and the first block with higher confidence (that is, the confidence is greater than a preset threshold) is used as the hallucination recognition block of the large language model.

[0083] The problem handling method of the present invention solves the hallucination problem of the large language model with the inherent ability of the large language model by activating the block (i.e., the hallucination recognition block) of whether the large language model can answer the question. On the one hand, no additional information provision, information processing, and information output are required. While solving the hallucination problem of the large language model, the reasoning efficiency of the large language model is improved; on the other hand, the hallucination recognition ability of the large language model will be improved as the comprehensive improvement of the large language model's capabilities, and the generalization of the large language model solved based on the block collaboration mode is better.

[0084] like Figure 2 As shown, an embodiment of the present invention further provides a problem handling device, the device comprising:

[0085] A first acquisition module 201 is used to acquire a first instruction, where the first instruction is an instruction for a first question;

[0086] A second acquisition module 202 is used to input the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is used to identify whether a result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language model;

[0087] A first processing module 203 is configured to change the word-gram output by the last layer of the large language model to the word-gram output by the hallucination recognition block, and output the word-gram as the answer to the first question when the word-gram output by the hallucination recognition block indicates that the first question has no answer;

[0088] The second processing module 204 is used to output the word-gram output by the last layer of the large language model as the answer to the first question when the word-gram output by the hallucination recognition block indicates that the first question has an answer.

[0089] Optionally, the second acquisition module 202 includes:

[0090] A first acquisition unit is used to input the first instruction into the large language model to obtain a latent vector of a hallucination recognition block of the large language model when the last layer of the large language model outputs a first word unit;

[0091] The first processing unit is used to decode the latent vector of the hallucination recognition block to obtain the word unit output by the hallucination recognition block.

[0092] Optionally, the device in the embodiment of the present invention further includes:

[0093] A third acquisition module is used to acquire other word-grams except the first word-gram output by the last layer of the large language model when the word-gram output by the hallucination recognition block indicates that the first question has an answer;

[0094] Accordingly, the second processing module 204 includes:

[0095] The second processing unit is used to output the first word unit and the other word units output by the last layer of the large language model as answers to the first question.

[0096] Optionally, the device in the embodiment of the present invention further includes:

[0097] A fourth acquisition module is used to acquire a set of hallucination instructions, wherein the questions corresponding to the instructions in the hallucination instruction set have no answers;

[0098] a fifth acquisition module, for inputting each instruction of the hallucination instruction set into the large language model, and obtaining a word unit and a confidence level of a target word unit corresponding to the word unit output at each layer of the large language model, wherein the target word unit is the first word unit outputted by the last layer of the large language model;

[0099] The third processing module is used to obtain the hallucination recognition block of the large language model by using the word unit and confidence level outputted at each layer of the large language model corresponding to each instruction and the target word unit.

[0100] Optionally, the third processing module includes:

[0101] A third processing unit is configured to obtain, for each of the instructions, a first block corresponding to a first word-gram based on the word-gram outputted at each layer of the large language model corresponding to the target word-gram, wherein the first word-gram includes at least one word-gram indicating that there is no answer to the question corresponding to the instruction;

[0102] A fourth processing unit, configured to obtain a confidence level corresponding to the first block based on the confidence level of the target word unit output at each layer of the large language model;

[0103] The fifth processing unit is used to select a first block whose confidence is greater than a preset threshold from the obtained multiple first blocks as the hallucination recognition block of the large language model.

[0104] The problem processing device of the embodiment of the present invention obtains a first instruction, where the first instruction is an instruction for a first question; then, the first instruction is input into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, where the hallucination recognition block is used to identify whether the result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language module; finally, when the word unit output by the hallucination recognition block indicates that there is no answer to the first question, the word unit output by the last layer of the large language model is changed to the word unit output by the hallucination recognition block, and is output as the answer to the first question; when the word unit output by the hallucination recognition block indicates that there is an answer to the first question, the word unit output by the last layer of the large language model is output as the answer to the first question, so that by activating the ability of the large language model to answer the question (i.e., the hallucination recognition block), the hallucination problem of the large language model is solved with the inherent ability of the large language model, thereby improving the accuracy of the large language model in solving the problem.

[0105] In order to better achieve the above goals, Figure 3 As shown, an embodiment of the present invention further provides a problem handling device, including a processor 300 and a transceiver 310, wherein the transceiver 310 receives and sends data under the control of the processor 300, and the processor 300 is used to perform the following process:

[0106] Obtaining a first instruction, where the first instruction is an instruction for a first question;

[0107] Inputting the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is used to identify whether a result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language model;

[0108] When the word-gram output by the hallucination recognition block indicates that there is no answer to the first question, the word-gram output by the last layer of the large language model is changed to the word-gram output by the hallucination recognition block, and is output as the answer to the first question;

[0109] When the word-gram output by the hallucination recognition block indicates that the first question has an answer, the word-gram output by the last layer of the large language model is output as the answer to the first question.

[0110] Optionally, the processor 300 is further configured to:

[0111] Inputting the first instruction into the large language model to obtain a latent vector of the hallucination recognition block of the large language model when the last layer of the large language model outputs the first word unit;

[0112] The latent vector of the hallucination recognition block is decoded to obtain the word unit output by the hallucination recognition block.

[0113] Optionally, the processor 300 is further configured to:

[0114] When the word unit output by the hallucination recognition block indicates that the first question has an answer, obtaining other word units except the first word unit output by the last layer of the large language model;

[0115] The first word-gram and the other word-grams output by the last layer of the large language model are output as answers to the first question.

[0116] Optionally, the processor 300 is further configured to:

[0117] Obtaining a set of hallucination instructions, wherein the questions corresponding to the instructions in the hallucination instruction set have no answers;

[0118] For each instruction of the hallucination instruction set, input the instruction into the large language model to obtain the word unit and confidence corresponding to the target word unit output at each layer of the large language model, wherein the target word unit is the first word unit outputted by the last layer of the large language model;

[0119] The hallucination recognition block of the large language model is obtained by using the word unit and confidence level of the target word unit corresponding to each instruction and outputted at each layer of the large language model.

[0120] Optionally, the processor 300 is further configured to:

[0121] For each of the instructions, based on the word-grams outputted at each layer of the large language model corresponding to the target word-gram, a first block corresponding to a first word-gram is obtained, wherein the first word-gram includes at least one word-gram indicating that there is no answer to the question corresponding to the instruction;

[0122] Obtaining the confidence corresponding to the first block based on the confidence corresponding to the target word output at each layer of the large language model;

[0123] A first block whose confidence level is greater than a preset threshold is selected from the obtained multiple first blocks as a hallucination recognition block of the large language model.

[0124] The problem processing device of the embodiment of the present invention obtains a first instruction, where the first instruction is an instruction for a first question; then, the first instruction is input into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, where the hallucination recognition block is used to identify whether the result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language module; finally, when the word unit output by the hallucination recognition block indicates that there is no answer to the first question, the word unit output by the last layer of the large language model is changed to the word unit output by the hallucination recognition block, and is output as the answer to the first question; when the word unit output by the hallucination recognition block indicates that there is an answer to the first question, the word unit output by the last layer of the large language model is output as the answer to the first question, so that by activating the ability of the large language model to answer the question (i.e., the hallucination recognition block), the hallucination problem of the large language model is solved with the inherent ability of the large language model, thereby improving the accuracy of the large language model in solving the problem.

[0125] An embodiment of the present invention also provides a problem handling device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, each process in the problem handling method embodiment described above is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0126] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, each process in the above-mentioned problem handling method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0127] An embodiment of the present invention further provides a computer program product, including computer instructions, which implement the steps in the problem handling method as described above when executed by a processor.

[0128] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0129] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A device that specifies functions in one or more processes and / or one or more blocks.

[0130] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable storage medium produce a paper product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that the computer or other programmable device executes a series of operating steps to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0132] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A problem solving method, characterized in that: include: Obtaining a first instruction, where the first instruction is an instruction for a first question; Inputting the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is used to identify whether a result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language model; When the word-gram output by the hallucination recognition block indicates that there is no answer to the first question, the word-gram output by the last layer of the large language model is changed to the word-gram output by the hallucination recognition block, and is output as the answer to the first question; When the word-gram output by the hallucination recognition block indicates that the first question has an answer, the word-gram output by the last layer of the large language model is output as the answer to the first question.

2. The method according to claim 1, characterized in that The step of inputting the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model includes: Inputting the first instruction into the large language model to obtain a latent vector of the hallucination recognition block of the large language model when the last layer of the large language model outputs the first word unit; The latent vector of the hallucination recognition block is decoded to obtain the word unit output by the hallucination recognition block.

3. The method according to claim 2, characterized in that The method further comprises: When the word unit output by the hallucination recognition block indicates that the first question has an answer, obtaining other word units except the first word unit output by the last layer of the large language model; Outputting the word unit output by the last layer of the large language model as the answer to the first question includes: The first word-gram and the other word-grams output by the last layer of the large language model are output as answers to the first question.

4. The method according to claim 1, characterized in that: The method further comprises: Obtaining a set of hallucination instructions, wherein the questions corresponding to the instructions in the hallucination instruction set have no answers; For each instruction of the hallucination instruction set, input the instruction into the large language model to obtain the word unit and confidence corresponding to the target word unit output at each layer of the large language model, wherein the target word unit is the first word unit outputted by the last layer of the large language model; The hallucination recognition block of the large language model is obtained by using the word unit and confidence level of the target word unit corresponding to each instruction and outputted at each layer of the large language model.

5. The method according to claim 4, characterized in that The method uses the word units and confidences of the target word units corresponding to the instructions and outputted at each layer of the large language model to obtain the hallucination recognition block of the large language model, including: For each of the instructions, based on the word-grams outputted at each layer of the large language model corresponding to the target word-gram, a first block corresponding to a first word-gram is obtained, wherein the first word-gram includes at least one word-gram indicating that there is no answer to the question corresponding to the instruction; Obtaining the confidence corresponding to the first block based on the confidence corresponding to the target word output at each layer of the large language model; A first block whose confidence level is greater than a preset threshold is selected from the obtained multiple first blocks as a hallucination recognition block of the large language model.

6. A problem handling device, characterized in that: include: A first acquisition module, used to acquire a first instruction, where the first instruction is an instruction for a first question; a second acquisition module, configured to input the first instruction into a large language model, and obtain a word unit output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is configured to identify whether a result generated by the large language model based on the input instruction contains content representing that there is no answer, and the hallucination recognition block includes at least one layer of the large language model; A first processing module is configured to change the word-gram output by the last layer of the large language model to the word-gram output by the hallucination recognition block, and output the word-gram as the answer to the first question when the word-gram output by the hallucination recognition block indicates that the first question has no answer; The second processing module is used to output the word-gram output by the last layer of the large language model as the answer to the first question when the word-gram output by the hallucination recognition block indicates that the first question has an answer.

7. A problem handling device, comprising a processor and a transceiver, wherein the transceiver receives and sends data under the control of the processor, characterized in that: The processor is configured to perform the following operations: Obtaining a first instruction, where the first instruction is an instruction for a first question; Inputting the first instruction into a large language model to obtain a word unit output by a hallucination recognition block of the large language model, wherein the hallucination recognition block is used to identify whether a result generated by the large language model based on the input instruction contains content representing no answer, and the hallucination recognition block includes at least one layer of the large language model; When the word-gram output by the hallucination recognition block indicates that there is no answer to the first question, the word-gram output by the last layer of the large language model is changed to the word-gram output by the hallucination recognition block, and is output as the answer to the first question; When the word-gram output by the hallucination recognition block indicates that the first question has an answer, the word-gram output by the last layer of the large language model is output as the answer to the first question.

8. A problem handling device, comprising a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that: When the processor executes the program, the problem handling method according to any one of claims 1 to 5 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the problem handling method as described in any one of claims 1 to 5 are implemented.

10. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps in the problem handling method according to any one of claims 1 to 5.

Images