Question and answer method, device and equipment based on large language model and storage medium
Through the large language model that divides work and works on the client and cloud devices, the problem of high computing complexity and cost in the application of the large language model on edge devices is solved, efficient question-and-answer reasoning and accuracy verification are achieved, and the cloud is reduced The pressure and cost of equipment.
Patent Information
- Application Number
- CN202411099663.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing large language models have high computational complexity and memory costs during the inference process, making them difficult to effectively apply on edge devices, and are difficult to maintain model quality and context learning capabilities.
By setting up large language models with different model structures and computing power on the client device and the cloud device, the client device is responsible for question type identification and candidate answer generation, and the cloud device is responsible for the accuracy verification of candidate answers, realizing division of labor and collaboration, and reducing the cloud device is responsible for the cloud device. The operating pressure and cost of the equipment.
While ensuring the inference quality of large language models, it improves inference efficiency, reduces the operating pressure of cloud devices and the operating cost of large language models, and realizes effective application on edge devices.
Smart Images

Figure CN119003729A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a question-answering method, apparatus, device and storage medium based on a large language model. Background Art
[0002] At present, large language models perform well in natural language processing tasks. However, how to maintain the quality and context learning ability of large language models while reducing the computational complexity and memory cost of the reasoning process of large language models and ensuring the application of large language models on edge devices has become a common concern in the industry. Summary of the invention
[0003] The present invention provides a question-answering method, apparatus, device and storage medium based on a large language model, which improves the reasoning efficiency of the large language model while ensuring the reasoning quality of the large language model.
[0004] According to one aspect of the present invention, a question answering method based on a large language model is provided, the method comprising:
[0005] The client device is used to obtain questions to be answered, and the question types of the questions to be answered are identified; wherein the question types include end-cloud collaboration questions;
[0006] The client device calls the first language model, identifies the question to be answered based on the question type of the question to be answered, generates at least two candidate answers, and sends the candidate answers to the cloud device; wherein the first language model is used to predict the candidate answers to the question to be answered in a character-by-character progressive manner, and the candidate answers refer to answers that have not been verified for accuracy;
[0007] The cloud device is used to call the second largest language model, and based on the semantics of the question to be answered, the accuracy of the at least two candidate answers is verified, a target answer is determined, and the target answer is returned to the client device.
[0008] According to another aspect of the present invention, a question-answering device based on a large language model is provided, the device comprising:
[0009] A type identification module, used to obtain questions to be answered using the client device, and identify the types of the questions to be answered; wherein the question types include end-cloud collaborative questions;
[0010] a candidate answer generation module, configured to use the client device to call the first language model, identify the question to be answered based on the question type of the question to be answered, generate at least two candidate answers, and send the candidate answers to the cloud device; wherein the first language model is used to predict candidate answers to the question to be answered in a character-by-character progressive manner, and the candidate answers refer to answers that have not been verified for accuracy;
[0011] The target answer generation module is used to use the cloud device to call the second largest language model, based on the semantics of the question to be answered, to verify the accuracy of the at least two candidate answers, to determine the target answer, and to return the target answer to the client device.
[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:
[0013] at least one processor; and
[0014] a memory communicatively connected to the at least one processor; wherein,
[0015] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the question-answering method based on the large language model described in any embodiment of the present invention.
[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the question-answering method based on a large language model described in any embodiment of the present invention when executed.
[0017] According to another aspect of the present invention, a computer program product is provided, including a computer program, which, when executed by a processor, implements the question-answering method based on a large language model described in any embodiment of the present invention.
[0018] The technical solution of the embodiment of the present invention, by setting large language models with different model structures and computing powers on the client device and the cloud device, enables the large language models set in the client device and the cloud device to work together and each be responsible for a different natural language processing process. While ensuring the reasoning quality of the large language model, it ensures the application of the large language model in the client device, reduces the operating pressure of the cloud device and the operating cost of the large language model.
[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0021] Figure 1 is a flowchart of a question-answering method based on a large language model provided according to Embodiment 1 of the present invention;
[0022] Figure 2 is a flowchart of a question-answering method based on a large language model provided according to Embodiment 2 of the present invention;
[0023] Figure 3 is a structural diagram of a question-answering device based on a large language model provided according to Embodiment 3 of the present invention;
[0024] Figure 4 It is a structural diagram of an electronic device for implementing the question-answering method based on a large language model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0027] Embodiment 1
[0028] Figure 1A flowchart of a question-answering method based on a large language model is provided for the first embodiment of the present invention. This embodiment is applicable to the situation of question-answering based on a large language model. The method can be applied to a terminal-cloud collaboration system including a client device and a cloud device. The method can be executed by a question-answering device based on a large language model. The question-answering device based on a large language model can be implemented in the form of hardware and / or software. The question-answering device based on a large language model can be configured in various general-purpose computing devices, such as a server device or a client device integrated with a large language model. Figure 1 As shown, the method includes:
[0029] S110: Using a client device to obtain questions to be answered, and identifying the types of the questions to be answered.
[0030] In an embodiment of the present invention, the end-cloud collaboration system can be composed of a client device and a cloud device, and is used for the system to respond to unanswered questions obtained by the client device. The client device is provided with a first largest language model, and the cloud device is provided with a second largest language model, and the complexity of the model structure and the computing power of the second largest language model are greater than those of the first largest language model.
[0031] Optionally, the first language model can be determined based on the computing power of the client device, and the second language model can be determined based on the computing power of the cloud device. Exemplarily, the client device can be various terminal devices, such as mobile phones and vehicle-mounted devices. On this basis, limited by the hardware limitations of the terminal device itself, the computing power level represented by the memory and operating environment of the terminal device is much smaller than that of the cloud device. The first language model set by the client device can be regarded as a simplified model of the second language model set by the cloud device. It is only necessary to partially delete the functions of the second language model without additional model training. Compared with running the second language model, the first language model runs faster and has a lower operating cost. Optionally, the types of the first language model and the second language model can be adaptively set according to those skilled in the art, and are not specifically limited here.
[0032] It should be noted that the number of questions to be answered obtained by the client device is relatively large and complicated. Therefore, it is necessary to identify the types of questions to be answered. In an embodiment of the present invention, the types of questions to be answered can be roughly divided into two categories, one is end-cloud collaborative questions, and the other is non-end-cloud collaborative questions. End-cloud collaborative questions may refer to questions with answers that require the participation of cloud devices to answer, and non-end-cloud collaborative questions may refer to questions that do not require cloud devices and can be answered by the client device itself, for example, chats between users and mobile phones such as "How are you?" and "Are you there?", as well as questions asked by users to the mobile phone about the time. The mobile phone can directly answer according to the time of the device itself.
[0033] Optionally, a client device is used to obtain the user's questions to be answered and identify the type of the questions to be answered, including: the client device inputs the obtained questions to be answered into a pre-trained question discrimination model; the question discrimination model determines whether the computing power of the client device can answer the questions to be answered based on the configuration parameters of the client device and the local knowledge data stored in the client device; if the client device cannot answer the questions to be answered, it is determined that the questions to be answered are end-cloud collaborative questions.
[0034] In an embodiment of the present invention, a question discrimination model can be used to determine the question type of a question to be answered. The question discrimination model will identify the input question to be answered, determine the semantics of the question to be answered, and determine the computing power of the client device in combination with the configuration parameters of the client device described in the question discrimination model and the local data to determine whether it is capable of replying to the question to be answered. If the question discrimination model determines that the client device itself cannot reply to the question to be answered, it will be determined that the question type of the question to be answered belongs to an end-cloud collaborative question. Otherwise, it will be determined that the question to be answered is a non-end-cloud collaborative question. For non-end-cloud collaborative questions, it can be directly handed over to the client device for processing. Exemplarily, if the logical complexity of the question to be answered is greater than a certain complexity threshold, the client device cannot directly process the question to be answered due to the limitation of its own hardware, or the reply to the question to be answered involves knowledge data other than the local data of the client device, which the client device cannot directly obtain, then the client device cannot process the question to be answered either. Optionally, the question discrimination model is a large language model, which is used to perform natural language processing on the question to be answered in combination with the device parameter configuration of the client device to determine the question type of the question to be answered. The complexity threshold can be adaptively set according to technical personnel in this field.
[0035] By identifying the types of questions to be answered, questions that are not end-to-end cloud collaborative questions are handed over to the device itself for processing, avoiding handing over questions to be answered directly to cloud devices for processing. Tasks are dynamically assigned to client devices or cloud devices, reducing the model utilization rate of cloud devices, thereby reducing the operating pressure of cloud devices and improving question-answering efficiency.
[0036] S120: Use the client device to call the first language model, identify the question to be answered based on the question type of the question to be answered, generate at least two candidate answers, and send the candidate answers to the cloud device.
[0037] S130: Using a cloud device to call the second largest language model, based on the semantics of the question to be answered, verify the accuracy of at least two candidate answers, determine a target answer, and return the target answer to the client device.
[0038] The first language model can be used to predict candidate answers to the question in a character-by-character manner. The candidate answer may refer to an answer that has not been verified for accuracy; the target answer may refer to a candidate answer that has passed the accuracy verification. Optionally, the first language model and the second language model are used to process the question type of the question to be answered that belongs to the end-cloud collaborative question.
[0039] Optionally, a client device calls the first largest language model to identify the question to be answered based on the question type of the question to be answered, and generate at least two candidate answers, including: when the question type of the question to be answered is an end-cloud collaborative question, the question to be answered is input into the first largest language model, so that the first largest language model generates at least two candidate answers in a manner that increases in sequence by the number of characters, and the candidate answers are sent to the cloud device.
[0040] In an optional embodiment of the present invention, when the question type of the question to be answered belongs to an end-cloud collaborative problem, the client device will hand over the question to be answered to the first largest language model for processing. After receiving the question to be answered, the first largest language model will perform natural language processing on the question to be answered and predict the answer to the question to be answered. It should be noted that due to the limitations of the client device's own hardware conditions, the first largest language model can only predict one character at a time and add the number of characters based on the last prediction. In other words, the first largest language model can only predict one character at a time, and the next prediction result of the first largest language model is predicted based on the last prediction result, that is, it predicts the next character of the last prediction result. For example, if the question to be answered is "Who are you", the candidate answer predicted by the first language model for the first time may be "I", the candidate answer predicted by the first language model for the second time may be "I am", and the candidate answer predicted by the first language model for the third time may be "I am a person", and each time the first language model predicts a candidate answer, it will send the candidate answer to the cloud device, and the cloud device will call the second language model to verify the accuracy of the candidate answer, and return the accuracy verification result to the client device, so that the client predicts the next candidate answer based on the accuracy verification result, until the second language model determines the target answer.
[0041] The first large language model predicts candidate answers in a character-by-character manner, and only predicts the next character that best matches the current character each time, which greatly reduces the reasoning time of the first large language model. Moreover, only one character is predicted each time, which does not cause operating pressure on the client device. This achieves the deployment of a large language model on the client device while further improving the reasoning speed.
[0042] The technical solution of the embodiment of the present invention, by setting large language models with different model structures and computing powers on the client device and the cloud device, enables the large language models set in the client device and the cloud device to work together and each be responsible for a different natural language processing process. While ensuring the reasoning quality of the large language model, it ensures the application of the large language model in the client device, reduces the operating pressure of the cloud device and the operating cost of the large language model.
[0043] Embodiment 2
[0044] Figure 2 This is a flowchart of a question-answering method based on a large language model provided in the second embodiment of the present invention. This embodiment further refines the above embodiment and provides a method of using a cloud device to call a second large language model to verify the accuracy of at least two candidate answers based on the semantics of the question to be answered. It should be noted that for the part not described in detail in the embodiment of the present invention, please refer to the relevant description of other embodiments, which will not be repeated here. Figure 2 As shown, the method includes:
[0045] S210: Using a client device to obtain questions to be answered, and identifying the types of the questions to be answered.
[0046] S220: Use the client device to call the first language model, identify the question to be answered based on the question type of the question to be answered, generate at least two candidate answers, and send the candidate answers to the cloud device.
[0047] S230. When the cloud device receives a candidate answer, it inputs the candidate answer into the second largest language model, so that the second largest language model verifies the accuracy of the candidate answer according to the semantics of the question to be answered, and feeds back the verification result to the client.
[0048] The accuracy check may include a semantic check, that is, the accuracy check may be to check whether the semantics of the candidate answer is logical, etc. Optionally, the semantic check may be a contextual semantic check based on the semantics of the question to be answered and the semantics of the candidate answer.
[0049] It should be noted that the first largest language model is only used to predict candidate answers to questions to be answered, but not to verify the correctness of candidate answers. The second largest language model is not used to predict candidate answers to questions to be answered, but to verify the correctness of candidate answers. This is because the model structure and computing power of the second largest model are greater than those of the first largest language model, and the first largest language model cannot implement complex logical judgments.
[0050] After receiving the candidate answers, the second largest language model will identify and perform natural language processing on the candidate answers, perform contextual semantic verification on the candidate answers based on the semantics of the question to be answered and the semantics of the candidate answers, and verify the accuracy of the candidate answers.
[0051] Optionally, after the verification result is fed back to the client, it also includes: if the verification result is passed, it indicates that the candidate answer is consistent with the question to be answered. After the client device receives the verification result, it predicts the next character of the candidate answer based on the candidate answer through the first largest language model to generate the next candidate answer; if the verification result is failed, it indicates that there is a semantic error in the candidate answer. After receiving the verification result, the client device deletes the last character of the candidate answer, and uses the first largest language model to re-predict the last character of the candidate answer based on the candidate answer after the last character is deleted, to generate the next candidate answer.
[0052] Exemplarily, the question to be answered is "Who are you"; the candidate answer given by the first language model for the first time is "I"; the verification of the received candidate answer by the second language model is "The question to be answered is: Who are you; the candidate answer given by the first language model is: I; please ask if it meets the requirements of the question to be answered; if it meets, return 1; if it does not meet, return 2; if the answer is completed, return 3". The result returned by the second language model is: 1. The candidate answer given by the first language model for the second time is "I am"; the verification of the received candidate answer by the second language model is "The question to be answered is: Who are you; the candidate answer given by the first language model is: I am; please ask if it meets the requirements of the question to be answered; if it meets, return 1; if it does not meet, return 2; if the answer is completed, return 3". The result returned by the second language model is: 1. The candidate answer given by the first language model for the third time is "I am a person"; the verification of the received candidate answer by the second language model is "The question to be answered is: Who are you; the candidate answer given by the first language model is: I am a person; please ask if it meets the requirements of the question to be answered; if it meets, return 1; if it does not meet, return 2; if the answer is completed, return 3". The result returned by the second language model is: 1. The fourth candidate answer given by the first language model is "I am human"; the second language model verifies the received candidate answers as "The question to be answered is: Who are you? The candidate answer given by the first language model is: I am human; May I ask if it meets the requirements of the question to be answered? Return 1 if it meets the requirements, 2 if it does not meet the requirements, and 3 if the answer is completed". The result returned by the second language model is: 2. At this time, the first language model will roll back the candidate answer generated last time by one character and re-predict. The fifth candidate answer given by the first language model is "I am artificial"; the second language model verifies the received candidate answers as "The question to be answered is: Who are you? The candidate answer given by the first language model is: I am artificial; May I ask if it meets the requirements of the question to be answered? Return 1 if it meets the requirements, 2 if it does not meet the requirements, and 3 if the answer is completed". The result returned by the second language model is: 1. The candidate answer given by the first language model for the sixth time is "I am artificial intelligence"; the verification of the received candidate answers by the second language model is "The question to be answered is: Who are you? The candidate answer given by the first language model is: I am artificial intelligence; please ask if it meets the requirements of the question to be answered; if it meets, return 1; if it does not meet, return 2; if the answer is completed, return 3". The result returned by the second largest language model is: 1. The candidate answer given by the first language model for the seventh time is "I am artificial intelligence"; the verification of the received candidate answers by the second language model is "The question to be answered is: Who are you? The candidate answer given by the first language model is: I am human functional intelligence; please ask if it meets the requirements of the question to be answered; if it meets, return 1; if it does not meet, return 2; if the answer is completed, return 3". The result returned by the second largest language model is: 1.The candidate answer given by the first language model for the eighth time is "I am a human-functional intelligent assistant"; the second language model verifies the received candidate answers as "The question to be answered is: Who are you? The candidate answer given by the first language model is: I am an artificial intelligence assistant; May I ask if it meets the requirements of the question to be answered? Return 1 if it meets the requirements, 2 if it does not meet the requirements, and 3 if the answer is completed." The result returned by the second largest language model is: 1. The candidate answer given by the first language model for the ninth time is "I am an artificial intelligence assistant"; the second language model verifies the received candidate answers as "The question to be answered is: Who are you? The candidate answer given by the first language model is: I am an artificial intelligence assistant; May I ask if it meets the requirements of the question to be answered? Return 1 if it meets the requirements, 2 if it does not meet the requirements, and 3 if the answer is completed." The result returned by the second largest language model is: 3, and returns the target answer "I am a human-functional intelligent assistant" to the client device.
[0053] Optionally, the second largest language model can determine the target answer by a matching threshold between the candidate answer and the question to be answered. When the accuracy check of the candidate answer passes, and the second largest language model determines that the matching degree between the candidate answer and the question to be answered exceeds a preset matching threshold, the candidate answer is used as the target answer. Optionally, the matching threshold can be adaptively set according to those skilled in the art.
[0054] By inferring and predicting candidate answers in a descending manner using the first language model, and verifying the correctness of the candidate answers based on the second language model, it is possible to avoid cloud devices directly reasoning and predicting the questions to be answered. By generating candidate answers by predicting characters on the client, the reasoning process is simplified and the computational complexity of the reasoning process is reduced.
[0055] The technical solution of the embodiment of the present invention can reduce the inference cost in the inference process and improve the inference efficiency of the large language model by performing inference prediction of candidate answers in a character-by-character manner on the client device and verifying the accuracy of the candidate answer results on the cloud device.
[0056] Embodiment 3
[0057] Figure 3 This is a schematic diagram of the structure of a question-answering device based on a large language model provided in Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:
[0058] A type identification module 310 is used to obtain questions to be answered using the client device and identify the types of the questions to be answered; wherein the question types include end-cloud collaborative questions;
[0059] The candidate answer generation module 320 is used to use the client device to call the first language model, identify the question to be answered based on the question type of the question to be answered, generate at least two candidate answers, and send the candidate answers to the cloud device; wherein the first language model is used to predict the candidate answers to the question to be answered in a character-by-character progressive manner, and the candidate answers refer to answers that have not been verified for accuracy;
[0060] The target answer generation module 330 is used to use the cloud device to call the second largest language model, based on the semantics of the question to be answered, to verify the accuracy of the at least two candidate answers, to determine the target answer, and to return the target answer to the client device.
[0061] The technical solution of the embodiment of the present invention, by setting large language models with different model structures and computing powers on the client device and the cloud device, enables the large language models set in the client device and the cloud device to work together and each be responsible for a different natural language processing process. While ensuring the reasoning quality of the large language model, it ensures the application of the large language model in the client device, reduces the operating pressure of the cloud device and the operating cost of the large language model.
[0062] Optionally, the target answer generation module 330 can be specifically used for: when the cloud device receives a candidate answer, it will input the candidate answer into the second largest language model, so that the second largest language model will verify the accuracy of the candidate answer according to the semantics of the question to be answered, and feedback the verification result to the client; wherein the accuracy verification includes semantic verification.
[0063] Optionally, the device further comprises an iteration module, for:
[0064] If the verification result is passed, it indicates that the candidate answer meets the question to be answered. After receiving the verification result, the client device predicts the next character of the candidate answer based on the candidate answer through the first language model to generate the next candidate answer;
[0065] If the verification result is failed, it indicates that there is a semantic error in the candidate answer. After receiving the verification result, the client device deletes the last character of the candidate answer, and uses the first language model to re-predict the last character of the candidate answer based on the candidate answer after deleting the last character, to generate the next candidate answer.
[0066] Optionally, the candidate answer generation module 320 can be specifically used for: when the question type of the question to be answered belongs to an end-cloud collaborative problem, the question to be answered is input into the first largest language model, so that the first largest language model generates at least two candidate answers in a manner of increasing number of characters, and sends the candidate answers to the cloud device. Optionally, the type identification module 310 can be specifically used for: the client device inputs the acquired question to be answered into a pre-trained question discrimination model; the question discrimination model determines whether the computing power of the client device can answer the question to be answered based on the configuration parameters of the client device and the local data stored in the client device; when the client device cannot answer the question to be answered, it is determined that the question to be answered is an end-cloud collaborative problem.
[0067] Optionally, the first largest language model can be determined according to the computing power of the client device, and the second largest language model can be determined according to the computing power of the cloud device.
[0068] The question-answering device based on a large language model provided in an embodiment of the present invention can execute the question-answering method based on a large language model provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0069] Embodiment 4
[0070] Figure 4 A schematic diagram of an electronic device 410 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0071] like Figure 4As shown, the electronic device 410 includes at least one processor 411, and a memory connected to the at least one processor 411 in communication, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.
[0072] A number of components in the electronic device 410 are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device 410 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0073] Processor 411 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. Processor 411 performs the various methods and processes described above, such as a question-answering method based on a large language model.
[0074] In some embodiments, the question-answering method based on a large language model may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the question-answering method based on a large language model described above may be performed. Alternatively, in other embodiments, the processor 411 may be configured to execute the question-answering method based on a large language model in any other appropriate manner (e.g., by means of firmware).
[0075] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0076] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0077] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0078] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0079] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0080] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0081] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0082] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A question-answering method based on a large language model, It is characterized in that Applied to a terminal-cloud collaboration system, the terminal-cloud collaboration system includes a client device and a cloud device, the client device is provided with a first large language model, the cloud device is provided with a second large language model, and the complexity of the model structure and the computing power of the second large language model are greater than those of the first large language model; the method includes: The client device is used to obtain questions to be answered, and the question types of the questions to be answered are identified; wherein the question types include end-cloud collaboration questions; The client device calls the first language model, identifies the question to be answered based on the question type of the question to be answered, generates at least two candidate answers, and sends the candidate answers to the cloud device; wherein the first language model is used to predict the candidate answers to the question to be answered in a character-by-character progressive manner, and the candidate answers refer to answers that have not been verified for accuracy; The cloud device is used to call the second largest language model, and based on the semantics of the question to be answered, the accuracy of the at least two candidate answers is verified, a target answer is determined, and the target answer is returned to the client device.
2. The method according to claim 1, It is characterized in that The cloud device is used to call the second largest language model, and based on the semantics of the question to be answered, the accuracy of the at least two candidate answers is verified, including: When the cloud device receives a candidate answer, it inputs the candidate answer into the second largest language model so that the second largest language model verifies the accuracy of the candidate answer according to the semantics of the question to be answered, and feeds back the verification result to the client; wherein the accuracy verification includes semantic verification.
3. The method according to claim 2, It is characterized in that After the verification result is fed back to the client, the method further includes: If the verification result is passed, it indicates that the candidate answer meets the question to be answered. After receiving the verification result, the client device predicts the next character of the candidate answer based on the candidate answer through the first language model to generate the next candidate answer; If the verification result is failed, it indicates that there is a semantic error in the candidate answer. After receiving the verification result, the client device deletes the last character of the candidate answer, and uses the first language model to re-predict the last character of the candidate answer based on the candidate answer after deleting the last character, to generate the next candidate answer.
4. The method according to claim 1, It is characterized in that The client device calls the first language model, identifies the question to be answered based on the question type of the question to be answered, and generates at least two candidate answers, including: When the question type of the question to be answered belongs to an end-cloud collaborative question, the question to be answered is input into the first largest language model so that the first largest language model generates at least two candidate answers in a manner of increasing number of characters, and sends the candidate answers to the cloud device.
5. The method according to claim 1, It is characterized in that The client device is used to obtain the user's unanswered question and identify the type of the unanswered question, including: The client device inputs the acquired question to be answered into a pre-trained question discrimination model; the question discrimination model determines whether the computing power of the client device can answer the question to be answered based on the configuration parameters of the client device and the local data stored in the client device; In the case that the client device cannot answer the question to be answered, it is determined that the question to be answered is a terminal-cloud collaboration question.
6. The method according to claim 1, It is characterized in that The first largest language model is determined according to the computing power of the client device, and the second largest language model is determined according to the computing power of the cloud device.
7. A question-answering device based on a large language model, It is characterized in that Applied to a terminal-cloud collaboration system, the terminal-cloud collaboration system includes a client device and a cloud device, the client device is provided with a first large language model, the cloud device is provided with a second large language model, and the complexity of the model structure and the computing power of the second large language model are greater than those of the first large language model; the device includes: A type identification module, used to obtain questions to be answered using the client device, and identify the types of the questions to be answered; wherein the question types include end-cloud collaborative questions; a candidate answer generation module, configured to use the client device to call the first language model, identify the question to be answered based on the question type of the question to be answered, generate at least two candidate answers, and send the candidate answers to the cloud device; wherein the first language model is used to predict candidate answers to the question to be answered in a character-by-character progressive manner, and the candidate answers refer to answers that have not been verified for accuracy; The target answer generation module is used to use the cloud device to call the second largest language model, based on the semantics of the question to be answered, to verify the accuracy of the at least two candidate answers, to determine the target answer, and to return the target answer to the client device.
8. An electronic device, It is characterized in that The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the question-answering method based on a large language model as described in any one of claims 1 to 6.
9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the question-answering method based on a large language model according to any one of claims 1 to 6 when executed.
10. A computer program product, It is characterized in that It comprises a computer program, which, when executed by a processor, implements the question-answering method based on a large language model as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Voice recognition method and device
CN112562673A
Voice information processing method and device thereof, storage medium and electronic device
CN113889116A
Data transmission method, device, system and equipment and storage medium
CN116938974A
Knowledge question and answer method, device and equipment and readable storage medium
CN117786068A
Interaction method, device and equipment based on large language model and readable storage medium
CN118363501A
Cited By
Post candidate information management method, system and equipment applying artificial intelligence
CN119938917A
A method, system, and device for managing job candidate information using artificial intelligence.
CN119938917B
Dynamic optimization-based question and answer big language model cluster collaborative question and answer method, system and equipment and medium
CN120336496A
Method and system for network query construction using retrieval-augmented generation (RAG), and applications thereof
US12626147B1
