Question answering method and device based on large language model, equipment and storage medium

Abstract

The invention provides a question answering method based on a large language model, and the method comprises the steps: carrying out the text similarity retrieval of a question proposed by a user in an internal knowledge base, and obtaining the answer content of an internal large language model based on a retrieval result. Meanwhile, the output content of the external large language model is obtained through the problem subjected to data desensitization, and the problem that internal sensitive information is leaked due to the fact that the problem is directly input into the external large language model is avoided. And calculating the text similarity coefficient of the output content of the external large language model and the text blocks in the internal knowledge base, and after multiple answer contents are obtained through circulation for multiple times, summarizing and answering by synthesizing the answer contents through the internal large language model. The quality and accuracy of the answer content output by the knowledge base question-answering system can be improved, meanwhile, leakage of internal sensitive information caused by directly using an external large model to assist an internal knowledge base in outputting the answer content is effectively avoided, and the information safety risk is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of large language models, and specifically relates to a question answering method, apparatus, device and storage medium based on a large language model. Background Art

[0002] As an important branch of big models, big language models are gradually changing our understanding and use of artificial intelligence due to their powerful language understanding and generation capabilities. With the advancement of technology, their potential in various fields will be more deeply explored and utilized.

[0003] For example, when managing a large number of internal documents, enterprises and institutions are currently faced with the challenge of how to efficiently use this information to support daily decision-making and business operations. As an intelligent information retrieval tool based on a large language model, the knowledge base question-answering system within the enterprise can convert natural language questions into accurate knowledge queries, providing great convenience for various units in managing internal documents.

[0004] However, the inventors have found that the existing knowledge base question-answering system within an enterprise has the following problems in practical application:

[0005] On the one hand, due to limited resource investment, the large language models deployed within enterprises and institutions are often not as advanced as external commercial large language models in terms of language understanding, reasoning and generation capabilities, resulting in low quality and poor accuracy of the answers output by the knowledge base question-answering system based on the large language model within the enterprise. On the other hand, if the external large language model is used to assist in the output of the answer content, although it may make up for the lack of high quality of the output content of the large language model, there is a risk of unintentional leakage of internal sensitive information during the interaction process, which seriously threatens the information security of enterprises and institutions. Summary of the invention

[0006] The purpose of the present invention is to solve the above-mentioned technical problems and provide a question answering method based on a large language model, which can improve the quality and accuracy of the answer content output by the internal knowledge base, while effectively preventing the leakage of internal sensitive information and reducing information security risks.

[0007] In order to solve the above problems, the present invention is implemented according to the following technical solutions:

[0008] In a first aspect, the present invention provides a question answering method based on a large language model, the method comprising:

[0009] S10, obtaining original question data input by the user;

[0010] S20, inputting the original question data into an internal knowledge base for retrieval to obtain retrieval result data, wherein the retrieval result data is answer data related to the original question data stored in the internal knowledge base;

[0011] S30, based on the search result data, obtaining result data output by the internal large language model for the original question data as the first answer content;

[0012] S40, inputting the original question data into an internal information desensitization model to perform data desensitization, and obtaining a sensitive word mapping relationship table and desensitized question data;

[0013] S50, obtaining result data output by the external large language model for the desensitized question data as the second answer content;

[0014] S60, based on the text similarity, respectively calculating the text similarity coefficients between the second answer content and each text block in the internal knowledge base;

[0015] S70, when the number of text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold is greater than or equal to 1, taking the text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold as the first reference answer data, obtaining the result output by the internal large language model for the original question data according to the first reference answer data, and taking the result as the third answer content; when the number of text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold is 0, taking the second answer content as the third answer content;

[0016] S80, repeating steps S50, S60 and S70 in sequence according to a preset number of executions to obtain a plurality of third answer contents;

[0017] S90, based on the sensitive word mapping relationship table obtained in step S40, restore the desensitized data of all third answer contents obtained in step S80;

[0018] S100, taking the first answer content and all the third answer contents after desensitizing the data as the second reference answer data, and obtaining the internal large language model to output a summary answer content for the original question data according to the second reference answer data.

[0019] Preferably, the method steps of inputting the original question data into an internal information desensitizing model for data desensitization to obtain the desensitized question data include: identifying sensitive words on the text of the original question data, and when the text contains the sensitive words, replacing the characters of the sensitive words to obtain desensitized words; wherein the sensitive words include citizens' personal information.

[0020] Preferably, sensitive words in the text of the original question data are identified by word segmentation technology or fuzzy matching technology.

[0021] Preferably, the method step of inputting the original question data into an internal information desensitizing model for data desensitization to obtain the desensitized question data also includes: identifying other sensitive words in the text of the original question data through an internal large language model, and when the text contains other sensitive words, replacing the characters of the other sensitive words to obtain other desensitized words; wherein the other sensitive words are sensitive information related to the internal part of the enterprise other than personal information of citizens.

[0022] Preferably, the original question data is input into an internal information desensitizing model for data desensitization, and the method steps for obtaining a sensitive word mapping relationship table include: when the text of the original question data contains the sensitive words and the other sensitive words, outputting a mapping relationship table between the sensitive words and the desensitized words, and between the other sensitive words and the other desensitized words.

[0023] In a second aspect, the present invention provides a question answering device based on a large language model, wherein the question answering device based on a large language model is configured to execute the question answering method based on a large language model, and the question answering device based on a large language model comprises:

[0024] An original question data acquisition module, wherein the original question data acquisition module is used to acquire original question data input by a user;

[0025] A retrieval result data acquisition module, the retrieval result data acquisition module is used to input the original question data into the internal knowledge base for retrieval to obtain retrieval result data, wherein the retrieval result data is answer data related to the original question data stored in the internal knowledge base;

[0026] A first answer content acquisition module, the first answer content acquisition module is used to obtain result data output by the internal large language model for the original question data based on the search result data as the first answer content;

[0027] The data desensitization module is used to input the original question data into the internal information desensitization model to perform data desensitization, and obtain a sensitive word mapping relationship table and desensitized question data;

[0028] A second answer content acquisition module, the second answer content acquisition module is used to obtain result data output by the external large language model for the desensitized question data as the second answer content;

[0029] A text similarity coefficient calculation module, the text similarity coefficient calculation module is used to calculate the text similarity coefficients between the second answer content and each text block in the internal knowledge base based on text similarity;

[0030] A screening module, wherein when the number of text blocks whose text similarity coefficients satisfy a preset text similarity coefficient threshold is greater than or equal to 1, the text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold are used as first reference answer data, and a result output by the internal large language model for the original question data according to the first reference answer data is obtained, and the result is used as a third answer content; when the number of text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold is 0, the second answer content is used as a third answer content;

[0031] A loop module, the loop module is used to repeatedly execute steps S50, S60 and S70 in sequence according to a preset number of executions to obtain a plurality of third answer contents;

[0032] A desensitized data recovery module, the desensitized data recovery module is used to recover the desensitized data of all third answer contents obtained by the circulation module based on the sensitive word mapping relationship table obtained by the data desensitization module;

[0033] A summary answer acquisition module is used to use the first answer content and all the third answer contents after desensitizing the data as second reference answer data, and obtain the internal large language model to output the summary answer content for the original question data based on the second reference answer data.

[0034] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0035] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a question answering method based on a large language model as described in any one of the first aspects above.

[0036] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is used to enable a processor to execute a question answering method based on a large language model as described in any one of the first aspects above.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention provides a method for answering questions based on a large language model, inputs questions raised by internal users of an enterprise into an internal knowledge base for retrieval, obtains multiple possible matching answer contents of the internal knowledge base, uses these answer contents as reference answers, and outputs the answer contents in the internal large language. At the same time, sensitive words in questions raised by internal users of an enterprise are replaced with characters through data desensitization, and the questions after replacing sensitive words are input into an external large language model to obtain the output contents of the external large language model, thereby avoiding the problem of internal sensitive information leakage caused by directly inputting questions raised by users into an external large language model. Next, by analyzing and calculating the text similarity coefficient between the output contents of the external large language model and each text block in the internal knowledge base, after multiple cycles to obtain multiple answer contents, the internal large language model is used to synthesize these answer contents to make a summary answer.

[0038] The large language model-based question answering method of the present invention can improve the quality and accuracy of the answer content output by the knowledge base question answering system, and at the same time effectively avoid the leakage of internal sensitive information caused by directly using an external large model to assist the internal knowledge base in outputting the answer content, thereby reducing information security risks.

[0039] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings, wherein:

[0041] Figure 1 is a structural diagram of a question answering method based on a large language model according to an embodiment of the present invention;

[0042] Figure 2 is an implementation flow chart of a question answering method based on a large language model according to an embodiment of the present invention;

[0043] Figure 3 is a module diagram of a question answering device based on a large language model according to an embodiment of the present invention;

[0044] Figure 4 It is a structural schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0046] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. Unless otherwise defined, the technical terms or scientific terms used in this specification should be understood by people with ordinary skills in the field to which this application belongs. The words "first", "second" and similar words used in this specification and claims do not indicate any order, quantity or importance, but are only used to distinguish different technical features.

[0047] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0048] Figure 1 is a structural diagram of a question answering method based on a large language model according to the present invention. The method can be executed by a question answering device based on a large language model. The device can be implemented in the form of hardware and / or software, and the device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0049] Step 101: Obtain original question data input by the user.

[0050] It should be noted that the original question data is the question that the user inputs in the internal knowledge base and wants to retrieve the answer. The internal knowledge base is a system or platform used to store, manage and share knowledge resources within an organization (such as an enterprise).

[0051] Step 102: input the original question data into the internal knowledge base for retrieval to obtain retrieval result data, wherein the retrieval result data is answer data related to the original question data stored in the internal knowledge base.

[0052] Step 103: Based on the search result data, obtain the result data output by the internal large language model for the original question data as the first answer content.

[0053] It should be noted that a large language model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. The large language model can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc., and is an important path to artificial intelligence. In the present invention, the internal large language model is a large language model that is only used by internal personnel in a certain unit, company, etc., and its performance is not as good as the external commercial large language model.

[0054] It should be noted that the internal large language model refers to the large language model deployed in the user's local environment (such as a company's internal computer, server, or local network). Since the model runs locally, sensitive data does not need to be uploaded to an external server, which enhances data privacy and security.

[0055] Step 104: Input the original question data into the internal information desensitization model to perform data desensitization, and obtain a sensitive word mapping relationship table and desensitized question data.

[0056] Specifically, the method steps for desensitizing the original question data include: (1) identifying sensitive words in the text of the original question data, and when the text of the original question data contains sensitive words, replacing the characters of the sensitive words to obtain desensitized words; wherein the sensitive words include personal information of citizens. Preferably, the text of the original question is identified with sensitive words by word segmentation technology or fuzzy matching technology. (2) identifying other sensitive words in the text of the original question data by an internal large language model, and when the text of the original question data contains other sensitive words, replacing the characters of other sensitive words to obtain other desensitized words, wherein the other sensitive words are sensitive information related to the internal part of the enterprise other than personal information of citizens, including business secrets (such as product formulas, design drawings, market strategies, customer lists, etc.), corporate financial information (such as financial reports, budgets, audit results, etc.), R&D materials (such as experimental data, technical documents, research results, etc. in the R&D process), supply chain information (such as supplier contact information, logistics data, inventory status, etc.), and other sensitive information related to the internal part of the enterprise. (3) When the text of the original question data contains sensitive words and other sensitive words, output the mapping relationship table between the sensitive words and the desensitized words, and between the other sensitive words and the other desensitized words. It can be understood that the mapping relationship refers to the corresponding relationship between two objects or sets, and the mapping relationship table is a table that records the corresponding relationship between the two objects or sets. In the present invention, the mapping relationship between sensitive words and desensitized words, and between other sensitive words and other desensitized words refers to the corresponding relationship between sensitive words and desensitized words, and the corresponding relationship between other sensitive words and other desensitized words, so that when the subsequent internal large language model makes a summary answer based on multiple answer contents, the desensitized words and other desensitized words can be restored according to the mapping relationship, so that the summary answer content finally input is more accurate. The mapping relationship is as follows:

[0057] Sensitive words Desensitization vocabulary 368468199901010101 3684**********0101 15012312311 150******** Zhang San Zhang Intelligent risk control decision engine system Some type of system The budget is 1.3 million yuan Budget is a certain amount

[0058] It should be noted that word segmentation technology refers to a technology that uses various matching methods to segment words based on the user's keyword string after query processing. Fuzzy matching technology is a data matching technology that compares two or more records and calculates the possibility that they belong to the same entity. An effective fuzzy matching algorithm can handle a range of data ambiguities, such as name / surname reversal, acronyms, abbreviations, phonetic and intentional spelling errors, abbreviations, added / deleted punctuation, etc.

[0059] In the present invention, citizen personal information refers to various information recorded electronically or otherwise that can identify the identity of a specific natural person or reflect the activities of a specific natural person alone or in combination with other information, including name, identity document number, communication contact information, address, account password, property status, whereabouts, etc.

[0060] In one embodiment, if the text of the original question data contains sensitive words, such as the ID number "123456789", after the data desensitization operation, the ID number will be replaced with "123***789" and the ID number "123***789" is a desensitized word. This can effectively prevent the leakage of internal sensitive information when citing an external large language model to assist in outputting the answer content of the question input by the internal user of the enterprise.

[0061] In one embodiment, if the original question data is "Our company intends to purchase a risk control decision intelligent decision engine system with a purchase budget of 1.3 million yuan. According to the purchase management method, what should be the next process?", where "risk control decision intelligent decision engine system" is the other sensitive information, after the data desensitization operation, the desensitized question obtained can be "Our company intends to purchase a certain type of system with a purchase budget of a certain amount. According to the purchase management method, what should be the next process?". That is, after the data is desensitized, other sensitive words are replaced and expressed in a fuzzy semantic way, so that when the external large language model is referenced to assist in outputting the answer content of the question input by the internal user of the enterprise, the leakage of internal sensitive information can be effectively avoided.

[0062] Step 105: Obtain result data output by the external large language model for the desensitized question data as the second answer content.

[0063] In the present invention, the external large language model is a large machine learning model developed by other organizations or companies for processing natural language. If the external large language model is used to directly output answers to questions related to the internal aspects of the enterprise, it is easy to cause the leakage of internal sensitive information, which may lead to the illegal acquisition or abuse of the core business data of the organization, thereby threatening its business secrets and customer privacy.

[0064] Step 106: Based on the text similarity, calculate the text similarity coefficients between the second answer content and each text block in the internal knowledge base.

[0065] It can be understood that text similarity refers to a method to measure the degree of similarity between two or more texts in terms of semantics, structure or content. A text block refers to a question and its answer stored in an internal knowledge base.

[0066] It should be noted that the method for calculating the text similarity coefficient includes but is not limited to converting text data into vector data, and then using the vector data to calculate the text similarity coefficient, such as calculating Euclidean distance, cosine similarity, etc. All methods that can calculate text similarity according to actual needs fall within the scope of protection of this application.

[0067] Step 107: When the number of text blocks whose text similarity coefficients satisfy a preset text similarity coefficient threshold is greater than or equal to 1, the text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold are used as first reference answer data, and the result output by the internal large language model for the original question data based on the first reference answer data is obtained, and the result is used as the third answer content; when the number of text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold is 0, the second answer content is used as the third answer content.

[0068] It should be noted that the preset text similarity coefficient threshold is a preset range of text similarity, and the range can be set according to actual application conditions.

[0069] Step 108: Repeat steps 105, 106 and 107 in sequence according to a preset number of executions to obtain multiple third answer contents.

[0070] It should be noted that the preset number of executions can be set according to actual application needs. In an actual application scenario, for example, step 105, step 106, and step 107 can be repeated 2 times, 3 times, 5 times, etc.

[0071] It is understandable that the result data output by the external large language model for the desensitized question data may not be completely accurate. Therefore, it is necessary to repeat steps 105, 106 and 107 multiple times in sequence according to the preset number of executions, and use the bagging idea of ​​machine learning (Bagging in machine learning is an integrated learning technology used to improve the stability and accuracy of the model. The core idea of ​​bagging is to build multiple models and combine their prediction results to reduce the overfitting of the model and improve the generalization ability), so that the external large language model can output multiple answers to the desensitized question data, and then summarize the multiple answers to finally reach a better answer.

[0072] Step 109: Based on the sensitive word mapping relationship table obtained in step 104, all the third answer contents obtained in step 108 are restored to desensitized data.

[0073] Step 110: Use the first answer content and the third answer content after all the desensitized data is restored as the second reference answer data, and obtain the internal large language model to output the summary answer content for the original question data based on the second reference answer data.

[0074] In summary, the present invention provides a method for answering questions based on a large language model, which searches the questions raised by internal users of an enterprise in the internal knowledge base and the internal large language model constructed within the enterprise, and obtains multiple possible matching answer contents of the internal knowledge base and the content output by the internal large language. At the same time, the sensitive words in the questions raised by internal users of the enterprise are replaced by characters through data desensitization, and the questions after replacing the sensitive words are input into the external large language model to obtain the output content of the external large language model, thereby avoiding the problem of internal sensitive information leakage caused by directly inputting the questions raised by the users into the external large language model. Then, by analyzing the text similarity between the output content of the external large language model and the multiple possible matching answer contents of the internal knowledge base, after multiple cycles to obtain multiple answer contents, the internal large language model is used to synthesize these answer contents to make a summary answer.

[0075] The large language model-based question answering method of the present invention can improve the quality and accuracy of the answer content output by the knowledge base question answering system, and at the same time effectively avoid the leakage of internal sensitive information caused by directly using an external large model to assist the internal knowledge base in outputting the answer content, thereby reducing information security risks.

[0076] Figure 2 is an implementation flow chart of a question answering method based on a large language model according to the present invention. Figure 2 As shown, this embodiment includes:

[0077] When a user asks a question in the company's internal knowledge base, the company's internal knowledge base matches the original question with the existing questions already entered in the internal knowledge base, and displays multiple answers related to the original question on the search result page. Then, the multiple answers displayed on the above result page are used as reference content, and the internal large language model is called to output an answer to the original question based on the reference content as the first answer content; for the original question, data desensitization operations are performed through word segmentation technology, fuzzy matching technology and internal large language model, and all sensitive words related to the company's internal information are desensitized to obtain the desensitized question. At the same time, a mapping relationship table between sensitive words and corresponding desensitized words is generated; then the external large language model is called to output an answer to the desensitized question as the second answer content. Calculate the text similarity coefficient between the second answer content and each text block stored in the internal knowledge base; if the number of text blocks whose text similarity coefficient exceeds the preset text similarity coefficient threshold is greater than or equal to 1, use the text block whose text similarity coefficient exceeds the preset text similarity coefficient threshold as a reference answer, and the internal large language model outputs an answer to the original question based on the reference answer as the third answer content; if the number of first answer contents whose text similarity exceeds the preset text similarity threshold is 0, use the above-mentioned second answer content as the third answer content; according to actual needs, preset the number of times to repeat the above-mentioned operation of generating the third answer content in sequence to obtain multiple third answer contents; finally, according to the above-mentioned mapping relationship table, restore all sensitive words in the obtained multiple third answer contents to the original sensitive words, use the first answer content and the multiple third answer contents after the sensitive words are restored as reference answers, and the internal large language model outputs a summary answer to the original question based on the reference answer.

[0078] Figure 3 is a module diagram of a question answering device based on a large language model according to the present invention. Figure 3 As shown, the module includes:

[0079] 301. Original question data acquisition module.

[0080] Specifically, the original question data acquisition module is used to acquire the original question data input by the user.

[0081] 302. Search result data acquisition module.

[0082] Specifically, the retrieval result data acquisition module is used to input the original question data into the internal knowledge base for retrieval to obtain the retrieval result data, wherein the retrieval result data is the answer data related to the original question data stored in the internal knowledge base.

[0083] 303. First answer content acquisition module.

[0084] Specifically, the first answer content acquisition module is used to obtain the result data output by the internal large language model for the original question data based on the retrieval result data, as the first answer content.

[0085] 304. Data desensitization module.

[0086] Specifically, the desensitized question acquisition module is used to input the original question data into the internal information desensitization model to perform data desensitization, and obtain a sensitive word mapping relationship table and desensitized question data.

[0087] 305. Second answer content acquisition module.

[0088] Specifically, the second answer content acquisition module is used to obtain the result data output by the external large language model for the desensitized question data as the second answer content.

[0089] 306. Text similarity coefficient calculation module.

[0090] Specifically, the text similarity coefficient calculation module is used to calculate the text similarity coefficients between the second answer content and each text block in the internal knowledge base based on the text similarity.

[0091] 307. Screening module.

[0092] Specifically, the screening module is used to use the text blocks whose text similarity coefficients meet the preset text similarity coefficient threshold as the first reference answer data when the number of text blocks whose text similarity coefficients meet the preset text similarity coefficient threshold is greater than or equal to 1, obtain the result output by the internal large language model for the original question data based on the first reference answer data, and use the result as the third answer content; when the number of text blocks whose text similarity coefficients meet the preset text similarity coefficient threshold is 0, use the second answer content as the third answer content.

[0093] 308. Loop module.

[0094] Specifically, the loop module is used to repeatedly execute the second answer content acquisition module, the text similarity coefficient calculation module and the screening module in sequence according to a preset number of executions to obtain multiple third answer contents.

[0095] 309. Desensitized data recovery module.

[0096] Specifically, the desensitized data recovery module is used to restore the desensitized data of all third answer contents obtained by the circulation module based on the sensitive word mapping relationship table obtained by the data desensitization module.

[0097] 310. Summary and answer acquisition module.

[0098] Specifically, the summary answer acquisition module is used to use the first answer content and the third answer content after all desensitized data are restored as the second reference answer data, and obtain the internal large language model to output the summary answer content for the original question data based on the second reference answer data.

[0099] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0100] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0101] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as solving a question answering method based on a large language model.

[0103] In some embodiments, a method for answering questions based on a large language model may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for answering questions based on a large language model described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a method for answering questions based on a large language model in any other appropriate manner (e.g., by means of firmware).

[0104] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0106] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0109] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0110] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements a question answering method based on a large language model as provided in the embodiment of the present invention.

[0111] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0113] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A question answering method based on a large language model, characterized in that: The method comprises: S10, obtaining original question data input by the user; S20, inputting the original question data into an internal knowledge base for retrieval to obtain retrieval result data, wherein the retrieval result data is answer data related to the original question data stored in the internal knowledge base; S30, based on the search result data, obtaining result data output by the internal large language model for the original question data as the first answer content; S40, inputting the original question data into an internal information desensitization model to perform data desensitization, and obtaining a sensitive word mapping relationship table and desensitized question data; S50, obtaining result data output by the external large language model for the desensitized question data as the second answer content; S60, based on the text similarity, respectively calculating the text similarity coefficients between the second answer content and each text block in the internal knowledge base; S70, when the number of text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold is greater than or equal to 1, taking the text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold as the first reference answer data, obtaining the result output by the internal large language model for the original question data according to the first reference answer data, and taking the result as the third answer content; when the number of text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold is 0, taking the second answer content as the third answer content; S80, repeating steps S50, S60 and S70 in sequence according to a preset number of executions to obtain a plurality of third answer contents; S90, based on the sensitive word mapping relationship table obtained in step S40, restore the desensitized data of all third answer contents obtained in step S80; S100, taking the first answer content and all the third answer contents after desensitizing the data as the second reference answer data, and obtaining the internal large language model to output a summary answer content for the original question data according to the second reference answer data.

2. A question answering method based on a large language model according to claim 1, characterized in that: The method steps of inputting the original problem data into the internal information desensitization model to perform data desensitization to obtain the desensitized problem data include: Performing sensitive word recognition on the text of the original question data, and when the text contains the sensitive words, replacing the characters of the sensitive words to obtain desensitized words; Among them, the sensitive words include citizens' personal information.

3. A question answering method based on a large language model according to claim 2, characterized in that: Sensitive words are identified in the text of the original question data through word segmentation technology or fuzzy matching technology.

4. A question answering method based on a large language model according to claim 2, characterized in that: The method step of inputting the original problem data into the internal information desensitization model to perform data desensitization to obtain the desensitized problem data also includes: Using an internal large language model, other sensitive words are identified in the text of the original question data. When other sensitive words are contained in the text, the characters of the other sensitive words are replaced to obtain other desensitized words. Among them, the other sensitive words are sensitive information related to the internal affairs of the enterprise other than personal information of citizens.

5. A question answering method based on a large language model according to claim 4, characterized in that: The original question data is input into the internal information desensitization model for data desensitization, and the method steps for obtaining a sensitive word mapping relationship table include: When the text of the original question data contains the sensitive words and the other sensitive words, a mapping relationship table between the sensitive words and the desensitized words, and between the other sensitive words and the other desensitized words is output.

6. A question answering device based on a large language model, characterized in that: The large language model-based question answering device is configured to execute the large language model-based question answering method, and the large language model-based question answering device includes: An original question data acquisition module, wherein the original question data acquisition module is used to acquire original question data input by a user; A retrieval result data acquisition module, the retrieval result data acquisition module is used to input the original question data into the internal knowledge base for retrieval to obtain retrieval result data, wherein the retrieval result data is answer data related to the original question data stored in the internal knowledge base; A first answer content acquisition module, the first answer content acquisition module is used to obtain result data output by the internal large language model for the original question data based on the search result data as the first answer content; The data desensitization module is used to input the original question data into the internal information desensitization model to perform data desensitization, and obtain a sensitive word mapping relationship table and desensitized question data; A second answer content acquisition module, the second answer content acquisition module is used to obtain result data output by the external large language model for the desensitized question data as the second answer content; A text similarity coefficient calculation module, the text similarity coefficient calculation module is used to calculate the text similarity coefficients between the second answer content and each text block in the internal knowledge base based on text similarity; A screening module, wherein when the number of text blocks whose text similarity coefficients satisfy a preset text similarity coefficient threshold is greater than or equal to 1, the text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold are used as first reference answer data, and a result output by the internal large language model for the original question data according to the first reference answer data is obtained, and the result is used as a third answer content; when the number of text blocks whose text similarity coefficients satisfy the preset text similarity coefficient threshold is 0, the second answer content is used as a third answer content; A loop module, the loop module is used to repeatedly execute steps S50, S60 and S70 in sequence according to a preset number of executions to obtain a plurality of third answer contents; A desensitized data recovery module, the desensitized data recovery module is used to recover the desensitized data of all third answer contents obtained by the circulation module based on the sensitive word mapping relationship table obtained by the data desensitization module; A summary answer acquisition module is used to use the first answer content and all the third answer contents after desensitizing the data as the second reference answer data, and obtain the internal large language model to output the summary answer content for the original question data based on the second reference answer data.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the question answering method based on a large language model as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, The computer program is used to enable a processor to implement a question answering method based on a large language model as described in any one of claims 1 to 5 when executed.

Images