Intelligent question answering system and method based on combination of large language model and NL2SQL

Abstract

The invention relates to an intelligent question and answer system and method based on a large language model in combination with NL2SQL. The system comprises multi-modal input processing, intention recognition, data desensitization, data recall, NL2SQL, compliance check, SQL execution, anti-desensitization and content generation. The multi-mode module is responsible for extracting documents, pictures and audio information and carrying out preprocessing. And the intention recognition module judges whether the NL2SQL module needs to generate an SQL query statement or not, and verifies the accuracy of the SQL query statement. The data desensitization module desensitizes the input by using rules and algorithms. And the data recall module retrieves documents and search results stored by the system. The NL2SQL module converts the user input into an SQL statement. And the SQL execution module runs the SQL statement and returns a result. And the content generation module outputs a final result according to the query result and the recall content. According to the method, content related to user query is obtained through a data recall module (such as sparse vector recall, BM25 recall, search engine recall and the like), and information sources of answers are further enriched.

Description

Technical Field

[0001] The present invention relates to an intelligent question-answering system and method based on a large language model combined with NL2SQL, belonging to the technical field of natural language processing. Background Art

[0002] With the rapid development of information technology, people's demand for obtaining and processing information is increasing. Against this background, intelligent question-answering systems have emerged. Intelligent question-answering systems are an artificial intelligence application that aims to answer natural language questions raised by users by simulating human conversations. This system usually combines technologies such as natural language processing, machine learning, and knowledge graphs to understand the semantics and intent of user questions and find the most relevant answers from massive data.

[0003] Existing intelligent question systems already have certain solutions, such as intelligent question-answering based on word matching, semantic understanding combined with question-answering libraries, and knowledge graphs, but all existing solutions have certain shortcomings, including 1. Limited ability to understand complex questions: At present, intelligent question-answering systems still face challenges in understanding complex questions and handling multi-step reasoning, which can easily lead to misunderstandings or inaccurate answers. 2. Insufficient context understanding: Intelligent question-answering systems have limited ability to understand context, making it difficult to handle complex dialogue situations, which can easily lead to incoherent or inaccurate answers.

[0004] The patent document with the patent number "CN107491485A" discloses a method for generating an execution plan, a planning unit and a distributed NewSQL database system. The problem with this system is that the execution plan is optimized by judging whether there are pre-stored SQL statements corresponding to the SQL statements in the shared cache pool. Although this method can improve query efficiency to a certain extent, it is limited by the number and types of pre-stored SQL statements and may not be able to flexibly respond to various complex query requirements. The system in the reference material can dynamically generate and execute SQL statements through the combination of a large language model and the NL2SQL module, and has higher flexibility and adaptability. It mainly relies on fixed processing flows and pre-stored SQL statements, and lacks intelligence and personalization. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention proposes an intelligent question answering system and method based on a large language model combined with NL2SQL.

[0006] The technical solution of the present invention is as follows:

[0007] On the one hand, the present invention provides an intelligent question-answering system based on a large language model combined with NL2SQL, including a multimodal module, an intent recognition module, a tool module, a data desensitization module, a data recall module, an NL2SQL module, a compliance check module, an SQL execution module, an anti-desensitization module, and a content generation module;

[0008] The multimodal module is used to obtain user input, including input documents, pictures and audio, and perform preprocessing, wherein the preprocessing includes document text extraction, optical character recognition (OCR), and audio-to-text conversion;

[0009] The intention recognition module is used to identify whether the user's input needs to enter the NL2SQL module query and verify the accuracy of the SQL statement generated by the NL2SQL module;

[0010] The data desensitization module includes preset desensitization rules and desensitization algorithms, which are used to desensitize the user's input;

[0011] The data recall module is used to obtain documents, indexes, and tool module historical search results stored in the system;

[0012] The NL2SQL module generates executable SQL statements based on user input;

[0013] The compliance checking module is used to check whether the SQL statement generated by the NL2SQL module is compliant;

[0014] The SQL execution module is used to execute the SQL statement generated by the NL2SQL module to obtain the query result;

[0015] The content generation module outputs the final query result of the system according to the query result of the SQL execution module and the content obtained by the data recall module.

[0016] As a preferred implementation, the intention recognition module performs intention recognition by the following method:

[0017] Divide the user's input into several minimum units, Tokens;

[0018] The vector composed of the probability of the intent type of the nth smallest unit Token is calculated:

[0019]

[0020] Among them, h n is the hidden state of the nth smallest unit Token, Represents the probability of the nth smallest unit token, W I represents the weight matrix, b Irepresents the bias vector, and softmax represents the activation function;

[0021] like If the value is less than the preset value, the NL2SQL module will not be entered. If it is greater than or equal to the preset value, it enters the NL2SQL module and verifies the accuracy of the SQL statement generated by the NL2SQL module by calculating the cross entropy loss value:

[0022]

[0023] Among them, N1 represents the number of intent types, N represents the length of the sequence, and p ic represents the probability that sample i belongs to intent type c, y ic represents the sign function, log(p ic ) represents the normalization of sample weights, Loss I Represents the cross entropy loss value;

[0024] If Loss I If the value is greater than or equal to the preset value, the NL2SQL module is called to regenerate the SQL statement. I If it is less than the preset value, the generated SQL statement is accurate.

[0025] As a preferred implementation, the tool module includes an Internet search tool, a document conversion tool, a voice recognition tool, an OCR tool, a weather query interface, a train schedule query tool, an airline ticket query tool, and a tourist attraction tool.

[0026] As a preferred implementation, the data recall module includes a sparse vector recall module, a BM25 recall submodule, a search engine recall module, and a reordering module;

[0027] The sparse vector recall module searches for content matching the user query statement from the vector library by means of vector matching;

[0028] The BM25 recall submodule searches the index library for the content that matches the document D and the user query statement by segmenting the words;

[0029] The search engine recall module retrieves the content by calling an external search engine;

[0030] The reordering module performs weighted sorting on the candidate contents obtained by the sparse vector recall module, the BM25 recall submodule, and the search engine recall module, and selects the first-ranked contents as the final contents.

[0031] As a preferred implementation, the NL2SQL module uses the open source large model SQLcoder-7b-2 as a base, and uses the open source large model training framework StarCoder to fine-tune the open source large model SQLcoder-7b-2.

[0032] As a preferred implementation, the inspection results of the compliance inspection module include compliance with specifications, recommended improvements, and required improvements.

[0033] As a preferred implementation, the sparse vector recall module vector matching method is:

[0034]

[0035] Wherein, v1 and v2 represent the vectors whose similarity is to be calculated, and Cosine Similarity(v1, v2) represents the vector matching result;

[0036] If the vector matching result is greater than or equal to a preset value, the corresponding content is marked as candidate content.

[0037] As a preferred implementation, the method for word matching of the BM25 recall submodule is:

[0038]

[0039] Among them, qi represents each word in the query word q, f(q i , D) represents the number of times the query word qi appears in document D, avgdl represents the average length of all documents, IDF(q i ) represents the inverse document frequency, k1 and b represent the preset full weights, and BM25(qD) represents the matching result between the query word q and the document D;

[0040] IDF(q i ) is calculated as:

[0041]

[0042] Where N represents the total number, and n represents the nth one in the total number N;

[0043] If the word matching result is greater than or equal to the preset value, the corresponding content is marked as candidate content.

[0044] On the other hand, the present invention also provides an intelligent question answering method based on a large language model combined with NL2SQL, comprising the following steps:

[0045] Constructing a multimodal module, the multimodal module is used to obtain user input, including input documents, pictures and audio, and perform preprocessing, the preprocessing including document text extraction, optical character recognition (OCR), and audio-to-text conversion;

[0046] Construct an intention recognition module, which is used to identify whether the user's input needs to enter the NL2SQL module for query. If it needs to enter the NL2SQL module, verify the accuracy of the SQL statement generated by the NL2SQL module;

[0047] Constructing a tool module, wherein the tool module includes an Internet search tool, a document conversion tool, a voice recognition tool, an OCR tool, a weather query interface, a train schedule query tool, an airline ticket query tool, and a tourist attraction tool;

[0048] Constructing a data desensitization module, wherein the data desensitization module includes preset desensitization rules and desensitization algorithms;

[0049] Constructing an anti-desensitization module, which restores the data according to the desensitization method used by the data desensitization module;

[0050] Constructing a data recall module, which is used to obtain documents, indexes, and tool module historical search results saved in the system;

[0051] Constructing a NL2SQL module, wherein the NL2SQL module generates executable SQL statements according to user input;

[0052] Construct a compliance checking module, which is used to check whether the SQL statements generated by the NL2SQL module are compliant;

[0053] Constructing a SQL execution module, wherein the SQL execution module is used to execute the SQL statement generated by the NL2SQL module to obtain the query result;

[0054] Constructing a content generation module, which outputs the final query result of the system according to the query result of the SQL execution module and the content obtained by the data recall module;

[0055] The multimodal module obtains user input, preprocesses the user input through the tool module, and transmits the preprocessed input to the intention recognition module to determine whether it is necessary to enter the NL2SQL module;

[0056] If it is necessary to enter the NL2SQL module, the user inputs the NL2SQL module to generate SQL statements, and the SQL statements are checked for compliance through the compliance check module. If they are not in compliance with the rules, the NL2SQL module is called to regenerate them. If they are in compliance with the rules, the SQL execution module is used, and the query results are passed to the content generation module to generate question and answer results;

[0057] If you do not need to enter the NL2SQL module, directly enter the data desensitization module to desensitize the user input; after desensitization, enter the data recall module to obtain historical records, and call the tool module to search. After the search is completed, the desensitized user input is restored through the anti-desensitization module. After restoration, it is input into the content generation module together with the results obtained from the data recall module to generate question and answer results.

[0058] The present invention has the following beneficial effects:

[0059] The data desensitization, data recall and compliance check modules of the present invention ensure the security and compliance of the data, while improving the efficiency and accuracy of data processing. In addition, the data recall module obtains content related to the user query in a variety of ways (such as sparse vector recall, BM25 recall, search engine recall, etc.), further enriching the information source of the answer. The NL2SQL module can convert the natural language questions input by the user into executable SQL statements, greatly simplifying the user query process. At the same time, the compliance check module and the SQL execution module ensure the compliance and execution efficiency of the SQL statements, thereby improving the overall performance of the question-answering system. The tool module includes a variety of practical tools (such as Internet search tools, document conversion tools, voice recognition tools, etc.), which provide users with more convenient and diverse query methods, further enhancing the practicality and user experience of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a connection diagram of the system modules of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0063] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0064] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0065] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0066] Embodiment 1:

[0067] See also Figure 1 , the present invention provides an intelligent question-answering system based on a large language model combined with NL2SQL, including a multimodal module, an intention recognition module, a tool module, a data desensitization module, a data recall module, an NL2SQL module, a compliance check module, an SQL execution module, an anti-desensitization module and a content generation module;

[0068] The multimodal module is used to obtain user input, including input documents, pictures and audio, and perform preprocessing, wherein the preprocessing includes document text extraction, optical character recognition (OCR), and audio-to-text conversion;

[0069] The intention recognition module is used to identify whether the user's input needs to enter the NL2SQL module query and verify the accuracy of the SQL statement generated by the NL2SQL module;

[0070] The data desensitization module includes preset desensitization rules and desensitization algorithms, which are used to desensitize the user's input;

[0071] The data recall module is used to obtain documents, indexes, and tool module historical search results stored in the system;

[0072] The NL2SQL module generates executable SQL statements based on user input;

[0073] NL2SQL (Natural Language to Structured Query Language) is a technology that allows users to query databases through natural language input and converts these natural language queries into SQL query statements, thereby implementing query operations on the database;

[0074] The compliance checking module is used to check whether the SQL statement generated by the NL2SQL module is compliant;

[0075] The SQL execution module is used to execute the SQL statement generated by the NL2SQL module to obtain the query result;

[0076] The content generation module outputs the final query result of the system according to the query result of the SQL execution module and the content obtained by the data recall module.

[0077] This embodiment is based on the MC-BERT model as a common encoder. At the intent recognition decoding end, the model passes the hidden state of each minimum unit Token through the softmax activation function, predicts the intent type of each minimum unit Token, and selects the intent type with more votes through a voting mechanism. This method can effectively avoid the situation where individual factors cause errors in the predicted intent type, and also reduces the error propagation of the intent information-guided slot filling task, making the model more stable.

[0078] As a preferred implementation, the encoding end of the intention recognition module performs intention recognition by the following method:

[0079] Divide the user's input into several minimum units, Tokens;

[0080] The vector composed of the probability of the intent type of the nth smallest unit Token is calculated:

[0081]

[0082] Among them, h n is the hidden state of the nth smallest unit Token, Represents the probability of the nth smallest unit token, W I represents the weight matrix, b I represents the bias vector, and softmax represents the activation function;

[0083] like If the value is less than the preset value, the NL2SQL module will not be entered. If it is greater than or equal to the preset value, it enters the NL2SQL module and verifies the accuracy of the SQL statement generated by the NL2SQL module by calculating the cross entropy loss value of the encoding end:

[0084]

[0085] Among them, N1 represents the number of intent types, N represents the length of the sequence, and p ic represents the probability that sample i belongs to intent type c, y ic represents the sign function, log(p ic ) represents the normalization of sample weights, Loss I Represents the cross entropy loss value;

[0086] If Loss I If the value is greater than or equal to the preset value, the NL2SQL module is called to regenerate the SQL statement.I If it is less than the preset value, the generated SQL statement is accurate.

[0087] Similarly, the slot filling decoding end also predicts the slot of each minimum unit Token through the softmax activation function. The vector formula and cross entropy loss value of the slot filling decoding end are consistent with the above encoding end and will not be repeated here.

[0088] As a preferred implementation, the tool module includes an Internet search tool, a document conversion tool, a voice recognition tool, an OCR tool, a weather query interface, a train schedule query tool, an airline ticket query tool, and a tourist attraction tool.

[0089] As a preferred implementation, the data recall module includes a sparse vector recall module, a BM25 recall submodule, a search engine recall module, and a reordering module;

[0090] The sparse vector recall module searches for content matching the user query statement from the vector library by means of vector matching;

[0091] The BM25 recall submodule searches the index library for the content that matches the document D and the user query statement by segmenting the words;

[0092] The search engine recall module retrieves the content by calling an external search engine;

[0093] The reordering module performs weighted sorting on the candidate contents obtained by the sparse vector recall module, the BM25 recall submodule, and the search engine recall module, and selects the first-ranked contents as the final contents.

[0094] As a preferred implementation, the NL2SQL module uses the open source large model SQLcoder-7b-2 as a base, and uses the open source large model training framework StarCoder to fine-tune the open source large model SQLcoder-7b-2.

[0095] As a preferred implementation, the inspection results of the compliance inspection module include compliance with specifications, recommended improvements, and required improvements;

[0096] The compliance with the specification means that the SQL statement is a standard query statement, does not involve the associated query of multiple large tables, does not involve dangerous operations such as data deletion;

[0097] The suggested improvements are that SQL involves time-consuming operations, etc.;

[0098] Dangerous operations involving data deletion must be improved;

[0099] As a preferred implementation, the sparse vector recall module vector matching method is:

[0100]

[0101] The function of this module is to convert the user input question text into a vector (embedding), and use the cosine similarity algorithm to query the N paragraph texts with the highest similarity from the existing document vector library (including question-answer pair vectors and document vectors). Here N can be 10.

[0102] Wherein, v1 and v2 represent vectors whose similarity is to be calculated, v1·v2 represents inner product, ||v1|| and ||v2|| represent the norms (modules) of vectors v1 and v2, and Cosine Similarity(v1, v2) represents the vector matching result;

[0103] If the vector matching result is greater than or equal to a preset value, the corresponding content is marked as candidate content.

[0104] As a preferred implementation, the method for word matching of the BM25 recall submodule is:

[0105]

[0106] Among them, qi represents each word in the query word q, f(q i , D) represents the number of times the query word qi appears in document D, avgdl represents the average length of all documents, IDF(q i ) represents the inverse document frequency, k1 and b represent the preset full weights, and BM25(qD) represents the matching result between the query word q and the document D;

[0107] IDF(q i ) is calculated as:

[0108]

[0109] Where N represents the total number, and n represents the nth one in the total number N;

[0110] If the word matching result is greater than or equal to the preset value, the corresponding content is marked as candidate content.

[0111] The SQL check rule types involved in the present invention are divided into the following categories according to the database object type:

[0112] TABLE: Indicates that this rule is applied to table objects.

[0113] SELECT: Indicates that this rule is applied to query statements.

[0114] INSERT: Indicates that this rule applies to insert statements.

[0115] UPDATE: Indicates that this rule applies to update statements.

[0116] DELETE: Indicates that this rule is applied to delete statements.

[0117] DDL: indicates that this rule is used to detect DDL.

[0118] DML: indicates that this rule is for DML detection.

[0119] DQL: Indicates that this rule is for DQL detection.

[0120] INDEX: Indicates that this rule is for testing INDEX creation statements.

[0121] Embodiment 2:

[0122] The present invention also provides an intelligent question answering method based on a large language model combined with NL2SQL, comprising the following steps:

[0123] Constructing a multimodal module, the multimodal module is used to obtain user input, including input documents, pictures and audio, and perform preprocessing, the preprocessing including document text extraction, optical character recognition (OCR), and audio-to-text conversion;

[0124] Construct an intention recognition module, which is used to identify whether the user's input needs to enter the NL2SQL module for query. If it needs to enter the NL2SQL module, verify the accuracy of the SQL statement generated by the NL2SQL module;

[0125] Constructing a tool module, wherein the tool module includes an Internet search tool, a document conversion tool, a voice recognition tool, an OCR tool, a weather query interface, a train schedule query tool, an airline ticket query tool, and a tourist attraction tool;

[0126] Constructing a data desensitization module, wherein the data desensitization module includes preset desensitization rules and desensitization algorithms;

[0127] Constructing an anti-desensitization module, which restores the data according to the desensitization method used by the data desensitization module;

[0128] Constructing a data recall module, which is used to obtain documents, indexes, and tool module historical search results saved in the system;

[0129] Constructing a NL2SQL module, wherein the NL2SQL module generates executable SQL statements according to user input;

[0130] Construct a compliance checking module, which is used to check whether the SQL statements generated by the NL2SQL module are compliant;

[0131] Constructing a SQL execution module, wherein the SQL execution module is used to execute the SQL statement generated by the NL2SQL module to obtain the query result;

[0132] Constructing a content generation module, which outputs the final query result of the system according to the query result of the SQL execution module and the content obtained by the data recall module;

[0133] The multimodal module obtains user input, preprocesses the user input through the tool module, and transmits the preprocessed input to the intention recognition module to determine whether it is necessary to enter the NL2SQL module;

[0134] If it is necessary to enter the NL2SQL module, the user inputs the NL2SQL module to generate SQL statements, and the SQL statements are checked for compliance through the compliance check module. If they are not in compliance with the rules, the NL2SQL module is called to regenerate them. If they are in compliance with the rules, the SQL execution module is used, and the query results are passed to the content generation module to generate question and answer results;

[0135] If you do not need to enter the NL2SQL module, directly enter the data desensitization module to desensitize the user input; after desensitization, enter the data recall module to obtain historical records, and call the tool module to search. After the search is completed, the desensitized user input is restored through the anti-desensitization module. After restoration, it is input into the content generation module together with the results obtained from the data recall module to generate question and answer results.

[0136] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0137] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-ONLy Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk and other media that can store program codes.

[0140] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An intelligent question answering system based on a large language model combined with NL2SQL, characterized in that: It includes multimodal module, intent recognition module, tool module, data desensitization module, data recall module, NL2SQL module, compliance check module, SQL execution module, anti-desensitization module and content generation module; The multimodal module is used to obtain user input, including input documents, pictures and audio, and perform preprocessing, wherein the preprocessing includes document text extraction, optical character recognition (OCR), and audio-to-text conversion; The intention recognition module is used to identify whether the user's input needs to enter the NL2SQL module query and verify the accuracy of the SQL statement generated by the NL2SQL module; The data desensitization module includes preset desensitization rules and desensitization algorithms, which are used to desensitize the user's input; The data recall module is used to obtain documents, indexes, and tool module historical search results stored in the system; The NL2SQL module generates executable SQL statements based on user input; The compliance checking module is used to check whether the SQL statement generated by the NL2SQL module is compliant; The SQL execution module is used to execute the SQL statement generated by the NL2SQL module to obtain the query result; The content generation module outputs the final query result of the system according to the query result of the SQL execution module and the content obtained by the data recall module.

2. According to claim 1, the intelligent question answering system based on a large language model combined with NL2SQL is characterized in that: The intent recognition module performs intent recognition by the following method: Divide the user's input into several minimum units, Tokens; The vector composed of the probability of the intent type of the nth smallest unit Token is calculated: Among them, h n is the hidden state of the nth smallest unit Token, Represents the probability of the nth smallest unit token, W I represents the weight matrix, b I represents the bias vector, and softmax represents the activation function; like If the value is less than the preset value, the NL2SQL module will not be entered. If it is greater than or equal to the preset value, it enters the NL2SQL module and verifies the accuracy of the SQL statement generated by the NL2SQL module by calculating the cross entropy loss value: Among them, N1 represents the number of intent types, N represents the length of the sequence, and p ic represents the probability that sample i belongs to intent type c, y ic represents the sign function, log(p ic ) represents the normalization of sample weights, Loss I Represents the cross entropy loss value; If Loss I If the value is greater than or equal to the preset value, the NL2SQL module is called to regenerate the SQL statement. I If it is less than the preset value, the generated SQL statement is accurate.

3. The intelligent question answering system based on a large language model combined with NL2SQL according to claim 1, characterized in that: The tool modules include an Internet search tool, a document conversion tool, a voice recognition tool, an OCR tool, a weather query interface, a train schedule query tool, an airline ticket query tool, and a tourist attraction tool.

4. The intelligent question answering system based on a large language model combined with NL2SQL according to claim 1, characterized in that: The data recall module includes a sparse vector recall module, a BM25 recall submodule, a search engine recall module and a reordering module; The sparse vector recall module searches for content matching the user query statement from the vector library by means of vector matching; The BM25 recall submodule searches the index library for the content that matches the document D and the user query statement by segmenting the words; The search engine recall module retrieves the content by calling an external search engine; The reordering module performs weighted sorting on the candidate contents obtained by the sparse vector recall module, the BM25 recall submodule, and the search engine recall module, and selects the first-ranked contents as the final contents.

5. The intelligent question answering system based on a large language model combined with NL2SQL according to claim 1, characterized in that: The NL2SQL module uses the open source large model SQLcoder-7b-2 as the base, and uses the open source large model training framework StarCoder to fine-tune the open source large model SQLcoder-7b-2.

6. The intelligent question answering system based on a large language model combined with NL2SQL according to claim 1, characterized in that: The inspection results of the compliance inspection module include compliance with specifications, recommended improvements, and required improvements.

7. The intelligent question-answering system based on a large language model combined with NL2SQL according to claim 4, characterized in that: The vector matching method of the sparse vector recall module is: Wherein, v1 and v2 represent the vectors whose similarity is to be calculated, and Cosine Similarity(v1, v2) represents the vector matching result; If the vector matching result is greater than or equal to a preset value, the corresponding content is marked as candidate content.

8. The intelligent question-answering system based on a large language model combined with NL2SQL according to claim 4, characterized in that: The method of word matching of the BM25 recall submodule is: Among them, qi represents each word in the query word q, f(q i , D) represents the number of times the query word qi appears in document D, avgdl represents the average length of all documents, IDF(q i ) represents the inverse document frequency, k1 and b represent the preset full weights, and BM25(q, D) represents the matching result between the query word q and the document D; IDF(q i ) is calculated as: Where N represents the total number, and n represents the nth one in the total number N; If the word matching result is greater than or equal to the preset value, the corresponding content is marked as candidate content.

9. An intelligent question answering method based on a large language model combined with NL2SQL, characterized in that: The following steps are involved: Constructing a multimodal module, the multimodal module is used to obtain user input, including input documents, pictures and audio, and perform preprocessing, the preprocessing including document text extraction, optical character recognition (OCR), and audio-to-text conversion; Construct an intention recognition module, which is used to identify whether the user's input needs to enter the NL2SQL module for query. If it needs to enter the NL2SQL module, verify the accuracy of the SQL statement generated by the NL2SQL module; Constructing a tool module, wherein the tool module includes an Internet search tool, a document conversion tool, a voice recognition tool, an OCR tool, a weather query interface, a train schedule query tool, an airline ticket query tool, and a tourist attraction tool; Constructing a data desensitization module, wherein the data desensitization module includes preset desensitization rules and desensitization algorithms; Constructing an anti-desensitization module, which restores the data according to the desensitization method used by the data desensitization module; Constructing a data recall module, which is used to obtain documents, indexes, and tool module historical search results saved in the system; Constructing a NL2SQL module, wherein the NL2SQL module generates executable SQL statements according to user input; Construct a compliance checking module, which is used to check whether the SQL statements generated by the NL2SQL module are compliant; Constructing a SQL execution module, wherein the SQL execution module is used to execute the SQL statement generated by the NL2SQL module to obtain the query result; Constructing a content generation module, which outputs the final query result of the system according to the query result of the SQL execution module and the content obtained by the data recall module; The multimodal module obtains user input, preprocesses the user input through the tool module, and transmits the preprocessed input to the intention recognition module to determine whether it is necessary to enter the NL2SQL module; If it is necessary to enter the NL2SQL module, the user inputs the NL2SQL module to generate SQL statements, and the SQL statements are checked for compliance through the compliance check module. If they are not in compliance with the rules, the NL2SQL module is called to regenerate them. If they are in compliance with the rules, the SQL execution module is used, and the query results are passed to the content generation module to generate question and answer results; If you do not need to enter the NL2SQL module, directly enter the data desensitization module to desensitize the user input; After desensitization, the data recall module is entered to obtain historical records, and the tool module is called to search. After the search is completed, the desensitized user input is restored through the anti-desensitization module. After restoration, it is input into the content generation module together with the results obtained from the data recall module to generate question and answer results.

10. The intelligent question answering method based on a large language model combined with NL2SQL according to claim 1, characterized in that: The intent recognition module performs intent recognition by the following method: Divide the user's input into several minimum units, Tokens; The vector composed of the probability of the intent type of the nth smallest unit Token is calculated: Among them, h n is the hidden state of the nth smallest unit Token, Represents the probability of the nth smallest unit token, W I represents the weight matrix, b I represents the bias vector, and softmax represents the activation function; like If the value is less than the preset value, the NL2SQL module will not be entered. If it is greater than or equal to the preset value, it enters the NL2SQL module and verifies the accuracy of the SQL statement generated by the NL2SQL module by calculating the cross entropy loss value: Among them, N1 represents the number of intent types, N represents the length of the sequence, and p ic represents the probability that sample i belongs to intent type c, y ic represents the sign function, log(p ic ) represents the normalization of sample weights, Loss I Represents the cross entropy loss value; If Loss I If the value is greater than or equal to the preset value, the NL2SQL module is called to regenerate the SQL statement. I , the generated SQL statement is correct if the default value is used.

Images