A method and system for intelligent question and answer generation based on large language model

Abstract

A method and system for generating intelligent questions and answers based on a large language model, which relates to the technical field of data information analysis. The method includes the following steps: obtaining electronic data and storing it in a relational database; training an Embedding Model; generating corresponding first vector data from the training data, and storing the first vector data, a first SQL statement, and the answer content in a vector database; obtaining user questions, vectorizing the user questions through the Embedding Model, obtaining second vector data, and generating prompt words through vector recall; inputting the prompt words into the first large language model, obtaining a second SQL statement, and executing the second SQL statement in the relational database to obtain query results. The present invention reduces the complexity of table design at the data warehousing stage, and stores the processed structured data in relational databases of different topics. Interact with corresponding relational databases for different topics to quickly query, locate, and analyze data.

Description

Technical Field

[0001] The present invention relates to the technical field of data information analysis, and in particular to an intelligent question and answer generation method and system based on a large language model. Background Art

[0002] In recent years, thanks to the rapid development of the Internet and the rapid progress of artificial intelligence technology, especially in 2022, OpenAI's ChatGPT was officially launched, marking the arrival of the era of large artificial intelligence models. This wave not only affects the underlying large models, infrastructure, and machine learning operations, but also profoundly affects and promotes the initial construction of the generative artificial intelligence ecosystem.

[0003] In the field of electronic data analysis, as an important way to handle information technology-related matters, electronic data analysis uses cutting-edge technology and follows strict procedures to comprehensively search electronic systems, extract and analyze relevant data, and provide strong support for the development of related businesses. However, when faced with electronic data, staff need to conduct business modeling and use modeling tools to extract relevant data, including structured and unstructured data, and then conduct in-depth analysis manually. In this process, challenges such as large data volume, scattered information, and easy omission of key information are often faced, which consumes a lot of manpower and time. Therefore, in order to address this pain point in electronic data analysis, simplifying business processes, improving the ability to handle massive electronic data analysis, and achieving efficient and accurate finding of useful clues have become key issues that need to be addressed in this field.

[0004] In the existing Text-to-SQL technology, the prompt word technology of the large language model is usually used to fill the massive data table structure information into the prompt word, generate SQL in combination with user questions, query the relational database, and finally generate results. However, in this method, there are low accuracy rates in generating SQL, weak data hits, and in the face of complex database designs, SQL syntax errors are generated for complex questions. There is also the problem of privacy leakage in some business scenarios. For example, in the process of case investigation, natural language questions are used to obtain relevant information about an object. The returned results will also include information about the object that is not related to the case, or information similar to the object name, leading to privacy leakage. Summary of the invention

[0005] In order to overcome the above problems, the present disclosure provides an intelligent question and answer generation method and system based on a large language model.

[0006] The technical solution of the present disclosure is as follows:

[0007] In a first aspect, the present disclosure provides a method for generating intelligent questions and answers based on a large language model, comprising the following steps:

[0008] Acquire electronic data, the electronic data including the answer content corresponding to the question, pre-process the electronic data to obtain structured data, and store it in a relational database;

[0009] Acquire multiple training data, each of which includes a question, a first SQL statement corresponding to the question, table structure information of a relational database, table index data, and a data dictionary, and train the Embedding Model through the training data; execute the first SQL statement in the relational database to obtain the answer content corresponding to the question;

[0010] Input the training data into the Embedding Model to generate corresponding first vector data, and store the first vector data, the first SQL statement and the answer content in a vector database;

[0011] Obtaining a user question, vectorizing the user question through the Embedding Model to obtain second vector data, performing vector recall on the second vector data, obtaining first vector data closest to the second vector data in the vector database, and obtaining the first SQL statement and answer content corresponding thereto, and generating a prompt word according to the user question, the first SQL statement, the answer content and the prompt word template;

[0012] Inputting the prompt word into the first language model to obtain a second SQL statement, and executing the second SQL statement in the relational database to obtain a query result;

[0013] The query results are inferred and summarized by the second largest language model, and then a response result is output.

[0014] Furthermore, the Embedding Model adopts an improved BERT model, and the improved BERT model includes:

[0015] An input layer, used for inputting the training data;

[0016] Embedding layer, which embeds the training data into word embedding, sentence embedding and position embedding to obtain an input vector;

[0017] A Bert layer converts the input vector into the first vector data;

[0018] The Bert layer includes a position encoder and a multi-layer bidirectional Transformer encoder, and the input vector is encoded by the position encoder and then input into the bidirectional Transformer encoder of the first layer;

[0019] The bidirectional Transformer encoder includes a first sub-connection layer and a second sub-connection layer connected in sequence, the first sub-connection layer includes a multi-head self-attention layer and a normalization layer, the multi-head self-attention layer and the normalization layer use residual connection, the second sub-connection layer includes a feedforward full connection layer and a normalization layer, the feedforward full connection layer and the normalization layer use residual connection; the position encoder uses relative position encoding, and after the relative position encoding is introduced, the multi-head self-attention layer is specifically:

[0020] ;

[0021] in, a ij Score for attention, Q i For the i query vector, K j For the j key vectors, W Q and W K are the linear transformation matrices of the query vector and key vector respectively, r ij is the relative position offset term, d k is the dimension of the key vector;

[0022] An output layer, outputting the first vector data;

[0023] The improved BERT model uses the following loss function:

[0024] ;

[0025] in, is the smoothing factor, y i For the i When the category is the correct category, the predicted probability of the correct category is, y f is the predicted probability of the incorrect class, and y f ≠y i , C is the number of categories, σ i For the i The adjustment factor for each class is used to adjust the loss contribution ratio according to the number of correct classes.

[0026] Furthermore, it includes a plurality of relational databases, each of which corresponds to a different subject;

[0027] After preprocessing, the electronic data are stored in relational databases related to their subject matter;

[0028] When obtaining the user's question, it also includes obtaining the question subject, and the second SQL statement is only executed in the relational database with the same subject as the question subject.

[0029] Furthermore, the electronic data is preprocessed to obtain structured data, specifically:

[0030] Performing data cleaning on the electronic data, including deduplication, missing value processing, format conversion and standardization;

[0031] The electronic data is subjected to data governance, specifically, the generated structured data is made into a single table query.

[0032] Furthermore, a prompt word is generated according to the user's question, the first SQL statement, the answer content and the prompt word template, specifically:

[0033] Fill the user's question, the first SQL statement and the answer content into the prompt word template;

[0034] The prompt word template also includes the role positioning of the large language model, the table structure of the database, the dictionary content of the table, the description information, the query content and the SQL sample;

[0035] The role positioning, database table structure, table dictionary content, and description information are preset information of the relational database.

[0036] Furthermore: when executing the second SQL statement, the SQL statement is directed to be executed to the relational database of the corresponding subject through the dynamic routing function.

[0037] In the second aspect, an intelligent question-answering generation system based on a large language model includes:

[0038] An electronic data preprocessing module, used to obtain electronic data, preprocess the electronic data to obtain structured data, and store it in a relational database; the electronic data includes the answer content corresponding to the question;

[0039] An Embedding Model training module is used to obtain multiple training data, each of which includes a question, a first SQL statement corresponding to the question, table structure information of a relational database, table index data, and a data dictionary. The Embedding Model is trained through the training data; the answer content corresponding to the question can be obtained by executing the first SQL statement in the relational database;

[0040] A vector database generation module, used for inputting the training data into the Embedding Model, generating corresponding first vector data, and storing the first vector data, the first SQL statement and the answer content into the vector database;

[0041] A prompt word generation module is used to obtain a user question, vectorize the user question through the Embedding Model to obtain second vector data, perform vector recall on the second vector data, obtain the first vector data closest to the second vector data in the vector database, obtain the first SQL statement and answer content corresponding to the first SQL statement, and generate a prompt word according to the user question, the first SQL statement, the answer content and the prompt word template;

[0042] An SQL statement generation and query module, used for inputting the prompt word into the first language model to obtain a second SQL statement, and executing the second SQL statement in the relational database to obtain a query result;

[0043] The answer output module infers and summarizes the query results through the second largest language model and then outputs the answer results.

[0044] Furthermore, the Embedding Model adopts an improved BERT model, and the improved BERT model includes:

[0045] An input layer, used for inputting the training data;

[0046] Embedding layer, which embeds the training data into word embedding, sentence embedding and position embedding to obtain an input vector;

[0047] A Bert layer converts the input vector into the first vector data;

[0048] The Bert layer includes a position encoder and a multi-layer bidirectional Transformer encoder, and the input vector is encoded by the position encoder and then input into the bidirectional Transformer encoder of the first layer;

[0049] The bidirectional Transformer encoder includes a first sub-connection layer and a second sub-connection layer connected in sequence, the first sub-connection layer includes a multi-head self-attention layer and a normalization layer, the multi-head self-attention layer and the normalization layer use residual connection, the second sub-connection layer includes a feedforward full connection layer and a normalization layer, the feedforward full connection layer and the normalization layer use residual connection; the position encoder uses relative position encoding, and after the relative position encoding is introduced, the multi-head self-attention layer is specifically:

[0050] ;

[0051] in,a ij Score for attention, Q i For the i query vector, K j For the j key vectors, W Q and W K are the linear transformation matrices of the query vector and key vector respectively, r ij is the relative position offset term, d k is the dimension of the key vector;

[0052] An output layer, outputting the first vector data;

[0053] The improved BERT model uses the following loss function:

[0054] ;

[0055] in, is the smoothing factor, y i For the i When the category is the correct category, the predicted probability of the correct category is, y f is the predicted probability of the incorrect class, and y f ≠y i , C is the number of categories, σ i For the i The adjustment factor for each class is used to adjust the loss contribution ratio according to the number of correct classes.

[0056] Furthermore, it includes a plurality of relational databases, each of which corresponds to a different subject;

[0057] After preprocessing, the electronic data are stored in relational databases related to their subject matter;

[0058] When obtaining the user's question, it also includes obtaining the question subject, and the second SQL statement is only executed in the relational database with the same subject as the question subject.

[0059] Furthermore, the electronic data is preprocessed to obtain structured data, specifically:

[0060] Performing data cleaning on the electronic data, including deduplication, missing value processing, format conversion and standardization;

[0061] Performing data governance on the electronic data, specifically making the generated structured data a single-table query;

[0062] Generate prompt words based on the user's question, the first SQL statement, the answer content and the prompt word template, specifically:

[0063] Fill the user's question, the first SQL statement and the answer content into the prompt word template;

[0064] The prompt word template also includes the role positioning of the large language model, the table structure of the database, the dictionary content of the table, the description information, the query content and the SQL sample;

[0065] The role positioning, database table structure, table dictionary content, and description information are preset information of the relational database.

[0066] The present disclosure has the following beneficial effects:

[0067] In the present disclosure, at the data storage stage, the complexity of table design is reduced as much as possible by designing a wide table and an anti-paradigm design method with redundant fields, and the processed structured data is stored in relational databases of different subjects. When users need to extract specific subject information from a large amount of electronic data, the present disclosure uses a large language model to interact with SQL generation, uses a large language model to convert natural language into SQL statements, interacts with the generated SQL statements with the corresponding relational database for different subjects, quickly queries and locates and analyzes data, and performs queries and summaries.

[0068] This paper adopts the improved BERT model, introduces relative position encoding, directly models the relative distance between words, and improves the sensitivity to word order. At the same time, the loss function of the BERT model is improved, and smoothing factors and adjustment factors are introduced to reduce the risk of overfitting, encourage the model to output a more dispersed probability distribution, and reduce the weight of easy-to-classify samples to focus on learning difficult samples.

[0069] The disclosure can effectively avoid the problem of privacy leakage in the business handling and case investigation of the public security system, because the data of the public security system usually involves the privacy information of citizens. The disclosure avoids the leakage of information unrelated to the business or case by setting up relational databases of different topics. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A flowchart of the steps of the intelligent question-answering generation method based on a large language model of the present invention;

[0071] Figure 2 It is a structural schematic diagram of a relational database of the present invention;

[0072] Figure 3 It is a schematic diagram of the composition structure of the intelligent question and answer generation system based on a large language model of the present invention. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0074] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of some known functions and known components.

[0075] The present disclosure is described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] First, refer to Figure 1 In one embodiment of the present disclosure, a method for generating intelligent questions and answers based on a large language model is provided, comprising the following steps:

[0077] Acquire electronic data, the electronic data including the answer content corresponding to the question, pre-process the electronic data to obtain structured data, and store it in a relational database;

[0078] Acquire multiple training data, each of which includes a question, a first SQL statement corresponding to the question, table structure information of a relational database, table index data, and a data dictionary, and train the Embedding Model through the training data; execute the first SQL statement in the relational database to obtain the answer content corresponding to the question;

[0079] Input the training data into the Embedding Model to generate corresponding first vector data, and store the first vector data, the first SQL statement and the answer content in a vector database;

[0080] Obtaining a user question, vectorizing the user question through the Embedding Model to obtain second vector data, performing vector recall on the second vector data, obtaining first vector data closest to the second vector data in the vector database, and obtaining the first SQL statement and answer content corresponding thereto, and generating a prompt word according to the user question, the first SQL statement, the answer content and the prompt word template;

[0081] Inputting the prompt word into the first language model to obtain a second SQL statement, and executing the second SQL statement in the relational database to obtain a query result;

[0082] The query results are inferred and summarized by the second largest language model, and then a response result is output.

[0083] The first language model and the second language model are pre-trained.

[0084] This embodiment realizes in-depth analysis of data information by combining vector retrieval with large language model technology. In particular, this method is applied to electronic data query in the public security system. Since the public security system has complicated business, various data are involved, query is difficult, and the personnel who perform query operations are not familiar with database operations. This method can effectively improve query efficiency and accuracy.

[0085] In some embodiments, when the training data is text data, a pre-trained word vector model such as Word2Vec, GloVe, or a more advanced Transformer architecture-based model such as BERT's embedding layer is used to obtain the vector representation. The training data is then input in the format required by the model. If the API is used to call the Embedding Model, the data structure of the input data needs to be constructed according to the requirements of the API documentation. For text data, it is necessary to first perform word segmentation processing, and then send it to the embedding model for processing to obtain the corresponding vector.

[0086] For vector databases, you can choose commonly used Milvus, Faiss, etc. These databases are optimized for the storage and retrieval of vector data. For example, Milvus is an open source vector database that supports large-scale vector data storage and efficient similarity retrieval.

[0087] In one embodiment of the present disclosure, the Embedding Model adopts an improved BERT model, and the improved BERT model includes:

[0088] An input layer, used for inputting the training data;

[0089] Embedding layer, which embeds the training data into word embedding, sentence embedding and position embedding to obtain an input vector;

[0090] A Bert layer converts the input vector into the first vector data;

[0091] The Bert layer includes a position encoder and a multi-layer bidirectional Transformer encoder, and the input vector is encoded by the position encoder and then input into the bidirectional Transformer encoder of the first layer;

[0092] The bidirectional Transformer encoder includes a first sub-connection layer and a second sub-connection layer connected in sequence, the first sub-connection layer includes a multi-head self-attention layer and a normalization layer, the multi-head self-attention layer and the normalization layer use residual connection, the second sub-connection layer includes a feedforward full connection layer and a normalization layer, the feedforward full connection layer and the normalization layer use residual connection; the position encoder uses relative position encoding, and after the relative position encoding is introduced, the multi-head self-attention layer is specifically:

[0093] ;

[0094] in, a ij Score for attention, Q i For the i query vector, K j For the j key vectors, W Q and W K are the linear transformation matrices of the query vector and key vector respectively, r ij is the relative position offset term, d k is the dimension of the key vector, T is the transpose of the matrix;

[0095] An output layer, outputting the first vector data;

[0096] The improved BERT model uses the following loss function:

[0097] ;

[0098] in, is the smoothing factor, y i For the i When the category is the correct category, the predicted probability of the correct category is, y f is the predicted probability of the incorrect class, and y f ≠y i , C is the number of categories, σ i For the i The adjustment factor for each class is used to adjust the loss contribution ratio according to the number of correct classes.

[0099] The traditional BERT model uses absolute position encoding, which has limited effect on long texts. The improved BERT model introduces relative position encoding, directly models the relative distance between words, and improves sensitivity to word order.

[0100] The above loss function can avoid overfitting and category imbalance and improve learning efficiency.

[0101] In one embodiment of the present disclosure,

[0102] In order to help users who do not have professional SQL knowledge to interact with the database through daily language, the present disclosure adopts Text-to-SQL technology. However, Text-to-SQL technology has the problem of low accuracy in generating SQL and weak data hit rate. In the face of complex database design, complex problems generate SQL syntax errors. At the same time, for private information, this method will also retrieve information that is not related to this question and answer, which may cause privacy leakage. For example, if you ask about Zhang San's call records, the returned information usually contains call records of people with similar or identical names to Zhang San.

[0103] Therefore, reference Figure 2 , in one embodiment of the present disclosure, includes a plurality of relational databases, each of which corresponds to a different subject;

[0104] After preprocessing, the electronic data are stored in relational databases related to their subject matter;

[0105] When obtaining the user's question, it also includes obtaining the question subject, and the second SQL statement is only executed in the relational database with the same subject as the question subject.

[0106] Specifically, you can use a person’s name or a case name as the subject.

[0107] When the case name is used as the subject, since the SQL statement is only executed in the relational database corresponding to the subject, and the relational database corresponding to the subject only stores electronic data related to the case, the answer will not contain information unrelated to the case, avoiding privacy leakage.

[0108] Similarly, when using a person's name as the subject (since names are highly repetitive, they can be coded, such as adding a combination of name and number, such as Zhang San 001, or using the ID number and seat theme), it can avoid collecting personnel information unrelated to the case or business during case investigation or business handling.

[0109] In the prior art, structured data is usually a paradigm design, which makes data cleaning more difficult. In order to reduce the difficulty of electronic data cleaning, in one embodiment of the present disclosure, the electronic data is preprocessed to obtain structured data, specifically:

[0110] Performing data cleaning on the electronic data, including deduplication, missing value processing, format conversion and standardization;

[0111] The electronic data is subjected to data governance, specifically, the generated structured data is made into a single table query.

[0112] Specifically, single-table design can be achieved through wide tables and field redundancy.

[0113] In one embodiment of the present disclosure, a prompt word is generated according to the user's question, the first SQL statement, the answer content and the prompt word template, specifically:

[0114] Fill the user's question, the first SQL statement and the answer content into the prompt word template;

[0115] The prompt word template also includes the role positioning of the large language model, the table structure of the database, the dictionary content of the table, the description information, the query content and the SQL sample;

[0116] The role positioning, database table structure, table dictionary content, and description information are preset information of the relational database.

[0117] In one implementation of the present disclosure, when executing the second SQL statement, the SQL statement is directed to be executed to the relational database of the corresponding subject through a dynamic routing function.

[0118] In a specific embodiment, the user asks a question: "When A and B are discussing where to travel in winter, who is the friend who is called the most times in WeChat communication?"

[0119] The second SQL statement output by the first language model is:

[0120] SELECT friend_account,communication_in_count FROM dwd_wx_friend WHERE case_id IN(SELECT case_id FROM dwd_collect_record WHERE case_name LIKE '%A and B are talking about where to travel in winter%') ORDER BY communication_in_count DESC LIMIT 1

[0121] Through the database dynamic routing function, the above SQL statement is executed in the relational database and directed to the theme mode related to "tourism" to realize data query. The query results are shown in Table 1:

[0122] Table 1

[0123]

[0124] Based on the query results given in Table 1, they are output to the second largest language model, and the response result is: After querying, when A and B are talking about which travel to go in winter, the friend who is called the most in WeChat communication is Json, with a total of 35 times. For another example, the user asks "query the records of fund records with a collection amount greater than 200 yuan", and the output response result is: there are 6 fund transaction records of the collection type, the amount is more than 200 yuan, and the maximum amount is 15230 yuan.

[0125] Second, reference Figure 3 The present disclosure provides an intelligent question-answering generation system based on a large language model, comprising:

[0126] An electronic data preprocessing module, used to obtain electronic data, preprocess the electronic data to obtain structured data, and store it in a relational database; the electronic data includes the answer content corresponding to the question;

[0127] An Embedding Model training module is used to obtain multiple training data, each of which includes a question, a first SQL statement corresponding to the question, table structure information of a relational database, table index data, and a data dictionary. The Embedding Model is trained through the training data; the answer content corresponding to the question can be obtained by executing the first SQL statement in the relational database;

[0128] A vector database generation module, used for inputting the training data into the Embedding Model, generating corresponding first vector data, and storing the first vector data, the first SQL statement and the answer content into the vector database;

[0129] A prompt word generation module is used to obtain a user question, vectorize the user question through the Embedding Model to obtain second vector data, perform vector recall on the second vector data, obtain the first vector data closest to the second vector data in the vector database, obtain the first SQL statement and answer content corresponding to the first SQL statement, and generate a prompt word according to the user question, the first SQL statement, the answer content and the prompt word template;

[0130] An SQL statement generation and query module, used for inputting the prompt word into the first language model to obtain a second SQL statement, and executing the second SQL statement in the relational database to obtain a query result;

[0131] The answer output module infers and summarizes the query results through the second largest language model and then outputs the answer results.

[0132] In one embodiment of the present disclosure, the Embedding Model adopts an improved BERT model, and the improved BERT model includes:

[0133] An input layer, used for inputting the training data;

[0134] Embedding layer, which embeds the training data into word embedding, sentence embedding and position embedding to obtain an input vector;

[0135] A Bert layer converts the input vector into the first vector data;

[0136] The Bert layer includes a position encoder and a multi-layer bidirectional Transformer encoder, and the input vector is encoded by the position encoder and then input into the bidirectional Transformer encoder of the first layer;

[0137] The bidirectional Transformer encoder includes a first sub-connection layer and a second sub-connection layer connected in sequence, the first sub-connection layer includes a multi-head self-attention layer and a normalization layer, the multi-head self-attention layer and the normalization layer use residual connection, the second sub-connection layer includes a feedforward full connection layer and a normalization layer, the feedforward full connection layer and the normalization layer use residual connection; the position encoder uses relative position encoding, and after the relative position encoding is introduced, the multi-head self-attention layer is specifically:

[0138] ;

[0139] in, a ij Score for attention, Q i For the i query vector, K j For the j key vectors, W Q and W K are the linear transformation matrices of the query vector and key vector respectively, r ij is the relative position offset term, d k is the dimension of the key vector;

[0140] An output layer, outputting the first vector data;

[0141] The improved BERT model uses the following loss function:

[0142] ;

[0143] in, is the smoothing factor, y i For the i When the category is the correct category, the predicted probability of the correct category is, y f is the predicted probability of the incorrect class, and y f ≠y i , C is the number of categories, σ i For the i The adjustment factor for each class is used to adjust the loss contribution ratio according to the number of correct classes.

[0144] In one embodiment of the present disclosure: a plurality of relational databases are included, and each of the relational databases corresponds to a different subject;

[0145] After preprocessing, the electronic data are stored in relational databases related to their subject matter;

[0146] When obtaining the user's question, it also includes obtaining the question subject, and the second SQL statement is only executed in the relational database with the same subject as the question subject.

[0147] In one embodiment of the present disclosure: the electronic data is preprocessed to obtain structured data, specifically:

[0148] Performing data cleaning on the electronic data, including deduplication, missing value processing, format conversion and standardization;

[0149] The electronic data is subjected to data governance, specifically, the generated structured data is made into a single table query.

[0150] In one embodiment of the present disclosure, a prompt word is generated according to the user's question, the first SQL statement, the answer content and the prompt word template, specifically:

[0151] Fill the user's question, the first SQL statement and the answer content into the prompt word template;

[0152] The prompt word template also includes the role positioning of the large language model, the table structure of the database, the dictionary content of the table, the description information, the query content and the SQL sample;

[0153] The role positioning, database table structure, table dictionary content, and description information are preset information of the relational database.

[0154] In one implementation of the present disclosure, when executing the second SQL statement, the SQL statement is directed to be executed to the relational database of the corresponding subject through a dynamic routing function.

[0155] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0156] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0157] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0158] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0159] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0160] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

[0161] There are a few points to note about this disclosure:

[0162] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to the general design.

[0163] (2) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to obtain new embodiments.

[0164] The above descriptions are merely embodiments of the present disclosure and are not intended to limit the patent scope of the present disclosure. Any equivalent structures made using the contents of the present disclosure and the drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present disclosure.

Claims

1. An intelligent question-answering generation method based on a large language model, characterized in that: The following steps are involved: Acquire electronic data, the electronic data including the answer content corresponding to the question, pre-process the electronic data to obtain structured data, and store it in a relational database; Acquire multiple training data, each of which includes a question, a first SQL statement corresponding to the question, table structure information of a relational database, table index data, and a data dictionary, and train the Embedding Model through the training data; Executing the first SQL statement in the relational database can obtain the answer content corresponding to the question; Input the training data into the Embedding Model to generate corresponding first vector data, and store the first vector data, the first SQL statement and the answer content in a vector database; Obtaining a user question, vectorizing the user question through the Embedding Model to obtain second vector data, performing vector recall on the second vector data, obtaining first vector data closest to the second vector data in the vector database, and obtaining the first SQL statement and answer content corresponding thereto, and generating a prompt word according to the user question, the first SQL statement, the answer content and the prompt word template; Inputting the prompt word into the first language model to obtain a second SQL statement, and executing the second SQL statement in the relational database to obtain a query result; The query result is inferred and summarized by the second language model, and then a response result is output; The Embedding Model adopts an improved BERT model, which includes: An input layer, used for inputting the training data; Embedding layer, which embeds the training data into word embedding, sentence embedding and position embedding to obtain an input vector; A Bert layer converts the input vector into the first vector data; The Bert layer includes a position encoder and a multi-layer bidirectional Transformer encoder, and the input vector is encoded by the position encoder and then input into the bidirectional Transformer encoder of the first layer; The bidirectional Transformer encoder includes a first sub-connection layer and a second sub-connection layer connected in sequence, the first sub-connection layer includes a multi-head self-attention layer and a normalization layer, the multi-head self-attention layer and the normalization layer use residual connection, the second sub-connection layer includes a feedforward full connection layer and a normalization layer, the feedforward full connection layer and the normalization layer use residual connection; the position encoder uses relative position encoding, and after the relative position encoding is introduced, the multi-head self-attention layer is specifically: ； in, a ij Score for attention, Q i For the i query vector, K j For the j key vectors, W Q and W K are the linear transformation matrices of the query vector and key vector respectively, r ij is the relative position offset term, d k is the dimension of the key vector; An output layer, outputting the first vector data; The improved BERT model uses the following loss function: ； in, is the smoothing factor, y i For the i When the category is the correct category, the predicted probability of the correct category is, y f is the predicted probability of the incorrect class, and y f ≠y i , C is the number of categories, σ i For the i The adjustment factor for each class is used to adjust the loss contribution ratio according to the number of correct classes.

2. The intelligent question-answering generation method based on a large language model according to claim 1, characterized in that: It includes multiple relational databases, each of which corresponds to a different subject; After preprocessing, the electronic data are stored in relational databases related to their subject matter; When obtaining the user's question, it also includes obtaining the question subject, and the second SQL statement is only executed in the relational database with the same subject as the question subject.

3. The intelligent question-answering generation method based on a large language model according to claim 2, characterized in that: The electronic data is preprocessed to obtain structured data, specifically: Performing data cleaning on the electronic data, including deduplication, missing value processing, format conversion and standardization; The electronic data is subjected to data governance, specifically, the generated structured data is made into a single table query.

4. The intelligent question-answering generation method based on a large language model according to claim 2, characterized in that: Generate prompt words based on the user's question, the first SQL statement, the answer content and the prompt word template, specifically: Fill the user's question, the first SQL statement and the answer content into the prompt word template; The prompt word template also includes the role positioning of the large language model, the table structure of the database, the dictionary content of the table, the description information, the query content and the SQL sample; The role positioning, database table structure, table dictionary content, and description information are preset information of the relational database.

5. The intelligent question-answering generation method based on a large language model according to claim 2, characterized in that: When the second SQL statement is executed, the SQL statement is directed to the relational database of the corresponding subject through the dynamic routing function.

6. An intelligent question-answering generation system based on a large language model, characterized in that: include: An electronic data preprocessing module, used to obtain electronic data, preprocess the electronic data to obtain structured data, and store it in a relational database; the electronic data includes the answer content corresponding to the question; An Embedding Model training module, used to obtain multiple training data, each of which includes a question, a first SQL statement corresponding to the question, table structure information of a relational database, table index data and a data dictionary, and train the Embedding Model through the training data; Executing the first SQL statement in the relational database can obtain the answer content corresponding to the question; A vector database generation module, used for inputting the training data into the Embedding Model, generating corresponding first vector data, and storing the first vector data, the first SQL statement and the answer content into the vector database; A prompt word generation module is used to obtain a user question, vectorize the user question through the Embedding Model to obtain second vector data, perform vector recall on the second vector data, obtain the first vector data closest to the second vector data in the vector database, obtain the first SQL statement and answer content corresponding to the first SQL statement, and generate a prompt word according to the user question, the first SQL statement, the answer content and the prompt word template; An SQL statement generation and query module, used for inputting the prompt word into the first language model to obtain a second SQL statement, and executing the second SQL statement in the relational database to obtain a query result; A response output module, which infers and summarizes the query results through the second language model and then outputs the response results; The Embedding Model adopts an improved BERT model, which includes: An input layer, used for inputting the training data; Embedding layer, which embeds the training data into word embedding, sentence embedding and position embedding to obtain an input vector; A Bert layer converts the input vector into the first vector data; The Bert layer includes a position encoder and a multi-layer bidirectional Transformer encoder, and the input vector is encoded by the position encoder and then input into the bidirectional Transformer encoder of the first layer; The bidirectional Transformer encoder includes a first sub-connection layer and a second sub-connection layer connected in sequence, the first sub-connection layer includes a multi-head self-attention layer and a normalization layer, the multi-head self-attention layer and the normalization layer use residual connection, the second sub-connection layer includes a feedforward full connection layer and a normalization layer, the feedforward full connection layer and the normalization layer use residual connection; the position encoder uses relative position encoding, and after the relative position encoding is introduced, the multi-head self-attention layer is specifically: ； in, a ij Score for attention, Q i For the i query vector, K j For the j key vectors, W Q and W K are the linear transformation matrices of the query vector and key vector respectively, r ij is the relative position offset term, d k is the dimension of the key vector; An output layer, outputting the first vector data; The improved BERT model uses the following loss function: ； in, is the smoothing factor, y i For the i When the category is the correct category, the predicted probability of the correct category is, y f is the predicted probability of the incorrect class, and y f ≠y i , C is the number of categories, σ i For the i The adjustment factor for each class is used to adjust the loss contribution ratio according to the number of correct classes.

7. The intelligent question-answering generation system based on a large language model according to claim 6, characterized in that: It includes multiple relational databases, each of which corresponds to a different subject; After preprocessing, the electronic data are stored in relational databases related to their subject matter; When obtaining the user's question, it also includes obtaining the question subject, and the second SQL statement is only executed in the relational database with the same subject as the question subject.

8. The intelligent question-answering generation system based on a large language model according to claim 7, characterized in that: The electronic data is preprocessed to obtain structured data, specifically: Performing data cleaning on the electronic data, including deduplication, missing value processing, format conversion and standardization; Performing data governance on the electronic data, specifically making the generated structured data a single-table query; Generate prompt words based on the user's question, the first SQL statement, the answer content and the prompt word template, specifically: Fill the user's question, the first SQL statement and the answer content into the prompt word template; The prompt word template also includes the role positioning of the large language model, the table structure of the database, the dictionary content of the table, the description information, the query content and the SQL sample; The role positioning, database table structure, table dictionary content, and description information are preset information of the relational database.

Images