Database AI question answering system and method based on large language model and retrieval enhancement
Through a database AI Q&A system combining large language model and vectorized search technology, the limitations of existing systems in intention recognition, key entity extraction and SQL query generation are solved, efficient and accurate database information query is achieved, and users are enhanced Experience and system performance.
Patent Information
- Application Number
- CN202411012782.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-19
AI Technical Summary
The existing database Q&A system based on natural language processing has limitations in intent recognition, key entity extraction, context processing and SQL query generation, resulting in low retrieval efficiency and accuracy, unable to meet the needs of complex query, and lack of dynamic learning and self-optimization. ability.
Adopt a database AI Q&A system based on large language model and vectorized search technology, including natural language processing module, vectorized search module and result verification module. Text data vectorization is used to extract keywords, databases are searched using vector search technology, and combined with The large language model generates natural language answers and verifies the correctness of the SQL query results.
It improves the efficiency and accuracy of database information query, can better understand user intentions, generate complete and accurate SQL queries, enhances the system's dynamic learning and self-optimization capabilities, and improves user experience.
Smart Images

Figure CN118981522A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of database retrieval technology, and in particular to a database AI question-answering system and method based on a large language model and retrieval enhancement. Background Art
[0002] With the rapid development of database technology, users have more and more demands for accessing and querying database information. However, traditional database query methods require users to master complex query languages and structures, which are not user-friendly enough for ordinary users. Database query systems based on natural language have become a hot topic of research. Existing database question-answering systems based on natural language processing still have certain limitations in understanding user intent and generating accurate answers.
[0003] In particular, intent recognition is inaccurate: Existing systems often rely on predefined rules and templates for intent recognition, lack flexibility, and are prone to misjudgment when faced with complex or ambiguous questions. For example, if a user's question contains multiple intents or context dependencies, the existing system may not be able to accurately recognize it.
[0004] Incomplete extraction of key entities: In natural language processing, identifying key entities in user questions is the basis for generating accurate answers. However, existing systems are prone to inaccurate extraction or omission of key entities when dealing with polysemous words, synonyms, or words with similar semantics, resulting in incomplete or incorrect queries.
[0005] Insufficient context processing capabilities: Most existing systems can only handle single-round conversations and cannot effectively use context information for multiple rounds of conversations. This makes it difficult for the system to understand and process the user's intention to ask questions or clarify questions in complex query scenarios.
[0006] Limited ability to generate SQL queries: Generating SQL queries requires accurate understanding of the user's query requirements and converting them into the corresponding database query language. When generating complex SQL queries, existing systems often have logical errors or omissions, and cannot meet the user's actual query needs.
[0007] Lack of dynamic learning and self-optimization: The existing system lacks effective use of user feedback and is unable to self-optimize and dynamically learn through feedback information during user interaction, resulting in slow improvement of system performance and inability to adapt to changing user needs.
[0008] Low retrieval efficiency and accuracy: The existing natural language database question-answering system has certain bottlenecks in efficiency and accuracy when retrieving large-scale data. It is unable to give accurate answers in a short time, affecting the user experience.
[0009] Therefore, a data retrieval system and method with high retrieval efficiency and accuracy is needed. Summary of the invention
[0010] In view of this, the purpose of the present invention is to provide a database AI question-answering system based on a large language model and retrieval enhancement. The system combines natural language processing and vectorized retrieval technology and adds a result verification module. It can efficiently and accurately answer database-related questions raised by users, thereby improving the user's experience of querying database information and ensuring the correctness of the query results.
[0011] In order to achieve the above object, the present invention provides the following technical solutions:
[0012] The database AI question-answering system based on a large language model and retrieval enhancement provided by the present invention includes a natural language processing module, a vectorized retrieval module, and a question-answering generation module;
[0013] The natural language processing module is used to process natural language questions input by users and extract key words;
[0014] The vectorized retrieval module: vectorizes the text data based on the extracted key words, searches in the database through vector retrieval technology, and obtains relevant data;
[0015] The question-answer generation module combines the natural language answers generated by the large language model and the retrieved data to generate the final answer.
[0016] Furthermore, it also includes a result verification module, which is used to verify the correctness of the generated SQL query result and make corrections based on the query result.
[0017] Further, the natural language processing module includes an input processing unit, an intention recognition unit and a question parsing unit;
[0018] The input processing unit is used to receive and pre-process the natural language sentence input by the user;
[0019] The intention recognition unit is used to recognize key words in the sentence input by the user;
[0020] The question parsing unit is used to parse the sentence input by the user into a query suitable for database retrieval.
[0021] Furthermore, the preprocessing in the input processing unit includes noise word removal, punctuation processing and word segmentation processing.
[0022] Further, the vectorized retrieval module includes a vector generation unit and a vector retrieval unit;
[0023] The vector generation unit converts the text data into a vector representation by using an embedding model;
[0024] The vector retrieval unit is used to calculate the similarity between the query vector Q and each vector V in the database, and retrieve relevant data according to the similarity;
[0025] The vector representation is calculated according to the following formula:
[0026] V=Model(T)
[0027] Among them, T is text data, V means converting text into a vector table through a large language model, and Model is a pre-trained model.
[0028] Furthermore, the vector similarity measure is calculated in any one of the following two ways:
[0029] The similarity is calculated according to the following formula:
[0030]
[0031] Where Q represents the query vector; Vi represents the vector in the database;
[0032] or
[0033] EuclideanDistance(Q,V i )=||QV i ||
[0034] Among them, EucliamDistance is the Euclidean distance.
[0035] Further, the question-answer generation module includes a data fusion unit and a natural language output unit;
[0036] The data fusion unit is used to fuse the search results obtained through the large language model and the results obtained through the SQL query and generate an answer statement that conforms to the natural language;
[0037] The natural language output unit is used to output the generated natural language answer.
[0038] The present invention also provides a question-answering method implemented by the above system, comprising the following steps:
[0039] Obtain input information, and extract key words based on the input information;
[0040] Vectorize text data based on key words;
[0041] Searching in the database by vector search technology to obtain search data;
[0042] Combine the natural language answers generated by the large language model and the retrieval data to generate and output answer information.
[0043] Further, the method further comprises the following steps:
[0044] Verify the correctness of the generated SQL query results based on the response information and make manual corrections if necessary.
[0045] Further, the method further comprises the following steps:
[0046] storing the generated vectors and related metadata in a vector library;
[0047] Build a knowledge base based on vector library.
[0048] The beneficial effects of the present invention are:
[0049] The database AI question-answering system based on a large language model and retrieval enhancement provided by the present invention combines the natural language processing capabilities of the large language model and the vectorized retrieval technology, and can efficiently and accurately answer database-related questions. The system includes a natural language processing module, a vectorized retrieval module, a question-answering generation module, and a result verification module, which can understand the questions raised by users, search related databases, and generate accurate answers, while verifying the correctness of the query results, solving the problems of difficulty in understanding user intentions and low accuracy in generating answers in the prior art.
[0050] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for illustration:
[0052] Figure 1 This is a block diagram of the AI question-answering system based on a large language model and retrieval enhancement database.
[0053] Figure 2 The flowchart of the database AI question answering method based on large language model and retrieval enhancement.
[0054] Figure 3 This is the actual flow chart of the database AI question answering method based on large language model and vectorized retrieval enhancement. DETAILED DESCRIPTION
[0055] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.
[0056] Example 1
[0057] like Figure 1 As shown, the database AI question-answering system based on a large language model and retrieval enhancement provided in this embodiment includes a natural language processing module, a vectorized retrieval module, a question-answering generation module, and a result verification module;
[0058] The natural language processing module is used to process natural language questions input by users, extract key words, and obtain key entities and intentions;
[0059] The natural language processing module in this embodiment includes an input processing unit, an intention recognition unit, and a question parsing unit;
[0060] The input processing unit is used to receive and pre-process natural language sentences input by users, where the sentences are used to express relevant questions that users need to search for;
[0061] The preprocessing includes removing noise words, punctuation processing and word segmentation processing;
[0062] For example, for the input question "How much land area will Ping'an Town provide in 2024?", after preprocessing, the word sequence may be ["Please ask", "2024", "Ping'an Town", "land supply", "area", "is", "how much"];
[0063] The intention recognition unit is used to recognize key words in the sentence input by the user, obtain the user's query intention and key entities according to the key words; perform intention classification and entity recognition through a large language model;
[0064] The large language model in this embodiment can adopt Wenxin Yiyan or Tongyi Qianwen;
[0065] For example, identify the key entities "2024", "Ping'an Town", "Land Supply Data Table" and "Area", and determine that the query intent is "query the land supply area";
[0066] The question parsing unit is used to parse the sentence input by the user into a query suitable for database retrieval;
[0067] Generate preliminary query templates based on the identified intent and key entities;
[0068] For example, the input statement is parsed into an SQL query template "SELECT sum(mj)as mj FROM gd WHEREyear=?and xzmc=?";
[0069] The vectorized retrieval module: vectorizes text data based on the extracted key words, searches in the database through vector retrieval technology, and obtains relevant data; the key entities and intentions retrieved by the user can be obtained based on the extracted key words;
[0070] The vectorized retrieval module in this embodiment includes a vector generation unit and a vector retrieval unit;
[0071] The vector generation unit converts the text data into a vector representation by using an embedding model;
[0072] The embedding model in this embodiment can adopt BERT, Word2Vec; the text data includes DDL, documents, reference SQL queries, etc.;
[0073] The vector representation in this embodiment is calculated according to the following formula:
[0074] V = EmbedModel(T)
[0075] Where T is text data, V is vector representation, and EmbedModel can be a pre-trained embedding model such as BERT and Word2Vec;
[0076] The vector retrieval unit is used to calculate the similarity between the query vector Q and each vector V in the database, and retrieve relevant data according to the similarity;
[0077] This embodiment uses the cosine similarity calculation formula for calculation:
[0078]
[0079] Where Q represents the query vector; V i Represents a vector in the database;
[0080] You can also use the Euclidean distance calculation formula to calculate the similarity:
[0081] EuclideanDistance(Q,V i )=||QV i ||
[0082] The vector generation unit vectorizes the relevant text data to generate a vector V.
[0083] The vector retrieval unit calculates the similarity between the query vector Q and each vector Vi in the database and retrieves relevant data;
[0084] Assume that the query vector Q is the user query vector generated by EmbedModel, and the vector Vi in the database is the vector representation of each document. The most relevant document is obtained by calculating the cosine similarity or Euclidean distance.
[0085] The question-answer generation module combines the natural language answers generated by the large language model and the retrieved data to generate the final answer;
[0086] The question-answer generation module in this embodiment includes a data fusion unit and a natural language output unit;
[0087] The data fusion unit is used to fuse the search results obtained through the large language model and the results obtained through the SQL query and generate an answer statement that conforms to the natural language;
[0088] The data fusion unit provided in this embodiment fuses the search results with the SQL query results to generate a natural language answer.
[0089] The natural language output unit is used to output the generated natural language answer;
[0090] The final answer output by the natural language output unit in this embodiment is: "The land supply area of Ping'an Town in 2024 is XXX square meters";
[0091] The result verification module is used to verify the correctness of the generated SQL query results and perform manual corrections when necessary;
[0092] The result verification module in this embodiment includes a result comparison unit and a manual correction unit;
[0093] The result comparison unit is used to verify the correctness of the query result;
[0094] The manual correction unit is used to modify the query result to obtain the correct result and vectorize the data and save it in the vector library.
[0095] like Figure 2 As shown, the system provided by this embodiment is used by the user in accordance with the following steps:
[0096] 1. User questions:
[0097] The user enters a natural language question, for example: "What is the land area available in Ping'an Town in 2024?"
[0098] 2. Send a search request to the knowledge base:
[0099] The system sends user questions to the knowledge base for retrieval.
[0100] In this embodiment, based on the input content, the system first pre-processes the user question, extracts key entities and intents, and then generates a query vector Q in preparation for vectorized retrieval;
[0101] 3. Knowledge base search:
[0102] The knowledge base uses vectorization technology to store the training set in the vector library;
[0103] Retrieve relevant information and return it to the system.
[0104] In this embodiment, the training data in the knowledge base (such as DDL, documents, reference SQL queries) is converted into vector representation through the embedding model and stored in the vector library. Receive the query vector Q, calculate the similarity between Q and each vector Vi in the vector library, and retrieve relevant information;
[0105] 4. Construct prompt words:
[0106] Based on the retrieved information, the system constructs prompt words suitable for processing by a large language model;
[0107] In this embodiment, the retrieved information is analyzed to extract key information and context; a prompt word template is generated in combination with the user's question; for example, after retrieving the information of the relevant land supply data table, a prompt word is constructed: "Please generate an SQL query statement to query the land supply area of Ping'an Town in 2024";
[0108] 5. Send to the large language model:
[0109] Send the constructed prompt words to the large language model.
[0110] In this embodiment, after receiving the prompt word, the large language model parses the prompt word content to understand the user's query needs; parse the prompt word: the large language model parses the key entities and intentions in the prompt word, such as "2024", "Ping'an Town", "land supply area"; generates an SQL query template based on the parsing results; fills the specific parameters into the query template to generate a complete SQL query statement;
[0111] 6. Generate SQL statements:
[0112] The SQL statements generated by the large language model feedback, such as: "SELECT sum(mj)as mj FROM gd WHEREyear=2024and xzmc='Ping'an Town'";
[0113] In this embodiment, the SQL statement is generated according to the following path: receiving prompt words -> parsing prompt words -> generating query templates -> filling in specific parameters -> generating complete SQL statements;
[0114] For example: prompt word construction:
[0115] System construction prompt: "Please generate an SQL query statement to query the land supply area of Ping'an Town in 2024";
[0116] Send to the large language model:
[0117] Send the prompt word to the large language model for processing.
[0118] Large language model parsing prompt words:
[0119] The large language model parses the prompt words and identifies the key entities "2024", "Ping'an Town", and "Land Supply Area".
[0120] The large language model understands the query intent: Query the land supply area of Ping'an Town in 2024.
[0121] Generate a query template:
[0122] Based on the parsing results, the large language model generates an SQL query template:
[0123] SELECT sum(mj)as mj FROM gd WHERE year=? and xzmc=?
[0125] Fill in specific parameters:
[0126] The large language model fills in the specific parameters into the query template:
[0127] SELECT sum(mj)as mj FROM gd WHERE year=2024and xzmc
[0128] ='Ping An Town';
[0129] Generate a complete SQL query statement:
[0130] The SQL query statement finally generated by the large language model is:
[0131] SELECT sum(mj)as mj FROM gd WHERE year=2024and xzmc
[0132] ='Ping An Town';
[0133] 7. Execute query database:
[0134] The system executes the generated SQL statement to query the database to obtain data results;
[0135] 8. Return data results:
[0136] The system returns the query results to the user.
[0137] 9. Construct statistical charts:
[0138] The system generates statistical charts based on the query results;
[0139] 10. Verify data correctness:
[0140] If the user confirms that the data is correct, the data will be vectorized and saved in the vector library to enrich the knowledge base training set.
[0141] Example 2
[0142] like Figure 3 As shown, the database AI question answering method based on a large language model and vectorized retrieval enhancement provided in this embodiment includes the following steps:
[0143] Obtain input information, and extract key words based on the input information;
[0144] Vectorize text data based on key words;
[0145] Searching in the database by vector search technology to obtain search data;
[0146] Combine the natural language answers generated by the large language model with the search data to generate and output answer information;
[0147] Verify the correctness of the generated SQL query results based on the response information and make manual corrections if necessary.
[0148] The system provided in this embodiment needs to perform data training, which is specifically performed in the following steps:
[0149] 1. Collect training data:
[0150] Collect database DDL, documents, and reference SQL queries as training sets.
[0151] 2. Specific description of vectorized retrieval
[0152] The vectorized retrieval module is one of the core technologies of this system. It converts text data into vector representation and uses vector similarity calculation to retrieve relevant data.
[0153] Vector generation formula:
[0154] Assuming the text data is T, it is converted into a vector representation V through a large language model:
[0155] V=Model(T)
[0156] Among them, Model can be a pre-trained model such as BERT and GPT.
[0157] Vector search formula:
[0158] Relevant data can be retrieved by calculating the similarity between the query vector Q and each vector Vi in the database. Commonly used vector similarity measurement methods include cosine similarity and Euclidean distance.
[0159] Cosine similarity calculation formula:
[0160]
[0161] Euclidean distance calculation formula:
[0162] EuclideanDistance(Q,V i )=||QV i ||3. In the vectorized retrieval module, by calculating the query vector Q and the vectors V in the database i The similarity can determine which data meets the requirements. The following are the specific steps:
[0163] (1) Vector generation
[0164] First, the query question entered by the user and the documents or records in the database are converted into vector representations.
[0165] User query vector Q:
[0166] The user’s natural language query is converted into a vector Q using an embedding model.
[0167] Q = EmbedModel(T query )
[0168] Among them, Tquery is the user's query text.
[0169] Database vector V:
[0170] Use the embedding model to convert each record or document in the database into a vector V i .
[0171] Vi=EmbedModel(T i )
[0172] Among them, Ti is the text data in the database.
[0173] (2) Similarity calculation
[0174] Compute the query vector Q and each database vector V i The common similarity calculation methods include cosine similarity and Euclidean distance.
[0175] Cosine similarity calculation formula:
[0176]
[0177] The result is between 0 and 1, with larger values indicating more similarity between the two vectors.
[0178] Euclidean distance calculation formula:
[0179] EuclideanDistance(Q,V i )=||QV i ||
[0180] The result is a non-negative real number, and the smaller the value, the more similar the two vectors are.
[0181] 4. Determine the search data that meets the requirements
[0182] Based on the similarity calculation results, determine which data meets the retrieval requirements. The following are the specific steps: Set the similarity threshold:
[0183] Set a similarity threshold θ to determine whether the retrieval results meet the requirements.
[0184] For example, when using cosine similarity, a commonly used threshold can be set to 0.8.
[0185] Retrieve the data that meets the requirements:
[0186] Traverse all database vectors V i , calculate their similarity with the query vector Q.
[0187] If the similarity is greater than or equal to the threshold θ, the data is considered to meet the retrieval requirements. Specific steps:
[0188] For each Vi calculate the similarity:
[0189] similarityi=CosineSimilarity(Q,V i )
[0190] Determine whether the threshold condition is met:
[0191] if similarity i ≥θ,then include V i in the results
[0192] 5. Retrieval operation examples
[0193] Assume the following database records and their vector representations:
[0194] Database Record 1: V11
[0195] Database Record 2: V2
[0196] Database Record 3: V3
[0197] The user query vector is QQQ, and the similarity threshold is set to 0.8.
[0198] Calculate similarity:
[0199] similarity1=CosineSimilarity(Q,V1)
[0200] similarity2=CosineSimilarity(Q,V2)
[0201] similarity3=CosineSimilarity(Q,V3)
[0202] Determine whether the requirements are met:
[0203] if similarity1≥0.8, then include V1 in the results
[0204] if similarity2≥0.8, then include V2 in the results
[0205] if similarity3≥0.8, then include V3 in the results
[0206] 6. Search results
[0207] After the above steps, the data that meets the similarity requirements will be retrieved and returned to the user.
[0208] In this way, using similarity calculation to determine the retrieval data that meets the requirements can effectively improve the accuracy and efficiency of the database question answering system.
[0209] 7.Store vectors and metadata:
[0210] The generated vectors and related metadata are stored in a vector library.
[0211] 8. Knowledge base construction:
[0212] Build a knowledge base based on the vector library for users to search and use when asking questions.
[0213] Example 3
[0214] This embodiment provides an enhanced retrieval process during 3D real-scene architectural design planning, which is as follows:
[0215] Query and analyze natural resource data in different regions, such as land use, mineral resources, water resources, etc.; integrate data scattered in different databases to achieve unified query and analysis; improve query efficiency and accuracy, and reduce the cost of users learning complex query languages.
[0216] 1. User questions:
[0217] The user inputs a natural language question, such as: "What is the reserve of the minerals being mined in Fulai Town in 2024?"
[0218] 2. Send a retrieval request to the knowledge base:
[0219] The system sends the user's question to the knowledge base for retrieval; for example, the user inputs: "What is the reserve of the minerals opened in Fulai Town in 2024?"; the system converts the question into a vectorized retrieval request.
[0220] 3. Knowledge base retrieval:
[0221] The knowledge base stores the training set in the vector library through vectorization technology; retrieves relevant information and returns it to the system. The specific processing process: uses an embedding model to convert the user's question and the documents or records in the database into vector representations. Calculates the similarity between the user query vector Q and the database vector Vi. For example, calculates the cosine similarity between the query vector Q and the vector Vi in the vector library to obtain the information most relevant to the query.
[0222] 4. Construct prompt words:
[0223] The system constructs prompt words suitable for processing by the large language model based on the retrieved information. For example, after retrieving information about the relevant land use data table, constructs the prompt word: "Please generate an SQL query statement to query the reserve of the minerals being mined in Fulai Town in 2024."
[0224] 5. Send to the large language model:
[0225] Sends the constructed prompt words to the large language model. The large language model processes the prompt words and generates an SQL query statement. The specific process: inputs the prompt words to the large language model (such as Wenxin Yiyan). The large language model parses the prompt words to understand the query requirements. Based on the prompt words and the internal knowledge of the model, generates the corresponding SQL query statement.
[0226] 6. Generate an SQL query statement:
[0227] The large language model returns the generated SQL statement, such as: "SELECT sum(cl) as cl FROM kc WHERE year=2024 and xzmc='Fulai Town' and status=‘opened’". The specific process: the large language model identifies the tables and fields to be queried based on the prompt words. Generates the corresponding SQL query statement. For example, the SQL statement generated based on the prompt word "large language model" is: "SELECT sum(cl) as cl FROM kc WHERE year=2024 and xzmc='Fulai Town' and status=‘opened’".
[0228] 7. Execute SQL query:
[0229] The system executes the generated SQL query statement to query the database and obtain data results.
[0230] Return data results:
[0231] 8. The system returns the query results to the user.
[0232] Construct statistical charts:
[0233] The system generates statistical charts based on the query results to help users analyze data and make decisions.
[0234] 9. Verify data accuracy:.
[0235] The system verifies the accuracy of the query results. If the data is correct, it is vectorized and stored in the vector library to enrich the knowledge base training set.
[0236] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. A database AI question-answering system based on a large language model and retrieval enhancement, characterized by: Includes natural language processing module, vectorized retrieval module, and question-answer generation module; The natural language processing module is used to process natural language questions input by users and extract key words; The vectorized retrieval module: vectorizes the text data based on the extracted key words, searches in the database through vector retrieval technology, and obtains relevant data; The question-answer generation module combines the natural language answers generated by the large language model and the retrieved data to generate the final answer.
2. The database AI question answering system based on a large language model and retrieval enhancement as claimed in claim 1, characterized in that: It also includes a result verification module, which is used to verify the correctness of the generated query results and make corrections based on the query results.
3. The database AI question answering system based on a large language model and retrieval enhancement as claimed in claim 1, characterized in that: The natural language processing module includes an input processing unit, an intention recognition unit and a question parsing unit; The input processing unit is used to receive and pre-process the natural language sentence input by the user; The intention recognition unit is used to recognize key words in the sentence input by the user; The question parsing unit is used to parse the sentence input by the user into a query suitable for database retrieval.
4. The database AI question answering system based on a large language model and retrieval enhancement as claimed in claim 3, characterized in that: The preprocessing in the input processing unit includes removing noise words, punctuation processing and word segmentation processing.
5. The database AI question answering system based on a large language model and retrieval enhancement as claimed in claim 1, characterized in that: The vectorized retrieval module includes a vector generation unit and a vector retrieval unit; The vector generation unit converts the text data into a vector representation by using an embedding model; The vector retrieval unit is used to calculate the similarity between the query vector Q and each vector V in the database, and retrieve relevant data according to the similarity; The vector representation is calculated according to the following formula: V=Model(T) Among them, T is text data, V means converting text into a vector table through a large language model, and Model is a pre-trained model.
6. The database AI question answering system based on a large language model and retrieval enhancement as claimed in claim 5, characterized in that: The vector similarity measure is calculated in either of the following two ways: The similarity is calculated according to the following formula: Where Q represents the query vector; Vi represents the vector in the database; or EuclideanDistance(Q,V i )=||Q-V i || Among them, EucliamDistance is the Euclidean distance.
7. The database AI question answering system based on a large language model and retrieval enhancement as claimed in claim 1, characterized in that: The question-answer generation module includes a data fusion unit and a natural language output unit; The data fusion unit is used to fuse the search results obtained through the large language model and the results obtained through the SQL query and generate an answer statement that conforms to the natural language; The natural language output unit is used to output the generated natural language answer.
8. A question-answering method implemented by the AI question-answering system based on a large language model and retrieval enhancement database according to any one of claims 1 to 7, characterized in that: The following steps are involved: Obtain input information, and extract key words based on the input information; Vectorize text data based on key words; Searching in the database by vector search technology to obtain search data; Combine the natural language answers generated by the large language model and the retrieval data to generate and output answer information.
9. The database AI question answering method based on large language model and vectorized retrieval enhancement as claimed in claim 8, characterized in that: The following steps are also included: Verify the correctness of the generated SQL query results based on the response information and make manual corrections if necessary.
10. The database AI question answering method based on a large language model and retrieval enhancement as claimed in claim 8, characterized in that: The following steps are also included: storing the generated vectors and related metadata in a vector library; Build a knowledge base based on vector library.
Citation Information
Cited By
Bidding field question and answer method and system based on retrieval enhancement generation architecture
CN119441446A
Data query method, device and equipment based on recall strategy optimization RAG
CN119807227A
Retrieval enhancement generation method and system based on large language model
CN120045750A
A retrieval-augmented generation method and system based on a large language model
CN120045750B
Financial question and answer intelligent planning and retrieval system and method based on big data
CN120067260A
