Large model SQL generation method and system based on feedback

Abstract

The invention belongs to the technical field of artificial intelligence, and discloses a large model SQL generation method and system based on feedback. The method comprises the following steps: generating a rewritten standardized query problem Qnew; performing business rule retrieval from the rewritten standardized query problem Qnew; similar question retrieval is carried out on the rewritten standardized query question Qnew; the large language model LLM is used for guiding generation of a database query language SQL or enhancing semantic understanding and automatically feeding back the generated database query language SQL; and judging the correctness of the database query language SQL and feeding back correctness or error. According to the method, user feedback data is stored through the vector database, and a semantic index of'problem-feedback-correct SQL 'is established, so that a system can quickly retrieve similar historical feedback when receiving a new query, and a large model is assisted to generate a more accurate SQL statement.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a method and system for generating large-scale SQL models based on feedback. Background Technology

[0002] Existing technologies for generating natural language SQL based on large models have significant shortcomings. Firstly, user-generated natural language expressions are often ambiguous and vague, with some business terms having similar meanings. For example, a user requesting "query the on-time performance data for flights departing last month" might mean "departure on-time performance" or "departure on-time performance," making it difficult for the large model to accurately understand the user's needs. This results in SQL statements that do not meet expectations, impacting data query and analysis efficiency. Secondly, existing solutions lack effective feedback mechanisms. When users find the generated SQL results inaccurate, this feedback cannot be integrated into the training data. The large model must rely on pre-set training data, making it difficult to continuously optimize the accuracy of SQL generation based on user feedback in real-world applications, leading to poor performance in complex query scenarios. Furthermore, traditional model training relies on periodically retraining model parameters, failing to respond to the latest user feedback in real time. For instance, if a user specifies "last month" as a calendar month rather than a financial month, the strategy cannot be updated until the entire training cycle is complete, resulting in significant lag. Summary of the Invention

[0003] To overcome the problems existing in related technologies, the present invention discloses a feedback-based method and system for generating SQL from a large model. Specifically, it relates to natural language processing and database technology, generating SQL based on the natural language understanding capabilities of a large model, and then improving the accuracy of the generated SQL through user feedback. This method is suitable for scenarios requiring natural language SQL generation, such as data querying and data analysis.

[0004] The technical solution is as follows: A feedback-based method for generating large-scale model SQL, comprising the following steps: S1 utilizes a large language model (LLM) to complete the missing or ambiguous parts of the extracted key elements in the query, and generates a rewritten standardized query question Q_new; key elements include query intent, query object, and time range; S2, retrieve business rules from the rewritten standardized query question Q_new; S3, perform similar question retrieval on the rewritten standardized query question Q_new; S4 transforms the retrieved content into structured prompt words and passes them to the Large Language Model (LLM). Through a multi-head attention mechanism, it captures key information in the prompt words, activates the correct regions, enhances semantic understanding, and influences the output sequence of the subsequent database query language (SQL) to complete the SQL query. S5 automatically provides feedback on the generated database query language SQL; S6 allows users to view the database query language (SQL) and execution results through the system interface, determine the correctness of the SQL query language and provide feedback on whether it is correct or incorrect, and complete manual feedback and correction.

[0005] In step S1, the Large Language Model (LLM) is used to complete the missing or ambiguous parts of the extracted key elements in the query, including: We utilize a large language model (LLM) to perform joint semantic modeling of the current query Q and the historical context H. A multi-head attention mechanism is used to capture implicit information from the historical context, identifying and extracting the following key elements: query intent, query object, and time range. Based on these elements, a query intent classifier and an element detection module are used to identify missing key elements in the query, and inference is performed to complete the query by combining candidate information from the context. The completion process employs a confidence scoring mechanism and semantic consistency judgment to generate a rewritten standardized query question Q_new. The extracted key elements are mapped to tables and fields in the database, and the large model is used to transform these key elements into part of the query conditions.

[0006] In step S2, business rule retrieval is performed from the rewritten standardized query question Q_new, including: using Named Entity Recognition (NER) technology to extract key entities T from the rewritten standardized query question Q_new, using an embedding algorithm to convert key entities T into vector representations, obtaining a key entity vector database, and retrieving a set of business rules S{R} in the key entity vector database whose cosine similarity to key entity T is greater than a threshold, thus completing the business rule retrieval.

[0007] Furthermore, using Named Entity Recognition (NER) technology, key entities T are extracted from the rewritten standardized query question Q_new, including: (1) Text preprocessing, word segmentation and vectorization; Word segmentation: Split Q_new according to Chinese characters or English words to generate a sequence S={w1,w2,…,wn}; Word embedding: Converting each unit into a vector using a pre-trained model; In the formula, The transformed vector, Units generated after splitting Chinese characters or English words; (2) Entity recognition; The annotation scheme uses the BIO / BIOES system to annotate entity boundaries; The sequence labeling model uses the BiLSTM-CRF model, which includes a BiLSTM layer to capture contextual features. And used to capture bidirectional semantic dependencies; where, For the first Contextual features of the unit No. The vector of the unit; For the first Contextual features of the unit; The CRF layer optimizes the label transition probability and avoids invalid label combinations. In optimizing label transition probabilities, the loss function The learning rate is updated to calculate the error between the predicted label and the true label. Used to optimize model parameters, through probability Will Converted into a probability distribution; where, For real labels, Number of units To predict probabilities, For model parameters, For learning rate, Output for model ; The Transformer / BERT model uses a self-attention mechanism to calculate global dependencies, and the output vector is directly input into the classification layer. In the formula, For query, As key, For the expression value, For order, For distance.

[0008] Furthermore, an embedding algorithm is used to transform the key entity T into a vector representation, and the key entity vector database is obtained, including: Step 1: Data preprocessing and entity cleaning; Entity normalization; cleaning special characters and stop words from the text, and standardizing capitalization and abbreviations; Block processing; large text is divided into blocks using a fixed window, with a set text block size. Overlapping windows Then the first The block interval is: ; Step 2, Embedding Model Selection and Vector Generation; Embedding model, mapping entity T to low-dimensional dense vectors. , For the embedded dimension; Knowledge Graph Entity TransE, Triple satisfy The loss function is: In the formula, For the interval hyperparameter, For the head entity, For the relationship, It is a tail entity; Text embedding: Context-sensitive vectors are generated using Transformer encoding; Step 3, vector calculation and standardization; similarity calculation, cosine similarity is... The Euclidean distance is ;in, This is the first vector after transformation. This is the second vector after transformation; the vector is normalized to... , .

[0009] In step S3, a similar question retrieval is performed on the rewritten standardized query question Q_new, including: The rewritten standardized query question Q_new is transformed into a vector representation using an embedding algorithm. A standardized query vector database is then obtained, and a set of historical feedback records S{Q,SQL_S} with a cosine similarity greater than a threshold is retrieved from the standardized query vector database to complete the similarity question retrieval. S{Q,SQL_S} is a subset of the standardized query vector database, which is a set of key-value pairs of historical questions and standardized SQL queries.

[0010] In step S5, the generated database query language SQL is automatically fed back, including: When the query is successfully executed and the set S{Q,SQL_S} obtained from the similar question retrieval is empty, the system combines the rewritten standardized query question Q_new with the corresponding SQL_S into a feedback data item. Then, this record is converted into a vector representation using an embedding algorithm and inserted into the standardized query vector database in step S3 for subsequent similar question matching.

[0011] Furthermore, the structure of the feedback data item is: {Q_new: the standardized query question rewritten by the user, SQL_S: the correct SQL}.

[0012] In step S6, if the feedback result is incorrect, the training data {Q_new,SQL_S} automatically fed back in step S5 will be deleted, the user will manually correct the database query language SQL to SQL_new, and the new training data {Q_new,SQL_new} will be saved.

[0013] Another object of the present invention is to provide a feedback-based large model SQL generation system, which implements the feedback-based large model SQL generation method, and the system includes: The question module is rewritten, and the large language model LLM is used to fill in the missing or ambiguous parts of the key elements extracted in the query, and generate the rewritten standardized query question Q_new; the key elements include query intent, query object, and time range; The business rule retrieval module retrieves business rules from the rewritten standardized query question Q_new. The similar question retrieval module performs similar question retrieval on the rewritten standardized query question Q_new; The SQL generation module is used to convert the retrieved content into structured prompt words and pass them to the Large Language Model (LLM). Through a multi-head attention mechanism, it captures key information in the prompt words, activates the correct regions, enhances semantic understanding, and influences the output sequence of the subsequent database query language (SQL) to complete the database query language (SQL) query. The automatic feedback module automatically provides feedback on the generated database query language SQL. The manual feedback and correction module allows users to view the database query language (SQL) and execution results through the system interface, determine the correctness of the SQL query language, and provide feedback on whether it is correct or incorrect, thus completing manual feedback and correction.

[0014] Furthermore, the feedback-based large-scale SQL generation system is mounted on a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the functions of the feedback-based large-scale SQL generation system.

[0015] Furthermore, the feedback-based large model SQL generation system also includes an information data processing terminal in this embodiment of the invention. The information data processing terminal is used to provide a user input interface to implement the functions of the feedback-based large model SQL generation system when executed on an electronic device.

[0016] Combining all the above technical solutions, the beneficial effects of this invention are as follows: First, addressing the lag inherent in traditional training methods: Traditional model training relies on periodically retraining model parameters, failing to respond in real-time to the latest user feedback. For example, when a user points out that "last month" should specifically refer to a calendar month rather than a financial month, the strategy cannot be updated until the entire training cycle is completed. This invention provides an SQL generation optimization scheme that does not rely on traditional model parameter updates. By storing user feedback data in a vector database and establishing a semantic index of "problem-feedback-correct SQL," the system can quickly retrieve similar historical feedback when receiving a new query, assisting the large model in generating more accurate SQL statements.

[0017] Secondly, traditional BI requires a process involving requirements analysis, development, deployment, and feedback / modification, which is inefficient, costly, and fails to meet users' personalized needs in a timely manner. This invention allows users to obtain their desired data and presentation format through simple dialogue, and supports multi-turn dialogues for follow-up questions or corrections, significantly reducing the cost of using BI and greatly improving efficiency. Existing SQL generation technologies based on large models suffer from problems such as the "large model illusion," low accuracy, and unstable generation results, rendering them largely impractical. The technical solution of this invention can solve these problems through minimal feedback training, significantly improving performance and enhancing the production usability of SQL generation. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure; Figure 1 This is a schematic diagram of the feedback-based large model SQL generation method provided in this embodiment of the invention; Figure 2 This is a flowchart of the feedback-based large model SQL generation method provided in this embodiment of the invention; Figure 3 This is a schematic diagram of the user feedback function interface provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the interface for viewing and editing historical training data provided in an embodiment of the present invention. Detailed Implementation

[0019] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0020] The innovation of this invention lies in the following: By introducing a pre-processing question rewriting module, this invention effectively reduces the difficulty of understanding complex contexts for large models and improves the accuracy of subsequent retrieval and generation; at the same time, the system is designed with an automatic feedback mechanism to dynamically improve training data and realize the automation and continuous optimization of generation capabilities; in addition, it also supports the integration of human feedback to overcome the inherent limitations of large models in specific semantic understanding or structural generation, thereby obtaining higher precision SQL generation results.

[0021] Example 1, as Figure 1 The principle of the feedback-based large model SQL generation method provided in this embodiment of the invention demonstrates the overall process from user input to SQL generation and feedback optimization.

[0022] Specifically, such as Figure 2 As shown, the feedback-based large model SQL generation method provided in this embodiment of the invention includes: S1 utilizes a large language model (LLM) to complete the missing or ambiguous parts of the extracted key elements in the query, and generates a rewritten standardized query question Q_new; key elements include query intent, query object, and time range; S2, retrieve business rules from the rewritten standardized query question Q_new; S3, perform similar question retrieval on the rewritten standardized query question Q_new; S4 transforms the retrieved content into structured prompt words and passes them to the Large Language Model (LLM). Through a multi-head attention mechanism, it captures key information in the prompt words, activates the correct regions, enhances semantic understanding, and influences the output sequence of the subsequent database query language (SQL) to complete the SQL query. S5 automatically provides feedback on the generated database query language SQL; S6 allows users to view the database query language (SQL) and execution results through the system interface, determine the correctness of the SQL query language and provide feedback on whether it is correct or incorrect, and complete manual feedback and correction.

[0023] For example, step S1 specifically includes: question rewriting: using a large language model (LLM) to perform joint semantic modeling on the current query Q and the historical context H, capturing implicit information in the historical context through a multi-head attention mechanism, and identifying and extracting the following key elements: query intent (such as statistics, comparison, filtering, etc.), query object (such as flights, orders, users, etc.), and time range (such as "today", "last week", "March 1st", etc.). Based on the above elements, using a query intent classifier and an element detection module, the missing key elements in the query (such as time range, query object, etc.) are identified, and inference is performed to complete the query by combining candidate information in the context. The completion process uses a confidence scoring mechanism and semantic consistency judgment to ensure the rationality and accuracy of the completed content, and finally generates the rewritten standardized query question Q_new. This standardized query has the characteristics of clear structure, complete semantics, and mapping to database query languages ​​(such as SQL). The extracted key elements are mapped to tables and fields in the database, so that these key elements (conditions and values) can be transformed into part of the query conditions with the help of the large model.

[0024] For example, if a user asks "How many flights were delayed today?", the system recognizes "today" as the current system date, and the standard question is: "Number of delayed flights on June 23, 2025". If a historical question contains "the number of flights on March 1st", and a user asks "what is the flight on-time rate?", the system will identify the implied date as "March 1st", and the standard question will be: "the flight on-time rate on March 1, 2025".

[0025] For example, step S2 specifically includes business rule retrieval: using Named Entity Recognition (NER) technology to extract key entity T from the rewritten standardized query question Q_new, using an embedding algorithm to convert key entity T into a vector representation, obtaining a business rule vector database, and retrieving a set of business rules S{R} in the business rule vector database whose cosine similarity to key entity T is greater than a threshold (e.g., 0.8), thus completing the business rule retrieval; Using Named Entity Recognition (NER) technology, key entities T are extracted from the rewritten standardized query question Q_new, including: (1) Text preprocessing, word segmentation and vectorization; Word segmentation: Split Q_new according to Chinese characters or English words to generate a sequence S={w1,w2,…,wn}; Word embedding: Converting each unit into a vector using a pre-trained model; In the formula, The transformed vector, Units generated after splitting Chinese characters or English words; (2) Entity recognition; The annotation scheme uses the BIO / BIOES system to annotate entity boundaries; The sequence labeling model uses the BiLSTM-CRF model, which includes a BiLSTM layer to capture contextual features. And used to capture bidirectional semantic dependencies; where, For the first Contextual features of the unit No. The vector of the unit; For the first Contextual features of the unit; The CRF layer optimizes the label transition probability and avoids invalid label combinations. In optimizing label transition probabilities, the loss function The learning rate is updated to calculate the error between the predicted label and the true label. Used to optimize model parameters, through probability Will Converted into a probability distribution; where, For real labels, Number of units To predict probabilities, For model parameters, For learning rate, Output for model ; The Transformer / BERT model uses a self-attention mechanism to calculate global dependencies, and the output vector is directly input into the classification layer. In the formula, For query, As key, For the expression value, For order, For distance.

[0026] For example, using an embedding algorithm to convert a key entity T into a vector representation, obtaining a key entity vector database includes: Step 1: Data preprocessing and entity cleaning; Entity normalization; cleaning special characters and stop words from the text, and standardizing capitalization and abbreviations; Block processing; large text is divided into blocks using a fixed window, with a set text block size. Overlapping windows Then the first The block interval is: ; Step 2, Embedding Model Selection and Vector Generation; Embedding model, mapping entity T to low-dimensional dense vectors. , For the embedded dimension; Knowledge Graph Entity TransE, Triple satisfy The loss function is: In the formula, For the interval hyperparameter, For the head entity, For the relationship, It is a tail entity; Text embedding: Context-sensitive vectors are generated using Transformer encoding; Step 3, vector calculation and standardization; similarity calculation, cosine similarity is... The Euclidean distance is ;in, This is the first vector after transformation. This is the second vector after transformation; the vector is normalized to... , To avoid the similarity being affected by excessively large vector lengths in long texts.

[0027] Index building and storage optimization; index type selection; HNSW (Hierarchical Navigable SmallWorld): a multi-level graph structure that supports efficient approximate nearest neighbor search (ANN); IVF (Inverted File Index): accelerates retrieval through clustering, suitable for billions of vectors; key configurations for vector databases; sharding: horizontally partitioning data for distributed storage; replicas: redundant backups to ensure availability, at least 3 replicas. Entity linking and context fusion; hierarchical entity tree, parsing relationships between entities (such as "company-department-employee"), extracting hierarchical information and injecting it into the context; The hybrid retrieval strategy involves user query → entity recognition → recalling relevant entities from the vector database → generating an answer by fusing attribute text.

[0028] Step S3 specifically includes: similar question retrieval: using an embedding algorithm to convert the rewritten standardized query question Q_new into a vector representation, obtaining a standardized query vector database, and retrieving a set of historical feedback records S{Q,SQL_S} with a cosine similarity greater than a threshold (e.g., 0.8) from the standardized query vector database to complete the similar question retrieval; S{Q,SQL_S} is a subset of the standardized query vector database, a set of key-value pairs of historical questions and standardized SQL.

[0029] For example, step S4 specifically includes converting the content retrieved in steps S2 and S3 into structured prompt words and passing them to the Large Language Model (LLM). The LLM captures key information in the prompt words through a multi-head attention mechanism, activates the correct regions, thereby enhancing semantic understanding and influencing the output sequence of the subsequent database query language (SQL), completing the SQL query, and thus improving the accuracy of the SQL query.

[0030] For example, if rule R includes "flight on-time rate calculation needs to exclude canceled flights", then when the Large Language Model (LLM) generates the database query language SQL for flight on-time rate, the relevant content will be added to the SQL query condition clause.

[0031] For example, step S5 specifically includes automatic feedback: when the query in step S4 is successfully executed and the set S{Q,SQL_S} obtained from the similar question retrieval in step S3 is empty, the system combines the rewritten standardized query question Q_new with the corresponding SQL_S into a feedback data item, with the structure: {Q_new: the user-rewritten standardized query question, SQL_S: the correct SQL}. This record is then converted into a vector representation using an embedding algorithm and inserted into the standardized query vector database of step S3 for subsequent similar question matching.

[0032] For example, step S6 specifically includes manual feedback / correction: The user can view the database query language SQL and execution results from step S4 through the system interface. Based on the user's business knowledge and SQL knowledge, the system comprehensively judges the correctness of the SQL syntax and logic, determines the correctness of the database query language SQL, and provides feedback of "correct" or "incorrect". If the feedback result is "incorrect", the system will delete the automatically fed-in training data {Q_new,SQL_S} and allow the user to manually correct the database query language SQL to SQL_new, saving the new training data {Q_new,SQL_new}.

[0033] As can be seen from the above embodiments, the present invention has the following advantages: Real-time optimization: By storing user feedback in a vector database, the accuracy of SQL generation can be improved in real time without retraining the model, solving the lag problem of periodic training of traditional models. Ambiguity resolution: By combining business rule mapping and historical feedback cases, semantic ambiguity in natural language can be effectively handled, such as clarifying the specific business meaning of "normal rate" and significantly improving the generation accuracy of complex queries.

[0034] Closed-loop feedback: Supports both automatic feedback and manual correction modes, allowing users to directly participate in data optimization, making the system more aligned with actual business scenarios and forming a virtuous cycle of "use-feedback-optimization". Highly efficient retrieval: Based on cosine similarity retrieval using a vector database, the retrieval time can be controlled within 50ms when the data volume is ≤100,000 records, ensuring rapid location of similar cases in massive feedback data.

[0035] Example 2: This invention provides a feedback-based large-scale model SQL generation system, which includes: The question module is rewritten, and the large language model LLM is used to fill in the missing or ambiguous parts of the key elements extracted in the query, and generate the rewritten standardized query question Q_new; the key elements include query intent, query object, and time range; The business rule retrieval module retrieves business rules from the rewritten standardized query question Q_new. The similar question retrieval module performs similar question retrieval on the rewritten standardized query question Q_new; The SQL generation module is used to convert the retrieved content into structured prompt words and pass them to the Large Language Model (LLM). Through a multi-head attention mechanism, it captures key information in the prompt words, activates the correct regions, enhances semantic understanding, and influences the output sequence of the subsequent database query language (SQL) to complete the database query language (SQL) query. The automatic feedback module automatically provides feedback on the generated database query language SQL. The manual feedback and correction module allows users to view the database query language (SQL) and execution results through the system interface, determine the correctness of the SQL query language, and provide feedback on whether it is correct or incorrect, thus completing manual feedback and correction.

[0036] Application Example 1: Context-based rewriting problem scenario.

[0037] User Input 1: What is the on-time flight rate today? Rewrite Question 1: On-time flight rate for June 23, 2025; User Input 2: What about yesterday? Rewrite Question 2: Flight on-time performance rate on June 22, 2025; User input 3: Passage success rate; Rewrite question 3: Passage success rate on 2025-06-22.

[0038] Application Example 2: Scenarios based on business rule correction; User input: How many flights are there on March 1st? Error generated without business rules: select COUNT(*) from flight; Extract business entity: number of flights; Business rule: number of flights - refers to the total number of non-reduced flights; Large model generates SQL: select COUNT(*) from flight where is_cut = 0.

[0039] Application Example 3: Scenarios based on historical training data correction.

[0040] User input: How many flights were delayed on March 1st? Business rules: Delayed flights refer to flights that are not reduced or canceled, depart from airports outside the international area, and whose flight mission is a regular flight, extra flight, or charter flight, where the actual departure time (for flights that have not actually taken off, the current time is used) - the scheduled departure time is greater than 15 minutes.

[0041] Error generated when there is no historical training data: SELECT COUNT(*) AS delayed_flights FROM igoc_bi_flight f LEFT JOIN bd_airport a ON f.pod_3code = a.iata_code WHERE f.flight_date = TO_DATE('2025-03-01', 'YYYY-MM-DD') AND f.is_cut = 0 AND f.is_cancel = 0 AND (a.area_type IS NULL OR a.area_type != 'I') AND f.flight_type IN ('regular shift', 'overtime', 'charter flight') AND COALESCE(f.atd, CURRENT_TIMESTAMP) - f.std>COALESCE(a.taxi_time,15) * INTERVAL '1' MINUTE like Figure 3 A diagram illustrating the user feedback function shows the interface and process for users to provide feedback.

[0042] There's an implicit rule in airport flight operations: times need to be rounded to the nearest minute before calculating the time difference, while the general model usually assumes the time difference is simply the subtraction of two times. Therefore, the SQL condition here is incorrect.

[0043] The user corrected the SQL content and saved it as training data. SELECT COUNT(*) AS delayed_flights_count FROM igoc_bi_flight f LEFT JOIN bd_airport a ON f.pod_3code = a.iata_code WHERE f.flight_date = TO_DATE('2025-03-01', 'YYYY-MM-DD') AND f.is_cut = 0 AND f.is_cancel = 0 AND (a.area_type IS NULL OR a.area_type != 'I') AND f.flight_type IN ('regular shift', 'overtime', 'charter flight') AND (TRUNC(COALESCE(f.atd, SYSDATE), 'MI') - TRUNC(f.std, 'MI')) *1440>COALESCE(a.taxi_time, 15); This SQL statement corrects the time difference calculation based on the result generated in the previous step. First, the time is rounded to the nearest minute before the calculation is performed.

[0044] User input: How many flights were delayed on June 1st? Training data retrieved: SELECT COUNT(*) AS delayed_flights_count FROM igoc_bi_flight f LEFT JOIN bd_airport a ON f.pod_3code = a.iata_code WHERE f.flight_date = TO_DATE('2025-03-01', 'YYYY-MM-DD') AND f.is_cut = 0 AND f.is_cancel = 0 AND (a.area_type IS NULL OR a.area_type != 'I') AND f.flight_type IN ('regular shift', 'overtime', 'charter flight') AND (TRUNC(COALESCE(f.atd, SYSDATE), 'MI') - TRUNC(f.std, 'MI')) *1440>COALESCE(a.taxi_time, 15); Generate correct SQL from large models: SELECT COUNT(*) AS delayed_flights_count FROM igoc_bi_flight f LEFT JOIN bd_airport a ON f.pod_3code = a.iata_code WHERE f.flight_date = TO_DATE('2025-06-01', 'YYYY-MM-DD') AND f.is_cut = 0 AND f.is_cancel = 0 AND (a.area_type IS NULL OR a.area_type != 'I') AND f.flight_type IN ('regular shift', 'overtime', 'charter flight') AND (TRUNC(COALESCE(f.atd, SYSDATE), 'MI') - TRUNC(f.std, 'MI')) *1440>COALESCE(a.taxi_time, 15).

[0045] For example, such as Figure 4 Screenshot of the interface for viewing and editing historical training data.

[0046] To further illustrate the effects of the embodiments of the present invention, the following experiments were conducted. Accuracy comparison experiment of different schemes. To verify the effectiveness of the present invention, a comparative experiment was conducted to test the SQL generation accuracy under different auxiliary strategies. The experimental conditions are as follows: Test set: Contains 2000 natural language queries in the civil aviation field. Evaluation index: Semantic matching accuracy of the generated SQL and the manually annotated correct SQL (i.e., the proportion of the generated SQL execution result consistent with the expectation). The comparison schemes are shown in Table 1.

[0047] Table 1 Comparison of Schemes The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A feedback-based method for generating large-scale model SQL, characterized in that, The method includes the following steps: S1 utilizes a large language model (LLM) to complete the missing or ambiguous parts of the extracted key elements in the query, and generates a rewritten standardized query question Q_new; key elements include query intent, query object, and time range; S2, retrieve business rules from the rewritten standardized query question Q_new; S3, perform similar question retrieval on the rewritten standardized query question Q_new; S4 transforms the retrieved content into structured prompt words and passes them to the Large Language Model (LLM). Through a multi-head attention mechanism, it captures key information in the prompt words, activates the correct regions, enhances semantic understanding, and influences the output sequence of the subsequent database query language (SQL) to complete the SQL query. S5 automatically provides feedback on the generated database query language SQL; S6 allows users to view the database query language (SQL) and execution results through the system interface, determine the correctness of the SQL query language and provide feedback on whether it is correct or incorrect, and complete manual feedback and correction.

2. The feedback-based large-scale model SQL generation method according to claim 1, characterized in that, In step S1, the Large Language Model (LLM) is used to complete the missing or ambiguous parts of the extracted key elements in the query, including: We utilize a large language model (LLM) to perform joint semantic modeling of the current query Q and the historical context H. A multi-head attention mechanism is used to capture implicit information from the historical context, identifying and extracting the following key elements: query intent, query object, and time range. Based on these elements, a query intent classifier and an element detection module are used to identify missing key elements in the query, and inference is performed to complete the query by combining candidate information from the context. The completion process employs a confidence scoring mechanism and semantic consistency judgment to generate a rewritten standardized query question Q_new. The extracted key elements are mapped to tables and fields in the database, and the large model is used to transform these key elements into part of the query conditions.

3. The feedback-based large-scale model SQL generation method according to claim 1, characterized in that, In step S2, business rule retrieval is performed from the rewritten standardized query question Q_new, including: using Named Entity Recognition (NER) technology to extract key entities T from the rewritten standardized query question Q_new, using an embedding algorithm to convert key entities T into vector representations, obtaining a key entity vector database, and retrieving a set of business rules S{R} in the key entity vector database whose cosine similarity to key entity T is greater than a threshold, thus completing the business rule retrieval.

4. The feedback-based large-scale model SQL generation method according to claim 3, characterized in that, Using Named Entity Recognition (NER) technology, key entities T are extracted from the rewritten standardized query question Q_new, including: (1) Text preprocessing, word segmentation and vectorization; Word segmentation: Split Q_new according to Chinese characters or English words to generate a sequence S={w1,w2,…,wn}; Word embedding: Converting each unit into a vector using a pre-trained model; In the formula, The transformed vector, Units generated after splitting Chinese characters or English words; (2) Entity recognition; The annotation scheme uses the BIO / BIOES system to annotate entity boundaries; The sequence labeling model uses the BiLSTM-CRF model, which includes a BiLSTM layer to capture contextual features. And used to capture bidirectional semantic dependencies; where, For the first Contextual features of the unit No. The vector of the unit; For the first Contextual features of the unit; The CRF layer optimizes the label transition probability and avoids invalid label combinations. In optimizing label transition probabilities, the loss function The learning rate is updated to calculate the error between the predicted label and the true label. Used to optimize model parameters, through probability Will Converted into a probability distribution; where, For real labels, Number of units To predict probabilities, For model parameters, For learning rate, Output for model ; The Transformer / BERT model uses a self-attention mechanism to calculate global dependencies, and the output vector is directly input into the classification layer. In the formula, For query, As key, For the expression value, For order, For distance.

5. The feedback-based large-model SQL generation method according to claim 3, characterized in that, The key entity T is transformed into a vector representation using an embedding algorithm, and a key entity vector database is obtained, including: Step 1: Data preprocessing and entity cleaning; Entity normalization; cleaning special characters and stop words from the text, and standardizing capitalization and abbreviations; Block processing; large text is divided into blocks using a fixed window, with a set text block size. Overlapping windows Then the first The block interval is: ; Step 2, Embedding Model Selection and Vector Generation; Embedding model, mapping entity T to low-dimensional dense vectors. , For the embedded dimension; Knowledge Graph Entity TransE, Triple satisfy The loss function is: In the formula, For the interval hyperparameter, For the head entity, For the relationship, It is a tail entity; Text embedding: Context-sensitive vectors are generated using Transformer encoding; Step 3, vector calculation and standardization; similarity calculation, cosine similarity is... The Euclidean distance is ;in, This is the first vector after transformation. This is the second vector after transformation; the vector is normalized to... , .

6. The feedback-based large-scale model SQL generation method according to claim 1, characterized in that, In step S3, a similar question retrieval is performed on the rewritten standardized query question Q_new, including: The rewritten standardized query question Q_new is transformed into a vector representation using an embedding algorithm. A standardized query vector database is then obtained, and a set of historical feedback records S{Q,SQL_S} with a cosine similarity greater than a threshold is retrieved from the standardized query vector database to complete the similarity question retrieval. S{Q,SQL_S} is a subset of the standardized query vector database, which is a set of key-value pairs of historical questions and standardized SQL queries.

7. The feedback-based large-scale model SQL generation method according to claim 1, characterized in that, In step S5, the generated database query language SQL is automatically fed back, including: When the query is successfully executed and the set S{Q,SQL_S} obtained from the similar question retrieval is empty, the system combines the rewritten standardized query question Q_new with the corresponding SQL_S into a feedback data item. Then, this record is converted into a vector representation using an embedding algorithm and inserted into the standardized query vector database in step S3 for subsequent similar question matching.

8. The feedback-based large-model SQL generation method according to claim 7, characterized in that, The structure of the feedback data item is: {Q_new: the user-rewritten standardized query question, SQL_S: the correct SQL}.

9. The feedback-based large-scale model SQL generation method according to claim 1, characterized in that, In step S6, if the feedback result is incorrect, the training data {Q_new,SQL_S} automatically fed back in step S5 will be deleted, the user will manually correct the database query language SQL to SQL_new, and the new training data {Q_new,SQL_new} will be saved.

10. A feedback-based large-scale model SQL generation system, characterized in that, The system implements the feedback-based large model SQL generation method according to any one of claims 1-9, and the system comprises: The question module is rewritten, and the large language model LLM is used to fill in the missing or ambiguous parts of the key elements extracted in the query, and generate the rewritten standardized query question Q_new; the key elements include query intent, query object, and time range; The business rule retrieval module retrieves business rules from the rewritten standardized query question Q_new. The similar question retrieval module performs similar question retrieval on the rewritten standardized query question Q_new; The SQL generation module is used to convert the retrieved content into structured prompt words and pass them to the Large Language Model (LLM). Through a multi-head attention mechanism, it captures key information in the prompt words, activates the correct regions, enhances semantic understanding, and influences the output sequence of the subsequent database query language (SQL) to complete the database query language (SQL) query. The automatic feedback module automatically provides feedback on the generated database query language SQL. The manual feedback and correction module allows users to view the database query language (SQL) and execution results through the system interface, determine the correctness of the SQL query language, and provide feedback on whether it is correct or incorrect, thus completing manual feedback and correction.

Images