Incremental analysis-based limited autoregressive decoding method and system, and medium

Abstract

The invention discloses a limited autoregressive decoding method and system based on incremental analysis and a medium, and the method comprises the steps: for a given Text-to-SQL task, formatting and coding an input natural language query problem and a corresponding database mode; generating an SQL query by adopting a large language model; the score is adjusted according to the log-softmax score predicted by the model and the currently assumed token, and in the step of generating each token of the SQL, the candidate tokens are subjected to incremental analysis to check whether the candidate tokens conform to grammar and semantic rules of the SQL; performing validity check on the generated SQL query through different preset check models; and endowing negative infinity scores to the tokens which do not pass the check, excluding the negative infinity scores in beam search, and outputting, evaluating and optimizing the effective SQL query sequence which passes the check. The execution precision of the language model can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a restricted autoregressive decoding method, system and medium based on incremental parsing. Background Art

[0002] When existing large-scale language models generate natural language and structured query languages ​​(such as SQL), the output space is usually unconstrained, which can easily lead to grammatical or logical errors and cannot guarantee the correctness of the generated code. Existing restricted decoding methods mostly reduce invalid outputs by customizing vocabulary or dedicated model structures, but these methods are often complex and difficult to apply to general language models. Therefore, a decoding method with strong compatibility that can ensure that the output strictly complies with SQL syntax and semantic constraints is needed.

[0003] The invention patent with the publication number CN118779342A and the name of the method, system, device and medium for converting natural language to SQL query statements parses the user's natural language query instructions through a universal large language model, uses knowledge graph and neural network technology to accurately match the target table and form SQL query semantics, and then intelligently converts it into SQL query statements through the SQL industry large model and thinking chain mode, and finally executes and returns the query results. It mainly includes: natural language parsing module, metadata recognition and semantic generation module, thinking chain generation SQL module, query execution and result display module. The system lacks real-time incremental parsing and verification, and it is difficult to ensure whether the SQL generated at each step conforms to the grammar and logic. In the case of multi-round dialogue or complex query, invalid or erroneous SQL generation may result; the method relies on multi-round dialogue history and prompt generation, but does not set dynamic checks, which may lead to logical inconsistencies or context-conflicting SQL queries in multi-round queries. This context management method is unstable in complex multi-round query scenarios; the system relies on SQL templates and has limited flexibility when facing variable or non-templated query requirements.

[0004] The publication number is CN118820286A, and the name is an invention patent for a text-to-SQL conversion method and device based on automated process supervision. It describes a text-to-SQL conversion method based on automated process supervision. The main purpose is to improve the accuracy and reliability of conversion through process supervision and automated data learning in the process of natural language to SQL conversion. Its main processes include: natural language parsing, SQL generation, automated process supervision, SQL optimization and verification, data annotation and learning. The process supervision of this method only performs error correction after anomalies are detected. There is a lack of a mechanism for incremental inspection at each step of SQL generation, resulting in some invalid SQL statements being generated only after subsequent inspections. The problem is discovered, which increases processing time and computing costs; the method lacks dynamic tracking and consistency management of context information in multi-round query scenarios, and it is difficult to maintain the consistency of the previous and subsequent logic in continuous queries, and may generate SQL queries that do not match user intentions; the system relies too much on user feedback to learn and improve the model, resulting in a slow learning and optimization process, and cannot dynamically adapt to complex or novel query requirements during the SQL generation process.

[0005] The above systems and methods are insufficient in context management and consistency assurance in multiple rounds of queries, making it difficult to ensure the logical coherence of complex continuous queries; they are relatively limited in real-time analysis and dynamic error correction, and only perform error correction after generation or when an exception occurs, lacking a step-by-step analysis and instant feedback mechanism; they are highly dependent on templates or user feedback, resulting in insufficient flexibility of the system in response to changing query requirements. Summary of the invention

[0006] The main purpose of the present invention is to propose a restricted autoregressive decoding method, system and medium based on incremental parsing, aiming to improve the execution accuracy of the language model.

[0007] To achieve the above object, the present invention provides a constrained autoregressive decoding method based on incremental parsing, the method comprising the following steps:

[0008] Step S10, preprocessing and model input: for a given Text-to-SQL task, format and encode the input natural language query question and the corresponding database schema;

[0009] Step S20, autoregressive encoding: using a large language model to generate SQL queries;

[0010] Step S30, incremental parsing integration: adjust the score according to the log-softmax score predicted by the model and the currently assumed token, and perform incremental parsing on the candidate tokens in the generation step of each token of SQL to check whether they conform to the grammatical and semantic rules of SQL;

[0011] Step S40, performing validity check on the generated SQL query using different preset check models;

[0012] In step S50, tokens that fail the check are assigned a negative infinite score and are excluded from the beam search, and valid SQL query sequences that pass the check are output, evaluated, and optimized.

[0013] A further technical solution of the present invention is that step S10 comprises:

[0014] Step S101, data integration: integrating the natural language query question and the schema information of the database into a single input sequence;

[0015] Step S102, input formatting: splicing the natural language query and the database model into a unified text form, using specific delimiters to ensure that each part is clearly distinguished;

[0016] Step S103, encoding: encoding the formatted text input to adapt to the processing requirements of the language model.

[0017] Step S104, input normalization: normalizing SQL keywords and identifiers.

[0018] A further technical solution of the present invention is that step S103 comprises:

[0019] Use word segmentation to convert the input into a token sequence that the model can process. For extremely long inputs, truncation is performed to retain natural language queries and key database information.

[0020] A further technical solution of the present invention is that before step S104, the following steps are further included:

[0021] Step S1031, database content enhancement: In some tasks, the context awareness capability of the model is enhanced by adding actual database content.

[0022] A further technical solution of the present invention is that step S20 comprises:

[0023] Step S201, input sequence encoding: taking the preprocessed and formatted token sequence as input, encoding it into an internal vector representation;

[0024] Step S202, autoregressive encoding process: the model starts from the start token <BOS> and gradually generates SQL queries. In each time step, the model calculates the probability distribution of all possible outputs of the next step based on the currently generated partial sequence, and uses beam search to maintain multiple candidate sequences;

[0025] Step S203, probability calculation and selection: the model uses the log-softmax function to assign probabilities to all possible output tokens, and retains several candidate paths during beam search through incremental parsing;

[0026] Step S204, gradually constructing the SQL query: the generated tokens are appended to the sequence in sequence, and this generation process continues until the model generates an end token <EOS>, indicating that the SQL query is complete.

[0027] A further technical solution of the present invention is that step S30 comprises:

[0028] Step S301, start decoding step: each time a token is generated, decoding is performed from the start mark;

[0029] Step S302, the model calculates the probability distribution of each candidate token: the language model uses the log-softmax function to calculate the probability distribution of all possible output tokens in the next step based on the generated partial sequence;

[0030] Step S303, preliminarily selecting the candidate tokens with the highest probability: according to the probability distribution given by the model, preliminarily selecting several candidate tokens with the highest probability as possible subsequent expansions;

[0031] Step S304, incremental parsing to check whether the candidate tokens comply with the syntax and semantics: incremental parsing checks are performed on the candidate tokens to verify whether they comply with the syntax and semantics rules of SQL;

[0032] Step S305, if the candidate token is valid, the candidate token that passes the incremental parsing check will be retained, and the process proceeds to step S307 to continue with the next step of generating SQL;

[0033] Step S306, if the candidate token is invalid, it is eliminated: the candidate token that fails the check will be assigned a negative infinite score;

[0034] Step S307: The candidate tokens retained are used as valid paths and enter the next step of token generation. The above steps are repeated until the SQL sequence is completely generated.

[0035] A further technical solution of the present invention is that the different inspection models pre-set in step S40 include a grammatical analysis model, an unguarded parsing model, and a guarded parsing model.

[0036] A further technical solution of the present invention is that step S50 comprises:

[0037] Use exact match accuracy, execution accuracy metrics, and test suites to evaluate the performance of generated SQL queries. Based on the performance evaluation, adjust hyperparameters such as beam search window size and check mode, or fine-tune large models to further improve performance on specific tasks.

[0038] To achieve the above objectives, the present invention also proposes a restricted autoregressive decoding system based on incremental parsing, characterized in that the system includes a memory, a processor, and a restricted autoregressive decoding program based on incremental parsing stored on the processor, and the restricted autoregressive decoding program based on incremental parsing executes the steps of the method described above when run by the processor.

[0039] To achieve the above objectives, the present invention also proposes a computer-readable storage medium, which stores a restricted autoregressive decoding program based on incremental parsing, and the restricted autoregressive decoding program based on incremental parsing executes the steps of the method described above when executed by a processor.

[0040] The beneficial effects of the incremental parsing-based constrained autoregressive decoding method, system and storage medium of the present invention are:

[0041] 1. Perform syntax and logic checks at each step of generating SQL statements to ensure that each generated fragment is a valid SQL statement and avoid generating invalid or erroneous queries.

[0042] 2. Through incremental parsing, context consistency can be continuously verified in multiple rounds of conversations to ensure that the SQL statements generated in each round are consistent with the previous query intent, which is suitable for more complex query scenarios.

[0043] 3. It does not rely on preset templates and specific SQL models, and can directly generate effective SQL statements in complex and dynamic queries, with higher flexibility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0045] Figure 1 It is a flow chart of a preferred embodiment of a restricted autoregressive decoding method based on incremental parsing of the present invention;

[0046] Figure 2 is a schematic flow chart of step S10;

[0047] Figure 3 is a schematic flow chart of step S20;

[0048] Figure 4 is a schematic flow chart of step S30;

[0049] Figure 5 It is a schematic diagram of the overall process of the restricted autoregressive decoding method based on incremental parsing of the present invention.

[0050] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The present invention proposes a constrained autoregressive decoding method based on incremental parsing, which can be applied to database query generation systems, intelligent data analysis and business intelligence platforms, AI dialogue systems and virtual assistants.

[0053] Please refer to Figure 1 and Figure 5 The preferred embodiment of the restricted autoregressive decoding method based on incremental parsing of the present invention comprises the following steps:

[0054] Step S10, preprocessing and model input: For a given Text-to-SQL task, the input natural language query question and the corresponding database schema are formatted and encoded.

[0055] For a given Text-to-SQL task, the input is usually a natural language query and the corresponding database schema. In order for these inputs to be better processed by the language model, they need to be properly formatted and encoded.

[0056] Step S20, autoregressive coding: using a large language model to generate SQL queries, wherein the large language model is, for example, TS or GPT.

[0057] Step S30, incremental parsing integration: adjust the score according to the log-softmax score predicted by the model and the currently assumed token. In the generation step of each token of SQL, the candidate tokens are incrementally parsed to check whether they conform to the grammatical and semantic rules of SQL.

[0058] This embodiment adopts an incremental parsing mechanism to gradually check the SQL fragments at each stage during the SQL generation process to ensure that each fragment complies with the SQL syntax and logic. This mechanism allows the system to verify and correct errors in real time when generating SQL statements, rather than waiting until the generation is complete to find errors, thereby reducing the possibility of invalid SQL generation. Incremental parsing improves the accuracy of generation through layer-by-layer verification at the lexical, grammatical, and semantic levels. In addition, since the syntax check is completed during the generation process, the system does not require large-scale error correction or optimization in the later stage, which significantly saves computing costs and time overhead and improves the efficiency of SQL generation.

[0059] Incremental parsing is the core of this embodiment, which allows the output to be dynamically checked and constrained at each step of the language model generation sequence. This method is different from traditional parsing because it does not check the entire sequence at once after the generation is completed, but proceeds step by step during the generation process. The advantage of this is that invalid tokens can be excluded immediately, thereby reducing the search space and improving decoding efficiency. Incremental parsing relies on a fast parser that can quickly check whether the current token conforms to SQL syntax and semantic rules at each step.

[0060] Step S40: Check the validity of the generated SQL query using different preset check models.

[0061] Among them, the pre-set different inspection models include a grammatical analysis model, an unguarded parsing model, and a guarded parsing model.

[0062] In step S50, tokens that fail the check are assigned a negative infinite score and are excluded from the beam search, and valid SQL query sequences that pass the check are output, evaluated, and optimized.

[0063] Specifically, this embodiment uses exact match accuracy, execution accuracy indicators and test suites to evaluate the performance of generating SQL queries. It is best to adjust hyperparameters such as beam search window size and check mode based on the performance evaluation, or fine-tune the large model to further improve the performance on specific tasks.

[0064] Through the above technical solution, this embodiment is compatible with various existing autoregressive models, and directly operates the output of the language model by rejecting some unacceptable tokens according to the judgment conditions in each decoding step, thereby improving the execution accuracy of the language model, especially in the Text-to-SQL translation task.

[0065] Please refer to Figure 2 In this embodiment, step S10 specifically includes:

[0066] Step S101, data integration: integrating the natural language query question and the schema information (table, column) of the database into a single input sequence.

[0067] Step S102, input formatting: concatenating the natural language query and the database schema into a unified text format, using specific separators such as “|” to ensure that each part is clearly distinguished;

[0068] Step S103, encoding: encoding the formatted text input to adapt to the processing requirements of the language model.

[0069] The step S103 specifically includes: using a word segmentation method such as BPE or WordPiece to convert the input into a token sequence that can be processed by the model. For extremely long inputs (such as tasks containing multiple tables or dialogue contexts), truncation is performed to retain natural language queries and key database information.

[0070] Step S104, input normalization: normalizing SQL keywords and identifiers.

[0071] In order to improve the consistency and accuracy of the system, SQL keywords and identifiers are standardized (such as converting all keywords to uppercase). This step can effectively reduce the complexity of syntax and matching.

[0072] In this embodiment, the step S104 may further include:

[0073] Step S1031, database content enhancement: In some tasks, the context awareness capability of the model is enhanced by adding actual database content.

[0074] Please refer to Figure 3 In this embodiment, step S20 specifically includes:

[0075] Step S201, input sequence encoding: taking the preprocessed and formatted token sequence as input, and encoding it into an internal vector representation.

[0076] Step S202, autoregressive encoding process: the model starts from the starting token <BOS> and gradually generates SQL queries. In each time step, the model calculates the probability distribution of all possible outputs of the next step based on the currently generated partial sequence, and uses beam search to maintain multiple candidate sequences.

[0077] Step S203, probability calculation and selection: The model uses the log-softmax function to assign probabilities to all possible output tokens, and retains several candidate paths during beam search through incremental parsing.

[0078] Step S204, gradually constructing the SQL query: the generated tokens are appended to the sequence in sequence, and this generation process continues until the model generates an end token <EOS>, indicating that the SQL query is complete.

[0079] Please refer to Figure 4 In this embodiment, step S30 specifically includes:

[0080] Step S301, start decoding step: each time a token is generated, decoding is performed from the start mark, which marks the start of the current generation step.

[0081] Step S302, the model calculates the probability distribution of each candidate token: the language model uses the log-softmax function to calculate the probability distribution of all possible output tokens in the next step based on the generated partial sequence.

[0082] Step S303, preliminarily select the candidate tokens with the highest probability: According to the probability distribution given by the model, preliminarily select several candidate tokens with the highest probability as possible subsequent expansions.

[0083] Step S304, incremental parsing checks whether the candidate tokens conform to syntax and semantics: Incremental parsing checks are performed on the candidate tokens to verify whether they conform to the syntax and semantic rules of SQL. Incremental parsing is used to ensure that the generated partial sequence remains valid at each step, and any grammatical or semantic inconsistencies are captured.

[0084] Step S305, if the candidate token is valid, the candidate token that passes the incremental parsing check will be retained, and the process proceeds to step S307 to continue with the next step of generating SQL;

[0085] Step S306: If the candidate token is invalid, it will be eliminated: the candidate tokens that fail the check will be assigned a negative infinite score, which means that these paths will be completely excluded in the subsequent generation process.

[0086] Step S307: The candidate tokens retained are used as valid paths and enter the next step of token generation. The above steps are repeated until the SQL sequence is completely generated.

[0087] The key technologies of the restricted autoregressive decoding method based on incremental parsing of the present invention involve the following aspects.

[0088] 1. Incremental parsing is the core of this method, which allows the output to be dynamically checked and constrained at each step of the language model generation sequence. This method is different from traditional parsing because it does not check the entire sequence at once after the generation is completed, but proceeds step by step during the generation process. The advantage of this is that invalid tokens can be excluded instantly, thereby reducing the search space and improving decoding efficiency. Incremental parsing relies on a fast parser, which can quickly check whether the current token conforms to SQL syntax and semantic rules at each step.

[0089] 2. Prediction score adjustment. This method integrates with existing greedy search and beam search algorithms by adjusting the scores predicted by the model. During the autoregressive decoding process, the model generates a probability distribution for each possible next token. Based on this distribution, the method selects the most likely k tokens and checks whether they comply with the syntax and semantic rules of SQL. If a token does not comply with the rules, its score is set to negative infinity so that the token is not considered in the beam search. This method effectively excludes invalid tokens from the search space without modifying the structure or training process of the model itself.

[0090] 3. SQL schema information. This method uses SQL schema information to enhance the parsing process. Schema information includes the names of tables and columns, as well as the relationships between columns and tables. This information is crucial for verifying whether the generated SQL queries are valid. For example, if a non-existent table or column is referenced in the generated query, the schema information can be used to identify this error and exclude these invalid queries. This ability to use schema information is the key to the method's ability to effectively improve the performance of text-to-SQL tasks.

[0091] 4. Multiple inspection modes. The present invention provides four different inspection modes to control the comprehensiveness of its inspection. These modes include:

[0092] a) Off: No checking is done. This is the baseline mode used to compare the performance of the method.

[0093] b) Lexical Analysis: It only checks whether the generated tokens are valid keywords, operators or identifiers.

[0094] c) Parsing without Guards: Check the output at the syntax level to ensure that the generated query conforms to the SQL syntax structure.

[0095] d) Parsing with Guards: In addition to syntax checking, additional guard rules are added to ensure that the generated queries are semantically valid, such as ensuring that the referenced tables and columns actually exist in the database schema. Specifically, these guard rules include the following key points:

[0096] (1) Range check of referenced tables and columns: If parsing to a pattern such as tid.cid, the guard rule requires that the tid table must be introduced in the FROM clause;

[0097] (2) Unique column resolution: When resolving to an independent column name cid, the guard rule will check whether only one table contains the column. If multiple tables contain the cid column, the resolution fails to avoid ambiguity.

[0098] (3) Subquery and alias binding verification: If an alias such as table1AS alias is bound to a subquery, the guard rule ensures that the alias is not bound repeatedly in the same scope.

[0099] (4) Fast failure mechanism: The guard mechanism verifies all possible semantic constraints early in the parsing process and quickly eliminates candidates that do not meet the conditions, avoiding unnecessary processing of invalid SQL statements later.

[0100] 5. Guards. Guards are a mechanism used in the most advanced parsing mode of this method to perform additional semantic checks when building the SQL abstract syntax tree (AST). Guard rules ensure that tables and columns referenced in SQL queries are ultimately correctly included in the FROM clause and that alias references are consistent. These guard rules can quickly identify and exclude semantically invalid queries, thereby improving the accuracy and efficiency of the entire system.

[0101] 6. Beam Ensemble Search. The method is tightly integrated with the beam search algorithm, by limiting the prediction to the k tokens with the highest probability in each decoding step and assigning a negative infinity score to those tokens that fail the check, effectively excluding them from the candidate set. This integration allows strict constraints and validation of the generated SQL queries while maintaining high efficiency.

[0102] The main key points and protection points of the limited autoregressive decoding method based on incremental parsing of the present invention are as follows:

[0103] 1. Incremental parsing mechanism. During the SQL generation process, the SQL fragments at each stage are checked step by step to ensure that each fragment conforms to the SQL syntax and logic. This mechanism allows the system to verify and correct errors in real time when generating SQL statements, rather than waiting until the generation is complete to find errors, thereby reducing the possibility of invalid SQL generation. Incremental parsing improves the accuracy of generation through layer-by-layer verification at the lexical, grammatical, and semantic levels. In addition, since the syntax check is completed during the generation process, the system does not need large-scale error correction or optimization in the later stage, which significantly saves computing costs and time overhead and improves the efficiency of SQL generation.

[0104] 2. Context consistency management. Context consistency management is used to maintain the logical coherence of queries and ensure that the SQL generated in each round can be connected with the previous query results or intentions. The system can automatically track context information in multiple rounds of queries to ensure that the SQL generated each time meets the user's overall query intentions, avoiding logical conflicts. By continuously tracking dynamic information in the query, the context of the previous round of queries is automatically associated to ensure that the data and conditions in multiple rounds of queries remain consistent. This context consistency management mechanism is particularly prominent in complex multi-round query scenarios and can meet the high requirements of users in continuous queries.

[0105] 3. Dynamic error correction and self-adaptation. Dynamic error correction and self-adaptation functions enable it to instantly identify and correct syntax and logic errors during the generation process. Unlike the traditional method of relying on user feedback to discover problems, real-time error checking and correction are performed during the generation stage, which reduces the system's dependence on user feedback and improves the accuracy of generation. The system can adapt to different demand scenarios by adjusting the SQL statement structure in real time, especially showing strong flexibility under complex or changing query requirements. This dynamic error correction mechanism ensures that the system can maintain the validity of SQL at every step and accelerates the overall responsiveness of the system.

[0106] 4. Flexibility without relying on templates. Complex SQL query statements can be generated without relying on fixed templates, which greatly improves the flexibility of the generation process. Unlike traditional systems that rely on preset templates, this method can directly generate SQL based on the user's natural language needs without being restricted by templates. This enables the system to adapt to different database structures and various application scenarios, and even highly customized query requirements can be accurately generated. This method generates complex SQL statements by autonomously parsing user intent, and supports complex SQL expressions such as multi-table associations, subqueries, and aggregations. This flexibility gives it a wider range of applicability and expressiveness.

[0107] The main goal of automated SQL generation and multi-round Text-to-SQL in the prior art is to automatically convert user text queries into SQL statements through natural language processing technology so that non-technical users can interact directly with the database. This type of method usually involves semantic parsing of user input, extracting entities, attributes and intents, and then using a large language model to generate SQL queries. However, traditional automated SQL generation methods are difficult to flexibly respond to complex query requirements and lack context management and logical consistency support in multi-round conversations. Multi-round Text-to-SQL methods need to maintain context consistency in consecutive conversation rounds so that the generated SQL statements can be connected to each other to meet the logical coherence in complex query scenarios. Through context tracking and dynamic adjustment, the system can automatically associate previous query intents in multi-round queries, but there are still problems such as insufficient context consistency and heavy reliance on fixed templates and rules. The present invention can effectively solve these technical problems through the above technical solution, and the present invention can achieve the following technical effects through the above technical solution:

[0108] 1. Perform syntax and logic checks at each step of generating SQL statements to ensure that each generated fragment is a valid SQL statement and avoid generating invalid or erroneous queries.

[0109] 2. Through incremental parsing, context consistency can be continuously verified in multiple rounds of conversations to ensure that the SQL statements generated in each round are consistent with the previous query intent, which is suitable for more complex query scenarios.

[0110] 3. It does not rely on preset templates and specific SQL models, and can directly generate effective SQL statements in complex and dynamic queries, with higher flexibility and adaptability.

[0111] To achieve the above objectives, the present invention also proposes a restricted autoregressive decoding system based on incremental parsing, the system comprising a memory, a processor, and a restricted autoregressive decoding program based on incremental parsing stored on the processor, the restricted autoregressive decoding program based on incremental parsing being executed by the processor to perform the steps of the method described in the above embodiment, which will not be repeated here.

[0112] To achieve the above objectives, the present invention also proposes a computer-readable storage medium, which stores a restricted autoregressive decoding program based on incremental parsing. The restricted autoregressive decoding program based on incremental parsing is executed by a processor to perform the steps of the method described in the above embodiment, which will not be repeated here.

[0113] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A constrained autoregressive decoding method based on incremental parsing, characterized in that: The method comprises the following steps: Step S10, preprocessing and model input: for a given Text-to-SQL task, format and encode the input natural language query question and the corresponding database schema; Step S20, autoregressive encoding: using a large language model to generate SQL queries; Step S30, incremental parsing integration: adjust the score according to the log-softmax score predicted by the model and the currently assumed token, and perform incremental parsing on the candidate tokens in the generation step of each token of SQL to check whether they conform to the grammatical and semantic rules of SQL; Step S40, performing validity check on the generated SQL query using different preset check models; In step S50, tokens that fail the check are assigned a negative infinite score and are excluded from the beam search, and valid SQL query sequences that pass the check are output, evaluated, and optimized.

2. The incremental parsing-based constrained autoregressive decoding method according to claim 1, characterized in that: The step S10 comprises: Step S101, data integration: integrating the natural language query question and the schema information of the database into a single input sequence; Step S102, input formatting: splicing the natural language query and the database model into a unified text form, using specific delimiters to ensure that each part is clearly distinguished; Step S103, encoding: encoding the formatted text input to adapt to the processing requirements of the language model. Step S104, input normalization: normalizing SQL keywords and identifiers.

3. The incremental parsing-based constrained autoregressive decoding method according to claim 2, characterized in that: The step S103 includes: Use word segmentation to convert the input into a token sequence that the model can process. For extremely long inputs, truncation is performed to retain natural language queries and key database information.

4. The incremental parsing-based constrained autoregressive decoding method according to claim 3, characterized in that: Before step S104, the following steps are also included: Step S1031, database content enhancement: In some tasks, the context awareness capability of the model is enhanced by adding actual database content.

5. The incremental parsing-based constrained autoregressive decoding method according to claim 4, characterized in that: The step S20 comprises: Step S201, input sequence encoding: taking the preprocessed and formatted token sequence as input, and encoding it into an internal vector representation; Step S202, autoregressive encoding process: the model starts from the start token <BOS> and gradually generates SQL queries. In each time step, the model calculates the probability distribution of all possible outputs of the next step based on the currently generated partial sequence, and uses beam search to maintain multiple candidate sequences; Step S203, probability calculation and selection: the model uses the log-softmax function to assign probabilities to all possible output tokens, and retains several candidate paths during beam search through incremental parsing; Step S204, gradually constructing the SQL query: the generated tokens are appended to the sequence in sequence, and this generation process continues until the model generates an end token <EOS>, indicating that the SQL query is complete.

6. The incremental parsing-based constrained autoregressive decoding method according to claim 5, characterized in that: The step S30 comprises: Step S301, start decoding step: each time a token is generated, decoding is performed from the start mark; Step S302, the model calculates the probability distribution of each candidate token: the language model uses the log-softmax function to calculate the probability distribution of all possible output tokens in the next step based on the generated partial sequence; Step S303, preliminarily selecting the candidate tokens with the highest probability: according to the probability distribution given by the model, preliminarily selecting several candidate tokens with the highest probability as possible subsequent expansions; Step S304, incremental parsing to check whether the candidate tokens comply with the syntax and semantics: incremental parsing checks are performed on the candidate tokens to verify whether they comply with the syntax and semantics rules of SQL; Step S305, if the candidate token is valid, the candidate token that passes the incremental parsing check will be retained, and the process proceeds to step S307 to continue with the next step of generating SQL; Step S306, if the candidate token is invalid, it is eliminated: the candidate token that fails the check will be assigned a negative infinite score; Step S307: The candidate tokens retained are used as valid paths and enter the next step of token generation. The above steps are repeated until the SQL sequence is completely generated.

7. The incremental parsing-based constrained autoregressive decoding method according to claim 6, characterized in that: The different checking models pre-set in step S40 include a grammatical analysis model, an unguarded parsing model, and a guarded parsing model.

8. The incremental parsing-based constrained autoregressive decoding method according to claim 7, characterized in that: The step S50 comprises: Use exact match accuracy, execution accuracy metrics, and test suites to evaluate the performance of generated SQL queries. Based on the performance evaluation, adjust hyperparameters such as beam search window size and check mode, or fine-tune large models to further improve performance on specific tasks.

9. A constrained autoregressive decoding system based on incremental parsing, characterized in that: The system comprises a memory, a processor and a restricted autoregressive decoding program based on incremental parsing stored on the processor, wherein the restricted autoregressive decoding program based on incremental parsing is executed by the processor to perform the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a restricted autoregressive decoding program based on incremental parsing, and the restricted autoregressive decoding program based on incremental parsing is executed by a processor to perform the steps of the method according to any one of claims 1 to 8.

Images