Grammar detection method, system and device, program product and storage medium
By separating SQL statements into statement units and building AST, combined with traversal detection of preset syntax rules, the difficulty of syntax error detection and repair in SQL statements is solved, and efficient and accurate syntax detection and repair are achieved.
Patent Information
- Application Number
- CN202411739838.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-18
AI Technical Summary
The prior art has problems such as time-consuming, error-prone and error-locating when detecting and repairing syntax errors in complex SQL statements, especially when SQL statements are too long.
By separating the target SQL statement into independent statement units and building an abstract syntax tree (AST), traverse the AST based on preset syntax rules, detect and record syntax errors, and generate detailed syntax detection results.
It improves the accuracy and efficiency of syntax detection, can quickly locate and fix syntax errors in SQL statements, reduces the work burden of developers, and improves the efficiency and accuracy of writing SQL statements.
Smart Images
Figure CN119645807A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a grammar detection method, system, device, program product and storage medium. Background Art
[0002] In database management systems, Structured Query Language (SQL) is a standard language for managing and processing data. With the development of information technology, the increasing complexity of business needs and the continuous increase in data volume, the demand for SQL statements has become more diverse and complex. In order to meet these needs, developers usually use various tools or automatic generators to create complex SQL queries.
[0003] The generated SQL statements need to be manually checked for grammatical correctness. This is because even the most advanced SQL generation tools may produce code that does not conform to grammatical specifications. The manual verification process is not only time-consuming, but also prone to errors. Accurately identifying grammatical errors is a challenge, especially for those staff who are unfamiliar with SQL syntax or lack experience. When the generated SQL statement is too long, if a grammatical error occurs, the error prompts provided by traditional database tools are often not specific enough. This makes it difficult for developers to quickly locate the exact location where the error occurred. The error prompt information simply points out that there is an error in a certain line, and does not provide more detailed contextual information, such as specific column names, table names, etc., making the repair work extremely difficult. Therefore, developers must spend a lot of time reviewing lengthy SQL statements line by line to ensure that they meet grammatical requirements. This practice greatly increases the workload of the development team and affects project progress and efficiency. Summary of the invention
[0004] Based on the above problems, the present application provides a grammar detection method, system, device, program product and storage medium.
[0005] The embodiments of the present application disclose the following technical solutions:
[0006] A first aspect of an embodiment of the present application provides a grammar detection method, characterized by comprising:
[0007] Get the target sentence;
[0008] Separating the target sentence into a plurality of independent sentence units;
[0009] Construct an abstract syntax tree for each statement unit, wherein the root node of the abstract syntax tree represents the entire target statement, the second-level nodes represent the keywords in the target statement, the third-level nodes represent the subquery statements, and the table name and field name are used as leaf nodes;
[0010] Based on the preset target sentence grammar rules, each layer of the abstract syntax tree is traversed to obtain a grammar detection result. The grammar detection result is used to indicate the erroneous part of the target sentence with the grammatical error and the error cause of the erroneous part when there is a grammatical error in the target sentence.
[0011] In a possible implementation, traversing each layer of the abstract syntax tree based on preset target sentence grammar rules includes:
[0012] Based on the preset grammar rules of the target sentence, it is detected whether there are keywords that meet the preset conditions in each layer of the abstract syntax tree, and whether the arrangement order of the keywords meets the preset arrangement order.
[0013] In a possible implementation, traversing each layer of the abstract syntax tree based on preset target sentence grammar rules to obtain a grammar detection result includes:
[0014] Check whether the type of each node conforms to the preset grammatical category; verify whether the connection relationship between nodes conforms to the hierarchy and order defined in the grammatical rules;
[0015] When an error node that does not conform to a grammatical rule or a connection relationship is detected, error information used to generate a grammar detection result is recorded, the error information including the position of the error node, the position of the error node, and the grammatical rule violated by the error node;
[0016] When the detection of each layer of the abstract syntax tree is completed, the error information recorded is integrated to generate a syntax detection result.
[0017] In a possible implementation, before obtaining the target sentence, the method further includes:
[0018] Obtain natural language information;
[0019] The natural language information is input into a target language model to generate a target sentence corresponding to the natural language information. The target language model is obtained by adjusting parameter information of a preset large language model based on a training sample set using the large model parameter fine-tuning technology LoRA.
[0020] In a possible implementation, the method of acquiring the training sample set includes:
[0021] Obtaining sample sentences whose sentence quality meets preset quality conditions;
[0022] Pair each sample sentence with its corresponding natural language description to obtain a key-value pair;
[0023] The key-value pairs corresponding to each sample sentence are stored in a preset file to obtain a training sample set.
[0024] In a possible implementation, the method further includes:
[0025] The grammar detection result, correction suggestion information of the grammar detection result, and the target sentence corresponding to the grammar detection result are sent to a user interaction interface, wherein the target sentence includes a portion with grammatical errors marked with highlights.
[0026] A second aspect of an embodiment of the present application provides a grammar detection method, comprising:
[0027] An acquisition unit, used for acquiring a target sentence;
[0028] A separation unit, used for separating the target sentence into a plurality of independent sentence units;
[0029] A construction unit, used to construct an abstract syntax tree for each statement unit, wherein the root node of the abstract syntax tree represents the entire target statement, the second-level nodes represent keywords in the target statement, the third-level nodes represent subquery statements, and the table name and field name are used as leaf nodes;
[0030] A traversal unit is used to traverse each layer of the abstract syntax tree based on preset target sentence grammar rules to obtain a grammar detection result. The grammar detection result is used to indicate the erroneous part of the target sentence with the grammatical error and the error cause of the erroneous part when there is a grammatical error in the target sentence.
[0031] A third aspect of an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the grammar detection method as described in the first aspect above is implemented.
[0032] A fourth aspect of the embodiments of the present application provides a computer program product. When the computer program product runs on a computer, the computer executes the grammar detection method as described in the first aspect above.
[0033] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the grammar detection method as described in the first aspect above.
[0034] Compared with the prior art, this application has the following beneficial effects:
[0035] Get the target statement to be tested. Split the entire target statement according to semantics or structure to form statement units that can be processed independently. By separating, each part can be analyzed in a more fine-grained manner, improving the accuracy and efficiency of detection. Create an abstract syntax tree for each statement unit. The root node represents the entire target statement. The second-level nodes represent the keywords in the target statement. The third-level nodes represent subquery statements. Table names and field names are used as leaf nodes. The abstract syntax tree provides a structured way to represent the target statement, which is convenient for subsequent syntax analysis and error location. Through the hierarchical structure, the composition and relationship of the statement can be clearly seen, which helps to find potential syntax errors. Use predefined syntax rules to traverse the abstract syntax tree layer by layer. Check whether each layer of nodes conforms to the preset syntax rules. By traversing layer by layer, the legitimacy of each node can be systematically and comprehensively checked. The preset syntax rules ensure the consistency and accuracy of detection. If there is a syntax error in the target statement, a syntax detection result indicating the specific location of the error part and the cause of the error is generated, indicating the error part and its cause, helping developers to quickly locate and fix the problem. It improves the efficiency of developers, reduces debugging time caused by syntax errors, and greatly improves the efficiency and accuracy of statement writing. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0037] Figure 1 A flowchart of a grammar detection method provided in an embodiment of the present application;
[0038] Figure 2 A schematic diagram of the parameter fine-tuning model structure provided in an embodiment of the present application;
[0039] Figure 3 A schematic diagram of an abstract syntax tree provided for an embodiment of the present application;
[0040] Figure 4 A structural diagram of a grammar detection system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0042] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the professional terms involved in the embodiments of the present application will be explained below.
[0043] Large Language Model: refers to a language model with a large parameter scale that is trained using abundant training data. Usually, a large language model can interact in a question-and-answer format, and the number of parameters ranges from billions to hundreds of billions.
[0044] Large model parameter fine-tuning technology (Low-Rank Adaptation, LoRA): You can use your own dataset to fine-tune the already trained large language model base model, allowing the large language model to learn the patterns in your own dataset and become more proficient in solving problems in professional fields.
[0045] Regulatory reporting: The People's Bank of China and the Financial Regulatory Bureau need to understand the operating conditions of each bank and introduce targeted policy measures, so all banks are required to report their data in the form of reports.
[0046] Structured Query Language (SQL) is a type of programming language. SQL is usually used to query databases. By writing SQL statements, you can add, delete, modify, and query data in the database.
[0047] The abstract syntax tree is a tree structure representation of the abstract syntax of the source code. Since this structure does not reflect every detail of the real syntax (for example, brackets are hidden in the tree structure), it is called an abstract structure. Compilers of various programming languages usually build a syntax tree to analyze whether there are errors in the syntax of the code.
[0048] To facilitate understanding of the technical solutions provided by the embodiments of the present application, the background technology involved in the embodiments of the present application will be described below.
[0049] As mentioned above, the quality of manually written SQL statements varies. Due to differences in the writer's technical level, degree of understanding of the business, and accuracy of regulatory requirements, the written SQL statements may have various flaws, such as syntax errors, logical errors, or inaccurate data extraction. These problems not only affect the accuracy of the data, but may also cause delays in regulatory reporting, bringing unnecessary risks to the banking industry. With the continuous changes in business and regulatory requirements and the frequent updates of upstream systems, technicians need to constantly encode and modify SQL statements. The accelerated pace of change has undoubtedly increased the coding burden on technicians and reduced work efficiency. At the same time, due to the limitations of manual coding, it is difficult to ensure that high-quality SQL statement writing and modification work can be completed in a short period of time.
[0050] In order to solve the above problems, the embodiment of the present application can systematically detect and correct grammatical errors by constructing an abstract syntax tree and traversing layer by layer based on preset grammatical rules. The grammar detection result not only points out the error part, but also provides the cause of the error, helping developers understand the problem and correct it. In this way, the quality of SQL statements can be significantly improved, the problems of inefficient execution and prone to errors can be reduced, and the burden on technicians when reviewing SQL statements can be reduced.
[0051] LoRA is used to fine-tune the parameters of the large language model so that the model can better fit the specific data patterns in the field of regulatory reporting, thereby improving the accuracy and applicability of generated SQL statements, including support for operations such as addition, deletion, modification and query.
[0052] The adjusted large language model is integrated into a regulatory reporting system development and operation platform with a good human-computer interaction interface. Users can directly enter Chinese descriptions to obtain corresponding SQL statements and syntax check results, which facilitates quick code modification, reduces the technical threshold for using large language models, and improves team collaboration efficiency.
[0053] It should be noted that the grammar detection method, system, device and medium provided in the present application can be applied to the field of computer technology. The above is only an example and does not limit the application field of the grammar detection method, system, device and medium provided in the present application. In addition, the embodiments of the present application may not limit the execution subject of grammar detection. For example, the grammar detection method of the embodiments of the present application can be applied to data processing devices such as terminal devices or servers. Among them, the terminal device can be an electronic device such as a computer, a personal digital assistant (PDA). The server can be an independent server, a cloud server or a cluster server composed of multiple servers.
[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0055] The following describes a grammar detection method provided by the present application through an embodiment. Figure 1 ,Should Figure 1 A flowchart of a grammar detection method provided in an embodiment of the present application, the method comprising:
[0056] S101. Obtain a target sentence.
[0057] The SQL target statements to be tested are obtained through user input, file reading or system interface calls. These statements may come from user-written queries, updates, deletions and other operations, or obtained from other systems or modules. This step provides basic data for subsequent syntax testing and ensures the smooth progress of the testing process.
[0058] In a possible implementation, before obtaining the target statement, the step further includes:
[0059] Acquire natural language information; input the natural language information into a target language model to generate a target sentence corresponding to the natural language information.
[0060] Natural language information can be Chinese description information input by the user. The user enters Chinese description information through the user interface (such as text box, voice input, etc.). The system receives and stores this natural language information, which can be saved in the form of strings. This allows non-professional users or people without programming knowledge to express the tasks or queries they want to perform through everyday language. This improves the usability and ease of use of the system, allowing more people to participate in complex system interactions.
[0061] The target language model is obtained by adjusting parameter information of a preset large language model based on a training sample set using the large model parameter fine-tuning technology LoRA.
[0062] The method for obtaining the training sample set includes: obtaining sample sentences whose sentence quality meets preset quality conditions; pairing each sample sentence with its corresponding natural language description to obtain a key-value pair; storing the key-value pair corresponding to each sample sentence in a preset file to obtain a training sample set.
[0063] Collect SQL statements that meet the preset quality conditions. Pair these SQL statements with their corresponding Chinese descriptions to form key-value pairs (X: Chinese description, Y: SQL statement), and store them in files by line. The purpose of this is to provide enough samples for the large language model to learn how to generate correct SQL statements based on natural language descriptions.
[0064] The process of determining the target language model may include:
[0065] A large language model is used for parameter fine-tuning. The parameter fine-tuning method uses the LoRA method, which adjusts the model by adding an additional low-rank adaptation layer without changing the original model weights. The advantage of this method is that only a small number of new parameters need to be trained while maintaining the strong performance of the original model. The specific method is as follows Figure 2 As shown, Figure 2 Schematic diagram of the parameter fine-tuning model structure provided in the embodiment of the present application. The method first freezes the weight parameters of the large model base model that have been trained, and on this premise, by adding additional layers to the model, only the parameters of these newly added layers are trained. The advantage of this method is that fewer new parameters are required, and the effect is better after the model parameters are merged.
[0066] The LoRA method adds a branch next to the original model, which contains a dimension reduction matrix A m×a and a dimension-raising matrix B a×n Initialize the matrix B with 0 values a×n , and the matrix A m×a Initialized with Gaussian distribution, that is
[0067] Let the parameter matrix W m×n The size is m×n, and the input vector X i First, through a dimension reduction matrix A m×a , the matrix size is m×a, where a is much smaller than n. i When outputting the branch, it passes through the dimension-enhanced matrix B a×n , B a×n The size of is a×n. After the training is completed, the matrix A m×a With matrix B a×n Multiply, that is, R m×n =A m×a B a×n x.
[0068] Matrix R m×n is the parameter matrix of the parameter fine-tuning branch. By decomposing the m×n matrix into two low-dimensional matrices, the number of parameters can be greatly reduced, the use of video memory can be reduced, and the training speed can be increased. a×n is initialized to 0, so if the fine-tuning branch is not trained, the matrix Am×a With B a×n After multiplication, it is still 0 and does not affect the final result of the model.
[0069] Finally, directly transform the matrix R m×n Add to the original model parameter matrix W m×n , the information in the data set can be introduced into the base model when the model is fine-tuned without significantly reducing the original ability of the model, that is, W m×n =R m×n x+W m×n x.
[0070] This part is processed using the PyTorch framework of the Python language. PyTorch provides the implementation of many commonly used machine learning methods, and you can directly call PyTorch's library functions to implement matrix multiplication and addition.
[0071] There is no fixed requirement for the learning rate and number of iterations set in parameter fine-tuning. These parameters are all hyperparameters. You can try the parameter settings multiple times. Generally, as long as the model can converge, good results can be achieved.
[0072] S102: Separate the target sentence into multiple independent sentence units.
[0073] Use the syntax features of SQL statements, such as semicolons (;) as statement separators, or separate the target statement into multiple independent statement units according to specific logical structures (such as BEGIN...END blocks). These units can be complete SOL statements or parts of SQL statements (such as subqueries).
[0074] By dividing the target sentence into multiple independent sentence units, the subsequent grammar detection process can be simplified so that each unit can be processed and analyzed separately, improving the accuracy and efficiency of detection.
[0075] S103. Construct an abstract syntax tree for each statement unit, wherein the root node of the abstract syntax tree represents the entire target statement, the second-level nodes represent keywords in the target statement, the third-level nodes represent subquery statements, and the table name and field name are leaf nodes.
[0076] Using parser techniques such as lexical analysis and syntax analysis, each statement unit is converted into an abstract syntax tree (AST). AST is a tree data structure used to represent the grammatical structure of a statement. In AST, the root node represents the entire statement, and the child nodes represent the various components of the statement, such as keywords, table names, field names, etc. Figure 3 As shown, Figure 3 A schematic diagram of an abstract syntax tree provided for an embodiment of the present application.
[0077] By building AST, the hierarchical structure and grammatical relationship of SQL statements can be clearly represented, providing strong support for subsequent rule-based grammar detection. At the same time, the AST construction process can also automatically identify and correct some simple grammatical errors, such as spelling errors or missing necessary grammatical elements.
[0078] S104: traverse each layer of the abstract syntax tree based on preset target sentence grammar rules to obtain a grammar detection result.
[0079] The grammar detection result is used to indicate the erroneous part of the target sentence having the grammar error and the error cause of the erroneous part when the target sentence has the grammar error.
[0080] AST is traversed and matched using the preset SQL grammar rule base. These rule bases contain the grammar specifications of the SQL language, such as the correct usage of keywords, naming rules for table names and field names, matching of data types, etc. During the traversal process, if a node or subtree of the AST is found to not comply with the grammar rules, the error details are recorded, including the error location, error type, and error cause.
[0081] Systematically check the legitimacy of each node to ensure that the overall statement conforms to the expected grammatical structure. By checking layer by layer, you can find grammatical errors hidden in the deep structure. Provide specific error location and cause to help developers quickly locate and correct the problem. Through automation, the workload of manual error finding is reduced, and development efficiency is improved.
[0082] In a possible implementation, traversing each layer of the abstract syntax tree based on preset target sentence grammar rules includes:
[0083] Based on the preset grammar rules of the target sentence, it is detected whether there are keywords that meet the preset conditions in each layer of the abstract syntax tree, and whether the arrangement order of the keywords meets the preset arrangement order.
[0084] When traversing the AST, you first need to identify the preset keywords and check whether these keywords appear in the preset order. For example, in SQL statements, SELECT usually appears before FROM; in programming languages, the conditional expression of an if statement must follow the if statement.
[0085] When traversing the AST, use recursion or iteration to visit each node. For each node, if it is a keyword node, record its position and type. Check whether the recorded keyword sequence conforms to the preset order rules. Ensure that the keywords in the target statement are arranged according to the correct grammatical structure to avoid logical errors caused by keyword misplacement.
[0086] Compared with traversing the AST and checking each node, focusing on detecting keywords and verifying their order can significantly improve detection efficiency. This implementation improves the accuracy and efficiency of syntax detection by focusing on detecting the existence and order of keywords in the AST, and enhances the adaptability and user-friendliness of the system.
[0087] In a possible implementation, after traversing each layer of the abstract syntax tree based on the preset target sentence grammar rule to obtain the grammar detection result, the method further includes:
[0088] The step of traversing each layer of the abstract syntax tree based on the preset target sentence syntax rule to obtain a syntax detection result includes:
[0089] Check whether the type of each node conforms to the preset grammatical category; verify whether the connection relationship between the nodes conforms to the hierarchy and order defined in the grammatical rules; when an error node that does not conform to the grammatical rules or connection relationship is detected, record the error information used to generate the grammar detection result, the error information includes the position of the error node, the position of the error node and the grammar rule violated by the error node; when the detection of each layer of the abstract syntax tree is completed, integrate the recorded error information to generate the grammar detection result.
[0090] This process traverses each layer of the abstract syntax tree based on the preset target statement syntax rules. The purpose of the traversal is to check whether each node and its connection relationship conforms to the SQL syntax rules.
[0091] During the traversal process, the first step is to check whether the type of each node conforms to the preset grammatical category. For example, check whether a node is a keyword in a SELECT statement (such as FROM, WHERE, etc.), or whether it is a table name, column name, etc.
[0092] In addition to detecting node types, it is also necessary to verify whether the connection relationship between nodes conforms to the hierarchy and order defined in the grammatical rules. For example, check whether the column name in the SELECT statement correctly follows the SELECT keyword, whether the WHERE clause correctly follows the FROM clause, etc.
[0093] When an error node that does not conform to grammatical rules or connection relationships is detected, the system will record the error information used to generate the grammar detection result. The error information includes the location of the error node (the specific location or level in the AST), the type of error node (such as keyword error, missing necessary parts, etc.), and the grammar rule violated by the error node.
[0094] When each level of the abstract syntax tree is checked, the system will integrate all the error messages recorded. These error messages are integrated into a complete syntax check result for subsequent analysis, correction suggestion generation or user feedback.
[0095] In a possible implementation, the method further includes:
[0096] The grammar detection result, correction suggestion information of the grammar detection result, and the target sentence corresponding to the grammar detection result are sent to a user interaction interface, wherein the target sentence includes a portion with grammatical errors marked with highlights.
[0097] Based on the detected syntax errors, the system provides specific correction suggestions, such as replacing incorrect keywords, adjusting statement structure, etc. These suggestions are designed to help users quickly correct errors and ensure the correctness of SQL statements. In the original target statement, the part with syntax errors will be highlighted (such as red underline, background highlight, etc.) so that users can see the problem at a glance. This visual feedback greatly improves the user's ability to identify and understand errors.
[0098] Check whether the type of each node belongs to a predefined syntax category, such as variable declaration, function call, etc. Confirm that the relationship between nodes follows the predetermined hierarchy and order. For example, in C language, the conditional expression of a while loop should directly follow the while keyword, and the loop body should be placed within the curly braces. When non-compliant situations are found, record the location, type and specific rules violated of the relevant nodes. After completing all levels of inspection, integrate all the collected error information to form the final syntax detection result.
[0099] The user interface provides intuitive syntax detection results and correction suggestions, as well as highlights the error parts, making it easier for users to understand and correct errors in SQL statements. The combination of highlighting and correction suggestions enables users to quickly locate the error location and take corresponding correction measures, thereby shortening the time for error correction.
[0100] In order to achieve the above functions, the embodiment of the present application provides a complete system architecture, which includes front-end, back-end services, and fine-tuning and syntax analysis modules of large models. This system will allow users to enter Chinese SQL descriptions through an interactive interface, and then return the generated SQL statements and syntax analysis results.
[0101] The front-end is developed using the Vue.js framework. It provides a user interface that allows users to enter Chinese SQL descriptions. It calls the interface provided by the server, sends requests and receives processing results. It displays the generated SQL statements and syntax analysis results.
[0102] For example, in the implementation process, you can use Vue CLI to create a new Vue project. Develop a simple form component that allows users to enter Chinese SQL descriptions. Create a button that triggers an API request to the backend when the user clicks it. Display the data received from the backend, including the generated SQL statement and syntax analysis results.
[0103] The backend is developed in Java language, based on Spring framework. Design RESTful interface to receive requests sent by the frontend. Integrate syntax analysis module to process Chinese SQL description entered by user, generate SQL statements and perform syntax analysis. Return the processing results to the frontend.
[0104] For example, in the implementation process, you can initialize a Spring Boot project, design a RESTful API interface to receive front-end requests and respond to results, and call a Python script in the controller to perform SQL generation and syntax analysis tasks.
[0105] The syntax analysis module can be extended or customized using existing SQL parsing libraries (such as Apache Calcite, JSQLParser, etc.). The generated SQL statements are parsed to check for syntax errors. Detailed error information is provided, including error locations, missing or redundant keywords, etc.
[0106] The above are some specific implementations of the grammar detection method provided in the embodiment of the present application. Based on this, the present application also provides a corresponding grammar detection system. The system provided in the embodiment of the present application will be introduced from the perspective of functional modularization. Figure 4 A structural diagram of a grammar detection system provided in an embodiment of the present application.
[0107] The system comprises:
[0108] An acquisition unit 110, configured to acquire a target sentence;
[0109] A separation unit 111, used to separate the target sentence into multiple independent sentence units;
[0110] A construction unit 112 is used to construct an abstract syntax tree for each statement unit, wherein the root node of the abstract syntax tree represents the entire target statement, the second-level nodes represent keywords in the target statement, the third-level nodes represent subquery statements, and the table name and field name are leaf nodes;
[0111] The traversal unit 113 is used to traverse each layer of the abstract syntax tree based on the preset target sentence grammar rules to obtain a grammar detection result. The grammar detection result is used to indicate the erroneous part of the target sentence with the grammatical error and the error cause of the erroneous part when there is a grammatical error in the target sentence.
[0112] The embodiments of the present application also provide corresponding devices and computer storage media for implementing the test method solutions provided in the embodiments of the present application.
[0113] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes so that the device executes the test method described in any embodiment of the present application.
[0114] The computer storage medium stores codes, and when the codes are executed, a device executing the codes implements the test method described in any embodiment of the present application.
[0115] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0116] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0117] It should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.
[0118] It should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0119] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0120] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0121] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A grammar detection method, characterized in that: include: Get the target sentence; Separating the target sentence into a plurality of independent sentence units; Construct an abstract syntax tree for each statement unit, wherein the root node of the abstract syntax tree represents the entire target statement, the second-level nodes represent the keywords in the target statement, the third-level nodes represent the subquery statements, and the table name and field name are used as leaf nodes; Based on the preset target sentence grammar rules, each layer of the abstract syntax tree is traversed to obtain a grammar detection result. The grammar detection result is used to indicate the erroneous part of the target sentence with the grammatical error and the error cause of the erroneous part when there is a grammatical error in the target sentence.
2. The method according to claim 1, characterized in that The traversing each layer of the abstract syntax tree based on the preset target sentence grammar rule includes: Based on the preset grammar rules of the target sentence, it is detected whether there are keywords that meet the preset conditions in each layer of the abstract syntax tree, and whether the arrangement order of the keywords meets the preset arrangement order.
3. The method according to claim 2, characterized in that The step of traversing each layer of the abstract syntax tree based on the preset target sentence syntax rule to obtain a syntax detection result includes: Check whether the type of each node conforms to the preset grammatical category; verify whether the connection relationship between nodes conforms to the hierarchy and order defined in the grammatical rules; When an error node that does not conform to a grammatical rule or a connection relationship is detected, error information used to generate a grammar detection result is recorded, the error information including the position of the error node, the position of the error node, and the grammatical rule violated by the error node; When the detection of each layer of the abstract syntax tree is completed, the error information recorded is integrated to generate a syntax detection result.
4. The method according to claim 1, characterized in that: Before obtaining the target sentence, the method further includes: Obtain natural language information; The natural language information is input into a target language model to generate a target sentence corresponding to the natural language information. The target language model is obtained by adjusting parameter information of a preset large language model based on a training sample set using the large model parameter fine-tuning technology LoRA.
5. The method according to claim 1, characterized in that The method of obtaining the training sample set includes: Obtaining sample sentences whose sentence quality meets preset quality conditions; Pair each sample sentence with its corresponding natural language description to obtain a key-value pair; The key-value pairs corresponding to each sample sentence are stored in a preset file to obtain a training sample set.
6. The method according to claim 1, characterized in that The method further comprises: The grammar detection result, correction suggestion information of the grammar detection result, and the target sentence corresponding to the grammar detection result are sent to a user interaction interface, wherein the target sentence includes a portion with grammatical errors marked with highlights.
7. A grammar detection system, characterized in that: include: An acquisition unit, used for acquiring a target sentence; A separation unit, used for separating the target sentence into a plurality of independent sentence units; A construction unit, used to construct an abstract syntax tree for each statement unit, wherein the root node of the abstract syntax tree represents the entire target statement, the second-level nodes represent keywords in the target statement, the third-level nodes represent subquery statements, and the table name and field name are used as leaf nodes; The traversal unit is used to traverse each layer of the abstract syntax tree based on the preset target sentence grammar rules to obtain a grammar detection result. The grammar detection result is used to indicate the erroneous part of the target sentence with the grammatical error and the error cause of the erroneous part when there is a grammatical error in the target sentence.
8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the grammar detection method according to any one of claims 1 to 6 is implemented.
9. A computer program product, characterized in that When the computer program product runs on a computer, the computer executes the grammar detection method according to any one of claims 1 to 6.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the grammar detection method according to any one of claims 1 to 6.