Data correction method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a data correction method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an original structured statement and original annotation information; determining knowledge data corresponding to the original structured statement in a target knowledge database based on a metadata query engine; the knowledge data comprises graph relation metadata and semantic vector metadata; according to the data correction large model, the graph relation metadata and the semantic vector metadata, error recognition is carried out on the original structured statement and the original annotation information, data correction is carried out on error data obtained through error recognition, and a target structured statement and target annotation information are obtained. By adopting the method, the accuracy of structured statement and annotation error correction can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a data correction method, apparatus, computer device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of artificial intelligence technology, the processing power of large language models can enable automated error correction of various types of data, thereby improving the efficiency of data error correction.

[0003] In traditional techniques, pre-trained large language models are used to automatically analyze the syntax of SQL (Structured Query Language) statements and their corresponding comments, identify syntax errors in the SQL statements and comments, and correct these errors using the large language model to obtain the correct SQL statements and comments.

[0004] However, in traditional technologies, the data dimensions input to large language models are relatively simple, resulting in large language models only supporting the identification and correction of grammatical errors, which in turn leads to poor accuracy in SQL statement and comment error correction. Summary of the Invention

[0005] Therefore, it is necessary to provide a data correction method, apparatus, computer device, computer-readable storage medium, and computer program product to address the aforementioned technical problems.

[0006] Firstly, this application provides a data correction method, including:

[0007] Obtain the original structured statements and original comment information;

[0008] The knowledge data corresponding to the original structured statement is determined in the target knowledge database based on the metadata query engine; the knowledge data includes graph relation metadata and semantic vector metadata.

[0009] Based on the data correction model, the graph relation metadata, and the semantic vector metadata, errors are identified in the original structured statement and the original annotation information. The erroneous data obtained from the error identification is then corrected to obtain the target structured statement and target annotation information.

[0010] In one embodiment, determining the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine includes:

[0011] The original structured statement is parsed according to a preset parsing strategy. If the parsing result is successful, the structural element corresponding to the original structured statement is obtained.

[0012] If the parsing result is a parsing failure, the original structured statement is parsed and extracted according to the structural parsing model to obtain the structural elements;

[0013] Based on the metadata query engine and the structured elements, the knowledge data corresponding to the original structured statement is determined in the target knowledge database.

[0014] In one embodiment, determining the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine includes:

[0015] Based on the metadata query engine and the original annotation information, a semantic vector is generated, and the semantic vector is retrieved in the target knowledge database to obtain semantic vector metadata.

[0016] In the target knowledge database, the existence detection and semantic association recommendation of each structural element in the original structured statement are performed based on the metadata query engine to obtain the graph relation metadata corresponding to each structural element.

[0017] In one embodiment, the semantic vector is generated based on the metadata query engine and the original annotation information, and the semantic vector is retrieved from the target knowledge database to obtain semantic vector metadata, including:

[0018] Based on the metadata query engine, the natural language text in the original annotation information is converted into a rule expression in the target format to obtain the target annotation information;

[0019] Based on the metadata query engine, the structural elements in the original structured statement are parsed, and each structural element is semantically bound to the target annotation information to obtain the association between the rule expression and the structural element;

[0020] The semantic vector corresponding to the original annotation information is generated based on the metadata query engine, and the semantic vector and the association relationship are retrieved in the target knowledge database to obtain the semantic vector metadata corresponding to each structural element.

[0021] In one embodiment, the step of identifying errors in the original structured statements and the original annotation information based on the data-corrected large model, the graph relation metadata, and the semantic vector metadata includes:

[0022] Based on the data correction model and the metadata of the graph relationship, the logical structure of the original structured statement and the original annotation information is analyzed to identify logical errors in the original structured statement and the original annotation information;

[0023] Based on the data correction model and the semantic vector metadata, the semantics of the original structured statement and the original annotation information are analyzed to identify semantic errors in the original structured statement and the original annotation information.

[0024] In one embodiment, the step of correcting the erroneous data obtained from error identification to obtain the target structured statement and target annotation information includes:

[0025] Based on the data correction model, the graph relation metadata, and the semantic vector metadata, the logical errors and / or semantic errors obtained from error identification are corrected to obtain the target structured statement and target annotation information.

[0026] Secondly, this application also provides a data correction device, comprising:

[0027] The acquisition module is used to acquire the original structured statements and original comment information;

[0028] The query module is used to determine the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine; the knowledge data includes graph relation metadata and semantic vector metadata;

[0029] The identification module is used to identify errors in the original structured statement and the original annotation information based on the data correction model, the graph relation metadata and the semantic vector metadata, and to correct the erroneous data obtained from the error identification to obtain the target structured statement and target annotation information.

[0030] In one embodiment, the query module is specifically used to parse the original structured statement according to a preset parsing strategy. If the parsing result is successful, the structural element corresponding to the original structured statement is obtained.

[0031] If the parsing result is a parsing failure, the original structured statement is parsed and extracted according to the structural parsing model to obtain the structural elements;

[0032] Based on the metadata query engine and the structured elements, the knowledge data corresponding to the original structured statement is determined in the target knowledge database.

[0033] In one embodiment, the query module is specifically used to generate a semantic vector based on the metadata query engine and the original annotation information, and to retrieve the semantic vector in the target knowledge database to obtain semantic vector metadata.

[0034] In the target knowledge database, the existence detection and semantic association recommendation of each structural element in the original structured statement are performed based on the metadata query engine to obtain the graph relation metadata corresponding to each structural element.

[0035] In one embodiment, the query module is specifically used to convert the natural language text in the original annotation information into a rule expression in the target format based on the metadata query engine, so as to obtain the target annotation information;

[0036] Based on the metadata query engine, the structural elements in the original structured statement are parsed, and each structural element is semantically bound to the target annotation information to obtain the association between the rule expression and the structural element;

[0037] The semantic vector corresponding to the original annotation information is generated based on the metadata query engine, and the semantic vector and the association relationship are retrieved in the target knowledge database to obtain the semantic vector metadata corresponding to each structural element.

[0038] In one embodiment, the identification module is specifically used to analyze the logical structure of the original structured statement and the original annotation information based on the data correction of the large model and the graph relationship metadata, and to identify logical errors in the original structured statement and the original annotation information;

[0039] Based on the data correction model and the semantic vector metadata, the semantics of the original structured statement and the original annotation information are analyzed to identify semantic errors in the original structured statement and the original annotation information.

[0040] In one embodiment, the identification module is specifically used to correct logical errors and / or semantic errors obtained from misidentification based on the data correction model, the graph relation metadata, and the semantic vector metadata, so as to obtain target structured statements and target annotation information.

[0041] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0042] Obtain the original structured statements and original comment information;

[0043] The knowledge data corresponding to the original structured statement is determined in the target knowledge database based on the metadata query engine; the knowledge data includes graph relation metadata and semantic vector metadata.

[0044] Based on the data correction model, the graph relation metadata, and the semantic vector metadata, errors are identified in the original structured statement and the original annotation information. The erroneous data obtained from the error identification is then corrected to obtain the target structured statement and target annotation information.

[0045] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0046] Obtain the original structured statements and original comment information;

[0047] The knowledge data corresponding to the original structured statement is determined in the target knowledge database based on the metadata query engine; the knowledge data includes graph relation metadata and semantic vector metadata.

[0048] Based on the data correction model, the graph relation metadata, and the semantic vector metadata, errors are identified in the original structured statement and the original annotation information. The erroneous data obtained from the error identification is then corrected to obtain the target structured statement and target annotation information.

[0049] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0050] Obtain the original structured statements and original comment information;

[0051] The knowledge data corresponding to the original structured statement is determined in the target knowledge database based on the metadata query engine; the knowledge data includes graph relation metadata and semantic vector metadata.

[0052] Based on the data correction model, the graph relation metadata, and the semantic vector metadata, errors are identified in the original structured statement and the original annotation information. The erroneous data obtained from the error identification is then corrected to obtain the target structured statement and target annotation information.

[0053] The aforementioned data correction method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire the structural elements and original annotation information corresponding to the original structured statement; determine the knowledge data corresponding to the structural elements in the target knowledge database based on a metadata query engine; the knowledge data includes graph relation metadata and semantic vector metadata; perform data augmentation on the original structured statement and original annotation information based on the graph relation metadata and semantic vector metadata to obtain the target data to be processed; identify errors in the target data to be processed based on the data correction model, and correct the erroneous data obtained from the error identification to obtain the target structured statement and target annotation information. Using this method, by retrieving the graph relation metadata and semantic vector metadata corresponding to the structural elements in the target knowledge database through a metadata query engine, the information dimensions of the original structured statement and original annotation information can be supplemented. This allows the data correction model to not only correct grammatical errors in the original structured statement and original annotation information, but also to simultaneously detect and correct related semantic text errors and logical errors in the original structured statement and original annotation information, enriching the dimensions of data correction and thus improving the accuracy of error correction for structured statements and annotations. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating a data correction method in one embodiment;

[0056] Figure 2 This is a flowchart illustrating the parsing of a raw structured statement in one embodiment;

[0057] Figure 3 This is a schematic diagram illustrating the process of retrieving semantic vector metadata and graph relation metadata in a target knowledge base in one embodiment.

[0058] Figure 4 This is a schematic diagram of the process for retrieving semantic vector metadata in one embodiment;

[0059] Figure 5 This is a flowchart illustrating the process of identifying logical and semantic errors in one embodiment.

[0060] Figure 6 This is a flowchart illustrating an example of a data correction method in one embodiment;

[0061] Figure 7 This is a structural block diagram of a data correction device in one embodiment;

[0062] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0064] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0065] In one embodiment, such as Figure 1 As shown, a data correction method is provided. This embodiment illustrates the method applied to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0066] Step 102: Obtain the original structured statements and original comment information.

[0067] In this embodiment, the original structured statement is an SQL statement, and the original comment information is the semantic comment text of the SQL statement. The original comment information can be Chinese table field descriptions and business terminology explanations, etc. The terminal receives the SQL script submitted by the user, which contains the original structured statement carrying the original comment information. However, there may be spelling errors, logical errors, or semantic errors in the original structured statement and the original comment information. For example, spelling errors may be caused by typos made by developers when writing SQL statements or comment information during development; logical errors are errors in the relationship between tables and fields in the SQL statement. For example, a SQL statement queries an object with field A1 in representation A, but field A1 does not exist in representation A, resulting in a logical error in the SQL statement; semantic errors are errors in the interpretation of the business function of a SQL statement. For example, a SQL statement queries data for business A, but the original comment information interprets the SQL statement as querying data for business B.

[0068] Step 104: Determine the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine.

[0069] The knowledge data includes graph relation metadata and semantic vector metadata. Graph metadata stores the logical relationships between tables, fields, and business entities in the form of a knowledge graph, while semantic vector metadata is a high-dimensional vector obtained by encoding text such as field descriptions and business terms through a pre-trained language model (e.g., a pre-trained BERT model).

[0070] In this embodiment, the terminal first uses an SQL parser to parse and extract the original structured statement and original comment information, determine the structural elements such as table names, field names and conditional expressions contained in the original structured statement, and simultaneously extract the semantic text in the original comment information.

[0071] Before performing data correction, the terminal needs to pre-build a target knowledge database, which includes a vector knowledge base and a graph knowledge base. Specifically, the terminal obtains initial metadata from the database management system and builds an initial knowledge database based on the initial metadata; it parses the initial metadata according to naming conventions to obtain metadata annotations, and maps the metadata annotations according to the word vector model and the thesaurus to obtain candidate semantic vector metadata; and it builds the vector knowledge base and the graph knowledge base based on the candidate semantic vector metadata and the initial knowledge database, respectively.

[0072] For the original structured statements and original annotation information, the terminal uses a metadata query engine to retrieve structural elements such as table names and field names from the target knowledge database. This determines the graph relation metadata and semantic vector metadata that match the query conditions formed by the table names, field names, and conditional expressions. The metadata query engine combines the graph knowledge base and the vector knowledge base to achieve the fusion of structured relation queries and unstructured semantic retrieval. Furthermore, the graph relation metadata corresponding to the retrieved structured statements is used to verify logical accuracy (e.g., whether fields exist in the table), and the semantic vector metadata is used to determine whether the annotation semantics in the original annotation information are consistent with standard business terminology and whether the annotation semantics conform to business authenticity—that is, whether the function performed by the current structured statement is consistent with the business function explained by the current original annotation information.

[0073] In an optional embodiment, the target knowledge database can also be a metadata knowledge base based on a knowledge graph, containing both relationships between entities and enhanced semantic information about entities. For example, the terminal uses a target knowledge database with a structured graph-based knowledge network. The knowledge graph of the target knowledge database preserves the original semantics of the metadata, as well as the logical relationships and business meanings between entities. Specifically, the terminal first automatically extracts structured metadata from the target database's system directory or metadata management platform, including key information such as table names and their business comments, data types and constraint attributes of each column, primary and foreign key definitions, and index composition. Then, the terminal abstracts the metadata into nodes and edges in the graph database. For example, each table and each column (field) is modeled as an independent node, and the logical relationships between tables and columns (fields), such as attribution, primary keys, and foreign keys, are transformed into directed edges. For example, "column → BELONGS_TO → table" indicates field attribution, and "column A → FOREIGN_KEY_TO → table B" or more precisely, "column A → REFERENCES → column B" indicates a cross-table reference relationship for table names. In addition, the terminal can also capture potential JOIN possibilities through heuristic relationships. To further enhance the semantic expressive power of the graph, the terminal performs naming standard parsing on table and column names. For example, it identifies the "_id" suffix as an identifier and the "user" root as a user-related entity. Natural language processing techniques are then used to extract semantic information such as keywords or business tags from table or column annotations. Finally, all nodes and relationships are persistently stored in the graph database and an efficient index is created.

[0074] Step 106: Based on the data correction model, graph relation metadata, and semantic vector metadata, errors are identified in the original structured statements and original annotation information. The erroneous data obtained from the error identification is then corrected to obtain the target structured statements and target annotation information.

[0075] In this embodiment, the terminal uses graph relation metadata and semantic vector metadata as supplementary information to the original structured statement and original annotation information, and inputs them together into the data correction model. The data correction model identifies spelling errors, logical errors, and semantic errors in the original structured statement and original annotation information, obtaining error identification results. These error identification results contain erroneous data. Subsequently, the terminal automatically corrects spelling errors and corrects logical and annotation errors in the SQL statement through the data correction model, obtaining semantically and logically correct target structured statement and target annotation information.

[0076] Specifically, for spelling errors, the data correction big data model can identify and correct spelling errors based on pre-trained reasoning capabilities. For logical errors in fields of SQL statements, the data correction big data model can identify and correct them through graph relation metadata and semantic vector metadata. For example, if the SQL statement queries the number of data in field A1 of table A, and the graph relation metadata records that table A contains A2, which is semantically close to field A1, and the semantic vector metadata indicates that the SQL statement is querying data in field A2, then the data correction big data model will modify field A1 in the SQL statement to field A2.

[0077] In the aforementioned data correction method, the metadata query engine retrieves the graph relation metadata and semantic vector metadata corresponding to the structural elements in the target knowledge database. This supplements the information dimensions of the original structured statements and original annotation information, enabling the large-scale data correction model to not only correct grammatical errors in the original structured statements and original annotation information, but also to simultaneously detect and correct related semantic text errors and logical errors in the original structured statements and original annotation information. This enriches the dimensions of data correction and thus improves the accuracy of error correction for structured statements and annotations.

[0078] In an exemplary embodiment, the deconstruction and parsing of the original structured statement can be achieved jointly based on a large data correction model and a preset parser, such as... Figure 2 As shown, step 104 includes steps 202 to 206. Wherein:

[0079] Step 202: Parse the original structured statement according to the preset parsing strategy. If the parsing result is successful, obtain the structure element corresponding to the original structured statement.

[0080] In this embodiment, the preset parsing strategy prioritizes using a deterministic SQL parser for AST parsing. When parsing fails due to syntax errors or non-standard syntax, it switches to a large data correction model for semantically driven structural element extraction. Therefore, the terminal first calls the preset SQL parser to perform structured parsing on the original structured statement, and obtains an AST (Abstract Syntax Tree) when the parsing is successful. The AST contains the structural elements of the original structured statement, including Table nodes, Column nodes, and Comment nodes.

[0081] For the `Table` node, when extracting the table name, if an alias exists, the alias is mapped back to the real table name. For example, if the original table name is `user_profile` and the alias is `SELECT u.id FROM user_profile u`, then the alias `u` is mapped back to the real table name `user_profile`. Specifically, when parsing the original structured statement, a scope tree is generated by traversing the AST using `traverse_scope`. Each scope (e.g., outer query, subquery) is assigned a unique `scope_id`, and an independent alias mapping table is maintained within each scope (e.g., `a` maps to `users`, `b` maps to `orders`, or `b` maps to `logs`). When the parsed field is `b.user_id`, the terminal searches for the real table name pointed to by `b` in the alias mapping of the corresponding scope based on the `scope_id` associated with the AST node containing that field. This ensures that even if the alias is repeated in different scopes (e.g., outer `b` refers to `orders`, inner `b` refers to `logs`), it can be accurately mapped to the real table within its respective scope, avoiding cross-scope alias confusion.

[0082] For Column nodes, the terminal extracts the field names and their corresponding tables. For example, it extracts the field name `u.id` and the table `user_profile.id` to which `u.id` belongs. For Comment nodes, the terminal extracts the comment text immediately adjacent to the original structured statement. Furthermore, complex syntax such as UNION, WITH, and subqueries are processed separately. For instance, the terminal recursively parses each branch using a preset SQL parser, extracting the tables and fields involved in each original structured statement to ensure structural integrity.

[0083] Step 204: If the parsing result is parsing failure, the original structured statement is parsed and extracted according to the large structure parsing model to obtain the structure elements.

[0084] In this embodiment, when the preset SQL parser throws a parsing exception due to syntax errors, non-standard writing (such as dialect-specific syntax), or missing structure, the terminal automatically triggers a fallback mechanism, which inputs the original structured statement in its entirety into the structure parsing big model. Based on the context semantics and the structured statement patterns in the training data, the structure parsing big model directly outputs a list of structured elements containing structured elements.

[0085] When the default parser is unable to extract the structured data from the original SQL statement, the large-scale structure parsing model acts as a semantically aware structure extractor, replacing the default SQL parser to parse the original structured statement and annotation information. The large-scale structure parsing model receives training data containing a large number of noisy SQL samples (including spelling and grammatical errors) and is fine-tuned through sequence labeling or text-to-structure tasks, enabling it to recognize intent and key elements under non-standard input. Therefore, the large-scale structure parsing model does not rely on strict grammar but infers structure through contextual pattern matching.

[0086] Step 206: Determine the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine and structure elements.

[0087] In this embodiment, the terminal submits the structural elements output by the preset SQL parser or the structural parsing large model to the metadata query engine. The metadata query engine then performs a multimodal query in the target knowledge database to determine the knowledge data corresponding to the original structured statement.

[0088] In this embodiment, when the parsing result is a parsing failure, a "fallback" mechanism is implemented through the large structure parsing model to parse and extract the original structured statement, thereby improving the fault tolerance of parsing and extraction. This allows regular original structured statements to be parsed according to the preset SQL parser, while abnormal original structured statements are parsed through the large structure parsing model, which can improve the accuracy of SQL parsing.

[0089] In an exemplary embodiment, knowledge data retrieval for graph relation metadata and semantic vector metadata, such as... Figure 3 As shown, step 104 includes steps 302 to 304. Wherein:

[0090] Step 302: Based on the metadata query engine and the original annotation information, generate semantic vectors, and retrieve semantic vector metadata from the target knowledge database.

[0091] The target knowledge database for vector retrieval is the vector knowledge base.

[0092] In this embodiment, the terminal performs vectorization processing on the original annotation information based on the metadata query engine to generate a semantic vector corresponding to the original annotation information. Then, based on this semantic vector, it performs vector similarity retrieval in the vector knowledge base to obtain the similarity between each candidate vector metadata and the semantic vector. Candidate vector metadata with a similarity greater than a preset similarity threshold are then identified as semantic vector metadata. Specifically, the original annotation information corresponding to the original SQL statement is first extracted into natural language text and transformed into a high-dimensional semantic vector through a pre-trained semantic encoder. The metadata query engine performs an approximate nearest neighbor search on this semantic vector in the vector index of the vector knowledge base, thereby recalling semantic information such as field definitions that match the business intent, forming semantic-level contextual support, and finally obtaining the semantic vector metadata.

[0093] Step 304: In the target knowledge database, based on the metadata query engine, the existence detection and semantic association recommendation of each structural element in the original structured statement are performed to obtain the graph relation metadata corresponding to each structural element.

[0094] The target knowledge database for structural element retrieval is the graph knowledge base.

[0095] In this embodiment, the terminal uses a metadata query engine to compare the structural elements in the original structured statement with the knowledge data in the graph knowledge base. It performs an existence check to verify whether the tables or fields in the original structured statement actually exist in the current database schema information. Specifically, the terminal calculates a comprehensive score based on the similarity (table name string similarity) between the table name in the structural element and the table name in the schema, and the intersection ratio between the field name in the structural element and the field set of the database table in the schema. Table names with matching scores greater than a preset threshold are selected, sorted by score, and the table name with the highest score is used as the recommended correction table name. Furthermore, the terminal uses predefined entity relationships and data lineage in the graph knowledge base to perform semantic association recommendations for each structural element based on the context of associated objects and node relationships. For example, when retrieving knowledge data for the field b.end_dt, the graph knowledge base returns structured knowledge data such as its table, data type, whether it is a validity period field, and typical value rules.

[0096] In this embodiment, by integrating semantic vector metadata and graph relation metadata, the terminal can determine whether the SQL is "syntactically correct" and whether it is "business-reasonable", thereby achieving deep understanding from surface parsing and enabling the detection of deep logical errors in complex business scenarios (e.g., financial risk control), thus improving the accuracy of data correction.

[0097] In one exemplary embodiment, such as Figure 4As shown, step 302 includes steps 402 to 406. Wherein:

[0098] Step 402: Based on the metadata query engine, the natural language text in the original annotation information is converted into a rule expression in the target format to obtain the target annotation information.

[0099] In this embodiment, the terminal uses a metadata query engine to convert the natural language format business description in the original annotation information into a verifiable rule expression. This rule expression is then used as the target annotation information contained in the original annotation information, and format conversion is performed using the target annotation information. For example, a natural language format business description might indicate that the end date is 3000-12-31, meaning it is currently valid. After conversion by the metadata query engine, the rule expression "END_DT=TO_DATE('3000-12-31','%Y-%m-%d')" is obtained. This transforms the ambiguous business description into a verifiable logical form for data correction within the large model, serving as supplementary knowledge data to the original annotation information.

[0100] Step 404: Based on the metadata query engine, the structural elements in the original structured statement are parsed, and each structural element is semantically bound to the target annotation information to obtain the association between the rule expression and the structural elements.

[0101] In this embodiment, the terminal constructs the correspondence between structural elements and target annotation information through a metadata query engine. Specifically, the metadata query engine combines the syntax rules of the SQL statement with the location of the target knowledge information (e.g., inline comments, block comments) to determine which part of the original structured statement the target annotation information most likely modifies. For example, target annotation information immediately following the WHERE clause is usually related to that condition; target annotation information following a SELECT column typically describes the business meaning of that column.

[0102] Then, the metadata query engine semantically aligns the target annotation information with the parsed structure elements, either by matching field names or by data type, business tags, or graph relationships, ultimately obtaining a structured binding result as the association between the rule expression and the structure element. For example, the structure element is a table named `orders` (aliased as `b`), with a column `b.end_dt` and a conditional expression `b.end_dt='3000-12-31'`. Furthermore, because `END_DT` in the rule expression and `b.end_dt` in the structure element have a high degree of naming match, the terminal uses the metadata query engine to determine the association between "b.end_dt" and "END_DT=TO_DATE('3000-12-31','%Y-%m-%d')".

[0103] Step 406: Generate semantic vectors corresponding to the original annotation information based on the metadata query engine, and retrieve the semantic vectors and relationships in the target knowledge database to obtain the semantic vector metadata corresponding to each structural element.

[0104] In this embodiment, the terminal calls a pre-trained semantic encoding model through a metadata query engine to encode the original annotation information into a high-dimensional semantic vector. For example, the terminal encodes the original annotation information "End date = 3000-12-31 indicates that it is currently valid" through the semantic encoding model to generate a semantic vector, so as to preserve the semantic integrity of the business intent.

[0105] Then, the metadata query engine identifies the current structural element for semantic retrieval based on the relationship between the regular expression and the structural element, and retrieves its semantic vector knowledge from the vector knowledge base to obtain the semantic vector metadata of the current structural element. Following the same principle, the metadata query engine iterates through all structural elements to obtain the semantic vector metadata corresponding to each structural element.

[0106] In one specific embodiment, the metadata query engine encodes the original annotation information "end date = 3000-12-31 indicates that it is currently valid" into a semantic vector V1, and determines that the current retrieval is for the field b.end_dt based on the rule expression of the original annotation information. Then, the terminal uses the metadata query engine to search the target knowledge database with V1 as the query vector and returns multiple semantic vector metadata.

[0107] In this embodiment, by converting the natural language text in the original annotation information into a rule expression in the target format, the semantic binding between each structural element and the target annotation information is realized, which can improve the accuracy of retrieving the semantic vector metadata corresponding to the structural elements.

[0108] In one exemplary embodiment, such as Figure 5 As shown, step 106 includes steps 502 to 504. Wherein:

[0109] Step 502: Based on the data correction of the large model and graph relationship metadata, analyze the logical structure of the original structured statements and original annotation information, and identify logical errors in the original structured statements and original annotation information.

[0110] In this embodiment, the terminal analyzes the logical structure of the original structured statement and original annotation information based on the data correction big model and graph relationship metadata, and identifies any logical errors. Specifically, the data correction big model uses the constraint rules provided by the graph (including foreign key relationships, NOT NULL constraints, and enumeration value ranges) for reasoning. For example, if WHERE user_id='abc', user_id is an integer primary key, or the annotation requires "only querying valid users," but the SQL does not include the status='ACTIVE' condition, the data correction big model determines that there is a logical inconsistency between the original structured statement and the original annotation information, and marks the logical error.

[0111] Step 504: Based on the data correction model and semantic vector metadata, analyze the semantics of the original structured statements and original annotation information to identify semantic errors in the original structured statements and original annotation information.

[0112] In this embodiment, the terminal corrects the large model and semantic vector metadata based on the data, analyzes the semantics of the original structured statement and original annotation information, and identifies semantic errors. The large model uses semantic vector metadata to understand business intent. For example, if the current date is October 2025, and the original annotation information is described as "statistics last month's sales," but the time condition described in the original structured statement is "order_date>='2025-09-01'," the terminal identifies a deviation in the time range of the semantic error in the original structured statement or original annotation information by comparing the time semantic vector with the actual expression; or, the field amount is mistakenly used for counting instead of summing, indicating a semantic error in the original annotation information.

[0113] In this embodiment, the original structured statements and original annotation information are supplemented by graph relation metadata and semantic vector metadata. This enables the identification of logical errors through structured graph relation metadata and semantic errors through semantic vector metadata. As a result, the large data correction model can perform data correction in more dimensions, improving the accuracy of error correction for structured statements and annotations.

[0114] In one exemplary embodiment, step 106 includes step 1061. Wherein:

[0115] Step 1061: Based on the data correction big model, graph relation metadata and semantic vector metadata, perform data correction on the logical errors and / or semantic errors obtained by error identification to obtain the target structured statement and target annotation information.

[0116] In this embodiment, the terminal utilizes the generation and reasoning capabilities of the data correction model to accurately correct identified logical and / or semantic errors based on the structured business knowledge of graph relationship metadata and the contextual semantic information of semantic vector metadata. Specifically, the data correction model takes the original structured statement, bound rule expressions, graph relationship metadata, and retrieved semantic vector metadata as input, comprehensively judges the error type and correction direction, and infers to generate a target structured statement that conforms to business semantics and system specifications. Simultaneously, it updates or standardizes the corresponding target annotation information to ensure that the corrected statement is not only grammatically correct but also accurate, explainable, and traceable in terms of business logic. For example, when the field b.end_dt is detected to be assigned a non-standard value of "2099-12-31", the data correction model corrects the assignment of the field to TO_DATE('3000-12-31','%Y-%m-%d') by using the rule in the graph relation metadata that "the effective deadline should be 3000-12-31" and the semantic vector metadata, while retaining the normalized target annotation information, thereby achieving end-to-end automation from error identification to semantic alignment correction.

[0117] In this embodiment, a large data correction model is used to correct grammatical errors in the original structured statements and original annotation information. Simultaneously, related semantic text errors and logical errors in the original structured statements and original annotation information are detected and corrected, which enriches the dimensions of data correction and thus improves the accuracy of error correction for structured statements and annotations.

[0118] In one specific embodiment, an example of a data processing method is provided, such as... Figure 6 As shown, where:

[0119] Step 601: Obtain the SQL input and comment text from the user.

[0120] Step 602: Can the parser parse the SQL and comment text? If it can, proceed to step 603; if parsing fails, proceed to step 604.

[0121] Step 603: Parse the SQL and comment text using the parser to obtain a syntax tree containing structured elements.

[0122] Step 604: Call the large model to parse the SQL and comment text to obtain a syntax tree containing structured elements.

[0123] Step 605: Retrieve data from the knowledge base using a scenario-based metadata query engine to obtain graph relation reasoning and vector semantic matching.

[0124] Step 606: Based on graph relation reasoning and vector semantic matching, error detection and marking are performed to obtain SQL syntax errors and comment semantic conflicts.

[0125] Step 607: Correct SQL syntax errors and comment semantic conflicts according to the intelligent error correction model, and output the corrected SQL statement and the updated comment text.

[0126] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0127] Based on the same inventive concept, this application also provides a data correction apparatus for implementing the data correction method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more data correction apparatus embodiments provided below can be found in the limitations of the data correction method described above, and will not be repeated here.

[0128] In one exemplary embodiment, such as Figure 7 As shown, a data correction device 700 is provided, including: an acquisition module 701, a query module 702, and an identification module 703, wherein:

[0129] Module 701 is used to obtain the original structured statements and original comment information;

[0130] The query module 702 is used to determine the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine; the knowledge data includes graph relation metadata and semantic vector metadata;

[0131] The identification module 703 is used to identify errors in the original structured statement and original annotation information based on the data correction large model, graph relation metadata and semantic vector metadata, and to correct the erroneous data obtained from the error identification to obtain the target structured statement and target annotation information.

[0132] In one embodiment, the query module 702 is specifically used to parse the original structured statement according to a preset parsing strategy. If the parsing result is successful, the structural element corresponding to the original structured statement is obtained.

[0133] If the parsing result is parsing failure, the original structured statement is parsed and extracted according to the large structure parsing model to obtain the structure elements;

[0134] Based on the metadata query engine and structural elements, the knowledge data corresponding to the original structured statements is determined in the target knowledge database.

[0135] In one embodiment, the query module 702 is specifically used to generate a semantic vector based on the metadata query engine and the original annotation information, and to retrieve the semantic vector metadata from the target knowledge database.

[0136] In the target knowledge database, the existence detection and semantic association recommendation of each structural element in the original structured statement are performed based on the metadata query engine to obtain the graph relation metadata corresponding to each structural element.

[0137] In one embodiment, the query module 702 is specifically used to convert the natural language text in the original annotation information into a rule expression in the target format based on the metadata query engine, so as to obtain the target annotation information;

[0138] Based on the metadata query engine, the structural elements in the original structured statement are parsed, and each structural element is semantically bound to the target annotation information to obtain the association between the rule expression and the structural element.

[0139] The semantic vectors corresponding to the original annotation information are generated based on the metadata query engine, and the semantic vectors and their relationships are used to retrieve the semantic vector metadata corresponding to each structural element from the target knowledge database.

[0140] In one embodiment, the identification module 703 is specifically used to correct the metadata of the large model and the graph relationship based on the data, analyze the logical structure of the original structured statements and the original annotation information, and identify logical errors in the original structured statements and the original annotation information.

[0141] Based on the data correction model and semantic vector metadata, the semantics of the original structured statements and original annotation information are analyzed to identify semantic errors in the original structured statements and original annotation information.

[0142] In one embodiment, the identification module 703 is specifically used to correct logical errors and / or semantic errors obtained from error identification based on the data correction large model, graph relation metadata and semantic vector metadata, so as to obtain target structured statements and target annotation information.

[0143] Each module in the aforementioned data correction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0144] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data correction method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0145] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0146] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0147] Obtain the original structured statements and original comment information;

[0148] The knowledge data corresponding to the original structured statements is determined in the target knowledge database based on the metadata query engine; the knowledge data includes graph relation metadata and semantic vector metadata.

[0149] Based on the data correction model, graph relation metadata, and semantic vector metadata, errors are identified in the original structured statements and original annotation information. The erroneous data obtained from the error identification is then corrected to obtain the target structured statements and target annotation information.

[0150] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0151] The original structured statement is parsed according to the preset parsing strategy. If the parsing result is successful, the structure element corresponding to the original structured statement is obtained.

[0152] If the parsing result is parsing failure, the original structured statement is parsed and extracted according to the large structure parsing model to obtain the structure elements;

[0153] Based on the metadata query engine and structural elements, the knowledge data corresponding to the original structured statements is determined in the target knowledge database.

[0154] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0155] Based on the metadata query engine and the original annotation information, a semantic vector is generated, and the semantic vector is retrieved from the target knowledge database to obtain the semantic vector metadata.

[0156] In the target knowledge database, the existence detection and semantic association recommendation of each structural element in the original structured statement are performed based on the metadata query engine to obtain the graph relation metadata corresponding to each structural element.

[0157] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0158] Based on the metadata query engine, the natural language text in the original annotation information is converted into a rule expression in the target format to obtain the target annotation information;

[0159] Based on the metadata query engine, the structural elements in the original structured statement are parsed, and each structural element is semantically bound to the target annotation information to obtain the association between the rule expression and the structural element.

[0160] The semantic vectors corresponding to the original annotation information are generated based on the metadata query engine, and the semantic vectors and their relationships are used to retrieve the semantic vector metadata corresponding to each structural element from the target knowledge database.

[0161] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0162] Based on the data correction of the large model and graph relationship metadata, the logical structure of the original structured statements and original annotation information is analyzed to identify logical errors in the original structured statements and original annotation information;

[0163] Based on the data correction model and semantic vector metadata, the semantics of the original structured statements and original annotation information are analyzed to identify semantic errors in the original structured statements and original annotation information.

[0164] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0165] Based on the data correction model, graph relation metadata, and semantic vector metadata, logical errors and / or semantic errors obtained from error identification are corrected to obtain target structured statements and target annotation information.

[0166] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0167] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0168] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0169] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0170] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0171] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data correction method characterized by, The method comprises: acquiring an original structured statement and original annotation information; determining knowledge data corresponding to the original structured statement in a target knowledge database based on a metadata query engine; the knowledge data comprises graph relationship metadata and semantic vector metadata; performing error identification on the original structured statement and the original annotation information according to a data correction large model, the graph relationship metadata and the semantic vector metadata, and performing data correction on error data obtained through error identification to obtain a target structured statement and target annotation information.

2. The method of claim 1, wherein, The determination of the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine comprises: parsing the original structured statement according to a preset parsing strategy, and obtaining a structure element corresponding to the original structured statement if the parsing result is successful; if the parsing result is unsuccessful, parsing and extracting the original structured statement according to a structure parsing large model to obtain the structure element; determining the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine and the structure element.

3. The method of claim 1, wherein, The determination of the knowledge data corresponding to the original structured statement in the target knowledge database based on the metadata query engine comprises: generating a semantic vector based on the metadata query engine and the original annotation information, and performing retrieval in the target knowledge database according to the semantic vector to obtain semantic vector metadata; performing existence detection and semantic association recommendation on each structure element in the original structured statement in the target knowledge database based on the metadata query engine to obtain graph relationship metadata corresponding to each structure element.

4. The method of claim 3, wherein, The generation of the semantic vector based on the metadata query engine and the original annotation information, and the retrieval in the target knowledge database according to the semantic vector to obtain semantic vector metadata comprises: converting natural language text in the original annotation information into a rule expression in a target format based on the metadata query engine to obtain target annotation information; performing semantic binding of each structure element and the target annotation information based on the metadata query engine to obtain an association relationship between the rule expression and the structure element by parsing the structure element in the original structured statement; generating a semantic vector corresponding to the original annotation information based on the metadata query engine, and performing retrieval in the target knowledge database based on the semantic vector and the association relationship to obtain semantic vector metadata corresponding to each structure element.

5. The method of claim 1, wherein, The error identification on the original structured statement and the original annotation information according to the data correction large model, the graph relationship metadata and the semantic vector metadata comprises: analyzing a logical structure of the original structured statement and the original annotation information according to the data correction large model and the graph relationship metadata to identify logical errors existing in the original structured statement and the original annotation information; According to the data correction large model and the semantic vector metadata, semantics of the original structured sentence and the original annotation information are analyzed, and semantic errors existing in the original structured sentence and the original annotation information are identified.

6. The method of claim 1, wherein, The data correction on the error data identified in the error identification is performed to obtain target structured sentences and target annotation information, including: The data correction on the logical errors and / or semantic errors identified in the error identification is performed based on the data correction large model, the graph relationship metadata, and the semantic vector metadata to obtain target structured sentences and target annotation information.

7. A data correction device characterized by comprising: The apparatus includes: An acquisition module configured to acquire original structured sentences and original annotation information; A query module configured to determine knowledge data corresponding to the original structured sentences in a target knowledge database based on a metadata query engine; the knowledge data includes graph relationship metadata and semantic vector metadata; An identification module configured to perform error identification on the original structured sentences and the original annotation information according to a data correction large model, the graph relationship metadata, and the semantic vector metadata, and perform data correction on error data identified in the error identification to obtain target structured sentences and target annotation information.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images