A data specification inspection method and system based on a data dictionary

Abstract

The application provides a data specification inspection method and system based on a data dictionary. The inspection method comprises the following steps: adding database connection information; querying database metadata to obtain a database modeling statement; using ANTLR to analyze the database modeling statement to obtain attribute information corresponding to the database; inputting feature information of a data element to obtain file information; querying the obtained file information; comparing field names, types and length information parsed in the modeling statement; and outputting a comparison result. The application provides a more intelligent, efficient and low-cost inspection method, and improves the quality and efficiency of information construction.

Description

Technical Field

[0001] This invention relates to the field of enterprise information system construction, and in particular to a data specification verification method and system based on a data dictionary. Background Technology

[0002] In the process of building enterprise information systems, the naming, types, and usage scenarios of fields vary across different business systems. Database modeling generally lacks unified field design standards, and the compliance of data fields is a key factor in ensuring data quality and system stability. A data dictionary is a data management tool that records information such as the definition, format, source, purpose, and limitations of data elements. By uniformly defining the Chinese and English names, types, meanings, and usage scenarios of dictionary items, inconsistencies in the data modeling process can be reduced. Based on this, ANTLR (Another Tool for Language Recognition) can process structured text data. By writing corresponding syntax rules, it can parse user-transmitted text or database modeling statements obtained by querying database metadata, extracting the corresponding field names, types, lengths, etc., and comparing them with the expected structure and rules of the data elements in the data dictionary. Using ANTLR for data dictionary validation can fully utilize ANTLR's parsing and generation capabilities to efficiently meet data dictionary validation requirements.

[0003] 1. Database field design based on manual inspection and review by business personnel.

[0004] 2. Verify the data by querying the name, type, and length in the database metadata and comparing them with the name, type, and length defined in the data dictionary.

[0005] 1. Issues with manual inspection of database field design

[0006] 1.1 Insufficient inspection due to human error

[0007] Relying on manual inspection by business personnel introduces subjective human factors, which may lead to omissions and result in incomplete and inaccurate inspection results for database field designs.

[0008] 1.2 High cost and time consumption

[0009] Manual inspection requires a significant investment of manpower and time, resulting in high costs and a lengthy process. During the implementation of information technology infrastructure, this method can lead to substantial cost pressures on projects, impacting their schedule and efficiency.

[0010] 2. Issues with querying database system metadata

[0011] 2.1 Depends on database metadata tables

[0012] Verification by querying database metadata tables relies on this metadata information. Different database systems have different metadata table structures, requiring additional manpower and time for adaptation, increasing the complexity and cost of the operation.

[0013] 2.2 Unable to intercept verification before database execution

[0014] This method can only perform checks when querying database system metadata, and cannot perform interceptive checks before system developers write modeling statements and send them to the database for execution. This results in the inability to promptly detect and correct potential field design problems before the actual database statements are executed. Summary of the Invention

[0015] In view of the above problems, the present invention is proposed to provide a data specification verification method and system based on a data dictionary to overcome or at least partially solve the above problems.

[0016] According to one aspect of the present invention, a data specification verification method based on a data dictionary is provided, the verification method comprising:

[0017] Add database connection information;

[0018] Query database metadata to obtain database modeling statements;

[0019] Use ANTLR to parse database modeling statements and obtain the corresponding attribute information of the database.

[0020] Enter the feature information of data elements and obtain file information;

[0021] Once the file information is retrieved, the field names, types, and lengths parsed from the modeling statement are compared, and the comparison results are output.

[0022] Optionally, the database connection information includes: connection name, connection address, port number, username, and password.

[0023] Optionally, after adding the database connection information, the process may also include: viewing the table structure and table data in the database on the IDE page, and writing SQL statements in the SQL editing interface.

[0024] Optionally, the query statement for obtaining database modeling data from database metadata specifically includes:

[0025] Query database metadata to obtain database modeling statements;

[0026] Receive the database modeling statements to be executed via the HTTP interface;

[0027] Intercept SQL statements written in the SQL editing interface.

[0028] Optionally, the attribute information corresponding to the database specifically includes: table name, field name, and type.

[0029] Optionally, the feature information of the data element specifically includes: definition, format, source, purpose, and limitations.

[0030] Optionally, the file information specifically includes: Chinese name, English name, type, and length information.

[0031] This invention also provides a data specification verification system based on a data dictionary, applying the aforementioned data specification verification method based on a data dictionary, wherein the verification method includes:

[0032] An IDE module used to add database connection information;

[0033] The database modeling statement retrieval module is used to query database metadata to obtain database modeling statements;

[0034] The syntax parsing module is used to parse database modeling statements using ANTLR to obtain the corresponding attribute information of the database.

[0035] The data dictionary module is used to input the characteristic information of data elements and obtain file information;

[0036] The verification module is used to query the obtained file information, compare the field names, types, and length information parsed into the modeling statement, and output the comparison results.

[0037] This invention provides a data specification verification method and system based on a data dictionary. The verification method includes: adding database connection information; querying database metadata to obtain database modeling statements; using ANTLR to parse the database modeling statements to obtain the corresponding attribute information of the database; inputting the feature information of data elements to obtain file information; querying the obtained file information; comparing the field names, types, and length information parsed from the modeling statements; and outputting the comparison results. This provides a more intelligent, efficient, and low-cost verification method, improving the quality and efficiency of information technology construction.

[0038] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 A flowchart of a data specification verification method based on a data dictionary is provided for an embodiment of the present invention. Detailed Implementation

[0041] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0042] The terms "comprising" and "having," and any variations thereof, in the specification, embodiments, claims, and drawings of this invention are intended to cover non-exclusive inclusion, such as including a series of steps or units.

[0043] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0044] A data specification verification system based on ANTLR and a data dictionary includes the following modules:

[0045] IDE module: Add database connection information, including connection name, connection address, port number, username, and password. View the table structure and data in the database on the IDE page, and write SQL statements in the SQL editing interface.

[0046] Database modeling statement acquisition module: Query database metadata to obtain database modeling statements, or receive database modeling statements to be executed via HTTP interface, or intercept SQL statements written in the SQL editing interface.

[0047] Syntax parsing module: Uses ANTLR to parse database modeling statements and obtain the corresponding table names, field names, and type information.

[0048] Data dictionary module: Enter the definition, format, source, purpose and restriction information of data elements.

[0049] The verification module retrieves the Chinese name, English name, type, and length information from the data dictionary module, compares it with the field name, type, and length information parsed from the modeling statement by the syntax parsing module, and outputs the comparison results.

[0050] The present invention provides a data specification verification system based on ANTLR and a data dictionary, comprising the following steps:

[0051] Figure 1 This is a flowchart illustrating the implementation of a data specification verification method based on ANTLR and a data dictionary, as provided in this embodiment of the invention. Figure 1 ,include:

[0052] Step 1: In the IDE module, fill in the database connection address, port number, username, password and other information and save it to the OceanBase database for storage.

[0053] Step 2: The first method involves writing the SQL statement to be executed in the IDE interface, clicking the execute button, and the program retrieving the executed SQL statement. The second method involves the business user clicking the verification button, and the system retrieving the table creation statement from the target database using the DBMS_METADATA.GET_DDL method or the show create table statement by querying the connection information stored in OceanBase in the first step. The third method involves the business user transmitting their written SQL statement to this system via a REST interface.

[0054] Step 3: Take the table creation statement obtained in the previous step, call the sqlAnalysis method, and use ANTLR to analyze the target structured query language SQL statement to obtain the syntax tree (ParseTree); traverse the ParseTree to obtain the table name, field name, type, and length information.

[0055] Step 4: Query all data element information in the data dictionary to obtain the Chinese name, English name, type, and length information of the data fields.

[0056] Step 5: Use the field names obtained in Step 3 to search for them in the English names of the data dictionary information obtained in Step 4. If the field cannot be found, mark it as not conforming to the rules. If the relevant field is found, compare the data type and length of the two fields in turn. If either of them does not match, mark the field as not conforming to the rules.

[0057] Step 6: Output all fields in the modeling statement and display the field names and whether the fields conform to the rules.

[0058] Beneficial effects:

[0059] Automated inspection mechanism:

[0060] Introducing an automated verification mechanism reduces reliance on business personnel. By using algorithms and rules to perform comprehensive verification of database field designs, human error can be avoided.

[0061] Interception verification function:

[0062] It can both verify existing database fields by querying database metadata and perform interception verification before system developers send modeling statements to the database for execution. By detecting and correcting field design problems in advance, it reduces the possibility of problems entering the database and improves the stability and quality of the system.

[0063] High efficiency and low cost:

[0064] By using automated and intelligent inspection methods, we have reduced manpower input and improved inspection efficiency, thereby reducing the cost pressure during the information technology construction process.

[0065] This invention provides a more intelligent, efficient, and low-cost verification method for database field design, thereby improving the quality and efficiency of information technology construction.

[0066] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A data standardization verification method based on a data dictionary, characterized in that, The testing method includes: Add database connection information. In the IDE module, fill in the database connection address, port number, username, and password information, and save it to the OceanBase database for storage. Retrieve database modeling statements from database metadata, including: Write the SQL statement to be executed in the IDE interface, click the execute button in the interface, and the program will retrieve the SQL statement to be executed; When the business personnel click the verification button, the system retrieves the table creation statement from the target database by querying the connection information stored in OceanBase in the first step, and using the DBMS_METADATA.GET_DDL method or the show create table statement. Business users can transmit their own SQL statements to this system via a REST interface; ANTLR is used to parse database modeling statements to obtain the corresponding attribute information of the database, including: The obtained table creation statement is processed by calling the sqlAnalysis method, which analyzes the target structured query language (SQL) statement using ANTLR to obtain a syntax tree (ParseTree). The ParseTree is then traversed to obtain the table name, field names, types, and length information. Enter the feature information of data elements and obtain file information, including: Query all data element information in the data dictionary to obtain the Chinese name, English name, type, and length information of the data fields; After retrieving the file information, the field names, types, and lengths parsed from the modeling statements are compared, and the comparison results are output, including: Use the obtained field name to search in the English name of the data dictionary information. If it cannot be found, mark the field as not conforming to the rule. If the relevant field is found, compare the two fields in turn to see if their types and lengths match. If any one of them does not match, mark the field as not conforming to the rule. Output all fields from the modeling statement and display the field names and whether the fields conform to the rules.

2. The data standardization verification method based on a data dictionary according to claim 1, characterized in that, The database connection information includes: connection name, connection address, port number, username, and password.

3. The data standardization verification method based on a data dictionary according to claim 1, characterized in that, After adding database connection information, the process also includes: viewing the table structure and table data in the database on the IDE page, and writing SQL statements in the SQL editing interface.

4. The data standardization verification method based on a data dictionary according to claim 1, characterized in that, The specific statements for querying database metadata to obtain database modeling statements include: Query database metadata to obtain database modeling statements; Receive the database modeling statements to be executed via the HTTP interface; Intercept SQL statements written in the SQL editing interface.

5. The data standardization verification method based on a data dictionary according to claim 1, characterized in that, The specific characteristics of the data elements include: definition, format, source, purpose, and limitations.

6. A data specification verification system based on a data dictionary, employing the data specification verification method based on a data dictionary as described in any one of claims 1-5, characterized in that, The testing method includes: An IDE module used to add database connection information; The database modeling statement retrieval module is used to query database metadata to obtain database modeling statements; The syntax parsing module is used to parse database modeling statements using ANTLR to obtain the corresponding attribute information of the database. The data dictionary module is used to input the characteristic information of data elements and obtain file information; The verification module is used to query the obtained file information, compare the field names, types, and length information parsed into the modeling statement, and output the comparison results.

Images