Physical data model quality auditing method and system based on metadata

Abstract

The invention discloses a physical data model quality auditing method and system based on metadata, and relates to the technical field of data auditing, and the method comprises the steps: creating a quality auditing rule of a physical data model, and obtaining the metadata of the physical data model through a converted result; checking the obtained metadata, and sending a generated review report to a designer of the corresponding problem model to complete updating of the quality of the physical data model; according to the method, the automation and accuracy of data model quality examination are remarkably improved, structural defects and data consistency problems in the data model can be quickly positioned and repaired, continuous tracking and optimization of the system on the model quality are ensured, and the system reliability is improved. And the visually displayed tracking report further improves the readability of the auditing result, so that a designer can efficiently identify and solve problems, and finally, the overall quality and performance of the data model are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data auditing, and in particular to a physical data model quality auditing method and system based on metadata. Background Art

[0002] At present, with the advent of the big data era and the rapid development of information technology, the importance of data in enterprise decision-making and operations has become increasingly prominent. Especially in the construction and management of physical data models, ensuring the quality of data models is crucial to improving system stability and data consistency. However, with the continuous increase in data volume and complexity, traditional manual inspection methods can no longer meet the needs of efficient and accurate data quality review. Therefore, metadata-based automated data quality audit methods have gradually become a key technology in enterprise digital transformation.

[0003] At present, in the existing technology, there are many deficiencies in the quality inspection of physical data models, including insufficient refinement of review rules, lack of automated support in the inspection process, and difficulty in tracking data quality. These deficiencies not only lead to errors and inconsistencies in data models, but also increase the complexity of maintenance and updating. To address these problems, the present invention proposes a metadata-based physical data model quality audit method, which realizes a comprehensive audit of the quality of physical data models through the creation of automated review rules, metadata inspection and visual tracking, and significantly improves the reliability and management efficiency of data models. Summary of the invention

[0004] In view of the problems existing in the existing metadata-based physical data model quality audit method and system, the present invention is proposed.

[0005] Therefore, the purpose of the present invention is to provide a physical data model quality audit method and system based on metadata. In view of the problems of insufficient efficiency of manual inspection and difficulty in data quality tracking in the prior art, the present invention adopts metadata-based automated review rule creation, automated inspection task generation and visual tracking technology to solve them.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a physical data model quality audit method based on metadata, which includes creating a quality review rule for the physical data model, converting a division result of the review rule, and obtaining metadata of the physical data model using the converted result;

[0008] Check the obtained metadata, generate a review report on the quality of the physical data model, and send the generated review report to the designer of the corresponding problem model to complete the update of the quality of the physical data model;

[0009] Record the review reports before and after the update, establish a tracking report on the quality of the physical data model, visualize the tracking report on the quality of the physical data model, build a chart distribution of the quality of the physical data model, and conduct an audit based on the distribution.

[0010] As a preferred solution of the metadata-based physical data model quality audit method of the present invention, wherein: the quality review rules for creating the physical data model include creating the quality review rules for the physical data model according to the execution language requirements of the database, and the quality review rules include structural integrity, data consistency and performance optimization. The system automatically collects and analyzes demand inputs from different business departments, and calculates the demand input matrix. The specific calculation formula of the demand input matrix is:

[0011]

[0012] Among them, R ij represents the weight of the jth requirement of the i-th category rule, ω k represents the weight of k business departments, N ijk Represents the score of the k-th business department on the j-th requirement of the i-th category rule.

[0013] As a preferred solution of the metadata-based physical data model quality audit method of the present invention, wherein: the conversion of the division results of the review rules includes the system automatically generating an SQL query inspection script according to different rule categories and weights, and the inspection tasks generated based on the inspection script, the inspection tasks include structural integrity inspection, data consistency inspection and performance optimization inspection;

[0014] The system inputs the matrix results according to the requirements and dynamically generates an inspection task list. The specific calculation formula of the inspection task list is:

[0015] T s ={T ij |R ij >threshold}

[0016] Among them, T s represents the set of inspection tasks that meet the threshold conditions, T ij represents the inspection task, R ij represents the weight of the jth requirement of the i-th category rule, and threshold represents the preset weight threshold;

[0017] The obtaining of the metadata of the physical data model includes the system automatically connecting to the database through the generated inspection task, executing the inspection task, and obtaining the metadata of the physical data model;

[0018] The system executes each task in sequence according to the generated inspection task list and collects metadata of the data model. The specific calculation formula for collecting metadata is:

[0019] M={m i |m i ∈execute(T ij )}

[0020] Among them, M represents the metadata set obtained, m i Represents a metadata item, execute represents the function that executes the inspection task, T ij Indicates an inspection task.

[0021] As a preferred solution of the metadata-based physical data model quality audit method of the present invention, wherein: the checking of the acquired metadata includes extracting the metadata of the physical data model from the database, customizing the field information of the metadata according to different metadata types, and checking the structural attributes in the field information;

[0022] If the field information lacks a primary key, the field information at this time is marked, the problem is recorded, and the recorded problem is optimized;

[0023] When the redundant indexes in the optimization process exceed the preset standard of the field information, the optimized field information does not meet the predefined data type specification. The preset standard includes objection to the problem of the record. The specific steps of the objection are as follows:

[0024] If the foreign key constraint between the field information does not satisfy the data consistency check, the foreign key relationship problem is recorded and marked for repair;

[0025] If the primary key in the direction pointed by the foreign key of the field information does not exist, the record mark repair process is recorded to clear the data.

[0026] As a preferred solution of the physical data model quality audit method based on metadata of the present invention, wherein: the generation of the physical data model quality review report includes, after completing the quality inspection of the physical data model, the system establishes different levels of review reports according to the inspection results, and determines the severity of the review report according to the number of inspections received by the physical data model;

[0027] When the same designer is responsible for N issues, the review report is allocated according to the recorded issues and designer information. The specific calculation formula for allocating the designer information is:

[0028] S={D i |M i ∈P(D i )}

[0029] Among them, S represents the allocation result of the designer, D i represents the designer, P(D i ) represents the set of problems for which the designer is responsible.

[0030] As a preferred solution of the physical data model quality audit method based on metadata of the present invention, wherein: the updating of the physical data model quality includes the system automatically checking the updated data model submitted by the designer based on the reported problems received by the designer to verify whether the problems have been fixed;

[0031] If the problem is detected to have been fixed, the system updates the review report status to Resolved;

[0032] If the detected problem is not repaired, record the reason for not repairing it and notify the designer to verify the various steps of repair;

[0033] Construct multiple thresholds T i , check the quality review rules, the specific check calculation formula is:

[0034] T s ={T i |R i >T i}

[0035] Among them, T s represents the set of repair items that have passed verification, T i Represents the constructed multiple thresholds, R i represents the verification result of the i-th repair;

[0036] When R i >T i When , the set of repair items that have passed the verification is in a qualified state;

[0037] When R i ≤T i When , the verified repair item set is in an unqualified state, at this time, the structural integrity, data consistency and performance optimization are checked;

[0038] The checking of structural integrity includes checking the table structure, table field types and constraint conditions of the structure;

[0039] The specific calculation formula for checking the structural integrity is:

[0040]

[0041] Where S represents the structural integrity score, ω j represents the weight of the j-th check, C jQualified variables representing structural integrity checks;

[0042] When C j ≤S, the structural integrity is passed and it is considered qualified; otherwise, when C j >S, record the detailed problem and notify the designer;

[0043] The data consistency check includes comparing the data consistency score with the qualified variables of the data consistency check. The specific calculation formula of the data consistency check is:

[0044]

[0045] Where D represents the data consistency score, d k Indicates the qualified variable of data consistency check, m indicates the total number of data, and k indicates the kth data;

[0046] When k ≤D, the data consistency is passed and it is recorded as qualified; otherwise, when d k >D, record the detailed problem and notify the designer;

[0047] The performance optimization check includes comparing the performance optimization score of the data with the performance optimization qualified variables of the data. The specific calculation formula of the performance optimization is:

[0048]

[0049] Among them, P represents the performance optimization score, p l represents the weight of the lth test, R l represents eligible variables for performance optimization;

[0050] When R l ≤P, the performance optimization is passed and it is recorded as qualified; otherwise, when R l >P, record the detailed problem and notify the designer.

[0051] As a preferred solution of the metadata-based physical data model quality audit method of the present invention, the establishment of a physical data model quality tracking report includes the system comprehensively evaluating the results of the inspection task and generating a repaired tracking report. The specific repair calculation formula is:

[0052]

[0053] Among them, R x It represents the tracking report of the repair results, S represents the structural integrity score, D represents the data consistency score, and P represents the performance optimization score;

[0054] Use heatmaps to display the quality tracking report of the established physical data model, and judge the repaired tracking report based on the depth of the heatmap color;

[0055] The darker the color of the heat map, the heavier the traces of the repaired tracking report. At this time, it is necessary to conduct a subsequent observation and audit of the repaired tracking report.

[0056] In a second aspect, an embodiment of the present invention provides a physical data model quality audit system based on metadata, which includes:

[0057] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the processor executes the computer program, any step of the above-mentioned metadata-based physical data model quality audit method is implemented.

[0058] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the above-mentioned metadata-based physical data model quality audit method is implemented.

[0059] The beneficial effects of the present invention are as follows: the present invention significantly improves the automation and accuracy of data model quality review through a metadata-based physical data model quality audit method; by creating quality review rules and automatically generating SQL query scripts, it can quickly locate and repair structural defects and data consistency issues in the data model; the dynamic generation of multiple thresholds and inspection task lists ensures the system's continuous tracking and optimization of model quality; the visually displayed tracking report further improves the readability of the audit results, enabling designers to efficiently identify and solve problems, ultimately greatly improving the overall quality and performance of the data model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0061] Figure 1 A specific flow chart of a metadata-based physical data model quality audit method and system provided for one embodiment of the present invention.

[0062] Figure 2 A schematic diagram of physical data detection of a metadata-based physical data model quality audit method and system provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0064] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0065] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0066] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0067] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0068] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0069] Example 1

[0070] Reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and provides a physical data model quality audit method based on metadata, comprising:

[0071] S1: Create quality review rules for the physical data model, perform conversion based on the division results of the review rules, and use the converted results to obtain metadata of the physical data model.

[0072] Among them, the quality review rules for creating the physical data model include creating the quality review rules for the physical data model according to the execution language requirements of the database. The quality review rules include structural integrity, data consistency and performance optimization. The system automatically collects and analyzes the demand inputs from different business departments and calculates the demand input matrix. The specific calculation formula of the demand input matrix is:

[0073]

[0074] Among them, R ij represents the weight of the jth requirement of the i-th category rule, ω k represents the weight of k business departments, N ijk Represents the score of the k-th business department on the j-th requirement of the i-th category rule.

[0075] Furthermore, quality review rules for the physical data model are created. The rules are designed based on the execution language requirements of the database. The quality review rules cover aspects such as structural integrity, data consistency, and performance optimization to ensure high quality standards of the data model in all dimensions. The system automatically collects and analyzes demand inputs from different business departments. Assume that the system collects requirements for data models from five business departments, where each department scores the requirements for each rule category. Specifically, department A scores 8 for structural integrity, department B scores 7, department C scores 9, and so on. The system calculates the demand input matrix and combines these scores to determine the weight and priority of each requirement. If structural integrity accounts for 60% of the overall score, data consistency accounts for 25%, and performance optimization accounts for 15%, the weight distribution is adjusted according to these proportions. The converted results provide a basis for obtaining metadata for the physical data model to ensure that the data model meets the specific requirements and quality standards of each business department. The quality score weight data is shown in Table 1 below:

[0076] Table 1 Quality score weight data table

[0077]

[0078] The table shows the scores of each business department on the quality review rules of the physical data model, including structural integrity, data consistency and performance optimization. Each department gave different scores based on these aspects, and the table also shows the weight distribution of these rules, with structural integrity accounting for 60%, data consistency accounting for 25%, and performance optimization accounting for 15%. The data is used to calculate the requirement input matrix to help determine the weight and priority of each requirement, thereby ensuring that the data model meets the high quality standards of each business department.

[0079] S1.1: The system automatically generates SQL query inspection scripts based on different rule categories and weights, and generates inspection tasks based on the inspection scripts. The inspection tasks include structural integrity inspection, data consistency inspection, and performance optimization inspection.

[0080] The system inputs the matrix results according to the requirements and dynamically generates the inspection task list. The specific calculation formula for the inspection task list is:

[0081] T s ={T ij |R ij >threshold}

[0082] Among them, T s represents the set of inspection tasks that meet the threshold conditions, T ij represents the inspection task, R ij represents the weight of the jth requirement of the i-th category rule, and threshold represents the preset weight threshold;

[0083] Acquiring the metadata of the physical data model includes, through the generated inspection task, the system automatically connects to the database, executes the inspection task, and acquires the metadata of the physical data model;

[0084] According to the generated inspection task list, the system executes each task in sequence and collects metadata of the data model. The specific calculation formula for collecting metadata is:

[0085] M={m i |m i ∈execute(T ij )}

[0086] Among them, M represents the metadata set obtained, m i Represents a metadata item, execute represents the function that executes the inspection task, T ij Indicates an inspection task.

[0087] Furthermore, the results of the review rule division are converted into specific SQL query check scripts. These scripts generate check tasks based on different rule categories and weights, including structural integrity check, data consistency check and performance optimization check. Assuming that the system has determined 10 structural integrity check tasks, 8 data consistency check tasks and 5 performance optimization check tasks, a check task list is dynamically generated based on the results of the demand input matrix. Once the check task list is generated, the system automatically connects to the database and executes each check task in turn. When performing the structural integrity check, the system may verify the integrity of the table structure and the constraints of the fields. By executing these check tasks, the metadata of the data model collected by the system will be organized into a data set for subsequent analysis and report generation.

[0088] S2: Check the obtained metadata, generate a review report on the quality of the physical data model, and send the generated review report to the designer of the corresponding problem model to complete the update of the quality of the physical data model.

[0089] Among them, checking the obtained metadata includes extracting metadata of the physical data model from the database, customizing field information of the metadata according to different metadata types, and checking structural attributes in the field information;

[0090] If the field information lacks a primary key, the field information at this time will be marked, the problem will be recorded, and the recorded problem will be optimized;

[0091] When the redundant indexes in the optimization process exceed the preset standard of the field information, the optimized field information does not meet the predefined data type specification. The preset standard includes objection to the record problem. The specific steps of the objection are as follows:

[0092] If the foreign key constraint between field information does not meet the data consistency check, the foreign key relationship problem is recorded and marked for repair;

[0093] If the primary key pointed to by the foreign key of the field information does not exist, the record mark repair process is recorded to clear the data.

[0094] S2.1: Generating a review report on the quality of the physical data model includes that after completing the quality check of the physical data model, the system establishes different levels of review reports based on the check results, and the severity of the review report is determined according to the number of checks received by the physical data model;

[0095] When the same designer is responsible for N issues, the review report is allocated based on the recorded issues and designer information. The specific calculation formula for allocating designer information is:

[0096] S={D i |M i∈P(D i )}

[0097] Among them, S represents the allocation result of the designer, D i represents the designer, P(D i ) represents the set of problems for which the designer is responsible.

[0098] Furthermore, when generating a physical data model quality review report, after completing the quality check, the system creates reports of different levels based on the inspection results. The severity of the report is determined by the number of inspections the physical data model undergoes. If a model undergoes 10 structural integrity checks, 8 data consistency checks, and 5 performance optimization checks, the system will generate a detailed review report based on these inspection results.

[0099] If the same designer is responsible for multiple issues, the system will assign the issues in the review report according to the designer's responsibilities. For example, Designer A is responsible for 3 issues and Designer B is responsible for 2 issues. The system will assign the review report to the corresponding designer based on the number of recorded issues and mark the person responsible for each issue. This assignment ensures that issues can be accurately tracked and handled.

[0100] S2.2: Completing the update of the quality of the physical data model includes automatically checking the updated data model submitted by the designer based on the reported problems received by the designer to verify whether the problems have been fixed;

[0101] If the problem is detected to have been fixed, the system updates the review report status to Resolved;

[0102] If the detected problem is not repaired, record the reason for not repairing it and notify the designer to verify the various steps of repair;

[0103] Construct multiple thresholds T i , check the quality review rules, the specific check calculation formula is:

[0104] T s ={T i |R i >T i}

[0105] Among them, T s represents the set of repair items that have passed verification, T i Represents the constructed multiple thresholds, R i represents the verification result of the i-th repair;

[0106] When R i >T i When , the set of repair items that have passed the verification is in a qualified state;

[0107] When Ri ≤T i When , the verified repair item set is in an unqualified state, at this time, the structural integrity, data consistency and performance optimization are checked;

[0108] Checking the structural integrity includes checking the table structure, table field types and constraints of the structure;

[0109] The specific calculation formula for checking the structural integrity is:

[0110]

[0111] Where S represents the structural integrity score, ω j represents the weight of the j-th check, C j Qualified variables representing structural integrity checks;

[0112] When C j ≤S, the structural integrity is passed and it is considered qualified; otherwise, when C j >S, record the detailed problem and notify the designer;

[0113] The data consistency check includes comparing the data consistency score with the qualified variables of the data consistency check. The specific calculation formula of the data consistency check is:

[0114]

[0115] Where D represents the data consistency score, d k Indicates the qualified variable of data consistency check, m indicates the total number of data, and k indicates the kth data;

[0116] When k ≤D, the data consistency is passed and it is recorded as qualified; otherwise, when d k >D, record the detailed problem and notify the designer;

[0117] The performance optimization check includes comparing the performance optimization score of the data with the performance optimization qualified variables of the data. The specific calculation formula for performance optimization is:

[0118]

[0119] Among them, P represents the performance optimization score, p l represents the weight of the lth test, R l represents eligible variables for performance optimization;

[0120] When R l ≤P, the performance optimization is passed and it is recorded as qualified; otherwise, when R l>P, record the detailed problem and notify the designer.

[0121] Preferably, in the step of completing the quality update of the physical data model, the system will automatically check the repaired data model submitted by the designer based on the report issues received by the designer. Assuming that the original review report points out that 3 unnecessary indexes are missing in the "Order" table, after the designer submits the repaired model, the system will verify whether these indexes have been added correctly. If all missing indexes have been supplemented, the system will update the status of the review report to "Resolved".

[0122] If the problem is not resolved, for example, the system still finds that an index is missing in the "Order" table, or some fields in the data table still do not have a primary key, the system will record the reason for the failure to repair and notify the designer to conduct further inspection and repair. At the same time, the system will perform quality checks based on the set multiple thresholds. If the pass standard for the structural integrity check is set to 90 points, and the repaired model scores 85 points, the model is marked as unqualified. In the data consistency check, if the total number of data items does not match or the data quality score is less than 80 points, the system will also record the problem. The standard for performance optimization inspection is to achieve at least 80% optimization effect. If the actual effect is lower than this standard, the system will mark it as unqualified. All problems found will be recorded in detail and the designer will be notified to make necessary adjustments and repairs.

[0123] In a preferred embodiment, a metadata-based physical data model quality audit system includes a creation module, which creates quality review rules for the physical data model and uses the converted results to obtain the metadata of the physical data model; an inspection module, which checks the obtained metadata and sends the generated review report to the designer of the corresponding problem model to complete the update of the physical data model quality; and a visualization display module, which establishes a tracking report on the quality of the physical data model and visualizes the tracking report on the quality of the physical data model.

[0124] The above-mentioned unit modules may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to the above-mentioned modules.

[0125] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0126] In summary, the present invention significantly improves the automation and accuracy of data model quality review through a metadata-based physical data model quality audit method. By creating quality review rules and automatically generating SQL query scripts, it can quickly locate and repair structural defects and data consistency issues in the data model. The dynamic generation of multiple thresholds and inspection task lists ensures the system's continuous tracking and optimization of model quality. The visually displayed tracking report further improves the readability of the audit results, enabling designers to efficiently identify and solve problems, ultimately greatly improving the overall quality and performance of the data model.

[0127] Example 2

[0128] Reference Figure 1 and Figure 2 , which is the second embodiment of the present invention, and this embodiment provides a physical data model quality audit method based on metadata. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through simulation experiments.

[0129] According to different rule categories and weights, an SQL query inspection script was generated, and an inspection task list was generated. Specifically, the inspection tasks for the data model included structural integrity checks, data consistency checks, and performance optimization checks. In the experiment, the preset weight threshold was set to 0.7, and the system dynamically generated an inspection task list, which included 50 structural integrity checks, 30 data consistency checks, and 20 performance optimization checks. After executing the inspection tasks, the system automatically connected to the database, obtained the metadata of the data model, and executed these inspection tasks in sequence. The results showed that among the 50 structural integrity checks, 40 passed and 10 failed; among the 30 data consistency checks, 25 passed and 5 failed; among the 20 performance optimization checks, 18 passed and 2 failed.

[0130] An audit report was generated based on the inspection results, and the problems were assigned to relevant designers for repair. Finally, the system updated the review report status, and the repair situation showed that 60% of the problems had been solved and 40% were still being processed. The system further recorded the reasons for the unrepaired problems and notified the designers to conduct detailed verification. Through multiple threshold checks, it was finally confirmed that the structural integrity score was 85 points (out of 100 points), the data consistency score was 80 points (out of 100 points), and the performance optimization score was 90 points (out of 100 points). These data provide a basis for subsequent quality improvements. The summary data of the experimental inspection results are shown in Table 2 below:

[0131] Table 2 Experimental inspection results summary data table

[0132]

[0133] The above table summarizes the results and quality scores after checking the data model. The table shows the number of tasks and pass status of the structural integrity check, data consistency check and performance optimization check, among which the pass rate of the structural integrity check is 80%, the pass rate of the data consistency check is 83%, and the pass rate of the performance optimization check is 90%. In addition, the completion status of the repair problems is also listed, showing that 60% of the problems have been solved and 40% of the problems are still being processed. The final score shows that the structural integrity score is 85 points, the data consistency score is 80 points, and the performance optimization score is 90 points, which provides data support for further quality improvement. The comparison between the present invention and the prior art is shown in Table 3 below:

[0134] Table 3 Comparison between the present invention and the prior art

[0135]

[0136] The above table summarizes the comparison between the present invention and the prior art, highlighting the significant advantages of the present invention in many aspects. Compared with the prior art, the present invention improves the inspection efficiency and accuracy by automatically generating SQL inspection scripts and task lists, improves the data quality and system performance through comprehensive data consistency inspection and optimization mechanisms, speeds up problem repair by automatically generating review reports and providing detailed repair guidance; and reduces manual intervention and labor costs through highly automated system operations, thereby effectively improving the overall quality and operational efficiency of the data model.

[0137] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A physical data model quality audit method based on metadata, characterized by: include, Create quality review rules for the physical data model, convert the results of the review rule division, and use the converted results to obtain the metadata of the physical data model; Check the obtained metadata, generate a review report on the quality of the physical data model, and send the generated review report to the designer of the corresponding problem model to complete the update of the quality of the physical data model; Record the review reports before and after the update, establish a tracking report on the quality of the physical data model, visualize the tracking report on the quality of the physical data model, build a chart distribution of the quality of the physical data model, and conduct an audit based on the distribution.

2. The physical data model quality audit method based on metadata according to claim 1, characterized in that: The quality review rules for creating the physical data model include creating the quality review rules for the physical data model according to the execution language requirements of the database. The quality review rules include structural integrity, data consistency and performance optimization. The system automatically collects and analyzes demand inputs from different business departments and calculates a demand input matrix. The specific calculation formula of the demand input matrix is: Among them, R ij represents the weight of the jth requirement of the i-th category rule, ω k represents the weight of k business departments, N ijk Represents the score of the k-th business department on the j-th requirement of the i-th category rule.

3. The physical data model quality audit method based on metadata according to claim 2, characterized in that: The conversion of the division results of the review rules includes the system automatically generating an SQL query inspection script according to different rule categories and weights, and the inspection tasks generated based on the inspection script, the inspection tasks including structural integrity inspection, data consistency inspection and performance optimization inspection; The system inputs the matrix results according to the requirements and dynamically generates an inspection task list. The specific calculation formula of the inspection task list is: T s ={T ij |R ij >threshold} Among them, T s represents the set of inspection tasks that meet the threshold conditions, T ij represents the inspection task, R ij represents the weight of the jth requirement of the i-th category rule, and threshold represents the preset weight threshold; The obtaining of the metadata of the physical data model includes the system automatically connecting to the database through the generated inspection task, executing the inspection task, and obtaining the metadata of the physical data model; The system executes each task in sequence according to the generated inspection task list and collects metadata of the data model. The specific calculation formula for collecting metadata is: M={m i |m i ∈execute(T ij )} Among them, M represents the metadata set obtained, m i Represents a metadata item, execute represents the function that executes the inspection task, T ij Indicates an inspection task.

4. The method for quality auditing of physical data models based on metadata according to claim 3, characterized in that: The checking of the obtained metadata includes extracting metadata of the physical data model from the database, customizing field information of the metadata according to different metadata types, and checking structural attributes in the field information; If the field information lacks a primary key, the field information at this time is marked, the problem is recorded, and the recorded problem is optimized; When the redundant indexes in the optimization process exceed the preset standard of the field information, the optimized field information does not meet the predefined data type specification. The preset standard includes objection to the problem of the record. The specific steps of the objection are as follows: If the foreign key constraint between the field information does not satisfy the data consistency check, the foreign key relationship problem is recorded and marked for repair; If the primary key in the direction pointed by the foreign key of the field information does not exist, the record mark repair process is recorded to clear the data.

5. The method for quality auditing of physical data models based on metadata according to claim 4, characterized in that: Generating the physical data model quality review report includes, after completing the quality check of the physical data model, the system establishes different levels of review reports according to the check results, and judging the severity of the review report according to the number of checks received by the physical data model; When the same designer is responsible for N issues, the review report is allocated according to the recorded issues and designer information. The specific calculation formula for allocating the designer information is: S={D i |M i ∈P(D i )} Among them, S represents the allocation result of the designer, D i represents the designer, P(D i ) represents the set of problems for which the designer is responsible.

6. The method for quality auditing of physical data models based on metadata according to claim 5, characterized in that: The updating of the quality of the physical data model includes the system automatically checking the updated data model submitted by the designer based on the reported problems received by the designer to verify whether the problems have been fixed; If the problem is detected to have been fixed, the system updates the review report status to Resolved; If the detected problem is not repaired, record the reason for not repairing it and notify the designer to verify the various steps of repair; Construct multiple thresholds T i , check the quality review rules, the specific check calculation formula is: T s ={T i |R i >T i } Among them, T s represents the set of repair items that have passed verification, T i Represents the constructed multiple thresholds, R i represents the verification result of the i-th repair; When R i >T i When , the set of repair items that have passed the verification is in a qualified state; When R i ≤T i When , the verified repair item set is in an unqualified state, at this time, the structural integrity, data consistency and performance optimization are checked; The checking of structural integrity includes checking the table structure, table field types and constraint conditions of the structure; The specific calculation formula for checking the structural integrity is: Where S represents the structural integrity score, ω j represents the weight of the j-th check, C j Qualified variables representing structural integrity checks; When C j ≤S, the structural integrity is passed and it is considered qualified; otherwise, when C j >S, record the detailed problem and notify the designer; The data consistency check includes comparing the data consistency score with the qualified variables of the data consistency check. The specific calculation formula of the data consistency check is: Where D represents the data consistency score, d k Indicates the qualified variable of data consistency check, m indicates the total number of data, and k indicates the kth data; When k ≤D, the data consistency is passed and it is recorded as qualified; otherwise, when d k >D, record the detailed problem and notify the designer; The performance optimization check includes comparing the performance optimization score of the data with the performance optimization qualified variables of the data. The specific calculation formula of the performance optimization is: Among them, P represents the performance optimization score, p l represents the weight of the lth test, R l represents eligible variables for performance optimization; When R l ≤P, the performance optimization is passed and it is recorded as qualified; otherwise, when R l >P, record the detailed problem and notify the designer.

7. The method for quality auditing of physical data models based on metadata according to claim 6, characterized in that: The establishment of the physical data model quality tracking report includes the system comprehensively evaluating the results of the inspection task and generating a repaired tracking report. The specific repair calculation formula is: Among them, R x It represents the follow-up report of the repair results, S represents the structural integrity score, D represents the data consistency score, and P represents the performance optimization score; Use heatmaps to display the quality tracking report of the established physical data model, and judge the repaired tracking report based on the depth of the heatmap color; The darker the color of the heat map, the heavier the traces of the repaired tracking report. At this time, it is necessary to conduct a follow-up observation and audit on the repaired tracking report.

8. A metadata-based physical data model quality audit system, based on the metadata-based physical data model quality audit method according to any one of claims 1 to 7, characterized in that: include, A creation module is used to create quality review rules for the physical data model and obtain metadata of the physical data model using the converted results; The inspection module inspects the acquired metadata and sends the generated inspection report to the designer of the corresponding problem model to complete the update of the quality of the physical data model; The visualization display module establishes a tracking report on the quality of the physical data model and displays the tracking report on the quality of the physical data model visually.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the metadata-based physical data model quality audit method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the metadata-based physical data model quality audit method described in any one of claims 1 to 7 are implemented.

Images