Metadata change auditing method and system, computer equipment and storage medium
The metadata change audit method automates the detection of expected vs. unexpected changes using a large language model and knowledge graph, reducing communication costs and enhancing audit efficiency.
Patent Information
- Application Number
- CN202411005615.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-07-25
AI Technical Summary
Existing metadata governance tools struggle with determining whether changes in business systems are expected or unexpected, leading to high communication costs and low audit efficiency due to the need for comprehensive judgments based on update instructions.
A metadata change audit method utilizing a large language model and knowledge graph to process metadata information, generating pre-change and actual change records for comparison, and generating alarms for unexpected changes.
Reduces communication costs and improves audit efficiency by automating the determination of expected vs. unexpected changes in metadata, eliminating the need for in-depth understanding of business system details.
Smart Images

Figure CN119066077A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information and data governance, and in particular to a metadata change audit method, system, computer equipment and storage medium. Background Art
[0002] At present, the digital economy has become a new driving force for national economic growth and an important breakthrough for economic transformation and promoting the transformation of human economic forms. The quantitative change of super data will trigger the qualitative change of super intelligence, leading to the rapid development of productivity. Data capabilities will act on the digital economy and achieve rapid development of the digital economy. As the core driving force for the innovative development of the digital economy, data elements not only play a role in the development of digital industrialization, but also include the development of industrial digitalization, that is, the radiation and driving effect of data elements on various sectors of the national economy.
[0003] During the digital transformation process, the IT department of an enterprise needs to manage a large number of information systems and integrate the data in the information systems into the data platform for unified processing to maximize the value of the data. However, the information system is not static and needs to be upgraded and updated along with the business to expand new business functions. For the IT department, how to control the changes in the information system, especially the changes in data, becomes an important issue. To solve the above problems, the metadata integration module in the existing data governance tools is usually used. However, if the database structure of the business system changes due to business changes, the data governance team cannot confirm whether the change in the metadata is an expected change or an unexpected change. It is necessary to make a comprehensive judgment based on the update instructions of the business system technical team and other information, resulting in high communication costs and low audit efficiency.
[0004] Therefore, there is an urgent need for a metadata change auditing method with high management efficiency and low communication cost. Summary of the invention
[0005] Therefore, the technical problem to be solved by the present invention is to overcome the problem in the related technology that when the metadata in the business system changes, in order to determine whether the change is an expected change or an unexpected change, the management personnel need to make a comprehensive judgment based on information such as the update instructions, resulting in high communication costs and low audit efficiency.
[0006] In order to solve the above technical problems, the present invention provides a metadata change audit method, which includes:
[0007] Collect target metadata information of business systems;
[0008] Performing standardization processing on the target metadata information to obtain standard metadata information;
[0009] Constructing a knowledge graph according to the target metadata information, the standard metadata information, and the corresponding relationship between the target metadata information and the standard metadata information;
[0010] Obtain change-related files, and input the change-related files and the knowledge graph into a large language model to determine pre-change information; the large language model is configured to determine keyword information according to the change-related files, and determine pre-change information according to the keyword information and the knowledge graph;
[0011] Generate a pre-change record according to the pre-change information;
[0012] Acquire actual change information, and generate an actual change record according to the actual change information;
[0013] The pre-change record and the actual change record are audited and compared.
[0014] In an optional implementation, the step of inputting the change-related file and the knowledge graph into a large language model to determine the pre-change information includes:
[0015] Inputting the change-related documents into a large language model to generate keyword information;
[0016] Inputting the keyword information and the knowledge graph into the large language model to determine standard change information; the large language model is configured to retrieve and determine the standard change information in the knowledge graph according to the keyword information;
[0017] The standard change information and the knowledge graph are input into the large language model to determine the pre-change information.
[0018] In an optional implementation, the obtaining actual change information includes:
[0019] When a change in target metadata information is detected, actual change information is generated and sent to the message middleware;
[0020] The actual change information is obtained from the message middleware.
[0021] In an optional implementation, generating a pre-change record according to the pre-change information includes:
[0022] Generate a pre-change record according to the pre-change information, the expected change method and the expected change time corresponding to the pre-change information;
[0023] The generating the actual change record according to the actual change information includes:
[0024] An actual change record is generated according to the actual change information, the actual change method corresponding to the actual change information, and the actual change time.
[0025] In an optional implementation, the auditing and comparing the pre-change record and the actual change record includes:
[0026] Determine a matching result between the pre-change record and the actual change record, and determine a matching status of the pre-change record according to the matching result; the matching result includes a pass or a fail, and the matching result includes an unmatched status or a matched status;
[0027] When the matching result between the pre-change record and the actual change record is failed, the matching status of the pre-change record is determined to be a non-matching status, and an unexpected change alarm message is generated; the unexpected change alarm message is used to prompt the management personnel that the matching result between the pre-change record and the actual change record is failed.
[0028] In an optional embodiment, the method further includes:
[0029] storing the matching status of the pre-change record and the pre-change record in a pre-change database;
[0030] Determine the unmatched change record in the pre-change database according to a preset period; the unmatched change record is a pre-change record in the pre-change database whose matching state is an unmatched state;
[0031] Determine metadata information corresponding to the unmatched change record according to the unmatched change record in the pre-change database;
[0032] Audit and compare the unmatched change records in the pre-change database with metadata information corresponding to the unmatched change records;
[0033] When the matching result between the unmatched change record in the pre-change database and the metadata information corresponding to the unmatched change record is failed, a pre-change extension alarm message is generated according to the unmatched change record in the pre-change database; the pre-change extension alarm message is used to remind the management personnel that the metadata information corresponding to the unmatched change record in the pre-change database has not changed.
[0034] In an optional implementation, the target metadata information of the business system is stored in a metadata database, and the target metadata information of the business system is collected, including:
[0035] Configuring connection information of the metadata database;
[0036] Testing whether a connection channel is connected; the connection channel is determined according to the connection information of the metadata database;
[0037] When the connection channel is connected, a metadata integration task is executed; the metadata integration task is used to collect target metadata information of the business system in the metadata database according to the connection information of the metadata database.
[0038] In a second aspect, the present invention provides a metadata change auditing system, the metadata change auditing system comprising:
[0039] Metadata integration module, used to collect target metadata information of business systems;
[0040] The metadata governance module is connected with the metadata integration module;
[0041] The metadata governance module is used to standardize the target metadata information to obtain standard metadata information; construct a knowledge graph according to the target metadata information, the standard metadata information, and the corresponding relationship between the target metadata information and the standard metadata information;
[0042] The pre-change service module is connected to the metadata governance module;
[0043] The pre-change service module is used to obtain change-related files, input the change-related files and the knowledge graph into the large language model to determine pre-change information, and generate a pre-change record according to the pre-change information; the large language model is configured to determine keyword information according to the change-related files, and determine the pre-change information according to the keyword information and the knowledge graph;
[0044] The metadata change audit module is connected to the pre-change service module;
[0045] The metadata change audit module is used to obtain actual change information, generate an actual change record based on the actual change information, and audit and compare the pre-change record with the actual change record.
[0046] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the metadata change audit method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.
[0047] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a single computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the metadata change audit method of the above-mentioned first aspect or any corresponding embodiment thereof.
[0048] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the metadata change audit method of the first aspect or any corresponding embodiment thereof.
[0049] The technical solution provided by the present invention has the following technical effects:
[0050] The present invention can determine the pre-change record from the change-related files through the text parsing and processing capabilities of the large language model and the knowledge graph constructed based on the metadata information. By auditing and comparing the pre-change record and the actual change record, it is determined whether the change corresponding to the actual change record is an expected change or an unexpected change. By performing audit comparisons through the above technical solution, managers do not need to have an in-depth understanding of the change technology and change details of the business system, which can assist managers in managing metadata information, reduce communication costs, reduce workload, and improve audit efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0052] Figure 1 is a flowchart of a metadata change audit method according to an embodiment of the present invention;
[0053] Figure 2 It is a schematic diagram of the overall process of the metadata change audit method according to an embodiment of the present invention;
[0054] Figure 3 is a schematic diagram of the structure of a metadata change audit system according to an embodiment of the present invention;
[0055] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0057] Therefore, an embodiment of the present invention provides a metadata change auditing method, system, computer device and storage medium to solve the problem in the related technology that when metadata in a business system changes, in order to determine whether the change is an expected change or an unexpected change, management personnel need to make a comprehensive judgment based on information such as update instructions, resulting in high communication costs and low auditing efficiency.
[0058] According to an embodiment of the present invention, an embodiment of a metadata change audit method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0059] Figure 1 It is a flowchart of a metadata change audit method according to an embodiment of the present invention.
[0060] like Figure 1 As shown, a metadata change audit method provided by the present invention in an embodiment of the present invention can be applied to a metadata change audit system. The metadata change audit system can be a metadata change audit system. The metadata change audit system can include: a metadata management module, a pre-change service module and a metadata change audit module. The metadata management module includes: a metadata integration module and a metadata governance module.
[0061] The metadata change audit method includes:
[0062] S101: Collect target metadata information of the business system.
[0063] In this embodiment, the target metadata information of the business system is stored in the metadata database, and S101 collecting the target metadata information of the business system specifically includes:
[0064] S1011: Configure connection information of the metadata database.
[0065] In this embodiment, the connection information of the metadata database may be used as data source connection information, and may include an access address, port, user name, password, etc. of the metadata database.
[0066] S1012: Test whether the connection channel is connected.
[0067] In this embodiment, the connection channel is determined according to the connection information of the metadata database. The connection channel is a communication channel between the metadata integration module and the metadata database.
[0068] S1013: When the connection channel is connected, a metadata integration task is executed. The metadata integration task is used to collect target metadata information of the business system in the metadata database according to the connection information of the metadata database.
[0069] In this embodiment, the target metadata information of the business system may include connection information of the metadata database, a table list, a field list, a system name, etc. The table list may include: table name, table Chinese name, and table annotation information. The field list may include: field name, table name, field Chinese name, field type, and field annotation information.
[0070] The metadata change audit method provided in the embodiment of the present invention can be applied to various types of business systems. As an example, if the business system is a human resource management system, the above S101 collects the target metadata information of the business system, specifically: the user configures the connection information of the metadata database of the human resource management system through the metadata integration module, fills in the access address, port, user name, password, etc. of the metadata database, and the metadata change audit system tests whether the connection channel to the metadata database is connected, and saves the connection information of the metadata database to the database of the metadata change audit system. The metadata integration module regularly schedules the metadata integration task, accesses the corresponding metadata database according to the connection information of the metadata database of the metadata change audit system, reads the target metadata information, and saves it to the database of the metadata change audit system.
[0071] S102: Standardize the target metadata information to obtain standard metadata information, and construct a knowledge graph according to the target metadata information, the standard metadata information, and the corresponding relationship between the target metadata information and the standard metadata information.
[0072] In this embodiment, the metadata governance module can execute the above-mentioned step S102. Standardizing the target metadata information to obtain standard metadata information specifically includes: performing word segmentation and synonym processing on the target metadata information to obtain words to be standardized, obtaining a pre-constructed data standard terminology table, and searching the data standard terminology table for standard terms corresponding to the words to be standardized, thereby obtaining standard metadata information. As an example, the table name, field name, field annotation information, etc. are segmented and standardized to obtain standard metadata information and the correspondence between the target metadata information and the standard metadata information. The knowledge graph constructed according to the target metadata information, the standard metadata information, and the correspondence between the target metadata information and the standard metadata information can be stored in the knowledge graph engine for semantic association, so as to facilitate the large language model to parse specific tables or fields from semantics.
[0073] As an example, data governance personnel use the data governance module of the metadata change audit system to standardize the target metadata information of the collected business system. After the data governance module performs word segmentation and synonym comparison on the table, field and corresponding annotation information, it matches the closest standard term in the data standard term table and forms an association relationship between the target metadata information and the corresponding standard term. For example: a target metadata information is collected from the human resources internal management system, and the "employee basic information table" contains a field named "employee name", and there is a standard term called "employee name" in the standard term table. At this time, "employee name" can be segmented into two words, "employee" and "name", "employee" is consistent with the standard term, and "name" and "name" are synonyms, so "employee name" is "employee name" after standardization. The above target metadata information includes "employee name", the words to be standardized include "employee", "name" and "name", the standard term includes "employee name", and the standard metadata information includes "employee name". The metadata change audit system saves the original field name and the standardized field name, and establishes an association between "employee name" and "employee name". After completing the standardization process, the data governance module can save the target metadata information, the standard metadata information after standardization, and the correspondence between the target metadata information and the standard metadata information to the knowledge graph engine. For example: the knowledge graph engine contains 2 entities, the field information entity: "employee name", the standard term entity: "employee name", and the connection between the two entities "standardization".
[0074] S103: Obtain change-related documents, input the change-related documents and the knowledge graph into the large language model to determine pre-change information, and generate a pre-change record based on the pre-change information.
[0075] In the art, a large language model is configured to determine keyword information based on change-related files, and determine pre-change information based on the keyword information and a knowledge graph.
[0076] In this field, the pre-change service module can perform the above step S103. Before the business system is changed, the technical staff needs to provide change-related documents. The change-related documents can be system upgrade plans or update instructions or DMLSQL document information.
[0077] In this embodiment, the large language model includes but is not limited to llama3, GPT-3, BERT, Transformer-X and other models. In this embodiment, based on the actual application scenario of the present invention, the large language model can be trained in a targeted manner to improve the accuracy of determining the pre-change information.
[0078] In the scenario targeted by the present invention, in addition to tasks related to data governance, managers also need to manage changes in business systems. Specifically, changes in the database structure due to changes in the business system, business upgrades, etc. The changes in the database will lead to changes in metadata. The technical means usually adopted by managers for the above changes are: manually check the change-related documents provided by the technical personnel, and then compare them with the changes in the metadata to confirm whether the structural changes of the database are in line with expectations.
[0079] In this process, the technical staff of the business system provides the management staff with a system upgrade plan or update instructions or DML SQL document information before the system upgrade. The management staff uploads the change-related files in the function provided by the metadata change audit system. After the pre-change service module receives and saves the change-related files, it uploads the files to the large language model and parses the pre-change information by asking questions. After the large language model performs semantic analysis on the change-related files, it analyzes the metadata information that needs to be changed, compares it with the metadata information in the knowledge graph engine, parses the specific table and field information that needs to be changed, generates pre-change records, and stores them in the pre-change database.
[0080] S104: Acquire actual change information, generate actual change records according to the actual change information, and audit and compare the preliminary change records with the actual change records.
[0081] In this embodiment, the metadata change audit module can execute the above step S104. The above-mentioned acquisition of actual change information specifically includes: when the target metadata information is detected to have changed, the actual change information is generated, and the actual change information is sent to the message middleware. The actual change information is obtained from the message middleware. In this embodiment, the metadata governance module can generate a metadata change event when it detects that there is a difference before and after the target metadata information, determine that the target metadata information has changed, generate actual change information based on the difference before and after the target metadata information, and send the actual change information to the message middleware. The metadata change audit module can obtain the actual change information from the message middleware.
[0082] In this embodiment, the above step S104 is performed for each pre-change record and the actual change information corresponding to the pre-change record.
[0083] In an optional implementation, S103 inputs the change-related files and the knowledge graph into the large language model to determine the pre-change information, including:
[0084] S1031: Input the change-related files into the large language model to generate keyword information.
[0085] S1032: Input keyword information and knowledge graph into the large language model to determine standard change information.
[0086] In this embodiment, the large language model is configured to retrieve and determine standard change information in the knowledge graph based on keyword information.
[0087] In this embodiment, the standard change information is the first parsing result of the large language model. You can first search and determine the parent entity in the knowledge graph based on the keyword information. The parent entity can be a database table name, a database annotation, a standardized table name, and the same is true for the field. Then, through the corresponding relationship, you can infer the subordinate entity, such as the table and field. In this embodiment, the table and field to be changed cannot be directly determined by the knowledge graph and keyword information, and only the first parsing result can be obtained. In order to obtain more accurate tables and fields to be changed, it is necessary to input the standard change information and the knowledge graph into the large language model for further parsing.
[0088] S1033: Input the standard change information and the knowledge graph into the large language model to determine the pre-change information.
[0089] In this embodiment, the pre-change information is the second parsing result of the large language model, and is the metadata information that should actually be changed in the business system. The pre-change information can be used as a metadata change application form for the business system. In this embodiment, a part of the change information in the standard change information can be used as the pre-change information through the parsing of the large language model. Through multiple parsing of the large language model, the second parsing based on the first parsing can improve the accuracy of determining the pre-change information.
[0090] As an example, if the large language model is llama3, S1031 inputs the change-related files into the large language model to generate keyword information, specifically including: based on the semantic analysis and code analysis capabilities of the llama3 large language model, using a specific prompt, asking questions to the llama3 large language model, and the llama3 large language model parses the content of the metadata database changes from the change-related files (change-related files include system upgrade plans, update instructions, and DML SQL and other unstructured documents) as keyword information, such as the system, table, field, and change time involved. For example: in the system upgrade plan and update instructions, the change description is generally described in text form. For example: in the "employee basic information table", a new "employee length of service" field is added, and three keyword information can be inferred: (1) Impact system: human resource management system, (2) Impact table: employee basic information table, (3) Impact content: new "employee length of service" field is added. If the uploaded information is a DMLSQL file, the above three keyword information can also be inferred through SQL statement parsing, and the table name and field name are relatively accurate. At the same time, the expected change method and expected change time can be analyzed from the upgrade plan and update instructions. The expected change method may include addition, replacement, deletion, etc.
[0091] After obtaining the keyword information, the standard change information can be determined by combining the llama3 large language model with the Graph RAG technology. The llama3 large language model retrieves the metadata information of the relevant tables and fields in the knowledge graph as the standard metadata information based on the keyword information. The standard metadata information and the knowledge graph are input into the large language model as context. Based on the GraphRAG technology, the large language model matches the knowledge graph, and finally parses the metadata information of the data table that actually needs to be changed by matching key data such as the system and table, and uses the metadata information of the data table that actually needs to be changed as the pre-change information.
[0092] After the standard change information and knowledge graph are input into the large language model to determine the pre-change information, the background service generates a pre-change record based on the pre-change information, the expected change method and the expected change time corresponding to the pre-change information. This pre-change record contains metadata identifier, table name change information, field change information, matching status, expected change time, etc., and saves the pre-change record in the system's pre-change database.
[0093] In an optional implementation, generating a pre-change record according to the pre-change information in S103 includes: generating a pre-change record according to the pre-change information, an estimated change method and an estimated change time corresponding to the pre-change information.
[0094] In this embodiment, the pre-change service module may generate a pre-change record according to the pre-change information, the expected change method and the expected change time corresponding to the pre-change information.
[0095] In an optional implementation, generating an actual change record according to the actual change information in S104 includes: generating an actual change record according to the actual change information, an actual change method corresponding to the actual change information, and an actual change time.
[0096] In this embodiment, the metadata change audit module may generate an actual change record according to the actual change information, the actual change method corresponding to the actual change information, and the actual change time.
[0097] In an optional implementation, S104 auditing and comparing the pre-change record with the actual change record includes:
[0098] Sa1: Determine the matching result between the pre-change record and the actual change record, and determine the matching status of the pre-change record according to the matching result.
[0099] In this embodiment, the matching result includes a pass or a fail, and the matching result includes an unmatched state or a matched state.
[0100] Sa2: When the matching result between the pre-change record and the actual change record is not passed, the matching state of the pre-change record is determined as a mismatched state, and unexpected change alarm information is generated.
[0101] In this embodiment, the unexpected change alarm information is used to prompt the management personnel that the matching result between the pre-change record and the actual change record is not passed. As an example, the unexpected change alarm information can be stored in a database, and when the background service queries the unexpected change alarm information, the unexpected change alarm information is prompted to the management personnel.
[0102] In this embodiment, when the matching result between the pre-change record and the actual change record is passed, the matching status of the pre-change record is determined to be a matched status, and the next pre-change record and its corresponding actual change record are audited and compared.
[0103] In this embodiment, the actual change record can be compared one by one with the pre-change record stored in the pre-change database through the metadata change audit module. The audit content is whether the change content such as the system name, table name, field name, etc. that actually triggers the change is consistent with the system name, table name, field name, etc. in the registered change information (pre-change record stored in the pre-change database). If so, the matching result of the pre-change record and the actual change record is determined to be passed, and the matching status of the pre-change record stored in the pre-change database is updated to a matched status, and the next change information is audited. If not, the matching result of the pre-change record and the actual change record is determined to be failed, and the matching status of the pre-change record stored in the pre-change database is updated to an unmatched status, and unexpected change alarm information is generated and stored in the database. After the background service queries the unexpected change alarm information, it prompts the management personnel.
[0104] In this embodiment, the above audit process may involve some details. For example, when a table is added, since the addition of tables and fields are metadata changes, in general, in the materials such as the upgrade plan provided, only the newly added table will be explained without explaining the specific field list. Therefore, in the specific audit process, there will be a situation where changes are filtered from each other. The above problem can be solved by conventional technical means in this field, which will not be specifically described in the present invention. The above embodiment mainly describes the way of change audit.
[0105] In the present invention, in addition to the above-mentioned change auditing method, the metadata change auditing method also includes another auditing method:
[0106] Sb1: Store the matching status of the pre-change record and the pre-change record in the pre-change database.
[0107] Sb2: Determine unmatched change records in the pre-change database according to a preset period.
[0108] In this embodiment, the unmatched change record is a pre-change record whose matching state in the pre-change database is an unmatched state.
[0109] Sb3: Determine the latest metadata information corresponding to the unmatched change record according to the unmatched change record in the pre-change database.
[0110] In this embodiment, the metadata information corresponding to the unmatched change record is the metadata information corresponding to the unmatched change record.
[0111] Sb4: Audit and compare the unmatched change records in the pre-change database with the metadata information corresponding to the unmatched change records.
[0112] Sb5: When the matching result between the unmatched change record in the pre-change database and the metadata information corresponding to the unmatched change record is failed, pre-change delay warning information is generated according to the unmatched change record in the pre-change database.
[0113] In this embodiment, the pre-change delay warning information is used to remind the management personnel that the metadata information corresponding to the unmatched change record in the pre-change database has not been changed, and to remind the management personnel to perform further processing.
[0114] The metadata change audit module can scan the pre-change information in the pre-change database according to a preset period, and determine the unmatched change records in the pre-change database. The unmatched change records are specifically pre-change records in the pre-change database whose matching status is in an unmatched state within a preset time period, which is equivalent to the pre-change information being in an unmatched state for a long time. At this time, it is necessary to determine the latest metadata information corresponding to the unmatched change record based on the unmatched change record in the pre-change database, and audit and compare the unmatched change record and the latest metadata information corresponding to the unmatched change record. When the matching result of the unmatched change record in the pre-change database and the metadata information corresponding to the unmatched change record is passed, the matching status of the unmatched change record is updated to a matched state.
[0115] The overall process of the metadata change audit method proposed in the embodiment of the present invention is as follows: Figure 2 As shown, metadata change auditing based on the technical solution of the present invention can identify unexpected changes through auditing, without the need for managers to have an in-depth understanding of the details of the changes in the business system, reducing the workload of managers in metadata changes. Managers do not need to have an in-depth understanding of the business system like technical personnel, reducing the communication between managers and technical personnel. During the communication process between managers and technical personnel, managers can improve the efficiency of communication with technical personnel based on information such as pre-change information, actual change information, and matching results. The technical solution of the present invention can compare and audit the system upgrade plan, update instructions, and actual changes in the database through the semantic understanding ability of the large language model and the knowledge graph, and assist managers in managing database changes.
[0116] It should be noted that the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.
[0117] In this embodiment, a metadata change audit system is also provided. A single system is used to implement the above-mentioned embodiments and optional implementation methods. The descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0118] Figure 3 It is a structural diagram of a metadata change audit system according to an embodiment of the present invention.
[0119] The present invention provides a metadata change audit system, such as Figure 3 As shown, the metadata change audit system includes:
[0120] The metadata integration module 11 is used to collect target metadata information of the business system.
[0121] The metadata governance module 12 is connected to the metadata integration module 11 .
[0122] The metadata governance module 12 is used to standardize the target metadata information to obtain standard metadata information, and to construct a knowledge graph based on the target metadata information, the standard metadata information, and the correspondence between the target metadata information and the standard metadata information.
[0123] The pre-change service module 13 is connected to the metadata governance module 12 .
[0124] The pre-change service module 13 is used to obtain change-related files, input the change-related files and the knowledge graph into the large language model to determine the pre-change information, and generate a pre-change record according to the pre-change information. The large language model is configured to determine keyword information according to the change-related files, and determine the pre-change information according to the keyword information and the knowledge graph.
[0125] The metadata change audit module 14 is connected to the pre-change service module 13 .
[0126] The metadata change audit module 14 is used to obtain actual change information, generate actual change records based on the actual change information, and audit and compare the pre-change records with the actual change records.
[0127] In an optional implementation, the target metadata information of the business system is stored in a metadata database, and the metadata integration module 11 includes:
[0128] The first processing unit is used to configure connection information of the metadata database.
[0129] The second processing unit is used to test whether the connection channel is connected. The connection channel is determined according to the connection information of the metadata database.
[0130] The third processing unit is used to execute a metadata integration task when the connection channel is connected. The metadata integration task is used to collect target metadata information of the business system in the metadata database according to the connection information of the metadata database.
[0131] In an optional implementation, the pre-change service module 13 includes:
[0132] The first processing unit is used to input the change-related files into the large language model to generate keyword information.
[0133] The second processing unit is used to input the keyword information and the knowledge graph into the large language model to determine the standard change information. The large language model is configured to retrieve and determine the standard change information in the knowledge graph according to the keyword information.
[0134] The third processing unit is used to input the standard change information and the knowledge graph into the large language model to determine the pre-change information.
[0135] In an optional implementation, the pre-change service module 13 further includes a fourth processing unit.
[0136] The fourth processing unit is used to generate a pre-change record according to the pre-change information, the expected change method corresponding to the pre-change information, and the expected change time.
[0137] In an optional implementation, the metadata governance module 12 is further configured to generate actual change information when a change in the target metadata information is detected, and send the actual change information to the message middleware.
[0138] The metadata change audit module 14 includes a first processing unit.
[0139] The first processing unit is used to obtain actual change information from the message middleware.
[0140] In an optional implementation, the metadata change audit module 14 further includes a second processing unit.
[0141] The second processing unit is used to generate an actual change record according to the actual change information, the actual change method corresponding to the actual change information, and the actual change time.
[0142] In an optional implementation, the metadata change audit module 14 further includes a third processing unit.
[0143] The third processing unit includes:
[0144] The first processing subunit is used to determine the matching result between the pre-change record and the actual change record, and determine the matching status of the pre-change record according to the matching result. The matching result includes pass or fail, and the matching result includes unmatched status or matched status.
[0145] The second processing subunit is used to determine the matching status of the pre-change record as a mismatched status and generate unexpected change warning information when the matching result between the pre-change record and the actual change record is not passed. The unexpected change warning information is used to remind the management personnel that the matching result between the pre-change record and the actual change record is not passed.
[0146] In an optional implementation, the metadata change audit module 14 further includes a fourth processing unit.
[0147] The fourth processing unit includes:
[0148] The first processing subunit is used to store the pre-change record and the matching status of the pre-change record in the pre-change database.
[0149] The second processing sub-unit is used to determine the unmatched change records in the pre-change database according to a preset period. The unmatched change records are pre-change records in the pre-change database whose matching status is unmatched.
[0150] The third processing sub-unit is used to determine metadata information corresponding to the unmatched change record according to the unmatched change record in the pre-change database.
[0151] The fourth processing sub-unit is used to audit and compare the unmatched change records in the pre-change database with the metadata information corresponding to the unmatched change records.
[0152] The fifth processing subunit is used to generate a pre-change extension warning message according to the unmatched change record in the pre-change database when the matching result between the unmatched change record in the pre-change database and the metadata information corresponding to the unmatched change record is not passed. The pre-change extension warning message is used to remind the management personnel that the metadata information corresponding to the unmatched change record in the pre-change database has not been changed.
[0153] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.
[0154] The metadata change audit system in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.
[0155] The embodiment of the present invention also provides a computer device having the above Figure 3 The metadata change auditing system shown.
[0156] See also Figure 4 , Figure 4Schematic diagram of the hardware structure of the computer device according to the embodiment of the present invention. Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In an optional embodiment, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor device). Figure 4 A processor 10 is taken as an example.
[0157] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.
[0158] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.
[0159] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function. The data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In an optional embodiment, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0160] The memory 20 may include a volatile memory, such as a random access memory. The memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive. The memory 20 may also include a combination of the above-mentioned types of memory.
[0161] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.
[0162] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc. Further, the storage medium can also include a combination of the above-mentioned types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.
[0163] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.
[0164] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.
Claims
1. A metadata change audit method, characterized in that: include: Collect target metadata information of business systems; Performing standardization processing on the target metadata information to obtain standard metadata information; Constructing a knowledge graph according to the target metadata information, the standard metadata information, and the corresponding relationship between the target metadata information and the standard metadata information; Obtain change-related documents, and input the change-related documents and the knowledge graph into a large language model to determine pre-change information; The large language model is configured to determine keyword information according to the change-related file, and determine pre-change information according to the keyword information and the knowledge graph; Generate a pre-change record according to the pre-change information; Acquire actual change information, and generate an actual change record according to the actual change information; The pre-change record and the actual change record are audited and compared.
2. The method according to claim 1, characterized in that The step of inputting the change-related file and the knowledge graph into a large language model to determine the pre-change information includes: Inputting the change-related documents into a large language model to generate keyword information; Inputting the keyword information and the knowledge graph into the large language model to determine standard change information; the large language model is configured to retrieve and determine the standard change information in the knowledge graph according to the keyword information; The standard change information and the knowledge graph are input into the large language model to determine the pre-change information.
3. The method according to claim 1, characterized in that The obtaining of actual change information includes: When a change in target metadata information is detected, actual change information is generated and sent to the message middleware; The actual change information is obtained from the message middleware.
4. The method according to claim 1, characterized in that: The generating a pre-change record according to the pre-change information includes: Generate a pre-change record according to the pre-change information, the expected change method and the expected change time corresponding to the pre-change information; The generating the actual change record according to the actual change information includes: An actual change record is generated according to the actual change information, the actual change method corresponding to the actual change information, and the actual change time.
5. The method according to claim 1, characterized in that The auditing and comparing the pre-change record and the actual change record includes: Determine a matching result between the pre-change record and the actual change record, and determine a matching status of the pre-change record according to the matching result; the matching result includes a pass or a fail, and the matching result includes an unmatched status or a matched status; When the matching result between the pre-change record and the actual change record is failed, the matching status of the pre-change record is determined to be a non-matching status, and an unexpected change alarm message is generated; the unexpected change alarm message is used to prompt the management personnel that the matching result between the pre-change record and the actual change record is failed.
6. The method according to claim 5, characterized in that The method further comprises: storing the matching status of the pre-change record and the pre-change record in a pre-change database; Determine the unmatched change record in the pre-change database according to a preset period; the unmatched change record is a pre-change record in the pre-change database whose matching state is an unmatched state; Determine metadata information corresponding to the unmatched change record according to the unmatched change record in the pre-change database; Audit and compare the unmatched change records in the pre-change database with metadata information corresponding to the unmatched change records; When the matching result between the unmatched change record in the pre-change database and the metadata information corresponding to the unmatched change record is failed, a pre-change extension alarm message is generated according to the unmatched change record in the pre-change database; the pre-change extension alarm message is used to remind the management personnel that the metadata information corresponding to the unmatched change record in the pre-change database has not changed.
7. The method according to claim 1, characterized in that The target metadata information of the business system is stored in a metadata database, and the target metadata information of the business system is collected, including: Configuring connection information of the metadata database; Testing whether a connection channel is connected; the connection channel is determined according to the connection information of the metadata database; When the connection channel is connected, a metadata integration task is executed; the metadata integration task is used to collect target metadata information of the business system in the metadata database according to the connection information of the metadata database.
8. A metadata change audit system, characterized in that: include: Metadata integration module, used to collect target metadata information of business systems; The metadata governance module is connected with the metadata integration module; The metadata governance module is used to perform standardization processing on the target metadata information to obtain standard metadata information; Constructing a knowledge graph according to the target metadata information, the standard metadata information, and the corresponding relationship between the target metadata information and the standard metadata information; The pre-change service module is connected to the metadata governance module; The pre-change service module is used to obtain change-related files, input the change-related files and the knowledge graph into the large language model to determine pre-change information, and generate a pre-change record according to the pre-change information; The large language model is configured to determine keyword information according to the change-related file, and determine pre-change information according to the keyword information and the knowledge graph; The metadata change audit module is connected to the pre-change service module; The metadata change audit module is used to obtain actual change information and generate actual change records according to the actual change information; The pre-change record and the actual change record are audited and compared.
9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the metadata change audit method described in any one of claims 1 to 7 by executing the computer instructions.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the metadata change audit method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Data exception update detection method and device, computer equipment and storage medium
CN112231181A
Bill auditing method and device
CN116912016A
Data change prediction method and system, electronic equipment and storage medium
CN117290589A
Method for efficiently exporting and importing files through RBD snapshots
CN117851348A
Method and system for updating web page index
KR1020100008885A
