Dynamic collection method, system and equipment for multi-level data directory

Abstract

The invention relates to the technical field of data processing, and provides a multi-level data directory dynamic collection method, system and equipment, and the method comprises the steps: receiving a directory operation application submitted by a user at a region node according to a preset data directory description strategy, and carrying out the examination and approval of the directory operation application; according to the approval result, updating the state of the corresponding directory entry at the region node, and storing the updated directory entry in a region directory library; dynamically reporting the approved directory entry with the updated state to a global node; and at the global node, collecting and combining directory entries from at least one region node, updating a global directory library, and providing a unified query service. According to the method, the accuracy and authority of directory information can be improved from a data source, the process and the clear responsibility of data directory management are realized, the flexibility and the timeliness of directory management are considered while the normalization is ensured, and the technical complexity and the cost of system integration and long-term maintenance are remarkably reduced.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, system and device for dynamic aggregation of multi-level data catalogs. Background Technology

[0002] In the current digital transformation process, data has become a core strategic asset for various organizations and industries. To achieve effective data governance, sharing, and value extraction, building a unified, standardized, and trustworthy data catalog system is an essential foundation. A data catalog is not only a systematic cataloging of data resources, but also a key support for realizing the assetization, service provision, and circulation of data, especially in the construction of large-scale organizational or national-level data infrastructure that spans regions, levels, and systems.

[0003] Currently, data catalog management technologies mainly fall into several typical patterns, but these methods exhibit significant limitations when dealing with complex, large-scale application scenarios:

[0004] Firstly, manual compilation-based directory management is common in organizations in their early stages of IT adoption or smaller in scale. This method typically relies on spreadsheets, documents, or internal knowledge bases for manual recording and maintenance. While simple to operate, it has significant drawbacks: manual updates are inefficient and prone to information lag; the lack of unified format and semantic standards makes it difficult to guarantee data quality; and it cannot support automated retrieval and integrated access, resulting in severely insufficient scalability and usability.

[0005] Secondly, independently deployed heterogeneous data catalog systems have become increasingly widespread with technological advancements. Different departments or regions often build their own data catalogs based on open-source or commercial tools such as Apache Atlas and Collibra. While effective on a local scale, the lack of top-level design and unified standards leads to significant differences between these systems in data models, metadata structures, and interface protocols, creating a "data silo" effect. This results in the inability to form a globally unified view of data assets, making cross-system data discovery and sharing extremely difficult, and hindering overall data governance effectiveness.

[0006] Furthermore, the initial aggregation method based on interface calls attempts to extract directory information from various independent systems through application programming interfaces (APIs) and aggregate it on a central platform. However, this approach faces severe challenges in practice. The core issue lies in the heterogeneity of directory descriptions: different systems often use inconsistent naming, formats, and semantic expressions for the same data attributes. For example, the data source field might be defined as "source," "dataSource," or "data source," and the update frequency might be expressed in multiple ways, such as "daily," "every natural day," etc. This semantic inconsistency necessitates extensive manual mapping and cleaning for automated aggregation, resulting in high implementation costs and difficulty in continuous maintenance. In addition, such methods often use timed batch processing to synchronize data, making real-time updates difficult and leading to information lag and inconsistencies between the central directory and regional directories. When new regional nodes need to be integrated, custom-developed interfaces and adaptation logic are usually required, resulting in poor system scalability and maintainability, and an overall architecture that struggles to adapt to dynamically changing business environments.

[0007] The shortcomings of the aforementioned existing technologies can be attributed to two main aspects: First, there is a lack of a universally accepted and binding data catalog description standard, making it impossible to unify data definition and semantic expression from the source; second, there is a lack of a supporting technical framework that enables standardized registration, process-based approval, and dynamic synchronization, making it difficult to achieve efficient and reliable multi-level catalog collection while ensuring data quality.

[0008] Chinese patent CN115269927A discloses a method and system for distributed data asset catalog aggregation. This solution addresses the fundamental issue of centralized aggregation of data catalogs, but it essentially employs a centralized, batch-processing, and post-verification technical paradigm. This paradigm has inherent shortcomings in key aspects of modern data asset management, such as real-time dynamic synchronization, source data governance, process-based accountability control, and flexible system expansion. The root cause of these shortcomings lies in the failure to establish an end-to-end governance framework that starts from the data registration source, permeates standardized constraints and process-based approval, and achieves event-driven synchronization.

[0009] Therefore, how to provide a method that can achieve standardized management, process-oriented governance, and dynamic synchronization of data catalogs from regional to global levels, and support the interconnection and reliable use of cross-domain data resources, has become an urgent technical problem to be solved. Summary of the Invention

[0010] In view of this, in order to overcome the shortcomings of the prior art, the present invention aims to provide a method, system and device for dynamic collection of multi-level data catalogs.

[0011] According to a first aspect of the present invention, a method for dynamic aggregation of a multi-level data directory is provided, the method comprising:

[0012] Step S1: At the regional node, based on the preset data directory description strategy, receive the directory operation application submitted by the user and approve the directory operation application;

[0013] Step S2: Based on the approval results, update the status of the corresponding directory entries at the regional node and store the updated directory entries in the regional directory library;

[0014] Step S3: Dynamically report approved and updated directory entries to the global nodes;

[0015] Step S4: At the global node, collect and merge directory entries from at least one regional node, update the global directory database, and provide a unified query service.

[0016] Optionally, the dynamic aggregation method for multi-level data directories of the present invention includes step S1 as follows:

[0017] The directory registration interface provided by the regional node receives directory operation requests submitted by users, which include adding registrations, changing content, or canceling directories.

[0018] The application content submitted by the user is validated for compliance based on the preset data catalog description strategy, which includes the structured fields, data types, format constraints and semantic rules of the data catalog.

[0019] Submit the verified application to the approval process, generate a task to be approved and notify the regional administrator for approval, receive the administrator's approval action, and record the approval result and approval comments.

[0020] Optionally, the approval process for the dynamic aggregation method of the multi-level data directory of the present invention includes:

[0021] Set up a catalog approval workflow unit at the regional node to manage the approval process. When a task to be approved is generated, send an approval notification to the designated regional administrator.

[0022] The approval interface displays the complete content of the catalog entries to be approved to the administrator, including the application type, submitter, application time, and detailed catalog information.

[0023] The approval interface provides two operation options: "Approve" and "Reject," and includes an area for inputting approval comments.

[0024] Record complete approval process information, including approval serial number, approval type, approval status, approval time, approver and approval comments, to form an auditable approval record.

[0025] Optionally, in the dynamic aggregation method of the multi-level data directory of the present invention, step S2 includes: setting up a regional directory status identification unit at the regional node for determining the status of the directory entry based on the approval result; if the approval result is approved and the directory operation is new registration, marking the corresponding directory entry as effective; if the approval result is approved and the directory operation is content change, marking the corresponding directory entry as updated and updating the directory content; if the approval result is approved and the directory operation is directory revocation, marking the corresponding directory entry as invalid; and storing the directory entries with completed status settings and their associated metadata in the regional directory database.

[0026] Optionally, in the dynamic aggregation method of the multi-level data directory of the present invention, step S3 includes: setting up a regional reporting service unit at the regional node for monitoring directory status update events; when a directory entry status update is detected and the corresponding approval result is passed, automatically triggering the reporting process; encapsulating the directory entry to be reported, its operation type, status information, and approval record into a reporting data packet; sending the reporting data packet to the global node through the directory reporting and communication module; and performing authentication and data encryption during the communication process.

[0027] Optionally, in the dynamic aggregation method of the multi-level data directory of the present invention, step S4, at the global node, involves aggregating and merging directory entries from at least one regional node, including:

[0028] Set up a global directory receiving unit at the global node to receive report data packets from regional nodes;

[0029] Parse the reported data packets and extract directory entries, their globally unique identifiers, operation types, and status information;

[0030] Based on the globally unique identifier, determine whether the directory entry is a new entry in the global directory library. If it is a new entry, create a corresponding record in the global directory library; if it is an existing entry, perform content merging or status update according to the operation type.

[0031] Update the version information and update time of directory entries in the global directory repository.

[0032] Optionally, in the dynamic aggregation method for multi-level data directories of the present invention, step S4 further includes:

[0033] A directory status management unit is set up at the global node to maintain the consistency of the directory status. This directory status management unit records the current status, effective time, expiration time and status change history of each directory entry.

[0034] When a directory update report is received from a regional node, the corresponding record in the directory status management unit is updated according to the status information in the report.

[0035] By comparing the directory status of regional nodes with that of global nodes, we ensure that the global directory repository and the state of each regional directory repository remain eventually consistent.

[0036] Optionally, in the dynamic aggregation method of the multi-level data directory of the present invention, in step S4, a unified query service is provided in the following manner:

[0037] A directory query interface unit is set up at the global node, providing two query methods: application programming interface and structured query language;

[0038] Configure search and filtering units that support combined searches based on multiple dimensions such as directory name, data source, region, industry category, and update time.

[0039] Configure a directory display interface unit that supports pagination, sorting, and detailed information viewing to present query results graphically.

[0040] According to a second aspect of the present invention, a dynamic aggregation system for a multi-level data directory is provided, the system comprising:

[0041] The application and approval module is used at the regional node to receive directory operation applications submitted by users and to approve such applications, based on a preset data directory description strategy.

[0042] The update and storage module is used to update the status of the corresponding directory entries at the regional nodes according to the approval results and store the updated directory entries in the regional directory library.

[0043] The reporting module is used to dynamically report approved and updated directory entries to the global nodes.

[0044] The aggregation and merging module is used to aggregate and merge directory entries from at least one regional node at the global node, update the global directory database, and provide a unified query service.

[0045] According to a third aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect of the present invention.

[0046] The dynamic aggregation method, system, and device for multi-level data catalogs of the present invention have the following beneficial technical effects:

[0047] First, improve the accuracy and authority of directory information from the data source. By enforcing a unified data directory description standard at functional nodes in each region, users are required to fill in standardized mandatory fields completely and accurately during registration, and undergo system verification of data type, format, and value range. A dedicated directory approval workflow unit is set up, and all addition, modification, or cancellation operations must be reviewed and approved by the regional administrator before taking effect. From both the registration entry point and process control levels, it is ensured that every piece of information entering the system directory is manually confirmed and traced for accountability, curbing the entry of erroneous, non-standard, or unauthorized data at the source, significantly improving the accuracy, completeness, and business authority of the entire domain data directory.

[0048] Second, streamline and clarify responsibilities in data catalog governance. Elevate data catalog maintenance from a simple technical operation to a standardized governance activity. By designing a complete "submission-approval-effectiveness-reporting" process and recording detailed operation logs for each step (including submitter, approver, operation type, timestamp, approval comments, etc.), a clear and auditable data management responsibility chain is established. This ensures that every change to the data catalog's status is traceable and accountable, not only meeting the core requirements of compliance and accountability in data governance but also providing comprehensive technical support for subsequent data quality assessments, issue tracing, and compliance audits.

[0049] Third, while ensuring standardization, the system also considers the flexibility and timeliness of catalog management. It adopts a combined model of "proactive registration" and "dynamic reporting." On the one hand, data holders or managers are allowed to initiate catalog registration or changes at any time according to business needs, giving the system good business flexibility. On the other hand, through the collaboration of regional reporting service units and the global catalog receiving unit, once the regional catalog has completed approval and status updates, the changed information can be synchronized to all functional nodes in real-time or near real-time. This event-driven dynamic synchronization mechanism overcomes the information delay problem of the traditional batch retrieval model, ensuring that the unified global data catalog can promptly reflect the latest data asset status of each region, thereby providing users with timely and reliable data discovery services.

[0050] Fourth, it significantly reduces the technical complexity and cost of system integration and long-term maintenance. By implementing a unified, standardized registration portal at regional nodes, the resolution of heterogeneity issues is moved to the data entry stage, rather than the backend aggregation and processing stage. For global nodes, the data they receive is already "clean" data that conforms to unified standards and has undergone standardized verification and approval, thus greatly simplifying the logic of data collection, merging, and storage. When a new regional node needs to be connected, only standardized regional functional nodes need to be deployed and ensured to comply with unified specifications, eliminating the need for complex interface adaptation development on the global side. This makes the system more scalable, and significantly improves the simplicity and maintainability of the overall technical architecture. Attached Figure Description

[0051] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart illustrating a dynamic aggregation method for a multi-level data directory according to an exemplary embodiment 1 of the present invention.

[0053] Figure 2 A schematic diagram of the architecture of a dynamic aggregation system for a multi-level data directory provided in Exemplary Embodiment 2 of the present invention;

[0054] Figure 3 A schematic diagram of the architecture of a dynamic aggregation system for a multi-level data directory provided in Exemplary Embodiment 3 of the present invention;

[0055] Figure 4 This is a schematic diagram illustrating the application principle of a dynamic data collection system for a multi-level data directory according to an exemplary embodiment 3 of the present invention.

[0056] Figure 5 This is a schematic diagram of the structure of the device provided by the present invention. Detailed Implementation

[0057] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0058] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0059] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0060] Example 1

[0061] Exemplary embodiment 1 of the present invention provides a dynamic aggregation method for multi-level data directories. Figure 1 This is a flowchart illustrating a dynamic aggregation method for a multi-level data directory according to an exemplary embodiment 1 of the present invention, as shown below. Figure 1 As shown, in this embodiment, the dynamic aggregation of multi-level data directories is implemented in the following manner:

[0062] Step S1: At the regional node, based on the preset data directory description strategy, receive the directory operation application submitted by the user and approve the directory operation application.

[0063] In this embodiment, the system receives directory operation applications submitted by users through the directory registration interface provided by the regional node. These directory operations include adding registrations, changing content, or canceling directories. The system performs normative verification on the application content submitted by users according to a preset data directory description strategy, which includes structured fields, data types, format constraints, and semantic rules of the data directory. The system submits the verified applications to the approval process, generates a task to be approved, and notifies the regional administrator for approval. The system receives the administrator's approval operation and records the approval results and opinions.

[0064] It should be noted that the approval process in this embodiment includes: setting up a catalog approval workflow unit at the regional node to manage the approval process; when a task to be approved is generated, sending an approval notification to the designated regional administrator; displaying the complete content of the catalog entry to be approved to the administrator through the approval operation interface, including the application type, submitter, application time, and detailed catalog information; providing two operation options, approval and rejection, in the approval operation interface, and setting up an approval opinion input area; recording complete approval process information, including approval serial number, approval type, approval status, approval time, approver, and approval opinion, forming an auditable approval record.

[0065] Step S2: Based on the approval results, update the status of the corresponding directory entries at the regional node and store the updated directory entries in the regional directory library.

[0066] In this embodiment, a regional directory status identification unit is set at the regional node to determine the status of the directory entry based on the approval result. If the approval result is approved and the directory operation is new registration, the corresponding directory entry is marked as effective. If the approval result is approved and the directory operation is content change, the corresponding directory entry is marked as updated and the directory content is updated. If the approval result is approved and the directory operation is directory revocation, the corresponding directory entry is marked as invalid. The directory entries with completed status settings and their associated metadata are stored in the regional directory library.

[0067] Step S3: Dynamically report the approved and updated directory entries to the global nodes.

[0068] In this embodiment, a regional reporting service unit is set up at the regional node to monitor directory status update events. When a directory entry status update is detected and the corresponding approval result is passed, the reporting process is automatically triggered. The directory entry to be reported, its operation type, status information, and approval record are encapsulated into a reporting data packet. The reporting data packet is sent to the global node through the directory reporting and communication module. Authentication and data encryption are performed during the communication process.

[0069] Step S4: At the global node, collect and merge directory entries from at least one regional node, update the global directory database, and provide a unified query service.

[0070] In this embodiment, at the global node, directory entries from at least one regional node are aggregated and merged in the following manner:

[0071] A global directory receiving unit is set up at the global node to receive reported data packets from regional nodes; the reported data packets are parsed to extract directory entries and their globally unique identifiers, operation types, and status information; based on the globally unique identifier, it is determined whether the directory entry is a new entry in the global directory library; if it is a new entry, a corresponding record is created in the global directory library; if it is an existing entry, content merging or status update is performed according to the operation type; the version information and update time of the directory entries in the global directory library are updated.

[0072] In practical applications, the method of this embodiment also includes the following state synchronization mechanism:

[0073] A directory status management unit is set up at the global node to maintain the consistency of the directory status. This directory status management unit records the current status, effective time, expiration time, and status change history of each directory entry. When a directory update report is received from a regional node, the corresponding record in the directory status management unit is updated according to the status information in the report. By comparing the directory status of the regional node and the global node, the status of the global directory library and each regional directory library is ensured to maintain eventual consistency.

[0074] In this embodiment, a unified query service is provided in the following manner:

[0075] A directory query interface unit is set up at the global node, providing two query methods: application programming interface and structured query language; a search and filtering unit is set up to support combined searches by multiple dimensions such as directory name, data source, region, industry classification, and update time; and a directory display interface unit is set up to support pagination, sorting, and detailed information viewing, displaying the query results in a graphical manner.

[0076] Example 2

[0077] Exemplary embodiment 2 of the present invention provides a dynamic aggregation system for a multi-level data directory. Figure 2 This is a schematic diagram of the architecture of a dynamic aggregation system for a multi-level data directory according to an exemplary embodiment 2 of the present invention, as shown below. Figure 2 As shown, the dynamic aggregation system for the multi-level data directory in this embodiment includes:

[0078] The application and approval module is used at the regional node to receive directory operation applications submitted by users and to approve such applications, based on a preset data directory description strategy.

[0079] The update and storage module is used to update the status of the corresponding directory entries at the regional nodes according to the approval results and store the updated directory entries in the regional directory library.

[0080] The reporting module is used to dynamically report approved and updated directory entries to the global nodes.

[0081] The aggregation and merging module is used to aggregate and merge directory entries from at least one regional node at the global node, update the global directory database, and provide a unified query service.

[0082] Example 3

[0083] Exemplary embodiment 3 of the present invention provides a dynamic aggregation system for a multi-level data directory. The dynamic aggregation system for a multi-level data directory in this embodiment includes:

[0084] Multiple independent regional nodes, each of which includes a directory registration interface unit, a directory operation submission unit, a directory approval workflow unit, a post-approval directory processing unit, a regional directory status identification unit, a regional directory storage unit, and a regional reporting service unit;

[0085] A global node includes a global directory receiving unit, a directory collection and merging unit, a global directory storage unit, a directory status management unit, a directory query interface unit, a retrieval and filtering unit, and a directory display interface unit;

[0086] Regional nodes and global nodes interact with each other through a directory reporting and communication module, which includes a communication protocol unit, a message transmission unit, and a security and authentication unit.

[0087] The following section provides a more detailed description of the dynamic data cataloging system for this embodiment in a specific scenario. Figure 3 This is a schematic diagram of the architecture of a dynamic aggregation system for a multi-level data directory provided in Exemplary Embodiment 3 of the present invention, as shown below. Figure 3 As shown in this embodiment, the dynamic aggregation system for a multi-level data directory includes: several regional functional nodes, one global functional node, a data directory storage and management module, a directory display and query module, a directory description specification and adaptation module, and a directory reporting and communication module. Each module can run on one or more servers, cloud hosts, or virtual machines, and is interconnected via a network. The regional functional nodes provide standardized registration entry points and approval processes, enabling precise manual management of the regional data directories, which are then aggregated into a unified global data directory through a multi-level reporting mechanism.

[0088] Specifically, in this embodiment, the composition and interrelationships of each part are as follows:

[0089] I. Regional Functional Nodes: Deployed within platforms in various regions or departments, these nodes serve as the central hub for the registration, approval, and management of data catalogs within their respective regions, providing users with an interactive management platform that adheres to various standards. In practical applications, regional functional nodes include:

[0090] 1. Catalog Registration Interface Unit: Provides a graphical user interface (GUI) or a set of application programming interfaces (APIs) for the holders or managers (users) of data resources / products. The design of this interface strictly adheres to the "National Data Infrastructure Construction Technical Document NDI-TR-2025-06 Data Infrastructure Data Catalog Description Specification," ensuring that users must fill in all required fields during registration and comply with the constraints on field types, formats, and value ranges.

[0091] 2. Directory Operation Submission Unit: Receives directory operation requests submitted by users through the directory registration interface unit 111, including three types: new registration, content modification, and directory cancellation. It encapsulates the user's submission request into a task awaiting approval and stores it in the approval queue.

[0092] 3. Directory Approval Workflow Unit: Used to manage the approval process for directory operations. When a new task is pending approval, it automatically notifies the regional functional node administrator. It provides an approval interface for administrators, displaying the complete content of the directory to be approved and offering "Approve" or "Reject" options, supporting the input of approval comments. It records the complete approval history, including approval serial number, approval type, subject name (submitter), subject identification code, approval status, and application time. After review, it records the approval status, approval comments, reporting results, and approval time, forming a traceable audit history list.

[0093] 4. Post-Approval Catalog Processing Unit: Receives the approval result from the catalog approval workflow unit. If the approval result is "Approved," the corresponding catalog entry and its operation type (add, update, cancel) are passed to the regional catalog status identification unit. If the approval result is "Rejected," the task status is updated to "Returned," and the submitting user is notified.

[0094] 5. Regional Directory Status Identification Unit: Based on the approved operation type, confirm the status of directory entries. For example, new registrations and changes correspond to an effective status, and the data directory displays the corresponding entries; directory cancellation corresponds to an invalid status, and the data directory no longer displays the corresponding entries.

[0095] 6. Regional Directory Storage Unit: Stores all registered and approved regional standard directories and their latest status for internal querying and historical tracing within the region.

[0096] 7. Regional Reporting Service Unit: When there are new approved catalog changes, it is automatically triggered to report these new, updated, and canceled catalog entries to the global functional nodes through the catalog reporting and communication module.

[0097] In this embodiment, users submit applications through the directory registration interface unit and the directory operation submission unit. The applications are then submitted to the directory approval workflow unit for approval. After approval, the approved directory processing unit hands the application over to the regional directory status identification unit to confirm the status, stores it in the regional directory storage unit, and finally reports it to the entire domain by the regional reporting service unit.

[0098] II. Global Functional Nodes: Used for receiving, aggregating, querying, and managing status, including:

[0099] Global directory receiving unit: Receives standardized directory change data from multiple regional functional nodes 10;

[0100] Directory aggregation and merging unit: Aggregate directory entries from different regions based on the globally unique identifier of the data resource / product;

[0101] Global directory storage unit: stores the aggregated global unified data directory and its historical versions;

[0102] Directory Status Management Unit: Maintains the status of directory additions, updates, and cancellations, as well as their effective / expired times.

[0103] III. Data Directory Storage and Management Module: This module includes a regional directory database and a global directory database. The regional directory database is used to persist the regional standard directories and statuses generated by functional nodes in each region. The global directory database is used to persist the unified global directory and version information.

[0104] IV. The directory display and search module includes:

[0105] Directory query interface unit: Provides API or SQL interface;

[0106] Search and filtering unit: Search by name, data source, product type, region, industry, geographic region, etc.

[0107] Directory display interface unit: Provides users with a unified data directory display and browsing experience across the entire domain.

[0108] V. Directory description specifications and adaptation modules, including:

[0109] Directory specification definition unit: uniformly defines data directory fields, types, and encoding rules;

[0110] Adaptation rule configuration unit: Configure field mapping rules from different data sources to a unified standard;

[0111] Specification parsing and verification unit: Provides specification parsing and verification capabilities for regional functional nodes and global functional nodes.

[0112] VI. Directory reporting and communication module, including:

[0113] Communication protocol unit: Supports HTTP / HTTPS, message queues, RPC, etc.;

[0114] Message Transmission Unit: Responsible for the transmission of directory data packets between regional functional nodes and global functional nodes;

[0115] Security and authentication unit: encrypts and authenticates the communication link.

[0116] Figure 4 This is a schematic diagram illustrating the application principle of a dynamic data collection system for a multi-level data directory according to Exemplary Embodiment 3 of the present invention, as shown below. Figure 4As shown in this embodiment, users, following standard specifications, submit applications for adding, modifying, or withdrawing data directories through the directory registration interface unit at the regional functional node. The applications enter the directory approval workflow unit for review by the regional administrator. After approval, the regional functional node confirms the directory status and reports it to the global functional node. The global functional node collects, synchronizes, and queries approved directory change information from various regions, maintaining the latest unified global data directory in the global directory database. Based on this database, the directory display and query module provides users with unified, accurate, and authoritative global data directory display and query services.

[0117] The multi-level data catalog dynamic aggregation method and system of this invention, by introducing standardized registration specifications, approval-driven process control, and a dynamic real-time reporting and synchronization mechanism, has the following beneficial technical effects in multi-level, cross-regional data catalog management scenarios:

[0118] First, improve the accuracy and authority of directory information from the data source. By enforcing a unified data directory description standard at functional nodes in each region, users are required to fill in standardized mandatory fields completely and accurately during registration, and undergo system verification of data type, format, and value range. A dedicated directory approval workflow unit is set up, and all addition, modification, or cancellation operations must be reviewed and approved by the regional administrator before taking effect. From both the registration entry point and process control levels, it is ensured that every piece of information entering the system directory is manually confirmed and traced for accountability, curbing the entry of erroneous, non-standard, or unauthorized data at the source, significantly improving the accuracy, completeness, and business authority of the entire domain data directory.

[0119] Second, streamline and clarify responsibilities in data catalog governance. Elevate data catalog maintenance from a simple technical operation to a standardized governance activity. By designing a complete "submission-approval-effectiveness-reporting" process and recording detailed operation logs for each step (including submitter, approver, operation type, timestamp, approval comments, etc.), a clear and auditable data management responsibility chain is established. This ensures that every change to the data catalog's status is traceable and accountable, not only meeting the core requirements of compliance and accountability in data governance but also providing comprehensive technical support for subsequent data quality assessments, issue tracing, and compliance audits.

[0120] Third, while ensuring standardization, the system also considers the flexibility and timeliness of catalog management. It adopts a combined model of "proactive registration" and "dynamic reporting." On the one hand, data holders or managers are allowed to initiate catalog registration or changes at any time according to business needs, giving the system good business flexibility. On the other hand, through the collaboration of regional reporting service units and the global catalog receiving unit, once the regional catalog has completed approval and status updates, the changed information can be synchronized to all functional nodes in real-time or near real-time. This event-driven dynamic synchronization mechanism overcomes the information delay problem of the traditional batch retrieval model, ensuring that the unified global data catalog can promptly reflect the latest data asset status of each region, thereby providing users with timely and reliable data discovery services.

[0121] Fourth, it significantly reduces the technical complexity and cost of system integration and long-term maintenance. By implementing a unified, standardized registration portal at regional nodes, the resolution of heterogeneity issues is moved to the data entry stage, rather than the backend aggregation and processing stage. For global nodes, the data they receive is already "clean" data that conforms to unified standards and has undergone standardized verification and approval, thus greatly simplifying the logic of data collection, merging, and storage. When a new regional node needs to be connected, only standardized regional functional nodes need to be deployed and ensured to comply with unified specifications, eliminating the need for complex interface adaptation development on the global side. This makes the system more scalable, and significantly improves the simplicity and maintainability of the overall technical architecture.

[0122] like Figure 5 As shown, the present invention also provides a device including a processor 310, a communication interface 320, a memory 330 for storing processor-executable computer programs, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 implements the aforementioned dynamic aggregation method for a multi-level data directory by running the executable computer program.

[0123] The computer program in memory 330, when implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0124] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected based on actual needs to achieve the purpose of this embodiment. Those skilled in the art can understand and implement this without any creative effort.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0126] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A dynamic aggregation method for a multi-level data directory, characterized in that, The method includes: Step S1: At the regional node, based on the preset data directory description strategy, receive the directory operation application submitted by the user and approve the directory operation application; Step S2: Based on the approval results, update the status of the corresponding directory entries at the regional node and store the updated directory entries in the regional directory library; Step S3: Dynamically report approved and updated directory entries to the global nodes; Step S4: At the global node, collect and merge directory entries from at least one regional node, update the global directory database, and provide a unified query service.

2. The dynamic aggregation method for multi-level data catalogs according to claim 1, characterized in that, Step S1 includes: The directory registration interface provided by the regional node receives directory operation requests submitted by users, which include adding registrations, changing content, or canceling directories. The application content submitted by the user is validated for compliance based on the preset data catalog description strategy, which includes the structured fields, data types, format constraints and semantic rules of the data catalog. Submit the verified application to the approval process, generate a task to be approved and notify the regional administrator for approval, receive the administrator's approval action, and record the approval result and approval comments.

3. The dynamic aggregation method for multi-level data catalogs according to claim 2, characterized in that, The approval process includes: Set up a catalog approval workflow unit at the regional node to manage the approval process. When a task to be approved is generated, send an approval notification to the designated regional administrator. The approval interface displays the complete content of the catalog entries to be approved to the administrator, including the application type, submitter, application time, and detailed catalog information. The approval interface provides two operation options: "Approve" and "Reject," and includes an area for inputting approval comments. Record complete approval process information, including approval serial number, approval type, approval status, approval time, approver and approval comments, to form an auditable approval record.

4. The dynamic aggregation method for multi-level data catalogs according to claim 1, characterized in that, Step S2 includes: setting up a regional directory status identification unit at the regional node to determine the status of directory entries based on the approval result; if the approval result is approved and the directory operation is new registration, marking the corresponding directory entry as effective; if the approval result is approved and the directory operation is content change, marking the corresponding directory entry as updated and updating the directory content; if the approval result is approved and the directory operation is directory revocation, marking the corresponding directory entry as invalid; and storing the directory entries with completed status settings and their associated metadata in the regional directory database.

5. The dynamic aggregation method for multi-level data catalogs according to claim 1, characterized in that, Step S3 includes: setting up a regional reporting service unit at the regional node to monitor directory status update events; when a directory entry status update is detected and the corresponding approval result is passed, the reporting process is automatically triggered; the directory entry to be reported, its operation type, status information, and approval record are encapsulated into a reporting data packet; the reporting data packet is sent to the global node through the directory reporting and communication module; and authentication and data encryption are performed during the communication process.

6. The dynamic aggregation method for multi-level data catalogs according to claim 1, characterized in that, In step S4, at the global node, directory entries from at least one regional node are aggregated and merged, including: Set up a global directory receiving unit at the global node to receive report data packets from regional nodes; Parse the reported data packets and extract directory entries, their globally unique identifiers, operation types, and status information; Based on the globally unique identifier, determine whether the directory entry is a new entry in the global directory library. If it is a new entry, create a corresponding record in the global directory library; if it is an existing entry, perform content merging or status update according to the operation type. Update the version information and update time of directory entries in the global directory repository.

7. The dynamic aggregation method for multi-level data catalogs according to claim 6, characterized in that, Step S4 also includes: A directory status management unit is set up at the global node to maintain the consistency of the directory status. This directory status management unit records the current status, effective time, expiration time and status change history of each directory entry. When a directory update report is received from a regional node, the corresponding record in the directory status management unit is updated according to the status information in the report. By comparing the directory status of regional nodes with that of global nodes, we ensure that the global directory repository and the state of each regional directory repository remain eventually consistent.

8. The dynamic aggregation method for multi-level data catalogs according to claim 1, characterized in that, In step S4, a unified query service is provided as follows: A directory query interface unit is set up at the global node, providing two query methods: application programming interface and structured query language; Configure search and filtering units that support combined searches based on multiple dimensions such as directory name, data source, region, industry category, and update time. Configure a directory display interface unit that supports pagination, sorting, and detailed information viewing to present query results graphically.

9. A dynamic data aggregation system for a multi-level data directory, characterized in that, The system includes: The application and approval module is used at the regional node to receive directory operation applications submitted by users and to approve such applications, based on a preset data directory description strategy. The update and storage module is used to update the status of the corresponding directory entries at the regional nodes according to the approval results and store the updated directory entries in the regional directory library. The reporting module is used to dynamically report approved and updated directory entries to the global nodes. The aggregation and merging module is used to aggregate and merge directory entries from at least one regional node at the global node, update the global directory database, and provide a unified query service.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method according to any one of claims 1-8.

Images