Enterprise data asset directory management method and system based on multi-source fusion

Abstract

The present application relates to the technical field of data management, and more particularly to a kind of enterprise data asset directory management method and system based on multi-source fusion, the method includes obtaining the metadata of the data asset of multiple source heterogeneous in enterprise, and the metadata is fused and analyzed to obtain data characteristic portrait;Matching data characteristic portrait corresponding directory construction model, to call directory structure template from preset directory template library by directory construction model;According to the directory construction process of the directory structure template simulation data asset corresponding;According to the directory construction process, the directory structure template is optimized, and enterprise data asset directory is generated according to the optimized directory structure template.The present application is based on the semantic matching mechanism of data characteristic portrait and directory construction model, automatically calls and optimizes directory structure template from template library, replaces traditional manual experience driven template selection mode, improves template reuse rate while reducing manual debugging cost.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and in particular to a method and system for managing enterprise data asset catalogs based on multi-source fusion. Background Technology

[0002] As enterprises deepen their digital transformation, the value of data assets as core production factors is becoming increasingly prominent. Building an efficient data asset catalog has become a crucial step in achieving data governance and application. A data asset catalog, through the structured organization of multi-source heterogeneous data, can clearly present the distribution, relationships, and usage rules of data assets, providing fundamental support for data retrieval, sharing, and value mining. However, existing data asset catalog construction relies on fixed templates or manual experience, lacking an intelligent matching mechanism for data characteristics. This makes it difficult to adapt to the dynamic changes of data assets in different business scenarios, resulting in a disconnect between the catalog structure and the actual data characteristics.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a method and system for managing enterprise data asset catalogs based on multi-source fusion. This aims to solve the technical problem that existing data asset catalog construction relies on fixed templates or manual experience, lacks an intelligent matching mechanism for data characteristics, and is difficult to adapt to the dynamic changes of data assets under different business scenarios, resulting in a disconnect between the catalog structure and the actual data characteristics.

[0005] To achieve the above objectives, the present invention provides a method for managing enterprise data asset catalogs based on multi-source fusion, the method comprising:

[0006] Obtain metadata of multi-source heterogeneous data assets within the enterprise, and perform fusion analysis on the metadata to obtain data feature profiles;

[0007] Match the directory construction model corresponding to the data feature profile, so as to call the directory structure template from the preset directory template library through the directory construction model;

[0008] Simulate the directory construction process corresponding to the data asset based on the directory structure template;

[0009] The directory structure template is optimized according to the construction process, and an enterprise data asset catalog is generated based on the optimized directory structure template.

[0010] Optionally, the metadata includes structural feature parameters. The process of acquiring multi-source heterogeneous data asset metadata within the enterprise and performing fusion analysis on the metadata to obtain a data feature profile includes:

[0011] Obtain the structural feature parameters of the data asset, wherein the structural feature parameters include data pattern parameters and lineage relationship parameters;

[0012] Ontology modeling is used to extract core entities and relationships from the structural feature parameters, and a data domain model and lineage mapping are constructed based on the core entities and relationships to obtain the correlation parameters;

[0013] The data association vector is obtained based on the association degree parameter, and the data association vector is standardized to obtain the data feature profile, wherein the data association vector includes concentration, sensitivity and activity.

[0014] Optionally, the step of calling a directory structure template from a preset directory template library through the directory construction model includes:

[0015] Obtain the template features corresponding to each template in the directory template library, and calculate the matching degree between each template feature and the data feature profile through the semantic relationship preset in the directory construction model;

[0016] When the matching degree exceeds a preset threshold, the preset template optimization rules are invoked to adapt and optimize the current template, and a directory structure template corresponding to the optimized template is generated.

[0017] Optionally, the process of simulating the directory construction process corresponding to the data asset based on the directory structure template includes:

[0018] A virtual directory topology model is constructed based on the node hierarchy of the directory structure template, and a dependency resolution model is obtained by resolving the dependency relationship of the virtual directory topology model through a directed acyclic graph.

[0019] An entity relationship equation is established using a graph embedding algorithm, and the hierarchical relationship data and permission mapping data of the dependency parsing model are calculated by the entity relationship equation and the path reasoning engine.

[0020] The feature constraint relationship between the hierarchical relationship data and the permission mapping data is obtained by extracting features from the hierarchical relationship data and the permission mapping data through a graph neural network.

[0021] The real-time metadata in the data management system is processed according to the aforementioned feature constraint relationship to obtain the structural evolution data and permission change data of the data assets during the catalog construction process;

[0022] By fusing the structural evolution data and the permission change data through a Bayesian network, the actual lineage links and actual permission distributions are obtained.

[0023] The rationality of the directory structure template is verified based on the actual lineage, the actual permission distribution, the hierarchical relationship data, and the permission mapping data.

[0024] Optionally, optimizing the directory structure template according to the construction process and generating an enterprise data asset catalog based on the optimized directory structure template includes:

[0025] When the directory structure template is verified to be reasonable, an information entropy-based evaluator is used to calculate the structure evaluation parameters and permission evaluation parameters corresponding to the directory structure template, and a template sequence to be optimized is generated based on the comparison results of the structure evaluation parameters and the permission evaluation parameters with preset thresholds respectively.

[0026] A random forest regressor is trained based on the template sequence to be optimized. The random forest regressor is used to predict the template optimization amount. The node parameters of the directory structure template are adjusted based on the template optimization amount to obtain the node relationship coordinates of the updated directory structure template.

[0027] The node relationship coordinates of the updated directory structure template are reconstructed using the minimum spanning tree algorithm, and the adjustment planning path is obtained by dynamically adjusting the planning based on the minimum spanning tree algorithm under the constraints of maximum level depth and permission convergence rate.

[0028] The centrality value at each node is calculated for the adjusted planning path. Redundant nodes are inserted in the centrality mutation region through the community detection algorithm to obtain the optimized directory structure template. The enterprise data asset catalog is then generated based on the optimized directory structure template.

[0029] Optionally, before acquiring the metadata of multi-source heterogeneous data assets within the enterprise, the method further includes:

[0030] Obtain the environmental parameters of the enterprise IT architecture to construct a data governance dynamics model corresponding to the IT architecture and perform an impact analysis on the data governance dynamics model to obtain the architecture dependency cloud map and permission distribution cloud map corresponding to the data assets under each preset initial directory template.

[0031] Extract the nodes of each initial directory template in the architecture dependency cloud and the permission gradient in the permission distribution cloud, and calculate the evolution rate and permission change rate of each initial directory template at each node to obtain the architecture dependency change curve of the data asset in the construction process under different initial directory templates.

[0032] Based on the architecture dependency change curve, select multiple first directory templates from several initial directory templates;

[0033] The first parameter in each of the first directory templates is evolved according to the preset template evolution algorithm to obtain several evolved template combinations and save them to the preset directory template library.

[0034] Optionally, the environmental parameters include system coupling parameters and data flow strategies. The process of obtaining the environmental parameters of the enterprise IT architecture, constructing a data governance dynamics model corresponding to the IT architecture, and performing an impact analysis on the data governance dynamics model to obtain the architecture dependency cloud map and permission distribution cloud map corresponding to the data assets under each preset initial directory template includes:

[0035] Obtain the system coupling parameters and data flow strategy of the enterprise IT architecture to calculate the governance impedance matrix corresponding to the IT architecture. The system coupling parameters include service call depth and dependency density, and the governance impedance matrix is ​​determined by the data flow strategy, service call depth and dependency density.

[0036] The impact value of metadata nodes is obtained through the governance impedance matrix and mapped to the topology nodes to establish a graph diffusion equation. The topology nodes are partitioned using a directed graph and the node density in the core business area is increased to twice the basic density.

[0037] The node centrality value is calculated based on the graph diffusion equation to obtain the influence field calculation equation, i.e., the data governance dynamics model; wherein, the node centrality value is determined by the in-degree vector and the out-degree vector;

[0038] Based on the influence field calculation equation, a set of nonlinear metadata evolution equations is established to obtain the architecture dependency cloud map and permission distribution cloud map corresponding to each preset initial directory template of the data asset; wherein, the set of nonlinear metadata evolution equations is solved by gradient descent until the loss function is less than a preset value.

[0039] Optionally, calculating the evolution rate and permission change rate of each initial directory template at each node to obtain the architecture dependency change curve of the data asset during the construction process under different initial directory templates includes:

[0040] Obtain the template topology corresponding to each initial directory template, and obtain the adjacency dataset of the nodes on the template topology; wherein, different initial directory templates correspond to different topology shapes, topology depths and topology connectivity;

[0041] The permission distribution value of the node is calculated using a page ranking algorithm; wherein, the permission distribution value is calculated through the mapping relationship between node importance and permission.

[0042] A data evolution dynamics equation is established based on the permission distribution value, and the Runge-Kutta method is used to solve for the permission change rate value of the node; wherein, the permission change rate value is obtained by calculating the derivative of permission with respect to time;

[0043] The state space equation is constructed based on the permission change rate of the node and the dependency on the architecture dependency cloud, and the dependency time function is obtained by Kalman filtering. The architecture dependency change curve is obtained based on the dependency time function.

[0044] Optionally, the step of selecting multiple first directory templates from a plurality of initial directory templates based on the architecture dependency change curve includes:

[0045] The current architecture dependency of each initial directory template is determined by the architecture dependency change curve. Constraints are constructed based on the preset maximum dependency of the data asset and the current architecture dependency of each initial directory template. Dependency constraint optimization parameters are calculated by the Lagrange multiplier method.

[0046] Based on the dependency constraint optimization parameters, a multi-objective optimization function is established for the dependency objective function and the permission objective function, and the mapping relationship between the template vector and the constraint response vector corresponding to each initial directory template is obtained through a support vector machine with a polynomial kernel function.

[0047] Based on the mapping relationship, a set of constrained optimization equations is constructed using the Karouch-Kun-Tucker conditions, and the optimal directory template and dual variables corresponding to each initial directory template are obtained by solving the interior point method.

[0048] If the norm difference between the optimal directory template and the initial template corresponding to the initial directory template is less than a preset convergence threshold, then the optimal directory template is stored as the first directory template.

[0049] Furthermore, to achieve the above objectives, the present invention also provides an enterprise data asset catalog management system based on multi-source fusion, the enterprise data asset catalog management system based on multi-source fusion comprising:

[0050] The data profiling module is used to acquire metadata of multi-source heterogeneous data assets within the enterprise, and to perform fusion analysis on the metadata to obtain data feature profiles.

[0051] The template matching module is used to match the directory construction model corresponding to the data feature profile, so as to call the directory structure template from the preset directory template library through the directory construction model;

[0052] A simulation module is constructed to simulate the directory construction process corresponding to the data asset based on the directory structure template.

[0053] An optimization generation module is used to optimize the directory structure template according to the construction process, and generate an enterprise data asset catalog based on the optimized directory structure template.

[0054] This invention provides a multi-source fusion-based enterprise data asset catalog management method. This method constructs a unified data feature profile by fusing and analyzing heterogeneous metadata from multiple sources within the enterprise, breaking down data silos and achieving standardized processing of cross-system and cross-format data. Compared to traditional manual sorting or single-data-source processing, this solution significantly improves the comprehensiveness and accuracy of data feature extraction, providing a more realistic data foundation for catalog construction and avoiding catalog structure deviations caused by data fragmentation. Based on a semantic matching mechanism between data feature profiles and catalog construction models, it automatically calls and optimizes catalog structure templates from a template library, replacing the traditional manual experience-driven template selection mode. By calculating the matching degree between template features and data features and triggering adaptive optimization rules, the catalog template can dynamically adapt to the structural complexity, business relevance, and permission requirements of different data assets, improving template reusability while reducing manual debugging costs. This is particularly suitable for enterprise environments with complex business scenarios and dynamically changing data. Utilizing technologies such as virtual catalog topology models, dependency parsing, and graph neural networks, the entire catalog construction process is simulated, simulating the structural evolution and permission mapping of data assets within the catalog in advance. By fusing structural evolution data and permission change data through Bayesian networks, the rationality of the directory structure's lineage and permission distribution is verified. This avoids the inefficient traditional approach of directly applying templates and repeatedly adjusting them later, shifting the problem-solving process to the construction phase and significantly reducing the time and resource costs of iterative directory optimization. Based on the structural and permission evaluation parameters fed back during the construction process, intelligent adjustments to template node parameters are achieved through random forest regressors and minimum spanning tree algorithms, constructing a dynamic optimization closed loop. Redundant nodes are inserted or node relationships are reconstructed under constraints of maximum hierarchical depth and permission convergence rate, ensuring that the generated directory structure not only meets current data characteristics but also possesses scalability to cope with future data asset changes, improving the flexibility and long-term usability of directory management. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the hardware operating environment of the enterprise data asset catalog management device based on multi-source fusion involved in the embodiments of the present invention;

[0056] Figure 2 This is a flowchart illustrating an embodiment of the enterprise data asset catalog management method based on multi-source fusion according to the present invention;

[0057] Figure 3 This is a structural block diagram of an embodiment of the enterprise data asset catalog management system based on multi-source fusion of the present invention.

[0058] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0059] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0060] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of an enterprise data asset catalog management device based on multi-source fusion, which is part of the hardware operating environment of the embodiment of the present invention.

[0061] like Figure 1 As shown, the enterprise data asset catalog management device based on multi-source fusion may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0062] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on enterprise data asset catalog management devices based on multi-source fusion, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0063] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an enterprise data asset catalog management program based on multi-source fusion.

[0064] exist Figure 1In the enterprise data asset catalog management device based on multi-source fusion shown, the network interface 1004 is mainly used to connect to the backend server and communicate with the backend server; the user interface 1003 is mainly used to connect to peripheral devices; the enterprise data asset catalog management device based on multi-source fusion calls the enterprise data asset catalog management program based on multi-source fusion stored in the memory 1005 through the processor 1001, and executes the enterprise data asset catalog management method based on multi-source fusion provided in the embodiment of the present invention.

[0065] Based on the above hardware structure, an embodiment of the enterprise data asset catalog management method based on multi-source fusion of the present invention is proposed.

[0066] Reference Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the enterprise data asset catalog management method based on multi-source fusion of the present invention, which presents an embodiment of the enterprise data asset catalog management method based on multi-source fusion of the present invention.

[0067] In one embodiment, the enterprise data asset catalog management method based on multi-source fusion includes the following steps:

[0068] Step S100: Obtain metadata of multi-source heterogeneous data assets within the enterprise, and perform fusion analysis on the metadata to obtain data feature profiles;

[0069] Multi-source heterogeneous metadata refers to metadata from different data sources, such as databases, file systems, and API interfaces, that differ in format, structure, and semantics. Examples include table structure information in relational databases, field definitions in log files, and storage paths for unstructured data. This type of metadata can be extracted from various enterprise systems using metadata acquisition tools. Data feature profiles are multi-dimensional descriptions of data assets, including statistical features, structural features, and business tags, used as a basis for subsequent catalog construction decisions. Fusion analysis unifies heterogeneous metadata into a standardized form through data cleaning, alignment, and integration techniques, such as using ETL tools or ontology-based semantic alignment methods. In a specific embodiment, this process first removes redundant or erroneous information through cleaning operations, then uses ontology mapping or machine learning models to semantically align the metadata, eliminating naming differences between different systems, and finally generates data feature profiles through statistical analysis and business rule definitions. This operation solves the metadata fragmentation problem caused by data silos in traditional methods, providing a unified data foundation for subsequent steps.

[0070] Step S200: Match the directory construction model corresponding to the data feature profile, so as to call the directory structure template from the preset directory template library through the directory construction model;

[0071] The directory construction model is a rule-based or machine learning-based algorithm used to match data features with directory templates, such as a similarity-based recommendation model or a rule-engine-based template selection strategy. The directory template library contains pre-stored templates with different topologies, including tree, network, and star structures, each associated with its applicable data feature parameters. The semantic matching mechanism uses Natural Language Processing (NLP) or knowledge graph technology to analyze the semantic relationship between data feature profiles and template descriptions, such as using word vector models to calculate the similarity between feature keywords and template tags. The technical operation involves inputting key parameters from the data feature profiles into the directory construction model. The model calculates the matching degree between each template and the current data features using preset rules or a trained classifier. Templates with high matching degrees are prioritized, triggering adaptive optimization rules, such as adjusting node levels or permission configurations. For example, if data features show high business relevance, the model may choose a network template instead of a tree template. This process replaces manual trial-and-error template selection, improving the accuracy and efficiency of template invocation.

[0072] Step S300: Simulate the directory construction process corresponding to the data assets based on the directory structure template;

[0073] The virtual directory topology model is a mathematical abstraction of the directory structure, representing node relationships and dependency paths through a graph structure. Dependency resolution is used to analyze logical or physical dependencies between data assets, such as a table field depending on the API output of another system. Graph neural networks (such as GraphSAGE or GAT) are used to predict directory evolution paths or detect abnormal dependencies, while Bayesian networks integrate structural evolution data and permission change data to verify the rationality of lineage chains and permission distribution. The technical operation first constructs a virtual directory topology model, simulating the impact of adding, deleting, and modifying directory nodes on the overall structure. Graph neural networks predict the evolution path of data assets within the directory, and Bayesian networks analyze the impact of permission changes on access control. For example, if a node permission change may trigger a chain reaction, the model will mark the risk point in advance. This simulation process transforms the traditional linear "build-and-verify" process into a closed loop of "verify-before-implementation," reducing later adjustment costs.

[0074] Step S400: Optimize the directory structure template according to the construction process, and generate the enterprise data asset catalog based on the optimized directory structure template.

[0075] Structural evaluation parameters include metrics that measure the rationality of the directory structure, such as hierarchy depth, node redundancy, and scalability. Permission evaluation parameters assess whether permission configurations meet the principle of least privilege or business requirements, such as permission convergence rate, i.e., the efficiency of permission hierarchy compression. The Random Forest Regressor predicts the optimal parameter adjustment direction through the ensemble of multiple decision trees, such as predicting the optimal depth of node levels. The Minimum Spanning Tree Algorithm is used to optimize the directory topology, such as reducing redundant nodes while maintaining connectivity. Redundant nodes are backup nodes added to cope with future data expansion, improving structural resilience. Reconstructing node relationships optimizes the topology logic by adjusting node connection methods, such as changing parent-child relationships to sibling relationships. The technical operation first collects structural and permission evaluation parameters based on simulation results. If a node's permissions have not converged, and structural evaluation parameters such as the current directory depth exceeding a threshold are present, the Random Forest Regressor predicts parameter adjustment direction based on historical optimization data, and the Minimum Spanning Tree Algorithm optimizes node connections to reduce redundancy. If data characteristics indicate that new related data may be added in the future, the algorithm will insert redundant nodes or reconstruct relationships to enhance scalability. The final generated directory structure not only conforms to the current data characteristics but also has dynamic adaptability, avoiding the frequent reconstruction caused by the rigid structure of traditional directories.

[0076] This embodiment provides a multi-source fusion-based enterprise data asset catalog management method. It constructs data feature profiles through the fusion analysis of heterogeneous metadata from multiple sources, calls suitable catalog structure templates from a template library based on a semantic matching mechanism, and performs simulation verification of the catalog construction process using graph neural networks and Bayesian networks. Finally, it optimizes the structure using a random forest regressor and a minimum spanning tree algorithm. This achieves the following technical effects: eliminating metadata fragmentation caused by data silos, improving the accuracy and efficiency of template calls, identifying structural or permission risks in advance through simulation verification, reducing later maintenance costs, and reserving future expansion space while meeting current needs. Therefore, it provides an efficient, intelligent, and sustainable solution for enterprise data governance.

[0077] In one embodiment, the metadata includes structural feature parameters, acquiring multi-source heterogeneous data asset metadata within the enterprise, and performing fusion analysis on the metadata to obtain a data feature profile, including:

[0078] Obtain the structural characteristic parameters of the data assets, including data pattern parameters and lineage parameters.

[0079] Data schema parameters can be metadata describing the data structure, such as the field types of database tables, the storage format of unstructured data, or the primary and foreign key relationships of data tables. These can be obtained by parsing table structure definition files or API documentation. For example, data schema parameters may include data type definitions for fields in relational databases or document structure templates for NoSQL databases. Lineage parameters can be metadata recording data sources, transformation paths, and dependencies. For example, data in a report field may originate from aggregation operations across multiple database tables, or data may have undergone cleaning steps using an ETL tool. For example, lineage parameters can be extracted by tracking data flow logs or ETL job configuration files. Technical operations can use metadata collection tools to extract raw metadata from various enterprise systems and filter out structural characteristic parameters. For example, parsing Hadoop logs can extract transformation dependencies between data tables, or reading foreign key constraint definitions in databases can identify entity relationships. This process ensures that the metadata contains complete information on structure and lineage, providing foundational data for subsequent modeling.

[0080] Ontology modeling is used to extract core entities and relationships from structural feature parameters, and a data domain model and lineage mapping are constructed based on the core entities and relationships to obtain the correlation parameters.

[0081] Ontology modeling can define concepts, attributes, and relationships within a business domain using formal languages. For example, defining the "customer" entity as a class with attributes like "order" and "contact information," and establishing an ownership relationship for "customer orders." Data domain models can be entity-relationship graphs built upon business logic. For instance, connecting the "sales order" entity with "product" and "warehouse" entities through relationships like "containment" and "inventory association." A lineage map can represent the path of data from its original source to its final use in a graph structure. For example, data from field A is transformed by an ETL tool to become the input for report field B. Association parameters can quantify the strength of associations between entities, such as calculating the shortest path length between two entities in the lineage map, the number of data sources they commonly depend on, or the frequency of collaboration in business processes. Technical operations can use ontology editing tools to semantically analyze structural feature parameters, identifying core entities and their relationships. For example, based on foreign key constraints in data schema parameters, connecting the "customer" and "order" entities to establish an ownership relationship for "customer orders." Simultaneously, a lineage map is generated based on lineage parameters, recording the source and transformation path of each data node. The calculation of the association parameter can involve graph algorithms (such as PageRank) or rule-based scoring systems. For example, if two entities share multiple transformation steps in a lineage graph, their association parameter value will be higher. This process makes implicit structural and lineage information explicit into computable parameters, providing a basis for the subsequent generation of data association vectors.

[0082] Data association vectors are obtained based on association parameters, and data association vectors are standardized to obtain data feature profiles, where data association vectors include concentration, sensitivity, and activity.

[0083] Concentration can refer to the degree of concentration of data storage or processing, such as 80% of a certain type of data being stored in a single database. Sensitivity can refer to the level of data privacy or security, such as marking fields containing personally identifiable information as highly sensitive. Activity can refer to the frequency of data updates or accesses, such as a table being updated daily and accessed by multiple systems. Standardization can be the process of converting parameters with different dimensions to a uniform scale, such as using Min-Max standardization or Z-score standardization. Technical operations can integrate correlation parameters with other feature parameters to form a data correlation vector. For example, concentration is calculated by the distribution entropy value of data storage, sensitivity is automatically labeled by field classification rules, and activity is determined by the number of accesses in statistical logs. Standardization methods are then used to eliminate differences in dimensions and ensure a balanced weighting of each dimension. For example, the vector for a data asset could be concentration 0.8, sensitivity 0.6, and activity 0.9. This process makes the data feature profile more accurately reflect the multi-dimensional characteristics of the data, providing a quantitative basis for subsequent catalog template matching.

[0084] This embodiment provides a multi-source fusion-based enterprise data asset catalog management method. By acquiring and fusing structural feature parameters of heterogeneous data from multiple sources, it constructs a data domain model and lineage mapping to quantify the strength of relationships between entities. Finally, through standardized processing of multi-dimensional data association vectors, it generates feature profiles, achieving the following technical effects: Explicit modeling of structure and lineage transforms implicit business logic and data dependencies into computable parameters, reducing omissions or errors from manual sorting; quantitative analysis based on multi-dimensional indicators such as concentration, sensitivity, and activity comprehensively considers the technical and business attributes of the data, avoiding the limitations of single-dimensional decision-making; standardized processing ensures that feature vectors of different data assets are compared on a unified scale; for example, highly sensitive data, even with low activity, can be prioritized for inclusion in access control design; and dynamically adapting association parameters allows the node hierarchy or permission division of catalog templates to be adjusted according to the strength of entity associations, for example, merging strongly associated entities into the same catalog branch to improve retrieval efficiency. In summary, this method significantly improves the precision and practicality of data feature profiling through structured modeling and multi-dimensional quantification, providing a reliable data foundation for catalog template matching and dynamic optimization, and enhancing the intelligence and adaptability of the overall solution.

[0085] In one embodiment, the directory structure template is called from a preset directory template library through a directory building model, including:

[0086] Obtain the template features corresponding to each template in the directory template library, and calculate the matching degree between each template feature and the data feature profile through the semantic relationships preset in the directory construction model;

[0087] When the matching degree exceeds the preset threshold, the preset template optimization rules are invoked to adapt and optimize the current template, and the directory structure template corresponding to the optimized template is generated.

[0088] Template features can be a set of attribute parameters of the directory template itself, used to characterize the template's structural characteristics and business constraints. These can be obtained through parametric modeling of pre-stored templates in the template library. For example, template features may include quantitative indicators such as structural complexity, business relevance, and permission configuration rules. For instance, the template features of a tree-structured template might include parameters such as "maximum level is 5" and "supports departmental-level permission isolation." Data feature profiles can be a set of attribute parameters of a dataset, used to describe the dynamic characteristics and business requirements of the data. These can be obtained through data element analysis or business rule extraction. For example, data feature profiles might include parameters such as data dynamic change rate, business sensitivity level, and multi-system interaction frequency. For instance, the feature profile of a dataset might include descriptions such as "daily update frequency exceeds the threshold" and "sensitive data accounts for more than 30%." Matching degree can be a quantitative indicator of the similarity between template features and data feature profiles, generated through the semantic relationship calculation module built into the directory construction model. For example, this can be achieved through weighted similarity calculation, such as combining structural similarity weight (0.4), permission matching weight (0.3), and business association weight (0.3) for weighted summation, or by using a classification model to classify and score the adaptability of the template and data features.

[0089] Template optimization rules can be a set of pre-defined executable strategies used to dynamically adjust the template structure to meet data feature requirements. These rules can be generated through a business rule engine or a machine learning model. For example, rules may include "add cache nodes if the data update frequency is higher than a threshold" or "force the insertion of permission verification nodes if sensitive data accounts for more than 30%". For instance, when data features show high business relevance but the original template design is a flat structure, the rules may require expanding the node hierarchy to a mesh topology; if data permission requirements are strict but the template is open by default, the rules may insert intermediate permission control nodes.

[0090] The above technical operation involves traversing all templates in the catalog template library, extracting template feature parameters for each template, such as structural complexity and supported permission types. Then, the template features are matched against key parameters in the data feature profile (such as data dynamic change rate and business sensitivity level) in a multi-dimensional manner. When the calculated matching degree exceeds a preset threshold (e.g., 0.8), the system automatically triggers the corresponding rule in the template optimization rule base to dynamically adjust the template structure. The optimized template structure needs to be re-verified for its compatibility with data features to ensure that the threshold requirements are still met after adjustment. This process, through a dual mechanism of quantitative matching and rule-driven approach, solves the problems of reliance on experience and limited adjustment scope in traditional manual optimization. For example, in the past, a template might have been adopted directly without discovering hidden permission defects, while the newly added threshold filtering and rule triggering can identify and correct such problems in advance, improving the accuracy of template adaptation.

[0091] This embodiment provides a multi-source fusion-based enterprise data asset catalog management method. It filters candidate templates by acquiring template features and calculating their matching degree with data feature profiles. Combined with preset thresholds to trigger template optimization rules, the structure is dynamically adjusted, ultimately generating an adapted catalog structure template. The quantitative parameters of template features and the multi-dimensional matching mechanism of data feature profiles avoid template selection bias caused by subjective judgment. Automated optimization actions driven by the rule engine, such as adding / deleting nodes and adjusting permission configurations, reduce the time cost of manual modifications. The dual control of threshold filtering and rule triggering can identify and correct initial template design defects in advance, reducing the risk of an unreasonable catalog structure. Through the above collaborative operation, this solution enhances the intelligent decision-making capability of the catalog construction model, accurately matching current data features while responding to potential future changes through preset rules, significantly reducing the need for manual intervention and improving construction efficiency.

[0092] In one embodiment, the process of simulating the directory construction process corresponding to the data assets based on the directory structure template includes:

[0093] A virtual directory topology model is constructed based on the node hierarchy of the directory structure template, and a dependency resolution model is obtained by resolving the dependency relationship of the virtual directory topology model through a directed acyclic graph.

[0094] The virtual directory topology model can be a mathematical model that abstractly represents the hierarchy and logical relationships between directory nodes through a graph structure. It can be obtained by mapping the node hierarchy of the directory template to graph nodes and defining the edge relationships between nodes. For example, the virtual directory topology model can include topological structures such as parent-child node membership relationships or cross-system data dependency paths. The directed acyclic graph (DAG) can be a graph structure used to describe unidirectional dependencies between data assets. It ensures the determinism of data flow through edge relationships that do not allow loops. For example, a DAG can include directed edges representing data sources or transformation logic, such as an edge relationship where a table field depends on the output of another system's API. The dependency resolution model can be a mathematical expression describing the dependency logic between nodes. For example, the dependency resolution model can include mathematical relationships such as the scope of data update impact determined by topological sorting or depth-first search algorithms. Technical operations can be achieved by mapping the node hierarchy of the directory template to graph nodes and defining the dependencies between nodes through the edges of the DAG. For example, if the data of node A needs to be processed through node B, a directed edge from A to B is added. Then, the critical path and dependent links are identified by using topological sorting or depth-first search algorithms. This can solve the problems of low efficiency and easy omission in traditional dependency analysis by manual sorting and ensure that the simulation process covers all potential dependencies.

[0095] An entity relationship equation is established using a graph embedding algorithm, and the hierarchical relationship data and permission mapping data of the dependency parsing model are calculated by the entity relationship equation and the path reasoning engine.

[0096] The graph embedding algorithm can be a technique that transforms a graph structure into a low-dimensional vector representation. For example, graph embedding algorithms can include methods such as Node2Vec or DeepWalk to capture topological features and semantic relationships between nodes. The entity relationship equation can be a mathematical expression constructed based on the graph embedding result. For example, entity relationship equations can include a set of equations defining the permission transmission logic between parent and child nodes through matrix operations. The path reasoning engine can be a system based on rule-based or logical reasoning. For example, a path reasoning engine can include a reasoning tool that uses Prolog or Drools engines to derive paths between nodes. The technical operation can use graph embedding algorithms to transform nodes and edges in the virtual directory topology into vector representations, and then construct entity relationship equations based on vector operations or rule definitions, such as quantifying hierarchical membership relationships or permission inheritance rules between nodes. Finally, the path reasoning engine and the DAG structure are combined to calculate all possible path combinations, such as the impact range of a node's permission change on downstream nodes, thereby achieving automated derivation of dependencies and permission logic, avoiding subjective biases from manual definitions.

[0097] By using graph neural networks to extract features from hierarchical relationship data and permission mapping data, the feature constraint relationship between hierarchical relationship data and permission mapping data is obtained.

[0098] Graph Neural Networks (GNNs) can be deep learning models specifically designed for processing graph-structured data. For example, GNNs may include models like GraphSAGE or GAT, used to capture features of multi-hop relationships between nodes. Feature constraints can be mathematical expressions describing the implicit association rules between hierarchical and permission data. For example, feature constraints may include constraints such as "the permission convergence rate must be inversely proportional to the hierarchical depth." Technical operations can use GNNs to perform joint feature extraction on hierarchical relationship data (such as node depth and parent-child relationships) and permission mapping data (such as role access rules). For example, attention mechanisms or message passing algorithms can be used to identify the constraints between the two. For instance, if a node's hierarchy is too deep but the permission convergence rate is insufficient, it can be marked as a risk point. This data-driven approach discovers implicit constraints that are difficult to detect using traditional methods, improving the accuracy of simulation results.

[0099] Based on feature constraints, real-time metadata in the data management system is processed to obtain structural evolution data and permission change data of data assets during the catalog construction process;

[0100] Real-time metadata can be dynamically updated data asset metadata. For example, real-time metadata may include dynamic information such as real-time field changes in database tables or adjustments to permission policies. Structure evolution data can be records describing the dynamic process of directory structure changes with metadata. For example, structure evolution data may include the trajectory of structural changes such as hierarchical expansion caused by adding nodes. Permission change data can be data recording the trajectory of permission configuration adjustments as business needs change. For example, permission change data may include configuration change records such as the shrinking or expansion of a role's permission scope. Technical operations can collect metadata change events in the data management system in real time, such as field additions or permission policy updates. Combined with feature constraints, these changes can be simulated using a rule engine or stream processing framework to simulate their impact on the directory structure. For example, when a table field is marked as sensitive data, permission convergence rules can be automatically generated and node isolation operations can be triggered, thereby achieving real-time dynamic environment directory structure simulation and ensuring it is synchronized with business needs.

[0101] By fusing structural evolution data and permission change data through Bayesian networks, the actual lineage links and actual permission distributions can be obtained.

[0102] In this context, Bayesian networks can be inference tools based on probabilistic graphical models. For example, a Bayesian network can describe the dependencies between variables using conditional probability tables (CPTs), such as the joint probability distribution of structural evolution data and permission change data. The actual lineage link can be a complete record of the data source and transformation path obtained from simulation. For example, the actual lineage link can include the complete path of data from the source system to the directory node. The actual permission distribution can be the permission configuration result generated after simulation. For example, the actual permission distribution can include configuration information such as access control lists for each node. Technical operations can use structural evolution data (such as the frequency of node additions and deletions) and permission change data (such as the number of role permission adjustments) as input variables, and utilize Bayesian networks for probabilistic inference to calculate the actual lineage link and actual permission distribution. For example, probabilistic analysis can be used to verify whether the negative correlation between the probability of permission changes for a certain node and the length of the lineage link conforms to security specifications, thereby achieving joint verification of structural and permission configurations.

[0103] The rationality of the directory structure template is verified based on the actual lineage, actual permission distribution, hierarchical relationship data, and permission mapping data.

[0104] The verification process is completed by comparing the simulation results with preset rationality indicators (such as permission convergence rate and lineage integrity). For example, if the permission distribution obtained from the simulation meets the principle of least privilege and business isolation requirements, the directory template is deemed reasonable.

[0105] This embodiment provides a multi-source fusion-based enterprise data asset catalog management method. It constructs a virtual catalog topology model and parses its dependencies to accurately cover all potential paths. It employs graph embedding algorithms and a path reasoning engine to automatically deduce hierarchy and permission logic, utilizes graph neural networks to discover implicit constraints, and combines real-time metadata processing to simulate structural evolution and permission changes in a dynamic environment. Finally, it uses Bayesian networks to fuse multi-dimensional data and quantify the verification results. This method achieves the technical effects of improving the automation and accuracy of catalog construction, enhancing dynamic adaptability, expanding verification dimensions, and reducing trial-and-error costs. Through multi-technology collaboration, this method upgrades traditional offline verification to an intelligent simulation engine with deep analysis and risk prediction capabilities, significantly improving the reliability and foresight of catalog construction.

[0106] In one embodiment, the directory structure template is optimized according to the construction process, and an enterprise data asset catalog is generated based on the optimized directory structure template, including:

[0107] When the directory structure template is verified to be reasonable, an information entropy-based evaluator is used to calculate the structure evaluation parameters and permission evaluation parameters corresponding to the directory structure template, and a sequence of templates to be optimized is generated based on the comparison results of the structure evaluation parameters and permission evaluation parameters with preset thresholds.

[0108] The information entropy-based evaluator can be a computational model that measures the uncertainty of directory structure parameters using entropy values ​​from information theory. This can be obtained through statistical analysis of node hierarchy distribution or permission configuration complexity. For example, an information entropy-based evaluator could include a node hierarchy distribution uniformity evaluation model or a permission configuration dispersion calculation module. The structure evaluation parameters can be a set of indicators that quantify the rationality of the directory structure. These can be obtained through topology analysis or business rule verification. For example, structure evaluation parameters could include the maximum directory hierarchy depth, node redundancy, and path length. The permission evaluation parameters can be quantitative indicators that assess the security of permission configuration. These can be obtained through role-based access conflict detection or permission convergence efficiency analysis. For example, permission evaluation parameters could include permission convergence rate and role-based access conflict probability. The preset threshold can be an optimization benchmark value set according to enterprise business needs or industry standards. This can be determined through historical data statistics or expert experience. For example, the preset threshold could include specific values ​​such as "maximum allowed hierarchy depth ≤ 5" or "permission convergence rate ≥ 0.8". By comparing the differences between the structural evaluation parameters and the permission evaluation parameters and preset thresholds, for example, unqualified template parameters can be marked as optimization requirements, thus forming a sequence of templates to be optimized. This process uses quantitative indicators to screen optimization targets, avoiding the blindness of manual adjustments and accurately identifying template defects that need to be prioritized.

[0109] A random forest regressor is trained based on the template sequence to be optimized. The random forest regressor predicts the amount of template optimization and the node parameters of the directory structure template are adjusted based on the amount of template optimization to obtain the node relationship coordinates of the updated directory structure template.

[0110] The random forest regressor can be a prediction model based on ensemble learning, trained using historical optimization datasets. For example, historical optimization data can include multi-dimensional data such as parameter adjustment records and effect feedback. Template optimization quantity can be a numerical indicator quantifying the magnitude of parameter adjustments, determined through regression prediction or parameter sensitivity analysis. For example, template optimization quantity can include specific values ​​such as the reduction in hierarchy depth and the number of redundant nodes inserted. Node parameters can be a set of numerical parameters describing the attributes of nodes in the topological space, defined through node relationship coordinate mapping or business rule encoding. For example, node parameters can include parameters such as hierarchy depth, permission level identifier, and business relevance weight. Node relationship coordinates can be a multi-dimensional numerical representation describing the topological location and connectivity of nodes, generated through parameter combination or spatial mapping algorithms. For example, node relationship coordinates can be numerically expressed using a three-dimensional coordinate system (hierarchical depth, permission level, and business relevance). By training the random forest regressor to predict optimization quantities and adjust node parameters, for example, the hierarchy depth parameter of a third-level node can be adjusted from 3.0 to 2.5, thereby changing its hierarchy dimension value in the node relationship coordinates. This process uses machine learning models to replace manual trial and error, significantly improving the accuracy and efficiency of parameter adjustment.

[0111] The node relationship coordinates of the updated directory structure template are reconstructed using the minimum spanning tree algorithm, and the adjustment planning path is obtained by dynamically adjusting the planning based on the minimum spanning tree algorithm under the constraints of maximum level depth and permission convergence rate.

[0112] The minimum spanning tree algorithm can be an optimization algorithm for constructing low-complexity topologies. Topology reconstruction can be achieved through edge weight calculation or node merging strategies. For example, the minimum spanning tree algorithm can be an improved version of Kruskal's algorithm or Prim's algorithm. The maximum level depth constraint can be a topology rule that limits the upper limit of directory levels. This can be implemented through level depth threshold verification or path length restrictions. For example, the maximum level depth constraint can be set to "the directory level must not exceed 5 levels." The permission convergence rate constraint can be a topology rule that controls the efficiency of permission configuration. This can be achieved through indicators such as role permission aggregation efficiency or access path compression rate. For example, the permission convergence rate constraint can require "convergence from administrator permissions to ordinary users within three levels." The dynamic adjustment plan can be a set of topology optimization paths that satisfy multiple constraints. This can be generated using strategies such as greedy algorithms or branch and bound methods. For example, the dynamic adjustment plan can include operation sequences such as node merging order and level adjustment priority. The minimum spanning tree algorithm generates adjustment plan paths under constraints. For example, redundant nodes can be merged first rather than adding new levels to avoid exceeding the maximum level depth threshold. This process seeks the optimal structural adjustment scheme while ensuring the constraints are met, effectively balancing structural simplicity and functional integrity.

[0113] To adjust the planning path, calculate the centrality value at each node, insert redundant nodes in the centrality mutation region using the community detection algorithm, obtain the optimized directory structure template, and generate the enterprise data asset catalog based on the optimized directory structure template.

[0114] The centrality value can be a quantitative indicator measuring the importance of a node in the network topology. It can be determined through methods such as betweenness centrality calculation or degree centrality analysis. For example, the centrality value can include the frequency of path traversal or the number of direct connections of a node. The community detection algorithm can be an analytical tool for identifying the community structure of the topology. It can be implemented through algorithms such as the Louvain algorithm or label propagation. For example, the community detection algorithm can identify groups of financial data nodes with close business connections. Redundant nodes can be backup nodes that enhance structural resilience. They can be generated through preset parameter templates or adaptive configuration strategies. For example, redundant nodes can be configured with the same permission inheritance rules as the original nodes but have independent business labels. By calculating centrality mutation regions and inserting redundant nodes, for example, backup nodes can be added next to critical paths to share traffic pressure. This process accurately locates areas that need enhanced scalability through topology analysis and community mining, avoiding the resource waste caused by global redundancy and improving the resilience of the directory structure to cope with future data growth.

[0115] This embodiment achieves the following technical effects by employing techniques such as using an information entropy evaluator to screen optimization objects, a random forest regressor to predict parameter adjustments, a minimum spanning tree algorithm to generate constraint-satisfying paths, and a community detection algorithm to locate redundant node insertion regions: It transforms abstract structural rationality into calculable quantitative indicators, significantly improving the objectivity of the optimization process; it uses machine learning models to replace manual trial and error, drastically shortening the parameter adjustment cycle; it generates feasible structural adjustment schemes under multiple constraints, avoiding the failure of traditional methods due to neglecting constraints; it precisely enhances the scalability of key areas through topology analysis, reducing the reconstruction pressure brought by future data growth; and it outputs the final generated directory structure in a standardized format and integrates it into business systems, ensuring seamless integration between the technical solution and actual needs. The synergistic effect of these techniques upgrades directory optimization from local adjustments to global planning, significantly improving the compliance, scalability, and long-term applicability of the generated directory.

[0116] In one embodiment, before acquiring metadata of multi-source heterogeneous data assets within an enterprise, the method further includes: acquiring environmental parameters of the enterprise's IT architecture to construct a data governance dynamics model corresponding to the IT architecture and performing an impact analysis on the data governance dynamics model to obtain an architecture dependency cloud map and a permission distribution cloud map corresponding to each of the preset initial directory templates for the data assets.

[0117] Among them, the data governance dynamics model can be a mathematical model based on system dynamics or complex systems theory, used to simulate the impact of IT architecture parameters on the data governance process. For example, it can describe the dynamic impact of architecture changes on data dependencies through differential equations. This model can be constructed by obtaining environmental parameters of the enterprise IT architecture through API calls or configuration file parsing. An architecture dependency cloud map can be a visualization tool that uses a graph structure to represent the dependencies between components or data entities in the enterprise IT architecture. Nodes represent system or data assets, and edge weights reflect the strength of dependencies or coupling. A permission distribution cloud map can be a visualization tool that shows the distribution patterns of permissions among architecture nodes in the form of probability density distribution or heatmaps. For example, it can use color gradients to represent the sensitivity level or access permission level of different nodes.

[0118] By constructing a data governance dynamics model and inputting environmental parameters for simulation, an architecture dependency cloud and a permission distribution cloud can be generated. In a specific embodiment, this process is achieved by simulating the evolution of dependencies and permission diffusion paths under different scenarios. For example, the model may simulate the impact of a new system connection on existing data dependency chains, or the impact of permission delegation on access control. This operation, by quantifying the correlation between architecture parameters and data governance, provides a dynamic environmental benchmark for subsequent template selection, thereby improving the environmental adaptability of directory construction.

[0119] Extract the nodes of each initial directory template in the architecture dependency cloud and the permission gradient in the permission distribution cloud, and calculate the evolution rate and permission change rate of each initial directory template at each node to obtain the architecture dependency change curve of the data asset in the construction process under different initial directory templates.

[0120] In this context, nodes can be elements representing system or data assets in the architecture dependency cloud, and their set is obtained by parsing the cloud structure. Permission gradients can be parameters that quantify the gradient of permission distribution among nodes in the cloud, obtained by calculating the difference in permission levels between adjacent nodes or entropy values. Evolution rate can be an indicator measuring how quickly a directory template adapts to architectural changes, such as the number of node relationship adjustments per unit time or the efficiency of dependency path reconstruction. Permission change rate can be an indicator describing the sensitivity of permission configuration to changes in architectural parameters, such as the impact of permission level adjustments on data access permissions.

[0121] By extracting node sets and permission gradients, and combining them with time series analysis or differential calculations, the evolution rate and permission change rate of templates under architectural change scenarios can be quantified. In a specific embodiment, if a template needs to frequently adjust its node hierarchy when the system expands, its evolution rate is low; if the permission gradient changes drastically, its permission change rate is high. By plotting architectural dependency change curves, with the horizontal axis representing time / scenario and the vertical axis representing evolution rate, the adaptability of different templates can be intuitively compared, thus providing a quantitative basis for subsequent selection.

[0122] Based on the architecture dependency change curve, select multiple first directory templates from several initial directory templates;

[0123] This operation analyzes the architecture dependency change curves and selects initial templates with evolution rates higher than a threshold and stable permission change rates as the first directory template. For example, selecting a template that can maintain a high evolution rate and stable permission convergence even when architecture parameters fluctuate ensures better dynamic adaptability in practical applications.

[0124] The first parameter in each first directory template is evolved according to the preset template evolution algorithm to obtain several evolved template combinations and save them to the preset directory template library.

[0125] The template evolution algorithm can be a parameter optimization algorithm based on genetic algorithms, particle swarm optimization, or neuroevolution, which improves adaptability by iteratively optimizing template parameters. The first parameter can be the template's structural and permission parameters, encoded as a numerical vector. Template combinations can be multiple optimized template variants generated by the evolutionary algorithm, containing different topologies or parameter configurations. Through parameter encoding, defining multi-objective optimization functions, and evolutionary operations such as crossover and mutation, template combinations adapted to different architectural scenarios can be generated. In a specific embodiment, after the evolved template combination's performance in the dynamic model is verified through simulation, it is saved to a catalog template library as a candidate template pool for subsequent steps, thereby improving the quality and scalability of the template library.

[0126] This embodiment quantifies the correlation between templates and architecture by constructing a data governance dynamics model and generating architecture dependency clouds and permission distribution clouds. It extracts node and permission gradients and calculates evolution rates and permission change rates to form a dynamic adaptability evaluation benchmark. Combined with screening and evolutionary algorithms, it generates optimized template combinations and stores them in a template library. This achieves the following technical effects: First, the template combinations generated by the evolutionary algorithm are dynamically verified, reducing adjustment costs in subsequent steps due to template mismatches with the actual architecture. Second, the introduction of architecture dependency clouds and permission distribution clouds shifts template screening from experience-driven to data-driven, improving the scientific nature of template selection. Finally, by simulating the impact of architecture changes through a dynamics model, high-risk templates are identified in advance and parameters are optimized, avoiding directory failures caused by architecture changes. Furthermore, the evolved template combinations contain multiple variants adapted to different architecture scenarios, enhancing the scalability of directory construction and compatibility with future architecture evolution.

[0127] In one embodiment, environmental parameters include system coupling parameters and data flow strategies. The environmental parameters of the enterprise IT architecture are obtained, a data governance dynamics model corresponding to the IT architecture is constructed, and an impact analysis is performed on the data governance dynamics model. This yields an architecture dependency cloud map and a permission distribution cloud map corresponding to each data asset under a preset initial directory template, including:

[0128] Obtain the system coupling parameters and data flow strategies of the enterprise IT architecture to calculate the governance impedance matrix corresponding to the IT architecture. The system coupling parameters include service call depth and dependency density, and the governance impedance matrix is ​​determined by the data flow strategy, service call depth, and dependency density.

[0129] System coupling parameters can be indicators that quantify the complexity of IT architecture, including service call depth and dependency density, which can be obtained through API scanning tools or dependency graph analysis. Data flow policies can be a set of rules defined by the enterprise, including data sharing scope and transmission path restrictions; for example, their content may include cross-departmental data access permissions or data transmission encryption requirements. A governance impedance matrix can be a mathematical structure describing the governance resistance between nodes, with its element values ​​calculated using formulas such as: matrix element value = data flow policy weight × (service call depth + dependency density coefficient), where the data flow policy weight is assigned different coefficients based on the sharing scope. In a specific embodiment, rules in the data flow policy document can be extracted through configuration parsing, and service call depth can be statistically analyzed using call chain analysis tools, ultimately generating a governance impedance matrix through matrix operations.

[0130] The impact values ​​of metadata nodes are obtained by governing the impedance matrix and mapped to the topology nodes to establish a graph diffusion equation. The topology nodes are partitioned into a directed graph and the node density in the core business area is increased to twice the basic density.

[0131] Metadata node impact values ​​can be numerical values ​​characterizing the criticality of a node in the IT architecture, calculated through matrix multiplication. Topology nodes can be logical units in a directed graph, and their density adjustment strategy can be to double the number or weight of nodes in core business areas. The graph diffusion equation can be a mathematical model describing the propagation law of metadata impact. In a specific embodiment, the initial impact value of core business nodes is set to a high value, and by adjusting the diffusion coefficient and the weight of the source item, the density of nodes in critical areas is increased to twice the base value, thereby enhancing their performance in the model.

[0132] The node centrality value is calculated based on the graph diffusion equation to obtain the influence field calculation equation, i.e., the data governance dynamics model; where the node centrality value is determined by the in-degree vector and the out-degree vector.

[0133] Node centrality can be a metric for measuring a node's importance in a network; for example, it can be calculated as a weighted sum of the in-degree and out-degree vectors. The influence field calculation equation can be a mathematical expression integrating the diffusion equation and node centrality, such as substituting the node centrality value into the diffusion coefficient or source term. In one specific embodiment, regions with high node centrality may accelerate the influence diffusion rate, while regions with high governance resistance values ​​suppress propagation by reducing the diffusion coefficient, ultimately forming a set of equations describing the dynamic distribution of metadata.

[0134] A set of nonlinear metadata evolution equations is established based on the influence field calculation equations to obtain the architecture dependency cloud map and permission distribution cloud map corresponding to each preset initial directory template of the data asset; among them, the nonlinear metadata evolution equations are solved by gradient descent until the loss function is less than the preset value.

[0135] The nonlinear metadata evolution equation set can be a complex set of equations containing polynomial or exponential terms. Gradient descent can be an iterative optimization algorithm that adjusts parameters by calculating the gradient direction of the loss function. In a specific embodiment, after initializing the node influence value distribution, multiple iterations are performed to bring the loss function below a preset threshold, ultimately generating an architecture dependency cloud map and a permission distribution cloud map.

[0136] This embodiment constructs a governance impedance matrix by quantifying system coupling parameters and data flow strategies. It then combines graph diffusion equations and node centrality values ​​to form an influence field calculation equation. Finally, it generates a cloud map by solving a set of nonlinear evolution equations and using gradient descent. This approach enhances the model's ability to quantitatively analyze dynamic changes in IT architecture, strengthens the influence weight of core business nodes, supports long-term governance strategy simulation and prediction, and reduces the computational complexity of large-scale architectures. This method transforms abstract architectural parameters into a computable model through mathematical modeling, reducing bias from human assumptions. Simultaneously, it achieves accurate environmental adaptation benchmark construction through density adjustment and efficient solution strategies.

[0137] In one embodiment, the evolution rate and permission change rate of each initial directory template at each node are calculated to obtain the architecture dependency change curve of the data asset during the construction process under different initial directory templates, including:

[0138] Obtain the template topology corresponding to each initial directory template, and obtain the adjacency dataset of the nodes on the template topology; where different initial directory templates correspond to different topology shapes, topology depths and topology connectivity;

[0139] A template topology can be a mathematical model describing the structural form of a directory template, and its definition includes, but is not limited to, topological shape, topological depth, and topological connectivity. For example, a tree topology has a hierarchical parent-child node relationship, while a mesh topology allows for multi-path connections between nodes. A template topology can be obtained by parsing the structure definition file of the directory template, specifically by recursively traversing the directory hierarchy or reading predefined topology parameter configurations. An adjacency dataset can be a collection that quantifies the direct connections between nodes, typically stored in the form of an adjacency matrix or adjacency list, used to characterize the strength of dependencies or communication frequency between nodes. In one specific embodiment, the adjacency dataset can be generated by traversing the node relationships of the template topology, for example, recording the direct child nodes or associated node identifiers of each node.

[0140] The permission distribution value of nodes is calculated using a page ranking algorithm; the permission distribution value is obtained by calculating the mapping relationship between node importance and permissions.

[0141] Page ranking algorithms can be iterative computation methods based on graph structures to evaluate the importance score of nodes in the topology. This algorithm distributes node scores to neighboring nodes according to preset probabilities and converges after multiple iterations, ultimately generating a numerical result reflecting node centrality. The permission mapping relationship can be a rule that associates node importance scores with actual permission levels, such as classifying permission levels based on score percentiles. In a specific embodiment, the calculation of permission distribution values ​​includes: initializing the initial scores of all nodes, constructing a graph structure based on the adjacency dataset, performing iterative calculations until convergence, and finally mapping the scores to specific permission levels in conjunction with business rules.

[0142] A data evolution dynamics equation is established based on the permission distribution value, and the Runge-Kutta method is used to solve for the permission change rate value of the node; wherein, the permission change rate value is obtained by calculating the derivative of permission with respect to time;

[0143] Data evolution dynamics equations can be differential equations describing the changes in permission distribution over time or architectural parameters, such as expressing the relationship between the derivative of the permission vector with respect to time and changes in adjacent nodes. The Runge-Kutta method can be a numerical algorithm for solving such equations, improving computational accuracy through multi-order derivative estimation. The Runge-Kutta method iteratively calculates each time step using four-order iterations, ultimately generating a node permission change rate curve. The permission change rate value quantifies the sensitivity of permission configuration to architectural changes, such as the instantaneous rate of change of a node's permission value over time.

[0144] The state space equation is constructed based on the node permission change rate and the dependency on the architecture dependency cloud, and the dependency time function is obtained by Kalman filtering. The architecture dependency change curve is obtained based on the dependency time function.

[0145] State-space equations can be mathematical tools for modeling system states and observed variables as a system of equations. Kalman filtering can be a recursive algorithm that optimizes state estimates through prediction and update phases. In a specific embodiment, the construction of state-space equations includes combining the permission change rate value with dependencies in the architecture dependency cloud to form a state vector. Kalman filtering then predicts the state at the next time step and corrects the estimated value by combining it with actual observation data, ultimately outputting a dependency time function. This function can generate an architecture dependency change curve, visually demonstrating the decay or enhancement trend of dependencies over time.

[0146] This embodiment establishes a node connection model by parsing the topology of the initial directory template and generating an adjacency dataset. It utilizes a page ranking algorithm combined with permission mapping rules to achieve dynamic permission distribution evaluation. Based on dynamic equations and numerical algorithms, it quantifies the permission change rate. Finally, it generates a dependency change curve by fusing observation data through state-space modeling and Kalman filtering. This achieves the following technical effects: improving the accuracy and robustness of architecture dependency change prediction through dynamic modeling and advanced algorithms; enhancing the system's adaptability to complex structures by being compatible with directory templates of different topological shapes; quantifying the sensitivity of node permissions to architecture changes, supporting adaptive adjustments to the data asset directory; and clearly characterizing the evolution trend of dependency relationships through dependency time functions, assisting in optimizing directory template selection to reduce failure risks.

[0147] In one embodiment, multiple first directory templates are selected from several initial directory templates based on the architecture dependency change curve. This includes: determining the current architecture dependency of each initial directory template based on the architecture dependency change curve; constructing constraints based on the preset maximum dependency of the data asset and the current architecture dependency of each initial directory template; calculating the dependency constraint optimization parameters using the Lagrange multiplier method; establishing a multi-objective optimization function based on the dependency objective function and the permission objective function based on the dependency constraint optimization parameters; obtaining the mapping relationship between the template vector and the constraint response vector corresponding to each initial directory template using a support vector machine with a polynomial kernel function; constructing a system of constraint optimization equations based on the mapping relationship using the Karouch-Kuntuak conditions; solving the optimal directory template and dual variables corresponding to each initial directory template using the interior point method; and storing the optimal directory template as the first directory template if the norm difference between the optimal directory template and the initial template corresponding to the initial directory template is less than a preset convergence threshold.

[0148] The architecture dependency change curve can be a quantitative model describing the dynamic changes in the dependency relationship between the data asset catalog template and the enterprise architecture. It can be generated by analyzing parameters such as the number of dependencies between nodes or the total weight. For example, this curve can reflect the changing trend of the coupling degree between architecture components under different template configurations. The Lagrange multiplier method can be a mathematical method for handling constrained optimization problems. By introducing Lagrange multipliers, the constraints are transformed into unconstrained problems. For example, it can use the difference between the current dependency amount of the architecture and the preset maximum dependency amount as the weight parameter of the constraint. The preset maximum dependency amount can be the upper limit of the dependency relationship that the enterprise architecture can bear. It is usually determined according to the system design specifications or historical operating data. For example, this value can include the maximum allowed number of dependencies between nodes or a weight threshold. The dependency objective function can be a mathematical expression for quantitatively evaluating the dependency risk of the catalog template. For example, it can be defined as a penalty function when the dependency amount exceeds the preset maximum value. The permission objective function can be an indicator function for measuring the convergence efficiency of template permissions. For example, it can include parameters such as the probability of permission conflicts or the convergence speed of role inheritance rules. Support vector machines with multinomial kernel functions can be nonlinear classification or regression models. They establish complex relationships by mapping low-dimensional feature spaces to high-dimensional spaces. For example, they can model the nonlinear correlation between template parameters and constraint response indicators. Template vectors can be numerical representations describing the characteristics of the template parameters. For example, they can include discrete or continuous variables such as hierarchy depth and inter-node dependency weights. Constraint response vectors can be a set of indicators quantifying the degree to which constraints are satisfied. For example, they can include dependency over-limit risk values ​​and permission conflict probability values. Karouch-Kuntuak conditions can be necessary conditions for the optimality of nonlinear programming problems, used to verify whether the solution satisfies the local optimality of constraint optimization. For example, they can check whether dependency constraints and permission constraints are simultaneously satisfied. Interior-point methods can be optimization algorithms that iteratively search for paths within the feasible region. For example, they can progressively adjust template parameters to approach the optimal solution while maintaining the continuous satisfaction of constraints. Norm difference can be a mathematical measure of the distance between two vectors in space. For example, it can be calculated using Euclidean distance or Mahalanobis distance, used to determine whether the optimization result has reached a stable state. The construction of a multi-objective optimization function can be achieved by linearly combining or weighting the dependency objective function with the permission objective function. For example, the weight coefficients can be dynamically adjusted according to business priorities. The mapping relationship establishment process of the support vector machine includes sub-steps such as kernel transformation of the feature space, parameter training, and model validation. For example, the order of the polynomial kernel function can be selected based on the cross-validation results. The construction of the constrained optimization equation system must include both equality constraints and inequality constraints. For example, equality constraints can define parameter boundary conditions, and inequality constraints can limit the dependency to no more than a preset threshold.The iterative solution process of the interior point method gradually approaches the optimal solution through the central path algorithm. For example, it can update parameters and adjust dual variables through Newton's method to maintain feasibility.

[0149] This embodiment quantifies the dependency characteristics of the initial directory template through architecture dependency change curves, transforms constraints into optimization parameters using the Lagrange multiplier method, establishes a multi-objective optimization model by combining the nonlinear mapping capability of support vector machines, and further employs the Karouch-Kuntucker condition and interior-point method to achieve efficient solution under constraints. Finally, the convergence of optimization is judged by norm difference, and the optimal directory template that meets the constraints is selected. This method accurately quantifies the matching degree between the template and the architecture through mathematical modeling, directly incorporates hard constraints such as architecture dependencies and permission convergence efficiency into the optimization process, and uses nonlinear mapping to capture complex relationships. Ultimately, while ensuring convergence speed and stability, it avoids the risk of structural mutation caused by over-optimization, thereby significantly improving the scientific nature and convergence efficiency of template selection and providing a reliable foundation for subsequent directory construction.

[0150] Furthermore, this embodiment of the invention also proposes a storage medium storing a multi-source fusion-based enterprise data asset catalog management program. When the multi-source fusion-based enterprise data asset catalog management program is executed by a processor, it implements the steps of the multi-source fusion-based enterprise data asset catalog management method described above.

[0151] In addition, refer to Figure 3 This invention also proposes a multi-source fusion-based enterprise data asset catalog management system, which includes:

[0152] The data profiling module 10 is used to acquire metadata of multi-source heterogeneous data assets within the enterprise, and to perform fusion analysis on the metadata to obtain data feature profiles.

[0153] Template matching module 20 is used to match the directory construction model corresponding to the data feature profile, so as to call the directory structure template from the preset directory template library through the directory construction model;

[0154] A simulation module 30 is constructed to simulate the directory construction process corresponding to the data asset based on the directory structure template.

[0155] The optimization generation module 40 is used to optimize the directory structure template according to the construction process, and generate an enterprise data asset catalog based on the optimized directory structure template.

[0156] Other embodiments or specific implementations of the enterprise data asset catalog management system based on multi-source fusion described in this invention can be referred to the above-mentioned method embodiments, and will not be repeated here.

[0157] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0158] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the module claims listing several systems, several of these systems may be specifically embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as names.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal user device (which may be a mobile phone, computer, server, air conditioner, or network user device, etc.) to execute the methods described in the various embodiments of the present invention.

[0160] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for managing enterprise data asset catalogs based on multi-source fusion, characterized in that, The method includes: Obtain metadata of multi-source heterogeneous data assets within the enterprise, and perform fusion analysis on the metadata to obtain data feature profiles; Match the directory construction model corresponding to the data feature profile, so as to call the directory structure template from the preset directory template library through the directory construction model; Simulate the directory construction process corresponding to the data asset based on the directory structure template; The directory structure template is optimized according to the construction process, and an enterprise data asset catalog is generated based on the optimized directory structure template. The process of simulating the directory construction process corresponding to the data asset based on the directory structure template includes: A virtual directory topology model is constructed based on the node hierarchy of the directory structure template, and a dependency resolution model is obtained by resolving the dependency relationship of the virtual directory topology model through a directed acyclic graph. An entity relationship equation is established using a graph embedding algorithm, and the hierarchical relationship data and permission mapping data of the dependency parsing model are calculated by the entity relationship equation and the path reasoning engine. The feature constraint relationship between the hierarchical relationship data and the permission mapping data is obtained by extracting features from the hierarchical relationship data and the permission mapping data through a graph neural network. The real-time metadata in the data management system is processed according to the aforementioned feature constraint relationship to obtain the structural evolution data and permission change data of the data assets during the catalog construction process; By fusing the structural evolution data and the permission change data through a Bayesian network, the actual lineage links and actual permission distributions are obtained. The rationality of the directory structure template is verified based on the actual lineage, the actual permission distribution, the hierarchical relationship data, and the permission mapping data.

2. The enterprise data asset catalog management method based on multi-source fusion as described in claim 1, characterized in that, The metadata includes structural feature parameters. The process of acquiring multi-source heterogeneous data asset metadata within the enterprise and performing fusion analysis on the metadata to obtain a data feature profile includes: Obtain the structural feature parameters of the data asset, wherein the structural feature parameters include data pattern parameters and lineage relationship parameters; Ontology modeling is used to extract core entities and relationships from the structural feature parameters, and a data domain model and lineage mapping are constructed based on the core entities and relationships to obtain the correlation parameters; The data association vector is obtained based on the association degree parameter, and the data association vector is standardized to obtain the data feature profile, wherein the data association vector includes concentration, sensitivity and activity.

3. The enterprise data asset catalog management method based on multi-source fusion as described in claim 1, characterized in that, The step of calling a directory structure template from a preset directory template library through the directory construction model includes: Obtain the template features corresponding to each template in the directory template library, and calculate the matching degree between each template feature and the data feature profile through the semantic relationship preset in the directory construction model; When the matching degree exceeds a preset threshold, the preset template optimization rules are invoked to adapt and optimize the current template, and a directory structure template corresponding to the optimized template is generated.

4. The enterprise data asset catalog management method based on multi-source fusion as described in claim 1, characterized in that, The step of optimizing the directory structure template according to the construction process and generating an enterprise data asset catalog based on the optimized directory structure template includes: When the directory structure template is verified to be reasonable, an information entropy-based evaluator is used to calculate the structure evaluation parameters and permission evaluation parameters corresponding to the directory structure template, and a template sequence to be optimized is generated based on the comparison results of the structure evaluation parameters and the permission evaluation parameters with preset thresholds respectively. A random forest regressor is trained based on the template sequence to be optimized. The random forest regressor is used to predict the template optimization amount. The node parameters of the directory structure template are adjusted based on the template optimization amount to obtain the node relationship coordinates of the updated directory structure template. The node relationship coordinates of the updated directory structure template are reconstructed using the minimum spanning tree algorithm, and the adjustment planning path is obtained by dynamically adjusting the planning based on the minimum spanning tree algorithm under the constraints of maximum level depth and permission convergence rate. The centrality value at each node is calculated for the adjusted planning path. Redundant nodes are inserted in the centrality mutation region through the community detection algorithm to obtain the optimized directory structure template. The enterprise data asset catalog is then generated based on the optimized directory structure template.

5. The enterprise data asset catalog management method based on multi-source fusion as described in claim 1, characterized in that, Before acquiring metadata of multi-source heterogeneous data assets within the enterprise, the process also includes: Obtain the environmental parameters of the enterprise IT architecture to construct a data governance dynamics model corresponding to the IT architecture and perform an impact analysis on the data governance dynamics model to obtain the architecture dependency cloud map and permission distribution cloud map corresponding to the data assets under each preset initial directory template. Extract the nodes of each initial directory template in the architecture dependency cloud and the permission gradient in the permission distribution cloud, and calculate the evolution rate and permission change rate of each initial directory template at each node to obtain the architecture dependency change curve of the data asset in the construction process under different initial directory templates. Based on the architecture dependency change curve, select multiple first directory templates from several initial directory templates; The first parameter in each of the first directory templates is evolved according to the preset template evolution algorithm to obtain several evolved template combinations and save them to the preset directory template library.

6. The enterprise data asset catalog management method based on multi-source fusion as described in claim 5, characterized in that, The environmental parameters include system coupling parameters and data flow strategies. The process involves obtaining the environmental parameters of the enterprise IT architecture, constructing a data governance dynamics model corresponding to the IT architecture, performing an impact analysis on the data governance dynamics model, and obtaining the architecture dependency cloud map and permission distribution cloud map corresponding to the data assets under each preset initial directory template. Obtain the system coupling parameters and data flow strategy of the enterprise IT architecture to calculate the governance impedance matrix corresponding to the IT architecture. The system coupling parameters include service call depth and dependency density, and the governance impedance matrix is ​​determined by the data flow strategy, service call depth and dependency density. The influence values ​​of metadata nodes are obtained through the governance impedance matrix and mapped to the topology nodes to establish a graph diffusion equation. The topology nodes are partitioned using a directed graph and the node density in the core business area is increased to twice the basic density. The node centrality value is calculated based on the graph diffusion equation to obtain the influence field calculation equation, i.e., the data governance dynamics model; wherein, the node centrality value is determined by the in-degree vector and the out-degree vector; Based on the influence field calculation equation, a set of nonlinear metadata evolution equations is established to obtain the architecture dependency cloud map and permission distribution cloud map corresponding to each preset initial directory template of the data asset; wherein, the set of nonlinear metadata evolution equations is solved by gradient descent until the loss function is less than a preset value.

7. The enterprise data asset catalog management method based on multi-source fusion as described in claim 5, characterized in that, The calculation of the evolution rate and permission change rate of each initial directory template at each node, to obtain the architecture dependency change curve of the data asset during the construction process under different initial directory templates, includes: Obtain the template topology corresponding to each initial directory template, and obtain the adjacency dataset of the nodes on the template topology; wherein, different initial directory templates correspond to different topology shapes, topology depths and topology connectivity; The permission distribution value of the node is calculated using a page ranking algorithm; wherein, the permission distribution value is calculated through the mapping relationship between node importance and permission. A data evolution dynamics equation is established based on the permission distribution value, and the Runge-Kutta method is used to solve for the permission change rate value of the node; wherein, the permission change rate value is obtained by calculating the derivative of permission with respect to time; The state space equation is constructed based on the permission change rate of the node and the dependency on the architecture dependency cloud, and the dependency time function is obtained by Kalman filtering. The architecture dependency change curve is obtained based on the dependency time function.

8. The enterprise data asset catalog management method based on multi-source fusion as described in claim 5, characterized in that, The step of selecting multiple first directory templates from several initial directory templates based on the architecture dependency change curve includes: The current architecture dependency of each initial directory template is determined by the architecture dependency change curve. Constraints are constructed based on the preset maximum dependency of the data asset and the current architecture dependency of each initial directory template. Dependency constraint optimization parameters are calculated by the Lagrange multiplier method. Based on the dependency constraint optimization parameters, a multi-objective optimization function is established for the dependency objective function and the permission objective function, and the mapping relationship between the template vector and the constraint response vector corresponding to each initial directory template is obtained through a support vector machine with a polynomial kernel function. Based on the mapping relationship, a set of constrained optimization equations is constructed using the Karouch-Kun-Tucker conditions, and the optimal directory template and dual variables corresponding to each initial directory template are obtained by solving the interior point method. If the norm difference between the optimal directory template and the initial template corresponding to the initial directory template is less than a preset convergence threshold, then the optimal directory template is stored as the first directory template.

9. A multi-source fusion-based enterprise data asset catalog management system, characterized in that, The enterprise data asset catalog management system based on multi-source fusion includes: The data profiling module is used to acquire metadata of multi-source heterogeneous data assets within the enterprise, and to perform fusion analysis on the metadata to obtain data feature profiles. The template matching module is used to match the directory construction model corresponding to the data feature profile, so as to call the directory structure template from the preset directory template library through the directory construction model; A simulation module is constructed to simulate the directory construction process corresponding to the data asset based on the directory structure template. An optimization generation module is used to optimize the directory structure template according to the construction process, and generate an enterprise data asset catalog based on the optimized directory structure template; The process of simulating the directory construction process corresponding to the data asset based on the directory structure template includes: A virtual directory topology model is constructed based on the node hierarchy of the directory structure template, and a dependency resolution model is obtained by resolving the dependency relationship of the virtual directory topology model through a directed acyclic graph. An entity relationship equation is established using a graph embedding algorithm, and the hierarchical relationship data and permission mapping data of the dependency parsing model are calculated by the entity relationship equation and the path reasoning engine. The feature constraint relationship between the hierarchical relationship data and the permission mapping data is obtained by extracting features from the hierarchical relationship data and the permission mapping data through a graph neural network. The real-time metadata in the data management system is processed according to the aforementioned feature constraint relationship to obtain the structural evolution data and permission change data of the data assets during the catalog construction process; By fusing the structural evolution data and the permission change data through a Bayesian network, the actual lineage links and actual permission distributions are obtained. The rationality of the directory structure template is verified based on the actual lineage, the actual permission distribution, the hierarchical relationship data, and the permission mapping data.

Images