Data asset management system and method based on data asset directory

Abstract

The invention provides a data asset management system and method based on a data asset catalogue, and the system comprises a data classification and structure management module which is used for dividing data into examples, examples and cases, and carrying out the standardized definition, index establishment and retrieval support of the storage forms of all kinds of data through a metadata management module; the data asset life cycle management module is used for recording and controlling the state of the data assets in each stage in the life cycle from development to commercialization in the process of converting the data into the data assets, and describing process information of various artificial products generated by the data assets in the life cycle and mutual relations of the process information through the metadata management module; and the data asset blood relationship management module is used for recording dependency relationships between data sources and data applications and between the data applications through a hierarchical management strategy and a data asset reference model between different management levels so as to form a traceable chain between the data assets.

Description

Technical Field

[0001] The present invention belongs to the technical field of data asset management, and particularly relates to a data asset management system and method based on a data asset catalog. Background Art

[0002] In the digital age, data has become the core asset driving social and economic development. The sharing and efficient utilization of data are regarded as the keys to enhancing enterprise competitiveness, promoting technological innovation, and facilitating industrial upgrading. However, with the explosive growth of data types and data volumes, how to achieve efficient and reliable data sharing and asset management while ensuring data security has become an urgent problem to be solved. Secure data sharing not only concerns the trust of data users and the rights and interests of data providers but also directly affects the compliance and security of data in various application scenarios. Reasonably designing and implementing effective data asset management methods is an important way to achieve secure and transparent data sharing among different participants, which can promote the maximum utilization and value release of data. Currently, in multiple fields such as healthcare, finance, transportation, and education, the demand for data sharing is increasing, and the complexity and security requirements of data asset management are also rising simultaneously. Therefore, how to ensure the security of data assets during sharing and the integrity and flexibility during management has become an important topic in the field of information technology.

[0003] However, the existing technologies still have significant deficiencies in realizing data utilization, life cycle management, and lineage tracing. First, the data utilization rate is low. In order to protect data security, current data asset management systems often adopt strict access control measures, but these measures often limit the flexible application of data in actual applications, resulting in the potential value of data assets not being fully released. Second, data asset management is insufficient. Many existing systems lack systematic full-life cycle management, and the life cycle stages of data assets from development, utilization to commercial use are not fully covered. This limitation makes it difficult to perform version control and change tracking of data assets and hard to ensure the reliability and manageability of data throughout the life cycle. Finally, it is difficult to effectively trace the data lineage while performing full-life cycle management. In the existing technologies for full-life cycle management, the tracing of data lineage often relies on the analysis and post-processing of metadata. This indirect method results in the inability to directly record the lineage during management, thus limiting the real-time performance and integrity of data tracing.

[0004] The existing data asset catalog technologies have deficiencies in some aspects.

[0005] First, the data manager needs to manually set the public scope of data assets and control access permissions to ensure data security and privacy. Although this setting method provides necessary security guarantees, it often increases the workload and complexity of the manager, resulting in low utilization of data resources. At the same time, in data sharing and access, overly strict or unreasonable permission settings may limit the circulation and application of data, failing to fully stimulate the potential value of data.

[0006] Second, the life cycle management of data assets depends on the recording of technical metadata, business metadata, and management metadata of artifacts throughout the life cycle. The system needs to continuously monitor and summarize information to obtain an overview of the life cycle. However, existing technologies have limitations in terms of automation level and analysis depth, and cannot present the complete life cycle information of data assets in real time and efficiently. This makes data managers face problems such as low efficiency and incomplete information when dealing with changes and historical tracking of data assets.

[0007] Third, the identification of data asset lineage depends on in-depth analysis and processing of metadata. Existing technologies usually obtain the lineage between assets through manual configuration and system analysis. However, due to the complex data chain, involving multiple levels and platforms, existing lineage analysis tools are insufficient in terms of accuracy and comprehensiveness. Users need more manual intervention to adjust and confirm the lineage, increasing the operation difficulty and the risk of errors. This technical limitation results in inefficient and unintuitive data asset management in data tracing and impact path tracking, affecting the quality of data governance and decision support.

[0008] Therefore, existing data asset catalog technologies have obvious deficiencies in data asset utilization rate, comprehensiveness of life cycle management, and accuracy and automation of lineage. Summary of the Invention

[0009] The object of the present invention is to solve the problems of low utilization rate of data assets, incomplete life cycle management, and inaccurate lineage analysis existing in the prior art, and a data asset management system and method based on a data asset catalog are proposed. First, through metadata and three types of "instances", "examples", and "cases", the classification and structured management of data are realized. Under this framework, data realizes a high degree of openness and sharing on the premise of ensuring security, thereby stimulating the innovation potential of users and significantly improving the utilization efficiency of data assets. Second, the full life cycle tracking and management of data assets from development to commercialization cover the process information and their interrelationships of various artifacts generated from the development to the commercial stage. This management method ensures that the changes and status of data assets at each stage are clearly grasped, thereby improving the efficiency and accuracy of data asset management. Finally, a hierarchical management strategy is adopted, and the reference relationships between different management levels (such as project portfolios, projects, versions, and version status, where versions are data applications that can exist independently in a project) are designed to construct an accurate data asset lineage, which can trace in detail the sources and dependencies of data and various artifacts generated during the development process, significantly enhancing the traceability and compliance of data assets.

[0010] To achieve the above object, the present invention adopts the following technical solutions.

[0011] The data classification and structure management module of the data asset management system based on a data asset catalog is used to classify data into instances, examples, and cases, and use the metadata management module to standardize the storage form of various types of data, establish indexes, and support retrieval;

[0012] The metadata module is used to standardize the description of data through metadata tables to ensure the consistency and identifiability of the basic information of data in all management stages;

[0013] The permission management module is used to perform authorization and access control during the data classification process, enabling users to access corresponding data according to the set access permission range;

[0014] The data asset life cycle management module is used to record and control the status of each stage in the life cycle of data assets from development to commercialization during the process of data being transformed into data assets, and through the metadata management module, describe the process information and their interrelationships of various artifacts generated by data assets during the life cycle;

[0015] The data asset lineage management module is used to record the dependency relationships between data sources and data applications, as well as between data applications through a hierarchical management strategy and a data asset reference model between different management levels, forming a traceable chain between data assets.

[0016] Furthermore, the metadata module defines the metadata structure of a single data item and the description of a single field.

[0017] Furthermore, the data classification and structure management module includes an instance indexing unit, which is used to establish an instance indexing mechanism. The instance indexing mechanism realizes the location and reference of data in different storage environments through an instance index table; the instance index table records the logical identifier, storage location, and its paths in different systems of the data.

[0018] Furthermore, the permission management module includes a review unit and a verification unit, where,

[0019] The review unit is used to review and record the permission granting and confirmation of a single artifact during its life cycle, record the details of a single access operation, and ensure the full traceability and strict access control of data assets during their life cycle;

[0020] The verification unit is used to verify and record the legal usage information of the lineage relationship between different data applications.

[0021] Furthermore, the data asset life cycle management module includes an analysis unit, which is used to analyze by combining the metadata in the data asset catalog with the system logs and change records, monitor the status changes of data assets in real time, analyze the change history of data assets, and form complete life cycle information.

[0022] Furthermore, the data asset life cycle management module includes a rights confirmation unit, which is used to record the rights confirmation information of a single data asset during the commercialization stage to ensure the legality and property rights of data assets during the commercialization process; the rights confirmation information of the data assets is recorded in the metadata table to ensure that the use and change of a single data asset comply with the preset legal and compliance requirements.

[0023] Furthermore, it also includes that the data asset lineage management module includes a layering unit, which is used to execute a hierarchical management strategy, organize data assets into project groups, projects, versions, and version statuses in sequence, and present the subordinate relationship and dependency relationship of data assets at different levels between the data source and data applications, as well as between data applications. The update, change, and application of data assets are monitored through lineage tracing.

[0024] To achieve the above object, the present invention also provides a data asset management method based on a data asset catalog, which applies the data asset management system based on a data asset catalog as described above, including:

[0025] The data classification and structure management module classifies data into instances, examples, and cases, and the metadata management module is used to standardize the storage forms of various types of data, establish indexes, and support retrieval; access control is performed through the permission management module to achieve authorization and confirmation of various types of data.

[0026] During the process of data being transformed into data assets, the data asset lifecycle management module establishes the process information and interrelationships of various artifacts generated in the lifecycle of data assets from development to commercialization. The permission management module checks and records various permission grants and confirmations for individual artifacts in the lifecycle, realizing full traceability and access control in the data asset lifecycle.

[0027] The data asset lineage management module records the dependencies between data sources and data applications, as well as between data applications, forming a traceable path between data assets; the permission management module checks and records the legal usage information of the lineage between different data applications.

[0028] To achieve the above object, the present invention also provides an electronic device, including a memory and a processor. A program is stored on the memory and runs on the processor. When the processor runs the program, it executes the steps of the data asset management system based on the data asset catalog as described above.

[0029] To achieve the above object, the present invention also provides a computer-readable storage medium, on which computer instructions are stored. When the computer instructions run, they execute the steps of the data asset management system based on the data asset catalog as described above.

[0030] The present invention proposes a data asset management system and method based on a data asset catalog, having the following beneficial effects:

[0031] 1) Based on traditional metadata management and combined with the requirements of data privacy protection, the present invention proposes an innovative data classification and structured management method. This method divides data into three forms: "instances", "examples", and "cases", ensuring the openness and sharing of data while protecting data security and privacy. Through this refined classification method, data management is more orderly, meeting the needs of data sharing while effectively preventing the leakage of sensitive data.

[0032] 2) The present invention provides a complete set of data asset lifecycle management methods, covering the entire process from the development to the commercial use of data assets. This method details the status and changes of data assets at each stage, as well as various artifacts generated during the development process, ensuring a high degree of traceability for the changes and status of data assets. Through this comprehensive management, every change to the data assets can be accurately recorded and monitored, enhancing the transparency and operability of data management.

[0033] 3) The present invention constructs an accurate data asset lineage through a hierarchical management strategy and the reference relationships between data assets. This mechanism supports a comprehensive traceability of the data source and change paths, ensuring the compliance and security of data assets. By subdividing data assets into project groups, projects, versions, and version statuses, and establishing reference relationships between different levels, the dependency and subordination relationships of the data are clearly presented, thereby enhancing the data governance and risk control capabilities.

[0034] 4) The present invention integrates key technical elements such as data protection, data asset lifecycle management, and data asset lineage traceability to construct a complete and efficient technical process. This process makes data asset management more systematic and standardized, providing a replicable and popularizable solution for enterprise-level data asset management systems. Enterprises can effectively manage and utilize their data assets in a complex data environment with the help of this technical process, significantly enhancing the transparency and efficiency of data governance, and promoting the development of data-driven decision-making and applications. Other features and advantages of the present invention will be described in the subsequent specification, and in part, will be obvious from the specification, or will be understood by implementing the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and together with the embodiments of the present invention, are used to explain the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0036] Figure 1 is a schematic diagram of the data asset management system architecture based on the data asset catalog of the present invention;

[0037] Figure 2 is a flowchart of the data asset management method based on the data asset catalog of the present invention;

[0038] Figure 3 is a schematic diagram of the "one yuan and three examples" data logic model relationship constructed for the embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of the full lifecycle management data logic model relationship constructed for the embodiment of the present invention;

[0040] Figure 5 Schematic diagram of the asset reference data logic model relationship constructed for the embodiments of the present invention. Detailed implementation manners

[0041] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.

[0042] Data asset: Data resources that are legally owned or controlled by a specific entity, can be measured in currency, and can bring economic benefits or social benefits.

[0043] Data asset catalog: A technology for systematically managing and sharing data assets, providing an integrated view of data resources to help an organization achieve centralized management and use of data assets.

[0044] Metadata: Detailed descriptive information about data, including the structure, content, format, source, and usage restrictions of the data. Metadata is usually divided into business metadata, technical metadata, and management metadata.

[0045] Lifecycle: The entire process of data from creation, storage, use, sharing to final destruction.

[0046] Lineage: The interdependent and flowing relationships between data and the artifacts generated during the application process.

[0047] Instance: A complete and specific presentation form of a certain data set, which covers all elements, attributes, and related information involved in the data set, representing an actually existing and independently identifiable and researchable data entity.

[0048] Example: Extracting some data items from a complete data entity or set and performing desensitization processing on them for the purpose of illustration, demonstration, or preliminary analysis.

[0049] Case: A comprehensive project example based on an actual application scenario, which takes a specific data asset as the core and elaborates in detail the whole process from data acquisition, sorting, analysis to final application to solve practical problems or achieve specific business goals.

[0050] Operational Data Store: A collection of raw data collected from information systems.

[0051] Data lake: A data set obtained by standardizing raw data.

[0052] Embodiment 1

[0053] Figure 1Schematic diagram of a data asset management system based on a data asset catalog according to the present invention, as Figure 1 shown, the data asset management system based on a data asset catalog of the present invention includes a data classification and structure management module, a data asset life cycle management module, a data asset lineage management module, a metadata module, and a permission management module. Among them,

[0054] The data classification and structure management module is used to classify data into three types: "instance", "example", and "case", and use the metadata management module to standardize the storage form (such as form) of various types of data, establish indexes, and support retrieval.

[0055] Optionally, in this process, the metadata management module not only describes and defines the data structure in detail, but also ensures that the data can be retrieved efficiently and accurately. At the same time, the data classification and structure management module configures and verifies the access permissions of different types of data through the permission management module to ensure the security of data during sharing and use.

[0056] In this embodiment, data classification and structure management is the starting point of data asset life cycle management. The premise of this stage is that the collection and storage of raw data have been completed, and the system already has the necessary data storage environment and basic data structure. The classification and structure management of data mainly uses the technical means of the metadata management module and the permission management module to ensure the safe and efficient management and sharing of data in different application scenarios.

[0057] In this embodiment, metadata is a detailed data description about data, covering information such as the structure, content, format, source, and usage restrictions of data. The goal of metadata management is to provide users with context information about the data to help them understand the nature and use of the data more accurately and make effective decisions. Metadata can be divided into multiple types: Business metadata describes the usage information of data in business processes and application scenarios. It includes the business definitions, business rules, data ownership, usage descriptions, and business contexts of data assets. Management metadata involves details of data maintenance and storage (such as storage location, access permissions, version control); technical metadata covers details of data processing and storage (such as encoding format, compression algorithm). These types of metadata together constitute a complete description of data assets.

[0058] In this embodiment, in terms of the management of data access permissions, the data asset catalog provides strong basic support. Users can browse and identify the required data assets through the catalog and initiate access requests based on the catalog information. The data asset catalog ensures the compliance and security of data access by setting the public scope and approval process. Different levels of access permissions (such as read-only, de-sensitized access) are determined according to specific requirements to meet diverse data usage scenarios.

[0059] The metadata module is the basis for data classification. The system standardizes the description of data through a unified metadata table to ensure the consistency and identifiability of the basic information of data (such as ID, name, description, data type, etc.) in all management stages. The metadata of each data item not only defines the structure of the data but also contains detailed descriptions of each field. This metadata management method provides a solid foundation for subsequent data tracking, access control, and applications, ensuring that data can flow consistently between different systems and platforms.

[0060] Optionally, the data classification and structure management module includes an instance index unit not shown in the figure. An instance index mechanism is established in the instance index unit to locate and reference data in different storage environments through an instance index table. The instance index table records the logical identifier, storage location, and its paths in different systems (such as data lakes, data warehouses, etc.) of the data, ensuring cross-platform traceability and consistency of the data. This mechanism not only guarantees the cross-system implementation of data lifecycle management but also ensures data privacy and security.

[0061] The permission management module is used to perform authorization and access control during the data classification process. Through strict access permission settings, users can only access corresponding data according to their permission scope. The example data display function is particularly important at this stage. The system provides partial data sample displays to stimulate users' innovative thinking and ensure that users can still obtain valuable information for data exploration under limited access permissions. The example data is associated with the metadata to ensure that the displayed content is consistent with the data structure while avoiding exposing sensitive data. In this way, users can obtain inspiring information conducive to innovation while ensuring data security.

[0062] In this embodiment, as Figure 3 shown, the case display function further enhances the level of data application. The system stores the successfully applied data solutions through a case table and helps users understand the application and performance of data in actual scenarios through the association with the metadata. Each case not only shows the application of data in a specific project but also provides the background information of data usage and its dependencies. This enables users to fully understand the effects of data in different scenarios when applying data, thereby promoting the development and application of innovative solutions.

[0063] As Figure 3 shown, metadata is data that describes data. For the three main types of data, namely instances, examples, and cases, table metadata and field metadata are set, as well as metadata tables such as instance table indexes, example table indexes, and case indexes, that is, the "one metadata and three examples" framework.

[0064] Table metadata is descriptive information about data, like a "navigation map" of a data lake, which comprehensively and meticulously depicts the data tables in the data lake in the form of a directory. Metadata not only includes the overall overview of the table, but also involves the content structure of the table, the management scope, and the information related to the administrator, etc. It is the cornerstone of data management. An example table is a broad form of expression. One piece of metadata describing a table can correspond to multiple example index tables. These example tables provide a specific reference for understanding the metadata and help to intuitively recognize the object characteristics described by the metadata.

[0065] Specifically at the data storage level, ODS (Operational Data Store) stores the original data from various information systems. Since the table names defined by different users are different, the table names obtained from different systems will be uniformly saved in the ODS as a backup of the original tables. Then, after unified management and standardization processing of these tables, they are stored in the data lake to provide a standardized data basis for subsequent analysis, and the example table index can be queried in the data lake through the example table name. A case is a project example based on a specific application scenario, such as a parking record table in a parking lot, which can be used to analyze scenarios such as employees' commuting patterns and optimize parking lot scheduling. Metadata details the specific content of these applications and analyses. One case index may be associated with multiple table metadata, and one piece of table metadata may also be applied to multiple cases. There is a complex relationship entityization logic between the two.

[0066] The data asset lifecycle management module is used to comprehensively record and control the status of data assets at each stage from development to commercialization during the process of data being transformed into data assets.

[0067] Optionally, through the metadata management module, the process information of various artifacts generated by the data asset throughout its lifecycle (including the data used, application scenarios, technical models, etc.) and their mutual relationships (such as the continuous change of status) are described in detail.

[0068] Optionally, the permission management module is responsible for reviewing and recording the permission granting and confirmation of each artifact during the lifecycle (such as ownership, usage rights, disposal rights, etc.) to ensure full traceability and strict access control of data assets throughout their lifecycle.

[0069] In this embodiment, in the "data asset lifecycle management module", these data are given scenario-based application value, permission control, and lifecycle records, reflecting their economic value and strategic significance in specific businesses and commercialization, thus completing the transformation from data to data assets.

[0070] In this embodiment, as Figure 4As shown, data asset lifecycle management aims to track and record in detail the entire process of data assets from development, rights confirmation to commercial use. This management process requires in-depth analysis by combining the metadata in the data asset catalog with the system logs and change records. Although the catalog can contain basic metadata (such as creation and update dates), complete lifecycle management relies on the system's analysis unit to monitor the status changes of data assets in real time. Through periodic task scheduling and analysis tools, the system can analyze the change history of data assets, form complete lifecycle information, and provide a more comprehensive view of asset dynamics for data managers.

[0071] As an example, during the development process, as Figure 4 shown, the status has inheritance, the current status is associated with the previous status, and the final status is determined in the version. Each status is closely associated with stakeholders. A certain status may involve multiple stakeholders, and conversely, a stakeholder may also be associated with multiple statuses. After the final status is determined in the version, a rights record needs to be made. For example, during the version establishment stage, it is necessary to clarify the stakeholders to whom the version belongs, or define the stakeholders to whom it belongs at a specific time node. When the version meets the commercial conditions, once it is commercially used, a commercial record will be generated. Each time it is commercially used, it is associated with a specific instance. It should be noted that the instance data used in the commercial stage may be different from that in the development stage. The development stage may use data from the previous year, while the commercial stage uses data from the current year. Each time these data are used, corresponding associations are established. At the same time, during the commercial process, the generated revenue records are all recorded in detail and accurately.

[0072] In this embodiment, the lifecycle management of data assets covers the entire process from data development to commercial use, ensuring that the status and change information of data in each stage are clearly tracked and managed. The premise of this stage is that the data classification and structure management stage has been completed, the data has been accurately classified, and the metadata has described the basic information of the data, such as data identification, type, structure, etc. At this time, the system begins to manage the development process of data assets, using technical means such as refined metadata management, version control, and permission management modules to ensure the security, traceability, and compliance of data assets.

[0073] During the development process of data assets, the system records and traces each stage of the data and the artifacts generated during the process through the metadata management module, thereby realizing the full life cycle management of data assets. Each stage of the development process - whether it is the "prototyping" or "pilot application" stage - is updated in status by the system through an automated process, and the relevant change information is recorded. These metadata not only include basic information such as the name, description, version number, and development date of each data asset, but also cover the status changes of data assets, such as "under development" or "completed", ensuring the transparency and traceability of data asset development.

[0074] When the data asset enters the commercial stage, the system will initiate the confirmation of rights management. At this time, the data asset has undergone complete development and testing and is ready to enter actual application. The confirmation of rights management is a key step to ensure the legal compliance of data assets during the process of transfer and use. In the commercial stage, the system will record in detail the ownership, usage rights, relevant agreements, etc. of each data asset to ensure that all data has clear legality and property rights during the commercial process. This information will be recorded in the metadata table to ensure that the use and change of each data asset comply with the pre-set legal and compliance requirements.

[0075] The confirmation of rights management includes tracking and recording the owners, users, and relevant legal agreements of each data asset. The system will mark the confirmation status of each data version and provide relevant agreement documents and review records when needed. This mechanism not only ensures the legal use of data, but also provides a clear path for the commercialization of data, avoiding property disputes and legal risks in the process of data transfer.

[0076] After the data enters the commercial stage, the system will continuously manage the permissions of data assets to ensure that the usage of data assets meets the authorization requirements. Through access control, only authorized users can access relevant data assets, and the system will record detailed information about each access operation, including the identity of the visitor, access time, operation content, etc. This refined permission management mechanism ensures that data can be used within the compliance framework at each usage stage after the end of the development cycle.

[0077] The data asset lineage management module is used to record the dependency relationships between data sources and data applications, as well as between data applications themselves, through a hierarchical management strategy and a data asset reference model between different management levels, forming a traceable chain between data assets.

[0078] Optionally, the permission management module verifies and records (manually verifies and records) the legal usage information of the lineage relationships between different data applications to prevent unauthorized access and data tampering, ensuring that the use of data assets meets the permission requirements.

[0079] In this embodiment, lineage management is a process for analyzing and presenting the dependency relationships and flow paths among data assets. Although a data asset catalog can provide preliminary association information, detailed lineage analysis requires in-depth link analysis with the aid of system tools. The system performs lineage analysis based on the metadata and data usage records in the catalog to trace the origin and destination of data, and presents the input, output, and processing links of data among different assets. Through this analysis, users can identify the upstream and downstream relationships and potential impact paths of data assets, thereby improving the transparency and traceability of data management.

[0080] In this embodiment, the tracing and visual display of data asset lineage are key aspects in ensuring transparent data management and enhancing data control capabilities. By applying metadata models and graphical display technologies, users can clearly understand the origin, transfer paths, and associations with other data at various stages of the data asset life cycle. This feature not only helps achieve precise management and optimization of data assets but also provides strong support for subsequent data compliance.

[0081] This embodiment also includes a hierarchical management strategy that organizes data assets into project portfolios, projects, versions, and version statuses in sequence, and establishes reference relationships among data assets at different levels. Through this strategy, the subordination and dependency relationships of data assets at different levels are clearly presented, thus laying a solid foundation for subsequent lineage tracing. After the data classification and structure management and data asset life cycle management phases are completed, relevant information such as the basic information, access rights, version control, and usage rules of data assets are recorded in the metadata. At this time, the updates, changes, and applications of data assets can be comprehensively monitored through lineage tracing.

[0082] As Figure 5 shown, in actual operation, the system constructs and maintains data asset lineage through the metadata model in the metadata management module, and clearly records the status, generation time, and association information of each data asset version in the metadata table. When a data asset is generated, updated, or iterated, a unique version number is automatically generated in the data asset lineage management module, and a reference relationship is established with the previous version using the version control mechanism. These reference relationships not only record the dependencies among assets but also elaborate on the change information contained in each asset, enabling clear tracing paths for data throughout the entire process from initial generation to final commercial use. Establish reference relationships in the data asset lineage management module and save them in the metadata model of the data asset lineage management module ( Figure 5) In the metadata management module, as assets are continuously iterated, the system will automatically update the reference relationships in the metadata table and indicate the inheritance and reference relationships of versions in the version association table. When data is updated or changed, the system can quickly identify the source, historical changes, and the degree of association with other data through these reference relationships, thereby avoiding version conflicts, ensuring the coherence and consistency of the data structure and logic, and enabling users to quickly trace back the evolution history of the data.

[0083] As an example, as Figure 5 shown, a program portfolio contains multiple projects, and each project has multiple versions. These versions will be upgraded as the models in the projects are improved. The applications exist in the projects in the form of versions, and each version has a corresponding status display, and the status will change at different stages of the development process. The versions are associated through version numbers. Each application has a previous version. When building an application, if the newly created application is associated with the previous new application and there is a relevant new version, relevant information needs to be added to the previous version to reflect the association and inheritance between versions. Each status is equivalent to a running sub-version and is associated with the instance index table. When associating, first clarify which type of table to associate with and simultaneously associate the table metadata, and then adjust the association table according to the status change to closely connect the status with the data instances used, ensuring that the data management in the development process matches the status change.

[0084] After development is completed, the final status is formed and recorded, and the version is released. After release, it is necessary to confirm whether it can be commercially used and set the commercial use flag, including two states: commercially available and not commercially available. It also includes establishing reference relationships. After the version is released, asset reference relationships are established for the technologies that can be referenced in the version. The asset reference relationships include that the released version can be referenced by the status of the version that is being developed. The reference relationship does not include self-reference, and the status in another asset version can reference a released version.

[0085] In this embodiment, while achieving precise lineage tracing, introducing the graphical display function of the data asset lineage relationship management module significantly improves the visualization and understandability of the data asset lineage relationship. Through the integrated graphical interface, users can intuitively view the reference relationships and dependency relationships between data assets. This graphical method not only helps data managers track the flow and evolution of data assets in real time, but also enables managers to conduct quick risk assessments and impact analyses when data changes, and timely discover potential version conflicts, security risks, or management loopholes. Thus, the supervision and optimization of data assets become more efficient and accurate.

[0086] In this embodiment, the permission management module plays a key role in the management of lineage relationships. The system uses the permission management module to check and record information on the legitimate use of lineage relationships between different data applications, preventing unauthorized access and data tampering. This mechanism ensures that the use of data assets complies with permission requirements and maintains the integrity and security of the data.

[0087] In this embodiment, the data asset catalog is designed specifically for the systematic management of data assets. Its core function is to provide an integrated view of data resources, assisting the organization in achieving centralized management of data assets. By stratifying and classifying data assets, the data asset catalog can standardize the description and organization of these assets, enabling users to more clearly understand and utilize data resources. The data asset catalog contains metadata such as the identification, category, description, management attribution, and update cycle of the data, and is maintained and updated through automatic collection and manual supplementation to ensure the accuracy and consistency of data asset information.

[0088] Embodiment 2

[0089] Figure 2 is a flowchart of a data asset management method based on a data asset catalog according to the present invention. As Figure 2 shown, a data asset management method based on a data asset catalog according to the present invention includes:

[0090] S1. The existing data is divided into instances, examples, and cases through the data classification and structure management module, and the storage forms of various types of data are standardized, indexed, and supported for retrieval using the metadata management module; access control is performed through the permission management module to achieve authorization and confirmation of various types of data.

[0091] S2. During the process of data being transformed into data assets through the data asset lifecycle management module, the process information and interrelationships of various artifacts generated during the lifecycle of the data asset from development to commercialization are established. Through the permission management module, various permissions granted and confirmed for individual artifacts in the lifecycle are checked and recorded, realizing full traceability and access control during the data asset lifecycle.

[0092] S3. The lineage relationships of different data applications at different management levels are established through the data asset lineage relationship management module, and the dependencies between the data sources and data applications, as well as between data applications, are recorded, forming a traceable relationship between data assets; the permission management module is used to check and record information on the legitimate use of lineage relationships between different data applications.

[0093] In this embodiment, metadata management is a pervasive function that serves business modules such as data classification, data lifecycle management, and data lineage relationship management.

[0094] A data asset management system and method based on a data asset catalog proposed by the present invention are optimized and innovated on the basis of existing data asset catalog management, and a new data classification and structured management method is proposed. This method ensures that while data is open and shared, a high level of security and compliance is maintained, thereby improving the utilization rate of data and promoting data-driven innovation. Compared with traditional methods, the present invention shows significant advantages in enhancing data value and innovation ability. The present invention also draws on the conceptual framework of data full-life cycle management in the ISO 8183 standard and integrates it into the management of the data asset catalog. By combining the standard framework and self-developed technologies, the present invention realizes the full-process management of data assets. In addition, by constructing reference relationships between data assets at different levels, the lineage relationship of data assets is accurately constructed, enhancing the traceability and compliance of data.

[0095] The present invention also provides an electronic device, including a memory and a processor. A program is stored on the memory and runs on the processor. When the processor runs the program, it executes the steps of the above-mentioned data asset management method based on a data asset catalog.

[0096] The present invention also provides a computer-readable storage medium, on which computer instructions are stored. When the computer instructions run, they execute the above-mentioned data asset management method based on a data asset catalog. For the data asset management method based on a data asset catalog, refer to the introduction in the foregoing part and will not be elaborated herein.

[0097] Those of ordinary skill in the art can understand that the above are only preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A data asset management system based on a data asset catalog, characterized in that: include: Data classification and structure management module, which is used to classify data into instances, examples and cases, and use the metadata management module to standardize the storage form of various types of data, establish indexes and support retrieval; The metadata module is used to describe the data in a standardized manner through a metadata table to ensure the consistency and identifiability of the basic information of the data in all management stages; The permission management module is used to perform authorization and access control in the process of data classification, so that users can access the corresponding data according to the set access permission range; A data asset lifecycle management module is used to record and control the status of each stage of the data asset lifecycle from development to commercialization during the process of data conversion into data assets, and to describe the process information of various artifacts generated by data assets during the lifecycle and their interrelationships through the metadata management module; The data asset lineage management module is used to record the dependencies between data sources and data applications, as well as between data applications, through hierarchical management strategies and data asset reference models between different management levels, to form a traceable chain between data assets.

2. A data asset management system based on a data asset catalog according to claim 1, characterized in that: The metadata module defines the metadata structure of a single data item and the description of a single field.

3. A data asset management system based on a data asset catalog according to claim 1, characterized in that: The data classification and structure management module includes an instance index unit, which is used to establish an instance index mechanism. The instance index mechanism realizes the positioning and reference of data in different storage environments through an instance index table; the instance index table records the logical identification, storage location and path of the data in different systems.

4. A data asset management system based on a data asset catalog according to claim 1, characterized in that: The rights management module includes a review unit and a verification unit, wherein: The review unit is used to review and record the authorization and confirmation of a single artifact during its life cycle, record the detailed information of a single access operation, and ensure that data assets are fully traceable and strictly controlled during their life cycle; The verification unit is used to verify and record the legal use information of the blood relationship between different data applications.

5. A data asset management system based on a data asset catalog according to claim 1, characterized in that: The data asset lifecycle management module includes an analysis unit, which is used to analyze the metadata in the data asset directory and the system logs and change records, monitor the status changes of data assets in real time, analyze the change history of data assets, and form complete lifecycle information.

6. A data asset management system based on a data asset catalog according to claim 1, characterized in that: The data asset lifecycle management module includes a title confirmation unit, which is used to record the title confirmation information of a single data asset during the commercial stage to ensure the legitimacy and property rights of the data asset during the commercial process; the title confirmation information of the data asset is recorded in the metadata table to ensure that the use and changes of a single data asset comply with pre-set legal and compliance requirements.

7. A data asset management system based on a data asset catalog according to claim 1, characterized in that: It also includes that the data asset lineage management module includes a hierarchical unit, which is used to execute a hierarchical management strategy, organize data assets into project groups, projects, versions and version status in sequence, and present the subordination and dependency relationships of data assets at different levels between data sources and data applications, and between data applications. The update, change and application of data assets are monitored through lineage tracing.

8. A data asset management method based on a data asset catalog, using the data asset management system based on a data asset catalog as described in claims 1-7, characterized in that: include: The data classification and structure management module divides the data into instances, examples and cases, and uses the metadata management module to standardize the storage form of various types of data, establish indexes and support retrieval; access control is performed through the authority management module to realize the authorization and confirmation of various types of data; In the process of data conversion into data assets, the data asset lifecycle management module establishes process information and interrelationships of various artifacts generated in the lifecycle of data assets from development to commercialization. The permission management module checks and records the granting and confirmation of various permissions for individual artifacts in the lifecycle, thus achieving full traceability and access control in the data asset lifecycle. The data asset lineage management module records the dependencies between data sources and data applications, as well as between data applications, to form a traceable path between data assets; Use the permission management module to check and record the legal use information of the blood relationship between different data applications.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a program running on the processor, and the processor executes the data asset management method based on a data asset catalog as described in claim 8 when running the program.

10. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed, the data asset management method based on the data asset directory described in claim 8 is executed.

Images