Data asset management system and method

Abstract

The invention relates to the technical field of data asset management, in particular to a data asset management system and method. The method comprises the following steps: acquiring a data asset information set from different data sources in real time and performing tag structure sub-column division to obtain data asset information tag description structured sub-columns, and meanwhile, performing time sequence creation synchronization processing and integrating and storing the data asset information tag description structured sub-columns into a data asset management database; performing use frequency and sensitivity evaluation analysis on each piece of data asset sub-information in the data asset management database, and performing data asset access utility evaluation analysis to obtain data access utility of each piece of data asset sub-information; and performing dynamic access right control and full life cycle management design on each piece of data asset sub-information in the data asset management database, and performing automatic management and report feedback recording to generate a data asset information management report. The transparency, safety and efficiency of data asset management can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data asset management, and in particular to a data asset management system and method. Background Art

[0002] In modern enterprises and organizations, data assets have become core assets, and the complexity and importance of their management are constantly increasing. Machine learning algorithms can be used to automatically classify and label data assets, and through in-depth analysis of data content and attributes, data types and sensitivity levels can be automatically identified to improve the accuracy and efficiency of data classification. Dynamic storage management technology is also introduced to automatically adjust storage resources according to the frequency and importance of data assets, and to optimize the allocation and utilization of storage space by real-time monitoring of data access and storage requirements. At the same time, the use and changes of data assets are continuously tracked by deploying a real-time data monitoring system, and potential anomalies and risks are timely identified through intelligent analysis and early warning mechanisms, and alerts are automatically generated and correction suggestions are provided. However, traditional data asset management methods usually rely on static rules and manual operations, and cannot efficiently handle large-scale and dynamically changing data assets, making data asset management inefficient. Summary of the invention

[0003] Based on this, it is necessary for the present invention to provide a data asset management system and method to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a data asset management method includes the following steps:

[0005] Step S1: by collecting data asset information sets from enterprise databases, file systems, and API data sources in real time, and dividing the data asset information sets into tag structure sub-columns to obtain data asset information tag description structured sub-columns; creating time series synchronization processing for each data asset information tag description structured sub-column and integrating and storing them in the data asset management database in the data processing unit;

[0006] Step S2: Performing a usage frequency and sensitivity evaluation analysis on each data asset sub-information in the data asset management database to obtain the data usage frequency and data sensitivity of each data asset sub-information; performing a data asset access utility evaluation analysis on each data asset sub-information in the data asset management database based on the data usage frequency and data sensitivity of each data asset sub-information to obtain the data access utility of each data asset sub-information;

[0007] Step S3: Dynamically control the access rights of each data asset sub-information in the data asset management database based on the data usage frequency and data access utility of each data asset sub-information, and obtain the dynamic access control rights of the data asset management database;

[0008] Step S4: Perform full life cycle management design for the corresponding data asset information in the data asset management database according to the dynamic access control authority of the data asset management database to generate full life cycle management rules for the data asset information; perform automated management and report feedback records for the corresponding data asset information in the data asset management database based on the full life cycle management rules for the data asset information to generate a data asset information management report.

[0009] Further, step S1 includes the following steps:

[0010] Step S11: acquiring a subset of information related to data assets from an enterprise database data source in real time to obtain a subset of database source data asset information;

[0011] Step S12: obtaining a file system source data asset information subset by collecting a data asset-related information subset from a file system data source in real time;

[0012] Step S13: obtaining an API source data asset information subset by collecting an information subset related to the data asset from the API data source in real time;

[0013] Step S14: performing information set processing on the database source data asset information subset, the file system source data asset information subset and the API source data asset information subset to obtain a data asset information set; dividing the data asset information set into label structure sub-columns to obtain data asset information label description structured sub-columns;

[0014] Step S15: Create a time series synchronization process for each data asset information tag description structured sub-column and integrate and store it in the data asset management database.

[0015] Furthermore, the step S14 of dividing the data asset information set into sub-columns of the label structure includes the following steps:

[0016] Performing information content semantic analysis on each piece of data asset information in the data asset information set to obtain content semantic data corresponding to the data asset content of each piece of data asset information;

[0017] Perform data format attribute mining and analysis on each data asset information in the data asset information set to obtain the data format attribute of each data asset information corresponding to the data asset content;

[0018] Based on the data format attributes of the data asset content corresponding to each data asset information, the content semantic data of the data asset content corresponding to each data asset information is classified and labeled to obtain various data content attribute labels corresponding to the data asset content, wherein various data content attribute labels include creation time, data type, storage location, data purpose and modification time;

[0019] According to various data content attribute labels corresponding to the data asset content, each data asset information in the data asset information set is divided into attribute label descriptions to obtain a data asset information content attribute label description sub-column;

[0020] Each data asset content in the data asset information content attribute label description sub-column is structurally transformed to obtain a data asset information label description structured sub-column.

[0021] Further, step S15 includes the following steps:

[0022] Step S151: Perform creation time label sub-column screening processing on each data asset information label description structured sub-column to obtain a data asset information creation time label structured sub-column and a data asset information remaining label structured sub-column;

[0023] Step S152: based on the data asset information creation time tag structured sub-column, perform creation time sequence synchronization processing on the remaining tag structured sub-columns of the data asset information to obtain the tag structured sub-columns of each data asset information under the same creation time sequence dimension;

[0024] Step S153: Designing the data integration storage format for each data asset information tag structured sub-column under the same creation time sequence dimension through the data asset management database in the data processing unit to obtain the data asset information tag sub-column integration storage format specification;

[0025] Step S154: Integrate and store each data asset information tag structured sub-column under the same creation time sequence dimension into the data asset management database in the data processing unit according to the data asset information tag sub-column integration storage format specification.

[0026] Further, step S2 includes the following steps:

[0027] Step S21: Perform access log recording processing on each data asset sub-information in the data asset management database to obtain data access log record information of each data asset sub-information;

[0028] Step S22: Analyze the number of accesses and time distribution of the data access log record information of each data asset sub-information to obtain the number of data accesses and data access time distribution of each data asset sub-information;

[0029] Step S23: Perform usage frequency statistics analysis on the number of data accesses of each data asset sub-information based on the data access time distribution of each data asset sub-information to obtain the data usage frequency of each data asset sub-information;

[0030] Step S24: using the data asset information sensitivity calculation formula to perform information sensitivity evaluation calculation on each data asset sub-information in the data asset management database, and obtaining the data sensitivity level of each data asset sub-information;

[0031] Step S25: Based on the data usage frequency and data sensitivity of each data asset sub-information, a data asset access utility evaluation and analysis is performed on each data asset sub-information corresponding to the data asset management database to obtain the data access utility of each data asset sub-information.

[0032] Furthermore, the calculation formula for the data asset information sensitivity in step S24 is specifically:

[0033]

[0034] In the formula, S i is the data sensitivity of the i-th data asset sub-information, n is the total number of data asset sub-information in the data asset management database, i is the item index parameter of the data asset sub-information, R i is the data asset availability parameter of the i-th data asset sub-information, α is the data asset availability impact weight coefficient, D i is the leakage risk measurement parameter of the i-th data asset sub-information, β is the weight coefficient of the data asset leakage risk impact, γ is the exponential attenuation coefficient of the control leakage risk impact, C i is the data asset confidentiality parameter of the i-th data asset sub-information, δ is the data asset confidentiality impact weight coefficient, m is the total number of additional attribute information in the data asset sub-information, j is the item measurement parameter of the additional attribute information, P i,j is the specific value of the i-th data asset sub-information on the j-th additional attribute information, κ j is the influence weight coefficient of the jth additional attribute information, and η is the correction coefficient of the data sensitivity.

[0035] Further, step S25 includes the following steps:

[0036] Step S251: classifying the data sensitivity level of each data asset sub-information corresponding to each data asset sub-information in the data asset management database based on the data sensitivity level of each data asset sub-information, and obtaining the data sensitivity level of each data asset sub-information;

[0037] Step S252: performing access utility weighted analysis on each data asset sub-information corresponding to each data asset sub-information in the data asset management database based on the data usage frequency and data sensitivity level of each data asset sub-information, so as to obtain a data access utility weighted value set for each data asset sub-information;

[0038] Step S253: Perform data asset access utility evaluation and analysis on each data asset sub-information corresponding to the data asset management database according to the data access utility weighted value set of each data asset sub-information to obtain the data access utility of each data asset sub-information.

[0039] Further, step S3 includes the following steps:

[0040] Step S31: Performing management role access business requirement analysis on each data asset sub-information in the data asset management database to obtain the database management role access business requirement for each data asset sub-information;

[0041] Step S32: performing data usage constraint analysis on each data asset sub-information in the data asset management database based on the data usage frequency of each data asset sub-information to obtain data usage constraint conditions for each data asset sub-information;

[0042] Step S33: performing access utility restriction analysis on each data asset sub-information in the data asset management database based on the data access utility of each data asset sub-information, and obtaining the data access utility restriction range of each data asset sub-information;

[0043] Step S34: Based on the data usage constraints and data access utility restriction range of each data asset sub-information, dynamic access permission control is performed on the database management role access business requirements of each data asset sub-information to obtain dynamic access control permissions for the data asset management database.

[0044] Further, step S4 includes the following steps:

[0045] Step S41: performing management role authority matching and mapping on the dynamic access control authority of the data asset management database to obtain the data asset management role dynamic access matching and mapping control authority;

[0046] Step S42: Based on the data asset management role dynamic access matching mapping control authority, the corresponding data asset information in the data asset management database is analyzed for the full life cycle stage authority characteristics to obtain the data asset full life cycle stage management role authority characteristic set;

[0047] Step S43: Formulate full life cycle management rules for the management role authority feature set of the data asset full life cycle stage to generate full life cycle management rules for data asset information;

[0048] Step S44: Based on the data asset information full life cycle management rules, the corresponding data asset information in the data asset management database is automatically managed and processed to obtain the data asset information full life cycle automatic management results;

[0049] Step S45: Record the management report feedback of the automatic management results of the data asset information throughout its life cycle to generate a data asset information management report.

[0050] Furthermore, the present invention also provides a data asset management system for executing the data asset management method as described above, the data asset management system comprising:

[0051] The data asset information structured storage module is used to collect data asset information sets from databases, file systems, and API data sources in real time, and divide the data asset information sets into tag structure sub-columns to obtain data asset information tag description structured sub-columns; create time-series synchronization processing for each data asset information tag description structured sub-column and integrate and store them in the data asset management database;

[0052] The usage frequency and access utility analysis module is used to perform usage frequency and sensitivity evaluation analysis on each data asset sub-information in the data asset management database, and obtain the data usage frequency and data sensitivity of each data asset sub-information; based on the data usage frequency and data sensitivity of each data asset sub-information, perform data asset access utility evaluation analysis on each data asset sub-information in the data asset management database, and obtain the data access utility of each data asset sub-information;

[0053] A database dynamic access permission control module is used to dynamically control the access permission of each data asset sub-information in the data asset management database based on the data usage frequency and data access utility of each data asset sub-information, thereby obtaining dynamic access control permissions for the data asset management database;

[0054] The data asset full life cycle management module is used to perform full life cycle management design for the corresponding data asset information in the data asset management database based on the dynamic access control permissions of the data asset management database, so as to generate full life cycle management rules for data asset information; based on the full life cycle management rules for data asset information, the corresponding data asset information in the data asset management database is automatically managed and report feedback records are recorded to generate data asset information management reports.

[0055] Beneficial effects of the present invention:

[0056] 1. Compared with the prior art, the data asset management method proposed in the present invention has the beneficial effect that by collecting a subset of information related to data assets from the enterprise database data source, the real-time and accuracy of data assets can be ensured. Through this method, the latest database content can be obtained to ensure that the data reflected in the management system is the latest and most timely, which can reduce the risk of information lag and make data asset management more reliable and efficient. By collecting a subset of information related to data assets from the file system data source in real time, the data assets stored in the file system can be effectively monitored and managed. As one of the main forms of data storage, the file system contains a large number of files and documents, which often carry important business information and historical records. Real-time collection can ensure that the management of data assets in the file system is the latest, and timely reflect the addition, modification or deletion of files, thereby providing basic data guarantee for subsequent processing. By collecting a subset of information related to data assets from the API data source in real time, data from external systems or third-party services can be obtained. This method enables effective data integration with other systems or platforms, enhances the interoperability of the system and the richness of data, and collects API data in real time. Not only can it provide the latest external data, but it can also help understand the impact of the external environment on its own data assets. At the same time, by processing the subsets of data asset information collected from different data sources, the data asset information from multiple sources can be comprehensively summarized. This step generates a unified data asset information set through the processing of the information set, which is of great significance for the comprehensive management and analysis of data. The data asset information is also structured through the label structure sub-column division, and each data asset label is given to make its description clearer and more orderly. This structured description method helps to quickly identify and classify data assets and improve the efficiency of data retrieval. By creating time series synchronization processing for each data asset information label description structured sub-column and integrating and storing it in the data asset management database, it helps to maintain the time consistency and system consistency of data asset information. The time series synchronization processing ensures the consistency of data at different time points and avoids data inconsistency problems caused by time differences. This step integrates and stores the structured label description information in the data asset management database, providing a reliable data foundation for subsequent data analysis, report generation and decision support, so that large-scale and dynamically changing data asset information can be efficiently processed, while supporting more accurate business decisions and strategic planning. Secondly, by conducting a usage frequency statistical analysis on each data asset sub-information in the data asset management database, this process aims to combine the number of data accesses with the time pattern to evaluate the frequency of use of data assets. This analysis can identify the actual usage of data assets and determine which data assets are frequently used and which are less frequently used.Through statistical analysis, managers can understand the actual needs of data and make more accurate resource allocation decisions. In addition, by conducting a corresponding sensitivity assessment on each data asset sub-information in the data asset management database, the sensitivity of each data asset is determined. This assessment takes into account factors such as data content, access rights, and potential risks, thereby determining the confidentiality and security requirements of the data. Through sensitivity assessment, it can be ensured that highly sensitive data assets receive appropriate protection measures to avoid damage caused by data leakage or abuse. In addition, based on the data usage frequency and data sensitivity of each data asset sub-information in the data asset management database, a data asset access utility assessment analysis is performed on each corresponding data asset sub-information. The purpose of this process is to comprehensively consider the frequency of data use and sensitivity, and evaluate the overall utility of data assets. This assessment can identify the most valuable data assets in the management and use process, thereby improving the overall effectiveness of data asset management. Then, by dynamically controlling the access rights of each data asset sub-information in the data asset management database based on the data usage frequency and data access utility of each data asset sub-information, the access rights of the database management role can be dynamically controlled by comprehensively considering the data usage frequency and data access utility. This process allows real-time adjustment and optimization of data access rights based on actual business needs and data access conditions. The dynamic access right control processing process can flexibly adjust access policies based on changes in data usage and utility to ensure data security and effectiveness. This method can effectively respond to changes in a dynamic business environment and reduce the limitations of static permission configuration, thereby providing more accurate and timely data asset protection. Finally, by conducting a full life cycle management design for the corresponding data asset information in the data asset management database based on the dynamic access control permissions of the data asset management database, the corresponding management rules can be designed to provide operational guidance and control standards for each stage of the data life cycle, ensuring the consistency and standardization of data asset management. By formulating clear management rules, data leakage or compliance risks caused by improper permission settings or non-standard operations can be avoided. This step helps to establish a systematic management framework, making data asset management more transparent and controllable, thereby improving the overall efficiency and effectiveness of data asset management and providing guarantees for the compliance and security of the enterprise.By automatically managing and processing the corresponding data asset information in the data asset management database based on the data asset information life cycle management rules, it is possible to achieve efficient and continuous monitoring and control of data assets. Automated processing not only reduces human operation errors, but also responds to changes in data management needs in real time, improving management flexibility and accuracy. This step makes the management of data assets more efficient throughout the life cycle, and can automatically execute rules, update and maintain data status in real time, thereby reducing manual intervention and improving the reliability and management efficiency of overall data asset management. This automated management result is also convenient for comprehensive analysis and report generation of data assets. In addition, by reporting and recording the data asset management results obtained after automated management, systematic management reports can be formed. These reports not only record the results of automated management, but also provide a comprehensive evaluation of the management process and effects. This process helps to comprehensively review all aspects of data asset management, identify problems and room for improvement, and the generated reports can provide important references for decision makers, help optimize data management strategies and processes, and thus improve the management effect of data assets and the scientific nature of business decisions.

[0057] 2. The data asset management system proposed in the present invention is generally composed of a data asset information structured storage module, a usage frequency and access utility analysis module, a database dynamic access permission control module and a data asset full life cycle management module. It can implement any data asset management method described in the present invention, and is used to combine the operations between computer programs running on each module to implement the data asset management method. The internal structures of the system cooperate with each other, which can greatly reduce duplication of work and manpower investment, and can quickly and effectively provide a more accurate and efficient data asset management process, thereby simplifying the operating procedures of the data asset management system. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0059] Figure 1 A schematic diagram of the steps of the data asset management method of the present invention;

[0060] Figure 2 for Figure 1 Detailed step flow diagram of step S1;

[0061] Figure 3 for Figure 2 Detailed step flow chart of step S15 in FIG. DETAILED DESCRIPTION

[0062] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0063] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0064] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0065] To achieve this, please refer to Figures 1 to 3 The present invention provides a data asset management method, the method comprising the following steps:

[0066] Step S1: by collecting data asset information sets from enterprise databases, file systems, and API data sources in real time, and dividing the data asset information sets into tag structure sub-columns to obtain data asset information tag description structured sub-columns; creating time series synchronization processing for each data asset information tag description structured sub-column and integrating and storing them in the data asset management database in the data processing unit;

[0067] Step S2: Performing a usage frequency and sensitivity evaluation analysis on each data asset sub-information in the data asset management database to obtain the data usage frequency and data sensitivity of each data asset sub-information; performing a data asset access utility evaluation analysis on each data asset sub-information in the data asset management database based on the data usage frequency and data sensitivity of each data asset sub-information to obtain the data access utility of each data asset sub-information;

[0068] Step S3: Dynamically control the access rights of each data asset sub-information in the data asset management database based on the data usage frequency and data access utility of each data asset sub-information, and obtain the dynamic access control rights of the data asset management database;

[0069] Step S4: Perform full life cycle management design for the corresponding data asset information in the data asset management database according to the dynamic access control authority of the data asset management database to generate full life cycle management rules for the data asset information; perform automated management and report feedback records for the corresponding data asset information in the data asset management database based on the full life cycle management rules for the data asset information to generate a data asset information management report.

[0070] In the embodiment of the present invention, please refer to Figure 1 FIG. 1 is a flow chart of the steps of the data asset management method of the present invention. In this example, the data asset management method includes the following steps:

[0071] Step S1: by collecting data asset information sets from enterprise databases, file systems, and API data sources in real time, and dividing the data asset information sets into tag structure sub-columns to obtain data asset information tag description structured sub-columns; creating time series synchronization processing for each data asset information tag description structured sub-column and integrating and storing them in the data asset management database in the data processing unit;

[0072] In an embodiment of the present invention, by collecting information data related to data assets in real time in an enterprise database, and by configuring a database connection interface, using SQL query language to extract relevant data asset information from a target database table, this process obtains information related to data assets, such as the name of a database table, field type, number of records, etc., by executing specific SELECT statements. These query statements must accurately reflect the structure and content of data assets, thereby obtaining a subset of database source data asset information. By using a corresponding file system retrieval tool (such as Python's os module or Java's NIO) to collect information data related to data assets from a file system data source in real time, traverse a specified file directory according to the screening criteria of file path and file type, identify qualified files, and for each file, retrieve and collect a subset of file system source data asset information by reading the file's metadata (such as file name, creation date, size, etc.) and file content (such as file text or structured data). In addition, by collecting information data related to data assets in real time from the corresponding API data source, configuring the parameters of the API request, including the request method (such as GET or POST), the request header and the request body, and initiating the API request by using an HTTP client library (such as Python's Requests library), obtaining the returned JSON or XML data, parsing the response data to extract information related to the data asset, such as the asset ID, attribute description, etc., so as to integrate and obtain the API source data asset information subset. At the same time, by integrating the data asset information subsets of different data sources previously collected in real time from different data sources, the information subsets obtained from different data sources are standardized and formatted, and the data from different sources are merged by using a data integration tool (such as Apache NiFi or Talend), and duplicate and redundant information is removed, and the various information subsets are integrated through a data fusion algorithm (such as data matching and merging technology) to generate a unified data asset information set, thereby obtaining a data asset information set. We also define label classification standards based on the actual content and structure of data assets, and use data annotation tools (such as Labelbox or custom scripts) to assign labels to data asset information sets, divide data asset information into different label categories (such as data asset type, data format, storage location, creation time, and modification time, etc.), and subdivide the label structure into sub-columns of different label categories to ensure the accuracy and effectiveness of label division, thereby obtaining structured sub-columns describing data asset information labels.Then, by performing time series synchronization processing on the structured sub-columns described by the data asset information tags obtained previously according to the corresponding creation time, a time series data processing framework (such as Apache Kafka and Apache Flink) is established to achieve creation timestamp marking of each structured sub-column, and by using time series data synchronization algorithms (such as time window technology) to process data, the time series consistency and integrity of the structured sub-columns described by the data asset information tags are ensured. At the same time, the data after time series synchronization processing is integrated and stored in the data asset management database (such as PostgreSQL or MongoDB), and a data backup and recovery mechanism is set up to ensure long-term stable storage and access to data.

[0073] Step S2: Performing a usage frequency and sensitivity evaluation analysis on each data asset sub-information in the data asset management database to obtain the data usage frequency and data sensitivity of each data asset sub-information; performing a data asset access utility evaluation analysis on each data asset sub-information in the data asset management database based on the data usage frequency and data sensitivity of each data asset sub-information to obtain the data access utility of each data asset sub-information;

[0074] In an embodiment of the present invention, a detailed access log is generated in the data asset management database for each data asset sub-information access behavior using a logging tool (such as ELK Stack or Splunk), and each access event to the data asset sub-information is recorded to obtain data access log information for each data asset sub-information. The data access log information of each data asset sub-information is analyzed by using a data analysis tool (such as Apache Hadoop or the Pandas library in Python), and the access frequency of each data asset sub-information is quantitatively calculated by statistically analyzing the number of accesses to each data asset sub-information, thereby obtaining the data usage frequency of each data asset sub-information. At the same time, information sensitivity evaluation calculations are performed in combination with relevant parameters of the data asset sub-information to quantitatively calculate the corresponding sensitivity value, thereby obtaining the data sensitivity degree of each data asset sub-information. Then, by combining the data usage frequency and data sensitivity of each data asset sub-information obtained from the previous statistical analysis, a weighted scoring method is used to evaluate the access utility of each data asset sub-information corresponding to the data asset management database, so as to calculate the scores of the two dimensions according to the usage frequency and sensitivity, and combine the scores of the two dimensions to obtain the final access utility score through the weighted average method. This step is usually automatically calculated using data analysis tools or custom scripts. For example, the corresponding calculation formula can be: In the formula, E kScore the final access utility of the i-th data asset sub-information, F i is the data usage frequency of the i-th data asset sub-information, a is the usage frequency weight coefficient, S i is the data sensitivity of the ith data asset sub-information, and b is the sensitivity weight coefficient, which ensures the accuracy and objectivity of the evaluation results, and finally obtains the data access utility of each data asset sub-information.

[0075] Step S3: Dynamically control the access rights of each data asset sub-information in the data asset management database based on the data usage frequency and data access utility of each data asset sub-information, and obtain the dynamic access control rights of the data asset management database;

[0076] In an embodiment of the present invention, a management role access business requirement analysis is performed on each data asset sub-information in the data asset management database, so as to collect the management role and corresponding business requirement of each data asset sub-information. This process is completed by accessing business requirement documents, interviewing management personnel, and analyzing existing permission settings. The management role information is extracted by using a permission management tool (such as LDAP or Active Directory), and the access requirements of each role to data assets are recorded in detail in combination with a business requirement analysis tool (such as a business process modeling tool BPMN). In addition, a data usage constraint analysis is performed on the corresponding data asset sub-information in the data asset management database by combining the data usage frequency of each data asset sub-information obtained by previous analysis. According to the analysis results, data usage constraint conditions are defined, such as implementing stricter access control on frequently accessed sensitive data, combining the usage frequency data with business requirements, and setting reasonable data usage restrictions. For example, if the access frequency of a data asset sub-information is high, it is necessary to limit the access time period or the number of accesses to the data. In addition, by combining the data access utility of each data asset sub-information obtained by previous quantitative calculation, the corresponding data asset sub-information in the data asset management database is analyzed for access utility restrictions, so as to define the data usage constraint conditions according to the analysis results. BI) generates an access utility graph to determine the utility range of data access. Then, by combining the data usage constraints and data access utility restriction range obtained from the previous analysis, the management role access business needs of each data asset sub-information obtained from the previous analysis are dynamically controlled to set the data usage constraints and access utility restriction range with the business needs of the management role by utilizing a dynamic permission control system (such as an RBAC-based permission management system or an ABAC-based access control policy). The data usage constraints and access utility restriction range are combined with the business needs of the management role to create a dynamic access control policy. The implementation process includes configuring the policy rules of the permission management system and dynamically adjusting the role permissions to meet the requirements of data usage and access utility. For example, for a highly sensitive and high access utility data asset, permissions will be automatically restricted to only necessary management roles, and stricter access auditing and control measures will be implemented to ultimately obtain dynamic access control permissions for the data asset management database.

[0077] Step S4: Perform full life cycle management design for the corresponding data asset information in the data asset management database according to the dynamic access control authority of the data asset management database to generate full life cycle management rules for the data asset information; perform automated management and report feedback records for the corresponding data asset information in the data asset management database based on the full life cycle management rules for the data asset information to generate a data asset information management report.

[0078] In an embodiment of the present invention, the dynamic access control permissions in the data asset management database obtained by the previous analysis are matched and mapped with the permissions of the management roles, so as to extract the access permission configuration of each management role from the permission management system, and map these permissions to the corresponding data asset information, and use the permission mapping tool (such as the role mapping module in the RBAC system) to map, match the permissions of each management role with the access requirements of the data assets, determine the dynamic access permissions of each role to the specific data assets, and perform the characteristic analysis of the full life cycle permissions on the corresponding data asset information in the data asset management database by combining the dynamic access control permissions obtained by the previous mapping assignment, so as to control the various life cycle stages of the data assets (such as creation, storage, use) through the life cycle management tool (such as the enterprise data management system). , sharing, and destruction), record the management roles and their permission characteristics involved in each stage, and analyze the permission requirements and restrictions of the roles at different life cycle stages. The full life cycle management rules are formulated based on the permission requirements and restrictions of the roles at different life cycle stages obtained by the previous analysis, so as to create the full life cycle management rules of data asset information by using a rule engine (such as Drools) or a policy formulation tool, and convert the permission characteristics obtained by the previous analysis into management rules. For example, access control policies and operation requirements are defined for different life cycle stages of data. The formulated rules include how to implement permission control, monitoring and audit requirements at different stages of data to ensure that the management of data assets throughout the life cycle complies with the set policies and security standards, so as to formulate and generate corresponding full life cycle management rules for data asset information. Then, the corresponding data asset information in the data asset management database is automatically managed and processed by combining the full life cycle management rules of data asset information obtained by the previous analysis, so as to apply the management rules to data assets by using an automated management system (such as a data governance platform or an automated workflow management system), and automatically perform data access control, permission allocation, data auditing and compliance checks according to the rules to obtain full life cycle automatic management results. Finally, by using report generation tools (such as Microsoft Power BI or Tableau), feedback records of management reports are made on the full life cycle automatic management results obtained after previous automated management, so as to summarize the automatic management processing results and generate detailed management reports. The report content includes the implementation of automated management, the effectiveness of authority control, the management effect of the life cycle stage, etc. Feedback records are generated regularly through the reporting system and provided to the management for decision-making reference. The report also includes performance indicators and improvement suggestions for data asset management to help optimize the management process of data assets. Finally, the records are generated to generate the corresponding data asset information management report.

[0079] Further, step S1 includes the following steps:

[0080] Step S11: acquiring a subset of information related to data assets from an enterprise database data source in real time to obtain a subset of database source data asset information;

[0081] Step S12: obtaining a file system source data asset information subset by collecting a data asset-related information subset from a file system data source in real time;

[0082] Step S13: obtaining an API source data asset information subset by collecting an information subset related to the data asset from the API data source in real time;

[0083] Step S14: performing information set processing on the database source data asset information subset, the file system source data asset information subset and the API source data asset information subset to obtain a data asset information set; dividing the data asset information set into label structure sub-columns to obtain data asset information label description structured sub-columns;

[0084] Step S15: Create a time series synchronization process for each data asset information tag description structured sub-column and integrate and store it in the data asset management database.

[0085] As an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Detailed step flow diagram of step S1 in FIG. 1 , in this embodiment, step S1 includes the following steps:

[0086] Step S11: acquiring a subset of information related to data assets from an enterprise database data source in real time to obtain a subset of database source data asset information;

[0087] In an embodiment of the present invention, information data related to data assets is collected in real time in an enterprise database, and the database connection interface is configured to extract relevant data asset information from the target database table using SQL query language. This process obtains information related to data assets, such as the name of the database table, field type, number of records, etc., by executing specific SELECT statements. These query statements must accurately reflect the structure and content of the data assets and are executed regularly through scheduling tools (such as Apache Airflow) to ensure the real-time and accuracy of data collection, and finally obtain a subset of database source data asset information.

[0088] Step S12: obtaining a file system source data asset information subset by collecting a data asset-related information subset from a file system data source in real time;

[0089] In an embodiment of the present invention, information data related to data assets is collected from a file system data source in real time by using a corresponding file system retrieval tool (such as Python's os module or Java's NIO), so as to traverse a specified file directory according to screening criteria of file path and file type, identify qualified files, and for each file, by reading the file's metadata (such as file name, creation date, size, etc.) and file content (such as file text or structured data), finally retrieve and collect a subset of the file system source data asset information.

[0090] Step S13: obtaining an API source data asset information subset by collecting an information subset related to the data asset from the API data source in real time;

[0091] In an embodiment of the present invention, information data related to data assets is collected in real time from the corresponding API data source, and the parameters of the API request are configured, including the request method (such as GET or POST), request header and request body, and an API request is initiated by using an HTTP client library (such as Python's Requests library), and the returned JSON or XML data is obtained. The response data is parsed to extract information related to the data assets, such as asset ID, attribute description, etc. To ensure the real-time nature of data collection, a scheduled task or event trigger mechanism should be set to automatically trigger the API call, and finally integrate to obtain a subset of API source data asset information.

[0092] Step S14: performing information set processing on the database source data asset information subset, the file system source data asset information subset and the API source data asset information subset to obtain a data asset information set; dividing the data asset information set into label structure sub-columns to obtain data asset information label description structured sub-columns;

[0093] In an embodiment of the present invention, the database source data asset information subset, the file system source data asset information subset and the API source data asset information subset previously collected in real time from different data sources are processed for data integration, so as to standardize and format the information subsets obtained from different data sources, and merge the data from different sources by using a data integration tool (such as Apache NiFi or Talend), and remove duplicate and redundant information, and integrate each information subset by a data fusion algorithm (such as data matching and merging technology) to generate a unified data asset information set. This process needs to consider the compatibility and consistency of the data to ensure the accuracy and completeness of the final information set, so as to obtain a data asset information set. At the same time, by defining a label classification standard according to the actual content and structure of the data asset, and by using a data annotation tool (such as Labelbox or a custom script) to assign labels to the data asset information set, the data asset information is divided into different label categories (such as data asset type, data format, storage location, creation time and modification time, etc.), and by analyzing the attributes and relevance of the data, the label structure is subdivided into sub-columns of different label categories to ensure the accuracy and effectiveness of the label division, and finally obtain the data asset information label description structured sub-column.

[0094] Step S15: Create a time series synchronization process for each data asset information tag description structured sub-column and integrate and store it in the data asset management database.

[0095] In an embodiment of the present invention, each structured subcolumn described by the data asset information tag obtained by the previous division is synchronized according to the corresponding creation time to establish a time series data processing framework (such as Apache Kafka and Apache Flink) to achieve creation timestamp marking of each structured subcolumn, and the data is processed by using a time series data synchronization algorithm (such as time window technology) to ensure the time series consistency and integrity of the structured subcolumns described by the data asset information tag. At the same time, the data after time series synchronization processing is integrated and stored in a data asset management database (such as PostgreSQL or MongoDB), and a data backup and recovery mechanism is set to ensure long-term stable storage and access to data.

[0096] Furthermore, the step S14 of dividing the data asset information set into sub-columns of the label structure includes the following steps:

[0097] Performing information content semantic analysis on each piece of data asset information in the data asset information set to obtain content semantic data corresponding to the data asset content of each piece of data asset information;

[0098] In an embodiment of the present invention, natural language processing (NLP) technology is used to perform information content semantic recognition and analysis on each data asset information in a previously integrated data asset information set, so as to input the data asset information into a semantic analysis model. The model includes modules such as word segmentation, part-of-speech tagging, and named entity recognition. For example, pre-trained language models such as BERT or GPT-3 are used to extract keywords, entities, and the relationships between them in the data asset content through these models, so as to obtain content semantic data of each data asset information. These content semantic data can reflect the actual semantic meaning of the data assets, form semantic labels containing topics, contexts, and related concepts, and finally obtain content semantic data of the data asset content corresponding to each data asset information.

[0099] Preferably, data format attribute mining and analysis is performed on each data asset information in the data asset information set to obtain data format attributes of data asset content corresponding to each data asset information;

[0100] In an embodiment of the present invention, data format attributes of each data asset information in a data asset information set are mined and analyzed, so that during the operation, the information content of each data asset is formatted to identify its data structure and format characteristics, and the data is read and parsed by using a data format detection tool (such as the pandas library in Python or Apache Avro) to parse and obtain the format attributes of the data asset information, such as data type, field length, data structure, etc., and record them as data format attributes, and finally obtain the data format attributes of the data asset content corresponding to each data asset information.

[0101] Preferably, based on the data format attributes of the data asset content corresponding to each data asset information, the content semantic data of the data asset content corresponding to each data asset information is subjected to data asset content attribute classification and labeling processing to obtain various data content attribute labels corresponding to the data asset content, wherein the various data content attribute labels include creation time, data type, storage location, data usage and modification time;

[0102] In an embodiment of the present invention, the content attributes of the content semantic data of the data asset content corresponding to each data asset information are classified and labeled by combining the data format attributes of the data asset content corresponding to each data asset information obtained by previous mining and analysis, so as to classify and label the data asset content by combining the semantic data obtained by previous analysis with the format attributes obtained by mining and analysis, and analyze the data asset content by using a label classification algorithm (such as a decision tree, random forest and other machine learning algorithms) to identify the attributes of the data asset content such as creation time, data type, storage location, data usage and modification time. For example, the data assets are trained using a classification model to obtain label categories and corresponding attribute information, and the data content is marked and classified according to these attributes, and finally various data content attribute labels corresponding to the data asset content are obtained, wherein the various data content attribute labels include creation time, data type, storage location, data usage and modification time.

[0103] Preferably, attribute label description division is performed on each data asset information in the data asset information set according to various data content attribute labels corresponding to the data asset content, so as to obtain a data asset information content attribute label description sub-column;

[0104] In an embodiment of the present invention, attribute labels are described and divided for each corresponding data asset information in a data asset information set by combining various types of data content attribute labels corresponding to the data asset content obtained by previous analysis, so that the previously generated attribute labels are applied to the data asset information set, and each data asset is divided into different label description sub-columns according to its information content. During the implementation process, attribute labels are assigned to each data asset through a label description generation tool (such as Excel's filtering function or a custom script), and the data asset information under the same attribute label is divided into corresponding sub-columns. For example, corresponding label description sub-columns are generated for labels such as creation time and data type, so as to organize and classify the information content of each data asset according to these labels, form a detailed attribute label description, and finally obtain the data asset information content attribute label description sub-column.

[0105] Preferably, each data asset content in the data asset information content attribute label description sub-column is structurally transformed to obtain a data asset information label description structured sub-column.

[0106] In an embodiment of the present invention, a data conversion tool (such as the DataFrame structure of the pandas library in Python) is used to perform data structuring conversion processing on each data asset content in the data asset information content attribute label description subcolumn obtained through the previous division, so as to convert the attribute label description subcolumn obtained through the previous division into a structured data format, and by mapping the information in the attribute label description subcolumn according to the data field, a standardized data table is generated. For example, the label attribute fields such as creation time and data type are used as column headers, and the information of each data asset is filled in the corresponding column by row. This process can convert the original label description into a structured data format, and finally obtain a data asset information label description structured subcolumn.

[0107] Further, step S15 includes the following steps:

[0108] Step S151: Perform creation time label sub-column screening processing on each data asset information label description structured sub-column to obtain a data asset information creation time label structured sub-column and a data asset information remaining label structured sub-column;

[0109] Step S152: based on the data asset information creation time tag structured sub-column, perform creation time sequence synchronization processing on the remaining tag structured sub-columns of the data asset information to obtain the tag structured sub-columns of each data asset information under the same creation time sequence dimension;

[0110] Step S153: Designing the data integration storage format for each data asset information tag structured sub-column under the same creation time sequence dimension through the data asset management database in the data processing unit to obtain the data asset information tag sub-column integration storage format specification;

[0111] Step S154: Integrate and store each data asset information tag structured sub-column under the same creation time sequence dimension into the data asset management database in the data processing unit according to the data asset information tag sub-column integration storage format specification.

[0112] As an embodiment of the present invention, refer to Figure 3 As shown, Figure 2 Detailed step flow diagram of step S15 in FIG. 1 , in this embodiment, step S15 includes the following steps:

[0113] Step S151: Perform creation time label sub-column screening processing on each data asset information label description structured sub-column to obtain a data asset information creation time label structured sub-column and a data asset information remaining label structured sub-column;

[0114] In an embodiment of the present invention, records containing creation time label descriptions are retrieved from each data asset information label description structured subcolumn obtained after previous structured conversion to extract the creation time label subcolumn. This can be accomplished through SQL queries or filtering functions in data analysis tools. For example, the query SELECT label_description,creation_time FROM data_assets WHERE label_description IS NOTNULL is executed to obtain all data asset information records with creation time label descriptions, thereby storing the creation time label subcolumn separately from the remaining label subcolumns, and finally obtaining the data asset information creation time label structured subcolumn and the data asset information remaining label structured subcolumns.

[0115] Step S152: based on the data asset information creation time tag structured sub-column, perform creation time sequence synchronization processing on the remaining tag structured sub-columns of the data asset information to obtain the tag structured sub-columns of each data asset information under the same creation time sequence dimension;

[0116] In an embodiment of the present invention, the remaining label structured sub-columns of the filtered data asset information are synchronized in time series by combining the previously extracted data asset information creation time label structured sub-column, including associating the creation time label with the records of each data asset information label, and then sorting them according to the creation time to ensure that all label information is aligned in the time dimension. This can be done by using the python code data_frame.sort_values(by='creation_time') to sort the remaining label structured sub-columns, and applying the sorted data to other label columns, thereby synchronizing under the same time sequence dimension, and finally obtaining each data asset information label structured sub-column under the same creation time sequence dimension.

[0117] Step S153: Designing the data integration storage format for each data asset information tag structured sub-column under the same creation time sequence dimension through the data asset management database in the data processing unit to obtain the data asset information tag sub-column integration storage format specification;

[0118] In an embodiment of the present invention, a data asset management database preset in a data processing unit is used to design a storage format after data integration for each data asset information label structured sub-column under the same creation time sequence dimension that has previously undergone time series synchronization processing, so as to define a storage format specification, including column name, data type, and index setting. For example, a table structure can be created, which contains fields such as "creation time", "label 1", and "label 2". The data types of the fields are VARCHAR, DATE, etc., and indexes are set to improve query performance. A corresponding table structure is created through a database design tool or an SQL script, such as executing CREATE TABLE asset_data(creation_time DATE, label1 VARCHAR(255), label2 VARCHAR(255)) to ensure that the data asset information can be integrated and stored in the defined format, and finally a data asset information label sub-column integrated storage format specification is designed.

[0119] Step S154: Integrate and store each data asset information tag structured sub-column under the same creation time sequence dimension into the data asset management database in the data processing unit according to the data asset information tag sub-column integration storage format specification.

[0120] In an embodiment of the present invention, by combining the data asset information label sub-column integration storage format specifications obtained in the previous design, each data asset information label structured sub-column under the same creation time sequence dimension obtained after time series synchronization processing is integrated and stored in the data asset management database of the data processing unit. The specific operation includes importing the previously obtained synchronized data into the database, so as to insert the data into a pre-created table by using a data import tool or SQL command, for example, by executing INSERT INTO asset_data(creation_time,label1,label2)VALUES(data_creation_time,data_label1,data_label2), the data is inserted into the database table, ensuring that the data format complies with the design specifications during the insertion process, and performing necessary data verification and error handling to maintain the accuracy and consistency of the data.

[0121] Further, step S2 includes the following steps:

[0122] Step S21: Perform access log recording processing on each data asset sub-information in the data asset management database to obtain data access log record information of each data asset sub-information;

[0123] In an embodiment of the present invention, a detailed access log is generated by using a logging tool (such as ELK Stack or Splunk) for the access behavior of each data asset sub-information in the data asset management database. Each access event to the data asset sub-information, including user ID, access time, access type (read, modify, delete, etc.) will be recorded. The log data is collected through a specific API interface and stored in the log database to ensure that each access to each data asset sub-information can be fully recorded to ensure the real-time and integrity of the access log, and finally record the data access log record information for each data asset sub-information.

[0124] Step S22: Analyze the number of accesses and time distribution of the data access log record information of each data asset sub-information to obtain the number of data accesses and data access time distribution of each data asset sub-information;

[0125] In an embodiment of the present invention, data access log record information of each data asset sub-information in the log database is analyzed by using a data analysis tool (such as Apache Hadoop or the Pandas library in Python) to statistically analyze the number of accesses to each data asset sub-information and draw a time distribution graph. The number of accesses is calculated by querying the relevant fields in the log database, and the time distribution analysis is performed by counting the access timestamps to generate an access heat map. This step ensures the visualization of the data access frequency and its time distribution, and finally obtains the data access number and data access time distribution of each data asset sub-information.

[0126] Step S23: Perform usage frequency statistics analysis on the number of data accesses of each data asset sub-information based on the data access time distribution of each data asset sub-information to obtain the data usage frequency of each data asset sub-information;

[0127] In an embodiment of the present invention, a statistical analysis tool (such as R language or Python's Scipy library) is used to perform a statistical calculation of the frequency of data access corresponding to each data asset sub-information by combining the data access time distribution of each data asset sub-information obtained in the previous analysis, so as to quantitatively calculate the access frequency of each data asset sub-information. The access frequency statistics are obtained by aggregating the time distribution data by hour, day or week to obtain the frequency distribution of access, and finally the data usage frequency of each data asset sub-information is obtained.

[0128] Step S24: using the data asset information sensitivity calculation formula to perform information sensitivity evaluation calculation on each data asset sub-information in the data asset management database, and obtaining the data sensitivity level of each data asset sub-information;

[0129] In an embodiment of the present invention, a suitable data asset information sensitivity calculation formula is constructed by combining the quantity measurement parameters of data asset sub-information, data asset availability parameters, data asset availability impact weight coefficient, leakage risk measurement parameters, data asset leakage risk impact weight coefficient, leakage risk control impact exponential decay coefficient, data asset confidentiality parameters, data asset confidentiality impact weight coefficient, quantity measurement parameters of additional attribute information, specific values ​​on additional attribute information, impact weight coefficient of additional attribute information and related parameters to perform information sensitivity assessment calculation on each data asset sub-information in the data asset management database, so as to quantitatively calculate the corresponding sensitivity degree value, and finally obtain the data sensitivity degree of each data asset sub-information.

[0130] Step S25: Based on the data usage frequency and data sensitivity of each data asset sub-information, a data asset access utility evaluation and analysis is performed on each data asset sub-information corresponding to the data asset management database to obtain the data access utility of each data asset sub-information.

[0131] In the embodiment of the present invention, the access utility of each data asset sub-information corresponding to the data asset management database is evaluated and calculated by using a weighted scoring method in combination with the data usage frequency and data sensitivity of each data asset sub-information obtained by previous statistical analysis, so as to calculate the scores of the two dimensions according to the usage frequency and the sensitivity, and combine the scores of the two dimensions to obtain the final access utility score by a weighted average method. This step is usually automatically calculated using a data analysis tool or a custom script. For example, the corresponding calculation formula can be: In the formula, E k Score the final access utility of the i-th data asset sub-information, F i is the data usage frequency of the i-th data asset sub-information, a is the usage frequency weight coefficient, S i is the data sensitivity of the ith data asset sub-information, and b is the sensitivity weight coefficient, which ensures the accuracy and objectivity of the evaluation results, and finally obtains the data access utility of each data asset sub-information.

[0132] Furthermore, the calculation formula for the data asset information sensitivity in step S24 is specifically:

[0133]

[0134] In the formula, S i is the data sensitivity of the i-th data asset sub-information, n is the total number of data asset sub-information in the data asset management database, i is the item index parameter of the data asset sub-information, R iis the data asset availability parameter of the i-th data asset sub-information, α is the data asset availability impact weight coefficient, D i is the leakage risk measurement parameter of the i-th data asset sub-information, β is the weight coefficient of the data asset leakage risk impact, γ is the exponential attenuation coefficient of the control leakage risk impact, C i is the data asset confidentiality parameter of the i-th data asset sub-information, δ is the data asset confidentiality impact weight coefficient, m is the total number of additional attribute information in the data asset sub-information, j is the item measurement parameter of the additional attribute information, P i,j is the specific value of the i-th data asset sub-information on the j-th additional attribute information, κ j is the influence weight coefficient of the jth additional attribute information, and η is the correction coefficient of the data sensitivity.

[0135] The present invention obtains a data asset information sensitivity calculation formula by using a specific mathematical model and after verification, which is used to evaluate the information sensitivity of each data asset sub-information in the data asset management database. The data asset information sensitivity calculation formula comprehensively considers data availability, leakage risk, confidentiality and additional attributes, making the evaluation of data sensitivity more comprehensive and accurate. Through the exponential decay of leakage risk and the logarithmic adjustment of confidentiality, the formula can dynamically reflect the changes in data sensitivity and adapt to different risk scenarios. Secondly, the weight coefficients (such as α, β, δ, κ) in the calculation formula j ) allows the relative importance of influencing factors to be adjusted according to actual needs, thereby improving the flexibility and accuracy of sensitivity assessment. In addition, the formula also introduces a correction factor to adjust the sensitivity level according to actual conditions, thereby enhancing the applicability and reliability of the calculation formula. By evaluating data access frequency and sensitivity, it can effectively support data management decisions, optimize data access control and protection strategies, and thus improve the management efficiency of data assets. Therefore, this formula fully considers the data sensitivity level S of the i-th data asset sub-information i , the total number of data asset sub-information n in the data asset management database, the item index parameter i of the data asset sub-information, and the data asset availability parameter R of the i-th data asset sub-information i , data asset availability impact weight coefficient α, leakage risk measurement parameter D of the i-th data asset sub-information i , data asset leakage risk impact weight coefficient β, control leakage risk impact exponential decay coefficient γ, data asset confidentiality parameter C of the i-th data asset sub-information i , data asset confidentiality impact weight coefficient δ, the total number of additional attribute information in the data asset sub-information m, the item measurement parameter j of the additional attribute information, the specific value P of the i-th data asset sub-information on the j-th additional attribute information i,j, the influence weight coefficient κ of the jth additional attribute information j , the correction coefficient of data sensitivity η, according to the data sensitivity S of the i-th data asset sub-information i The correlation between the above parameters forms a functional relationship:

[0136]

[0137] This formula can realize the information sensitivity evaluation calculation process for each data asset sub-information in the data asset management database. At the same time, by introducing the correction coefficient η of the data sensitivity level, it can be adjusted according to the errors occurring in the calculation process, thereby improving the accuracy and applicability of the data asset information sensitivity calculation formula.

[0138] Further, step S25 includes the following steps:

[0139] Step S251: classifying the data sensitivity level of each data asset sub-information corresponding to each data asset sub-information in the data asset management database based on the data sensitivity level of each data asset sub-information, and obtaining the data sensitivity level of each data asset sub-information;

[0140] In an embodiment of the present invention, the sensitivity level of each data asset sub-information corresponding to each data asset sub-information in the data asset management database is divided by combining the data sensitivity level of each data asset sub-information obtained by previous quantitative calculation, so as to divide it according to a pre-set sensitivity classification standard (such as high (for example, the data sensitivity level is 86-100), medium (for example, the data sensitivity level is 56-85), and low (for example, the data sensitivity level is 0-55)). The classification standard can be based on factors such as the data type and the privacy level of the personal or business information involved, and adopts specific methods, such as data classification tools or manual review methods, to ensure that each data asset sub-information is appropriately identified with a sensitivity level, and finally the data sensitivity level of each data asset sub-information is divided.

[0141] Step S252: performing access utility weighted analysis on each data asset sub-information corresponding to each data asset sub-information in the data asset management database based on the data usage frequency and data sensitivity level of each data asset sub-information, so as to obtain a data access utility weighted value set for each data asset sub-information;

[0142] In an embodiment of the present invention, access utility weighted analysis is performed on the corresponding data asset sub-information in the data asset management database by combining the data usage frequency of each data asset sub-information obtained by previous statistical analysis and the data sensitivity level obtained by classification, so as to analyze and obtain the access utility weighted weight coefficient corresponding to the data sensitivity level and the usage frequency, thereby obtaining the corresponding weighted value set, including the sensitivity weight coefficient and the usage frequency weight coefficient, which can reflect the utility impact parameters of the corresponding usage frequency and sensitivity level of each data asset sub-information in actual access, and finally obtain the data access utility weighted value set for each data asset sub-information.

[0143] Step S253: Perform data asset access utility evaluation and analysis on each data asset sub-information corresponding to the data asset management database according to the data access utility weighted value set of each data asset sub-information to obtain the data access utility of each data asset sub-information.

[0144] In the embodiment of the present invention, a weighted average calculation is performed based on the data access utility weighted value set of each data asset sub-information obtained by the previous analysis and in combination with the data usage frequency and data sensitivity of the corresponding data asset sub-information in the data asset management database to quantitatively calculate the final access utility score. For example, the corresponding calculation formula may be: In the formula, E k Score the final access utility of the i-th data asset sub-information, F i is the data usage frequency of the i-th data asset sub-information, a is the usage frequency weight coefficient, S i is the data sensitivity of the ith data asset sub-information, b is the sensitivity weight coefficient, and the data access utility of each data asset sub-information is finally quantitatively calculated.

[0145] Further, step S3 includes the following steps:

[0146] Step S31: Performing management role access business requirement analysis on each data asset sub-information in the data asset management database to obtain the database management role access business requirement for each data asset sub-information;

[0147] In an embodiment of the present invention, a business demand analysis of management role access is performed on each data asset sub-information in the data asset management database, so as to collect the management role and corresponding business demand of each data asset sub-information. This process is completed by accessing business demand documents, interviewing management personnel, and analyzing existing permission settings. The management role information is extracted by using permission management tools (such as LDAP or Active Directory), and combined with business demand analysis tools (such as business process modeling tools BPMN) to record in detail the access requirements of each role to the data assets. For example, if a data asset sub-information involves financial data, the access requirements of the finance department role will be recorded in detail and associated with the data asset, and finally the database management role access business requirements of each data asset sub-information are obtained.

[0148] Step S32: performing data usage constraint analysis on each data asset sub-information in the data asset management database based on the data usage frequency of each data asset sub-information to obtain data usage constraint conditions for each data asset sub-information;

[0149] In an embodiment of the present invention, a data usage constraint analysis is performed on the corresponding data asset sub-information in the data asset management database by combining the data usage frequency of each data asset sub-information obtained by previous analysis, so as to process the access log of the data asset by using a data analysis platform (such as Apache Spark or Python's Pandas library), calculate the usage frequency of each data asset sub-information, and define data usage constraints based on the analysis results, such as implementing stricter access control on frequently accessed sensitive data, combining usage frequency data with business needs, and setting reasonable data usage restrictions. For example, if the access frequency of a data asset sub-information is high, it is necessary to limit the access time period or number of accesses to the data to reduce the risk of data leakage, and finally obtain the data usage constraint for each data asset sub-information.

[0150] Step S33: performing access utility restriction analysis on each data asset sub-information in the data asset management database based on the data access utility of each data asset sub-information, and obtaining the data access utility restriction range of each data asset sub-information;

[0151] In an embodiment of the present invention, by combining the data access utility of each data asset sub-information obtained by previous quantitative calculation, an access utility restriction analysis is performed on the corresponding data asset sub-information in the data asset management database, so as to generate an access utility chart by using a data visualization tool (such as Tableau or Microsoft Power BI) to determine the utility range of data access, wherein the data access utility restriction range is set according to the access frequency and data sensitivity. For example, for high-utility and high-sensitivity data assets, strict access control restrictions are set, such as restricting access rights or strengthening identity authentication requirements, and finally the data access utility restriction range of each data asset sub-information is obtained.

[0152] Step S34: Based on the data usage constraints and data access utility restriction range of each data asset sub-information, dynamic access permission control is performed on the database management role access business requirements of each data asset sub-information to obtain dynamic access control permissions for the data asset management database.

[0153] In an embodiment of the present invention, dynamic access rights are controlled by combining the data usage constraints and data access utility restriction range of each data asset sub-information previously analyzed with the database management role access business needs of each data asset sub-information previously analyzed, so as to set it by utilizing a dynamic permission control system (such as an RBAC-based permission management system or an ABAC-based access control policy), combining the data usage constraints and the access utility restriction range with the business needs of the management role, and creating a dynamic access control policy. The implementation process includes configuring the policy rules of the permission management system, and dynamically adjusting the role permissions to meet the requirements of data usage and access utility. For example, for a highly sensitive and high access utility data asset, permissions will be automatically restricted to only necessary management roles, and stricter access auditing and control measures will be implemented, ultimately obtaining dynamic access control permissions for the data asset management database.

[0154] Further, step S4 includes the following steps:

[0155] Step S41: performing management role authority matching and mapping on the dynamic access control authority of the data asset management database to obtain the data asset management role dynamic access matching and mapping control authority;

[0156] In an embodiment of the present invention, the dynamic access control permissions in the data asset management database obtained by previous analysis are matched and mapped with the permissions of the management roles, so as to extract the access permission configuration of each management role from the permission management system, and map these permissions to the corresponding data asset information, and use a permission mapping tool (such as the role mapping module in the RBAC system) to perform mapping, match the permissions of each management role with the access requirements of the data assets, and determine the dynamic access permissions of each role to specific data assets through access control lists (ACLs) or permission configuration tables. The generated mapping results include the role and its corresponding data asset access permissions, and finally obtain the dynamic access matching and mapping control permissions of the data asset management role.

[0157] Step S42: Based on the data asset management role dynamic access matching mapping control authority, the corresponding data asset information in the data asset management database is analyzed for the full life cycle stage authority characteristics to obtain the data asset full life cycle stage management role authority characteristic set;

[0158] In an embodiment of the present invention, a full life cycle permission feature analysis is performed on the corresponding data asset information in the data asset management database by combining the dynamic access matching mapping control permissions of the data asset management role obtained by previous mapping assignment, so as to analyze the various life cycle stages (such as creation, storage, use, sharing, and destruction) of data assets through life cycle management tools (such as enterprise data management systems), record the management roles and their permission features involved in each stage, and analyze the permission requirements and restrictions of the roles at different life cycle stages. For example, in the data creation stage, a role with higher permissions is required to enter data, while in the data use stage, it is necessary to ensure that only specific roles can access it. The generated full life cycle stage management role permission feature set includes the permission requirements and feature descriptions of each life cycle stage, and finally a full life cycle stage management role permission feature set of data assets is obtained.

[0159] Step S43: Formulate full life cycle management rules for the management role authority feature set of the data asset full life cycle stage to generate full life cycle management rules for data asset information;

[0160] In an embodiment of the present invention, full life cycle management rules are formulated for the set of management role permission features of the full life cycle stages of data assets obtained by previous analysis, so as to create full life cycle management rules for data asset information by using a rule engine (such as Drools) or a policy formulation tool. The permission feature set obtained by previous analysis is converted into management rules. For example, access control policies and operation requirements are defined for different life cycle stages of data. The formulated rules include how to implement permission control, monitoring and auditing requirements at different stages of data to ensure that the management of data assets throughout the entire life cycle complies with the set policies and security standards, and finally the corresponding full life cycle management rules for data asset information are formulated and generated.

[0161] Step S44: Based on the data asset information full life cycle management rules, the corresponding data asset information in the data asset management database is automatically managed and processed to obtain the data asset information full life cycle automatic management results;

[0162] In an embodiment of the present invention, the corresponding data asset information in the data asset management database is automatically managed and processed by combining the data asset information full life cycle management rules obtained through previous analysis, so that the management rules can be applied to data assets by utilizing an automated management system (such as a data governance platform or an automated workflow management system), and data access control, permission allocation, data auditing and compliance checks are automatically performed according to the rules. For example, when data enters a new life cycle stage, access rights are automatically adjusted, permission settings are updated, and necessary management operations are performed. The results include the automated management status and related logs of the data asset information throughout its life cycle, and ultimately the automated management results of the data asset information throughout its life cycle are obtained.

[0163] Step S45: Record the management report feedback of the automatic management results of the data asset information throughout its life cycle to generate a data asset information management report.

[0164] In an embodiment of the present invention, a report generation tool (such as Microsoft Power BI or Tableau) is used to record the feedback of the management report on the automatic management results of the entire life cycle of data asset information previously obtained after automated management, so as to summarize the automatic management processing results using a report generation tool (such as Microsoft Power BI or Tableau) and generate a detailed management report. The report content includes the execution status of automated management, the effectiveness of authority control, the management effect of the life cycle stage, etc. Feedback records are generated regularly through the reporting system and provided to the management for decision-making reference. The report also includes performance indicators and improvement suggestions for data asset management to help optimize the management process of data assets. Finally, the records are generated to generate the corresponding data asset information management report.

[0165] Furthermore, the present invention also provides a data asset management system for executing the data asset management method as described above, the data asset management system comprising:

[0166] The data asset information structured storage module is used to collect data asset information sets from databases, file systems, and API data sources in real time, and divide the data asset information sets into tag structure sub-columns to obtain data asset information tag description structured sub-columns; create time-series synchronization processing for each data asset information tag description structured sub-column and integrate and store them in the data asset management database;

[0167] The usage frequency and access utility analysis module is used to perform usage frequency and sensitivity evaluation analysis on each data asset sub-information in the data asset management database, and obtain the data usage frequency and data sensitivity of each data asset sub-information; based on the data usage frequency and data sensitivity of each data asset sub-information, perform data asset access utility evaluation analysis on each data asset sub-information in the data asset management database, and obtain the data access utility of each data asset sub-information;

[0168] A database dynamic access permission control module is used to dynamically control the access permission of each data asset sub-information in the data asset management database based on the data usage frequency and data access utility of each data asset sub-information, thereby obtaining dynamic access control permissions for the data asset management database;

[0169] The data asset full life cycle management module is used to perform full life cycle management design for the corresponding data asset information in the data asset management database based on the dynamic access control permissions of the data asset management database, so as to generate full life cycle management rules for data asset information; based on the full life cycle management rules for data asset information, the corresponding data asset information in the data asset management database is automatically managed and report feedback records are recorded to generate data asset information management reports.

[0170] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0171] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A data asset management method, characterized in that: The following steps are involved: Step S1: by collecting data asset information sets from enterprise databases, file systems, and API data sources in real time, and dividing the data asset information sets into tag structure sub-columns to obtain data asset information tag description structured sub-columns; creating time series synchronization processing for each data asset information tag description structured sub-column and integrating and storing them in the data asset management database in the data processing unit; Step S2: Performing a usage frequency and sensitivity evaluation analysis on each data asset sub-information in the data asset management database to obtain the data usage frequency and data sensitivity of each data asset sub-information; performing a data asset access utility evaluation analysis on each data asset sub-information in the data asset management database based on the data usage frequency and data sensitivity of each data asset sub-information to obtain the data access utility of each data asset sub-information; Step S3: Dynamically control the access rights of each data asset sub-information in the data asset management database based on the data usage frequency and data access utility of each data asset sub-information, and obtain the dynamic access control rights of the data asset management database; Step S4: Perform full life cycle management design on the corresponding data asset information in the data asset management database according to the dynamic access control authority of the data asset management database to generate full life cycle management rules for the data asset information; Based on the data asset information full life cycle management rules, the corresponding data asset information in the data asset management database is automatically managed and report feedback is recorded to generate a data asset information management report.

2. The data asset management method according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: acquiring a subset of information related to data assets from an enterprise database data source in real time to obtain a subset of database source data asset information; Step S12: obtaining a file system source data asset information subset by collecting a data asset-related information subset from a file system data source in real time; Step S13: obtaining an API source data asset information subset by collecting an information subset related to the data asset from the API data source in real time; Step S14: performing information set processing on the database source data asset information subset, the file system source data asset information subset and the API source data asset information subset to obtain a data asset information set; dividing the data asset information set into label structure sub-columns to obtain data asset information label description structured sub-columns; Step S15: Create a time series synchronization process for each data asset information tag description structured sub-column and integrate and store it in the data asset management database.

3. The data asset management method according to claim 2, characterized in that: The step S14 of dividing the data asset information set into sub-columns of the label structure includes the following steps: Performing information content semantic analysis on each piece of data asset information in the data asset information set to obtain content semantic data corresponding to the data asset content of each piece of data asset information; Perform data format attribute mining and analysis on each data asset information in the data asset information set to obtain the data format attribute of each data asset information corresponding to the data asset content; Based on the data format attributes of the data asset content corresponding to each data asset information, the content semantic data of the data asset content corresponding to each data asset information is classified and labeled to obtain various data content attribute labels corresponding to the data asset content, wherein various data content attribute labels include creation time, data type, storage location, data purpose and modification time; According to various data content attribute labels corresponding to the data asset content, each data asset information in the data asset information set is divided into attribute label descriptions to obtain a data asset information content attribute label description sub-column; Each data asset content in the data asset information content attribute label description sub-column is structurally transformed to obtain a data asset information label description structured sub-column.

4. The data asset management method according to claim 2, characterized in that: Step S15 includes the following steps: Step S151: Perform creation time label sub-column screening processing on each data asset information label description structured sub-column to obtain a data asset information creation time label structured sub-column and a data asset information remaining label structured sub-column; Step S152: based on the data asset information creation time tag structured sub-column, perform creation time sequence synchronization processing on the remaining tag structured sub-columns of the data asset information to obtain the tag structured sub-columns of each data asset information under the same creation time sequence dimension; Step S153: Designing the data integration storage format for each data asset information tag structured sub-column under the same creation time sequence dimension through the data asset management database in the data processing unit to obtain the data asset information tag sub-column integration storage format specification; Step S154: Integrate and store each data asset information tag structured sub-column under the same creation time sequence dimension into the data asset management database in the data processing unit according to the data asset information tag sub-column integration storage format specification.

5. The data asset management method according to claim 1, characterized in that: Step S2 includes the following steps: Step S21: Perform access log recording processing on each data asset sub-information in the data asset management database to obtain data access log record information of each data asset sub-information; Step S22: Analyze the number of accesses and time distribution of the data access log record information of each data asset sub-information to obtain the number of data accesses and data access time distribution of each data asset sub-information; Step S23: Perform usage frequency statistics analysis on the number of data accesses of each data asset sub-information based on the data access time distribution of each data asset sub-information to obtain the data usage frequency of each data asset sub-information; Step S24: using the data asset information sensitivity calculation formula to perform information sensitivity evaluation calculation on each data asset sub-information in the data asset management database, and obtaining the data sensitivity level of each data asset sub-information; Step S25: Based on the data usage frequency and data sensitivity of each data asset sub-information, a data asset access utility evaluation and analysis is performed on each data asset sub-information corresponding to the data asset management database to obtain the data access utility of each data asset sub-information.

6. The data asset management method according to claim 5, characterized in that: The data asset information sensitivity calculation formula in step S24 is specifically: In the formula, S i is the data sensitivity of the i-th data asset sub-information, n is the total number of data asset sub-information in the data asset management database, i is the item index parameter of the data asset sub-information, R i is the data asset availability parameter of the i-th data asset sub-information, α is the data asset availability impact weight coefficient, D i is the leakage risk measurement parameter of the i-th data asset sub-information, β is the weight coefficient of the data asset leakage risk impact, γ is the exponential attenuation coefficient of the control leakage risk impact, C i is the data asset confidentiality parameter of the i-th data asset sub-information, δ is the data asset confidentiality impact weight coefficient, m is the total number of additional attribute information in the data asset sub-information, j is the item measurement parameter of the additional attribute information, P i，j is the specific value of the i-th data asset sub-information in the j-th additional attribute information, k j is the influence weight coefficient of the jth additional attribute information, and η is the correction coefficient of the data sensitivity.

7. The data asset management method according to claim 5, characterized in that: Step S25 includes the following steps: Step S25 1: Based on the data sensitivity of each data asset sub-information, classify the data sensitivity level of each data asset sub-information corresponding to the data asset management database to obtain the data sensitivity level of each data asset sub-information; Step S252: performing access utility weighted analysis on each data asset sub-information corresponding to each data asset sub-information in the data asset management database based on the data usage frequency and data sensitivity level of each data asset sub-information, so as to obtain a data access utility weighted value set for each data asset sub-information; Step S253: Perform data asset access utility evaluation and analysis on each data asset sub-information corresponding to the data asset management database according to the data access utility weighted value set of each data asset sub-information to obtain the data access utility of each data asset sub-information.

8. The data asset management method according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: Perform management role access business requirement analysis on each data asset sub-information in the data asset management database to obtain the database management role access business requirement of each data asset sub-information; Step S32: performing data usage constraint analysis on each piece of data asset sub-information in the data asset management database based on the data usage frequency of each piece of data asset sub-information, and obtaining data usage constraint conditions for each piece of data asset sub-information; Step S33: performing access utility restriction analysis on each data asset sub-information in the data asset management database based on the data access utility of each data asset sub-information, and obtaining the data access utility restriction range of each data asset sub-information; Step S34: Based on the data usage constraints and data access utility restriction range of each data asset sub-information, dynamic access permission control is performed on the database management role access business requirements of each data asset sub-information to obtain dynamic access control permissions for the data asset management database.

9. The data asset management method according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing management role authority matching and mapping on the dynamic access control authority of the data asset management database to obtain the data asset management role dynamic access matching and mapping control authority; Step S42: Based on the data asset management role dynamic access matching mapping control authority, the corresponding data asset information in the data asset management database is analyzed for the full life cycle stage authority characteristics to obtain the data asset full life cycle stage management role authority characteristic set; Step S43: Formulate full life cycle management rules for the management role authority feature set of the data asset full life cycle stage to generate full life cycle management rules for data asset information; Step S44: Based on the data asset information full life cycle management rules, the corresponding data asset information in the data asset management database is automatically managed and processed to obtain the data asset information full life cycle automatic management results; Step S45: Record the management report feedback of the automatic management results of the data asset information throughout its life cycle to generate a data asset information management report.

10. A data asset management system, characterized in that: Used to execute the data asset management method according to claim 1, the data asset management system comprises: The data asset information structured storage module is used to collect data asset information sets from databases, file systems, and API data sources in real time, and divide the data asset information sets into tag structure sub-columns to obtain data asset information tag description structured sub-columns; create time-series synchronization processing for each data asset information tag description structured sub-column and integrate and store them in the data asset management database; The usage frequency and access utility analysis module is used to perform usage frequency and sensitivity evaluation analysis on each data asset sub-information in the data asset management database, and obtain the data usage frequency and data sensitivity of each data asset sub-information; based on the data usage frequency and data sensitivity of each data asset sub-information, perform data asset access utility evaluation analysis on each data asset sub-information in the data asset management database, and obtain the data access utility of each data asset sub-information; A database dynamic access permission control module is used to dynamically control the access permission of each data asset sub-information in the data asset management database based on the data usage frequency and data access utility of each data asset sub-information, thereby obtaining dynamic access control permissions for the data asset management database; The data asset full life cycle management module is used to perform full life cycle management design for the corresponding data asset information in the data asset management database based on the dynamic access control permissions of the data asset management database, so as to generate full life cycle management rules for data asset information; based on the full life cycle management rules for data asset information, the corresponding data asset information in the data asset management database is automatically managed and report feedback records are recorded to generate data asset information management reports.

Images