Data asset management method and system

Abstract

The invention is suitable for the technical field of data asset management, and provides a data asset management method and system, and the method comprises the steps: determining an appointed specific type of a target data asset, extracting a transmission and storage operation log corresponding to the appointed specific type from a log management platform of a target enterprise, and obtaining an initial sensitivity index of the target data asset. According to the method, the influence scale factor and the implicit relevance variation amplitude of the data resources in the migration of the storage environment are comprehensively considered, so that the dynamic adjustment of the sensitivity index is realized, and the method has remarkable beneficial effects. Firstly, the scheme can dynamically adapt to the change of a storage environment, and the sensitivity evaluation is optimized in real time according to the actual migration frequency and implicit relevance change of data, so that the flexibility is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data asset management, and in particular, relates to a data asset management method and system. Background Art

[0002] With the rapid growth of data scale and the diversification of storage environments, the sensitivity assessment and management of data assets have gradually become the core issues of enterprise data governance. Existing technologies usually use static rules to assess the sensitivity of data assets, such as fixed sensitivity level classification based on data type, content or business scenario. This static sensitivity assessment method has certain effects when the data generation and storage environment are stable, but in the face of complex multi-storage environments and multi-node migration scenarios, it cannot effectively reflect the actual impact of storage environment changes on the implicit correlation and sensitivity of data.

[0003] In current technical solutions, although there are some static protection measures for sensitive data, such as classified storage, encryption processing or access permission settings based on fixed rules, these methods ignore the dynamic impact of storage environment migration on data sensitivity. For example, data migration between different storage nodes may lead to enhanced implicit correlation and increase the potential risk of privacy leakage, while existing technologies fail to capture and adjust sensitivity assessment results in real time. At the same time, in the case of insufficient sensitivity, highly sensitive data may not be adequately protected, while low-sensitivity data may occupy too many resources, making it impossible to achieve reasonable resource allocation and maximize storage efficiency.

[0004] It can be seen that the existing technology has significant deficiencies in the dynamic adjustment of sensitivity assessment. Its main defects include: it cannot fully reflect the changes in the implicit correlation of data during the migration of the storage environment, the assessment results are static and lack of pertinence, and it is difficult to provide appropriate protection for highly sensitive data. At the same time, the assessment bias caused by insufficient sensitivity may further affect the classification operation of data storage and the implementation effect of protection measures. Summary of the invention

[0005] The purpose of the present invention is to provide a data asset management method and system, aiming to solve the problems raised in the background technology.

[0006] The present invention is implemented as follows: a data asset management method, the method comprising:

[0007] Determine the designated specific category of the target data asset, and extract the transmission and storage operation logs corresponding to the designated specific category from the log management platform of the target enterprise to obtain the initial sensitivity index of the target data asset;

[0008] Filter the data resources that exceed the preset storage time limit in the transmission and storage operation logs, and aggregate these data resources into an analysis data set, and statistically analyze the proportion of data resources that have undergone storage environment migration in the analysis data set, and set the proportion as the impact ratio factor;

[0009] Select a standard reference sample from the analysis data set, the selection condition is that the data resource has a storage environment migration record of more than a preset number of times and is most consistent with the characteristic information of the target data resource, and extract the secondary storage operation log of the standard reference sample;

[0010] Parse the secondary storage operation logs of the standard reference sample, quantify the change range of implicit correlation each time the storage environment is migrated, and calculate the average change range;

[0011] Based on the impact scale factor and the average change amplitude, the initial sensitivity index of the target data resource is dynamically adjusted.

[0012] As a further limitation of the technical solution of the embodiment of the present invention, the storage environment migration includes migration from a local system to a cloud platform, distributed migration across cloud nodes, and switching of storage media.

[0013] As a further limitation of the technical solution of the embodiment of the present invention, the steps of screening data resources exceeding a preset storage time limit in the transmission and storage operation logs, aggregating these data resources into an analysis data set, statistically analyzing the proportion of data resources that have undergone storage environment migration in the analysis data set, and setting the proportion as an influencing proportional factor include:

[0014] Parse the transmission and storage operation logs to determine the storage time limit and storage environment migration records bound to each data resource;

[0015] Filter out data resources that have exceeded the preset storage limit and aggregate them into an analytical data set;

[0016] Analyze the data resources in the data set that have undergone storage environment migration, calculate their proportions, and set them as impact ratio factors.

[0017] As a further limitation of the technical solution of the embodiment of the present invention, the steps of selecting a standard reference sample from the analysis data set, selecting a data resource having a storage environment migration record of more than a preset number of times and most consistent with the characteristic information of the target data resource, and extracting the secondary storage operation log of the standard reference sample include:

[0018] Filter out all data resources with storage environment migration records from the analysis data set, and count the number of storage environment migrations for each data resource;

[0019] Among the screened data resources, select data resources with storage environment migration records of more than a preset number of times, and further screen out standard reference samples with the highest similarity to the target data resources based on characteristic information such as data structure, purpose and scale;

[0020] Extract the secondary storage operation log of the standard reference sample.

[0021] As a further limitation of the technical solution of the embodiment of the present invention, the steps of parsing the secondary storage operation log of the standard reference sample, quantifying the change range of the implicit correlation each time the storage environment is migrated, and calculating the average change range include:

[0022] Extracting implicit correlation data related to each storage environment migration from the secondary storage operation log of the standard reference sample, wherein the implicit correlation data includes the number of access path changes;

[0023] Analyze the extracted implicit correlation data and calculate the change range of implicit correlation caused by each storage environment migration;

[0024] Sum up the implicit affinity changes of all secondary storage environment migrations and calculate their average change.

[0025] As a further limitation of the technical solution of the embodiment of the present invention, the step of dynamically adjusting the initial sensitivity index of the target data resource based on the impact scale factor and the average change amplitude includes:

[0026] Retrieve the sensitivity index optimization formula, and dynamically adjust the initial sensitivity index of the target data resource in combination with the impact scale factor and the average change amplitude to obtain a dynamically optimized sensitivity index;

[0027] The dynamic optimization sensitivity index replaces the initial sensitivity index and applies it to the storage classification operation of the target data resource;

[0028] The sensitivity index optimization formula is: , where S dynamic Refers to the dynamic optimization sensitivity index, S initial refers to the initial sensitivity index, P refers to the influencing proportional factor, and a refers to the adjustment coefficient of the influencing proportional factor. refers to the average change range, b refers to the adjustment coefficient of the average change range, and n refers to the total number of migrations of the storage environment of the standard reference sample. It refers to the change in implicit correlation caused by the migration of the storage environment of the standard reference sample for the i-th time;

[0029] Said The specific calculation process is: ,in Refers to the number of access paths of the standard reference sample before the i-th storage environment migration. Refers to the number of access paths of the standard reference sample after the i-th storage environment migration.

[0030] A data asset management system, the system comprising: a data acquisition module, an impact ratio factor determination module, a standard reference sample determination module, an average change amplitude calculation module and a sensitivity index adjustment module, wherein:

[0031] A data acquisition module is used to determine the specified specific category of the target data asset, and extract the transmission and storage operation logs corresponding to the specified specific category from the log management platform of the target enterprise to obtain the initial sensitivity index of the target data asset;

[0032] The impact ratio factor determination module is used to screen the data resources in the transmission and storage operation logs that exceed the preset storage time limit, and aggregate these data resources into an analysis data set, statistically analyze the proportion of data resources that have undergone storage environment migration in the analysis data set, and set the proportion as the impact ratio factor;

[0033] The storage environment migration includes migration from a local system to a cloud platform, distributed migration across cloud nodes, and switching of storage media;

[0034] A standard reference sample determination module is used to select a standard reference sample from the analysis data set, the selection condition being a data resource that has a storage environment migration record of more than a preset number of times and is most consistent with the characteristic information of the target data resource, and extracting the secondary storage operation log of the standard reference sample;

[0035] The average change range calculation module is used to parse the secondary storage operation log of the standard reference sample, quantify the implicit correlation change range during each storage environment migration, and calculate the average change range;

[0036] The sensitivity index adjustment module is used to dynamically adjust the initial sensitivity index of the target data resource based on the impact scale factor and the average change amplitude.

[0037] As a further limitation of the technical solution of the embodiment of the present invention, the impact proportional factor determination module specifically includes:

[0038] A log parsing unit, used to parse the transmission and storage operation logs, determine the storage time limit and storage environment migration record bound to each data resource;

[0039] An analysis data set generation unit, used to filter out data resources that exceed a preset storage time limit and aggregate them into an analysis data set;

[0040] The impact ratio factor determination unit is used to analyze the data resources in the data set that have undergone storage environment migration, calculate their proportions and set them as impact ratio factors.

[0041] As a further limitation of the technical solution of the embodiment of the present invention, the standard reference sample determination module specifically includes:

[0042] A first data set screening unit, used to screen out all data resources with storage environment migration records from the analysis data set, and count the number of storage environment migrations of each data resource;

[0043] The second data set screening unit is used to select data resources with storage environment migration records of more than a preset number of times from the screened data resources, and further screen out standard reference samples with the highest similarity to the target data resources based on characteristic information such as data structure, purpose and scale;

[0044] The secondary storage operation log extraction unit is used to extract the secondary storage operation log of the standard reference sample.

[0045] As a further limitation of the technical solution of the embodiment of the present invention, the average change amplitude calculation module specifically includes:

[0046] An implicit correlation data acquisition unit, used to extract implicit correlation data related to each storage environment migration from the secondary storage operation log of the standard reference sample, wherein the implicit correlation data includes the number of access path changes;

[0047] A change range calculation unit, used for analyzing the extracted implicit correlation data and calculating the change range of the implicit correlation caused by each storage environment migration;

[0048] The average change range determination unit is used to summarize the implicit correlation change ranges of all secondary storage environment migrations and calculate the average change range thereof.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] This technical solution achieves dynamic adjustment of the sensitivity index by comprehensively considering the impact ratio factor of data resources in the storage environment migration and the change range of implicit relevance, which has significant beneficial effects. First, the solution can dynamically adapt to changes in the storage environment and optimize the sensitivity assessment in real time according to the actual migration frequency of data and changes in implicit relevance, thereby improving flexibility.

[0051] Secondly, the sensitivity index is modified by combining multi-dimensional factors, which is more accurate than traditional static evaluation and can more truly reflect the actual sensitivity level of data resources. In addition, the optimized sensitivity index provides a reliable basis for storage classification operations, ensuring that highly sensitive data is fully protected during storage, effectively reducing the risk of data leakage.

[0052] At the same time, the solution improves the overall efficiency of the storage system by reasonably allocating storage resources and avoiding resource waste caused by sensitivity assessment bias. This method is highly versatile and can be applied to different types of data resources and multiple storage scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of a method provided by an embodiment of the present invention;

[0054] Figure 2 A flow chart of calculating the impact proportional factor in the method provided in an embodiment of the present invention;

[0055] Figure 3 A flowchart of determining a standard reference sample corresponding to a target data resource in the method provided in an embodiment of the present invention;

[0056] Figure 4 A flow chart of determining the average variation range corresponding to the standard reference sample in the method provided in an embodiment of the present invention;

[0057] Figure 5 A flow chart of calculating a dynamic optimization sensitivity index in a method provided in an embodiment of the present invention;

[0058] Figure 6 An application architecture diagram of a system provided by an embodiment of the present invention;

[0059] Figure 7 A structural block diagram of an impact scale factor determination module in a system provided by an embodiment of the present invention;

[0060] Figure 8 A structural block diagram of a standard reference sample determination module in a system provided by an embodiment of the present invention;

[0061] Fig. 9 This is a structural block diagram of an average variation range calculation module in a system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0063] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0064] Specifically, a data asset management method includes the following steps:

[0065] Step S100, determining the designated specific category of the target data asset, and extracting the transmission and storage operation logs corresponding to the designated specific category from the log management platform of the target enterprise, to obtain the initial sensitivity index of the target data asset.

[0066] In the embodiment of the present invention, the background of the present invention is to classify data assets by sensitivity index before uploading and storing data assets, so as to achieve refined management and security protection of data. In the actual scenario of data management, different types of data assets have different sensitivity requirements. Reasonable evaluation and classification of sensitivity index can improve data storage efficiency, optimize storage resource allocation, and reduce the risk of sensitive data leakage. Therefore, classification based on sensitivity index has become an important requirement of data management before uploading and storing data assets.

[0067] Specifying specific categories means classifying the target data assets into specific types, such as financial data, customer information, operation logs, product information, etc. This classification is to clarify the characteristics and uses of data assets, thereby providing a reference for the subsequent calculation of sensitivity indexes. The classification of specific categories is usually based on standards such as the source, use, business relevance or data structure of the data, which can be annotated through the enterprise's metadata management system or business rules.

[0068] The target enterprise's log management platform is a platform for recording data operation behaviors, usually used to collect and store the transmission and storage operation logs of the enterprise's data assets. These logs are usually automatically generated by the enterprise's storage system, data transmission tools or data platform, and record the data operation process, including upload time, storage location, access records and other information. These logs can be extracted through existing log collection tools (such as ELK Stack, Splunk, etc.), and combined with the enterprise's data management system for sorting and analysis to obtain transmission and storage operation records related to specific categories.

[0069] The initial sensitivity index is a basic assessment of the sensitivity of data assets under static conditions, and is usually calculated using existing technologies. Its calculation is usually generated through data sensitivity assessment tools or algorithms in combination with information such as the type, content, purpose, and compliance requirements of the data. For example, a rule-based sensitivity assessment tool can evaluate the sensitivity level of data through automatic scanning of data content (such as keyword matching and regular expression detection); a machine learning model can predict the sensitivity index of a target data asset by training historical data sensitivity records. These technologies have been widely used in data classification and compliance management, and are existing application technologies that can be directly referenced in the present invention.

[0070] Furthermore, the data asset management method further includes the following steps:

[0071] Step S200, filtering data resources in the transmission and storage operation logs that exceed a preset storage time limit, and aggregating these data resources into an analysis data set, statistically analyzing the proportion of data resources that have undergone storage environment migration in the analysis data set, and setting the proportion as an impact ratio factor.

[0072] Specifically, Figure 2 A flow chart for calculating the impact scaling factor is shown.

[0073] The steps of filtering data resources exceeding a preset storage time limit in the transmission and storage operation logs, aggregating these data resources into an analysis data set, and statistically analyzing the proportion of data resources that have undergone storage environment migration in the analysis data set, and setting the proportion as an influencing proportional factor specifically include the following steps:

[0074] Step S201, parsing the transmission and storage operation logs to determine the storage time limit and storage environment migration record bound to each data resource;

[0075] Step S202, filtering out data resources that exceed a preset storage time limit and aggregating them into an analysis data set;

[0076] Step S203, analyzing the data resources in the data set that have undergone storage environment migration, counting their proportions and setting them as impact ratio factors.

[0077] The storage environment migration includes migration from a local system to a cloud platform, distributed migration across cloud nodes, and switching of storage media.

[0078] In an embodiment of the present invention, log parsing and analysis technology is applied in the process of parsing transmission and storage operation logs and determining the storage time limit and storage environment migration record bound to each data resource. Specifically, the log management platform or storage system will generate structured or semi-structured operation logs, which can be parsed and extracted by log analysis tools (such as ELKStack, Splunk, etc.). Using regular expressions, log query languages ​​(such as KQL, SQL, etc.) or customized scripts, the storage time limit and migration records related to data resources can be quickly extracted. In addition, log cleaning and conversion tools (such as Logstash, Fluentd) can also be used to pre-process the log content, unify the format and fields, and ensure the integrity and accuracy of the log data.

[0079] Data screening and filtering techniques are used in the process of filtering out data resources that have exceeded the preset storage time limit and aggregating them into an analytical data set. By performing conditional queries on the parsed log data (such as through SQL WHERE clauses or rule-based filtering conditions), data resources that have exceeded the preset storage time limit can be filtered out. Combined with data integration tools (such as Apache Nifi, Talend) or distributed processing frameworks (such as Apache Spark, Hadoop), large-scale log data can be efficiently filtered and aggregated to ultimately generate a data set for analysis.

[0080] Data statistics and analysis techniques are applied in the process of analyzing the data resources in the data set that have undergone storage environment migration, counting their proportions and setting them as influencing scale factors. First, use data statistics tools (such as Pandas, NumPy) or query languages ​​to group and perform statistical operations on the data set, and calculate the number of data resources that have undergone storage environment migration and their proportion of the total number of data resources. Second, use scripts or rules to map the calculation results to influencing scale factors for subsequent use. In addition, in order to process large-scale data sets, big data processing frameworks (such as Apache Flink, Dask) can also be introduced for real-time or batch computing.

[0081] The combination of the above-mentioned technical means can effectively realize the whole process from log parsing, data screening to statistical analysis, ensuring the accurate and efficient calculation of the influencing proportional factors.

[0082] Furthermore, the data asset management method further includes the following steps:

[0083] Step S300, select a standard reference sample from the analysis data set, the selection condition is that the data resource has a storage environment migration record of more than a preset number of times and is most consistent with the characteristic information of the target data resource, and extract the secondary storage operation log of the standard reference sample.

[0084] Specifically, Figure 3 A flow chart for determining a standard reference sample corresponding to a target data resource is shown.

[0085] Among them, selecting a standard reference sample from the analysis data set, selecting a data resource with a storage environment migration record of more than a preset number of times and the most consistent with the characteristic information of the target data resource, and extracting the secondary storage operation log of the standard reference sample specifically includes the following steps:

[0086] Step S301, filtering out all data resources with storage environment migration records from the analysis data set, and counting the number of storage environment migrations of each data resource;

[0087] Step S302, selecting data resources with storage environment migration records of a preset number or more from the screened data resources, and further screening out standard reference samples with the highest similarity to the target data resources based on characteristic information such as data structure, purpose and scale;

[0088] Step S303, extracting the secondary storage operation log of the standard reference sample.

[0089] In an embodiment of the present invention, the significance of setting the preset number of times is to ensure that the selected standard reference sample can effectively reflect the impact of storage environment migration on the change of implicit data correlation. By setting a reasonable lower limit of the number of migrations, data resources with sufficient migration records and more representative change characteristics can be screened out to avoid the deviation of low-frequency migration data in sensitivity adjustment analysis. The basis mainly comes from historical data statistics, business needs and the dynamic characteristics of the storage environment. For example, for certain frequently migrated distributed storage data types, a reasonable preset number of times can be determined by analyzing historical migration records.

[0090] The specific implementation process of "selecting standard reference samples with the highest similarity to the target data resources based on characteristic information such as data structure, purpose and scale" includes: first, extracting the structure type (such as field attributes, storage format), purpose (such as business scenario, access mode) and data scale (such as data volume, storage space occupancy) of all candidate data resources; then, establishing similarity measurement rules based on the characteristics of the target data resources, such as calculating structural similarity, purpose matching and scale ratio differences; finally, sorting the candidate data resources based on the comprehensive similarity score, and selecting the data resources with the highest scores as standard reference samples.

[0091] The method of extracting the "secondary storage operation log of the standard reference sample" is to locate the corresponding transmission and storage operation logs based on the unique identifier of the reference sample (such as resource ID or file path) through the log management platform of the target enterprise. During the extraction process, the log entries related to the storage environment migration can be further filtered through filtering conditions to form secondary storage operation logs specifically for analysis. These logs can be extracted through existing log management tools (such as ELK Stack, Splunk) or customized scripts to ensure data integrity and accuracy.

[0092] Furthermore, the data asset management method further includes the following steps:

[0093] Step S400, parse the secondary storage operation log of the standard reference sample, quantify the change range of the implicit correlation each time the storage environment is migrated, and calculate the average change range.

[0094] Specifically, Figure 4 A flow chart for determining the average magnitude of change corresponding to a standard reference sample is shown.

[0095] The following steps are used to parse the secondary storage operation log of the standard reference sample, quantify the change range of the implicit correlation each time the storage environment is migrated, and calculate the average change range:

[0096] Step S401, extracting implicit correlation data related to each storage environment migration from the secondary storage operation log of the standard reference sample, wherein the implicit correlation data includes the number of access path changes;

[0097] Step S402, analyzing the extracted implicit correlation data, and calculating the implicit correlation change range caused by each storage environment migration;

[0098] Step S403: Summarize the implicit correlation change ranges of all secondary storage environment migrations and calculate the average change range.

[0099] In the embodiment of the present invention, implicit correlation data refers to indicators of changes in the implicit relationship between data resources and other data or systems during the storage environment migration process. These data can reflect the impact of storage environment migration on the correlation characteristics of data resources, and thus provide a basis for evaluating changes in data sensitivity. Implicit correlation data is usually difficult to define directly through explicit rules, but can be quantified through indirect observation of storage behavior.

[0100] In addition to the number of access path changes, the types of implicit association data can also include call frequency fluctuations and increases or decreases in cross-node associations. Call frequency fluctuations refer to the changes in the frequency of data access or calls after the storage environment is migrated, which can be quantified by comparing the total number of calls before and after the migration; cross-node association increases or decreases refer to the increase or decrease in the number of connections between data and storage nodes or associated data after the storage environment is migrated, which can be calculated by counting the difference in the number of node associations before and after the migration. The calculation method for the magnitude of these two types of changes is the same as the number of path changes, and can be expressed as: .

[0101] This calculation method is applicable to all types of implicit correlation data, and can uniformly quantify their range of change, providing data support for subsequent average change range calculation and dynamic adjustment of sensitivity index.

[0102] Furthermore, the data asset management method further includes the following steps:

[0103] Step S500 , dynamically adjusting the initial sensitivity index of the target data resource based on the impact scale factor and the average change amplitude.

[0104] Specifically, Figure 5 A flow chart for calculating a dynamically optimized sensitivity index is shown.

[0105] The dynamic adjustment of the initial sensitivity index of the target data resource based on the impact scale factor and the average change amplitude specifically includes the following steps:

[0106] Step S501, retrieve the sensitivity index optimization formula, and dynamically adjust the initial sensitivity index of the target data resource in combination with the impact scale factor and the average change amplitude to obtain a dynamically optimized sensitivity index;

[0107] Step S502: Replace the initial sensitivity index with the dynamically optimized sensitivity index and apply it to the storage classification operation of the target data resource.

[0108] The sensitivity index optimization formula is: , where S dynamic Refers to the dynamic optimization sensitivity index, S initial refers to the initial sensitivity index, P refers to the influencing proportional factor, and a refers to the adjustment coefficient of the influencing proportional factor. refers to the average change range, b refers to the adjustment coefficient of the average change range, and n refers to the total number of migrations of the storage environment of the standard reference sample. It refers to the change in implicit correlation caused by the migration of the storage environment of the standard reference sample for the i-th time;

[0109] Said The specific calculation process is: ,in Refers to the number of access paths of the standard reference sample before the i-th storage environment migration. Refers to the number of access paths of the standard reference sample after the i-th storage environment migration.

[0110] In an embodiment of the present invention, the significance of selecting the impact scale factor and the average change range to adjust the initial sensitivity index is to comprehensively consider the dynamic impact of storage environment migration on data sensitivity. The impact scale factor reflects the frequency and prevalence of the target data resource in the storage environment migration. The higher the proportion of migration times, the greater the risk that the data may be exposed to different storage environments, so the sensitivity index needs to be improved. The average change range quantifies the actual degree of change in the implicit correlation of the data during the migration process, such as changes in access paths, fluctuations in call frequency, etc. This change directly reflects the impact of storage environment migration on data privacy and correlation, thereby providing a more scientific basis for evaluation. By combining these two factors, the actual situation of sensitivity changes of data resources during the storage life cycle can be dynamically and accurately reflected.

[0111] In the specific calculation process, the impact scale factor and the average change amplitude are multiplied by the adjustment coefficient to adjust the initial sensitivity index, the purpose of which is to improve the adaptability to the dynamic environment. The impact scale factor directly reflects the impact of the migration frequency on the sensitivity index in the form of a proportion, while the average change amplitude uses the amplitude of the implicit correlation change as a dynamic evaluation standard. These two are quantified as the increment of the sensitivity index through the effect of the adjustment coefficient, and added to the initial sensitivity index to dynamically improve the sensitivity index of the target data resource. This improvement is not a fixed value, but a dynamic adjustment based on specific migration characteristics and implicit correlation changes. It can more accurately evaluate the security requirements of data in different storage environments, thereby guiding the optimization of storage classification operations and the implementation of data protection strategies.

[0112] For example, the initial sensitivity index of a target data resource is 50. After analysis, it is found that the proportion of data resources that have experienced storage environment migration in the analysis data set is 0.4 (40%). In the implicit correlation data analysis, the data resource has experienced 3 storage environment migrations in the storage life cycle. The number of access paths before and after the first migration is 10 and 12 respectively, the number of access paths before and after the second migration is 12 and 15 respectively, and the number of access paths before and after the third migration is 15 and 18 respectively. The implicit correlation change range is calculated as 0.2 (first time), 0.25 (second time), and 0.2 (third time). The average of these change ranges is 0.2167.

[0113] The adjustment parameter a (adjustment parameter affecting the proportional factor) is set to 0.5, and the adjustment parameter b (adjustment parameter of the average change amplitude) is set to 0.8. After combining the above parameters, the optimized dynamic sensitivity index is calculated to be 63.67.

[0114] Furthermore, Figure 6 The application architecture diagram of the system provided by the embodiment of the present invention is shown.

[0115] In another preferred embodiment of the present invention, a data asset management system includes:

[0116] The data acquisition module 100 is used to determine the specified specific category of the target data asset, and extract the transmission and storage operation logs corresponding to the specified specific category from the log management platform of the target enterprise to obtain the initial sensitivity index of the target data asset.

[0117] In the embodiment of the present invention, the background of the present invention is to classify data assets by sensitivity index before uploading and storing data assets, so as to achieve refined management and security protection of data. In the actual scenario of data management, different types of data assets have different sensitivity requirements. Reasonable evaluation and classification of sensitivity index can improve data storage efficiency, optimize storage resource allocation, and reduce the risk of sensitive data leakage. Therefore, classification based on sensitivity index has become an important requirement of data management before uploading and storing data assets.

[0118] Specifying specific categories means classifying the target data assets into specific types, such as financial data, customer information, operation logs, product information, etc. This classification is to clarify the characteristics and uses of data assets, thereby providing a reference for the subsequent calculation of sensitivity indexes. The classification of specific categories is usually based on standards such as the source, use, business relevance or data structure of the data, which can be annotated through the enterprise's metadata management system or business rules.

[0119] The target enterprise's log management platform is a platform for recording data operation behaviors, usually used to collect and store the transmission and storage operation logs of the enterprise's data assets. These logs are usually automatically generated by the enterprise's storage system, data transmission tools or data platform, and record the data operation process, including upload time, storage location, access records and other information. These logs can be extracted through existing log collection tools (such as ELK Stack, Splunk, etc.), and combined with the enterprise's data management system for sorting and analysis to obtain transmission and storage operation records related to specific categories.

[0120] The initial sensitivity index is a basic assessment of the sensitivity of data assets under static conditions, and is usually calculated using existing technologies. Its calculation is usually generated through data sensitivity assessment tools or algorithms in combination with information such as the type, content, purpose, and compliance requirements of the data. For example, a rule-based sensitivity assessment tool can evaluate the sensitivity level of data through automatic scanning of data content (such as keyword matching and regular expression detection); a machine learning model can predict the sensitivity index of a target data asset by training historical data sensitivity records. These technologies have been widely used in data classification and compliance management, and are existing application technologies that can be directly referenced in the present invention.

[0121] Furthermore, the data asset management system also includes:

[0122] The impact ratio factor determination module 200 is used to screen data resources in the transmission and storage operation logs that exceed the preset storage time limit, and summarize these data resources into an analysis data set, statistically analyze the proportion of data resources that have undergone storage environment migration in the data set, and set the proportion as the impact ratio factor.

[0123] Specifically, Figure 7 It shows a structural block diagram of the impact proportional factor determination module 200 in the system provided by an embodiment of the present invention.

[0124] Among them, in the preferred implementation manner provided by the present invention, the impact proportional factor determination module 200 specifically includes:

[0125] The log parsing unit 201 is used to parse the transmission and storage operation logs to determine the storage time limit and storage environment migration record bound to each data resource;

[0126] An analysis data set generating unit 202 is used to filter out data resources that exceed a preset storage time limit and aggregate them into an analysis data set;

[0127] The impact ratio factor determination unit 203 is used to analyze the data resources in the data set that have undergone storage environment migration, calculate their proportions and set them as impact ratio factors.

[0128] The storage environment migration includes migration from a local system to a cloud platform, distributed migration across cloud nodes, and switching of storage media.

[0129] In an embodiment of the present invention, log parsing and analysis technology is applied in the process of parsing transmission and storage operation logs and determining the storage time limit and storage environment migration record bound to each data resource. Specifically, the log management platform or storage system will generate structured or semi-structured operation logs, which can be parsed and extracted by log analysis tools (such as ELKStack, Splunk, etc.). Using regular expressions, log query languages ​​(such as KQL, SQL, etc.) or customized scripts, the storage time limit and migration records related to data resources can be quickly extracted. In addition, log cleaning and conversion tools (such as Logstash, Fluentd) can also be used to pre-process the log content, unify the format and fields, and ensure the integrity and accuracy of the log data.

[0130] Data screening and filtering techniques are used in the process of filtering out data resources that have exceeded the preset storage time limit and aggregating them into an analytical data set. By performing conditional queries on the parsed log data (such as through SQL WHERE clauses or rule-based filtering conditions), data resources that have exceeded the preset storage time limit can be filtered out. Combined with data integration tools (such as Apache Nifi, Talend) or distributed processing frameworks (such as Apache Spark, Hadoop), large-scale log data can be efficiently filtered and aggregated to ultimately generate a data set for analysis.

[0131] Data statistics and analysis techniques are applied in the process of analyzing the data resources in the data set that have undergone storage environment migration, counting their proportions and setting them as influencing scale factors. First, use data statistics tools (such as Pandas, NumPy) or query languages ​​to group and perform statistical operations on the data set, and calculate the number of data resources that have undergone storage environment migration and their proportion of the total number of data resources. Second, use scripts or rules to map the calculation results to influencing scale factors for subsequent use. In addition, in order to process large-scale data sets, big data processing frameworks (such as Apache Flink, Dask) can also be introduced for real-time or batch computing.

[0132] The combination of the above-mentioned technical means can effectively realize the whole process from log parsing, data screening to statistical analysis, ensuring the accurate and efficient calculation of the influencing proportional factors.

[0133] Furthermore, the data asset management system also includes:

[0134] The standard reference sample determination module 300 is used to select a standard reference sample from the analysis data set, the selection condition is a data resource with storage environment migration records of more than a preset number of times and the most consistent with the characteristic information of the target data resource, and extract the secondary storage operation log of the standard reference sample.

[0135] Specifically, Figure 8 It shows a structural block diagram of the standard reference sample determination module 300 in the system provided by an embodiment of the present invention.

[0136] Among them, in the preferred embodiment provided by the present invention, the standard reference sample determination module 300 specifically includes:

[0137] The first data set screening unit 301 is used to screen out all data resources with storage environment migration records from the analysis data set, and count the number of storage environment migrations of each data resource;

[0138] The second data set screening unit 302 is used to select data resources with storage environment migration records of a preset number or more from the screened data resources, and further screen out standard reference samples with the highest similarity to the target data resources according to characteristic information such as data structure, purpose and scale;

[0139] The secondary storage operation log extraction unit 303 is used to extract the secondary storage operation log of the standard reference sample.

[0140] In an embodiment of the present invention, the significance of setting the preset number of times is to ensure that the selected standard reference sample can effectively reflect the impact of storage environment migration on the change of implicit data correlation. By setting a reasonable lower limit of the number of migrations, data resources with sufficient migration records and more representative change characteristics can be screened out to avoid the deviation of low-frequency migration data in sensitivity adjustment analysis. The basis mainly comes from historical data statistics, business needs and the dynamic characteristics of the storage environment. For example, for certain frequently migrated distributed storage data types, a reasonable preset number of times can be determined by analyzing historical migration records.

[0141] The specific implementation process of "selecting standard reference samples with the highest similarity to the target data resources based on characteristic information such as data structure, purpose and scale" includes: first, extracting the structure type (such as field attributes, storage format), purpose (such as business scenario, access mode) and data scale (such as data volume, storage space occupancy) of all candidate data resources; then, establishing similarity measurement rules based on the characteristics of the target data resources, such as calculating structural similarity, purpose matching and scale ratio differences; finally, sorting the candidate data resources based on the comprehensive similarity score, and selecting the data resources with the highest scores as standard reference samples.

[0142] The method of extracting the "secondary storage operation log of the standard reference sample" is to locate the corresponding transmission and storage operation logs based on the unique identifier of the reference sample (such as resource ID or file path) through the log management platform of the target enterprise. During the extraction process, the log entries related to the storage environment migration can be further filtered through filtering conditions to form secondary storage operation logs specifically for analysis. These logs can be extracted through existing log management tools (such as ELK Stack, Splunk) or customized scripts to ensure data integrity and accuracy.

[0143] Furthermore, the data asset management system also includes:

[0144] The average change range calculation module 400 is used to parse the secondary storage operation log of the standard reference sample, quantify the implicit correlation change range during each storage environment migration, and calculate the average change range.

[0145] Specifically, Fig. 9 It shows a structural block diagram of the average variation range calculation module 400 in the system provided by an embodiment of the present invention.

[0146] Among them, in the preferred implementation manner provided by the present invention, the average change amplitude calculation module 400 specifically includes:

[0147] An implicit correlation data acquisition unit 401 is used to extract implicit correlation data related to each storage environment migration from the secondary storage operation log of the standard reference sample, wherein the implicit correlation data includes the number of access path changes;

[0148] A change range calculation unit 402 is used to analyze the extracted implicit correlation data and calculate the change range of the implicit correlation caused by each storage environment migration;

[0149] The average change range determining unit 403 is used to summarize the implicit relevance change ranges of all secondary storage environment migrations and calculate the average change range thereof.

[0150] In the embodiment of the present invention, implicit correlation data refers to indicators of changes in the implicit relationship between data resources and other data or systems during the storage environment migration process. These data can reflect the impact of storage environment migration on the correlation characteristics of data resources, and thus provide a basis for evaluating changes in data sensitivity. Implicit correlation data is usually difficult to define directly through explicit rules, but can be quantified through indirect observation of storage behavior.

[0151] In addition to the number of access path changes, the types of implicit association data can also include call frequency fluctuations and increases or decreases in cross-node associations. Call frequency fluctuations refer to the changes in the frequency of data access or calls after the storage environment is migrated, which can be quantified by comparing the total number of calls before and after the migration; cross-node association increases or decreases refer to the increase or decrease in the number of connections between data and storage nodes or associated data after the storage environment is migrated, which can be calculated by counting the difference in the number of node associations before and after the migration. The calculation method for the magnitude of these two types of changes is the same as the number of path changes, and can be expressed as: .

[0152] This calculation method is applicable to all types of implicit correlation data, and can uniformly quantify their range of change, providing data support for subsequent average change range calculation and dynamic adjustment of sensitivity index.

[0153] Furthermore, the data asset management system also includes:

[0154] The sensitivity index adjustment module 500 is used to dynamically adjust the initial sensitivity index of the target data resource based on the impact scale factor and the average change amplitude.

[0155] In an embodiment of the present invention, the significance of selecting the impact scale factor and the average change range to adjust the initial sensitivity index is to comprehensively consider the dynamic impact of storage environment migration on data sensitivity. The impact scale factor reflects the frequency and prevalence of the target data resource in the storage environment migration. The higher the proportion of migration times, the greater the risk that the data may be exposed to different storage environments, so the sensitivity index needs to be improved. The average change range quantifies the actual degree of change in the implicit correlation of the data during the migration process, such as changes in access paths, fluctuations in call frequency, etc. This change directly reflects the impact of storage environment migration on data privacy and correlation, thereby providing a more scientific basis for evaluation. By combining these two factors, the actual situation of sensitivity changes of data resources during the storage life cycle can be dynamically and accurately reflected.

[0156] In the specific calculation process, the impact scale factor and the average change amplitude are multiplied by the adjustment coefficient to adjust the initial sensitivity index, the purpose of which is to improve the adaptability to the dynamic environment. The impact scale factor directly reflects the impact of the migration frequency on the sensitivity index in the form of a proportion, while the average change amplitude uses the amplitude of the implicit correlation change as a dynamic evaluation standard. These two are quantified as the increment of the sensitivity index through the effect of the adjustment coefficient, and added to the initial sensitivity index to dynamically improve the sensitivity index of the target data resource. This improvement is not a fixed value, but a dynamic adjustment based on specific migration characteristics and implicit correlation changes. It can more accurately evaluate the security requirements of data in different storage environments, thereby guiding the optimization of storage classification operations and the implementation of data protection strategies.

[0157] For example, the initial sensitivity index of a target data resource is 50. After analysis, it is found that the proportion of data resources that have experienced storage environment migration in the analysis data set is 0.4 (40%). In the implicit correlation data analysis, the data resource has experienced 3 storage environment migrations in the storage life cycle. The number of access paths before and after the first migration is 10 and 12 respectively, the number of access paths before and after the second migration is 12 and 15 respectively, and the number of access paths before and after the third migration is 15 and 18 respectively. The implicit correlation change range is calculated as 0.2 (first time), 0.25 (second time), and 0.2 (third time). The average of these change ranges is 0.2167.

[0158] The adjustment parameter a (adjustment parameter affecting the proportional factor) is set to 0.5, and the adjustment parameter b (adjustment parameter of the average change amplitude) is set to 0.8. After combining the above parameters, the optimized dynamic sensitivity index is calculated to be 63.67.

[0159] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0160] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0161] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0163] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A data asset management method, characterized in that: The method comprises: Determine the designated specific category of the target data asset, and extract the transmission and storage operation logs corresponding to the designated specific category from the log management platform of the target enterprise to obtain the initial sensitivity index of the target data asset; Filter the data resources that exceed the preset storage time limit in the transmission and storage operation logs, and aggregate these data resources into an analysis data set, and statistically analyze the proportion of data resources that have undergone storage environment migration in the analysis data set, and set the proportion as the impact ratio factor; Select a standard reference sample from the analysis data set, the selection condition is that the data resource has a storage environment migration record of more than a preset number of times and is most consistent with the characteristic information of the target data resource, and extract the secondary storage operation log of the standard reference sample; Parse the secondary storage operation logs of the standard reference sample, quantify the change range of implicit correlation each time the storage environment is migrated, and calculate the average change range; Based on the impact scale factor and the average change amplitude, the initial sensitivity index of the target data resource is dynamically adjusted.

2. The data asset management method according to claim 2, characterized in that: The storage environment migration includes migration from a local system to a cloud platform, distributed migration across cloud nodes, and switching of storage media.

3. The data asset management method according to claim 1, characterized in that: The steps of filtering data resources exceeding a preset storage time limit in the transmission and storage operation logs, and aggregating these data resources into an analysis data set, statistically analyzing the proportion of data resources that have undergone storage environment migration in the analysis data set, and setting the proportion as an influencing proportional factor include: Parse the transmission and storage operation logs to determine the storage time limit and storage environment migration records bound to each data resource; Filter out data resources that exceed the preset storage time limit and aggregate them into an analytical data set; Analyze the data resources in the data set that have undergone storage environment migration, calculate their proportions, and set them as impact ratio factors.

4. The data asset management method according to claim 1, characterized in that: The steps of selecting a standard reference sample from the analysis data set, selecting a data resource having storage environment migration records of more than a preset number of times and most consistent with characteristic information of the target data resource, and extracting a secondary storage operation log of the standard reference sample include: Filter out all data resources with storage environment migration records from the analysis data set, and count the number of storage environment migrations for each data resource; Among the screened data resources, select data resources with storage environment migration records of more than a preset number of times, and further screen out standard reference samples with the highest similarity to the target data resources based on characteristic information such as data structure, purpose and scale; Extract the secondary storage operation log of the standard reference sample.

5. The data asset management method according to claim 4, characterized in that: The steps of parsing the secondary storage operation log of the standard reference sample, quantifying the change range of the implicit correlation each time the storage environment is migrated, and calculating the average change range include: Extracting implicit correlation data related to each storage environment migration from the secondary storage operation log of the standard reference sample, wherein the implicit correlation data includes the number of access path changes; Analyze the extracted implicit correlation data and calculate the change range of implicit correlation caused by each storage environment migration; Sum up the implicit affinity changes of all secondary storage environment migrations and calculate their average change.

6. The data asset management method according to claim 5, characterized in that: Based on the impact scale factor and the average change amplitude, the steps of dynamically adjusting the initial sensitivity index of the target data resource include: Retrieve the sensitivity index optimization formula, and dynamically adjust the initial sensitivity index of the target data resource in combination with the impact scale factor and the average change amplitude to obtain a dynamically optimized sensitivity index; The dynamic optimization sensitivity index replaces the initial sensitivity index and applies it to the storage classification operation of the target data resource; The sensitivity index optimization formula is: , where S dynamic Refers to the dynamic optimization sensitivity index, S initial refers to the initial sensitivity index, P refers to the influencing proportional factor, and a refers to the adjustment coefficient of the influencing proportional factor. refers to the average change range, b refers to the adjustment coefficient of the average change range, and n refers to the total number of migrations of the storage environment of the standard reference sample. It refers to the change in implicit correlation caused by the migration of the storage environment of the standard reference sample for the i-th time; Said The specific calculation process is: ,in Refers to the number of access paths of the standard reference sample before the i-th storage environment migration. Refers to the number of access paths of the standard reference sample after the i-th storage environment migration.

7. A data asset management system, characterized in that: The system comprises: a data acquisition module, an impact ratio factor determination module, a standard reference sample determination module, an average variation range calculation module and a sensitivity index adjustment module, wherein: A data acquisition module is used to determine the specified specific category of the target data asset, and extract the transmission and storage operation logs corresponding to the specified specific category from the log management platform of the target enterprise to obtain the initial sensitivity index of the target data asset; The impact ratio factor determination module is used to screen the data resources in the transmission and storage operation logs that exceed the preset storage time limit, and aggregate these data resources into an analysis data set, statistically analyze the proportion of data resources that have undergone storage environment migration in the analysis data set, and set the proportion as the impact ratio factor; The storage environment migration includes migration from a local system to a cloud platform, distributed migration across cloud nodes, and switching of storage media; A standard reference sample determination module is used to select a standard reference sample from the analysis data set, the selection condition being a data resource that has a storage environment migration record of more than a preset number of times and is most consistent with the characteristic information of the target data resource, and extracting the secondary storage operation log of the standard reference sample; The average change range calculation module is used to parse the secondary storage operation log of the standard reference sample, quantify the implicit correlation change range during each storage environment migration, and calculate the average change range; The sensitivity index adjustment module is used to dynamically adjust the initial sensitivity index of the target data resource based on the impact scale factor and the average change amplitude.

8. The data asset management method according to claim 7, characterized in that: The impact proportional factor determination module specifically includes: A log parsing unit, used to parse the transmission and storage operation logs, determine the storage time limit and storage environment migration record bound to each data resource; An analysis data set generation unit, used to filter out data resources that exceed a preset storage time limit and aggregate them into an analysis data set; The impact ratio factor determination unit is used to analyze the data resources in the data set that have undergone storage environment migration, calculate their proportions and set them as impact ratio factors.

9. The data asset management method according to claim 8, characterized in that: The standard reference sample determination module specifically includes: A first data set screening unit, used to screen out all data resources with storage environment migration records from the analysis data set, and count the number of storage environment migrations of each data resource; The second data set screening unit is used to select data resources with storage environment migration records of more than a preset number of times from the screened data resources, and further screen out standard reference samples with the highest similarity to the target data resources based on characteristic information such as data structure, purpose and scale; The secondary storage operation log extraction unit is used to extract the secondary storage operation log of the standard reference sample.

10. The data asset management method according to claim 9, characterized in that: The average variation range calculation module specifically includes: An implicit correlation data acquisition unit, used to extract implicit correlation data related to each storage environment migration from the secondary storage operation log of the standard reference sample, wherein the implicit correlation data includes the number of access path changes; A change range calculation unit, used for analyzing the extracted implicit correlation data and calculating the change range of the implicit correlation caused by each storage environment migration; The average change range determination unit is used to summarize the implicit correlation change ranges of all secondary storage environment migrations and calculate the average change range thereof.

Images