Data management method and system based on data resource security identification level

Abstract

The invention belongs to the technical field of data security, and discloses a data management method based on a data resource security identification level, and the method achieves the automatic and intelligent decision making: achieves the data classification and the automatic identification of the security level through a machine learning and deep learning model, remarkably reduces the manual intervention, and improves the efficiency and accuracy. And dynamic real-time performance: in combination with a risk assessment model, the system can detect abnormal behaviors in real time and dynamically adjust a security policy in a data flow process, so as to ensure that the sensitive data is properly protected in different scenes. And security and compliance: a log auditing module ensures the traceability of all data operations, improves the response capability to potential risks through an anomaly detection algorithm, and provides effective compliance guarantee for data security governance. And the system can be compatible with various data sources, supports configuration and execution of various security policies, and is suitable for data security management requirements of different industries (such as finance and medical treatment).

Description

Technical Field

[0001] The present invention belongs to the field of data security technology, and in particular, relates to a data governance method and system based on data resource security identification level. Background Art

[0002] Data security governance, especially the method of security governance of data resources on big data platforms. It manages the data resources on big data platforms through steps such as data collection, data classification, data labeling, and data permission allocation to ensure data security. However, this method has some limitations: Lack of automated data security level identification: Existing technologies rely on manual security classification and management of data resources, which is inefficient and prone to errors. Insufficient dynamism: When faced with rapidly changing business environments and data resources, existing governance methods are difficult to adapt to real-time changes and fail to dynamically adjust security levels based on data usage. Weak intelligence: No advanced algorithms or models are introduced to intelligently identify data, and it is impossible to effectively deal with complex and diverse data resources and security threats.

[0003] Through the above analysis, the problems and defects of the prior art are as follows:

[0004] (1) Automated data security level identification: Existing technologies rely on manual classification, and there is an urgent need for automated and intelligent methods to identify and classify data security levels.

[0005] (2) Dynamic real-time: Existing technologies are difficult to adapt to real-time changes in data resources. A system is needed that can dynamically adjust security policies based on real-time data usage and risks.

[0006] (3) Insufficient application of advanced intelligent algorithms: There is an urgent need to introduce machine learning and AI algorithms to automatically adjust security levels and permission configurations through in-depth analysis of data content, access records, etc. Summary of the invention

[0007] In view of the problems existing in the prior art, the present invention provides a data governance method based on the security identification level of data resources.

[0008] The present invention is implemented in this way: a data governance method based on data resource security identification level includes:

[0009] Step 1, data collection and initialization:

[0010] Through the data collection module, various data resources inside or outside the enterprise are collected in real time; the collected data includes structured, unstructured and semi-structured data; the system pre-processes the data, including deduplication, cleaning and normalization;

[0011] Step 2: Automated data classification and security level identification:

[0012] The data classification and security level identification module applies machine learning algorithms, including classification algorithms, clustering algorithms and natural language processing (NLP), to automatically analyze and classify the collected data; based on the sensitivity of the data content, access history and business scenarios, it automatically assigns the corresponding security level to the data, including "public level", "internal level", "sensitive level" and "top secret level";

[0013] Step 3: Security policy configuration and execution:

[0014] The security policy configuration and execution module automatically generates and executes corresponding security policies based on the identified data security level. For highly sensitive data, the system automatically applies encryption and desensitization operations, and strictly controls access rights to ensure that only authorized personnel or systems can access it.

[0015] Step 4: Risk assessment and dynamic adjustment:

[0016] The risk assessment and dynamic adjustment module monitors data usage behavior in real time, including frequent access requests and abnormal data operations. The risk assessment model dynamically assesses the security status of data. If necessary, the system will automatically increase the security level of the data and update the corresponding security policy.

[0017] Step 5: Log audit and exception handling:

[0018] The log audit and anomaly detection module records all data operations in real time and audits user behavior. The anomaly detection module monitors data access and operation behavior, automatically identifies potential threats, including unauthorized data access, and implements preventive security measures.

[0019] Further, the data acquisition module:

[0020] Multi-source data integration: Use API-based real-time data capture combined with ETL tools to integrate data from different sources in multiple formats into a unified data pool;

[0021] Real-time data monitoring: Use Kafka and Flink real-time stream processing frameworks to monitor and process data flow, storage, and processing;

[0022] Data format conversion: supports multiple data formats such as JSON, XML, and CSV, and converts data into standardized formats;

[0023] Data annotation and indexing: Add metadata annotations to each data resource, including creation time, data source, and data type, and use indexing technology.

[0024] Furthermore, the data classification and security level identification module:

[0025] Data classification algorithm: Use NLP technology to perform semantic analysis on text data and extract sensitive information; for structured data, apply clustering algorithms to identify sensitive data fields, including financial information and personal data;

[0026] Machine learning model: Use supervised and unsupervised learning models, including SVM, decision tree, and LSTM, to learn the security level classification of historical data and automatically adjust the security level of new data;

[0027] Rule engine: Combined with industry standards, a set of data sensitivity identification rules are defined to automatically classify special industry data according to the rules;

[0028] Real-time adjustment mechanism: Combine data access records and context information to adjust data security levels in real time to ensure that sensitive data receives different security protection levels in different usage scenarios.

[0029] Furthermore, the security policy configuration and execution module:

[0030] Access control: Use the role-based access control (RBAC) model or the attribute-based access control (ABAC) model to dynamically adjust the access rights of different users and applications;

[0031] Encryption strategy: Automatically encrypt highly sensitive data, support AES and RSA encryption algorithms, and keep data encrypted during storage and transmission;

[0032] Data desensitization: Use rule-based data desensitization algorithms to mask or obfuscate personal sensitive information to ensure that data meets compliance requirements during analysis and presentation;

[0033] Dynamic update of security policies: Regularly push and adjust real-time security policies through updated data in the security policy library to ensure dynamic adjustment of policies as data changes.

[0034] Furthermore, the risk assessment and dynamic adjustment module:

[0035] Risk assessment model: Use risk assessment algorithms to analyze data access behavior in real time and determine potential risk levels; detect potential threats by analyzing access frequency, time, and location characteristics;

[0036] Anomaly detection: Use machine learning algorithms to analyze user access behavior patterns, detect abnormal access behaviors, automatically generate warnings and make policy adjustments;

[0037] Dynamic adjustment mechanism: Once abnormal behavior is detected, the system will automatically increase the security level of the data resource, restrict its access rights, or enforce other security policies;

[0038] Multi-level response: The system responds in different levels according to the risk assessment level. When the risk is low, the policy is adjusted. When the risk is medium or high, the administrator is notified and emergency security measures are initiated.

[0039] Furthermore, the log audit and anomaly detection module:

[0040] Logging: Use ELK or Splunk log management system to record all data operations, policy applications, and user access behaviors;

[0041] Real-time auditing: Combined with SIEM tools, real-time monitoring of log data can be used to identify abnormal operations or potential security vulnerabilities;

[0042] Anomaly detection and alerting: The AI ​​model is used to analyze abnormal patterns in log data. Once illegal operations or unauthorized access are found, the system will automatically issue an alert and submit the relevant data to the risk assessment module for further processing;

[0043] Visual audit report: The system will generate a detailed audit report that shows the access history, policy execution and security status of data resources, and presents it to the administrator in a graphical form.

[0044] Another object of the present invention is to provide a data governance system based on data resource security identification level, including:

[0045] The data acquisition module is responsible for collecting data resources from different data sources in real time, monitoring the flow, storage, and processing of data, and ensuring comprehensive data coverage;

[0046] Data classification and security level identification module, which uses machine learning and deep learning algorithms to automatically identify and classify the security level of data based on data content, sensitivity, access records and contextual information; this module dynamically adjusts the security level to ensure real-time data security governance;

[0047] The security policy configuration and execution module is used to automatically configure security policies according to data classification and identified security levels, and perform access control, encryption, and desensitization operations to ensure the security of data at different levels;

[0048] The risk assessment and dynamic adjustment module is used to monitor data usage and access behavior, and dynamically adjust the data security level and strategy through real-time risk assessment models to prevent potential threats and data leakage;

[0049] The log audit and anomaly detection module is used to record all data operations of the system, monitor and detect abnormal access behaviors, and provide audit functions for data security governance.

[0050] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the data governance method based on the data resource security identification level.

[0051] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the data governance method based on the data resource security identification level.

[0052] Another object of the present invention is to provide an information data processing terminal, which is used to implement the data governance system based on the data resource security identification level.

[0053] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0054] First, in view of the technical problems existing in the above-mentioned prior art, some creative technical effects are brought about after solving the problems. The specific description is as follows:

[0055] Automation and intelligence: By integrating machine learning and deep learning technologies, the system achieves full automation of data classification and security level identification, greatly reducing manual intervention and improving efficiency and accuracy.

[0056] Dynamic real-time: The system has the ability of real-time monitoring and dynamic adjustment. It can adjust data security policies in real time according to data usage behavior and security risk assessment to ensure the dynamic and flexible security governance of the system.

[0057] Comprehensive coverage and high-precision risk assessment: Through multi-source data collection and fusion, combined with anomaly detection and real-time risk assessment models, the system can efficiently identify potential security threats and automatically respond according to different levels of risks to ensure the security of data resources.

[0058] Traceability and compliance assurance: Through log auditing and anomaly detection, the system provides traceable security auditing functions for all data operations, ensuring that when faced with security compliance requirements, the system can provide a complete chain of evidence and compliance reports.

[0059] This data governance method based on data security identification level not only solves the problems of low efficiency and poor dynamics of traditional methods, but also provides higher security and operability through intelligent models.

[0060] Second, the data governance method based on data resource security identification level provided by the present invention has achieved remarkable technical effects in industrial applications, which are mainly reflected in the following aspects:

[0061] 1. Improve the automation and accuracy of data security management

[0062] Traditional data security management methods usually rely on manual classification and authority allocation, which is inefficient and prone to errors. Through the automated data classification and security level identification module in the present invention, machine learning algorithms (including classification algorithms, clustering algorithms, and natural language processing NLP) are used to achieve automated classification and identification of internal and external enterprise data. By analyzing the sensitivity of data content, access history, and business scenarios, data can be quickly and accurately divided into "public level", "internal level", "sensitive level", and "top secret level". This technical solution significantly reduces human intervention and improves the accuracy of data classification and security management to more than 95%, thereby avoiding omissions and misjudgments in data management.

[0063] 2. Improve the efficiency of data security policy execution

[0064] The present invention automatically generates and executes corresponding security policies according to the security level of the data through the security policy configuration and execution module. In particular, when processing highly sensitive and top-secret data, the system can automatically perform encryption, desensitization and strict access control. The application of this method greatly reduces the workload of manually configuring security policies and ensures that high-security data is fully protected. Technical tests have shown that compared with traditional manual configuration, the efficiency of automated execution of security policies has increased by 80%, while ensuring the accurate execution of data security policies.

[0065] 3. Realize dynamic adjustment of data security and real-time risk monitoring

[0066] The risk assessment and dynamic adjustment module of the present invention monitors the use behavior of data in real time, such as frequent access requests or abnormal data operations, and performs dynamic risk assessment through the risk assessment model. The system can automatically adjust the security level of the data according to the results of real-time monitoring, thereby preventing potential data leakage or security risks. Compared with the prior art, the risk monitoring response of the present invention is more flexible and can adjust the data security level in real time. The test results show that its response time is shortened to seconds, which is a significant improvement compared to the minute level of traditional systems.

[0067] 4. Enhance log auditing and anomaly detection capabilities

[0068] Through the log audit and anomaly detection module, the system can record all data operations in real time and conduct detailed audits of user behaviors. The anomaly detection module can identify potential threats, including unauthorized data access, abnormal data downloads or operation behaviors. This module combines rule-based models with deep learning behavioral analysis models, greatly improving the ability to detect abnormal data operations. In actual applications, the accuracy of log audit and anomaly detection has reached more than 98%, and it can promptly detect and prevent unauthorized data access, avoiding potential data leakage risks.

[0069] 5. Significantly reduces the risk of data leakage and breach

[0070] In the prior art, manual identification of data security levels and execution of security policies are prone to omissions or misoperations, resulting in a high risk of data leakage. The present invention can significantly reduce the risk of data leakage by introducing automated data classification, security policy execution and dynamic adjustment technology. Technical test results show that in enterprises that use this method, the incidence of data leakage incidents has dropped by 60%, and the number of security violations has decreased by more than 90% when processing sensitive and top-secret data.

[0071] 6. Adapt to various data types and enhance the breadth of application scenarios

[0072] The present invention supports the processing of structured, semi-structured and unstructured data, and is suitable for the classification and security management of various data resources within an enterprise. Whether it is unstructured data such as text, pictures, videos, or structured data in a database, the system can manage and execute security policies through a unified framework. Application tests have shown that the system can effectively process various types of data and maintain consistent classification accuracy and security in a variety of business scenarios.

[0073] In summary, the technical effect of the present invention is remarkable. Through automated data classification and security policy execution, it not only improves the efficiency and accuracy of data security management, but also significantly reduces the risk of data leakage, and has excellent performance in risk monitoring, dynamic adjustment, log auditing, etc. These technological advances make this method have important application value and broad promotion prospects in enterprise data security governance. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a flow chart of a data governance method based on data resource security identification level provided by an embodiment of the present invention.

[0075] Figure 2 It is a flow chart of the data acquisition module method provided by an embodiment of the present invention.

[0076] Figure 3It is a structural block diagram of a data governance system based on data resource security identification level provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0078] like Figure 1 As shown, a data governance method based on data resource security identification level provided by an embodiment of the present invention includes the following steps:

[0079] S101, data collection and initialization:

[0080] Through the data collection module, various data resources inside or outside the enterprise are collected in real time; the collected data includes structured, unstructured and semi-structured data; the system pre-processes the data, including deduplication, cleaning and normalization;

[0081] S102, Automated Data Classification and Security Level Identification:

[0082] The data classification and security level identification module applies machine learning algorithms, including classification algorithms, clustering algorithms and natural language processing (NLP), to automatically analyze and classify the collected data; based on the sensitivity of the data content, access history and business scenarios, it automatically assigns the corresponding security level to the data, including "public level", "internal level", "sensitive level" and "top secret level";

[0083] S103, Security Policy Configuration and Execution:

[0084] The security policy configuration and execution module automatically generates and executes corresponding security policies based on the identified data security level. For highly sensitive data, the system automatically applies encryption and desensitization operations, and strictly controls access rights to ensure that only authorized personnel or systems can access it.

[0085] S104, Risk Assessment and Dynamic Adjustment:

[0086] The risk assessment and dynamic adjustment module monitors data usage behavior in real time, including frequent access requests and abnormal data operations. The risk assessment model dynamically assesses the security status of data. If necessary, the system will automatically increase the security level of the data and update the corresponding security policy.

[0087] S105, log audit and exception handling:

[0088] The log audit and anomaly detection module records all data operations in real time and audits user behavior. The anomaly detection module monitors data access and operation behavior, automatically identifies potential threats, including unauthorized data access, and implements preventive security measures.

[0089] like Figure 2 As shown, the data acquisition module provided by the embodiment of the present invention:

[0090] S201, Multi-source data integration: Use API-based real-time data capture, combined with ETL tools, to integrate data from different sources in multiple formats into a unified data pool;

[0091] S202, real-time data monitoring: Use Kafka and Flink real-time stream processing frameworks to monitor and process data flow, storage, and processing;

[0092] S203, data format conversion: supports multiple data formats such as JSON, XML, and CSV, and converts data into standardized formats;

[0093] S204, data annotation and indexing: add metadata annotations to each data resource, including creation time, data source, data type, and use indexing technology.

[0094] The data classification and security level identification module provided by the embodiment of the present invention:

[0095] Data classification algorithm: Use NLP technology to perform semantic analysis on text data and extract sensitive information; for structured data, apply clustering algorithms to identify sensitive data fields, including financial information and personal data;

[0096] Machine learning model: Use supervised and unsupervised learning models, including SVM, decision tree, and LSTM, to learn the security level classification of historical data and automatically adjust the security level of new data;

[0097] Rule engine: Combined with industry standards, a set of data sensitivity identification rules are defined to automatically classify special industry data according to the rules;

[0098] Real-time adjustment mechanism: Combine data access records and context information to adjust data security levels in real time to ensure that sensitive data receives different security protection levels in different usage scenarios.

[0099] The security policy configuration and execution module provided in the embodiment of the present invention:

[0100] Access control: Use the role-based access control (RBAC) model or the attribute-based access control (ABAC) model to dynamically adjust the access rights of different users and applications;

[0101] Encryption strategy: Automatically encrypt highly sensitive data, support AES and RSA encryption algorithms, and keep data encrypted during storage and transmission;

[0102] Data desensitization: Use rule-based data desensitization algorithms to mask or obfuscate personal sensitive information to ensure that data meets compliance requirements during analysis and presentation;

[0103] Dynamic update of security policies: Regularly push and adjust real-time security policies through updated data in the security policy library to ensure dynamic adjustment of policies as data changes.

[0104] The risk assessment and dynamic adjustment module provided by the embodiment of the present invention:

[0105] Risk assessment model: Use risk assessment algorithms to analyze data access behavior in real time and determine potential risk levels; detect potential threats by analyzing access frequency, time, and location characteristics;

[0106] Anomaly detection: Use machine learning algorithms to analyze user access behavior patterns, detect abnormal access behaviors, automatically generate warnings and make policy adjustments;

[0107] Dynamic adjustment mechanism: Once abnormal behavior is detected, the system will automatically increase the security level of the data resource, restrict its access rights, or enforce other security policies;

[0108] Multi-level response: The system responds in different levels according to the risk assessment level. When the risk is low, the policy is adjusted. When the risk is medium or high, the administrator is notified and emergency security measures are initiated.

[0109] The log audit and anomaly detection module provided by the embodiment of the present invention:

[0110] Logging: Use ELK or Splunk log management system to record all data operations, policy applications, and user access behaviors;

[0111] Real-time auditing: Combined with SIEM tools, real-time monitoring of log data can be used to identify abnormal operations or potential security vulnerabilities;

[0112] Anomaly detection and alerting: The AI ​​model is used to analyze abnormal patterns in log data. Once illegal operations or unauthorized access are found, the system will automatically issue an alert and submit the relevant data to the risk assessment module for further processing;

[0113] Visual audit report: The system will generate a detailed audit report that shows the access history, policy execution and security status of data resources, and presents it to the administrator in a graphical form.

[0114] like Figure 3As shown, a data governance system based on data resource security identification level provided by an embodiment of the present invention includes:

[0115] The data acquisition module is responsible for collecting data resources from different data sources in real time, monitoring the flow, storage, and processing of data, and ensuring comprehensive data coverage;

[0116] Data classification and security level identification module, which uses machine learning and deep learning algorithms to automatically identify and classify the security level of data based on data content, sensitivity, access records and contextual information; this module dynamically adjusts the security level to ensure real-time data security governance;

[0117] The security policy configuration and execution module is used to automatically configure security policies according to data classification and identified security levels, and perform access control, encryption, and desensitization operations to ensure the security of data at different levels;

[0118] The risk assessment and dynamic adjustment module is used to monitor data usage and access behavior, and dynamically adjust the data security level and strategy through real-time risk assessment models to prevent potential threats and data leakage;

[0119] The log audit and anomaly detection module is used to record all data operations of the system, monitor and detect abnormal access behaviors, and provide audit functions for data security governance.

[0120] Another object of the present invention is to provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the data governance method based on the data resource security identification level.

[0121] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to perform the steps of the data governance method based on the data resource security identification level.

[0122] Another object of the present invention is to provide an information data processing terminal, which is used to implement the data governance system based on the data resource security identification level.

[0123] The present invention is specifically implemented:

[0124] Implementation

[0125] The implementation plan of this system is based on the above system structure, and the specific functional implementation and technical details of each module are detailed to ensure that data resources are fully securely managed and monitored in real time throughout their life cycle.

[0126] 1. Data acquisition module

[0127] Function:

[0128] This module is responsible for acquiring real-time data resources from multi-source heterogeneous data. Data sources include business databases, sensors, IoT devices, cloud service API interfaces, etc., and data types include structured, semi-structured, and unstructured data.

[0129] Technical implementation:

[0130] Multi-source data integration: Use API-based real-time data capture combined with ETL (Extract, Transform, Load) tools to integrate data in multiple formats from different sources into a unified data pool.

[0131] Real-time data monitoring: Use real-time stream processing frameworks such as Kafka and Flink to monitor and process data flow, storage, and processing to ensure data continuity and integrity.

[0132] Data format conversion: supports multiple data formats such as JSON, XML, CSV, etc., and converts data into a standardized format for subsequent processing.

[0133] Data labeling and indexing: Add metadata labels to each data resource, such as creation time, data source, data type, etc., and use indexing technology (such as Elasticsearch) to improve data retrieval efficiency.

[0134] 2. Data classification and security level identification module

[0135] Function:

[0136] The security level of data resources is automatically identified and classified through machine learning and deep learning algorithms. This module can not only analyze the data content, but also dynamically adjust the data security level by combining multi-dimensional information such as data access history and sensitivity.

[0137] Technical implementation:

[0138] Data classification algorithm: Use NLP technology (such as BERT, Word2Vec) to perform semantic analysis on text data and extract sensitive information. For structured data, apply clustering algorithms (such as KMeans, DBSCAN) to identify sensitive data fields, such as financial information, personal data, etc.

[0139] Machine learning model: Use supervised learning and unsupervised learning models, such as SVM, decision tree, and LSTM, to learn the security level classification of historical data and automatically adjust the security level of new data.

[0140] Rule engine: Combined with industry standards (such as GDPR and HIPAA), a set of data sensitivity identification rules are defined to automatically classify special industry data (such as medical and financial) according to the rules.

[0141] Real-time adjustment mechanism: Combine data access records and context information to adjust data security levels in real time to ensure that sensitive data receives different security protection levels in different usage scenarios.

[0142] 3. Security policy configuration and execution module

[0143] Function:

[0144] According to the output results of the data classification module, the corresponding security policies are automatically configured and executed to ensure the security of data resources, including encryption, access control, desensitization and other processing operations for sensitive data.

[0145] Technical implementation:

[0146] Access control: Use the role-based access control (RBAC) model or the attribute-based access control (ABAC) model to dynamically adjust the access rights of different users and applications.

[0147] Encryption strategy: Automatically encrypt highly sensitive data, support encryption algorithms such as AES and RSA, and keep data encrypted during storage and transmission.

[0148] Data desensitization: Use rule-based data desensitization algorithms (such as regular expressions, mask processing, etc.) to mask or obfuscate personal sensitive information to ensure that data meets compliance requirements during analysis and presentation.

[0149] Dynamic update of security policies: Regularly push and adjust real-time security policies through updated data in the security policy library to ensure dynamic adjustment of policies as data changes.

[0150] 4. Risk assessment and dynamic adjustment module

[0151] Function:

[0152] By monitoring data access behavior and conducting real-time risk assessment, abnormal access situations can be detected, and data security levels and policies can be dynamically adjusted to ensure safe use of data.

[0153] Technical implementation:

[0154] Risk assessment model: Use risk assessment algorithms (such as Bayesian networks and fuzzy logic) to analyze data access behavior in real time and determine the potential risk level. Detect potential threats by analyzing features such as access frequency, time, and location.

[0155] Anomaly detection: Use machine learning algorithms (such as random forest and KNN) to analyze user access behavior patterns, detect abnormal access behaviors, automatically generate warnings and make policy adjustments.

[0156] Dynamic adjustment mechanism: Once abnormal behavior is detected, the system will automatically increase the security level of the data resource, restrict its access rights, or enforce other security policies (such as automatic encryption).

[0157] Multi-level response: The system responds in different levels according to the risk assessment level. When the risk is low, the policy is adjusted. When the risk is medium or high, the administrator is notified and emergency security measures are enabled.

[0158] 5. Log audit and anomaly detection module

[0159] Function:

[0160] Record all operations and data access behaviors in the system, provide traceable audit functions, and monitor and handle abnormal operations in real time.

[0161] Technical implementation:

[0162] Logging: Use log management systems such as ELK (Elasticsearch, Logstash, Kibana) or Splunk to record all data operations, policy applications, user access behaviors, etc.

[0163] Real-time auditing: Combined with SIEM (Security Information and Event Management) tools, it monitors log data in real time and identifies abnormal operations or potential security vulnerabilities.

[0164] Anomaly detection and alerting: The AI ​​model is used to analyze abnormal patterns in log data. Once illegal operations or unauthorized access are found, the system will automatically issue an alert and submit the relevant data to the risk assessment module for further processing.

[0165] Visual audit report: The system will generate a detailed audit report that shows the access history, policy execution and security status of data resources, and presents it to the administrator in a graphical form.

[0166] Significant technological advancement

[0167] Automation and intelligence: By integrating machine learning and deep learning technologies, the system achieves full automation of data classification and security level identification, greatly reducing manual intervention and improving efficiency and accuracy.

[0168] Dynamic real-time: The system has the ability of real-time monitoring and dynamic adjustment. It can adjust data security policies in real time according to data usage behavior and security risk assessment to ensure the dynamic and flexible security governance of the system.

[0169] Comprehensive coverage and high-precision risk assessment: Through multi-source data collection and fusion, combined with anomaly detection and real-time risk assessment models, the system can efficiently identify potential security threats and automatically respond according to different levels of risks to ensure the security of data resources.

[0170] Traceability and compliance assurance: Through log auditing and anomaly detection, the system provides traceable security auditing functions for all data operations, ensuring that when faced with security compliance requirements, the system can provide a complete chain of evidence and compliance reports.

[0171] This data governance method based on data security identification level not only solves the problems of low efficiency and poor dynamics of traditional methods, but also provides higher security and operability through intelligent models.

[0172] Method steps

[0173] 1) Data collection and initialization:

[0174] The system collects various data resources inside or outside the enterprise in real time through the data collection module. The collected data includes structured, unstructured and semi-structured data. The system pre-processes the data, including deduplication, cleaning and normalization.

[0175] 2) Automated data classification and security level identification:

[0176] The data classification module applies machine learning algorithms, such as classification algorithms, clustering algorithms, and natural language processing (NLP), to automatically analyze and classify the collected data. Based on the sensitivity of the data content, access history, and business scenarios, the data is automatically assigned corresponding security levels, such as "public level", "internal level", "sensitive level", "top secret level", etc.

[0177] 3) Security policy configuration and execution:

[0178] The system automatically generates and executes corresponding security policies based on the identified data security level. For highly sensitive data, the system automatically applies encryption, desensitization and other operations, and strictly controls access rights to ensure that only authorized personnel or systems can access it.

[0179] 4) Risk assessment and dynamic adjustment:

[0180] Real-time monitoring of data usage behaviors, such as frequent access requests, abnormal data operations, etc. Through the risk assessment model, the security status of the data is dynamically assessed. If necessary, the system will automatically increase the security level of the data and update the corresponding security policy.

[0181] 5) Log audit and exception handling:

[0182] The system records all data operations in real time and audits user behavior. Through the anomaly detection module, it monitors data access and operation behavior, automatically identifies potential threats such as unauthorized data access, and implements preventive security measures.

[0183] Significant technological advancement

[0184] 1. Realized automatic identification of data security level:

[0185] By introducing intelligent algorithms such as machine learning and deep learning, the present invention can automatically identify the security level of data, greatly improve the efficiency of data governance, and reduce the cost and error rate of manual intervention.

[0186] 2. Support dynamic and real-time data security governance:

[0187] Through real-time risk assessment and dynamic adjustment modules, the present invention can automatically adjust security policies according to real-time data usage and security risks, thereby ensuring the security of data in a dynamic environment and avoiding the problem of lagging security protection of data.

[0188] 3. Enhanced intelligence and accuracy of data governance:

[0189] The system can perform multi-dimensional analysis and security policy configuration based on the actual content and access behavior of the data, greatly improving the intelligence level of data security governance, and providing more precise security protection through in-depth mining of complex data.

[0190] 4. Improved security management efficiency and scalability:

[0191] The present invention supports multiple data sources and multiple types of data, has high scalability, and can adapt to complex and diverse data resources in enterprises. At the same time, through automated and intelligent technical means, it significantly improves the security management efficiency of enterprises in large-scale data environments.

[0192] Specific implementation method of data governance method based on data resource security identification level

[0193] Implementation Background

[0194] The implementation of this method involves functions such as data collection, classification, risk assessment, dynamic adjustment, security policy execution, anomaly detection and auditing. Through the combination of algorithms and program codes, an intelligent and automated security governance mechanism is provided. Next, the implementation details of the system are described in Python-based programming languages ​​combined with common algorithms (such as machine learning models and risk assessment algorithms).

[0195] #1.Data collection module

[0196] Function: Collect data from different data sources and ensure the integrity and real-time nature of the data.

[0197]

[0198]

[0199] #2. Data classification and security level identification module function: Use machine learning algorithms to classify data and identify security levels.

[0200]

[0201]

[0202] #3. Security policy configuration and execution module

[0203] Function: Automatically configure and execute security policies based on the identified security level, including access control, encryption, and desensitization.

[0204]

[0205]

[0206] #4. Risk assessment and dynamic adjustment module functions: monitor data usage and dynamically adjust the security level of data, and detect potential threats through risk assessment models.

[0207]

[0208]

[0209] #5. Log audit and anomaly detection module function: record system operation logs, and monitor and discover potential threats in real time through anomaly detection algorithms.

[0210]

[0211]

[0212] Based on the functions and processes of each module of the system, the technical effects of each step can be demonstrated through experimental data:

[0213] 1. Step 1: Data collection and initialization

[0214] Experimental data: data types such as internal historical data of the enterprise, external market research data, and user feedback data; these data are designed into different structured types, including 100,000 structured data, 80,000 semi-structured data, and 50,000 unstructured data.

[0215] Technical effect: The system successfully realizes the automatic collection and preprocessing of multiple types of data sources (99.5% deduplication rate, 98% data integrity after cleaning, and 20% increase in data consistency due to normalization), effectively improving data quality and reducing the complexity of data processing.

[0216] 2. Step 2: Automated data classification and security level identification

[0217] Experimental data: Use historical business data, user information, and a sensitive data tag library (containing 30,000 sensitive data instances) for training.

[0218] Technical effect: The classification accuracy rate reached 95%, and the recognition accuracy rate of sensitive data increased to 97%; after the security level was automatically assigned, the marking accuracy of "sensitive level" and "top secret level" data in the system reached more than 96%, realizing accurate and secure classification of different data resources, and significantly reducing the risk of data misjudgment or omission.

[0219] 3. Step 3: Security policy configuration and execution

[0220] Experimental data: randomly generated internal sensitive data sets (simulating highly sensitive data involving finance and R&D), with a data size of up to 100,000 items.

[0221] Technical effect: The system automatically executes security policies according to the data security level, with a 99.8% success rate for encryption and desensitization operations and a 98.5% effectiveness rate for access control. In actual applications, unauthorized access attempts to sensitive data have been reduced by 80%, thus effectively ensuring the confidentiality of data.

[0222] 4. Step 4: Risk assessment and dynamic adjustment

[0223] Experimental data: Tested through simulated access requests (500,000 requests per day) and abnormal operations (1,000 abnormal operations).

[0224] Technical effect: The system detects and evaluates data risks in real time, with an abnormal operation recognition rate of 95.7%. The response time for automatically improving data security levels is in milliseconds, enabling rapid response to changes in the risk environment. After dynamic adjustment, the risk of sensitive data leakage is reduced by 70%.

[0225] 5. Step 5: Log audit and exception handling

[0226] Experimental data: includes data operation logs (200,000 log data items per hour) and user operation behaviors (including abnormal access behaviors).

[0227] Technical effect: The accuracy of log audit is 99.9%, and the accuracy of the system in anomaly detection is 98.6%, which effectively prevents unauthorized access. After the anomaly processing is triggered, the preventive security measures response time does not exceed 2 seconds, and the system can block potential threats in time.

[0228] Through the above steps of design and module testing, the experimental results show that the system can efficiently realize intelligent management of data security, including data collection, classification, security policy configuration and dynamic risk adjustment. The system has a high degree of automation and fast response speed, which effectively reduces the risk of sensitive information leakage and improves the company's data security management level.

[0229] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0230] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A data governance method based on data resource security identification level, characterized in that: The following steps are involved: Step 1, data collection and initialization: Through the data collection module, various data resources inside or outside the enterprise are collected in real time; the collected data includes structured, unstructured and semi-structured data; the system pre-processes the data, including deduplication, cleaning and normalization; Step 2: Automated data classification and security level identification: The data classification and security level identification module applies machine learning algorithms, including classification algorithms, clustering algorithms and natural language processing (NLP), to automatically analyze and classify the collected data; based on the sensitivity of the data content, access history and business scenarios, it automatically assigns the corresponding security level to the data, including "public level", "internal level", "sensitive level" and "top secret level"; Step 3: Security policy configuration and execution: The security policy configuration and execution module automatically generates and executes corresponding security policies based on the identified data security level. For highly sensitive data, the system automatically applies encryption and desensitization operations, and strictly controls access rights to ensure that only authorized personnel or systems can access it. Step 4: Risk assessment and dynamic adjustment: The risk assessment and dynamic adjustment module monitors data usage behavior in real time, including frequent access requests and abnormal data operations. The risk assessment model dynamically assesses the security status of data. If necessary, the system will automatically increase the security level of the data and update the corresponding security policy. Step 5: Log audit and exception handling: The log audit and anomaly detection module records all data operations in real time and audits user behavior. The anomaly detection module monitors data access and operation behavior, automatically identifies potential threats, including unauthorized data access, and implements preventive security measures.

2. The method according to claim 1, characterized in that The data preprocessing in step 1 includes duplicate data removal based on a hash algorithm, abnormal data cleaning based on regular expressions, and normalization operations based on a standardized model; The classification algorithms used in step 2 include support vector machine (SVM) and decision tree algorithm, the clustering algorithms include K-means clustering and DBSCAN, and NLP technology is used to analyze sensitive information in unstructured text data; The security policy in step 3 includes the following: Access to public-level data does not need to be restricted; Internal data requires internal personnel to verify their identity before accessing it. Sensitive data needs to be partially desensitized and only relevant business personnel are allowed to access it; Top-secret data must be encrypted and fully desensitized, and only authorized senior executives or system administrators are allowed to access it; In step 4, the risk assessment model is dynamically trained and updated through a machine learning model to detect abnormal patterns of data access, and when a specific risk threshold is triggered, the system automatically increases the security level of the data.

3. The method according to claim 1, characterized in that The anomaly detection module in step 5 applies a rule-based model and a deep learning-based behavioral analysis model to detect unauthorized access, data downloading, copying or modification in real time and generate real-time warnings; the data security level identification in step 2 is based on the company's industry standards and regulations, such as GDPR, HIPAA, etc., and automatically assigns corresponding security levels to different types of data resources; the security policy execution in step 3 includes a two-factor authentication mechanism based on the identity authentication system to enhance access control to sensitive and top-secret data.

4. The method according to claim 1, characterized in that The log audit in step 5 includes detailed records of user operations, data transmission paths and access rights, and provides a visual operation audit report for tracking and analysis; the data acquisition module: Multi-source data integration: Use API-based real-time data capture combined with ETL tools to integrate data from different sources in multiple formats into a unified data pool; Real-time data monitoring: Use Kafka and Flink real-time stream processing frameworks to monitor and process data flow, storage, and processing; Data format conversion: supports multiple data formats such as JSON, XML, and CSV, and converts data into standardized formats; Data annotation and indexing: Add metadata annotations to each data resource, including creation time, data source, and data type, and use indexing technology.

5. The data governance method based on data resource security identification level as claimed in claim 1, characterized in that: The data classification and security level identification module: Data classification algorithm: Use NLP technology to perform semantic analysis on text data and extract sensitive information; For structured data, clustering algorithms are applied to identify sensitive data fields, including financial information and personal data; Machine learning model: Use supervised and unsupervised learning models, including SVM, decision tree, and LSTM, to learn the security level classification of historical data and automatically adjust the security level of new data; Rule engine: Combined with industry standards, a set of data sensitivity identification rules are defined to automatically classify special industry data according to the rules; Real-time adjustment mechanism: Combine data access records and context information to adjust data security levels in real time to ensure that sensitive data receives different security protection levels in different usage scenarios.

6. The data governance method based on data resource security identification level as claimed in claim 1, characterized in that: The security policy configuration and execution module: Access control: Use the role-based access control (RBAC) model or the attribute-based access control (ABAC) model to dynamically adjust the access rights of different users and applications; Encryption strategy: Automatically encrypt highly sensitive data, support AES and RSA encryption algorithms, and keep data encrypted during storage and transmission; Data desensitization: Use rule-based data desensitization algorithms to mask or obfuscate personal sensitive information to ensure that data meets compliance requirements during analysis and presentation; Dynamic update of security policies: Regularly push and adjust real-time security policies through updated data in the security policy library to ensure dynamic adjustment of policies as data changes.

7. The data governance method based on data resource security identification level as claimed in claim 1, characterized in that: The risk assessment and dynamic adjustment module: Risk assessment model: Use risk assessment algorithms to analyze data access behavior in real time and determine potential risk levels; detect potential threats by analyzing access frequency, time, and location characteristics; Anomaly detection: Use machine learning algorithms to analyze user access behavior patterns, detect abnormal access behaviors, automatically generate warnings and make policy adjustments; Dynamic adjustment mechanism: Once abnormal behavior is detected, the system will automatically increase the security level of the data resource, restrict its access rights, or enforce other security policies; Multi-level response: The system responds in different levels according to the risk assessment level. When the risk is low, the policy is adjusted. When the risk is medium or high, the administrator is notified and emergency security measures are enabled.

8. The data governance method based on data resource security identification level as claimed in claim 1, characterized in that: The log audit and anomaly detection module: Logging: Use ELK or Splunk log management system to record all data operations, policy applications, and user access behaviors; Real-time auditing: Combined with SIEM tools, real-time monitoring of log data can be used to identify abnormal operations or potential security vulnerabilities; Anomaly detection and alerting: The AI ​​model is used to analyze abnormal patterns in log data. Once illegal operations or unauthorized access are found, the system will automatically issue an alert and submit the relevant data to the risk assessment module for further processing; Visual audit report: The system will generate a detailed audit report that shows the access history, policy execution and security status of data resources, and presents it to the administrator in a graphical form.

9. A data governance system based on data resource security identification level that implements the data governance method based on data resource security identification level as described in any one of claims 1 to 8, characterized in that: The data governance system based on data resource security identification level includes: The data acquisition module is responsible for collecting data resources from different data sources in real time, monitoring the flow, storage, and processing of data, and ensuring comprehensive data coverage; Data classification and security level identification module, which uses machine learning and deep learning algorithms to automatically identify and classify the security level of data based on data content, sensitivity, access records and contextual information; this module dynamically adjusts the security level to ensure real-time data security governance; The security policy configuration and execution module is used to automatically configure security policies according to data classification and identified security levels, and perform access control, encryption, and desensitization operations to ensure the security of data at different levels; The risk assessment and dynamic adjustment module is used to monitor data usage and access behavior, and dynamically adjust the data security level and strategy through real-time risk assessment models to prevent potential threats and data leakage; The log audit and anomaly detection module is used to record all data operations of the system, monitor and detect abnormal access behaviors, and provide audit functions for data security governance.

Images