Intelligent authority management system of data lake platform

Abstract

The invention discloses an intelligent authority management system of a data lake platform, and relates to the technical field of safety management of data, a data classification module is used for determining classification and grading identifiers of data files according to the knowing range of data information; the authority management and control module is used for judging the access authority of the user according to the user role, the application scene and the data security level which are mutually associated; the dynamic adjustment module is used for updating the security level identifier and the corresponding security policy according to a known range adjustment request initiated by a management department; the background monitoring module is used for monitoring all data and corresponding authority configuration and automatically repairing abnormal authority configuration according to a preset security policy; according to the invention, full-process management and control of dynamic permission adjustment, multi-dimensional permission adaptation and intelligent exception repair are realized.

Description

Technical Field

[0001] This invention relates to the field of data security management technology, specifically to an intelligent access control system for a data lake platform. Background Technology

[0002] During the AI ​​transformation process, enterprises use AI data lakes to collect core data from various departments, such as R&D drawings, financial salaries, and customer privacy. The scope of knowledge and access permissions for different data are clearly limited, and a data access management system is needed to manage them.

[0003] A document titled "A Database Row Access Control Method and System" (Document No. CN107239711A) discloses a data access control system. This system performs preliminary parsing of user-input Structured Query Language (SQL) queries within a pre-defined data access control system to obtain an SQL parse tree containing unresolved nodes. It then refines the information of these unresolved nodes using metadata. Next, it compares the current user information with pre-stored user access information in the data access control system to obtain the user's access restriction information. Finally, it combines the access restriction information with the refined unresolved nodes to generate filter nodes. These filter nodes are then added to the unresolved nodes in the SQL parse tree to generate the final SQL parse tree. The user's SQL query result is then obtained from the final SQL parse tree. While this document addresses the problem of coarse granularity in traditional table-level or column-level access control, it still has the following shortcomings:

[0004] Firstly, the technical solution in this paper explicitly states that the permission restriction information only includes name, age, contact information, and job title, and does not support custom fields or dynamically expanded dimensions. However, in practical applications, the database may involve multiple permission division criteria such as department and project to which the data belongs, and fixed dimensions cannot meet the needs of complex scenarios. Secondly, it lacks a dynamic permission adjustment mechanism. When a user's job title changes, the pre-stored permission information needs to be manually modified and the comparison process needs to be retried. It cannot support dynamic scenarios such as permission inheritance or transfer, and its adaptability is limited. Summary of the Invention

[0005] This invention provides an intelligent permission management system for a data lake platform, enabling full-process control of dynamic permission adjustment, multi-dimensional permission adaptation, and intelligent anomaly repair.

[0006] This invention provides the following technical solution: an intelligent access control system for a data lake platform, comprising:

[0007] The data classification module is used to determine the classification and grading identifiers of data files based on the scope of knowledge of the data.

[0008] The access control module is used to determine a user's access permissions based on the interrelated user roles, application scenarios, and data security levels.

[0009] The dynamic adjustment module is used to update the security classification identifier and corresponding security policy based on the scope of knowledge adjustment request initiated by the management department.

[0010] The background monitoring module is used to monitor all data and corresponding permission configurations, and automatically repair abnormal permission configurations according to preset security policies.

[0011] As a further technical solution of the present invention, the data classification module includes:

[0012] The scope of knowledge is divided into units, and the data lake platform and enterprises collaborate to determine the scope of data knowledge, which is divided into access scopes such as R&D, finance, HR, marketing, and the entire company.

[0013] The data tag generation unit assigns corresponding data tags to different files during data file encryption, based on the aforementioned permission scope.

[0014] As a further technical solution of the present invention, the access control module includes:

[0015] The attribute binding unit uses attribute-based encryption technology to associate and bind user roles, application scenarios, data security levels, and access permissions.

[0016] The access control unit determines a user's access permissions based on the interrelated user roles, application scenarios, and data security levels.

[0017] As a further technical solution of the present invention, the dynamic adjustment module includes:

[0018] The application unit allows management departments to initiate requests to adjust the scope of knowledge through a visual interface, and fill in the target scope of knowledge and the reason for the adjustment.

[0019] The update unit automatically updates the security classification identifier and corresponding security policy after the information security department approves the adjustment request.

[0020] As a further technical solution of the present invention, the background monitoring module includes:

[0021] The unit is periodically scanned to check the owner attributes, access control lists, and permission inheritance relationships of data files at a preset frequency.

[0022] The real-time monitoring unit captures permission modifications, owner changes, and abnormal access operations through a kernel-level hook mechanism, achieving a response time within seconds.

[0023] The anomaly handling unit automatically repairs abnormal permission configurations according to preset security policies based on the hierarchical alarms triggered by the system.

[0024] The audit log unit records abnormal events, repair actions, and associated user information, reducing the manual workload for administrators.

[0025] As a further technical solution of the present invention, the collaborative process of the data classification module, the access control module, the dynamic adjustment module, and the background monitoring module is as follows:

[0026] Step 1: The scope of knowledge of the data is determined by the knowledge scope division unit in collaboration with the enterprise. Then, the data tag generation unit assigns corresponding data tags to different files when encrypting the data files according to the knowledge scope.

[0027] Step 2: The attribute binding unit uses attribute-based encryption technology to associate and bind user role, application scenario, and data security level with access permissions. Then, the access control unit determines the user's access permissions based on the above association.

[0028] Step 3: The application unit supports the management department to initiate a request to adjust the scope of knowledge through the visual interface and fill in the target scope of knowledge and the reason for adjustment. After the information security department approves the request, the update unit will automatically update the security level identifier and the corresponding security policy.

[0029] Step 4: The periodic scanning unit checks the owner attributes, access control lists, and permission inheritance relationships of data files at a preset frequency. At the same time, the kernel-level hook mechanism is used to capture permission modifications, owner changes, and abnormal access operations by the real-time monitoring unit and achieve a response within seconds. Then, the exception handling unit automatically repairs the abnormal permission configuration according to the preset security policy based on the triggered hierarchical alarms. Finally, the audit log unit records the abnormal events, repair actions, and associated user information.

[0030] The present invention has the following beneficial effects:

[0031] By associating user roles, application scenarios, and data security levels, automatic adaptation is facilitated. When user attributes or scenarios change, permissions are automatically updated without manual reconfiguration. This ensures that the appropriate user can access encrypted data, but other users or applications cannot open the encrypted text. Automatic repair and auditing reduce the manual workload for administrators. These mechanisms not only achieve the core goals of secure, sustainable, and self-managed knowledge bases but also upgrade access control from passive defense to proactive intelligence. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the collaborative process of each module in this invention.

[0033] Figure 2 This is a structural block diagram of the background monitoring module in this invention. Detailed Implementation

[0034] The technical solutions of the embodiments of this specification will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of this specification and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of this specification.

[0035] In the description of this application, terms such as "front," "rear," "inner," "outer," "upper," "lower," "left," and "right" are used only to indicate orientation or positional relationship for the convenience of describing the embodiments and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.

[0036] All data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0037] Before introducing the technical solutions described in this manual, the application scenarios and related technologies of the technical solutions will be introduced.

[0038] During the AI ​​engineering process in enterprises, different intelligent agents and AI applications will be developed based on the different application scenarios of different business departments. Therefore, different application scenarios require various different data. Much of this data is not allowed to be accessed across departments. We must ensure that the exposure of related data does not expand during the enterprise's AI engineering process. For example, the company's payroll should only be accessible to the finance department or HR, and should not be exposed to employees in other departments due to the implementation of AI engineering; core design drawings from the R&D department should only be used for training AI models to assist R&D, and should not be accessed by the marketing department's precision marketing system; customer privacy data from the marketing department should only be used for personalized responses in intelligent customer service scenarios, and should not be accessed by the R&D department's AI applications. These scenarios all require the data lake platform to have comprehensive data file classification, grading, and security level management capabilities, achieving end-to-end control from "data attribute definition" to "access control." To this end, this application provides an intelligent access control system for a data lake platform. Through the collaborative work of a data classification module, an access control module, a dynamic adjustment module, and a backend monitoring module, it achieves precise matching, dynamic updating, and intelligent control of data security levels and permissions. This invention can effectively prevent cross-departmental data leakage and abuse of sensitive data in AI projects, reduce access control maintenance costs, and is applicable to core data access management scenarios of AI data lakes in various enterprises.

[0039] Please see Figure 1-2 As shown, an intelligent access control system for a data lake platform includes:

[0040] The data classification module is used to determine the classification and grading identifiers of data files based on the scope of knowledge of the data.

[0041] The access control module is used to determine a user's access permissions based on the interrelated user roles, application scenarios, and data security levels.

[0042] The dynamic adjustment module is used to update the security classification identifier and corresponding security policy based on the scope of knowledge adjustment request initiated by the management department.

[0043] The background monitoring module is used to monitor all data and corresponding permission configurations, and automatically repair abnormal permission configurations according to preset security policies.

[0044] The data classification module includes:

[0045] The scope of knowledge is divided into units, and the data lake platform and enterprises collaborate to determine the scope of data knowledge, which is divided into access scopes such as R&D, finance, HR, marketing, and the entire company.

[0046] The data tag generation unit assigns corresponding data tags to different files during data file encryption, based on the aforementioned permission scope.

[0047] In this application, the classification and grading of data files are determined based on the scope of access to the data. During the initial processing of data assets, the scope of data access will be discussed with the enterprise, and the data in all knowledge bases will be identified and defined based on the discussion results. The scope of access can be divided into different permission ranges such as R&D, finance, HR, marketing, production, supply chain, and company-wide, according to business needs. Based on different permission ranges, different data files will be tagged with corresponding data tags during data file encryption.

[0048] The access control module includes:

[0049] The attribute binding unit uses attribute-based encryption technology to associate and bind user roles, application scenarios, data security levels, and access permissions.

[0050] The access control unit determines a user's access permissions based on the interrelated user roles, application scenarios, and data security levels.

[0051] The above employs an attribute-based encryption (ABE) and security level labeling multi-dimensional permission model to associate user roles (such as R&D engineers, HR specialists, and AI algorithm engineers), application scenarios (such as model training, intelligent customer service, precision marketing, and intelligent agent question answering), and data security levels (such as top secret and confidential) for easy automatic adaptation. For example, the data permission scope for an R&D engineer is R&D; attribute-based encryption is used to bind and encrypt the user role and this permission scope, while also associating it with the corresponding application scenario and data security level.

[0052] For example, the "R&D design drawings" data in the R&D department can only be accessed by AI intelligent agents in R&D application scenarios, but not by intelligent marketing AI intelligent agents in the marketing system.

[0053] The finance department's intelligent reporting system (AI application) accesses "finance-salary data" in the "generate monthly report" scenario, but the AI ​​cannot answer inquiries from other departments regarding financial data.

[0054] The dynamic adjustment module includes:

[0055] The application unit allows management departments to initiate requests to adjust the scope of knowledge through a visual interface, and fill in the target scope of knowledge and the reason for the adjustment.

[0056] The update unit automatically updates the security classification identifier and corresponding security policy after the information security department approves the adjustment request.

[0057] In this application, the Trusted Data Lake Platform supports a dynamic security level (scope of knowledge) adjustment mechanism to address changing needs throughout the data lifecycle: management departments can manually initiate a scope of knowledge adjustment application through the "Knowledge Base Management Platform Interface" (e.g., adjusting the scope of knowledge of "R&D Department" to "Entire Company"), fill in the reason for the adjustment, and after review by the information security department, the system will automatically update the security level identifier and security policy.

[0058] In summary, through the aforementioned data file classification and classification and security level identification management mechanism, the data lake platform achieves full-link control with "clear data attributes, accurate security level identification, and adapted security policies," completely solving problems such as "cross-departmental data exposure" and "abuse of sensitive data" in enterprise AI projects. This allows enterprises to flexibly access data according to business needs while ensuring the security of core business secrets.

[0059] The background monitoring module includes:

[0060] The unit is periodically scanned to check the owner attributes, access control lists, and permission inheritance relationships of data files at a preset frequency.

[0061] The real-time monitoring unit captures permission modifications, owner changes, and abnormal access operations through a kernel-level hook mechanism, achieving a response time within seconds.

[0062] The anomaly handling unit automatically repairs abnormal permission configurations according to preset security policies based on the hierarchical alarms triggered by the system.

[0063] The audit log unit records abnormal events, repair actions, and associated user information, reducing the manual workload for administrators.

[0064] The background daemon uses a dual-mode approach of "periodic scanning and real-time monitoring" to build an intelligent access control closed loop: periodic scanning is performed once per hour by default (with support for enterprise-defined frequencies), comprehensively verifying the owner attributes, ACL (Access Control List) rules, and permission inheritance relationships of knowledge base files, with a focus on covering the permission configuration of sensitive data (such as R&D design drawings and financial statements); real-time monitoring uses kernel-level hooks (such as the inotify mechanism in Linux systems and FileSystemWatcher in Windows) to capture operations such as file permission modifications, owner changes, and abnormal access (such as unauthorized users frequently accessing top-secret files), achieving "second-level response". When an anomaly is detected (such as a marketing department user being mistakenly granted write permissions to a file in the R&D department, or the file owner being maliciously changed to an external account), the system first triggers a tiered alert (sensitive data anomalies trigger a high-level alert, while non-sensitive data triggers a normal alert). Then, it automatically executes remediation actions according to the company's preset security policies. For example, the owner of core files in the R&D department must be the R&D director, and the ACL rules must restrict it to "read-only for R&D personnel and write-only for R&D managers." The daemon will compare the current configuration with the preset rules, automatically restore the correct owner information and ACL permissions, and write the abnormal event (including trigger time, anomaly type, and file paths involved), remediation actions, and associated users (if traced back to the operator through logs) in detail to the system log, forming a complete "discovery-alert-remediation-audit" chain.

[0065] These mechanisms not only achieve the core goals of "secure, sustainable, and self-managed knowledge base," but also upgrade access control from "passive defense" to "proactive intelligence": through automatic repair and auditing, they reduce the manual workload of administrators (estimated to reduce access control maintenance costs by approximately 70%). For example, after a financial company adopted this platform, the number of business interruptions caused by access issues dropped from 3 times per month to 0 times, and the incidence of data breaches decreased by 95%, fully validating the effectiveness of intelligent and secure access control—ensuring data security while supporting the rapid iteration of enterprise AI applications and business innovation.

[0066] The collaborative process of the data classification module, access control module, dynamic adjustment module, and backend monitoring module is as follows:

[0067] Step 1: The scope of knowledge of the data is determined by the knowledge scope division unit in collaboration with the enterprise. Then, the data tag generation unit assigns corresponding data tags to different files when encrypting the data files according to the knowledge scope.

[0068] Step 2: The attribute binding unit uses attribute-based encryption technology to associate and bind user role, application scenario, and data security level with access permissions. Then, the access control unit determines the user's access permissions based on the above association.

[0069] Step 3: The application unit supports the management department to initiate a request to adjust the scope of knowledge through the visual interface and fill in the target scope of knowledge and the reason for adjustment. After the information security department approves the request, the update unit will automatically update the security level identifier and the corresponding security policy.

[0070] Step 4: The periodic scanning unit checks the owner attributes, access control lists, and permission inheritance relationships of data files at a preset frequency. At the same time, the kernel-level hook mechanism is used to capture permission modifications, owner changes, and abnormal access operations by the real-time monitoring unit and achieve a response within seconds. Then, the exception handling unit automatically repairs the abnormal permission configuration according to the preset security policy based on the triggered hierarchical alarms. Finally, the audit log unit records the abnormal events, repair actions, and associated user information.

[0071] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. An intelligent permission management system of a data lake platform, characterized in that, The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. 2.The intelligent permission management system of a data lake platform according to claim 1, characterized in that, The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. 3.The intelligent permission management system of a data lake platform according to claim 1, characterized in that, The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. 4.The intelligent permission management system of a data lake platform according to claim 1, characterized in that, The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. 5.The intelligent permission management system of a data lake platform according to claim 1, characterized in that, The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. 6.The intelligent permission management system of a data lake platform according to claim 1, characterized in that, The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to a data classification and permission management method and system. The application relates to Step 4, periodically check the owner attribute, access control list and permission inheritance relationship of the data file by the preset frequency of the scanning unit, and capture the permission modification, owner change and abnormal access operation by the real-time monitoring unit through the kernel-level hook mechanism and realize the second-level response, then repair the abnormal permission configuration according to the triggered hierarchical alarm by the abnormal processing unit according to the preset security strategy, finally record the abnormal event, repair action and associated user information through the audit log unit.

Images