Metadata permission auditing system implementation method and metadata permission auditing system
By designing a metadata permission audit system based on SpringBoot and Elasticsearch, using the common REST API and audit monitoring SDK, the problems of metadata permission management and audit in big data security are solved, and efficient and secure management of data resources are achieved.
Patent Information
- Application Number
- CN202510046403.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-13
- Publication Date
- 2025-04-22
AI Technical Summary
How to achieve efficient and efficient management of big data security through access control and permission audit of metadata, especially how to protect the use of data resources when facing massive data.
Design a metadata permission audit system, based on the SpringBoot framework and Elasticsearch distributed storage system, and implement the management and audit monitoring of metadata permission policies through the common REST API to ensure that the permissions for data access and operation are audited and monitored.
It realizes unified management and audit monitoring of metadata permission policies, ensures the security of data resources, and provides users with a safer and more convenient metadata usage environment.
Smart Images
Figure CN119862594A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of big data security technology, and in particular to a method for implementing a metadata authority audit system. Background Art
[0002] According to IDC, the amount of global data doubles approximately every two years. The advent of big data has led to a revolution, prompting the rapid development of big data and cloud service technologies. Faced with the application of massive amounts of data in storage, mining, and analysis, governments and enterprises have increasingly strong demands for big data security, and data security plays an increasingly important role. How to more safely and timely protect the use rights of data resources has become a challenge that the industry is increasingly concerned about.
[0003] In the era of big data, data is no longer a simple record of information, but has become an increasingly important information asset. Faced with massive amounts of data, how to authorize and authenticate data has become an increasingly important topic. Data permissions refer to the control of data resource visibility for system users, and mainly include three elements: 1. Subject. In a narrow sense, the subject refers to the user alone. In a broad sense, the identity, role, position, etc. related to the user concept can be called the subject; 2. Resources, that is, data that needs to control the scope of access; 3. Rules, that is, the conditions applicable to the subject for specific data resources. So how to achieve data isolation through data authorization, protect user data privacy, and how to maintain data authorization information have also become issues that enterprises and governments are constantly exploring.
[0004] Data auditing refers to compliance management that conducts fine-grained audits of data resource access operations, issues alerts for risky behaviors encountered by data resources, and blocks attacks. By recording, analyzing, and reporting on user access to data resources, compliance reports can be generated and incidents can be traced back to their source. Data auditing plays an important role in the audit of data security governance. Only by comprehensively monitoring and recording data operations, access, and risk conditions can compliance checks on data use be completed, and data access and operation behaviors be supervised.
[0005] How to achieve secure and efficient data management through metadata access control and permission auditing is an urgent problem that needs to be solved by enterprises and governments. Summary of the invention
[0006] The technical task of the present invention is to address the above shortcomings and provide a metadata permission audit system implementation method and a metadata permission audit system, which supports the uniformity and universality of permission policy management, and at the same time audits and monitors data access and operation permissions to ensure the security of data resources and create a safer and more convenient use environment for users to use metadata.
[0007] The technical solution adopted by the present invention to solve its technical problem is:
[0008] A method for implementing a metadata permission audit system, which implements metadata permission audit based on the SpringBoot framework and the Elasticsearch distributed storage system. Through a common REST API, users can manage and maintain metadata permission policy information and audit and analyze access operation behaviors of permission policies.
[0009] Users manage permission policy information through the REST API;
[0010] The audit monitoring SDK monitors access to permission policy information and analyzes audit logs at regular intervals.
[0011] This method conveniently maintains permission policy information through the general REST API; timely monitors and warns the use of permission policy information through the audit monitoring SDK package; creates a safer and more convenient environment for users to use metadata. Solve the problem of metadata access control and permission auditing.
[0012] Furthermore, the general REST API allows users to manage and maintain the permission policy information of metadata, and publishes the permission policy access, creation, update and deletion functions as REST API interfaces, allowing data resource managers and other application services to call them;
[0013] The audit monitoring SDK is used to monitor the calls of the data permission policy system REST API and generate audit monitoring logs based on the calling operation behavior;
[0014] Combine the data permission policy management system and audit monitoring SDK to implement metadata permission and audit supervision functions.
[0015] Furthermore, for the creation and update of permission policies, the permission policy management of data sources, data tables, and fields is implemented through a unified and universal JSON object format.
[0016] Furthermore, the management of permission policy information is implemented through a common JSON object format, including the following information:
[0017] 1) Data resource information, including basic metadata information included in the permission policy, supports maintenance and management of multi-level metadata information, including databases, data tables, and table fields;
[0018] 2) Permission information, including the user's permission to use data resources. Users can customize their permissions;
[0019] 3) Subject information, including the subject's identity type. Users can customize settings and maintain (user, role, user group, etc.), and also support maintaining the user subject's ID and name;
[0020] 4) Basic information of the permission policy, including creation time, validity period, etc.
[0021] The data permission policy of this method supports the general JSON data format and is not restricted by the data resource type and user subject. Any data permission policy information that follows this data format can be maintained in the data permission policy management system. The generality of the data permission policy mentioned in this method means that there is no programmatic constraint on data resources and authorization subjects. Data resource managers can authorize data resources at any level without having to consider the permissions of their parent resources or child resources.
[0022] Furthermore, the management of the data resource information, in order to achieve the universality of data resource management, the universal authority policy data format includes: data resource type, data resource id, data resource name, parent resource id;
[0023] The management of the permission information supports all permissions for operations such as adding, querying, updating, and deleting data tables, and all permissions for operations such as querying, modifying, and deleting table fields. Data resource managers can maintain customized permission policy information for data resources.
[0024] All operations on metadata will go through the data permission system. Based on the metadata information to be accessed or operated, the permission policy information corresponding to the metadata will be obtained, and the operation behavior will be compared with the authorized operation behavior in the permission policy. Based on the comparison results, it will decide whether to allow it to continue to operate on the metadata.
[0025] Furthermore, the audit monitoring SDK monitors access to permission policy information and periodically analyzes audit logs.
[0026] The permission policy management system is monitored in real time through the audit monitoring SDK package. All REST APIs of the permission policy management system are connected to the audit monitoring SDK package. All operations that access permission policy information through the REST API will be monitored by the audit monitoring SDK package. The monitoring indicators include: metadata information, access time, data authentication results, user subjects, operation behaviors and other information. The monitoring results of data permission policy information are maintained in the form of audit logs in the ES storage system.
[0027] Furthermore, the audit monitoring SDK extends the scheduled audit function based on the xxl-job open source component. The manager obtains the audit monitoring log at a fixed period by setting scheduled tasks, expands based on the audit log, performs statistical analysis based on metadata or authorized subjects, generates metadata access statistical reports, or generates alarm information based on metadata abnormal access logs.
[0028] The present invention also claims to protect a metadata permission audit system, which implements metadata permission audit based on the SpringBoot framework and the Elasticsearch distributed storage system, allows users to manage and maintain metadata permission policy information through a common REST API, and audits and analyzes access operation behaviors of permission policies;
[0029] Users manage permission policy information through the REST API;
[0030] The audit monitoring SDK monitors access to permission policy information and periodically analyzes audit logs;
[0031] The system specifically implements metadata permission auditing through the above method.
[0032] The present invention also claims protection for a metadata authority audit system implementation device, comprising: at least one memory and at least one processor;
[0033] The at least one memory is used to store a machine-readable program;
[0034] The at least one processor is used to call the machine-readable program to implement the above method.
[0035] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which implement the above method when executed by a processor.
[0036] Compared with the prior art, the metadata authority audit system implementation method and metadata authority audit system of the present invention have the following beneficial effects:
[0037] 1. The present invention can realize a unified and universal metadata rights policy management system and maintain rights policy information.
[0038] 2. The present invention can monitor the operation behavior on the authority policy and generate an audit log. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is a flowchart of a method for implementing a metadata authority audit system provided by an embodiment of the present invention;
[0040] Figure 2This is a data resource permission information maintenance code example provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0041] The present invention will be further described below in conjunction with specific embodiments.
[0042] The embodiment of the present invention provides a method for implementing a metadata permission audit system, which implements metadata permission audit based on the SpringBoot framework and the Elasticsearch distributed storage system, and allows users to manage and maintain metadata permission policy information through a common REST API, and audit and analyze access operation behaviors of permission policies;
[0043] Users manage permission policy information through the REST API;
[0044] The audit monitoring SDK monitors access to permission policy information and analyzes audit logs at regular intervals.
[0045] This method conveniently maintains permission policy information through the general REST API; timely monitors and warns the use of permission policy information through the audit monitoring SDK package; creates a safer and more convenient environment for users to use metadata. Solve the problem of metadata access control and permission auditing.
[0046] This method is specifically implemented as follows:
[0047] 1. Implemented based on SpringBoot framework and Elasticsearch distributed storage system. Publish the permission policy access, creation, update and deletion functions as REST API interfaces, allowing data resource managers and other application services to call them;
[0048] The audit monitoring SDK is used to monitor the calls of the data permission policy system REST API and generate audit monitoring logs based on the calling operation behavior;
[0049] Combine the data permission policy management system and audit monitoring SDK to implement metadata permission and audit supervision functions.
[0050] 2. For the creation and update of permission policies, the permission policy management of data sources, data tables, and fields is implemented through a unified and universal JSON object format.
[0051] 3. This method implements the management of permission policy information through the common JSON object format, which mainly includes the following information:
[0052] (1) Data resource information, specifically refers to the basic metadata information contained in the permission policy, and supports the maintenance and management of multi-level metadata information (database, data table, table field). To achieve the universality of data resource management, the universal permission policy data format provided by this method includes: data resource type, data resource id, data resource name, and parent resource id;
[0053] (2) Permission information, which specifically refers to the user's permission to use data resources. Users can customize their permissions (add, query, delete, modify, etc.);
[0054] (3) Subject information, including the subject's identity type. Users can customize settings and maintain (user, role, user group, etc.). It also supports maintaining the user subject's ID and name.
[0055] (4) Basic information of the permission policy, including creation time, validity period, etc.
[0056] The data permission policy of this method supports the general JSON data format and is not restricted by the data resource type and user subject. Any data permission policy information that follows this data format can be maintained in the data permission policy management system. The generality of the data permission policy mentioned in this method means that there is no programmatic constraint on data resources and authorization subjects. Data resource managers can authorize data resources at any level without having to consider the permissions of their parent resources or child resources.
[0057] For the permission information of data resources, the generality referred to in this method supports all permissions for operations such as adding, querying, updating, and deleting data tables, and all permissions for operations such as querying, modifying, and deleting table fields. Data resource managers can maintain customized permission policy information for data resources. Figure 2 shown.
[0058] In system applications, all operations on metadata will pass through the data permission system of the present invention. According to the metadata information to be accessed or operated, the permission policy information corresponding to the metadata is obtained, and the operation behavior will be compared with the authorized operation behavior in the permission policy. Based on the comparison result, it is decided whether to allow it to continue to operate on the metadata.
[0059] 4. This method provides an audit monitoring SDK package, which is used to monitor the permission policy management system in real time. All REST APIs of the permission policy management system are connected to the audit monitoring SDK package. All operations that access permission policy information through the REST API will be monitored by the audit monitoring SDK package. The monitoring indicators include: metadata information, access time, data authentication results, user subjects, operation behaviors and other information. The monitoring results of data permission policy information are maintained in the form of audit logs in the ES storage system.
[0060] The audit monitoring SDK provided by this method is based on the xxl-job open source component to expand the scheduled audit function. The manager obtains the audit monitoring log at a fixed period by setting a scheduled task, expands based on the audit log, performs statistical analysis based on metadata or authorized subjects, generates metadata access statistics reports, or generates alarm information based on metadata abnormal access logs.
[0061] like Figure 1 The figure shows the functional implementation flow chart of this method. Users will set permission policies for metadata according to business needs. The permission policies can be created, updated, deleted and queried through the REST API of the permission policy management module. When users operate metadata, they will first obtain the permission policy information of metadata through the REST API interface of the permission policy management, perform permission verification, and generate an audit log with the verification results and push it to the audit monitoring SDK to save it in the ES database. At the same time, the audit monitoring SDK will also monitor the REST API of the permission policy management to form an audit access log of the permission policy information. When a user attempts to operate and access metadata without permission, an audit alarm log will be generated. The scheduled monitoring task created by the audit monitoring SDK based on the xxl-job open source component will periodically analyze the audit alarm log and inform the relevant system administrators of the audit alarm results.
[0062] The embodiment of the present invention also provides a metadata permission audit system, which implements metadata permission audit based on the SpringBoot framework and the Elasticsearch distributed storage system, and allows users to manage and maintain metadata permission policy information through a common REST API, and audit and analyze access operation behaviors of permission policies;
[0063] Users manage permission policy information through the REST API;
[0064] The audit monitoring SDK monitors access to permission policy information and periodically analyzes audit logs;
[0065] By providing a unified REST API interface, the management and maintenance of permission policy information is realized. At the same time, an audit monitoring SDK package is provided to monitor the use, access, and editing of permission policies through audit logs, and to record abnormal access to permission policies to ensure secure access to data resource permissions. The system specifically implements metadata permission auditing through the metadata permission auditing system implementation method described in the above embodiment.
[0066] The general REST API allows users to manage and maintain metadata permission policy information, publishes permission policy access, creation, update and deletion functions as REST API interfaces, and allows data resource managers and other application services to call them;
[0067] The audit monitoring SDK is used to monitor the calls of the data permission policy system REST API and generate audit monitoring logs based on the calling operation behavior;
[0068] Combine the data permission policy management system and audit monitoring SDK to implement metadata permission and audit supervision functions.
[0069] For the creation and update of permission policies, the permission policy management of data sources, data tables, and fields is implemented through a unified and universal JSON object format.
[0070] The permission policy information is managed through the common JSON object format, which mainly includes the following information:
[0071] (1) Data resource information, specifically refers to the basic metadata information contained in the permission policy, and supports the maintenance and management of multi-level metadata information (database, data table, table field). To achieve the universality of data resource management, the universal permission policy data format provided by this method includes: data resource type, data resource id, data resource name, and parent resource id;
[0072] (2) Permission information, which specifically refers to the user's permission to use data resources. Users can customize their permissions (add, query, delete, modify, etc.);
[0073] (3) Subject information, including the subject's identity type. Users can customize settings and maintain (user, role, user group, etc.). It also supports maintaining the user subject's ID and name.
[0074] (4) Basic information of the permission policy, including creation time, validity period, etc.
[0075] The permission information of data resources refers to the general support for all permissions for operations such as adding, querying, updating, and deleting data tables, and all permissions for operations such as querying, modifying, and deleting table fields. Data resource managers can maintain customized permission policy information for data resources.
[0076] In system applications, all operations on metadata will pass through the data permission system of the present invention. According to the metadata information to be accessed or operated, the permission policy information corresponding to the metadata is obtained, and the operation behavior will be compared with the authorized operation behavior in the permission policy. Based on the comparison result, it is decided whether to allow it to continue to operate on the metadata.
[0077] The permission policy management system is monitored in real time through the audit monitoring SDK package. All REST APIs of the permission policy management system are connected to the audit monitoring SDK package. All operations that access permission policy information through the REST API will be monitored by the audit monitoring SDK package. The monitoring indicators include: metadata information, access time, data authentication results, user subjects, operation behaviors and other information. The monitoring results of data permission policy information are maintained in the form of audit logs in the ES storage system.
[0078] The audit monitoring SDK of this system is based on the xxl-job open source component to expand the scheduled audit function. The manager obtains the audit monitoring log by setting scheduled tasks at regular intervals, expands based on the audit log, performs statistical analysis based on metadata or authorized subjects, generates metadata access statistical reports, or generates alarm information based on metadata abnormal access logs.
[0079] Users will set permission policies for metadata according to business needs. The permission policies can be created, updated, deleted, and queried through the REST API of the permission policy management module. When users operate metadata, they will first obtain the permission policy information of the metadata through the REST API interface of the permission policy management, perform permission verification, and generate an audit log with the verification results and push it to the audit monitoring SDK to save it in the ES database. At the same time, the audit monitoring SDK will also monitor the REST API of the permission policy management to form an audit access log of the permission policy information. When a user attempts to operate and access metadata without permission, an audit alarm log will be generated. The scheduled monitoring task created by the audit monitoring SDK based on the xxl-job open source component will periodically analyze the audit alarm log and inform the relevant system administrators of the audit alarm results.
[0080] The embodiment of the present invention also provides a metadata authority audit system implementation device, including: at least one memory and at least one processor;
[0081] The at least one memory is used to store a machine-readable program;
[0082] The at least one processor is used to call the machine-readable program to implement the metadata authority audit system implementation method described in the above embodiment.
[0083] The embodiment of the present invention also provides a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the method for implementing the metadata authority audit system described in the above embodiment is implemented. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code for implementing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.
[0084] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
[0085] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.
[0086] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.
[0087] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.
[0088] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.
Claims
1. A method for implementing a metadata authority audit system, characterized in that: Implement metadata permission audit based on SpringBoot framework and Elasticsearch distributed storage system. Through the common REST API, users can manage and maintain metadata permission policy information and audit and analyze access operation behaviors of permission policies. Users manage permission policy information through the REST API; The audit monitoring SDK monitors access to permission policy information and analyzes audit logs at regular intervals.
2. A method for implementing a metadata authority audit system according to claim 1, characterized in that: The general REST API allows users to manage and maintain metadata permission policy information, publishes permission policy access, creation, update and deletion functions as REST API interfaces, and allows data resource managers and other application services to call them; The audit monitoring SDK is used to monitor the calls of the data permission policy system REST API and generate audit monitoring logs based on the calling operation behavior; Combine the data permission policy management system and audit monitoring SDK to implement metadata permission and audit supervision functions.
3. A method for implementing a metadata authority audit system according to claim 1, characterized in that: For the creation and update of permission policies, the permission policy management of data sources, data tables, and fields is implemented through a unified and universal JSON object format.
4. A method for implementing a metadata authority audit system according to claim 1 or 3, characterized in that: The permission policy information is managed through the common JSON object format, including the following information: 1) Data resource information, including basic metadata information included in the permission policy, supports maintenance and management of multi-level metadata information, including databases, data tables, and table fields; 2) Permission information, including the user's permission to use data resources. Users can customize their permissions; 3) Subject information, including the subject's identity type, which can be customized and maintained by the user, and also supports the maintenance of the user subject's ID and name; 4) Basic information of the permission policy, including creation time and validity period.
5. A method for implementing a metadata authority audit system according to claim 4, characterized in that: The management of the data resource information, the general permission policy data format includes: data resource type, data resource id, data resource name, parent resource id; The management of the permission information supports all permissions for adding, querying, updating, and deleting operations on data tables, and all permissions for querying, modifying, and deleting operations on table fields. Data resource managers can maintain customized permission policy information for data resources. All operations on metadata will go through the data permission system. Based on the metadata information to be accessed or operated, the permission policy information corresponding to the metadata will be obtained, and the operation behavior will be compared with the authorized operation behavior in the permission policy. Based on the comparison results, it will decide whether to allow it to continue to operate on the metadata.
6. A method for implementing a metadata authority audit system according to claim 1, characterized in that: The audit monitoring SDK monitors access to permission policy information and periodically analyzes audit logs. The permission policy management system is monitored in real time through the audit monitoring SDK package. All REST APIs of the permission policy management system are connected to the audit monitoring SDK package. All operations that access permission policy information through the REST API will be monitored by the audit monitoring SDK package. The monitoring indicators include: metadata information, access time, data authentication results, user subjects, and operation behavior information. The monitoring results of data permission policy information are maintained in the form of audit logs in the ES storage system.
7. A method for implementing a metadata authority audit system according to claim 1 or 6, characterized in that: The audit monitoring SDK extends the scheduled audit function based on the xxl-job open source component. The manager obtains the audit monitoring log at a fixed period by setting a scheduled task, expands based on the audit log, performs statistical analysis based on metadata or authorized subjects, generates metadata access statistical reports, or generates alarm information based on metadata abnormal access logs.
8. A metadata authority audit system, characterized in that: The system implements metadata permission auditing based on the SpringBoot framework and the Elasticsearch distributed storage system. Through the common REST API, users can manage and maintain metadata permission policy information and audit and analyze access operation behaviors of permission policies. Users manage permission policy information through the REST API; The audit monitoring SDK monitors access to permission policy information and periodically analyzes audit logs; The system specifically implements metadata authority auditing through any of the methods described in claims 1 to 7.
9. A metadata authority audit system implementation device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.
10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.