Distributed storage and data warehouse permission synchronization method and system

Abstract

The present invention provides a permission synchronization method and system for distributed storage and data warehouse, which relates to the technical field of permission management, including analyzing user operation data of distributed storage, training machine learning models, and generating directory permission intelligent mapping tables; analyzing the structural information of data tables in data warehouses, and generating multi-dimensional table permission evaluation models; combining the two to generate an adaptive permission rule base, and deploying it to each node of distributed storage; when permissions change, the system calculates the impact range according to the adaptive permission rule base, and generates a permission update instruction sequence; the system monitors the execution status of instructions, identifies abnormal nodes, and repairs based on permission propagation trees to ensure permission consistency. The present invention can realize automatic synchronization of distributed storage and data warehouse permissions, improve permission management efficiency, reduce manual intervention costs, and enhance data security.

Description

Technical Field

[0001] The present invention relates to the technical field of authority management, and in particular to a method and system for synchronizing authorities of distributed storage and data warehouse. Background Art

[0002] As the scale of data continues to expand, enterprise applications are increasingly using a hybrid architecture of distributed storage systems and data warehouses to store and manage data. In this architecture, data is stored in different physical nodes, and users need to access data across multiple storage nodes. In order to ensure data security, it is necessary to establish a unified permission management mechanism between the distributed storage system and the data warehouse to achieve dynamic adjustment and synchronization of permissions. This requires the permission management system to accurately identify user access patterns, evaluate the security level of data, and maintain permission consistency between storage nodes when permissions are changed.

[0003] However, there are still some problems with the existing technology. The traditional static permission configuration method cannot adapt to the dynamically changing data access needs of users, resulting in the problem of too loose or too strict permission allocation; in a distributed environment, the permission updates of each storage node have a time dependency, but the existing technology lacks a systematic analysis of the permission change propagation path, which easily leads to inconsistent permission updates; when anomalies occur in permission updates, the existing technology lacks effective anomaly detection and automatic repair mechanisms, and requires manual intervention, affecting system availability; the existing permission synchronization scheme does not fully consider the impact of data value and correlation on permission management, making it difficult to achieve precise permission control.

[0004] In summary, it is necessary to solve the technical problem of permission synchronization between distributed storage systems and data warehouses. Specifically, by analyzing the user access behavior characteristics through machine learning methods, a multi-dimensional permission evaluation model for data tables is established to achieve adaptive adjustment of permission rules; by building a permission change dependency graph, the sequential execution of permission updates between nodes is ensured; through permission propagation analysis and automatic repair mechanisms, permission anomalies are discovered and handled in a timely manner to improve system reliability. The present invention can solve the problems in the prior art. Summary of the invention

[0005] The embodiment of the present invention provides a method and system for synchronizing permissions between distributed storage and data warehouse, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] A method for synchronizing permissions between a distributed storage and a data warehouse is provided, comprising:

[0008] Obtain user operation data in a distributed storage system, generate a behavior data set, perform feature extraction on the behavior data set, and calculate a user access pattern feature matrix; input the user access pattern feature matrix into a pre-trained machine learning model, calculate the directory access frequency weight, time sensitivity coefficient, and permission propagation factor, and generate a directory permission intelligent mapping table; read the structural information of the data table in the data warehouse, calculate the value score, field association strength, and cross-table access impact of the data table, and generate a multi-dimensional table permission evaluation model; perform weighted calculation on the directory permission intelligent mapping table and the parameters in the multi-dimensional table permission evaluation model to generate an adaptive permission rule library;

[0009] Deploy permission synchronization agents on physical nodes of distributed storage, and distribute the adaptive permission rule library to each permission synchronization agent according to the consistent hashing algorithm; monitor permission change events, extract permission change instructions, match the permission change instructions with the rules in the adaptive permission rule library, generate rule matching results, calculate the impact range of the permission change based on the rule matching results, and form an initial permission change set; perform dependency analysis on the initial permission change set to generate a permission change dependency graph; sort permission change operations based on the topological structure of the permission change dependency graph to generate a permission update instruction sequence; write the permission update instruction sequence into a message queue to generate a permission change snapshot containing an execution timestamp;

[0010] Read the permission change snapshot and extract the execution status information of the permission update instruction sequence; compare the execution status information with the permission change dependency graph to identify abnormal nodes of the permission update; generate a permission propagation tree based on the position of the abnormal node in the permission change dependency graph; calculate the permission consistency score of each propagation path in the permission propagation tree; when the permission consistency score is lower than a preset threshold, extract the rule matching result corresponding to the abnormal node; generate a permission repair plan according to the rule matching result; convert the permission repair plan into a repair instruction sequence; execute the repair instruction sequence to update the permission status; write the repaired permission status and repair process data into the adaptive permission rule library.

[0011] In an optional embodiment,

[0012] Obtain user operation data in a distributed storage system, generate a behavior data set, perform feature extraction on the behavior data set, and calculate a user access pattern feature matrix; input the user access pattern feature matrix into a pre-trained machine learning model, calculate the directory access frequency weight, time sensitivity coefficient, and permission propagation factor, and generate a directory permission intelligent mapping table; read the structural information of the data table in the data warehouse, calculate the value score, field association strength, and cross-table access impact of the data table, and generate a multi-dimensional table permission evaluation model; perform weighted calculation on the directory permission intelligent mapping table and the parameters in the multi-dimensional table permission evaluation model to generate an adaptive permission rule base, including:

[0013] Obtain user access time, operation type, and access path data in the audit log of the distributed storage system, and use the sliding time window method to preprocess the user access time, operation type, and access path data to generate a behavior data set; based on the behavior data set, extract time series features, operation features, and path features, and vectorize the time series features, operation features, and path features to generate a user access pattern feature matrix;

[0014] Input the user access pattern feature matrix into the pre-trained machine learning model, obtain the directory access frequency weight by multiplying and adding the time decay factor and the number of accesses, calculate the time sensitivity coefficient by the exponential decay function, and calculate the permission propagation factor by the intersection and union ratio of the directory permission set; construct a directory access relationship graph, set the directory as the graph node, set the user access path as the graph edge, extract the progressive propagation feature between directories through the graph convolution network based on the directory access relationship graph, and modify the permission propagation factor based on the progressive propagation feature; generate a directory permission intelligent mapping table based on the modified permission propagation factor, the directory access frequency weight and the time sensitivity coefficient;

[0015] Read the structural information of the data table in the data warehouse, calculate the sensitivity score, citation frequency and data volume score based on the structural information, and perform weighted calculation on the sensitivity score, citation frequency and data volume score to obtain the data table value score; calculate the field association coefficient and field importance score based on the structural information, and perform weighted calculation on the field association coefficient and field importance score to obtain the field association strength; calculate the access frequency and dependency between tables based on the structural information, and perform the product accumulation of the access frequency and dependency between tables to obtain the cross-table access impact, and generate a multi-dimensional table permission evaluation model based on the data table value score, field association strength and cross-table access impact;

[0016] The directory permission intelligent mapping table is weightedly calculated with the data table value score, field association strength and cross-table access impact in the multi-dimensional table permission evaluation model to generate permission rules, and the permission rules are stored in an adaptive permission rule library.

[0017] In an optional embodiment,

[0018] Construct a directory access relationship graph, set the directory as a graph node, set the user access path as a graph edge, extract the progressive propagation features between directories through a graph convolutional network based on the directory access relationship graph, and modify the permission propagation factor based on the progressive propagation features, including:

[0019] Building an initial directory graph based on the access path data in the behavior data set, and setting the direct access frequency between directory nodes as the initial weight of the edge;

[0020] Calculate the shortest path distance between any two directory nodes in the initial directory graph, construct a spatial adjacency matrix based on the shortest path distance, and multiply the spatial adjacency matrix by the initial weight of the edge to obtain the spatial association strength of the directory node;

[0021] Extracting access timing information from the behavior data set, calculating the access time interval between directory node pairs, normalizing the access time interval to obtain a time decay coefficient, and multiplying the time decay coefficient by the spatial correlation strength to obtain a spatiotemporal weighted adjacency matrix;

[0022] Performing sparse processing on the spatiotemporal weighted adjacency matrix, retaining edges with weight values ​​greater than a preset weight threshold, and obtaining an optimized directory access relationship graph;

[0023] A multi-layer graph convolutional network is used to extract features from the optimized directory access relationship graph, wherein the first graph convolutional layer extracts local access pattern features, the second graph convolutional layer fuses multi-hop access path features, and the third graph convolutional layer generates a progressive propagation feature vector of the directory node;

[0024] A similarity matrix between directory nodes is calculated based on the progressive propagation feature vector, and a dot product operation is performed on the similarity matrix and the original permission propagation factor to obtain a modified permission propagation factor.

[0025] In an optional embodiment,

[0026] Monitor permission change events, extract permission change instructions, match permission change instructions with rules in the adaptive permission rule library, generate rule matching results, calculate the impact range of permission changes based on the rule matching results, and form an initial permission change set; perform dependency analysis on the initial permission change set to generate a permission change dependency graph, including:

[0027] The permission synchronization agent monitors permission change events, extracts the operation type, target object and permission attribute from the permission change events, and generates permission change instructions; constructs a dictionary tree, inserts the rule pattern string into the dictionary tree, and constructs an invalidation pointer for the node in the dictionary tree, the invalidation pointer points to the node corresponding to the longest suffix of the current node; matches are performed in the dictionary tree along the permission change instruction, and when the match fails, the invalidation pointer is used to fall back to the node corresponding to the longest suffix to continue matching until the match is successful or falls back to the root node, and a directly affected object set is generated based on the objects corresponding to the successfully matched rules; based on the directly affected object set, the associated rule chain is recursively queried, and each object in the directly affected object set is combined with the corresponding rule chain to form an initial permission change set;

[0028] Perform dependency analysis on each object in the initial permission change set and the corresponding rule chain, start traversal from the starting rule in the rule chain, add the visited rules to the visited set, and recursively visit the associated rules of the current rule; if the associated rule is already in the visited set, record the direct dependency relationship between the current rule and the associated rule; when all the associated rules of the rule have been visited, remove the current rule from the visited set and return to the upper-level rule; iteratively calculate the reachability between the rules through matrix multiplication to obtain the indirect dependency relationship; set the permission objects in the initial permission change set as nodes of the dependency graph, and set the direct dependency and the indirect dependency as edges of the dependency graph; count the number of times the rule is triggered in the historical records, and divide it by the total number of executions to obtain the trigger probability; count the number of permission objects affected by the rule, and divide it by the total number of permission objects in the system to obtain the degree of influence; set the product of the trigger probability and the degree of influence as the dependency strength, set the dependency strength as the edge weight of the dependency graph, and generate a permission change dependency graph.

[0029] In an optional embodiment,

[0030] Based on the topological structure of the permission change dependency graph, the permission change operations are sorted to generate a permission update instruction sequence; the permission update instruction sequence is written into a message queue to generate a permission change snapshot containing an execution timestamp, including:

[0031] Calculate the in-degree of each node in the permission change dependency graph, and add nodes with zero in-degree to the candidate permission node set;

[0032] Calculate the sum of the out-edge weights of each candidate authority node in the candidate authority node set, select the target authority node with the largest sum of out-edge weights, add the change operation corresponding to the target authority node to the authority update instruction sequence, remove the out-edge of the target authority node, update the in-degree value of the successor authority node adjacent to the target authority node, and remove the target authority node from the candidate authority node set; when the in-degree value of the successor authority node becomes zero, add the successor authority node to the candidate authority node set; repeat until the candidate authority node set is empty;

[0033] An incremental counter is used to assign a timestamp to each instruction in the permission update instruction sequence; a permission update topic is created in a distributed message queue, a preparation request containing update content is sent to all participating nodes, and a preparation completion response is waited for all participating nodes to return; after receiving a successful response from all participating nodes, a submission request is sent to all participating nodes, and the permission update instruction sequence is written into the permission update topic; the permission update instruction sequence, the timestamp, and the node relationship and edge weight information of the permission change dependency graph are converted into a binary format to generate a permission change snapshot.

[0034] In an optional embodiment,

[0035] Reading the permission change snapshot, extracting the execution status information of the permission update instruction sequence; comparing the execution status information with the permission change dependency graph, identifying abnormal nodes of the permission update; generating a permission propagation tree based on the position of the abnormal node in the permission change dependency graph includes:

[0036] The permission change snapshot is constructed as a permission state matrix, which is composed of matrix elements of permission node rows, time series columns and execution state information; the permission state matrix is ​​subjected to difference operation with the permission change dependency graph to obtain a state deviation matrix; when the deviation value in the state deviation matrix exceeds a preset state deviation threshold, the corresponding permission node is marked as an abnormal node;

[0037] Based on the abnormal node, the forward propagation layer and the reverse propagation layer of the permission change dependency graph are constructed. The forward propagation layer traverses downward layer by layer through the permission inheritance relationship, and the reverse propagation layer traverses upward layer by layer through the permission dependency relationship. In each layer of the forward propagation layer and the reverse propagation layer, the propagation impact factor is calculated and determined according to the state deviation value of the abnormal node, the topological distance between nodes, and the cumulative attenuation coefficient of the path. When the propagation impact factor is lower than a preset cutoff threshold, the propagation boundary of the corresponding layer is determined, and the traversal in the corresponding direction is stopped. The traversal results of the forward propagation layer and the reverse propagation layer are merged to generate a permission propagation tree.

[0038] In an optional embodiment,

[0039] Calculating the permission consistency score of each propagation path in the permission propagation tree; when the permission consistency score is lower than a preset threshold, extracting the rule matching result corresponding to the abnormal node; generating a permission repair plan according to the rule matching result; converting the permission repair plan into a repair instruction sequence; executing the repair instruction sequence to update the permission status; writing the repaired permission status and repair process data into the adaptive permission rule library includes:

[0040] For each propagation path in the permission propagation tree, the permission consistency score is calculated by the propagation impact factor on the propagation path and the permission value difference between adjacent permission nodes; when the permission consistency score is lower than the preset score threshold, the rule matching result corresponding to the abnormal node is extracted;

[0041] Obtaining the rule weight value and rule influence direction of each rule from the rule matching result, calculating the correction direction of the current authority value and the target authority value of the abnormal node, sorting in descending order according to the rule weight values, selecting the rule with the largest rule weight value and the same rule influence direction as the correction direction as the repair rule; applying the repair rule to the abnormal node, determining the rule repair parameter based on the state deviation value of the abnormal node; generating a repair instruction sequence according to the repair rule and the repair parameter;

[0042] Execute the repair instruction sequence within a preset time window, and record the start execution time, completion execution time, execution status code, and state change amount after execution of each repair instruction; when it is detected that the execution status code is abnormal or the state change amount does not meet the preset change amount threshold, trigger the corresponding repair instruction to retry execution;

[0043] Calculate the degree of improvement of the state deviation before and after executing the repair instruction sequence, the state fluctuation during the execution process and the execution success rate of the repair instruction; write the degree of improvement of the state deviation, the state fluctuation, the execution success rate and the repaired permission state into the adaptive permission rule library.

[0044] According to a second aspect of the embodiments of the present invention,

[0045] A distributed storage and data warehouse permission synchronization system is provided, comprising:

[0046] The first unit is used to obtain user operation data in the distributed storage system, generate a behavior data set, perform feature extraction on the behavior data set, and calculate a user access pattern feature matrix; input the user access pattern feature matrix into a pre-trained machine learning model, calculate the directory access frequency weight, time sensitivity coefficient and permission propagation factor, and generate a directory permission intelligent mapping table; read the structural information of the data table in the data warehouse, calculate the value score, field association strength and cross-table access influence of the data table, and generate a multi-dimensional table permission evaluation model; perform weighted calculation on the directory permission intelligent mapping table and the parameters in the multi-dimensional table permission evaluation model to generate an adaptive permission rule base;

[0047] The second unit is used to deploy permission synchronization agents on physical nodes of distributed storage, distribute the adaptive permission rule library to each permission synchronization agent according to the consistent hashing algorithm; monitor permission change events, extract permission change instructions, match the permission change instructions with the rules in the adaptive permission rule library, generate rule matching results, calculate the impact range of the permission change based on the rule matching results, and form an initial permission change set; perform dependency analysis on the initial permission change set to generate a permission change dependency graph; sort the permission change operations based on the topological structure of the permission change dependency graph to generate a permission update instruction sequence; write the permission update instruction sequence into a message queue to generate a permission change snapshot containing an execution timestamp;

[0048] The third unit is used to read the permission change snapshot and extract the execution status information of the permission update instruction sequence; compare the execution status information with the permission change dependency graph to identify the abnormal nodes of the permission update; generate a permission propagation tree based on the position of the abnormal node in the permission change dependency graph; calculate the permission consistency score of each propagation path in the permission propagation tree; when the permission consistency score is lower than a preset threshold, extract the rule matching result corresponding to the abnormal node; generate a permission repair plan according to the rule matching result; convert the permission repair plan into a repair instruction sequence; execute the repair instruction sequence to update the permission status; write the repaired permission status and repair process data into the adaptive permission rule library.

[0049] According to a third aspect of the embodiments of the present invention,

[0050] An electronic device is provided, comprising:

[0051] processor;

[0052] a memory for storing processor-executable instructions;

[0053] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0054] A fourth aspect of the embodiments of the present invention is:

[0055] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0056] In an embodiment of the present invention, a machine learning model is used to learn user access patterns, intelligently generate permission mapping tables and rule bases, and sort permission update operations based on a dependency graph, thereby reducing unnecessary permission change operations and improving permission synchronization efficiency; a multi-dimensional table permission evaluation model and a consistent hashing algorithm are used to achieve fine-grained permission control, and permission change dependency analysis and abnormal node identification are used to promptly repair permission errors, effectively preventing permission leakage and abuse; a permission rule base is automatically generated, permission anomalies are automatically identified and repaired, and permission status is automatically updated without human intervention, thereby reducing the complexity and cost of permission management and realizing automated permission management. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of a flow chart of a method for synchronizing permissions between distributed storage and a data warehouse according to an embodiment of the present invention;

[0058] Figure 2 It is a structural diagram of a distributed storage and data warehouse permission synchronization system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0061] Figure 1 FIG. 1 is a flow chart of a method for synchronizing permissions between distributed storage and a data warehouse according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0062] S101. Obtain user operation data in a distributed storage system, generate a behavior data set, perform feature extraction on the behavior data set, and calculate a user access pattern feature matrix; input the user access pattern feature matrix into a pre-trained machine learning model, calculate the directory access frequency weight, time sensitivity coefficient, and permission propagation factor, and generate a directory permission intelligent mapping table; read the structural information of the data table in the data warehouse, calculate the value score, field association strength, and cross-table access impact of the data table, and generate a multi-dimensional table permission evaluation model; perform weighted calculation on the directory permission intelligent mapping table and the parameters in the multi-dimensional table permission evaluation model to generate an adaptive permission rule base;

[0063] In this embodiment, by acquiring user operation data and generating a behavior data set, feature extraction is performed, which can accurately describe the user's access pattern and provide a data-driven basis for subsequent permission management; by calculating the directory access frequency weight, time sensitivity coefficient and permission propagation factor, a directory permission intelligent mapping table is generated, which can dynamically and intelligently adjust the directory permission setting, thereby improving the flexibility and adaptability of permission management; by calculating the value score, field association strength and cross-table access influence of the data table, a multi-dimensional permission evaluation model is generated, which can comprehensively analyze the permission dependency relationship between data tables to ensure that the permission setting is more reasonable and accurate; by combining the directory permission intelligent mapping table and the multi-dimensional table permission evaluation model, an adaptive permission rule base is generated through weighted calculation, which can dynamically adjust the permission rules according to system changes, thereby improving the automation and intelligence level of permission management.

[0064] S102. Deploy permission synchronization agents on physical nodes of distributed storage, distribute the adaptive permission rule base to each permission synchronization agent according to the consistent hashing algorithm; monitor permission change events, extract permission change instructions, match the permission change instructions with the rules in the adaptive permission rule base, generate rule matching results, calculate the impact range of the permission change based on the rule matching results, and form an initial permission change set; perform dependency analysis on the initial permission change set to generate a permission change dependency graph; sort permission change operations based on the topological structure of the permission change dependency graph to generate a permission update instruction sequence; write the permission update instruction sequence into a message queue to generate a permission change snapshot containing an execution timestamp;

[0065] In this embodiment, by deploying a permission synchronization agent on the physical nodes of the distributed storage, and combining the consistent hashing algorithm to dynamically allocate an adaptive permission rule base, it is ensured that the permission rules can be efficiently and evenly distributed to each node, so as to achieve the synchronization and consistency of distributed permissions; the permission synchronization agent can monitor permission change events in real time, automatically extract and match permission change instructions, calculate the impact range of permission changes based on the rule matching results, and quickly form a preliminary permission change set, thereby achieving precise control and response to permission changes; through dependency analysis and generating permission change dependency graphs, the system can understand the mutual relationship and impact path of permission changes, ensure the rationality of permission changes, and avoid permission conflicts and inconsistencies; based on the topological structure of the permission change dependency graph, the permission change operations are sorted, and a permission update instruction sequence is generated, and permission updates are executed in an orderly and efficient manner, reducing conflicts and delays in permission changes; by generating permission change snapshots containing execution timestamps, a complete permission change history record is provided, which is helpful for permission auditing and problem troubleshooting, and ensures the transparency and traceability of permission changes.

[0066] S103. Read the permission change snapshot and extract the execution status information of the permission update instruction sequence; compare the execution status information with the permission change dependency graph to identify abnormal nodes of the permission update; generate a permission propagation tree based on the position of the abnormal node in the permission change dependency graph; calculate the permission consistency score of each propagation path in the permission propagation tree; when the permission consistency score is lower than a preset threshold, extract the rule matching result corresponding to the abnormal node; generate a permission repair plan based on the rule matching result; convert the permission repair plan into a repair instruction sequence; execute the repair instruction sequence to update the permission status; write the repaired permission status and repair process data into the adaptive permission rule library.

[0067] In this embodiment, by reading the permission change snapshot and comparing the execution status information, the system can quickly identify abnormal nodes in the permission update, ensure the accuracy of the permission change process, and prevent permission setting errors or conflicts; generate a permission propagation tree based on the abnormal nodes, and calculate the permission consistency score of each propagation path, which helps to comprehensively evaluate the impact scope and consistency of the permission change, thereby avoiding potential permission propagation problems; when the permission consistency score is lower than the threshold, the system can automatically extract the rule matching results and generate a permission repair plan to ensure the correctness and consistency of the permission configuration without manual intervention, thereby improving management efficiency; the repair plan is converted into a repair instruction sequence and executed, automatically updating the permission status, ensuring dynamic adjustment and optimization of system permissions, and ensuring that the permission configuration continues to comply with security policies and actual needs; the repaired permission status and repair process data will be written into the adaptive permission rule library, providing a complete permission change history record, enhancing the transparency, auditability and traceability of permission management, and facilitating compliance inspection and problem tracking.

[0068] In an optional implementation, user operation data in a distributed storage system is obtained to generate a behavior data set, feature extraction is performed on the behavior data set, and a user access pattern feature matrix is ​​calculated; the user access pattern feature matrix is ​​input into a pre-trained machine learning model, directory access frequency weight, time sensitivity coefficient and permission propagation factor are calculated, and a directory permission intelligent mapping table is generated; structural information of a data table in a data warehouse is read, a value score, field association strength and cross-table access influence of the data table are calculated, and a multi-dimensional table permission evaluation model is generated; weighted calculation is performed on the directory permission intelligent mapping table and the parameters in the multi-dimensional table permission evaluation model to generate an adaptive permission rule base, including:

[0069] Obtain user access time, operation type, and access path data in the audit log of the distributed storage system, and use the sliding time window method to preprocess the user access time, operation type, and access path data to generate a behavior data set; based on the behavior data set, extract time series features, operation features, and path features, and vectorize the time series features, operation features, and path features to generate a user access pattern feature matrix;

[0070] Input the user access pattern feature matrix into the pre-trained machine learning model, obtain the directory access frequency weight by multiplying and adding the time decay factor and the number of accesses, calculate the time sensitivity coefficient by the exponential decay function, and calculate the permission propagation factor by the intersection and union ratio of the directory permission set; construct a directory access relationship graph, set the directory as the graph node, set the user access path as the graph edge, extract the progressive propagation feature between directories through the graph convolution network based on the directory access relationship graph, and modify the permission propagation factor based on the progressive propagation feature; generate a directory permission intelligent mapping table based on the modified permission propagation factor, the directory access frequency weight and the time sensitivity coefficient;

[0071] Read the structural information of the data table in the data warehouse, calculate the sensitivity score, citation frequency and data volume score based on the structural information, and perform weighted calculation on the sensitivity score, citation frequency and data volume score to obtain the data table value score; calculate the field association coefficient and field importance score based on the structural information, and perform weighted calculation on the field association coefficient and field importance score to obtain the field association strength; calculate the access frequency and dependency between tables based on the structural information, and perform the product accumulation of the access frequency and dependency between tables to obtain the cross-table access impact, and generate a multi-dimensional table permission evaluation model based on the data table value score, field association strength and cross-table access impact;

[0072] The directory permission intelligent mapping table is weightedly calculated with the data table value score, field association strength and cross-table access impact in the multi-dimensional table permission evaluation model to generate permission rules, and the permission rules are stored in an adaptive permission rule library.

[0073] In a specific implementation, first, data such as user access time, operation type, and access path are collected from the audit log of the distributed storage system. For example, user A accessed the / data / sales / 202310 directory at 8:00 on October 27, 2023, and performed a read operation; user B accessed the / data / marketing / 202310 directory at 8:05 on October 27, 2023, and performed a write operation. All this information will be recorded and collected.

[0074] Next, the sliding time window method is used to preprocess the collected data to generate a behavior data set. For example, if the time window is set to one day, the user operation data within one day will be aggregated together. The read operation of user A accessing the / data / sales / 202310 directory on October 27, 2023, and the write operation of user B accessing the / data / marketing / 202310 directory on the same day will be included in the behavior data set on October 27, 2023.

[0075] Based on the generated behavioral data set, time series features, operation features, and path features are extracted. Time series features include access time period (morning, afternoon, evening), access frequency (number of visits per hour), etc.; operation features include read, write, delete, etc.; path features include the hierarchical depth of the access directory, the name of the access directory, etc. These features are then vectorized to form a user access pattern feature matrix. For example, the feature vector of user A can be expressed as [morning, 2 times per hour, read, 3 layers, sales], and the feature vector of user B can be expressed as [morning, 1 time per hour, write, 3 layers, marketing].

[0076] The user access pattern feature matrix is ​​input into the pre-trained machine learning model to calculate the directory access frequency weight, time sensitivity coefficient, and permission propagation factor. The directory access frequency weight is calculated by adding the time decay factor to the number of accesses. For example, the access weight in the last 7 days is higher than the access weight 7 days ago. The time sensitivity coefficient is calculated using an exponential decay function. For example, the closer the access time is to the present, the higher the weight. The permission propagation factor is calculated by the intersection and union of the directory permission set. For example, if a user has read permission for a parent directory, then he or she may also have read permission for a child directory.

[0077] In order to more accurately calculate the permission propagation factor, a directory access relationship graph is constructed. The directories are set as graph nodes and the user access paths are set as graph edges. For example, if a user accesses / data / sales / 202310, the three directories / data, / data / sales, and / data / sales / 202310 form a path, which are connected by edges in the graph. The graph convolutional network is used to extract the progressive propagation features between directories, and the permission propagation factor is corrected based on these features. For example, if the user frequently accesses the subdirectories under the / data / sales directory, the permission propagation factor of the / data / sales directory will be higher.

[0078] Based on the modified permission propagation factor, directory access frequency weight and time sensitivity coefficient, a directory permission intelligent mapping table is generated. The table records the access frequency weight, time sensitivity coefficient and permission propagation factor of each directory.

[0079] Read the structural information of the data table in the data warehouse, and calculate the value score, field association strength, and cross-table access impact of the data table. The data table value score is calculated by weighting the sensitivity score, reference frequency, and data volume score. The field association strength is calculated by weighting the field association coefficient and the field importance score. The cross-table access impact is calculated by accumulating the access frequency and dependency between tables.

[0080] Generate a multi-dimensional table permission evaluation model based on data table value scores, field association strength, and cross-table access impact.

[0081] Finally, the directory permission intelligent mapping table is weightedly calculated with the data table value score, field association strength and cross-table access impact in the multi-dimensional table permission evaluation model to generate permission rules, which are then stored in the adaptive permission rule library.

[0082] In this embodiment, by analyzing user access behavior, the user's data access rights can be controlled more accurately to prevent sensitive data leakage; the adaptive permission rule library can automatically adjust the permission rules according to user behavior, reduce manual intervention, and reduce management costs; users can obtain corresponding access rights according to actual needs, avoid unnecessary permission application processes, and improve user experience.

[0083] In an optional implementation, a directory access relationship graph is constructed, directories are set as graph nodes, user access paths are set as graph edges, and based on the directory access relationship graph, progressive propagation features between directories are extracted through a graph convolutional network. Based on the progressive propagation features, the permission propagation factor is modified, including:

[0084] Building an initial directory graph based on the access path data in the behavior data set, and setting the direct access frequency between directory nodes as the initial weight of the edge;

[0085] Calculate the shortest path distance between any two directory nodes in the initial directory graph, construct a spatial adjacency matrix based on the shortest path distance, and multiply the spatial adjacency matrix by the initial weight of the edge to obtain the spatial association strength of the directory node;

[0086] Extracting access timing information from the behavior data set, calculating the access time interval between directory node pairs, normalizing the access time interval to obtain a time decay coefficient, and multiplying the time decay coefficient by the spatial correlation strength to obtain a spatiotemporal weighted adjacency matrix;

[0087] Performing sparse processing on the spatiotemporal weighted adjacency matrix, retaining edges with weight values ​​greater than a preset weight threshold, and obtaining an optimized directory access relationship graph;

[0088] A multi-layer graph convolutional network is used to extract features from the optimized directory access relationship graph, wherein the first graph convolutional layer extracts local access pattern features, the second graph convolutional layer fuses multi-hop access path features, and the third graph convolutional layer generates a progressive propagation feature vector of the directory node;

[0089] A similarity matrix between directory nodes is calculated based on the progressive propagation feature vector, and a dot product operation is performed on the similarity matrix and the original permission propagation factor to obtain a modified permission propagation factor.

[0090] In a specific implementation, first, a user behavior data set in the system is collected, which contains the path and time information of the user's access to different directories. For example, user A accessed the directory / home / user / documents at 10:00 on January 1, 2024, and then accessed the directory / home / user / pictures at 10:05.

[0091] Next, we build an initial directory graph based on the collected access path data. Each directory in the system is set as a node in the graph. If the user directly accesses directory B from directory A, a directed edge is established between directory A and directory B. The initial weight of the edge is set to the direct access frequency from directory A to directory B. For example, if the user accesses directory / home / user / documents 10 times from directory / home / user, the initial weight of the edge is 10.

[0092] Then, calculate the shortest path distance between any two directory nodes in the initial directory graph. The shortest path distance can be calculated using a breadth-first search algorithm. Based on the calculated shortest path distance, construct a spatial adjacency matrix. The element value of the spatial adjacency matrix is ​​the reciprocal of the shortest path distance between two directory nodes. Multiply the spatial adjacency matrix by the initial weight of the edge to obtain the spatial association strength of the directory nodes. For example, if the initial weights of directory A and directory B are 10 and the shortest path distance is 2, the spatial association strength is 10×(1 / 2)=5.

[0093] Next, extract the access timing information in the behavior dataset and calculate the access time interval between directory node pairs. For example, a user first visits directory A and then visits directory B, and the time interval between the two visits is 5 minutes. After normalizing the access time interval, the time decay coefficient is obtained. For example, the time interval can be normalized using an exponential decay function. Multiply the time decay coefficient by the spatial association strength to obtain the spatiotemporal weighted adjacency matrix. For example, if the time decay coefficient is 0.8 and the spatial association strength is 5, the value of the corresponding element in the spatiotemporal weighted adjacency matrix is ​​0.8×5=4.

[0094] The spatiotemporal weighted adjacency matrix is ​​sparsely processed, and edges with weight values ​​greater than a preset weight threshold are retained to obtain an optimized directory access relationship graph. For example, if the weight threshold is set to 3, only edges with weight values ​​greater than 3 are retained.

[0095] A three-layer graph convolutional network is used to extract features from the optimized directory access relationship graph. The first graph convolutional layer extracts local access pattern features, for example, which directories are visited next after a certain directory is frequently visited. The second graph convolutional layer fuses multi-hop access path features, for example, considering the access pattern of users passing through multiple intermediate directories to finally reach the target directory. The third graph convolutional layer generates a progressive propagation feature vector of the directory node, which represents the propagation characteristics of the directory in the access path.

[0096] The similarity matrix between directory nodes is calculated based on the generated progressive propagation feature vector. For example, the cosine similarity can be used to calculate the similarity. The similarity matrix is ​​dot-producted with the original permission propagation factor to obtain the modified permission propagation factor. For example, if the original permission propagation factor is 0.5 and the similarity is 0.8, the modified permission propagation factor is 0.5×0.8=0.4.

[0097] In this embodiment, the security of permission management is improved: by considering the access relationship and propagation characteristics between directories, the propagation of permissions can be controlled more accurately to avoid security risks caused by excessive permission propagation; the permission propagation factor can be dynamically adjusted according to the actual access situation, making permission management more flexible and adaptable to different application scenarios; by automatically extracting features and correcting the permission propagation factor through a graph convolutional network, manual intervention can be reduced and the efficiency of permission management can be improved.

[0098] In an optional implementation, monitoring permission change events, extracting permission change instructions, matching permission change instructions with rules in an adaptive permission rule library, generating rule matching results, calculating the impact range of permission changes based on the rule matching results, and forming an initial permission change set; performing dependency analysis on the initial permission change set to generate a permission change dependency graph includes:

[0099] The permission synchronization agent monitors permission change events, extracts the operation type, target object and permission attribute from the permission change events, and generates permission change instructions; constructs a dictionary tree, inserts the rule pattern string into the dictionary tree, and constructs an invalidation pointer for the node in the dictionary tree, the invalidation pointer points to the node corresponding to the longest suffix of the current node; matches are performed in the dictionary tree along the permission change instruction, and when the match fails, the invalidation pointer is used to fall back to the node corresponding to the longest suffix to continue matching until the match is successful or falls back to the root node, and a directly affected object set is generated based on the objects corresponding to the successfully matched rules; based on the directly affected object set, the associated rule chain is recursively queried, and each object in the directly affected object set is combined with the corresponding rule chain to form an initial permission change set;

[0100] Perform dependency analysis on each object in the initial permission change set and the corresponding rule chain, start traversal from the starting rule in the rule chain, add the visited rules to the visited set, and recursively visit the associated rules of the current rule; if the associated rule is already in the visited set, record the direct dependency relationship between the current rule and the associated rule; when all the associated rules of the rule have been visited, remove the current rule from the visited set and return to the upper-level rule; iteratively calculate the reachability between the rules through matrix multiplication to obtain the indirect dependency relationship; set the permission objects in the initial permission change set as nodes of the dependency graph, and set the direct dependency and the indirect dependency as edges of the dependency graph; count the number of times the rule is triggered in the historical records, and divide it by the total number of executions to obtain the trigger probability; count the number of permission objects affected by the rule, and divide it by the total number of permission objects in the system to obtain the degree of influence; set the product of the trigger probability and the degree of influence as the dependency strength, set the dependency strength as the edge weight of the dependency graph, and generate a permission change dependency graph.

[0101] In a specific implementation, first, the system monitors permission change events through the permission synchronization agent. For example, if the administrator modifies the access rights of user A to file B, the permission synchronization agent will capture the event. The operation type (modification), target object (file B) and permission attribute (access rights) are extracted from the event, and a permission change instruction is generated: "Modify the access rights of file B".

[0102] Next, the system uses the pre-built adaptive permission rule base for rule matching. The rule base uses a dictionary tree structure and builds an invalidation pointer for each node for fast matching. For example, the rule base contains the rules "All users have read permissions to all files" and "User A has write permissions to file B." The permission change instruction "Modify the access permissions of file B" is matched with the rule base. The matching process is carried out along the dictionary tree. If the match fails, the invalidation pointer is used to fall back to the node corresponding to the longest suffix to continue matching. In this example, "Modify the access permissions of file B" is successfully matched with "User A has write permissions to file B." Based on the successfully matched rules, a set of directly affected objects is generated, which is user A and file B in this example.

[0103] Then, based on the set of directly affected objects, the system recursively queries the associated rule chains to form an initial permission change set. For example, if there is another rule "Users in the group to which user A belongs have read permission to file B", and user A belongs to group C, then this rule and "User A has write permission to file B" form a rule chain. The directly affected objects (user A and file B) are combined with the corresponding rule chains to form an initial permission change set.

[0104] Next, perform dependency analysis on the initial permission change set to generate a permission change dependency graph. Start traversing from the starting rule in the rule chain, for example, start from "User A has write permission to file B". Add the visited rules to the visited set. Recursively access the associated rules of the current rule, for example, "Users in the group that user A belongs to have read permission to file B". If the associated rule is already in the visited set, record the direct dependency relationship from the current rule to the associated rule. When all associated rules of the rule have been visited, remove the current rule from the visited set and return to the upper-level rule. Through multiple iterations, the direct dependency relationship between all rules can be obtained. For example, "User A has write permission to file B" directly depends on "Users in the group that user A belongs to have read permission to file B".

[0105] To obtain indirect dependencies, the system uses a graph traversal approach to calculate the reachability between rules through multiple iterations. For example, if rule X depends on rule Y, and rule Y depends on rule Z, then rule X indirectly depends on rule Z.

[0106] Finally, the system sets the permission objects (user A, file B, group C) in the initial permission change set as nodes of the dependency graph, and sets the direct and indirect dependencies as edges of the dependency graph. The dependency strength is calculated by counting the number of times the rule is triggered in the historical records and the number of permission objects affected, and is set as the edge weight of the dependency graph. For example, if the historical triggering number of "user A has write permission to file B" is 100 times and the total number of executions in the system is 1000 times, the trigger probability is 0.1. If the number of permission objects affected by the rule is 10 and the total number of permission objects in the system is 100, the degree of impact is 0.1. The dependency strength is the product of the trigger probability and the degree of impact, that is, 0.01. Finally, a permission change dependency graph with nodes and weighted edges is generated.

[0107] In this embodiment, by automatically analyzing the impact scope of permission changes, the workload of manual analysis is reduced, thereby improving the efficiency of permission management; by accurately evaluating the impact of permission changes, potential security risks can be avoided, thereby enhancing the security of permission management; by constructing a permission change dependency graph, permissions can be managed more finely and a more flexible permission control strategy can be implemented.

[0108] In an optional implementation, based on the topological structure of the permission change dependency graph, the permission change operations are sorted to generate a permission update instruction sequence; the permission update instruction sequence is written into a message queue, and the permission change snapshot containing the execution timestamp is generated, including:

[0109] Calculate the in-degree of each node in the permission change dependency graph, and add nodes with zero in-degree to the candidate permission node set;

[0110] Calculate the sum of the out-edge weights of each candidate authority node in the candidate authority node set, select the target authority node with the largest sum of out-edge weights, add the change operation corresponding to the target authority node to the authority update instruction sequence, remove the out-edge of the target authority node, update the in-degree value of the successor authority node adjacent to the target authority node, and remove the target authority node from the candidate authority node set; when the in-degree value of the successor authority node becomes zero, add the successor authority node to the candidate authority node set; repeat until the candidate authority node set is empty;

[0111] An incremental counter is used to assign a timestamp to each instruction in the permission update instruction sequence; a permission update topic is created in a distributed message queue, a preparation request containing update content is sent to all participating nodes, and a preparation completion response is waited for all participating nodes to return; after receiving a successful response from all participating nodes, a submission request is sent to all participating nodes, and the permission update instruction sequence is written into the permission update topic; the permission update instruction sequence, the timestamp, and the node relationship and edge weight information of the permission change dependency graph are converted into a binary format to generate a permission change snapshot.

[0112] In a specific implementation, first, a permission change dependency graph is constructed. The graph has permission nodes as vertices and dependencies between permission change operations as edges. Each vertex represents a specific permission, such as read and write permissions for a resource. Each edge represents a dependency between two permission change operations. For example, a user must be granted read permission before write permission can be granted. The weight of an edge represents the strength of the dependency. For example, if certain permissions have a higher priority for change, the weight of the corresponding edge will be greater. For example, a user needs to obtain read permissions for file A and file B, as well as write permissions for file A. If the write permission for file A depends on the read permission for file A, then in the graph, the "read file A" node points to the "write file A" node, and the weight of this edge can be set to a higher value.

[0113] Next, calculate the in-degree of each node in the graph, that is, the number of edges pointing to the node. Add nodes with an in-degree of zero to the candidate permission node set. A node with an in-degree of zero indicates that its corresponding permission change operation has no pre-dependencies and can be executed first. In the above example, the in-degree of the "Read File B" and "Read File A" nodes is zero, so they are added to the candidate permission node set.

[0114] Then, select the target permission node with the largest sum of outgoing edge weights from the candidate permission node set. This means that the node with the largest sum of subsequent operation weights that depends on the current node is selected for priority execution. Add the change operation corresponding to the target permission node to the permission update instruction sequence. Remove the outgoing edge of the target permission node, and update the in-degree value of the successor permission node adjacent to the target permission node. The target permission node is processed and removed from the candidate permission node set. When the in-degree value of the successor permission node becomes zero, add the successor permission node to the candidate permission node set. Repeat this step until the candidate permission node set is empty. Assume that the weight of the edge from "read file A" to "write file A" is 10, and "read file B" has no edges pointing to other nodes. Then select "read file A" and add its corresponding "grant user permission to read file A" operation to the instruction sequence. Then remove the outgoing edge of "read file A" and update the in-degree value of "write file A". At this time, the in-degree of "write file A" becomes 0, and it is added to the candidate permission node set. Finally, select "Write File A" and add "Grant the user permission to write file A" to the command sequence, then select "Read File B" and add "Grant the user permission to read file B" to the command sequence.

[0115] Each instruction in the permission update instruction sequence is assigned a timestamp using an incremental counter to ensure the execution order of the instructions. For example, the first instruction has a timestamp of 1, the second instruction has a timestamp of 2, and so on.

[0116] Create a permission update topic in the distributed message queue. Send a prepare request containing the update content to all participating nodes, and wait for all participating nodes to return a preparation completion response. After receiving a successful response from all participating nodes, send a commit request to all participating nodes and write the permission update instruction sequence into the permission update topic. This ensures data consistency among all nodes.

[0117] Finally, the permission update instruction sequence, timestamp, node relationship, and edge weight information of the permission change dependency graph are converted into binary format to generate a permission change snapshot. For example, Protocol Buffers can be used to serialize this information into binary data for storage and transmission.

[0118] In this embodiment, by sorting the permission change operations, unnecessary waiting and conflicts are avoided, thereby improving the efficiency of permission updates; through distributed message queues and a two-phase commit mechanism, data consistency of all participating nodes is ensured, avoiding errors caused by data inconsistency; the generated permission change snapshot contains complete permission change information, which facilitates subsequent audits and rollback operations, thereby improving the reliability and security of the system.

[0119] In an optional implementation, reading the permission change snapshot, extracting the execution status information of the permission update instruction sequence; comparing the execution status information with the permission change dependency graph, identifying abnormal nodes of the permission update; and generating a permission propagation tree based on the position of the abnormal node in the permission change dependency graph includes:

[0120] The permission change snapshot is constructed as a permission state matrix, which is composed of matrix elements of permission node rows, time series columns and execution state information; the permission state matrix is ​​subjected to difference operation with the permission change dependency graph to obtain a state deviation matrix; when the deviation value in the state deviation matrix exceeds a preset state deviation threshold, the corresponding permission node is marked as an abnormal node;

[0121] Based on the abnormal node, the forward propagation layer and the reverse propagation layer of the permission change dependency graph are constructed. The forward propagation layer traverses downward layer by layer through the permission inheritance relationship, and the reverse propagation layer traverses upward layer by layer through the permission dependency relationship. In each layer of the forward propagation layer and the reverse propagation layer, the propagation impact factor is calculated and determined according to the state deviation value of the abnormal node, the topological distance between nodes, and the cumulative attenuation coefficient of the path. When the propagation impact factor is lower than a preset cutoff threshold, the propagation boundary of the corresponding layer is determined, and the traversal in the corresponding direction is stopped. The traversal results of the forward propagation layer and the reverse propagation layer are merged to generate a permission propagation tree.

[0122] In a specific implementation, first, read the permission change snapshot data. The snapshot records the execution status of each permission change operation, such as "success", "failure", "partial success", etc. Assume that the permission change system records the access permission changes of users A, B, and C to resources X, Y, and Z, and stores the snapshot data in JSON format.

[0123] Next, extract the execution status information of the permission update instruction sequence. Extract the execution status of each permission update operation from the permission change snapshot to form a status sequence. For example, the status of user A's permission update operation on resource X is "successful", the status of user B's permission update operation on resource Y is "failed", and the status of user C's permission update operation on resource Z is "successful".

[0124] Then, the execution status information is compared with the permission change dependency graph to identify abnormal nodes of permission update. The permission change dependency graph describes the dependency relationship between different permission nodes. For example, the permission change of user A depends on the authorization of the administrator, and the permission change of user B depends on the authorization of user A. By comparing the state sequence with the dependency graph, nodes with abnormal execution status can be identified. For example, if the permission update operation status of user A is "failed" and the authorization operation status of the administrator is "successful", the permission node of user A is marked as an abnormal node.

[0125] Next, a permission propagation tree is generated based on the position of the abnormal node in the permission change dependency graph.

[0126] First, the permission change snapshot is constructed as a permission status matrix. The rows of the matrix represent permission nodes, the columns represent time series, and the matrix elements represent the execution status information of the corresponding nodes at the corresponding time points. For example, the first row of the matrix represents user A, the first column represents the first permission change, and the matrix element (1,1) represents the execution status of user A at the time of the first permission change. Assuming that user A's permission for resource X is "successful" in the first change and "failed" in the second change, the matrix element (1,1) is "successful" and (1,2) is "failed".

[0127] Then, the state deviation matrix is ​​obtained by performing a difference operation between the permission state matrix and the permission change dependency graph. The state deviation matrix reflects the difference between the permission node state and the expected state. For example, if the expected state of user A's permission for resource X is "success", but the actual state is "failure", then the corresponding element value in the state deviation matrix is ​​"failure" - "success" = negative value, indicating a negative deviation.

[0128] Next, when the deviation value in the state deviation matrix exceeds the preset state deviation threshold, the corresponding authority node is marked as an abnormal node. For example, if the state deviation threshold is set to -0.5, all nodes with deviation values ​​less than -0.5 are marked as abnormal nodes.

[0129] Then, the forward propagation layer and the reverse propagation layer of the permission change dependency graph are constructed based on the abnormal nodes. The forward propagation layer traverses downwards layer by layer through the permission inheritance relationship, and the reverse propagation layer traverses upwards layer by layer through the permission dependency relationship. For example, if user B's permission depends on user A, user A is the reverse propagation layer node of user B, and user B is the forward propagation layer node of user A.

[0130] In each layer of the forward propagation layer and the reverse propagation layer, the propagation impact factor is calculated based on the state deviation value of the abnormal node, the topological distance between nodes, and the cumulative attenuation coefficient of the path. The propagation impact factor indicates the degree of influence of the abnormal node on other nodes. For example, if user A is an abnormal node and user B depends on user A, the propagation impact factor of user A on user B will decay as the topological distance increases.

[0131] When the propagation impact factor is lower than the preset cutoff threshold, the propagation boundary of the corresponding layer is determined and the traversal in the corresponding direction is stopped. For example, if the cutoff threshold is set to 0.1, all nodes with a propagation impact factor less than 0.1 are considered to be beyond the propagation boundary and the traversal is stopped.

[0132] Finally, the traversal results of the forward propagation layer and the backward propagation layer are combined to generate a permission propagation tree. The permission propagation tree clearly shows the impact range of abnormal nodes.

[0133] In this embodiment, by identifying abnormal nodes and analyzing their scope of influence, potential security risks can be discovered in a timely manner, and corresponding measures can be taken to prevent permission abuse and data leakage; by automatically analyzing the impact of permission changes, manual intervention can be reduced and the efficiency of permission management can be improved; by analyzing the permission propagation tree, the scope of influence of permission changes can be more accurately controlled to avoid unexpected permission changes leading to system failures or business interruptions.

[0134] In an optional implementation, calculating the permission consistency score of each propagation path in the permission propagation tree; when the permission consistency score is lower than a preset threshold, extracting the rule matching result corresponding to the abnormal node; generating a permission repair plan according to the rule matching result; converting the permission repair plan into a repair instruction sequence; executing the repair instruction sequence to update the permission status; writing the repaired permission status and repair process data into the adaptive permission rule library includes:

[0135] For each propagation path in the permission propagation tree, the permission consistency score is calculated by the propagation impact factor on the propagation path and the permission value difference between adjacent permission nodes; when the permission consistency score is lower than the preset score threshold, the rule matching result corresponding to the abnormal node is extracted;

[0136] Obtaining the rule weight value and rule influence direction of each rule from the rule matching result, calculating the correction direction of the current authority value and the target authority value of the abnormal node, sorting in descending order according to the rule weight values, selecting the rule with the largest rule weight value and the same rule influence direction as the correction direction as the repair rule; applying the repair rule to the abnormal node, determining the rule repair parameter based on the state deviation value of the abnormal node; generating a repair instruction sequence according to the repair rule and the repair parameter;

[0137] Execute the repair instruction sequence within a preset time window, and record the start execution time, completion execution time, execution status code, and state change amount after execution of each repair instruction; when it is detected that the execution status code is abnormal or the state change amount does not meet the preset change amount threshold, trigger the corresponding repair instruction to retry execution;

[0138] Calculate the degree of improvement of the state deviation before and after executing the repair instruction sequence, the state fluctuation during the execution process and the execution success rate of the repair instruction; write the degree of improvement of the state deviation, the state fluctuation, the execution success rate and the repaired permission state into the adaptive permission rule library.

[0139] In a specific implementation, first, a permission propagation tree is constructed. The permission propagation tree reflects the inheritance and transfer relationship of permissions in the system. The root node of the tree represents the highest permission, and the child nodes inherit the permissions of the parent node and can be passed down. Each node contains a permission value, which indicates the permission level of the node. For example, the root node permission value is 10, and its child node permission values ​​may be 8, 7, etc. The lower the value, the lower the permission level.

[0140] Then, the permission consistency score is calculated. Each propagation path in the permission propagation tree is traversed, and the permission consistency score is calculated based on the permission value difference between adjacent permission nodes on the path and the predefined propagation impact factor. The propagation impact factor is used to measure the acceptable range of the difference in permission values ​​of adjacent nodes. For example, if the parent node permission value is 8, the child node permission value is 5, and the propagation impact factor is 0.2, the permission consistency score of the path segment is high; if the child node permission value is 1, the score is low, indicating that there may be permission anomalies.

[0141] Next, extract the abnormal nodes and rule matching results. When the permission consistency score of a propagation path is lower than the preset threshold, it is considered that some nodes on the path have permission anomalies. Extract these abnormal nodes and match the rules according to the adaptive permission rule base. The rule base contains a series of predefined permission repair rules, each of which corresponds to a specific permission anomaly scenario. For example, the rule base may contain rules such as "the child node permission must not be less than half of the parent node permission".

[0142] Select the best repair rule. For each abnormal node, multiple repair rules may be matched. Select the best repair rule based on the weight value of each rule and the rule impact direction. The rule weight value indicates the importance of the rule, and the rule impact direction indicates the direction (increase or decrease) of the rule's correction to the permission value. For example, if the current permission value of the abnormal node is 1 and the target permission value is 5, select the rule with the rule impact direction of "increase". Among multiple candidate rules, select the rule with the largest weight value as the final repair rule.

[0143] Generate and execute a repair instruction sequence. Determine the rule repair parameters and generate a repair instruction sequence based on the selected repair rule and the state deviation value of the abnormal node (the difference between the current permission value and the target permission value). The repair instruction sequence contains a series of operation instructions for modifying the permission value of the abnormal node. For example, if the repair rule is "set the child node permission to half of the parent node permission", and the parent node permission value is 8, the generated repair instruction sequence is "set the child node permission to 4". Execute the repair instruction sequence within the preset time window and record relevant data during the execution process, such as the execution status code, state change, etc. If an execution abnormality is detected, the retry mechanism is triggered.

[0144] Update the adaptive permission rule base. Update the adaptive permission rule base according to the execution results of the repair instruction sequence. Write the data such as the permission status after repair, the degree of improvement of state deviation, the state fluctuation, the success rate of repair instruction execution, etc. into the rule base for reference and optimization of subsequent permission repair. For example, if the execution success rate of a repair rule is always low, you can adjust the weight value of the rule or modify the rule itself.

[0145] Exemplarily, assume that there is an abnormal node in the permission propagation tree, whose current permission value is 1 and the target permission value is 5. According to the matching results of the rule base, there are two candidate rules: Rule A: increase the permission value by 2, with a weight value of 0.8; Rule B: increase the permission value by 1, with a weight value of 0.5. Since the weight value of Rule A is higher and the rule impact direction is the same as the correction direction, Rule A is selected as the repair rule. According to the state deviation value of 4, the rule repair parameter is determined to be 2. The generated repair instruction sequence is "increase the permission value by 2". After executing the instruction, the permission value of the abnormal node becomes 3. The repaired permission value, the degree of improvement of the state deviation and other data are written into the adaptive permission rule base for subsequent permission repair.

[0146] In this embodiment, permission anomalies can be automatically identified and repaired without manual intervention, thereby improving permission management efficiency; through permission consistency scoring and adaptive rule base, it is ensured that permission configuration complies with predefined rules and policies to avoid situations where permissions are too high or too low; timely repair of permission anomalies can effectively prevent unauthorized access and data leakage, thereby improving the security of the system.

[0147] Figure 2 FIG. 1 is a schematic diagram of the structure of a distributed storage and data warehouse permission synchronization system according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0148] The first unit is used to obtain user operation data in the distributed storage system, generate a behavior data set, perform feature extraction on the behavior data set, and calculate a user access pattern feature matrix; input the user access pattern feature matrix into a pre-trained machine learning model, calculate the directory access frequency weight, time sensitivity coefficient and permission propagation factor, and generate a directory permission intelligent mapping table; read the structural information of the data table in the data warehouse, calculate the value score, field association strength and cross-table access influence of the data table, and generate a multi-dimensional table permission evaluation model; perform weighted calculation on the directory permission intelligent mapping table and the parameters in the multi-dimensional table permission evaluation model to generate an adaptive permission rule base;

[0149] The second unit is used to deploy permission synchronization agents on physical nodes of distributed storage, distribute the adaptive permission rule library to each permission synchronization agent according to the consistent hashing algorithm; monitor permission change events, extract permission change instructions, match the permission change instructions with the rules in the adaptive permission rule library, generate rule matching results, calculate the impact range of the permission change based on the rule matching results, and form an initial permission change set; perform dependency analysis on the initial permission change set to generate a permission change dependency graph; sort the permission change operations based on the topological structure of the permission change dependency graph to generate a permission update instruction sequence; write the permission update instruction sequence into a message queue to generate a permission change snapshot containing an execution timestamp;

[0150] The third unit is used to read the permission change snapshot and extract the execution status information of the permission update instruction sequence; compare the execution status information with the permission change dependency graph to identify the abnormal nodes of the permission update; generate a permission propagation tree based on the position of the abnormal node in the permission change dependency graph; calculate the permission consistency score of each propagation path in the permission propagation tree; when the permission consistency score is lower than a preset threshold, extract the rule matching result corresponding to the abnormal node; generate a permission repair plan according to the rule matching result; convert the permission repair plan into a repair instruction sequence; execute the repair instruction sequence to update the permission status; write the repaired permission status and repair process data into the adaptive permission rule library.

[0151] According to a third aspect of the embodiments of the present invention,

[0152] An electronic device is provided, comprising:

[0153] processor;

[0154] a memory for storing processor-executable instructions;

[0155] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0156] A fourth aspect of the embodiments of the present invention is:

[0157] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0158] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium on which are loaded computer-readable program instructions for executing various aspects of the present invention. Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solution from the scope of the technical solution of the embodiments of the present invention.

Claims

1. A method for synchronizing permissions between distributed storage and data warehouse, characterized in that: include: Acquire user operation data in a distributed storage system, generate a behavior data set, perform feature extraction on the behavior data set, and calculate a user access pattern feature matrix; Input the user access pattern feature matrix into a pre-trained machine learning model, calculate the directory access frequency weight, time sensitivity coefficient and permission propagation factor, and generate a directory permission intelligent mapping table; Read the structural information of the data table in the data warehouse, calculate the value score, field association strength and cross-table access impact of the data table, and generate a multi-dimensional table permission evaluation model; Performing weighted calculation on the parameters in the directory permission intelligent mapping table and the multi-dimensional table permission evaluation model to generate an adaptive permission rule library; Deploy permission synchronization agents on physical nodes of distributed storage, and distribute the adaptive permission rule library to each permission synchronization agent according to a consistent hashing algorithm; Monitor permission change events, extract permission change instructions, match permission change instructions with rules in the adaptive permission rule library, generate rule matching results, calculate the impact scope of permission changes based on the rule matching results, and form an initial permission change set; Perform dependency analysis on the initial permission change set to generate a permission change dependency graph; Based on the topological structure of the permission change dependency graph, the permission change operations are sorted to generate a permission update instruction sequence; Writing the permission update instruction sequence into a message queue to generate a permission change snapshot including an execution timestamp; Reading the permission change snapshot and extracting execution status information of the permission update instruction sequence; Compare the execution status information with the permission change dependency graph to identify abnormal nodes of permission update; Generate a permission propagation tree based on the position of the abnormal node in the permission change dependency graph; Calculating the permission consistency score of each propagation path in the permission propagation tree; When the authority consistency score is lower than a preset threshold, extracting the rule matching result corresponding to the abnormal node; Generate a permission repair plan according to the rule matching result; Converting the permission repair scheme into a repair instruction sequence; executing the repair instruction sequence to update the permission status; The repaired permission status and the repair process data are written into the adaptive permission rule library.

2. The method according to claim 1, characterized in that Acquire user operation data in a distributed storage system, generate a behavior data set, perform feature extraction on the behavior data set, and calculate a user access pattern feature matrix; Input the user access pattern feature matrix into a pre-trained machine learning model, calculate the directory access frequency weight, time sensitivity coefficient and permission propagation factor, and generate a directory permission intelligent mapping table; Read the structural information of the data table in the data warehouse, calculate the value score, field association strength and cross-table access impact of the data table, and generate a multi-dimensional table permission evaluation model; Performing weighted calculation on the directory permission intelligent mapping table and the parameters in the multi-dimensional table permission evaluation model to generate an adaptive permission rule base includes: Obtain user access time, operation type, and access path data in the audit log of the distributed storage system, and use the sliding time window method to preprocess the user access time, operation type, and access path data to generate a behavior data set; based on the behavior data set, extract time series features, operation features, and path features, and vectorize the time series features, operation features, and path features to generate a user access pattern feature matrix; Input the user access pattern feature matrix into the pre-trained machine learning model, obtain the directory access frequency weight by multiplying and adding the time decay factor and the number of accesses, calculate the time sensitivity coefficient by the exponential decay function, and calculate the permission propagation factor by the intersection and union ratio of the directory permission set; construct a directory access relationship graph, set the directory as the graph node, set the user access path as the graph edge, extract the progressive propagation feature between directories through the graph convolution network based on the directory access relationship graph, and modify the permission propagation factor based on the progressive propagation feature; generate a directory permission intelligent mapping table based on the modified permission propagation factor, the directory access frequency weight and the time sensitivity coefficient; Read the structural information of the data table in the data warehouse, calculate the sensitivity score, citation frequency and data volume score based on the structural information, and perform weighted calculation on the sensitivity score, citation frequency and data volume score to obtain the data table value score; calculate the field association coefficient and field importance score based on the structural information, and perform weighted calculation on the field association coefficient and field importance score to obtain the field association strength; calculate the access frequency and dependency between tables based on the structural information, and perform the product accumulation of the access frequency and dependency between tables to obtain the cross-table access impact, and generate a multi-dimensional table permission evaluation model based on the data table value score, field association strength and cross-table access impact; The directory permission intelligent mapping table is weightedly calculated with the data table value score, field association strength and cross-table access impact in the multi-dimensional table permission evaluation model to generate permission rules, and the permission rules are stored in an adaptive permission rule library.

3. The method according to claim 2, characterized in that Construct a directory access relationship graph, set the directory as a graph node, set the user access path as a graph edge, extract the progressive propagation features between directories through a graph convolutional network based on the directory access relationship graph, and modify the permission propagation factor based on the progressive propagation features, including: Building an initial directory graph based on the access path data in the behavior data set, and setting the direct access frequency between directory nodes as the initial weight of the edge; Calculate the shortest path distance between any two directory nodes in the initial directory graph, construct a spatial adjacency matrix based on the shortest path distance, and multiply the spatial adjacency matrix by the initial weight of the edge to obtain the spatial association strength of the directory node; Extracting access timing information from the behavior data set, calculating the access time interval between directory node pairs, normalizing the access time interval to obtain a time decay coefficient, and multiplying the time decay coefficient by the spatial correlation strength to obtain a spatiotemporal weighted adjacency matrix; Performing sparse processing on the spatiotemporal weighted adjacency matrix, retaining edges with weight values ​​greater than a preset weight threshold, and obtaining an optimized directory access relationship graph; A multi-layer graph convolutional network is used to extract features from the optimized directory access relationship graph, wherein the first graph convolutional layer extracts local access pattern features, the second graph convolutional layer fuses multi-hop access path features, and the third graph convolutional layer generates a progressive propagation feature vector of the directory node; A similarity matrix between directory nodes is calculated based on the progressive propagation feature vector, and a dot product operation is performed on the similarity matrix and the original permission propagation factor to obtain a modified permission propagation factor.

4. The method according to claim 1, characterized in that: Monitor permission change events, extract permission change instructions, match permission change instructions with rules in the adaptive permission rule library, generate rule matching results, calculate the impact scope of permission changes based on the rule matching results, and form an initial permission change set; Perform dependency analysis on the initial permission change set to generate a permission change dependency graph including: The permission synchronization agent monitors permission change events, extracts the operation type, target object and permission attribute from the permission change events, and generates permission change instructions; constructs a dictionary tree, inserts the rule pattern string into the dictionary tree, and constructs an invalidation pointer for the node in the dictionary tree, the invalidation pointer points to the node corresponding to the longest suffix of the current node; matches are performed in the dictionary tree along the permission change instruction, and when the match fails, the invalidation pointer is used to fall back to the node corresponding to the longest suffix to continue matching until the match is successful or falls back to the root node, and a directly affected object set is generated based on the objects corresponding to the successfully matched rules; based on the directly affected object set, the associated rule chain is recursively queried, and each object in the directly affected object set is combined with the corresponding rule chain to form an initial permission change set; Perform dependency analysis on each object in the initial permission change set and the corresponding rule chain, start traversal from the starting rule in the rule chain, add the visited rules to the visited set, and recursively visit the associated rules of the current rule; if the associated rule is already in the visited set, record the direct dependency relationship between the current rule and the associated rule; when all the associated rules of the rule have been visited, remove the current rule from the visited set and return to the upper-level rule; iteratively calculate the reachability between the rules through matrix multiplication to obtain the indirect dependency relationship; set the permission objects in the initial permission change set as nodes of the dependency graph, and set the direct dependency and the indirect dependency as edges of the dependency graph; count the number of times the rule is triggered in the historical records, and divide it by the total number of executions to obtain the trigger probability; count the number of permission objects affected by the rule, and divide it by the total number of permission objects in the system to obtain the degree of influence; set the product of the trigger probability and the degree of influence as the dependency strength, set the dependency strength as the edge weight of the dependency graph, and generate a permission change dependency graph.

5. The method according to claim 4, characterized in that Based on the topological structure of the permission change dependency graph, the permission change operations are sorted to generate a permission update instruction sequence; Writing the permission update instruction sequence into a message queue and generating a permission change snapshot including an execution timestamp includes: Calculate the in-degree of each node in the permission change dependency graph, and add nodes with zero in-degree to the candidate permission node set; Calculate the sum of the out-edge weights of each candidate authority node in the candidate authority node set, select the target authority node with the largest sum of out-edge weights, add the change operation corresponding to the target authority node to the authority update instruction sequence, remove the out-edge of the target authority node, update the in-degree value of the successor authority node adjacent to the target authority node, and remove the target authority node from the candidate authority node set; when the in-degree value of the successor authority node becomes zero, add the successor authority node to the candidate authority node set; repeat until the candidate authority node set is empty; An incremental counter is used to assign a timestamp to each instruction in the permission update instruction sequence; a permission update topic is created in a distributed message queue, a preparation request containing update content is sent to all participating nodes, and a preparation completion response is waited for all participating nodes to return; after receiving a successful response from all participating nodes, a submission request is sent to all participating nodes, and the permission update instruction sequence is written into the permission update topic; the permission update instruction sequence, the timestamp, and the node relationship and edge weight information of the permission change dependency graph are converted into a binary format to generate a permission change snapshot.

6. The method according to claim 1, characterized in that Read the permission change snapshot and extract the execution status information of the permission update instruction sequence; compare the execution status information with the permission change dependency graph to identify abnormal nodes of the permission update; Generating a permission propagation tree based on the position of the abnormal node in the permission change dependency graph includes: The permission change snapshot is constructed as a permission state matrix, which is composed of matrix elements of permission node rows, time series columns and execution state information; the permission state matrix is ​​subjected to difference operation with the permission change dependency graph to obtain a state deviation matrix; when the deviation value in the state deviation matrix exceeds a preset state deviation threshold, the corresponding permission node is marked as an abnormal node; Based on the abnormal node, the forward propagation layer and the reverse propagation layer of the permission change dependency graph are constructed. The forward propagation layer traverses downward layer by layer through the permission inheritance relationship, and the reverse propagation layer traverses upward layer by layer through the permission dependency relationship. In each layer of the forward propagation layer and the reverse propagation layer, the propagation impact factor is calculated and determined according to the state deviation value of the abnormal node, the topological distance between nodes, and the cumulative attenuation coefficient of the path. When the propagation impact factor is lower than a preset cutoff threshold, the propagation boundary of the corresponding layer is determined, and the traversal in the corresponding direction is stopped. The traversal results of the forward propagation layer and the reverse propagation layer are merged to generate a permission propagation tree.

7. The method according to claim 6, characterized in that Calculating the permission consistency score of each propagation path in the permission propagation tree; when the permission consistency score is lower than a preset threshold, extracting the rule matching result corresponding to the abnormal node; Generate a permission repair plan according to the rule matching result; Converting the permission repair scheme into a repair instruction sequence; Execute the repair instruction sequence to update the permission status; Writing the repaired permission status and the repair process data into the adaptive permission rule base includes: For each propagation path in the permission propagation tree, the permission consistency score is calculated by the propagation impact factor on the propagation path and the permission value difference between adjacent permission nodes; when the permission consistency score is lower than the preset score threshold, the rule matching result corresponding to the abnormal node is extracted; Obtaining the rule weight value and rule influence direction of each rule from the rule matching result, calculating the correction direction of the current authority value and the target authority value of the abnormal node, sorting in descending order according to the rule weight values, selecting the rule with the largest rule weight value and the same rule influence direction as the correction direction as the repair rule; applying the repair rule to the abnormal node, determining the rule repair parameter based on the state deviation value of the abnormal node; generating a repair instruction sequence according to the repair rule and the repair parameter; Execute the repair instruction sequence within a preset time window, and record the start execution time, completion execution time, execution status code, and state change amount after execution of each repair instruction; when it is detected that the execution status code is abnormal or the state change amount does not meet the preset change amount threshold, trigger the corresponding repair instruction to retry execution; Calculate the degree of improvement of the state deviation before and after executing the repair instruction sequence, the state fluctuation during the execution process and the execution success rate of the repair instruction; write the degree of improvement of the state deviation, the state fluctuation, the execution success rate and the repaired permission state into the adaptive permission rule library.

8. A distributed storage and data warehouse permission synchronization system, used to implement the method according to any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain user operation data in the distributed storage system, generate a behavior data set, perform feature extraction on the behavior data set, and calculate a user access pattern feature matrix; Input the user access pattern feature matrix into a pre-trained machine learning model, calculate the directory access frequency weight, time sensitivity coefficient and permission propagation factor, and generate a directory permission intelligent mapping table; Read the structural information of the data table in the data warehouse, calculate the value score, field association strength and cross-table access impact of the data table, and generate a multi-dimensional table permission evaluation model; Performing weighted calculation on the parameters in the directory permission intelligent mapping table and the multi-dimensional table permission evaluation model to generate an adaptive permission rule library; The second unit is used to deploy a permission synchronization agent on a physical node of the distributed storage, and distribute the adaptive permission rule library to each permission synchronization agent according to a consistent hashing algorithm; Monitor permission change events, extract permission change instructions, match permission change instructions with rules in the adaptive permission rule library, generate rule matching results, calculate the impact scope of permission changes based on the rule matching results, and form an initial permission change set; Perform dependency analysis on the initial permission change set to generate a permission change dependency graph; Based on the topological structure of the permission change dependency graph, the permission change operations are sorted to generate a permission update instruction sequence; Writing the permission update instruction sequence into a message queue to generate a permission change snapshot including an execution timestamp; A third unit is used to read the permission change snapshot and extract the execution status information of the permission update instruction sequence; Compare the execution status information with the permission change dependency graph to identify abnormal nodes of permission update; Generate a permission propagation tree based on the position of the abnormal node in the permission change dependency graph; Calculating the permission consistency score of each propagation path in the permission propagation tree; When the authority consistency score is lower than a preset threshold, extracting the rule matching result corresponding to the abnormal node; Generate a permission repair plan according to the rule matching result; Converting the permission repair scheme into a repair instruction sequence; executing the repair instruction sequence to update the permission status; The repaired permission status and the repair process data are written into the adaptive permission rule library.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images