Intelligent management method for cloud storage access authority based on reinforcement learning

Abstract

The invention discloses a reinforcement learning-based cloud storage access permission intelligent management method. The method comprises the following steps of S1, constructing a hierarchical reinforcement learning model; s2, collecting historical access data of a user in the cloud storage system; s3, configuring an initial permission and resource allocation according to a user group permission strategy, and applying a configuration result to a low-layer operation module; s4, feeding back to a hierarchical reinforcement learning model; s5, preferentially retaining the files or data blocks accessed at high frequency in the cache; s6, the hierarchical reinforcement learning model optimizes an authority strategy through a real-time reward mechanism; s7, dynamically recovering the data blocks or files accessed at low frequency; s8, performing synchronous optimization of the access authority and the cache strategy; s9, temporarily freezing the high-risk authority; and S10, generating a new permission strategy and a cache strategy. According to the invention, the response speed and the resource utilization efficiency of the system are improved, and the security and the fault-tolerant capability of the system are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of cloud storage technology, and in particular to a cloud storage access rights intelligent management method based on reinforcement learning. Background Art

[0002] With the rapid development of cloud computing and big data technology, more and more enterprises and individual users are migrating data storage and management to the cloud. Cloud storage systems provide users with convenient storage services with their efficiency, scalability and flexibility. However, with the increase in the number of users in the cloud environment and the complexity of their access needs, traditional permission management and resource scheduling methods have gradually exposed significant problems and limitations.

[0003] In existing cloud storage systems, access rights are usually managed through pre-defined static policies. Administrators manually assign access rights based on user roles or functions and configure corresponding resources for different user groups. The static permission management model can initially meet the needs of some fixed scenarios, but with the dynamic changes in user access behavior and the increasing demand for real-time scheduling of cloud resources, traditional static configurations are gradually unable to cope with complex and changing access scenarios. In order to cope with changes in user needs, administrators need to frequently update permissions and resource configurations, resulting in increased management costs and error rates.

[0004] To improve management efficiency, some cloud storage systems have introduced rule-based automated management methods that trigger permission changes and resource allocations through pre-set conditions and rules. However, rule-based management methods lack flexibility and adaptability, and it is difficult to automatically adjust permissions based on users' real-time behavior. In addition, in existing permission management technologies, administrators usually tend to grant excessive authorization beyond actual needs in order to reduce users' frequent permission applications, which increases the security risks of the system.

[0005] In summary, existing technologies have obvious deficiencies in dealing with dynamic user behaviors, achieving flexible permission management, and avoiding over-authorization, and are unable to meet the efficient and secure management requirements of cloud storage systems. Therefore, a new solution is urgently needed that can automatically optimize permissions and resource allocation based on users' real-time behaviors and access history, and achieve smarter and safer cloud storage permission management. Summary of the invention

[0006] One purpose of the present invention is to propose a cloud storage access rights intelligent management method based on reinforcement learning, which improves the response speed and resource utilization efficiency of the system and enhances the security and fault tolerance of the system.

[0007] A cloud storage access rights intelligent management method based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0008] S1. Construct a hierarchical reinforcement learning model. The hierarchical reinforcement learning model includes a high-level policy module and a low-level operation module. The high-level policy module is responsible for the management of user groups and the initial allocation of resource permissions. The low-level operation module is used for access permission control of specific files or data blocks and dynamic cache strategy adjustment.

[0009] S2, collecting historical access data of users in the cloud storage system, and inputting the historical access data into the high-level policy module to generate an initial user group permission policy;

[0010] S3. Configure initial permissions and resource allocation according to the user group permission policy, and apply the configuration results to the low-level operation module so that each user group has the minimum access rights required to complete its role tasks when accessing for the first time;

[0011] S4. When a user requests to access cloud storage resources for the first time, the low-level operation module calls the cache management system to load the accessed data blocks or files into the cache area, and records the specific behavior data of the access and feeds it back to the hierarchical reinforcement learning model;

[0012] S5, the hierarchical reinforcement learning model dynamically adjusts the cache strategy based on the data access frequency in the cache, the usage pattern of the user group, and the historical behavior pattern, so that the files or data blocks with high frequency access are preferentially retained in the cache;

[0013] S6. The hierarchical reinforcement learning model optimizes the permission strategy through a real-time reward mechanism. It gives positive rewards and adjusts user permissions for access behaviors that comply with system security specifications; it gives negative rewards for abnormal behaviors and triggers an automatic permission reduction mechanism.

[0014] S7. When the system detects that the cache area resources are insufficient, the hierarchical reinforcement learning model dynamically recycles the data blocks or files with low frequency of access based on the current user's access priority, cache resource occupancy and system load status;

[0015] S8. When the system load increases or user access requests are frequent, the high-level policy module re-evaluates the access requirements of each user group and passes the adjusted user group permission policy to the low-level operation module to perform synchronous optimization of access rights and cache policies;

[0016] S9. When abnormal behaviors occur frequently, the system triggers a secondary verification process based on the feedback from the hierarchical reinforcement learning model, requiring the user to perform additional identity verification and temporarily freezing their high-risk permissions;

[0017] S10. Regularly evaluate and optimize permission policies and cache policies, feed back historical access data and cache hit rates to high-level policy modules, and generate new permission policies and cache policies.

[0018] Optionally, the S1 includes:

[0019] S11. Construct a hierarchical reinforcement learning model, including a high-level policy module and a low-level operation module; the high-level policy module is used to manage the initial allocation of user groups and resource permissions, and the low-level operation module is used to control access permissions for specific files or data blocks and adjust dynamic cache strategies;

[0020] S12. In the high-level strategy module, define the state space in represents the status of the i-th user group, u i is the feature vector of the user group, including roles and permission levels, r i is the resource access record, f i To collect visit frequency statistics;

[0021] S13. In the high-level strategy module, define the action space in Indicates the action of assigning permissions to user groups, assigning permission set P j Access rights for specific resources;

[0022] S14. In the high-level strategy module, establish a reward function:

[0023]

[0024] Among them, E s =∑ k δ(p k )·ω k represents the system security index, δ(p k ) is the permission p k The risk indicator function, ω k is the risk weight of the system security indicator authority, E r =∑ k ψ(p k )·c k represents the resource utilization index, ψ(p k ) is the permission p k The resource usage of k is the resource consumption cost, E u =∑ k φ(p k )·μ k , represents the user satisfaction index, φ(p k ) is the permission p k Contribution to user task completion, μ k For the user to have permissions p k The demand intensity, λ1, λ2, λ3 are adjustment parameters used to balance system security, resource utilization and user satisfaction;

[0025] S15. In the low-level operation module, define the state space in Indicates the status of the lth data block or file, d l is the data feature vector, including size and type, c l The cache status is 1, which means it is in the cache, or 0, which means it is not in the cache. l is the frequency of visits;

[0026] S16. In the low-level operation module, define the action space in Cache operations performed on data blocks or files, including:

[0027] a load To load a data block into the cache;

[0028] a retain To keep data blocks in cache;

[0029] a replace To replace the data block in the cache with a new data block;

[0030] a evict To remove a data block from the cache;

[0031] S17. In the low-level operation module, establish a reward function:

[0032]

[0033] in, represents the improvement in cache hit rate, Δh is the increase in cache hit times, and h total is the total number of visits, C c =∑ l c l ·s l Indicates cache resource consumption, s l is the size of data block l, γ1 and γ2 are adjustment parameters used to balance the improvement of cache hit rate and cache resource consumption;

[0034] S18. Establish a collaborative strategy between the high-level strategy module and the low-level operation module through a hierarchical strategy function:

[0035] π(s,a)=π H (s H ,a H )·π L (s L ,a L |s H ,a H );

[0036] Where s=(s H ,s L ) is the joint state, a=(a H ,a L ) is a joint action, π H (s H ,a H ) is a high-level policy that determines the permission allocation of user groups, π L (s L ,a L |s H ,a H ) is a low-level strategy, which selects a cache operation based on the output of the high-level strategy and the current data block status;

[0037] S19. Use reinforcement learning algorithm to optimize high-level strategy modules and low-level operation modules to maximize the overall expected cumulative reward:

[0038]

[0039] Among them, θ H ,θ L are the parameters of the high-level strategy module and the low-level operation module, and γ is the discount factor used to balance short-term and long-term benefits;

[0040] Update the policy parameters through the policy gradient method:

[0041]

[0042] Among them, α H ,α L is the learning rate of the high-level strategy module and the low-level operation module, and They are the gradients of the parameters of the high-level strategy module and the low-level operation module, respectively, guiding the optimization direction of the strategy.

[0043] Optionally, S2 includes:

[0044] S21. Collecting historical access data of users in the cloud storage system, where the historical access data includes access time, access frequency, role information, and operation type;

[0045] S22. Constructing access record matrix Where m ij Represents user u i In time period t j The number of access behaviors in a period, n is the number of users, t is the number of time periods, and each element in the matrix reflects the access frequency of the user in a specific time period;

[0046] S23. Define the user role set U, each user u i With the eigenvector x i =(r i ,o i ), r i Indicates the user's role type, o i Represents the set of common operation types of the user; the feature vector is used to describe the access characteristics of the user group;

[0047] S26. Generate a time-weighted access matrix W = [w ij ]:

[0048] w ij =α·f ij ·exp(-β·Δt j );

[0049] Among them, f ij For user u i For operation j The access frequency, Δt j is the time interval between the operation and the current time, α and β are adjustment parameters for balancing historical access data and recent data;

[0050] S27, historical access data matrix M, user feature vector x i The time-weighted access matrix W is input into the high-level policy module to determine the optimal initial permission policy for the user group by maximizing the product of access frequency and time weighting:

[0051]

[0052] Where P = [p ij ] indicates user u i For operation j The initial authority matrix, p ij =1 means granting permission, p ij =0 means no permission is granted;

[0053] S28, according to the user group authority policy, divide the users into multiple authority level groups G = {g1, g2, ..., g m}, each group g i Include several users and match appropriate resource permissions based on their historical access data;

[0054] S29. Apply the generated initial permission strategy and group division results to the permission management process and serve as the initial configuration of the high-level policy module to provide a basis for the specific permission allocation and cache strategy adjustment of the low-level operation module.

[0055] Optionally, the S3 includes:

[0056] S31, according to the user group authority policy, the user group authority matrix P group Each element p in ij With user group u i For operation j The initial permission match of ij =1 means granting permission, p ij =0 means no permission is granted;

[0057] S32. Define resource allocation vector R i Represented as user group u i the mix of resources allocated;

[0058] S33, map permissions and resources, and establish a mapping function φ(u i ,o j ) means when user group u i Granted Operation o j When the user has the permission, the system allocates the corresponding resource combination R according to the user's needs. i :

[0059]

[0060] Among them, (u i ,o j ) is the user group u i Execute operation j The required resource combination, ω j For operation j The weight of R indicates its importance in the system. j For operation o j The total amount of resource type allocated, μ ij For user group u i For operation j The demand intensity is calculated based on the historical access frequency, T max is the maximum effective time window of the permission, T ij For operation j The remaining time of the corresponding permission;

[0061] S34. Generate an initial authority-resource allocation matrix according to the mapping function:

[0062] Q=[q ij ];

[0063] Among them, q ij =1 indicates user group u i Operation obtained j The minimum resource support required, q ij =0 means no resources are allocated;

[0064] S35, passing the generated initial authority-resource allocation matrix Q to the lower-level operation module, so that the lower-level operation module matches the user's authority and performs corresponding resource allocation when the user requests for the first time;

[0065] S36. When a user first accesses the cloud storage system, the low-level operation module calls the cache management system to load the required data blocks into the cache area according to the initial permission-resource allocation matrix Q, and allocates computing resources and network bandwidth for the operation of the user group;

[0066] S37, set the timeliness constraint of the operation for each user group and define the timeliness function τ(u i ,o j ), when the permission expires or the user task is completed, the low-level operation module i ,o j ) automatically releases the allocated resources and updates the permission status of the user group based on the access records.

[0067] Optionally, the S6 includes:

[0068] S61. Based on the access behavior of the user group, the reward value of the current behavior is calculated according to the reward function of S2. Behaviors that comply with the system security specifications will receive positive rewards, and abnormal behaviors will receive negative rewards.

[0069] S62. When an access behavior that complies with security specifications is detected, the system expands the user's authority scope according to the reward value, allowing the user to access more resources;

[0070] S63. When abnormal behavior is detected, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group;

[0071] S64, defining a recovery strategy function ρ(t) for the user group whose rights are triggered to be reduced, where ρ(t) indicates that some or all rights are restored after time t; if no new abnormal behavior is found within the specified time, the rights will be automatically restored;

[0072] S65. Feedback the permission adjustment result to the high-level policy module to update the permission policy of the user group, and use it as a reference in the next permission evaluation.

[0073] Optionally, the S10 includes:

[0074] S101. Periodically collect historical access data and cache hit rate data in the system to construct an evaluation data set:

[0075] D = (H, C);

[0076] Where H = [h ij ] indicates user group u iFor resources j The access frequency matrix, C = [c lm ] indicates data block d l The hit status in the cache, c lm =1 means hit, c lm =0 means miss;

[0077] S102: Define a cache evaluation function η(C) according to the cache hit rate data C to calculate the cache hit rate:

[0078]

[0079] Among them, L is the number of data blocks, M is the number of access requests, and the cache hit rate η(C) is used to evaluate the execution effect of the cache strategy;

[0080] S103. According to the user's historical access data H, define the permission evaluation index ζ(H) to calculate the matching degree of resource access:

[0081]

[0082] Where n is the number of user groups, k is the number of resources, and p ij For user group u i For resources j The permission status of the,matching degree ζ(H) indicates the consistency between the actual access and the,permission allocation;

[0083] S104: Feedback the cache evaluation result η(C) and the authority evaluation result ζ(H) to the high-level policy module and construct an optimization objective function

[0084]

[0085] Among them, λ1, λ2, and λ3 are weight parameters, which are used to balance access matching degree, cache hit rate, and resource consumption respectively;

[0086] S105, according to the optimization objective function Generate a new permission policy matrix P ′ =[p ′ ij ] and the cache strategy matrix Q ′ =[q ′ lm ], where p ′ ij Indicates the adjusted user group permission status, q ′ lm Indicates the new cache strategy;

[0087] S106, passing the new permission strategy and cache strategy to the lower-level operation module for application, so that the system performs permission management and cache adjustment according to the optimized strategy in the next cycle;

[0088] S107, define the convergence condition ∈ of the strategy update, determine whether the optimization meets the convergence standard, and if the convergence condition is met, confirm that the strategy optimization is completed; otherwise, continue the optimization iteration until the convergence condition is met;

[0089] S108. In the next evaluation cycle, the system repeats the evaluation and optimization process of S101-S107 according to the new historical access data and cache hit rate data, so that the permission strategy and cache strategy continue to maintain the best state.

[0090] The beneficial effects of the present invention are:

[0091] (1) The present invention adopts a hierarchical reinforcement learning model and realizes intelligent optimization of permission and resource allocation through the collaborative work of high-level policy modules and low-level operation modules. The high-level policy module manages the initial permission allocation of user groups, and the low-level operation module is responsible for the permission control of specific files or data blocks and the adjustment of dynamic cache strategies. Compared with traditional static permission configuration, the present invention dynamically analyzes user behavior data through reinforcement learning algorithms and generates permission strategies that are adjusted in real time. The hierarchical reinforcement learning model can dynamically optimize permission and resource configuration according to user access patterns and system loads without the need for administrator intervention, ensuring that the system maintains high efficiency and stability under high load conditions.

[0092] (2) The present invention realizes adaptive cache strategy optimization by constructing a time-weighted access matrix and a cache hit rate evaluation function. The system prioritizes retaining high-frequency access data blocks in the cache based on the user's historical access data and resource usage, and intelligently recycles low-frequency data blocks when cache resources are insufficient. Compared with the fixed rule-based strategy in traditional cache management, the reinforcement learning model of the present invention can dynamically adjust the allocation of cache resources and realize iterative update of the cache strategy through a policy gradient optimization algorithm. The adaptive cache management method significantly improves the system's response speed and resource utilization efficiency, ensuring the provision of high-quality services under different load conditions.

[0093] (3) The present invention designs a real-time permission reduction and recovery mechanism. When the system detects abnormal access behavior, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group and prevent unauthorized access and resource abuse. The permission recovery time is set by the recovery strategy function. If no new abnormal behavior is found within the specified time, part or all of the user's permissions are automatically restored. While improving the security of the system, it avoids accidental harm to legitimate users and improves the fault tolerance of the system. Compared with traditional manual management methods, the reduction and recovery mechanism of the present invention has higher flexibility and accuracy, and significantly reduces the security risks of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0095] Figure 1 A flowchart of a method for intelligent management of cloud storage access rights based on reinforcement learning proposed by the present invention;

[0096] Figure 2 This is a structural schematic diagram of a hierarchical reinforcement learning model in a cloud storage access rights intelligent management method based on reinforcement learning proposed by the present invention. DETAILED DESCRIPTION

[0097] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0098] refer to Figure 1-Figure 2 , a cloud storage access rights intelligent management method based on reinforcement learning, comprising the following steps:

[0099] S1. Construct a hierarchical reinforcement learning model. The hierarchical reinforcement learning model includes a high-level policy module and a low-level operation module. The high-level policy module is responsible for the management of user groups and the initial allocation of resource permissions. The low-level operation module is used for access permission control of specific files or data blocks and dynamic cache strategy adjustment.

[0100] S2, collecting historical access data of users in the cloud storage system, and inputting the historical access data into the high-level policy module to generate an initial user group permission policy;

[0101] S3. Configure initial permissions and resource allocation according to the user group permission policy, and apply the configuration results to the low-level operation module so that each user group has the minimum access rights required to complete its role tasks when accessing for the first time;

[0102] S4. When a user requests to access cloud storage resources for the first time, the low-level operation module calls the cache management system to load the accessed data blocks or files into the cache area, and records the specific behavior data of the access and feeds it back to the hierarchical reinforcement learning model;

[0103] S5, the hierarchical reinforcement learning model dynamically adjusts the cache strategy based on the data access frequency in the cache, the usage pattern of the user group, and the historical behavior pattern, so that the files or data blocks with high frequency access are preferentially retained in the cache;

[0104] S6. The hierarchical reinforcement learning model optimizes the permission strategy through a real-time reward mechanism. It gives positive rewards and adjusts user permissions for access behaviors that comply with system security specifications; it gives negative rewards for abnormal behaviors and triggers an automatic permission reduction mechanism.

[0105] S7. When the system detects that the cache area resources are insufficient, the hierarchical reinforcement learning model dynamically recycles the data blocks or files with low frequency of access based on the current user's access priority, cache resource occupancy and system load status;

[0106] S8. When the system load increases or user access requests are frequent, the high-level policy module re-evaluates the access requirements of each user group and passes the adjusted user group permission policy to the low-level operation module to perform synchronous optimization of access rights and cache policies;

[0107] S9. When abnormal behaviors occur frequently, the system triggers a secondary verification process based on the feedback from the hierarchical reinforcement learning model, requiring the user to perform additional identity verification and temporarily freezing their high-risk permissions;

[0108] S10. Regularly evaluate and optimize permission policies and cache policies, feed back historical access data and cache hit rates to high-level policy modules, and generate new permission policies and cache policies.

[0109] In this implementation, S1 includes:

[0110] S11. Construct a hierarchical reinforcement learning model, including a high-level policy module and a low-level operation module; the high-level policy module is used to manage the initial allocation of user groups and resource permissions, and the low-level operation module is used to control access permissions for specific files or data blocks and adjust dynamic cache strategies;

[0111] S12. In the high-level strategy module, define the state space in represents the status of the i-th user group, u i is the feature vector of the user group, including roles and permission levels, r i is the resource access record, f i To collect visit frequency statistics;

[0112] S13. In the high-level strategy module, define the action space in Indicates the action of assigning permissions to user groups, assigning permission set P j Access rights for specific resources;

[0113] S14. In the high-level strategy module, establish a reward function:

[0114]

[0115] Among them, E s =Σ k δ(p k )·ω k represents the system security index, δ(p k ) is the permission p k The risk indicator function, ω k is the risk weight of the system security indicator authority, E r =Σ k ψ(p k )·c k represents the resource utilization index, ψ(p k ) is the permission p k The resource usage of k is the resource consumption cost, E u =Σ k φ(p k )·μ k , represents the user satisfaction index, φ(p k ) is the permission p k Contribution to user task completion, μ k For the user to have permissions p k The demand intensity, λ1, λ2, λ3 are adjustment parameters used to balance system security, resource utilization and user satisfaction;

[0116] S15. In the low-level operation module, define the state space in Indicates the status of the lth data block or file, d l is the data feature vector, including size and type, c l The cache status is 1, which means it is in the cache, or 0, which means it is not in the cache. l is the frequency of visits;

[0117] S16. In the low-level operation module, define the action space in Cache operations performed on data blocks or files, including:

[0118] aload To load a data block into the cache;

[0119] a retain To keep data blocks in cache;

[0120] a replace To replace the data block in the cache with a new data block;

[0121] a evict To remove a data block from the cache;

[0122] S17. In the low-level operation module, establish a reward function:

[0123]

[0124] in, represents the improvement in cache hit rate, Δh is the increase in cache hit times, and h total is the total number of visits, C c =Σ l c l ·s l Indicates cache resource consumption, s l is the size of data block l, γ1 and γ2 are adjustment parameters used to balance the improvement of cache hit rate and cache resource consumption;

[0125] S18. Establish a collaborative strategy between the high-level strategy module and the low-level operation module through a hierarchical strategy function:

[0126] π(s,a)=π H (s H ,a H )·π L (s L ,a L |s H ,a H );

[0127] Where s=(s H ,s L ) is the joint state, a=(a H ,a L ) is a joint action, π H (s H ,a H ) is a high-level policy that determines the permission allocation of user groups, π L (s L ,a L |s H ,a H ) is a low-level strategy, which selects a cache operation based on the output of the high-level strategy and the current data block status;

[0128] S19. Use reinforcement learning algorithm to optimize high-level strategy modules and low-level operation modules to maximize the overall expected cumulative reward:

[0129]

[0130] Among them, θ H ,θ L are the parameters of the high-level strategy module and the low-level operation module, and γ is the discount factor used to balance short-term and long-term benefits;

[0131] Update the policy parameters through the policy gradient method:

[0132]

[0133] Among them, α H ,α L is the learning rate of the high-level strategy module and the low-level operation module, and They are the gradients of the parameters of the high-level strategy module and the low-level operation module, respectively, guiding the optimization direction of the strategy.

[0134] In this implementation, S2 includes:

[0135] S21. Collecting historical access data of users in the cloud storage system, where the historical access data includes access time, access frequency, role information, and operation type;

[0136] S22. Constructing access record matrix Where m ij Represents user u i In time period t j The number of access behaviors in a period, n is the number of users, t is the number of time periods, and each element in the matrix reflects the access frequency of the user in a specific time period;

[0137] S23. Define the user role set U, each user u i With the eigenvector x i =(r i ,o i ), r i Indicates the user's role type, o i Represents the set of common operation types of the user; the feature vector is used to describe the access characteristics of the user group;

[0138] S26. Generate a time-weighted access matrix W = [w ij ]:

[0139] w ij =α·f ij ·exp(-β·Δt j );

[0140] Among them, f ij For user u i For operation j The access frequency, Δt j is the time interval between the operation and the current time, α and β are adjustment parameters for balancing historical access data and recent data;

[0141] S27, historical access data matrix M, user feature vector x i The time-weighted access matrix W is input into the high-level policy module to determine the optimal initial permission policy for the user group by maximizing the product of access frequency and time weighting:

[0142]

[0143] Where P = [p ij ] indicates user u i For operation j The initial authority matrix, p ij =1 means granting permission, p ij =0 means no permission is granted;

[0144] S28, according to the user group authority policy, divide the users into multiple authority level groups G = {g1, g2, ..., g m}, each group g i Include several users and match appropriate resource permissions based on their historical access data;

[0145] S29. Apply the generated initial permission strategy and group division results to the permission management process and serve as the initial configuration of the high-level policy module to provide a basis for the specific permission allocation and cache strategy adjustment of the low-level operation module.

[0146] In this implementation, S3 includes:

[0147] S31, according to the user group authority policy, the user group authority matrix P group Each element p in ij With user group u i For operation j The initial permission match of ij =1 means granting permission, p ij =0 means no permission is granted;

[0148] S32. Define resource allocation vector R i Represented as user group u i the mix of resources allocated;

[0149] S33, map permissions and resources, and establish a mapping function φ(u i ,oj ) means when user group u i Granted Operation o j When the user has the permission, the system allocates the corresponding resource combination R according to the user's needs. i :

[0150]

[0151] Among them, (u i ,o j ) is the user group u i Execute operation j The required resource combination, ω j For operation j The weight of R indicates its importance in the system. j For operation o j The total amount of resource type allocated, μ ij For user group u i For operation j The demand intensity is calculated based on the historical access frequency, T max is the maximum effective time window of the permission, T ij For operation j The remaining time of the corresponding permission;

[0152] S34. Generate an initial authority-resource allocation matrix according to the mapping function:

[0153] Q=[q ij ];

[0154] Among them, q ij =1 indicates user group u i Operation obtained j The minimum resource support required, q ij =0 means no resources are allocated;

[0155] S35, passing the generated initial authority-resource allocation matrix Q to the lower-level operation module, so that the lower-level operation module matches the user's authority and performs corresponding resource allocation when the user requests for the first time;

[0156] S36. When a user first accesses the cloud storage system, the low-level operation module calls the cache management system to load the required data blocks into the cache area according to the initial permission-resource allocation matrix Q, and allocates computing resources and network bandwidth for the operation of the user group;

[0157] S37, set the timeliness constraint of the operation for each user group and define the timeliness function τ(u i ,o j ), when the permission expires or the user task is completed, the low-level operation module i,o j ) automatically releases the allocated resources and updates the permission status of the user group based on the access records.

[0158] In this implementation, S6 includes:

[0159] S61. Based on the access behavior of the user group, the reward value of the current behavior is calculated according to the reward function of S2. Behaviors that comply with the system security specifications will receive positive rewards, and abnormal behaviors will receive negative rewards.

[0160] S62. When an access behavior that complies with security specifications is detected, the system expands the user's authority scope according to the reward value, allowing the user to access more resources;

[0161] S63. When abnormal behavior is detected, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group;

[0162] S64, defining a recovery strategy function ρ(t) for the user group whose rights are triggered to be reduced, where ρ(t) indicates that some or all rights are restored after time t; if no new abnormal behavior is found within the specified time, the rights will be automatically restored;

[0163] S65. Feedback the permission adjustment result to the high-level policy module to update the permission policy of the user group, and use it as a reference in the next permission evaluation.

[0164] In this implementation, S10 includes:

[0165] S101. Periodically collect historical access data and cache hit rate data in the system to construct an evaluation data set:

[0166] D = (H, C);

[0167] Where H = [h ij ] indicates user group u i For resources j The access frequency matrix, C = [c lm ] indicates data block d l The hit status in the cache, c lm =1 means hit, c lm =0 means miss;

[0168] S102: Define a cache evaluation function η(C) according to the cache hit rate data C to calculate the cache hit rate:

[0169]

[0170] Among them, L is the number of data blocks, M is the number of access requests, and the cache hit rate η(C) is used to evaluate the execution effect of the cache strategy;

[0171] S103. According to the user's historical access data H, define the permission evaluation index ζ(H) to calculate the matching degree of resource access:

[0172]

[0173] Where n is the number of user groups, k is the number of resources, and p ij For user group u i For resources j The permission status of the,matching degree ζ(H) indicates the consistency between the actual access and the,permission allocation;

[0174] S104: Feedback the cache evaluation result η(C) and the authority evaluation result ζ(H) to the high-level policy module and construct an optimization objective function

[0175]

[0176] Among them, λ1, λ2, and λ3 are weight parameters, which are used to balance access matching degree, cache hit rate, and resource consumption respectively;

[0177] S105, according to the optimization objective function Generate a new permission policy matrix P ′ =[p ′ ij ] and the cache strategy matrix Q ′ =[q ′ lm ], where p ′ ij Indicates the adjusted user group permission status, q ′ lm Indicates the new cache strategy;

[0178] S106, passing the new permission strategy and cache strategy to the lower-level operation module for application, so that the system performs permission management and cache adjustment according to the optimized strategy in the next cycle;

[0179] S107, define the convergence condition ∈ of the strategy update, determine whether the optimization meets the convergence standard, and if the convergence condition is met, confirm that the strategy optimization is completed; otherwise, continue the optimization iteration until the convergence condition is met;

[0180] S108. In the next evaluation cycle, the system repeats the evaluation and optimization process of S101-S107 according to the new historical access data and cache hit rate data, so that the permission strategy and cache strategy continue to maintain the best state.

[0181] Embodiment 1:

[0182] In order to verify the feasibility and effectiveness of the present invention in the intelligent management of cloud storage access rights, the experimenters used two publicly available data sets in the experiment: Amazon Access Logs and Google Cluster Data. Among them, the Amazon Access Logs data set contains the access logs of different users to commodity resources on the e-commerce platform, covering the access behavior from January 2023 to June 2023; the Google Cluster Data data set records the dynamic requests and access data of users to computing resources in the multi-tenant cloud computing platform, and the data time range is from November 2022 to April 2023.

[0183] These two data sets provide real access scenario data for the training and testing of the present invention. The experimenters divided the two data sets into training sets and test sets in a ratio of 8:2 and conducted comparative tests in different permission management scenarios to verify the performance of the present invention in optimizing system response speed, resource utilization and security.

[0184] The experimenters extracted the frequency and access time of users visiting product pages and the access operation types from the Amazon Access Logs dataset and extracted the frequency of users’ resource requests, tenant roles, and requested resource types from the Google Cluster Data. The experimenters integrated the two sets of data to train the reinforcement learning model. The specific data dimensions are as follows:

[0185] User group feature vector: role, access frequency, operation type (such as read, write, modify).

[0186] Resource request characteristics: resource type, amount of resources occupied, and request response time.

[0187] Time window: records the validity period of user permissions and is used for permission reduction and recovery mechanisms.

[0188] In order to achieve standardized data processing, the experimenters used Min-Max scaling to normalize the access frequency and resource consumption to the [0,1] range, and used the policy gradient algorithm in reinforcement learning to optimize permissions and resource allocation.

[0189] In model training, the experimenters used the permission management data of Amazon Access Logs as the initial training samples, and the resource request data of Google Cluster Data was used to simulate the permission contraction and recovery mechanism under high-load scenarios. The training process used an adaptive moment estimation optimizer, with a total of 200 rounds of training, a learning rate of 0.001, and a small batch of 64. After each training, the system dynamically updates the permission strategy and cache strategy based on the user's behavior.

[0190] The experimenters compared the traditional static permission configuration method with the method of the present invention and used the following evaluation indicators on the two data sets:

[0191] Permission application reduction rate: Compare the number of permission applications by users in the system.

[0192] Average response time: The average response speed of access requests.

[0193] Resource utilization: Average utilization of cache and computing resources.

[0194] Number of data breach incidents: The number of privilege abuse and data breach incidents detected.

[0195] Table 1 below shows the comparison results between the method of the present invention and the traditional method on the Amazon Access Logs dataset:

[0196] Table 1 Comparison results on the Amazon Access Logs dataset

[0197] index Method of the present invention Traditional methods Average number of permission requests 1.5 times / week 7 times / week Average response time (seconds) 0.3 seconds 1.8 seconds Resource Utilization 87％ 58％ Data breach incidents 0 times 4 times / month

[0198] On the Google Cluster Data dataset, the method of the present invention is shown in Table 2 below:

[0199] Table 2 Comparison results on Google Cluster Data dataset

[0200] index Method of the present invention Traditional methods Average number of permission requests 2 times / week 9 times / week Average response time (seconds) 0.4 seconds 2.1 seconds Resource Utilization 90％ 62％ Data breach incidents 0 times 3 times / month

[0201] Through comparative tests, it can be seen that the reinforcement learning model of the present invention significantly reduces the number of user permission applications, with an average weekly reduction of 78.6% in the Amazon Access Logs data set and 77.8% in the Google Cluster Data data set. In addition, the method of the present invention greatly improves the resource utilization of the system, shortens the average response time by more than 80%, and effectively solves the problem of too frequent permission configuration and slow response speed in traditional methods.

[0202] The permission reduction and recovery mechanism of the present invention played an important role in high load and abnormal behavior detection. No data leakage incidents were detected during the test, while the traditional static configuration method detected at least 3 leakage incidents per month. The resource utilization rate of the system was also significantly improved, reaching 90% in the Google Cluster Data test, which is much higher than the 62% of the traditional method. The results prove the feasibility and effectiveness of the present invention in cloud storage systems, and provide a flexible, secure and efficient solution for complex and changeable access scenarios.

[0203] This Example 1 demonstrates the application process of the present invention in a real data set, verifying the excellent performance of the present invention in improving system response speed, optimizing resource utilization and enhancing system security. Through experiments and comparisons on two public data sets, the experimenters proved that the method of the present invention can not only cope with complex permission management requirements, but also dynamically adjust permissions and resource configuration under different load scenarios, significantly improving the management efficiency and security of cloud storage systems.

[0204] The present invention adopts a hierarchical reinforcement learning model, and realizes intelligent optimization of permission and resource allocation through the collaborative work of high-level policy modules and low-level operation modules. The high-level policy module manages the initial permission allocation of user groups, and the low-level operation module is responsible for the permission control of specific files or data blocks and the adjustment of dynamic cache strategies. Compared with traditional static permission configuration, the present invention dynamically analyzes user behavior data through reinforcement learning algorithms and generates real-time adjusted permission strategies. The hierarchical reinforcement learning model can dynamically optimize permission and resource configuration according to user access patterns and system loads without the need for administrator intervention, ensuring the high efficiency and stability of the system under high load conditions.

[0205] The present invention realizes adaptive cache strategy optimization by constructing a time-weighted access matrix and a cache hit rate evaluation function. The system prioritizes retaining high-frequency access data blocks in the cache based on the user's historical access data and resource usage, and intelligently recycles low-frequency data blocks when cache resources are insufficient. Compared with the fixed rule-based strategy in traditional cache management, the reinforcement learning model of the present invention can dynamically adjust the allocation of cache resources and realize iterative update of the cache strategy through a policy gradient optimization algorithm. The adaptive cache management method significantly improves the system's response speed and resource utilization efficiency, ensuring high-quality services under different load conditions.

[0206] The present invention designs a real-time permission reduction and recovery mechanism. When the system detects abnormal access behavior, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group and prevent unauthorized access and resource abuse. The permission recovery time is set by the recovery strategy function. When no new abnormal behavior is found within the specified time, part or all of the user's permissions are automatically restored. While improving the security of the system, accidental harm to legitimate users is avoided, and the fault tolerance of the system is improved. Compared with traditional manual management methods, the reduction and recovery mechanism of the present invention has higher flexibility and accuracy, and significantly reduces the security risks of the system.

[0207] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A cloud storage access rights intelligent management method based on reinforcement learning, characterized in that: The steps include: S1. Construct a hierarchical reinforcement learning model. The hierarchical reinforcement learning model includes a high-level policy module and a low-level operation module. The high-level policy module is responsible for the management of user groups and the initial allocation of resource permissions. The low-level operation module is used for access permission control of specific files or data blocks and dynamic cache strategy adjustment. S2, collecting historical access data of users in the cloud storage system, and inputting the historical access data into the high-level policy module to generate an initial user group permission policy; S3. Configure initial permissions and resource allocation according to the user group permission policy, and apply the configuration results to the low-level operation module so that each user group has the minimum access rights required to complete its role tasks when accessing for the first time; S4. When a user requests to access cloud storage resources for the first time, the low-level operation module calls the cache management system to load the accessed data blocks or files into the cache area, and records the specific behavior data of the access and feeds it back to the hierarchical reinforcement learning model; S5, the hierarchical reinforcement learning model dynamically adjusts the cache strategy based on the data access frequency in the cache, the usage pattern of the user group, and the historical behavior pattern, so that the files or data blocks with high frequency access are preferentially retained in the cache; S6. The hierarchical reinforcement learning model optimizes the permission strategy through a real-time reward mechanism. It gives positive rewards and adjusts user permissions for access behaviors that comply with system security specifications; it gives negative rewards for abnormal behaviors and triggers an automatic permission reduction mechanism. S7. When the system detects that the cache area resources are insufficient, the hierarchical reinforcement learning model dynamically recycles the data blocks or files with low frequency of access based on the current user's access priority, cache resource occupancy and system load status; S8. When the system load increases or user access requests are frequent, the high-level policy module re-evaluates the access requirements of each user group and passes the adjusted user group permission policy to the low-level operation module to perform synchronous optimization of access rights and cache policies; S9. When abnormal behaviors occur frequently, the system triggers a secondary verification process based on the feedback from the hierarchical reinforcement learning model, requiring the user to perform additional identity verification and temporarily freezing their high-risk permissions; S10. Regularly evaluate and optimize permission policies and cache policies, feed back historical access data and cache hit rates to high-level policy modules, and generate new permission policies and cache policies.

2. According to claim 1, a cloud storage access rights intelligent management method based on reinforcement learning is characterized in that: The S1 includes: S11. Construct a hierarchical reinforcement learning model, including a high-level policy module and a low-level operation module; the high-level policy module is used to manage the initial allocation of user groups and resource permissions, and the low-level operation module is used to control access permissions for specific files or data blocks and adjust dynamic cache strategies; S12. In the high-level strategy module, define the state space in represents the status of the i-th user group, u i is the feature vector of the user group, including roles and permission levels, r i is the resource access record, f i To collect visit frequency statistics; S13. In the high-level strategy module, define the action space in Indicates the action of assigning permissions to user groups, assigning permission set P j Access rights for specific resources; S14. In the high-level strategy module, establish a reward function: Among them, E s =∑ k δ(p k )·ω k represents the system security index, δ(p k ) is the permission p k The risk indicator function, ω k is the risk weight of the system security indicator authority, E r =∑ k ψ(p k )·c k represents the resource utilization index, ψ(p k ) is the permission p k The resource usage of k is the resource consumption cost, E u =∑ k φ(p k )·μ k , represents the user satisfaction index, φ(p k ) is the permission p k Contribution to user task completion, μ k For the user to have permissions p k The demand intensity, λ1, λ2, λ3 are adjustment parameters used to balance system security, resource utilization and user satisfaction; S15. In the low-level operation module, define the state space in Indicates the status of the lth data block or file, d l is the data feature vector, including size and type, c l The cache status is 1, which means it is in the cache, or 0, which means it is not in the cache. l is the frequency of visits; S16. In the low-level operation module, define the action space in Cache operations performed on data blocks or files, including: a load To load a data block into the cache; a retain To keep data blocks in cache; a replace To replace the data block in the cache with a new data block; a evict To remove a data block from the cache; S17. In the low-level operation module, establish a reward function: in, represents the improvement in cache hit rate, Δh is the increase in cache hit times, and h total is the total number of visits, C c =∑ l c l ·s l Indicates cache resource consumption, s l is the size of data block l, γ1 and γ2 are adjustment parameters used to balance the improvement of cache hit rate and cache resource consumption; S18. Establish a collaborative strategy between the high-level strategy module and the low-level operation module through a hierarchical strategy function: π(s,a)=π H (s H ,a H )·p L (s L ,a L |s H ,a H ); Where s=(s H ,s L ) is the joint state, a=(a H ,a L ) is a joint action, π H (s H ,a H ) is a high-level policy that determines the permission allocation of user groups, π L (s L ,a L |s H ,a H ) is a low-level strategy, which selects a cache operation based on the output of the high-level strategy and the current data block status; S19. Use reinforcement learning algorithm to optimize high-level strategy modules and low-level operation modules to maximize the overall expected cumulative reward: Among them, θ H ,θ L are the parameters of the high-level strategy module and the low-level operation module, and γ is the discount factor used to balance short-term and long-term benefits; Update the policy parameters through the policy gradient method: Among them, α H ,α L is the learning rate of the high-level strategy module and the low-level operation module, and They are the gradients of the parameters of the high-level strategy module and the low-level operation module, respectively, guiding the optimization direction of the strategy.

3. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: The S2 includes: S21. Collecting historical access data of users in the cloud storage system, where the historical access data includes access time, access frequency, role information, and operation type; S22. Constructing access record matrix Where m ij Represents user u i In time period t j The number of access behaviors in a period, n is the number of users, t is the number of time periods, and each element in the matrix reflects the access frequency of the user in a specific time period; S23. Define the user role set U, each user u i With the eigenvector x i =(r i ,o i ), r i Indicates the user's role type, o i Represents the set of regular operation types of the user; the feature vector is used to describe the access characteristics of the user group; S26. Generate a time-weighted access matrix W = [w ij ]: w ij =α·f ij ·exp(-β·Δt j ); Among them, f ij For user u i For operation j The access frequency, Δt j is the time interval between the operation and the current time, α and β are adjustment parameters for balancing historical access data and recent data; S27, historical access data matrix M, user feature vector x i The time-weighted access matrix W is input into the high-level policy module to determine the optimal initial permission policy for the user group by maximizing the product of access frequency and time weighting: Where P = [p ij ] indicates user u i For operation j The initial authority matrix, p ij =1 means granting permission, p ij =0 means no permission is granted; S28, according to the user group authority policy, divide the users into multiple authority level groups G = {g1, g2, ..., g m }, each group g i Include several users and match appropriate resource permissions based on their historical access data; S29. Apply the generated initial permission strategy and group division results to the permission management process and serve as the initial configuration of the high-level policy module to provide a basis for the specific permission allocation and cache strategy adjustment of the low-level operation module.

4. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: The S3 includes: S31, according to the user group authority policy, the user group authority matrix P group Each element p in ij With user group u i For operation j The initial permission match of ij =1 means granting permission, p ij =0 means no permission is granted; S32. Define resource allocation vector R i Represented as user group u i the mix of resources allocated; S33, map permissions and resources, and establish a mapping function φ(u i ,o j ) means when user group u i Granted Operation o j When the user has the permission, the system allocates the corresponding resource combination R according to the user's needs. i : Among them, (u i ,o j ) is the user group u i Execute operation j The required resource combination, ω j For operation o j The weight of R indicates its importance in the system. j For operation o j The total amount of resource type allocated, μ ij For user group u i For operation j The demand intensity is calculated based on the historical access frequency, T max is the maximum effective time window of the permission, T ij For operation o j The remaining time of the corresponding permission; S34. Generate an initial authority-resource allocation matrix according to the mapping function: Q=[q ij ]; Among them, q ij =1 indicates user group u i Operation obtained j The minimum resource support required, q ij =0 means no resources are allocated; S35, passing the generated initial authority-resource allocation matrix Q to the lower-level operation module, so that the lower-level operation module matches the user's authority and performs corresponding resource allocation when the user requests for the first time; S36. When a user first accesses the cloud storage system, the low-level operation module calls the cache management system to load the required data blocks into the cache area according to the initial permission-resource allocation matrix Q, and allocates computing resources and network bandwidth for the operation of the user group; S37, set the timeliness constraint of the operation for each user group and define the timeliness function τ(u i ,o j ), when the permission expires or the user task is completed, the low-level operation module i ,o j ) automatically releases the allocated resources and updates the permission status of the user group based on the access records.

5. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: The S6 includes: S61. Based on the access behavior of the user group, the reward value of the current behavior is calculated according to the reward function of S2. Behaviors that comply with the system security specifications will receive positive rewards, and abnormal behaviors will receive negative rewards. S62. When an access behavior that complies with security specifications is detected, the system expands the user's authority scope according to the reward value, allowing the user to access more resources; S63. When abnormal behavior is detected, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group; S64, defining a recovery strategy function ρ(t) for the user group whose rights are triggered to be reduced, where ρ(t) indicates that some or all rights are restored after time t; if no new abnormal behavior is found within the specified time, the rights will be automatically restored; S65. Feedback the permission adjustment result to the high-level policy module to update the permission policy of the user group, and use it as a reference in the next permission evaluation.

6. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: The S10 includes: S101. Periodically collect historical access data and cache hit rate data in the system to construct an evaluation data set: D = (H, C); Where H = [h ij ] indicates user group u i For resources j The access frequency matrix, C = [c lm ] indicates data block d l The hit status in the cache, c lm =1 means hit, c lm =0 means miss; S102: Define a cache evaluation function η(C) according to the cache hit rate data C to calculate the cache hit rate: Among them, L is the number of data blocks, M is the number of access requests, and the cache hit rate η(C) is used to evaluate the execution effect of the cache strategy; S103. According to the user's historical access data H, define the permission evaluation index ζ(H) to calculate the matching degree of resource access: Where n is the number of user groups, k is the number of resources, and p ij For user group u i For resources j The permission status of the,matching degree ζ(H) indicates the consistency between the actual access and the,permission allocation; S104: Feedback the cache evaluation result η(C) and the authority evaluation result ζ(H) to the high-level policy module and construct an optimization objective function Among them, λ1, λ2, and λ3 are weight parameters, which are used to balance access matching degree, cache hit rate, and resource consumption respectively; S105, according to the optimization objective function Generate a new permission policy matrix P′=[p′ ij ] and cache strategy matrix Q′=[q′ lm ], where p′ ij Indicates the adjusted user group permission status, q′ lm Indicates the new cache strategy; S106, passing the new permission strategy and cache strategy to the lower-level operation module for application, so that the system performs permission management and cache adjustment according to the optimized strategy in the next cycle; S107, define the convergence condition ∈ of the strategy update, determine whether the optimization meets the convergence standard, and if the convergence condition is met, confirm that the strategy optimization is completed; otherwise, continue the optimization iteration until the convergence condition is met; S108. In the next evaluation cycle, the system repeats the evaluation and optimization process of S101-S107 according to the new historical access data and cache hit rate data, so that the permission strategy and cache strategy continue to maintain the best state.

Images