Intelligent management method of cloud storage access rights based on reinforcement learning

Abstract

The present invention discloses a cloud storage access rights intelligent management method based on reinforcement learning, comprising the following steps: S1, building a hierarchical reinforcement learning model; S2, collecting historical access data of users in the cloud storage system; S3, configuring initial permissions and resource allocation according to the user group permission policy, and applying the configuration results to the low-level operation module; S4, feeding back to the hierarchical reinforcement learning model; S5, preferentially retaining frequently accessed files or data blocks in the cache; S6, the hierarchical reinforcement learning model optimizes the permission policy through a real-time reward mechanism; S7, dynamically recovering data blocks or files with low frequency of access; S8, performing synchronous optimization of access rights and caching policies; S9, temporarily freezing its high-risk permissions; and S10, generating new permission policies and caching policies. The present invention improves the response speed and resource utilization efficiency of the system, and enhances the security and fault tolerance of the system.

Description

Technical Field

[0001] The present invention relates to the field of cloud storage technology, and in particular to a method for intelligent management of cloud storage access rights based on reinforcement learning. Background Art

[0002] With the rapid development of cloud computing and big data technologies, more and more enterprises and individual users are migrating their data storage and management to the cloud. Cloud storage systems provide users with convenient storage services with their efficiency, scalability, and flexibility. However, with the growth of the number of users in cloud environments and the complexity of their access needs, traditional permission management and resource scheduling methods have gradually exposed significant problems and limitations.

[0003] In existing cloud storage systems, access permissions are usually managed through pre-defined static policies. Administrators manually assign access permissions based on user roles or functions and configure corresponding resources for different user groups. The static permission management model can initially meet the needs of some fixed scenarios. However, with the dynamic changes in user access behavior and the increasing demand for real-time scheduling of cloud resources, traditional static configurations are gradually unable to cope with complex and changing access scenarios. In order to cope with changes in user needs, administrators need to frequently update permissions and resource configurations, resulting in increased management costs and error rates.

[0004] To improve management efficiency, some cloud storage systems have introduced rule-based automated management methods that trigger permission changes and resource allocation through pre-set conditions and rules. However, rule-based management methods lack flexibility and adaptability, making it difficult to automatically adjust permissions based on users' real-time behavior. In addition, in existing permission management technologies, administrators often tend to grant excessive authorization beyond actual needs in order to reduce users' frequent permission applications, increasing system security risks.

[0005] In summary, existing technologies are clearly insufficient in addressing dynamic user behavior, enabling flexible permission management, and preventing over-authorization, making them difficult to meet the requirements for efficient and secure management of cloud storage systems. Therefore, a new solution is urgently needed that can automatically optimize permissions and resource allocation based on real-time user behavior and access history, achieving smarter and more secure cloud storage permission management. Summary of the Invention

[0006] One purpose of the present invention is to propose a method for intelligent management of cloud storage access rights based on reinforcement learning. The present invention improves the system's response speed and resource utilization efficiency, and enhances the system's security and fault tolerance.

[0007] According to an embodiment of the present invention, a method for intelligently managing cloud storage access rights based on reinforcement learning includes the following steps:

[0008] S1. Build a hierarchical reinforcement learning model. The hierarchical reinforcement learning model includes a high-level policy module and a low-level operation module. The high-level policy module is responsible for user group management and initial allocation of resource permissions, while the low-level operation module is used to control access permissions for specific files or data blocks and adjust dynamic cache policies.

[0009] S2. Collect historical access data of users in the cloud storage system and input the historical access data into the high-level policy module to generate the initial user group permission policy;

[0010] S3. Configure initial permissions and resource allocation based on the user group permission policy, and apply the configuration results to the low-level operation module so that each user group has the minimum access rights required to complete their role tasks upon first access.

[0011] S4. When a user requests to access cloud storage resources for the first time, the low-level operation module calls the cache management system to load the accessed data block or file into the cache area, and records the specific behavior data of the access and feeds it back to the hierarchical reinforcement learning model;

[0012] S5. The hierarchical reinforcement learning model dynamically adjusts the cache strategy based on the data access frequency in the cache, the usage patterns of user groups, and historical behavior patterns, so that frequently accessed files or data blocks are retained in the cache first;

[0013] S6. The hierarchical reinforcement learning model optimizes permission policies through a real-time reward mechanism. It provides positive rewards and adjusts user permissions for access behaviors that comply with system security specifications; it provides negative rewards for abnormal behaviors and triggers an automatic permission reduction mechanism.

[0014] S7. When the system detects insufficient cache area resources, the hierarchical reinforcement learning model dynamically reclaims infrequently accessed data blocks or files based on the current user's access priority, cache resource usage, and system load status;

[0015] S8. When the system load increases or user access requests become frequent, the high-level policy module re-evaluates the access requirements of each user group and passes the adjusted user group permission policy to the low-level operation module to simultaneously optimize the access rights and cache policy.

[0016] S9. When abnormal behavior occurs frequently, the system triggers a secondary verification process based on feedback from the hierarchical reinforcement learning model, requiring the user to undergo additional identity verification and temporarily freezing their high-risk permissions.

[0017] S10. Regularly evaluate and optimize permission policies and cache policies, feed historical access data and cache hit rates back to high-level policy modules, and generate new permission policies and cache policies.

[0018] Optionally, the S1 includes:

[0019] S11. Build a hierarchical reinforcement learning model, including a high-level policy module and a low-level operation module; the high-level policy module is used to manage the initial allocation of user groups and resource permissions, and the low-level operation module is used to control access permissions for specific files or data blocks and dynamically adjust cache policies;

[0020] S12. In the high-level strategy module, define the state space in Indicates the status of the i-th user group, u i is the feature vector of the user group, including roles and permission levels, r i For resource access records, f i To collect visit frequency statistics;

[0021] S13. In the high-level strategy module, define the action space in Indicates the action of assigning permissions to user groups, assigning permission set P j Access rights to specific resources;

[0022] S14. In the high-level strategy module, establish the reward function:

[0023]

[0024] Among them, E s =∑ k δ(p k )·ω k represents the system security index, δ(p k ) is the permission p k The risk indicator function, ω k is the risk weight of the system security indicator authority, E r =∑ k ψ(p k )·c k represents the resource utilization index, ψ(p k ) is the permission p k The resource usage of k is the resource consumption cost, E u =∑ k φ(p k )·μ k , represents the user satisfaction index, φ(p k ) is the permission p k Contribution to user task completion, μ k For the user to have permission p k The demand intensity of λ1, λ2, and λ3 are adjustment parameters used to balance system security, resource utilization, and user satisfaction.

[0025] S15. In the low-level operation module, define the state space in Indicates the status of the lth data block or file, d l is the data feature vector, including size and type, c l The cache status is 1, which means it is in the cache, or 0, which means it is not in the cache. l is the frequency of visits;

[0026] S16. In the low-level operation module, define the action space in Cache operations performed on data blocks or files, including:

[0027] a load To load the data block into the cache;

[0028] a retain To keep data blocks in cache;

[0029] a replace To replace the data block in the cache with a new data block;

[0030] a evict To remove a data block from the cache;

[0031] S17. In the low-level operation module, establish the reward function:

[0032]

[0033] in, represents the improvement of cache hit rate, Δh is the increase of cache hit times, h total is the total number of visits, C c =∑ l c l ·s l Indicates cache resource consumption, s l is the size of data block l, γ1 and γ2 are adjustment parameters used to balance the improvement of cache hit rate and cache resource consumption;

[0034] S18. Establish a collaborative strategy between the high-level strategy module and the low-level operation module through a hierarchical strategy function:

[0035] π(s,a)=π H (s H ,a H )·π L (s L ,a L |s H ,a H );

[0036] Where s=(s H ,s L ) is the joint state, a=(a H ,a L ) is a joint action, π H (s H ,a H ) is a high-level policy that determines the permission allocation of user groups, π L (s L ,a L |s H ,a H ) is a low-level strategy that selects a cache operation based on the output of the high-level strategy and the current data block status;

[0037] S19. Use reinforcement learning algorithms to optimize high-level strategy modules and low-level operation modules to maximize the overall expected cumulative reward:

[0038]

[0039] Among them, θ H ,θ L are the parameters of the high-level strategy module and the low-level operation module, and γ is the discount factor used to balance short-term and long-term benefits;

[0040] Update the policy parameters using the policy gradient method:

[0041]

[0042] Among them, α H ,α L is the learning rate of the high-level strategy module and the low-level operation module, and They are the gradients of the parameters of the high-level strategy module and the low-level operation module, respectively, guiding the optimization direction of the strategy.

[0043] Optionally, the S2 includes:

[0044] S21. Collect the user's historical access data in the cloud storage system, including access time, access frequency, role information, and operation type;

[0045] S22. Constructing an access record matrix where m ij Represents user u i In time period t j The number of access behaviors in a period, n is the number of users, t is the number of time periods, and each element in the matrix reflects the access frequency of the user in a specific time period;

[0046] S23. Define the user role set U, each user u i With eigenvector x i =(r i ,o i ), r i Indicates the user's role type, o i Represents the set of common operation types of the user; the feature vector is used to describe the access characteristics of the user group;

[0047] S26. Generate a time-weighted access matrix W = [w ij ]:

[0048] w ij =α·f ij ·exp(-β·Δt j );

[0049] Among them, f ij For user u i For operation o j The access frequency, Δt j is the time interval between the operation and the current time, α and β are adjustment parameters for balancing historical access data and recent data;

[0050] S27, historical access data matrix M, user feature vector x i The time-weighted access matrix W is input into the high-level policy module, which determines the optimal initial permission policy for the user group by maximizing the product of access frequency and time weighting:

[0051]

[0052] Where P = [p ij ] represents user u i For operation o j The initial authority matrix, p ij =1 means granting permission, p ij =0 means no permission is granted;

[0053] S28, according to the user group authority policy, divide the users into multiple authority level groups G = {g1, g2, ..., g m}, each group g i Include several users and match appropriate resource permissions based on their historical access data;

[0054] S29. Apply the generated initial permission strategy and group division results to the permission management process and serve as the initial configuration of the high-level policy module to provide a basis for the specific permission allocation and cache strategy adjustment of the low-level operation module.

[0055] Optionally, the S3 includes:

[0056] S31, according to the user group authority policy, the user group authority matrix P group Each element p in ij With user group u i For operation o j The initial permission match of ij =1 means granting permission, p ij =0 means no permission is granted;

[0057] S32. Define resource allocation vector R i Represented as user group u i the mix of resources allocated;

[0058] S33, map permissions to resources and establish a mapping function φ(u i ,o j ) means when user group u i Granted Operation o j When the user has the authority, the system allocates the corresponding resource combination R according to the user's needs. i :

[0059]

[0060] Among them, (u i ,o j ) is the user group u i Execute operation o j The required resource combination, ω j For operation o j The weight of R represents its importance in the system. j For operation o j The total amount of resource type allocated, μ ij For user group u i For operation o j The demand intensity is calculated based on the historical access frequency, T max is the maximum effective time window of the permission, T ij For operation o j The remaining time of the corresponding permission;

[0061] S34. Generate an initial authority-resource allocation matrix based on the mapping function:

[0062] Q=[q ij ];

[0063] Among them, q ij =1 indicates user group u i Operation obtained j Minimum resource support required, q ij =0 means no resources are allocated;

[0064] S35, passing the generated initial authority-resource allocation matrix Q to the lower-level operation module, so that the lower-level operation module matches the user's authority and performs corresponding resource allocation when the user makes the first request;

[0065] S36. When a user first accesses the cloud storage system, the low-level operation module calls the cache management system based on the initial permission-resource allocation matrix Q to load the required data blocks into the cache area, and at the same time allocates computing resources and network bandwidth for the user group's operations;

[0066] S37, set the timeliness constraint of the operation for each user group and define the timeliness function τ(u i ,o j ), when the permission expires or the user task is completed, the low-level operation module i ,o j ) automatically releases allocated resources and updates the permission status of the user group based on the access record.

[0067] Optionally, the S6 includes:

[0068] S61. Based on the access behavior of the user group, the reward value of the current behavior is calculated according to the reward function of S2. Behaviors that comply with system security specifications will receive positive rewards, and abnormal behaviors will receive negative rewards.

[0069] S62. When access behavior that complies with security regulations is detected, the system expands the user's authority scope based on the reward value, allowing the user to access more resources;

[0070] S63. When abnormal behavior is detected, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group;

[0071] S64. Define a recovery policy function ρ(t) for the user group whose privilege reduction is triggered, where ρ(t) represents the restoration of some or all privileges after time t. If no new abnormal behavior is found within the specified time, the privileges will be automatically restored.

[0072] S65: Feedback the permission adjustment result to the high-level policy module to update the permission policy of the user group and use it as a reference in the next permission evaluation.

[0073] Optionally, the S10 includes:

[0074] S101. Periodically collect historical access data and cache hit rate data from the system to construct an evaluation dataset:

[0075] D=(H,C);

[0076] Where H = [h ij ] indicates user group u iResources j The access frequency matrix, C = [c lm ] represents data block d l The hit status in the cache, c lm =1 means hit, c lm =0 means miss;

[0077] S102: Define a cache evaluation function η(C) based on the cache hit rate data C to calculate the cache hit rate:

[0078]

[0079] Where L is the number of data blocks, M is the number of access requests, and the cache hit rate η(C) is used to evaluate the execution effect of the cache strategy;

[0080] S103. Based on the user's historical access data H, define the permission evaluation index ζ(H) to calculate the matching degree of resource access:

[0081]

[0082] Among them, n is the number of user groups, k is the number of resources, and p ij For user group u i Resources j The permission status of the access permission is represented by the matching degree ζ(H), which indicates the consistency between the actual access permission and the permission assignment.

[0083] S104, feed back the cache evaluation result η(C) and the authority evaluation result ζ(H) to the high-level policy module, and construct the optimization objective function

[0084]

[0085] Among them, λ1, λ2, and λ3 are weight parameters, which are used to balance access matching degree, cache hit rate, and resource consumption respectively;

[0086] S105, according to the optimization objective function Generate a new permission policy matrix P ′ =[p ′ ij ] and cache strategy matrix Q ′ =[q ′ lm ], where p ′ ij Indicates the adjusted user group permission status, q ′ lm Indicates the new cache strategy;

[0087] S106: passing the new permission policy and cache policy to the lower-level operation module for application, so that the system can perform permission management and cache adjustment according to the optimized policy in the next cycle;

[0088] S107, defining a convergence condition ∈ for the strategy update, determining whether the optimization satisfies the convergence criteria, and confirming that the strategy optimization is complete if the convergence criteria are met; otherwise, continuing the optimization iteration until the convergence criteria are met;

[0089] S108. In the next evaluation cycle, the system repeats the evaluation and optimization process of S101-S107 based on the new historical access data and cache hit rate data to keep the permission strategy and cache strategy in the best state.

[0090] The beneficial effects of the present invention are:

[0091] (1) The present invention adopts a hierarchical reinforcement learning model, and realizes intelligent optimization of permission and resource allocation through the collaborative work of high-level policy modules and low-level operation modules. The high-level policy module manages the initial permission allocation of user groups, and the low-level operation module is responsible for the permission control of specific files or data blocks and the adjustment of dynamic cache strategies. Compared with traditional static permission configuration, the present invention dynamically analyzes user behavior data through reinforcement learning algorithms and generates real-time adjustment permission strategies. The hierarchical reinforcement learning model can dynamically optimize permission and resource allocation according to user access patterns and system load without the need for administrator intervention, ensuring the high efficiency and stability of the system under high load conditions.

[0092] (2) The present invention realizes adaptive cache strategy optimization by constructing a time-weighted access matrix and a cache hit rate evaluation function. The system prioritizes retaining frequently accessed data blocks in the cache based on the user's historical access data and resource usage, and intelligently recycles low-frequency data blocks when cache resources are insufficient. Compared with the fixed rule-based strategy in traditional cache management, the reinforcement learning model of the present invention can dynamically adjust the allocation of cache resources and realize iterative updates of cache strategies through a policy gradient optimization algorithm. The adaptive cache management method significantly improves the system's response speed and resource utilization efficiency, ensuring the provision of high-quality services under different load conditions.

[0093] (3) The present invention designs a real-time permission reduction and recovery mechanism. When the system detects abnormal access behavior, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group and prevent unauthorized access and resource abuse. The permission recovery time is set by the recovery strategy function. If no new abnormal behavior is found within the specified time, part or all of the user's permissions are automatically restored. While improving the security of the system, it avoids accidental harm to legitimate users and improves the fault tolerance of the system. Compared with the traditional manual management method, the reduction and recovery mechanism of the present invention has higher flexibility and accuracy, and significantly reduces the security risk of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0095] Figure 1 This is a flow chart of a method for intelligent management of cloud storage access rights based on reinforcement learning proposed by the present invention;

[0096] Figure 2 This is a structural diagram of the hierarchical reinforcement learning model in the intelligent management method of cloud storage access rights based on reinforcement learning proposed by the present invention. DETAILED DESCRIPTION

[0097] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.

[0098] refer to Figure 1-Figure 2 , a cloud storage access rights intelligent management method based on reinforcement learning, comprising the following steps:

[0099] S1. Build a hierarchical reinforcement learning model. The hierarchical reinforcement learning model includes a high-level policy module and a low-level operation module. The high-level policy module is responsible for user group management and initial allocation of resource permissions, while the low-level operation module is used to control access permissions for specific files or data blocks and adjust dynamic cache policies.

[0100] S2. Collect historical access data of users in the cloud storage system and input the historical access data into the high-level policy module to generate the initial user group permission policy;

[0101] S3. Configure initial permissions and resource allocation based on the user group permission policy, and apply the configuration results to the low-level operation module so that each user group has the minimum access rights required to complete their role tasks upon first access.

[0102] S4. When a user requests to access cloud storage resources for the first time, the low-level operation module calls the cache management system to load the accessed data block or file into the cache area, and records the specific behavior data of the access and feeds it back to the hierarchical reinforcement learning model;

[0103] S5. The hierarchical reinforcement learning model dynamically adjusts the cache strategy based on the data access frequency in the cache, the usage patterns of user groups, and historical behavior patterns, so that frequently accessed files or data blocks are retained in the cache first;

[0104] S6. The hierarchical reinforcement learning model optimizes permission policies through a real-time reward mechanism. It provides positive rewards and adjusts user permissions for access behaviors that comply with system security specifications; it provides negative rewards for abnormal behaviors and triggers an automatic permission reduction mechanism.

[0105] S7. When the system detects insufficient cache area resources, the hierarchical reinforcement learning model dynamically reclaims infrequently accessed data blocks or files based on the current user's access priority, cache resource usage, and system load status;

[0106] S8. When the system load increases or user access requests become frequent, the high-level policy module re-evaluates the access requirements of each user group and passes the adjusted user group permission policy to the low-level operation module to simultaneously optimize the access rights and cache policy.

[0107] S9. When abnormal behavior occurs frequently, the system triggers a secondary verification process based on feedback from the hierarchical reinforcement learning model, requiring the user to undergo additional identity verification and temporarily freezing their high-risk permissions.

[0108] S10. Regularly evaluate and optimize permission policies and cache policies, feed historical access data and cache hit rates back to high-level policy modules, and generate new permission policies and cache policies.

[0109] In this embodiment, S1 includes:

[0110] S11. Build a hierarchical reinforcement learning model, including a high-level policy module and a low-level operation module; the high-level policy module is used to manage the initial allocation of user groups and resource permissions, and the low-level operation module is used to control access permissions for specific files or data blocks and dynamically adjust cache policies;

[0111] S12. In the high-level strategy module, define the state space in Indicates the status of the i-th user group, u i is the feature vector of the user group, including roles and permission levels, r i For resource access records, f i To collect visit frequency statistics;

[0112] S13. In the high-level strategy module, define the action space in Indicates the action of assigning permissions to user groups, assigning permission set P j Access rights to specific resources;

[0113] S14. In the high-level strategy module, establish the reward function:

[0114]

[0115] Among them, E s =Σ k δ(p k )·ω k represents the system security index, δ(p k ) is the permission p k The risk indicator function, ω k is the risk weight of the system security indicator authority, E r =Σ k ψ(p k )·c k represents the resource utilization index, ψ(p k ) is the permission p k The resource usage of k is the resource consumption cost, E u =Σ k φ(p k )·μ k , represents the user satisfaction index, φ(p k ) is the permission p k Contribution to user task completion, μ k For the user to have permission p k The demand intensity of λ1, λ2, and λ3 are adjustment parameters used to balance system security, resource utilization, and user satisfaction.

[0116] S15. In the low-level operation module, define the state space in Indicates the status of the lth data block or file, d l is the data feature vector, including size and type, c l The cache status is 1, which means it is in the cache, or 0, which means it is not in the cache. l is the frequency of visits;

[0117] S16. In the low-level operation module, define the action space in Cache operations performed on data blocks or files, including:

[0118] aload To load the data block into the cache;

[0119] a retain To keep data blocks in cache;

[0120] a replace To replace the data block in the cache with a new data block;

[0121] a evict To remove a data block from the cache;

[0122] S17. In the low-level operation module, establish the reward function:

[0123]

[0124] in, represents the improvement of cache hit rate, Δh is the increase of cache hit times, h total is the total number of visits, C c =Σ l c l ·s l Indicates cache resource consumption, s l is the size of data block l, γ1 and γ2 are adjustment parameters used to balance the improvement of cache hit rate and cache resource consumption;

[0125] S18. Establish a collaborative strategy between the high-level strategy module and the low-level operation module through a hierarchical strategy function:

[0126] π(s,a)=π H (s H ,a H )·π L (s L ,a L |s H ,a H );

[0127] Where s=(s H ,s L ) is the joint state, a=(a H ,a L ) is a joint action, π H (s H ,a H ) is a high-level policy that determines the permission allocation of user groups, π L (s L ,a L |s H ,a H ) is a low-level strategy that selects a cache operation based on the output of the high-level strategy and the current data block status;

[0128] S19. Use reinforcement learning algorithms to optimize high-level strategy modules and low-level operation modules to maximize the overall expected cumulative reward:

[0129]

[0130] Among them, θ H ,θ L are the parameters of the high-level strategy module and the low-level operation module, and γ is the discount factor used to balance short-term and long-term benefits;

[0131] Update the policy parameters using the policy gradient method:

[0132]

[0133] Among them, α H ,α L is the learning rate of the high-level strategy module and the low-level operation module, and They are the gradients of the parameters of the high-level strategy module and the low-level operation module, respectively, guiding the optimization direction of the strategy.

[0134] In this embodiment, S2 includes:

[0135] S21. Collect the user's historical access data in the cloud storage system, including access time, access frequency, role information, and operation type;

[0136] S22. Constructing an access record matrix where m ij Represents user u i In time period t j The number of access behaviors in a period, n is the number of users, t is the number of time periods, and each element in the matrix reflects the access frequency of the user in a specific time period;

[0137] S23. Define the user role set U, each user u i With eigenvector x i =(r i ,o i ), r i Indicates the user's role type, o i Represents the set of common operation types of the user; the feature vector is used to describe the access characteristics of the user group;

[0138] S26. Generate a time-weighted access matrix W = [w ij ]:

[0139] w ij =α·f ij ·exp(-β·Δt j );

[0140] Among them, f ij For user u i For operation o j The access frequency, Δt j is the time interval between the operation and the current time, α and β are adjustment parameters for balancing historical access data and recent data;

[0141] S27, historical access data matrix M, user feature vector x i The time-weighted access matrix W is input into the high-level policy module, which determines the optimal initial permission policy for the user group by maximizing the product of access frequency and time weighting:

[0142]

[0143] Where P = [p ij ] represents user u i For operation o j The initial authority matrix, p ij =1 means granting permission, p ij =0 means no permission is granted;

[0144] S28, according to the user group authority policy, divide the users into multiple authority level groups G = {g1, g2, ..., g m}, each group g i Include several users and match appropriate resource permissions based on their historical access data;

[0145] S29. Apply the generated initial permission strategy and group division results to the permission management process and serve as the initial configuration of the high-level policy module to provide a basis for the specific permission allocation and cache strategy adjustment of the low-level operation module.

[0146] In this embodiment, S3 includes:

[0147] S31, according to the user group authority policy, the user group authority matrix P group Each element p in ij With user group u i For operation o j The initial permission match of ij =1 means granting permission, p ij =0 means no permission is granted;

[0148] S32. Define resource allocation vector R i Represented as user group u i the mix of resources allocated;

[0149] S33, map permissions to resources and establish a mapping function φ(u i ,oj ) means when user group u i Granted Operation o j When the user has the authority, the system allocates the corresponding resource combination R according to the user's needs. i :

[0150]

[0151] Among them, (u i ,o j ) is the user group u i Execute operation o j The required resource combination, ω j For operation o j The weight of R represents its importance in the system. j For operation o j The total amount of resource type allocated, μ ij For user group u i For operation o j The demand intensity is calculated based on the historical access frequency, T max is the maximum effective time window of the permission, T ij For operation o j The remaining time of the corresponding permission;

[0152] S34. Generate an initial authority-resource allocation matrix based on the mapping function:

[0153] Q=[q ij ];

[0154] Among them, q ij =1 indicates user group u i Operation obtained j Minimum resource support required, q ij =0 means no resources are allocated;

[0155] S35, passing the generated initial authority-resource allocation matrix Q to the lower-level operation module, so that the lower-level operation module matches the user's authority and performs corresponding resource allocation when the user makes the first request;

[0156] S36. When a user first accesses the cloud storage system, the low-level operation module calls the cache management system based on the initial permission-resource allocation matrix Q to load the required data blocks into the cache area, and at the same time allocates computing resources and network bandwidth for the user group's operations;

[0157] S37, set the timeliness constraint of the operation for each user group and define the timeliness function τ(u i ,o j ), when the permission expires or the user task is completed, the low-level operation module i,o j ) automatically releases allocated resources and updates the permission status of the user group based on the access record.

[0158] In this embodiment, S6 includes:

[0159] S61. Based on the access behavior of the user group, the reward value of the current behavior is calculated according to the reward function of S2. Behaviors that comply with system security specifications will receive positive rewards, and abnormal behaviors will receive negative rewards.

[0160] S62. When access behavior that complies with security regulations is detected, the system expands the user's authority scope based on the reward value, allowing the user to access more resources;

[0161] S63. When abnormal behavior is detected, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group;

[0162] S64. Define a recovery policy function ρ(t) for the user group whose privilege reduction is triggered, where ρ(t) represents the restoration of some or all privileges after time t. If no new abnormal behavior is found within the specified time, the privileges will be automatically restored.

[0163] S65: Feedback the permission adjustment result to the high-level policy module to update the permission policy of the user group and use it as a reference in the next permission evaluation.

[0164] In this embodiment, S10 includes:

[0165] S101. Periodically collect historical access data and cache hit rate data from the system to construct an evaluation dataset:

[0166] D=(H,C);

[0167] Where H = [h ij ] indicates user group u i Resources j The access frequency matrix, C = [c lm ] represents data block d l The hit status in the cache, c lm =1 means hit, c lm =0 means miss;

[0168] S102: Define a cache evaluation function η(C) based on the cache hit rate data C to calculate the cache hit rate:

[0169]

[0170] Where L is the number of data blocks, M is the number of access requests, and the cache hit rate η(C) is used to evaluate the execution effect of the cache strategy;

[0171] S103. Based on the user's historical access data H, define the permission evaluation index ζ(H) to calculate the matching degree of resource access:

[0172]

[0173] Among them, n is the number of user groups, k is the number of resources, and p ij For user group u i Resources j The permission status of the access permission is represented by the matching degree ζ(H), which indicates the consistency between the actual access permission and the permission assignment.

[0174] S104, feed back the cache evaluation result η(C) and the authority evaluation result ζ(H) to the high-level policy module, and construct the optimization objective function

[0175]

[0176] Among them, λ1, λ2, and λ3 are weight parameters, which are used to balance access matching degree, cache hit rate, and resource consumption respectively;

[0177] S105, according to the optimization objective function Generate a new permission policy matrix P ′ =[p ′ ij ] and cache strategy matrix Q ′ =[q ′ lm ], where p ′ ij Indicates the adjusted user group permission status, q ′ lm Indicates the new cache strategy;

[0178] S106: passing the new permission policy and cache policy to the lower-level operation module for application, so that the system can perform permission management and cache adjustment according to the optimized policy in the next cycle;

[0179] S107, defining a convergence condition ∈ for the strategy update, determining whether the optimization satisfies the convergence criteria, and confirming that the strategy optimization is complete if the convergence criteria are met; otherwise, continuing the optimization iteration until the convergence criteria are met;

[0180] S108. In the next evaluation cycle, the system repeats the evaluation and optimization process of S101-S107 based on the new historical access data and cache hit rate data to keep the permission strategy and cache strategy in the best state.

[0181] Example 1:

[0182] To verify the feasibility and effectiveness of this invention in intelligently managing cloud storage access permissions, the researchers used two publicly available datasets: Amazon Access Logs and Google Cluster Data. The Amazon Access Logs dataset contains access logs of different users to product resources on the e-commerce platform, covering access behavior from January to June 2023; the Google Cluster Data dataset records dynamic requests and access data for computing resources on a multi-tenant cloud computing platform, covering the period from November 2022 to April 2023.

[0183] These two datasets provide real access scenario data for the training and testing of this invention. The experimenters divided the two datasets into training and test sets in a ratio of 8:2 and conducted comparative tests in different permission management scenarios to verify the performance of this invention in optimizing system response speed, resource utilization, and security.

[0184] The experimenters extracted the frequency, access time, and access operation type of users visiting product pages from the Amazon Access Logs dataset and extracted the frequency, tenant role, and resource type of users' resource requests from the Google Cluster Data. The experimenters integrated the two sets of data to train the reinforcement learning model. The specific data dimensions are as follows:

[0185] User group feature vector: role, access frequency, operation type (such as read, write, modify).

[0186] Resource request characteristics: resource type, occupied resource amount, and request response time.

[0187] Time window: records the validity period of user permissions and is used for permission reduction and recovery mechanisms.

[0188] To achieve data standardization, the experimenters used Min-Max scaling to normalize access frequency and resource consumption to the [0,1] range, and used the policy gradient algorithm in reinforcement learning to optimize permissions and resource allocation.

[0189] During model training, the researchers used permission management data from Amazon Access Logs as the initial training sample, and resource request data from Google Cluster Data to simulate permission reduction and recovery mechanisms under high-load scenarios. The training process used an adaptive moment estimation optimizer for 200 epochs, with a learning rate of 0.001 and a mini-batch size of 64. After each training session, the system dynamically updated the permission and caching policies based on user behavior.

[0190] The experimenters compared the traditional static permission configuration method with the method of this invention and used the following evaluation metrics on the two datasets:

[0191] Permission application reduction rate: Compare the number of permission applications by users in the system.

[0192] Average response time: The average response speed of access requests.

[0193] Resource Utilization: Average utilization of cache and computing resources.

[0194] Number of data breach incidents: The number of detected privilege abuse and data breach incidents.

[0195] Table 1 below shows the comparison results between the proposed method and the traditional method on the Amazon Access Logs dataset:

[0196] Table 1 Comparison results on the Amazon Access Logs dataset

[0197] index Method of the present invention Traditional methods Average number of permission requests 1.5 times / week 7 times / week Average response time (seconds) 0.3 seconds 1.8 seconds Resource utilization 87％ 58％ Data breach incidents 0 times 4 times / month

[0198] On the Google Cluster Data dataset, the method of the present invention performs as shown in Table 2:

[0199] Table 2 Comparison results on the Google Cluster Data dataset

[0200] index Method of the present invention Traditional methods Average number of permission requests 2 times / week 9 times / week Average response time (seconds) 0.4 seconds 2.1 seconds Resource utilization 90％ 62％ Data breach incidents 0 times 3 times / month

[0201] Comparative testing demonstrates that the reinforcement learning model of this invention significantly reduces the number of user permission requests, with an average weekly reduction of 78.6% in the Amazon Access Logs dataset and 77.8% in the Google Cluster Data dataset. Furthermore, this method significantly improves system resource utilization, shortening average response time by over 80%, effectively addressing the issues of overly frequent permission configuration and slow response times encountered in traditional methods.

[0202] The permission reduction and recovery mechanism of the present invention played an important role in high-load and abnormal behavior detection. No data leakage incidents were detected during the test, while the traditional static configuration method detected at least three leakage incidents per month. The system's resource utilization rate was also significantly improved, reaching 90% in the Google Cluster Data test, far higher than the 62% of the traditional method. The results proved the feasibility and effectiveness of the present invention in cloud storage systems, providing a flexible, secure and efficient solution for complex and changeable access scenarios.

[0203] This Example 1 demonstrates the application of this invention on a real-world dataset, validating its superior performance in improving system response speed, optimizing resource utilization, and enhancing system security. Through experiments and comparisons on two public datasets, the researchers demonstrated that this method not only addresses complex permission management requirements but also dynamically adjusts permissions and resource allocation under varying load scenarios, significantly improving the management efficiency and security of cloud storage systems.

[0204] The present invention adopts a hierarchical reinforcement learning model and realizes intelligent optimization of permission and resource allocation through the collaborative work of high-level policy modules and low-level operation modules. The high-level policy module manages the initial permission allocation of user groups, and the low-level operation module is responsible for the permission control of specific files or data blocks and the adjustment of dynamic cache strategies. Compared with traditional static permission configuration, the present invention dynamically analyzes user behavior data through reinforcement learning algorithms and generates real-time adjusted permission strategies. The hierarchical reinforcement learning model can dynamically optimize permission and resource configuration according to user access patterns and system load without the need for administrator intervention, ensuring the high efficiency and stability of the system under high load conditions.

[0205] The present invention realizes adaptive cache strategy optimization by constructing a time-weighted access matrix and a cache hit rate evaluation function. Based on the user's historical access data and resource usage, the system prioritizes retaining frequently accessed data blocks in the cache and intelligently reclaims low-frequency data blocks when cache resources are insufficient. Compared with the fixed rule-based strategy in traditional cache management, the reinforcement learning model of the present invention can dynamically adjust the allocation of cache resources and realize iterative updates of cache strategies through a policy gradient optimization algorithm. The adaptive cache management method significantly improves the system's response speed and resource utilization efficiency, ensuring high-quality services under different load conditions.

[0206] The present invention designs a real-time permission reduction and recovery mechanism. When the system detects abnormal access behavior, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group, prevent unauthorized access and resource abuse, and set the permission recovery time through the recovery strategy function. When no new abnormal behavior is found within the specified time, part or all of the user's permissions are automatically restored. While improving the security of the system, it avoids accidental harm to legitimate users and improves the fault tolerance of the system. Compared with traditional manual management methods, the reduction and recovery mechanism of the present invention has higher flexibility and accuracy, and significantly reduces the security risks of the system.

[0207] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A cloud storage access rights intelligent management method based on reinforcement learning, characterized in that: The steps include: S1. Build a hierarchical reinforcement learning model. The hierarchical reinforcement learning model includes a high-level policy module and a low-level operation module. The high-level policy module is responsible for user group management and initial allocation of resource permissions, while the low-level operation module is used to control access permissions for specific files or data blocks and adjust dynamic cache policies. S2. Collect historical access data of users in the cloud storage system and input the historical access data into the high-level policy module to generate the initial user group permission policy; S3. Configure initial permissions and resource allocation based on the user group permission policy, and apply the configuration results to the low-level operation module so that each user group has the minimum access rights required to complete their role tasks upon first access. S4. When a user requests to access cloud storage resources for the first time, the low-level operation module calls the cache management system to load the accessed data block or file into the cache area, and records the specific behavior data of the access and feeds it back to the hierarchical reinforcement learning model; S5. The hierarchical reinforcement learning model dynamically adjusts the cache strategy based on the data access frequency in the cache, the usage patterns of user groups, and historical behavior patterns, so that frequently accessed files or data blocks are retained in the cache first; S6. The hierarchical reinforcement learning model optimizes permission policies through a real-time reward mechanism. It provides positive rewards and adjusts user permissions for access behaviors that comply with system security specifications; it provides negative rewards for abnormal behaviors and triggers an automatic permission reduction mechanism. S7. When the system detects insufficient cache area resources, the hierarchical reinforcement learning model dynamically reclaims infrequently accessed data blocks or files based on the current user's access priority, cache resource usage, and system load status; S8. When the system load increases or user access requests become frequent, the high-level policy module re-evaluates the access requirements of each user group and passes the adjusted user group permission policy to the low-level operation module to simultaneously optimize the access rights and cache policy. S9. When abnormal behavior occurs frequently, the system triggers a secondary verification process based on feedback from the hierarchical reinforcement learning model, requiring the user to undergo additional identity verification and temporarily freezing their high-risk permissions. S10. Regularly evaluate and optimize permission policies and cache policies, feed historical access data and cache hit rates back to the high-level policy module, and generate new permission policies and cache policies.

2. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: Said S1 comprises: S11. Build a hierarchical reinforcement learning model, including a high-level policy module and a low-level operation module; the high-level policy module is used to manage the initial allocation of user groups and resource permissions, and the low-level operation module is used to control access permissions for specific files or data blocks and dynamically adjust cache policies; S12. In the high-level strategy module, define the state space in Indicates the status of the i-th user group, u i is the feature vector of the user group, including roles and permission levels, r i For resource access records, f i To collect visit frequency statistics; S13. In the high-level strategy module, define the action space in Indicates the action of assigning permissions to user groups, assigning permission set P j Access rights to specific resources; S14. In the high-level strategy module, establish the reward function: Among them, E s =∑ k δ(p k )·ω k represents the system security index, δ(p k ) is the permission p k The risk indicator function, ω k is the risk weight of the system security indicator authority, E r =∑ k ψ(p k )·c k represents the resource utilization index, ψ(p k ) is the permission p k The resource usage of k is the resource consumption cost, E u =∑ k φ(p k )·μ k , represents the user satisfaction index, φ(p k ) is the permission p k Contribution to user task completion, μ k For the user to have permission p k The demand intensity of λ1, λ2, and λ3 are adjustment parameters used to balance system security, resource utilization, and user satisfaction. S15. In the low-level operation module, define the state space in Indicates the status of the lth data block or file, d l is the data feature vector, including size and type, c l The cache status is 1, which means it is in the cache, or 0, which means it is not in the cache. l is the frequency of visits; S16. In the low-level operation module, define the action space in Cache operations performed on data blocks or files, including: a load To load the data block into the cache; a retain To keep data blocks in cache; a replace To replace the data block in the cache with a new data block; a evict To remove a data block from the cache; S17. In the low-level operation module, establish the reward function: in, represents the improvement of cache hit rate, Δh is the increase of cache hit times, h total is the total number of visits, C c =∑ l c l ·s l Indicates cache resource consumption, s l is the size of data block l, γ1 and γ2 are adjustment parameters used to balance the improvement of cache hit rate and cache resource consumption; S18. Establish a collaborative strategy between the high-level strategy module and the low-level operation module through a hierarchical strategy function: π(s,a)=π H (s H ,a H )·p L (s L ,a L |s H ,a H ); Where s=(s H ,s L ) is the joint state, a=(a H ,a L ) is a joint action, π H (s H ,a H ) is a high-level policy that determines the permission allocation of user groups, π L (s L ,a L |s H ,a H ) is a low-level strategy that selects a cache operation based on the output of the high-level strategy and the current data block status; S19. Use reinforcement learning algorithms to optimize high-level strategy modules and low-level operation modules to maximize the overall expected cumulative reward: Among them, θ H ,θ L are the parameters of the high-level strategy module and the low-level operation module, and γ is the discount factor used to balance short-term and long-term benefits; Update the policy parameters using the policy gradient method: Among them, α H ,α L is the learning rate of the high-level strategy module and the low-level operation module, and They are the gradients of the parameters of the high-level strategy module and the low-level operation module, respectively, guiding the optimization direction of the strategy.

3. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: The S2 includes: S21. Collect the user's historical access data in the cloud storage system, including access time, access frequency, role information, and operation type; S22. Constructing an access record matrix where m ij Represents user u i In time period t j The number of access behaviors in a period, n is the number of users, t is the number of time periods, and each element in the matrix reflects the access frequency of the user in a specific time period; S23. Define the user role set U, each user u i With eigenvector x i =(r i ,o i ), r i Indicates the user's role type, o i Represents the set of common operation types of the user; the feature vector is used to describe the access characteristics of the user group; S26. Generate a time-weighted access matrix W = [w ij ]: w ij =α·f ij ·exp(-β·Δt j ); Among them, f ij For user u i For operation o j The access frequency, Δt j is the time interval between the operation and the current time, α and β are adjustment parameters for balancing historical access data and recent data; S27, historical access data matrix M, user feature vector x i The time-weighted access matrix W is input into the high-level policy module, which determines the optimal initial permission policy for the user group by maximizing the product of access frequency and time weighting: Where P = [p ij ] represents user u i For operation o j The initial authority matrix, p ij =1 means granting permission, p ij =0 means no permission is granted; S28, according to the user group authority policy, divide the users into multiple authority level groups G = {g1, g2, ..., g m }, each group g i Include several users and match appropriate resource permissions based on their historical access data; S29. Apply the generated initial permission strategy and group division results to the permission management process and serve as the initial configuration of the high-level policy module to provide a basis for the specific permission allocation and cache strategy adjustment of the low-level operation module.

4. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: The S3 includes: S31, according to the user group authority policy, the user group authority matrix P group Each element p in ij With user group u i For operation o j The initial permission match of ij =1 means granting permission, p ij =0 means no permission is granted; S32. Define resource allocation vector R i Represented as user group u i the mix of resources allocated; S33, map permissions to resources and establish a mapping function φ(u i ,o j ) means when user group u i Granted Operation o j When the user has the authority, the system allocates the corresponding resource combination R according to the user's needs. i : Among them, (u i ,o j ) is the user group u i Execute operation o j The required resource combination, ω j For operation o j The weight of R represents its importance in the system. j For operation o j The total amount of resource type allocated, μ ij For user group u i For operation o j The demand intensity is calculated based on the historical access frequency, T max is the maximum valid time window of the permission, T ij For operation o j The remaining time of the corresponding permission; S34. Generate an initial authority-resource allocation matrix based on the mapping function: Q=[q ij ]; Among them, q ij =1 indicates user group u i Operation obtained j Minimum resource support required, q ij =0 means no resources are allocated; S35, passing the generated initial authority-resource allocation matrix Q to the lower-level operation module, so that the lower-level operation module matches the user's authority and performs corresponding resource allocation when the user makes the first request; S36. When a user first accesses the cloud storage system, the low-level operation module calls the cache management system based on the initial permission-resource allocation matrix Q to load the required data blocks into the cache area, and at the same time allocates computing resources and network bandwidth for the user group's operations; S37, set the timeliness constraint of the operation for each user group and define the timeliness function τ(u i ,o j ), when the permission expires or the user task is completed, the low-level operation module i ,o j ) automatically releases allocated resources and updates the permission status of the user group based on the access record.

5. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: The S6 includes: S61. Based on the access behavior of the user group, the reward value of the current behavior is calculated according to the reward function of S2. Behaviors that comply with system security specifications will receive positive rewards, and abnormal behaviors will receive negative rewards. S62. When access behavior that complies with security regulations is detected, the system expands the user's authority scope based on the reward value, allowing the user to access more resources; S63. When abnormal behavior is detected, the permission reduction mechanism is immediately triggered to reduce the permissions of the user group; S64. Define a recovery policy function ρ(t) for the user group whose privilege reduction is triggered, where ρ(t) represents the restoration of some or all privileges after time t. If no new abnormal behavior is found within the specified time, the privileges will be automatically restored. S65: Feedback the permission adjustment result to the high-level policy module to update the permission policy of the user group and use it as a reference in the next permission evaluation.

6. The method for intelligent management of cloud storage access rights based on reinforcement learning according to claim 1, characterized in that: The S10 includes: S101. Periodically collect historical access data and cache hit rate data from the system to construct an evaluation dataset: D=(H,C); Where H = [h ij ] indicates user group u i Resources j The access frequency matrix, C = [c lm ] represents data block d l The hit status in the cache, c lm =1 means hit, c lm =0 means miss; S102: Define a cache evaluation function η(C) based on the cache hit rate data C to calculate the cache hit rate: Where L is the number of data blocks, M is the number of access requests, and the cache hit rate η(C) is used to evaluate the execution effect of the cache strategy; S103. Based on the user's historical access data H, define the permission evaluation index ζ(H) to calculate the matching degree of resource access: Among them, n is the number of user groups, k is the number of resources, and p ij For user group u i Resources j The permission status of the access permission is represented by the matching degree ζ(H), which indicates the consistency between the actual access permission and the permission assignment. S104, feed back the cache evaluation result η(C) and the authority evaluation result ζ(H) to the high-level policy module, and construct the optimization objective function Among them, λ1, λ2, and λ3 are weight parameters, which are used to balance access matching degree, cache hit rate, and resource consumption respectively; S105, according to the optimization objective function Generate a new permission policy matrix P′=[p′ ij ] and cache strategy matrix Q′=[q′ lm ], where p′ ij Indicates the adjusted user group permission status, q′ lm Indicates the new cache strategy; S106: passing the new permission policy and cache policy to the lower-level operation module for application, so that the system can perform permission management and cache adjustment according to the optimized policy in the next cycle; S107, defining a convergence condition ∈ for the strategy update, determining whether the optimization satisfies the convergence criteria, and confirming that the strategy optimization is complete if the convergence criteria are met; otherwise, continuing the optimization iteration until the convergence criteria are met; S108. In the next evaluation cycle, the system repeats the evaluation and optimization process of S101-S107 based on the new historical access data and cache hit rate data to keep the permission strategy and cache strategy in the best state.

Images