Method, apparatus and computer program product for generating retention policy

Abstract

The present disclosure relates to a method, an apparatus and a computer program product for generating a retention policy. The method includes determining a root node representing a current state and at least one child node including an alternative policy based on historical access data and a current data parameter. The method further includes generating at least one extension node including an alternative policy based on the prediction of the historical access data and the at least one child node. The method further includes generating a first state vector by encoding tree structure data including root nodes, child nodes, and extension nodes. And selecting an alternative strategy corresponding to the child node or the extension node based on the first state vector so as to generate a reservation strategy. Through the mode, the predicted future state can be integrated in the process of generating the retention strategy, and a more comprehensive view angle is provided for decision making, so that the accuracy and effectiveness of the strategy are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of data management, and more specifically, to a method, device and computer program product for generating a retention policy. BACKGROUND

[0002] In the field of data protection, the management of retention time for backup data is an important link. In the related art, the process of retention time management mainly relies on fixed, rule-based strategies. These strategies are usually pre-set and static, and they determine the retention period of data according to a pre-prepared schedule or simple business criteria.

[0003] With the continuous growth of data volume and the increasing complexity of business needs, pre-set data retention strategies combined with manual intervention and decision-making are widely adopted. This approach combines predefined rules with human judgment to adapt to data retention needs in different situations. This approach usually relies on the experience and judgment of information technology (IT) administrators, who need to flexibly adjust the retention period of data according to actual conditions. SUMMARY

[0004] Embodiments of the present disclosure propose a method, device and computer program product for generating a retention policy.

[0005] In a first aspect of embodiments of the present disclosure, a method for generating a retention policy is provided. The method includes determining a root node representing a current state and at least one child node including alternative strategies based on historical access data and current data parameters. The method further includes generating at least one expanded node including alternative strategies based on predictions of the historical access data and the at least one child node. The method further includes generating a first state vector by encoding tree structure data including the root node, the at least one child node and the at least one expanded node. The method further includes selecting an alternative strategy corresponding to the at least one child node or the at least one expanded node based on the first state vector to generate the retention policy.

[0006] In a second aspect of the embodiments of the present disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device storing one or more programs, when executed by the one or more processors, cause the one or more processors to implement a method for generating a reservation policy, the method including determining a root node representing a current state and at least one child node including alternative policies based on historical access data and current data parameters. The method further includes generating at least one expanded node including alternative policies based on a prediction of the historical access data and the at least one child node. The method further includes generating a first state vector by encoding tree structure data including the root node, the at least one child node, and the at least one expanded node. The method further includes selecting an alternative policy corresponding to the at least one child node or the at least one expanded node based on the first state vector to generate the reservation policy.

[0007] In a third aspect of the embodiments of the present disclosure, a computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements a method for generating a reservation policy, the method including determining a root node representing a current state and at least one child node including alternative policies based on historical access data and current data parameters. The method further includes generating at least one expanded node including alternative policies based on a prediction of the historical access data and the at least one child node. The method further includes generating a first state vector by encoding tree structure data including the root node, the at least one child node, and the at least one expanded node. The method further includes selecting an alternative policy corresponding to the at least one child node or the at least one expanded node based on the first state vector to generate the reservation policy.

[0008] It is to be understood that the description in the Summary section is not intended to identify key or essential features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be apparent from review of the following description. BRIEF DESCRIPTION OF DRAWINGS

[0009] The above and other features, aspects, and advantages of various embodiments of the present disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings, in which like reference numbers represent like elements throughout. In the drawings:

[0010] Figure 1 A schematic diagram illustrating an example environment in which various embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2 A flowchart illustrating a method for generating a reservation policy according to some embodiments of the present disclosure is shown;

[0012] Figure 3A schematic diagram showing a process of training a reinforcement learning model according to some embodiments of the present disclosure is shown;

[0013] Figure 4 A flowchart showing a process of selecting an optimal policy using a reinforcement learning model according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A schematic diagram showing a process of Monte Carlo tree search (MCTS) in a tree neural network (TNN) according to some embodiments of the present disclosure is shown;

[0015] Figure 6 A schematic diagram showing a process of generating a state vector using a graph neural network (GNN) according to some embodiments of the present disclosure is shown; and

[0016] Figure 7 A block diagram of a device in which a number of embodiments of the present disclosure can be implemented is shown. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described below in greater detail with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided so as to more completely and thoroughly understand the present disclosure. It is understood that the drawings and embodiments of the present disclosure are for exemplary purposes only and are not intended to limit the scope of protection of the present disclosure.

[0018] In the description of embodiments of the present disclosure, the term "includes" and its conjugates are open-ended, meaning that "includes but is not limited to". The term "based on" is intended to mean "based, at least in part, on" The term "one embodiment" or "an embodiment" means "at least one embodiment". The terms "a first", "a second", etc. can refer to different or the same objects. Other explicit or implicit definitions can also be included below.

[0019] In the field of data retention, policy-based systems were once a popular approach. This approach relies on pre-set, static policies to determine the retention period of data, which are usually based on fixed schedules or simple business logic. However, as the amount of data continues to grow, the business environment continues to change, and regulatory compliance requirements become increasingly stringent, policy-based systems gradually reveal their limitations. Due to the inability to dynamically adapt to these changes, these systems often result in unnecessary data accumulation or premature data deletion, wasting storage resources. At the same time, while a manual management and decision-making approach is combined in some situations, this approach not only consumes a large amount of manpower and time, but is also susceptible to human error and subjectivity, leading to inconsistency and unpredictability of data retention policies.

[0020] In related technologies, general predictive models are also used to generate data retention strategies. This approach estimates future storage needs or data relevance using basic predictive algorithms, thus providing a reference for formulating data retention strategies. However, due to the complexity and variability of the data environment, these simple predictive models often face challenges. They typically cannot accurately predict complex future scenarios involving multiple factors such as cost changes, regulatory shifts, or technological advancements. Furthermore, the predictions of these models are usually static, lacking real-time adaptability to constantly changing business environments and regulatory requirements, and therefore often fail to achieve the desired results in practical applications.

[0021] To address this, embodiments of this disclosure propose a scheme for generating retention policies. This scheme involves determining the root node representing the current state using current data parameters and identifying child nodes containing candidate retention policies using historical access data. Then, based on the historical access data and the child nodes, predictions are made to generate extended nodes containing candidate policies. A state vector is generated by encoding the tree structure data including the root node, child nodes, and extended nodes. Finally, candidate policies are selected based on the state vector to generate the retention policy. This approach integrates the predicted future state during retention policy generation, providing a more comprehensive perspective for decision-making, thereby improving the accuracy and effectiveness of the retention policy and increasing the utilization of storage resources. Furthermore, when the data environment changes, the policy can be adjusted accordingly, ensuring the real-time adaptability of the generated retention policy and enabling dynamic and intelligent management of data retention time.

[0022] Figure 1 A schematic diagram of an example environment 100 in which various embodiments of this disclosure may be implemented is shown. For example... Figure 1 As shown, the example environment 100 may include historical access data 101 and current data parameters 103. Historical access data 101 may be records of user or system access to resources such as files, databases, websites, and applications within a certain time period in the past, or it may be data such as user behavior, system responses, and environment variables. For example, historical access data 101 may include access time, accessing user (or system identifier), accessed resource type, access method (such as read, write, delete, etc.), and access result. Current data parameters 103 may be the data state or configuration of a system, application, or specific function at a certain moment. Current data parameters 103 may include various types of data and configuration items, such as system configuration parameters, database connection parameters, user preference settings, and business data status. Current data parameters 103 may be static, such as a system-predefined constant value, or dynamic, such as a value that changes in real time based on user behavior or system status.

[0023] In some embodiments, the root node 107 in the tree structure data 105 can be determined based on the current data parameter 103. The tree structure data 105 can be a hierarchical or nested data set. The tree structure data 105 may include a root node 107 representing the current data state. The root node 107 may have multiple child nodes 109. In this embodiment, each child node 109 may include alternative strategies. The alternative strategies may be predefined and can be selected according to actual needs. For example, the alternative strategy may be to delete retained data every day or every week. After determining the root node 107 and at least one child node 109, future possible scenarios can be predicted based on the alternative strategies of each child node 109 and the patterns or trends of historical access data 101, and future alternative strategies can be formulated to generate extended nodes 111. Each extended node 111 may include the predicted alternative strategies.

[0024] like Figure 1 As shown, in example environment 100, the tree structure data 105 can be encoded using encoding model 113 to generate state vector 115. Encoding model 113 can be a model used to capture the relationships and attributes between nodes in the tree structure data 105, encoding the entire tree structure data 105 into easily processed state vector 115, providing support for subsequent tasks and improving the efficiency and accuracy of processing the tree structure data 105. After generating state vector 115, it is used as the input vector of reinforcement learning model 117. The trained reinforcement learning model 117 selects alternative policies from child nodes 109 or expanded nodes 111 to generate retention policy 119.

[0025] As explained above, this scheme generates child nodes for candidate retention strategies, then predicts based on historical access data and the child nodes to generate extended nodes that include the candidate strategies. By encoding the tree structure data, including the root node, child nodes, and extended nodes, a state vector is generated. Finally, a candidate strategy is selected based on the state vector to generate the retention strategy. In this way, during the generation of the data retention strategy, a state vector that accurately reflects the overall data structure is generated. This state vector can incorporate predictions of future states, thus providing a more comprehensive and in-depth perspective for the decision-making process. This method of formulating forward-looking strategies not only enhances the accuracy and effectiveness of the retention strategy but also significantly improves the utilization efficiency of storage resources. Furthermore, when the data environment changes, the strategy can be adjusted accordingly, ensuring the real-time adaptability of the generated retention strategy and achieving dynamic and intelligent management of data retention time.

[0026] It should be understood that the architecture and functionality of the example environment 100 is described for illustrative purposes only and is not meant to limit the scope of the present disclosure. Embodiments of the present disclosure can be applied to other environments with different structures and / or functionality.

[0027] The following will be described in conjunction with Figures 2 to 6 The processes of the embodiments of the present disclosure are described in detail. For ease of understanding, the specific data mentioned in the following description are all exemplary and do not serve to limit the protection scope of the present disclosure. It can be understood that the embodiments described below can also include additional actions not shown and / or can omit the actions shown, and the scope of the present disclosure is not limited in this respect.

[0028] Figure 2 A flowchart of a method 200 for generating a retention policy according to some embodiments of the present disclosure is shown. At block 202, based on historical access data and current data parameters, a root node representing a current state and at least one child node including alternative policies are determined. For example, as shown in Figure 1 The historical access data 101 can be a record of access to resources by a user or a system over a past time period, which can include access time, user identification, resource type, access method, and access result, etc. to provide a basis for understanding user behavior, system performance, and resource usage. The current data parameters 103 can be the data state or configuration of a system, application, or specific function at a certain time, which is used to determine the root node 107 representing the current state. The root node 107 represents the current data state, while the child nodes 109 are developed around the root node 107, and each child node 107 can include alternative policies for the data state.

[0029] At block 204, at least one extended node including alternative policies is generated based on predictions of the historical access data and the child nodes. For example, as shown in Figure 1 The alternative policies of the child nodes 109 can be based on different assumptions, conditions, or goals, and each alternative policy has different advantages and risks. In embodiments of the present disclosure, a prediction model can be used to collect and analyze the historical access data 101. By analyzing the rules or trends in the historical access data 101, the impact of the alternative policies of the child nodes 109 on future situations can be predicted, and the predicted information can include changes in user demand, fluctuations in system performance, changes in external environment, etc. For each child node 109, multiple extended nodes 111 can be generated according to the predicted future situation, and the extended nodes 111 can be grandchildren nodes of the root node 107 or great-grandchildren nodes of the root node 107. The number of levels of the tree structure data 105 can be selected according to actual needs.

[0030] In some embodiments, before generating the extended node 111 using the prediction model, the environmental parameters in the prediction model can be set according to changes in the data environment, and the setting of the environmental parameters can be based on storage capacity, storage speed, storage cost, central processing unit (CPU) / graphics processing unit (GPU) computing power, user demand changes, and regulatory environment, etc. Adjusting the prediction model according to actual needs can generate a higher-accuracy alternative strategy, thereby improving the accuracy and effectiveness of the retention policy 119.

[0031] In some embodiments, the child node 109 and the extended node 111 can indicate the data volume, the data type, the alternative strategy, the storage parameter, and the preset condition. The data volume represents the quantity of data, the data type represents the nature of the data, the alternative strategy can include detailed information of the retention policy applicable to the data, the storage parameter can include the current and projected cost related to data storage, and the preset condition can include relevant legal and regulatory obligations that affect the retention decision.

[0032] At block 206, a first state vector is generated by encoding the tree structure data including the root node, the at least one child node, and the at least one extended node. For example, as shown in Figure 1 the tree structure data 105 can be encoded using the encoding model 113 to generate the state vector 115. In some embodiments, the encoding model 113 can employ a GNN model, which can accurately capture the relationships and attributes between the nodes of the tree structure data 105 and encode the entire tree structure data 105 into an easily processed state vector 115, providing support for subsequent tasks and improving the efficiency and accuracy of processing the tree structure data 105.

[0033] At block 208, based on the first state vector, an alternative strategy corresponding to at least one child node or at least one extension is selected to generate a retention policy. For example, as shown in Figure 1 the first state vector can be the state vector 115, and after generating the state vector 115, the state vector 115 is used as the input vector of the reinforcement learning model 117, and the trained reinforcement learning model 117 is used to select the optimal alternative strategy from the child node 109 or the extended node 111 to generate the retention policy 119.

[0034] In this way, the predicted future state can be integrated into the process of generating the retention policy, providing a more comprehensive perspective for decision-making, thereby improving the accuracy and effectiveness of the retention policy and improving the utilization of storage resources. In addition, when the data environment changes, the strategy can also be adjusted according to the changes to ensure the real-time adaptability of the generated retention policy, and to dynamically and intelligently manage the retention time of the data.

[0035] The following will be described in conjunction with Figures 3 to 7The process of generating the retention policy is described in detail. In this disclosure, the process is explained in the order of training the reinforcement learning model, selecting the optimal policy using the reinforcement learning model, updating the tree structure data, and generating the state vector. The specific data mentioned in the following description are exemplary and are not intended to limit the scope of this disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0036] Figure 3 A schematic diagram of a process 300 for training a reinforcement learning model according to some embodiments of this disclosure is shown. Figure 3 As shown, reinforcement learning model 313 can be Figure 1 The reinforcement learning model 117 is obtained after training the reinforcement learning model 313, which is an untrained reinforcement learning model 117. The experience pool 301 used to train the reinforcement learning model 313 may include training data 303 and a history retention policy 305. The training data 303 may be historical access data, that is, the training data 303 may be records of user or system access to resources in the past time period. The historical access data 101 may include information such as access time, user ID, resource type, access method and access result, providing a basis for understanding user behavior, system performance and resource usage. The history retention policy 305 may be a retention policy for data used in the past time period.

[0037] In some embodiments, the TNN model 307 can be used to generate training tree structure data based on the experience pool. The process of generating the training tree structure data includes using the TNN model 307 to generate training root nodes, training child nodes, and training expansion nodes based on the training data 303. The generation method of the training root nodes, training child nodes, and training expansion nodes is the same as... Figure 2 The generation methods for the root node, child nodes, and extension nodes are consistent, so they will not be elaborated here.

[0038] In some embodiments, after generating the training tree structure data, the training tree structure data can be encoded by the GNN model 309 to capture the relationships and attributes between the nodes of the tree structure data 105, thereby generating a training vector 311. The training vector 311 can be represented as:

[0039] v_i = f(W * concat(v_i, agg(children(v_i))) + b) (1)

[0040] where v_i represents the vector representation of node i, W represents the weight, b represents the bias, concat represents a function that concatenates the current state of the node with the aggregated child information, agg represents an aggregation function that combines child nodes and extended node data, and f represents a nonlinear activation function.

[0041] As Figure 3 mentioned, after generating the training vector 311, the training vector 311 can be input into the reinforcement learning model 313, and the training vector 311 contains the state information of the current environment. The reinforcement learning model 313 can adopt a Deep Q-Network (DQN) model. After receiving the training vector 311, the reinforcement learning model 313 selects the optimal policy 315 based on the training vector 311, and the storage controller 317 executes the optimal policy 315. The storage controller 317 can be a hardware component that controls and manages storage devices such as hard disks, solid state disks, and flash memories. The storage controller 317 feeds back the change of the data retention environment to the reinforcement learning model 313 in real time after executing the optimal policy 315, and stores the change of the data retention environment and the selected optimal policy 315 in the experience pool 301 as a database for training the reinforcement learning model 313. By continuously selecting the optimal policy 315, the purpose of training the reinforcement learning model 313 can be achieved.

[0042] Figure 4 A schematic diagram of a process 400 of selecting an optimal policy by using a reinforcement learning model according to some embodiments of the present disclosure. As Figure 4 shown, the tree structure data is encoded by using the GNN model to obtain a training vector 401. The training vector 401 is input into the reinforcement learning model 403, and the reinforcement learning model 403 can adopt a Deep Q-Network (DQN) model. The DQN model is a core decision component for generating a retention policy. The DQN model integrates the encoded information from the GNN model, and uses the training vector 401 that can reflect potential future scenarios and decision paths to learn and determine the optimal retention policy.

[0043] At block 405, the DQN model uses a reward learning framework to evaluate the quality of actions taken at each state. The DQN model includes multiple neural network layers, each layer integrating the information encoded by the GNN model, i.e., the training vector 401, and combining the training vector 401 with other related data, such as the current storage capacity memory cost parameters, etc. The DQN model learning process is driven by a reward mechanism that evaluates the results of actions from the aspects of cost efficiency, data availability, and compliance. The reward estimates the expected return of actions, guiding the selection of the most beneficial actions.

[0044] At box 407, the DQN model selects the optimal retention strategy based on the calculated reward. The reward calculated by the DQN model can be called the Q-value reward. After calculating the Q-value for each data instance based on the encoded tree structure data, the DQN module selects the optimal retention strategy based on the Q-value and outputs an action, such as retention, deletion, or archiving. Then, the selected retention strategy is executed in the data retention environment 409. Finally, the DQN model is trained using the execution results, allowing it to continuously improve the retention strategy to adapt to changing conditions and objectives. This improves adaptability to environmental changes while minimizing storage costs. The optimization algorithm can be expressed as:

[0045] Q(s,a) = Q(s,a) + \alpha [r + \gamma \max_{a'}Q(s',a') - Q(s,a)] (2)

[0046] Where Q(s,a) represents the reward value of taking action a in state s, a represents the learning rate, r represents the reward obtained after taking action in state s, γ represents the discount factor for future rewards, s' represents the next state after taking action a, and a' represents the possible action to be taken in the next state s'.

[0047] Figure 5 A schematic diagram of the MCTS process 500 in a TNN according to some embodiments of this disclosure. For example... Figure 5 As shown, after determining the root node 501 and child node 503, any child node 503 can be selected for prediction to generate the corresponding extended node 505. In box 507, the child node and extended node are simulated to generate simulation information. Since the extended node 505 is generated based on predictions of child node 503 and historical access data, the accuracy of the extended node 505 and the availability of alternative strategies cannot be guaranteed. Therefore, before encoding the tree structure data, the estimation of future scenarios can be simulated by simulating the execution of alternative strategies in child node 503 and extended node 505, thereby evaluating the feasibility of different alternative strategies.

[0048] At box 509, the tree structure data is updated based on the simulation information. For each child node 504 and extended node 505, simulation information 507 is generated, and the tree structure data is updated based on the simulation information 507 to correct the nodes. In some embodiments, the update process may involve determining the aggregated simulation information of the direct successor node for each child node 503 and extended node 505, and updating the child node 503 and extended node 505 based on the aggregated simulation information.

[0049] In some embodiments, the encoding of the tree structure data including the root node 501, the updated child node 503, and the updated extended node 505 can be performed to generate a state vector, and the generated state vector can be input into the reinforcement learning model to generate a reservation policy. By taking the pre-emptive measure of simulating the nodes before encoding, the performance of the alternative policy in actual operation can be predicted, and the detailed simulation results can be used to correct the tree structure data. The corrected nodes can indicate the potential risks and compliance adherence in future scenarios, and the corrected actions can ensure the accuracy and effectiveness of the tree structure data, thereby more accurately predicting future scenarios and improving the scientificity and effectiveness of the decision-making process.

[0050] Figure 6 A schematic diagram of a process 600 of generating a state vector using a GNN is shown according to some embodiments of the present disclosure. As shown, the task of the GNN model 603 is to encode the Monte Carlo tree 601 generated by the TNN model in the MCTS process. In embodiments of the present disclosure, the GNN model 603 is directly connected to the TNN module and receives the Monte Carlo tree 601 dynamically generated by the TNN module as its input. The GNN model 603 encodes by capturing the dependency between different potential future scenarios and decisions represented in the Monte Carlo tree 601. Figure 6

[0051] In some embodiments, the encoding process of the GNN model 603 can include encoding each node in the tree structure data to generate a node vector 605, where each node in the tree structure data represents a potential decision or state, and encoding the nodes connected to each other in the tree structure data to generate an edge vector 607, where the edge vector 607 represents the transition from one node to another, i.e., the transition from one state to another. By encoding the edge vector 607, the properties of the taken decisions or actions can be captured. By integrating the node vector 605 and the edge vector 607, the state vector 611 can be obtained. The GNN model 603 applies graph convolution operations to the node vector 605 and the edge vector 607, which enables the integration of node information and global tree structure and enhances the decision context. In embodiments of the present disclosure, the GNN model 603 generates the node vector 605 based on the features of each node itself and the aggregated information from its neighbors, and the node vector 605 can be generated using the following formula:

[0052] ​h_i^{(l+1)} = \sigma\left(W^{(l)} \cdot \text{AGGREGATE}\left(\{h_j^{(l)}: j \in \text{Neighbors}(i)\}\right) + b^{(l)}\right) (3)

[0053] where h_i^{(l)} denotes the feature vector of the i-th node in the l-th layer, W^{(l)} denotes the weight of the l-th layer, b^{(l)} denotes the bias of the l-th layer, AGGREGATE denotes the aggregation of the features of neighboring nodes, σ denotes a nonlinear activation function, and Neighbors denotes the neighboring nodes.

[0054] In some embodiments, the state vector 611 output by the GNN model 603 can be used as an input of a reinforcement learning model, and the reinforcement learning model is used to select a retention policy. In the case of using a reinforcement learning model alone, the complex state of the data retention environment is difficult to represent, the representable state is limited, and the relationship and dependency between different data entities and retention decisions cannot be effectively captured. In the embodiments of the present disclosure, the GNN model 603 is used to encode the Monte Carlo tree, and the complex decision path and the mutual dependency of the state can be represented by the state vector 611, thereby improving the accuracy and delicacy of understanding the data retention environment.

[0055] Figure 7 A schematic block diagram of an example device 700 that can be used to implement embodiments of the present disclosure is shown. As shown, the device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with computer program instructions stored in a read-only memory (ROM) 702 or loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for operation of the device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0056] Various components in the device 700 are connected to the I / O interface 705, including an input unit 706 such as a keyboard, a mouse, etc., an output unit 707 such as various types of displays, a speaker, etc., a storage unit 708 such as a magnetic disk, an optical disk, etc., and a communication unit 709 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0057] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the method 200. For example, in some embodiments, the method 200 can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the method 200 described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the method 200 by other any appropriate means, such as by means of firmware.

[0058] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), etc.

[0059] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / operations specified in the flowchart and / or block diagram block or blocks. The program code can execute entirely on a machine, partly on a machine, as a stand-alone software package, partly on a machine and partly on a remote machine or entirely on a remote machine or server.

[0060] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. Further, while operations are depicted in a particular, chronological sequence, this should not be understood as requiring such order or sequence of operations, or that all illustrated operations be performed to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, while specific implementations are discussed herein, the scope of the present disclosure is not limited to the specific details and representations herein. Certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0061] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method for generating a reservation policy, comprising: determining a root node representing a current state and at least one child node including alternative policies based on historical access data and current data parameters; generating at least one expanded node including alternative policies based on predictions of the historical access data and the at least one child node; generating a first state vector by encoding tree structure data including the root node, the at least one child node, and the at least one expanded node; and selecting an alternative policy corresponding to the at least one child node or the at least one expanded node based on the first state vector to generate the reservation policy.

2. The method of claim 1, wherein generating at least one expanded node including alternative policies comprises: generating at least one grandchild node including alternative policies based on predictions of the historical access data and the child node.

3. The method of claim 1, further comprising: determining aggregated simulation information of direct successor nodes for each child node and the expanded node; updating the child node and the expanded node based on the aggregated simulation information; generating a second state vector by encoding tree structure data including the root node, the updated child node, and the expanded node; and selecting an alternative policy corresponding to the updated child node or the expanded node based on the second state vector to generate a second reservation policy.

4. The method of claim 3, wherein determining aggregated simulation information of direct successor nodes comprises: simulating each child node and the expanded node corresponding alternative policies to generate simulation information; and determining aggregated simulation information of direct successor nodes based on the simulation information.

5. The method of claim 1, wherein generating at least one expanded node including alternative policies comprises: determining environmental parameters of a prediction model based on a predicted environment; and generating at least one expanded node including alternative policies based on predictions of the historical access data and the child node by the prediction model.

6. The method of claim 1, wherein generating a first state vector comprises: generating node vectors by encoding each node in the tree structure data by a graph neural network; generating edge vectors by encoding edges between interconnected nodes in the tree structure data by a graph neural network; and integrating the node vectors and the edge vectors to generate the first state vector.

7. The method of claim 1, further comprising: determining a training root node representing a current state and at least one training child node including alternative policies based on training data and current data parameters; generating at least one training expanded node including alternative policies based on predictions of the training data and the at least one training child node; generating a training vector by encoding tree structure data including the training root node, the at least one training child node, and the at least one training expanded node; and ​ ​ ​ ​ ​ selecting, by the reinforcement learning model, an alternative policy corresponding to the at least one training child node or the at least one training expansion node based on the training vector to generate a third reserve policy. 8.The method of claim 7, wherein generating the third reserve policy comprises: calculating, for each training child node and each training expansion node, a reward value of a corresponding alternative policy; and generating the third reserve policy based on the reward values. 9.The method of claim 8, further comprising: executing the third reserve policy to generate feedback information in a data reserve environment; training the reinforcement learning model based on the feedback information and historical reserve policies in the data reserve environment. 10.The method of claim 1, wherein selecting an alternative policy corresponding to the child node or the expansion node to generate the reserve policy comprises: selecting, by the trained reinforcement learning model, an alternative policy corresponding to the child node or the expansion node based on the first state vector to generate the reserve policy. 11.The method of claim 1, wherein the child node and / or the expansion node indicates a data amount, a data type, an alternative policy, a storage parameter, and a preset condition. 12.An electronic device comprising: at least one processor; and instructions coupled to the at least one processor and having stored thereon instructions that, when executed by the at least one processor, cause the electronic device to: determine a root node representing a current state and at least one child node including alternative policies based on historical access data and current data parameters; generate at least one expansion node including alternative policies based on a prediction of the historical access data and the at least one child node; generate a first state vector by encoding tree structure data including the root node, the at least one child node, and the at least one expansion node; and select an alternative policy corresponding to the at least one child node or the at least one expansion node based on the first state vector to generate a reserve policy. 13.The device of claim 12, wherein the instructions to generate at least one expansion node including alternative policies comprise instructions to: generate at least one grandchild node including alternative policies based on a prediction of the historical access data and the child node. 14.The device of claim 13, the actions further comprising: determining, for each child node and the expansion node, aggregated simulation information of a direct successor node; updating the child node and the expansion node based on the aggregated simulation information; generating a second state vector by encoding tree structure data including the root node, the updated child node, and the expansion node; and selecting an alternative policy corresponding to the updated child node or the expansion node based on the second state vector to generate a second reserve policy. 15.The device of claim 14, wherein the instructions to determine aggregated simulation information of a direct successor node comprise instructions to: ​ ​ ​ simulate each child node and the alternative policy corresponding to the expansion node to generate simulation information; and determine aggregated simulation information of direct successor nodes based on the simulation information.

16. The device of claim 12, wherein the instructions to generate at least one expansion node comprising an alternative policy comprise instructions to: determine environmental parameters of a prediction model based on a predicted environment; and generate at least one expansion node comprising an alternative policy based on the prediction of the historical access data and the child nodes by the prediction model.

17. The device of claim 12, wherein the instructions to generate the first state vector comprise instructions to: generate node vectors by encoding each node in the tree structured data by a graph neural network; generate edge vectors by encoding edges between interconnected nodes in the tree structured data by a graph neural network; and integrate the node vectors and the edge vectors to generate the first state vector.

18. The device of claim 12, the instructions further causing the electronic device to: determine a training root node representing a current state and at least one training child node comprising an alternative policy based on training data and current data parameters; generate at least one training expansion node comprising an alternative policy based on a prediction of the training data and the at least one training child node; generate a training vector by encoding tree structured data comprising the training root node, the at least one training child node, and the at least one training expansion node; and select an alternative policy corresponding to the at least one training child node or the at least one training expansion node by a reinforcement learning model based on the training vector to generate a third reserved policy.

19. The device of claim 18, wherein the instructions to generate the third reserved policy comprise instructions to: calculate a reward value of the corresponding alternative policy for each training child node and each training expansion node; and generate the third reserved policy based on the reward values.

20. A computer program product tangibly stored on a non-transitory computer readable medium and comprising machine executable instructions that, when executed, cause a machine to: determine a root node representing a current state and at least one child node comprising an alternative policy based on historical access data and current data parameters; generate at least one expansion node comprising an alternative policy based on a prediction of the historical access data and the at least one child node; generate a first state vector by encoding tree structured data comprising the root node, the at least one child node, and the at least one expansion node; and select an alternative policy corresponding to the at least one child node or the at least one expansion node based on the first state vector to generate a reserved policy. ​ ​

Images