Storage training calculation collaborative energy consumption optimization method based on reinforcement learning

Abstract

The invention discloses a storage training computing collaborative energy consumption optimization method based on reinforcement learning. The method comprises the following steps: S1, initializing a cloud computing environment; s2, generating a high-computation-intensive task mode and a low-storage-occupancy task mode; s3, constructing a hierarchical reinforcement learning model; s4, preferentially allocating more computing resources to the computing module for a high-computing-intensive task; s5, preferentially optimizing the allocation of computing resources and training resources for the low storage occupancy type task; s6, generating energy consumption budget and resource allocation strategy models of different modules; s7, dynamically adjusting the resource use ratio of the storage, calculation and training modules through the low-layer reinforcement learning strategy model; and S8, transmitting the energy consumption feedback data to the high-level strategy model and the low-level strategy model. According to the invention, the resource allocation of the storage module, the calculation module and the training module can be adaptively adjusted in time in a load changing environment, and the system can be ensured to keep optimal energy efficiency configuration under different loads.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a storage training computing collaborative energy consumption optimization method based on reinforcement learning. Background Art

[0002] In the prior art, energy consumption optimization in a cloud computing environment mainly relies on independent optimization methods for storage modules, computing modules, and training modules. Usually, resource allocation and energy consumption management are performed on the storage, computing, and training modules respectively to reduce the overall energy consumption of the system. A common practice in the existing technical solutions is to use data compression technology and data deduplication for the storage module to reduce storage space requirements, thereby reducing energy consumption. In the computing module, virtualization technology and load balancing are used to improve the utilization rate of computing resources to achieve energy-saving effects. In the training module, energy consumption is reduced by optimizing model parameters and reducing redundant computing methods.

[0003] Although existing technologies have optimized energy consumption in cloud computing environments to a certain extent, they mainly focus on the optimization of a single module and cannot achieve coordinated energy consumption management at the global level of the system. In a complex computing environment where multiple tasks are executed concurrently, different tasks often have different resource requirements and load conditions. Optimizing the energy consumption of storage modules, computing modules or training modules alone cannot effectively cope with resource competition and dynamic changes among multiple tasks. Existing optimization strategies lack a global perspective and it is difficult to coordinate the resource allocation of multiple modules, which makes it difficult to control the overall energy consumption level of the system. In addition, most of the existing technical means use static optimization strategies that cannot adaptively adjust resource allocation according to the real-time load changes of the tasks. In actual cloud computing environments, especially in scenarios with large dynamic changes in load, the effect of energy consumption optimization is greatly limited.

[0004] The main defects of the existing technology can be summarized as follows: First, the existing energy consumption optimization methods are mainly concentrated in a single module, which cannot achieve coordinated optimization among storage, computing and training modules, and the overall energy consumption management of the system is not effective enough; second, most of the existing technologies adopt static optimization methods, which lack the ability to adaptively adjust to real-time load changes, resulting in the inability to update resource allocation strategies in a load-changing environment, affecting the flexibility and accuracy of energy efficiency management; finally, the existing technology does not adequately handle the resource competition problem in a multi-task environment, and cannot effectively coordinate the resource requirements of multiple tasks, resulting in poor energy consumption optimization effect when the system has many concurrent tasks. In summary, the existing technology has a large room for improvement in global energy consumption management and real-time adaptive adjustment. Summary of the invention

[0005] One purpose of the present invention is to propose a storage-training-computing collaborative energy consumption optimization method based on reinforcement learning. The present invention can adaptively adjust the resource allocation of storage modules, computing modules and training modules in a load-changing environment in a timely manner to ensure that the system can maintain the optimal energy efficiency configuration under different loads.

[0006] A storage training computing collaborative energy consumption optimization method based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0007] S1. Initialize the cloud computing environment, set the energy consumption model and resource consumption status of the storage module, computing module and training module, obtain multiple concurrent tasks in the system, and define the resource requirements and load conditions of each task;

[0008] S2. Introduce an unsupervised learning model to perform cluster analysis on the tasks in the system, automatically classify the task types based on the resource requirements and load conditions of the tasks, and generate high-computation-intensive tasks and low-storage-occupancy task modes;

[0009] S3. Build a hierarchical reinforcement learning model, define the high-level strategy model as global energy consumption management and resource allocation, and the low-level strategy model as the specific resource usage optimization of each module storage, computing and training;

[0010] S4. For high computing intensive tasks, during the execution of the high-level policy model, more computing resources are allocated to the computing module first, and the energy consumption budget of the storage module is reduced. The energy consumption management target of the high-level tasks is dynamically adjusted to meet the high demand for computing resources and minimize the computing energy consumption in the global energy consumption.

[0011] S5. For low-storage tasks, the high-level policy model is adjusted to allocate fewer resources to the storage module through unsupervised learning classification to identify tasks with low storage requirements. At the same time, the allocation of computing resources and training resources is prioritized in the low-level policy model.

[0012] S6. Execute high-level reinforcement learning strategy models for different task types, optimize global resource allocation for multiple concurrent tasks based on global energy consumption targets and task patterns identified by unsupervised learning, and generate energy consumption budgets and resource allocation strategy models for different modules;

[0013] S7, for the low-level resource allocation of each concurrent task, dynamically adjust the resource usage ratio of storage, computing and training modules through the low-level reinforcement learning strategy model;

[0014] S8. Introduce a multi-level energy consumption feedback mechanism to monitor the energy consumption information of the system as a whole and the storage module, computing module, and training module in real time, and pass the energy consumption feedback data to the high-level strategy model and the low-level strategy model:

[0015] The high-level policy model dynamically adjusts the high-level resource allocation policy model according to real-time global energy consumption data, task load changes, and resource usage, and provides differentiated resource and energy consumption constraints for high-computation-intensive tasks and low-storage-occupancy tasks;

[0016] The low-level strategy model adaptively adjusts the resource allocation of storage modules, computing modules, and training modules through real-time task energy consumption feedback data, so as to achieve the optimal energy consumption configuration for different types of tasks under dynamic loads.

[0017] Optionally, the S1 includes the following steps:

[0018] S11, initialize the cloud computing environment. The cloud computing environment system architecture includes storage module, computing module and training module. Set the energy consumption model of the storage module to E s (t), the energy consumption model of the computing module is E c (t), the energy consumption model of the training module is E t (t), where t represents the time segment of task execution;

[0019] S12. Set the resource consumption of the storage module to be defined as R s (t), the resource consumption of the computing module is defined as R c (t), the resource consumption of the training module is defined as R t (t);

[0020] S13, the system monitors the execution of multiple concurrent tasks and obtains the T i The resource requirements of task T i The resource demand vector is:

[0021] R i =[R s,i ,R c,i ,R t,i ];

[0022] Among them, R s,i For task T i Resource requirements on the storage module, R c,i For task T i Resource requirements on the computing module, R t,i For task T i Resource requirements on training modules;

[0023] S14. Define the task load vector according to the system load:

[0024] L i =[L s,i ,L c,i ,L t,i ];

[0025] Among them, L s,i Represents task T i The load on the storage module, L c,i Represents task T i The load on the computing module, L t,i Represents task T i Load on training modules;

[0026] S15. Combine the task resource requirement vector and the task load vector to generate the global resource requirement and load matrix of the task:

[0027]

[0028] Among them, R i =[R s,i ,R c,i ,R t,i ] is the resource requirement vector of the i-th task, L i =[L s,i ,L c,i ,L t,i ] is the load vector of the i-th task, and n is the total number of concurrent tasks.

[0029] Optionally, S2 includes the following steps:

[0030] S21. Based on the global resource demand and load matrix of the task, an unsupervised learning model U is introduced. The similarity matrix D between tasks is calculated by using the task resource demand vector and the task load vector through the unsupervised learning model U. ij :

[0031]

[0032] Among them, D ij Represents task T i and T j The distance between m and α m They represent the importance weight coefficients of task resource requirements and load, R m,max and R m,min represents the maximum and minimum value in the resource demand vector, L m,max and L m,min represents the maximum and minimum values ​​in the load vector;

[0033] S23. Using the task similarity matrix D ij Perform cluster analysis to generate different task categories C = {C1, C2, ..., C k}, Task T i Category label As shown below:

[0034]

[0035] in, Represents task T i The task category C k ;|C k | for category C k The number of tasks in D ij Represents task T i With category C k Medium Task T j similarity;

[0036] S24. For the classified tasks, define the resource requirement vector of the high computationally intensive tasks:

[0037]

[0038] in, Represents task T i The proportion of computing resources, R h and L h are the resource requirements and load average vectors for highly computationally intensive tasks, respectively.

[0039] S25. Determine the storage resource requirements of low storage occupation tasks based on the clustering results generated by the unsupervised learning model:

[0040]

[0041] in, Represents task T i The storage resource ratio, R l and L l are the resource requirements and load average vectors of low storage occupancy tasks respectively.

[0042] Optionally, S3 includes the following steps:

[0043] S31. Construct a hierarchical reinforcement learning model. The hierarchical reinforcement learning model consists of a high-level strategy model P H and the low-level policy model P L The high-level policy model and the low-level policy model correspond to the global energy consumption management and resource utilization optimization of each module respectively;

[0044] S32. Define high-level policy model P H The state space is the global state S of the system H =[E s ,E c ,E t ,R h ,L h ], where Es 、E c 、E t are the current energy consumption of the storage module, computing module and training module respectively, R h and L h are the resource demand vector and load vector of high computationally intensive tasks respectively; the action space is the high-level resource allocation strategy A H =[A s ,A c ,A t ], A s , A c , A t Represents the resource ratios allocated to the storage module, computing module, and training module respectively; the high-level reward function R H The goal definition is based on minimizing global energy consumption and maximizing resource utilization efficiency:

[0045] R H =-(w s ·E s +w c ·E c +w t ·E t )+λ·η;

[0046] Among them, w s 、w c 、w t is the energy consumption weight coefficient of the storage module, computing module and training module, η represents the global resource utilization rate, and λ is the weighted coefficient of resource utilization efficiency;

[0047] S33. Define the low-level policy model P L The state space is the module internal state S L =[R s ,R c ,R t ,L s ,L c ,L t ], where R s , R c , R t They represent the resource requirements of the storage module, computing module, and training module respectively, and L s , L c , L t They represent the load of the storage module, computing module, and training module respectively; the action space is the resource usage adjustment strategy A of each module L =[A s,L ,A c,L ,A t,L ], the low-level reward function R L Defined as minimizing energy consumption and optimizing resource allocation for each module:

[0048] R L =-(α s ·E s +α c ·E c +α t ·E t );

[0049] Among them, α s , α c , α t is the energy consumption weight coefficient of the low-level storage module, computing module and training module;

[0050] S34, High-level strategy model P H Adjust the resource allocation strategy of storage module, computing module and training module through global energy consumption feedback A H , and is the low-level policy model P L Provide constraints, low-level policy model P L Under the constraints of high-level strategies, specific optimization and adjustment of resource usage are made to the storage module, computing module and training module to dynamically balance the resource allocation and energy consumption of the storage module, computing module and training module.

[0051] Optionally, S4 includes the following steps:

[0052] S41, in the high-level strategy model P H During the execution of , the resource requirement weight γ of the computing module is defined for high computing intensive tasks c , the energy consumption weight of the storage module γ s , where γ c >γ s Indicates the computing resource priority allocation strategy, which enables computing modules to obtain more resources first;

[0053] S42, based on the resource requirement vector R of the high computationally intensive task h and load vector L h Calculate the computing resource requirements and load of the current task, and dynamically adjust the resource allocation strategy A in the high-level strategy H , so that the resource allocation of the computing module A c Increase, the energy consumption budget E of the storage module s reduce;

[0054] S43, the adjusted high-level energy consumption management target is obtained by optimizing the function f opt It is expressed as:

[0055] f opt =min(w s ·E s +w c·E c +w t ·E t );

[0056] Among them, w s 、w c 、w t are the energy consumption weight coefficients of the storage module, computing module, and training module respectively;

[0057] S44. Through the hierarchical reinforcement learning model, the resource usage of the computing module is adjusted in real time according to the load feedback of the current task, so that the computing module matches the high computing requirements of the task. Through the continuous feedback and adjustment of the high-level strategy model, the highly computing-intensive tasks can achieve the optimal state of computing resource allocation under the global energy consumption management goal, and at the same time minimize the energy consumption of the computing module in the global energy consumption.

[0058] Optionally, S5 includes the following steps:

[0059] S51. Define the resource requirement weight γ′ of the storage module for the low storage occupation tasks identified by unsupervised learning classification. s and the resource requirement weight γ′ of the computing module c , where γ′ s <γ′ c This means that tasks with lower storage requirements are given priority to allocate less storage resources, and more computing resources are given priority to allocate more;

[0060] S52, based on the resource requirement vector R of the low storage occupation task l and load vector L l Calculate the storage resource requirements and load of the current task, and dynamically adjust the high-level policy model P H The resource allocation strategy A′ in H =[A′ s ,A′ c ,A′ t ],A′ s Indicates that the resource allocation of the storage module is reduced, A′ c Indicates that the resource allocation of the computing module has increased;

[0061] S53, by optimizing function f opt,l Adjust the high-level energy management goals to reduce the energy consumption of storage modules in low-storage tasks. The optimization function is defined as:

[0062] f opt,l =min(w′ s ·E′ s +w′ c ·E′ c +w′ t ·E′ t);

[0063] Among them, w′ s , w′ c , w′ t are the energy consumption weight coefficients of the storage module, computing module and training module respectively, E′ s , E′ c , E′ t They are the energy consumption of the adjusted storage module, computing module, and training module respectively;

[0064] S54, in the low-level strategy model P L Prioritize the allocation of computing resources and training resources, and dynamically adjust the resource usage strategy of computing and training modules based on the load feedback of the current task. L =[A′ c,L ,A′ t,L ], so that computing resources and training resources can be optimally allocated in low-storage tasks.

[0065] Optionally, the S6 includes the following steps:

[0066] S61. For different types of tasks, the tasks are divided into high-computation-intensive tasks and low-storage-occupancy tasks based on the task mode identified by the unsupervised learning model. Different resource weights are assigned to each type of task, and the computing resource weight of high-computation-intensive tasks is defined as γ c,h and the storage resource weight γ of low storage occupancy tasks s,l , where γ c,h >γ s,l , represents the priority allocation strategy of resources of different task types;

[0067] S62. Execute high-level reinforcement learning strategy model P H Define the state space as the global state S of the system H =[E s ,E c ,E t ,R h ,L h ,R l ,L l ], R h and L h are the resource requirements and loads of high computationally intensive tasks, R l and L l are the resource requirements and loads of low-storage tasks respectively, and the action space is the resource allocation strategy A H =[A s ,A c ,A t ], used to optimize the resource allocation and energy consumption budget of storage modules, computing modules and training modules;

[0068] S63, according to the global energy consumption target E total , construct energy consumption optimization function f opt,global :

[0069]

[0070] Among them, w m Represents the global energy consumption weight E of the storage module, computing module and training module m (t) represents the energy consumption of the storage module, computing module and training module at time t, Represents module resource requirements R m (t) for load L m The rate of change of (t), κ m Indicates the sensitivity of resource requirements to load changes, is the load L m The second derivative of (t) represents the acceleration change of the load over time, is an exponential decay term used to handle the relationship between resource demand and load. Introduce the resource requirement R of concurrent task i m,i (t) and load L m,i (t), indicating the mutual influence between concurrent tasks, η i The influence coefficient of concurrent tasks on global energy consumption optimization;

[0071] S64. Using high-level reinforcement learning strategy model P H Combined with the task mode generated by unsupervised learning, the resource requirements and load conditions of multiple concurrent tasks are globally evaluated, and the resource usage of the storage module, computing module, and training module are dynamically adjusted to generate energy consumption budgets and resource allocation strategy models for different modules.

[0072] S65. During the task execution process, continuously monitor the resource usage and energy consumption changes of the task, and adjust the high-level strategy model P in real time according to the global energy consumption feedback. H resource allocation strategy.

[0073] Optionally, the S7 includes the following steps:

[0074] S71. Define a low-level reinforcement learning strategy model P for each concurrent task L The state space is the resource usage state S of the storage module, computing module and training module L ;

[0075] S72, dynamically adjust the resource allocation ratio of the storage module, computing module and training module according to the resource requirements and load feedback of concurrent tasks, and define the low-level action space AL ;

[0076] S73, by optimizing function f opt,local Dynamically adjust the resource usage of storage modules, computing modules, and training modules:

[0077]

[0078] Among them, α m Represents the energy consumption weight coefficient of the storage module, computing module and training module, E m represents the actual energy consumption of the storage module, computing module and training module, β m (t) represents the time decay factor of the energy consumption of the storage module, computing module and training module, λ m Indicates the sensitivity factor of load change, Indicates the load L of the storage module, computing module and training module during task execution m (t) rate of change with respect to time, is an exponential function used to measure resource demand R m (t) and load L m (t) is adjusted by m represents the module's sensitivity to resource utilization, ∈ is a small constant to prevent the denominator from being zero, and γ m The control coefficient representing the resource utilization;

[0079] S74, low-level reinforcement learning strategy model P L According to the real-time load feedback of each task, the resource usage ratio of the storage module, computing module and training module is dynamically adjusted, and the low-level reinforcement learning strategy model P is adjusted according to the actual execution of concurrent tasks. L Continuously optimize the resource allocation strategies of storage modules, computing modules, and training modules to achieve optimal resource utilization of storage modules, computing modules, and training modules.

[0080] The beneficial effects of the present invention are:

[0081] (1) The present invention constructs high-level and low-level reinforcement learning strategy models, and performs hierarchical processing on global energy consumption management and resource optimization of storage modules, computing modules and training modules. In the high-level reinforcement learning strategy model, the resource allocation ratio of storage, computing and training modules is dynamically adjusted, and the coordinated optimization among storage modules, computing modules and training modules is realized in combination with global energy consumption feedback data. Compared with the traditional single module optimization strategy, the present invention can adaptively adjust the resource allocation of storage modules, computing modules and training modules according to the real-time load conditions of the system to achieve the optimal configuration of global energy efficiency. The collaborative management method effectively avoids the problem of resource waste caused by independent optimization of storage modules, computing modules and training modules, and significantly improves the overall energy efficiency of the system.

[0082] (2) The present invention performs cluster analysis on tasks through an unsupervised learning model, which can automatically identify high-computation-intensive tasks and low-storage-occupancy tasks, and dynamically adjust the resource allocation strategy of the high-level policy model according to the different resource requirements of the tasks, preferentially allocating more computing resources to high-computation-intensive tasks and reducing the use of storage resources for low-storage-occupancy tasks. This significantly improves the accuracy of resource allocation and avoids the problem of excessive or insufficient allocation of resources, thereby maximizing resource utilization and energy consumption management in a complex multi-tasking environment.

[0083] (3) The present invention can monitor the energy consumption status of the system in real time through the dynamic feedback mechanism of the multi-level reinforcement learning model, and dynamically adjust the resource usage ratio of the storage module, computing module and training module according to the load changes of the task, thereby adaptively adjusting the resource allocation of the storage module, computing module and training module in a load-changing environment, ensuring that the system can maintain the optimal energy efficiency configuration under different loads. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0085] Figure 1 A flowchart of a storage training computing collaborative energy consumption optimization method based on reinforcement learning proposed by the present invention;

[0086] Figure 2 This is a schematic diagram of the energy consumption feedback and dynamic adjustment mechanism between high-level and low-level strategy models in a storage training computing collaborative energy consumption optimization method based on reinforcement learning proposed by the present invention. DETAILED DESCRIPTION

[0087] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0088] refer to Figure 1-2 , a storage training computing collaborative energy consumption optimization method based on reinforcement learning, comprising the following steps:

[0089] S1. Initialize the cloud computing environment, set the energy consumption model and resource consumption status of the storage module, computing module and training module, obtain multiple concurrent tasks in the system, and define the resource requirements and load conditions of each task;

[0090] S2. Introduce an unsupervised learning model to perform cluster analysis on the tasks in the system, automatically classify the task types based on the resource requirements and load conditions of the tasks, and generate high-computation-intensive tasks and low-storage-occupancy task modes;

[0091] S3. Build a hierarchical reinforcement learning model, define the high-level strategy model as global energy consumption management and resource allocation, and the low-level strategy model as the specific resource usage optimization of each module storage, computing and training;

[0092] S4. For high computing intensive tasks, during the execution of the high-level policy model, more computing resources are allocated to the computing module first, and the energy consumption budget of the storage module is reduced. The energy consumption management target of the high-level tasks is dynamically adjusted to meet the high demand for computing resources and minimize the computing energy consumption in the global energy consumption.

[0093] S5. For low-storage tasks, the high-level policy model is adjusted to allocate fewer resources to the storage module through unsupervised learning classification to identify tasks with low storage requirements. At the same time, the allocation of computing resources and training resources is prioritized in the low-level policy model.

[0094] S6. Execute high-level reinforcement learning strategy models for different task types, optimize global resource allocation for multiple concurrent tasks based on global energy consumption targets and task patterns identified by unsupervised learning, and generate energy consumption budgets and resource allocation strategy models for different modules;

[0095] S7, for the low-level resource allocation of each concurrent task, dynamically adjust the resource usage ratio of storage, computing and training modules through the low-level reinforcement learning strategy model;

[0096] S8. Introduce a multi-level energy consumption feedback mechanism to monitor the energy consumption information of the system as a whole and the storage module, computing module, and training module in real time, and pass the energy consumption feedback data to the high-level strategy model and the low-level strategy model:

[0097] The high-level policy model dynamically adjusts the high-level resource allocation policy model according to real-time global energy consumption data, task load changes, and resource usage, and provides differentiated resource and energy consumption constraints for high-computation-intensive tasks and low-storage-occupancy tasks;

[0098] The low-level strategy model adaptively adjusts the resource allocation of storage modules, computing modules, and training modules through real-time task energy consumption feedback data, so as to achieve the optimal energy consumption configuration for different types of tasks under dynamic loads.

[0099] In this implementation, S1 includes the following steps:

[0100] S11, initialize the cloud computing environment. The cloud computing environment system architecture includes storage module, computing module and training module. Set the energy consumption model of the storage module to E s (t), the energy consumption model of the computing module is E c (t), the energy consumption model of the training module is E t (t), where t represents the time segment of task execution;

[0101] S12. Set the resource consumption of the storage module to be defined as R s (t), the resource consumption of the computing module is defined as R c (t), the resource consumption of the training module is defined as R t (t);

[0102] S13, the system monitors the execution of multiple concurrent tasks and obtains the T i The resource requirements of task T i The resource demand vector is:

[0103] R i =[R s,i ,R c,i ,R t,i ];

[0104] Among them, R s,i For task T i Resource requirements on the storage module, R c,i For task T i Resource requirements on the computing module, R t,i For task T i Resource requirements on training modules;

[0105] S14. Define the task load vector according to the system load:

[0106] L i =[L s,i ,L c,i ,L t,i ];

[0107] Among them, L s,i Represents task T i The load on the storage module, L c,i Represents task T i The load on the computing module, L t,i Represents task T i Load on training modules;

[0108] S15. Combine the task resource requirement vector and the task load vector to generate the global resource requirement and load matrix of the task:

[0109]

[0110] Among them, R i =[R s,i ,R c,i ,R t,i ] is the resource requirement vector of the i-th task, L i =[L s,i ,L c,i ,L t,i ] is the load vector of the i-th task, and n is the total number of concurrent tasks.

[0111] In this implementation, S2 includes the following steps:

[0112] S21. Based on the global resource demand and load matrix of the task, an unsupervised learning model U is introduced. The similarity matrix D between tasks is calculated by using the task resource demand vector and the task load vector through the unsupervised learning model U. ij :

[0113]

[0114] Among them, D ij Represents task T i and T j The distance between m and α m They represent the importance weight coefficients of task resource requirements and load, R m,max and R m,min represents the maximum and minimum value in the resource demand vector, L m,max and L m,min represents the maximum and minimum values ​​in the load vector;

[0115] S23. Using the task similarity matrix D ij Perform cluster analysis to generate different task categories C = {C1, C2, ..., C k}, Task T i Category label As shown below:

[0116]

[0117] in, Represents task T i The task category C k ;|C k | for category C k The number of tasks in D ij Represents task T i With category C k Medium Task T j similarity;

[0118] S24. For the classified tasks, define the resource requirement vector of the high computationally intensive tasks:

[0119]

[0120]

[0121] in, Represents task T i The proportion of computing resources, R h and L h are the resource requirements and load average vectors for highly computationally intensive tasks, respectively.

[0122] S25. Determine the storage resource requirements of low storage occupation tasks based on the clustering results generated by the unsupervised learning model:

[0123]

[0124] in, Represents task T i The storage resource ratio, R l and L l are the resource requirements and load average vectors of low storage occupancy tasks respectively.

[0125] In this implementation mode, S3 includes the following steps:

[0126] S31. Construct a hierarchical reinforcement learning model. The hierarchical reinforcement learning model consists of a high-level strategy model P H and the low-level policy model P L The high-level policy model and the low-level policy model correspond to the global energy consumption management and resource utilization optimization of each module respectively;

[0127] S32. Define high-level policy model P H The state space is the global state S of the system H =[E s ,E c ,E t ,Rh ,L h ], where E s 、E c 、E t are the current energy consumption of the storage module, computing module and training module respectively, R h and L h are the resource demand vector and load vector of high computationally intensive tasks respectively; the action space is the high-level resource allocation strategy A H =[A s ,A c ,A t ], A s , A c , A t Represents the resource ratios allocated to the storage module, computing module, and training module respectively; the high-level reward function R H The goal definition is based on minimizing global energy consumption and maximizing resource utilization efficiency:

[0128] R H =-(w s ·E s +w c ·E c +w t ·E t )+λ·η;

[0129] Among them, w s 、w c 、w t is the energy consumption weight coefficient of the storage module, computing module and training module, η represents the global resource utilization, and λ is the weighted coefficient of resource utilization efficiency;

[0130] S33. Define the low-level policy model P L The state space is the module internal state S L =[R s ,R c ,R t ,L s ,L c ,L t ], where R s , R c , R t They represent the resource requirements of the storage module, computing module, and training module respectively, and L s , L c , L t They represent the load of the storage module, computing module, and training module respectively; the action space is the resource usage adjustment strategy A of each module L =[A s,L ,A c,L ,A t,L ], the low-level reward function RL Defined as minimizing energy consumption and optimizing resource allocation for each module:

[0131] R L =-(α s ·E s +α c ·E c +α t ·E t );

[0132] Among them, α s , α c , α t is the energy consumption weight coefficient of the low-level storage module, computing module and training module;

[0133] S34, High-level strategy model P H Adjust the resource allocation strategy of storage module, computing module and training module through global energy consumption feedback A H , and is the low-level policy model P L Provide constraints, low-level policy model P L Under the constraints of high-level strategies, specific optimization and adjustment of resource usage are made to the storage module, computing module and training module to dynamically balance the resource allocation and energy consumption of the storage module, computing module and training module.

[0134] In this implementation, S4 includes the following steps:

[0135] S41, in the high-level strategy model P H During the execution of , the resource requirement weight γ of the computing module is defined for high computing intensive tasks c , the energy consumption weight of the storage module γ s , where γ c >γ s Indicates the computing resource priority allocation strategy, which enables computing modules to obtain more resources first;

[0136] S42, based on the resource requirement vector R of the high computationally intensive task h and load vector L h Calculate the computing resource requirements and load of the current task, and dynamically adjust the resource allocation strategy A in the high-level strategy H , so that the resource allocation of the computing module A c Increase, the energy consumption budget E of the storage module s reduce;

[0137] S43, the adjusted high-level energy consumption management target is obtained by optimizing the function f opt It is expressed as:

[0138] f opt =min(ws ·E s +w c ·E c +w t ·E t );

[0139] Among them, w s 、w c 、w t are the energy consumption weight coefficients of the storage module, computing module, and training module respectively;

[0140] S44. Through the hierarchical reinforcement learning model, the resource usage of the computing module is adjusted in real time according to the load feedback of the current task, so that the computing module matches the high computing requirements of the task. Through the continuous feedback and adjustment of the high-level strategy model, the highly computing-intensive tasks can achieve the optimal state of computing resource allocation under the global energy consumption management goal, and at the same time minimize the energy consumption of the computing module in the global energy consumption.

[0141] In this implementation, S5 includes the following steps:

[0142] S51. Define the resource requirement weight γ′ of the storage module for the low storage occupation tasks identified by unsupervised learning classification. s and the resource requirement weight γ′ of the computing module c , where γ′ s <γ′ c This means that tasks with lower storage requirements are given priority to allocate less storage resources, and more computing resources are given priority to allocate more;

[0143] S52, based on the resource requirement vector R of the low storage occupation task l and load vector L l Calculate the storage resource requirements and load of the current task, and dynamically adjust the high-level policy model P H The resource allocation strategy A′ in H =[A′ s ,A′ c ,A′ t ],A′ s Indicates that the resource allocation of the storage module is reduced, A′ c Indicates that the resource allocation of the computing module has increased;

[0144] S53, by optimizing function f opt,l Adjust the high-level energy management goals to reduce the energy consumption of storage modules in low-storage tasks. The optimization function is defined as:

[0145] f opt,l =min(w′ s ·E′ s +w′ c ·E′c +w′ t ·E′ t );

[0146] Among them, w′ s , w′ c , w′ t are the energy consumption weight coefficients of the storage module, computing module and training module respectively, E′ s , E′ c , E′ t They are the energy consumption of the adjusted storage module, computing module, and training module respectively;

[0147] S54, in the low-level strategy model P L Prioritize the allocation of computing resources and training resources, and dynamically adjust the resource usage strategy of computing and training modules based on the load feedback of the current task. L =[A′ c,L ,A′ t,L ], so that computing resources and training resources can be optimally allocated in low-storage tasks.

[0148] In this implementation, S6 includes the following steps:

[0149] S61. For different types of tasks, the tasks are divided into high-computation-intensive tasks and low-storage-occupancy tasks based on the task mode identified by the unsupervised learning model. Different resource weights are assigned to each type of task, and the computing resource weight of high-computation-intensive tasks is defined as γ c,h and the storage resource weight γ of low storage occupancy tasks s,l , where γ c,h >γ s,l , represents the priority allocation strategy of resources of different task types;

[0150] S62. Execute high-level reinforcement learning strategy model P H Define the state space as the global state S of the system H =[E s ,E c ,E t ,R h ,L h ,R l ,L l ], R h and L h are the resource requirements and loads of high computationally intensive tasks, R l and L l are the resource requirements and loads of low-storage tasks respectively, and the action space is the resource allocation strategy A H =[A s ,A c ,At ], used to optimize the resource allocation and energy consumption budget of storage modules, computing modules and training modules;

[0151] S63, according to the global energy consumption target E total , construct energy consumption optimization function f opt,global :

[0152]

[0153] Among them, w m Represents the global energy consumption weight E of the storage module, computing module and training module m (t) represents the energy consumption of the storage module, computing module and training module at time t, Represents module resource requirements R m (t) for load L m The rate of change of (t), κ m Indicates the sensitivity of resource requirements to load changes, is the load L m The second derivative of (t) represents the acceleration change of the load with time, is an exponential decay term, which is used to deal with the relationship between resource demand and load. Introduce the resource requirement R of concurrent task i m,i (t) and load L m,i (t), indicating the mutual influence between concurrent tasks, η i The influence coefficient of concurrent tasks on global energy consumption optimization;

[0154] S64. Using high-level reinforcement learning strategy model P H Combined with the task mode generated by unsupervised learning, the resource requirements and load conditions of multiple concurrent tasks are globally evaluated, and the resource usage of the storage module, computing module, and training module are dynamically adjusted to generate energy consumption budgets and resource allocation strategy models for different modules.

[0155] S65. During the task execution process, continuously monitor the resource usage and energy consumption changes of the task, and adjust the high-level strategy model P in real time according to the global energy consumption feedback. H resource allocation strategy.

[0156] In this implementation, S7 includes the following steps:

[0157] S71. Define a low-level reinforcement learning strategy model P for each concurrent task L The state space is the resource usage state S of the storage module, computing module and training module L ;

[0158] S72, dynamically adjust the resource allocation ratio of the storage module, computing module and training module according to the resource requirements and load feedback of concurrent tasks, and define the low-level action space A L ;

[0159] S73, by optimizing function f opt,local Dynamically adjust the resource usage of storage modules, computing modules, and training modules:

[0160]

[0161] Among them, α m Represents the energy consumption weight coefficient of the storage module, computing module and training module, E m represents the actual energy consumption of the storage module, computing module and training module, β m (t) represents the time decay factor of the energy consumption of the storage module, computing module and training module, λ m Indicates the sensitivity factor of load change, Indicates the load L of the storage module, computing module and training module during task execution m (t) rate of change with respect to time, is an exponential function used to measure resource demand R m (t) and load L m (t) is adjusted by m represents the module's sensitivity to resource utilization, ∈ is a small constant to prevent the denominator from being zero, and γ m The control coefficient representing the resource utilization;

[0162] S74, low-level reinforcement learning strategy model P L According to the real-time load feedback of each task, the resource usage ratio of the storage module, computing module and training module is dynamically adjusted, and the low-level reinforcement learning strategy model P is used according to the actual execution of concurrent tasks. L Continuously optimize the resource allocation strategies of storage modules, computing modules, and training modules to achieve optimal resource utilization of storage modules, computing modules, and training modules.

[0163] Embodiment 1:

[0164] In order to verify the feasibility and effectiveness of the present invention in practical applications, the experimenters used publicly available data sets Google Cluster Data V2 (GCD) and Alibaba Cloud Trace Benchmark (ACTB) for experiments. These two data sets respectively cover the multi-task concurrent processing of large cloud computing data centers and record the resource usage of different tasks in storage modules, computing modules and training modules.

[0165] Google Cluster Data V2 (GCD) contains task scheduling data of Google data centers within one month, covering the computing, storage, and bandwidth usage of approximately 6,700 tasks. The data set spans from January to February 2021. Alibaba Cloud Trace Benchmark (ACTB) contains multi-task execution data of Alibaba Cloud data centers within three months, with a data time range of March to June 2022. The data set records the resource usage of tasks, task scheduling information, and the energy consumption of the system during task execution.

[0166] The experimenters divided the two data sets into training set and test set in a ratio of 8:2 to simulate the storage, training and computing energy consumption optimization scenarios when the system executes multiple tasks concurrently.

[0167] The experimenters conducted experiments in a cloud computing environment with a 16-core CPU, 64GB of memory and 1TB of SSD. The system simultaneously ran high-computing-intensive tasks (machine learning model training tasks), low-storage-occupancy tasks (data flow analysis tasks) and hybrid tasks (video encoding tasks). To ensure the fairness of the experiment, the experimenters ran the traditional static resource allocation method and the reinforcement learning method proposed in this invention under the same hardware conditions. In the experiment, the experimenters used a hierarchical strategy model based on deep reinforcement learning and combined it with unsupervised learning to classify and optimize the tasks.

[0168] Storage, computing, and training resource optimization process:

[0169] The tasks in the GCD and ACTB datasets are clustered using an unsupervised learning model to identify high computationally intensive tasks and low storage occupancy tasks.

[0170] Use a high-level reinforcement learning model to allocate computing, storage, and training resources to different tasks, and adjust the resource allocation strategy based on the system's real-time energy consumption feedback.

[0171] In the low-level reinforcement learning model, the resource usage ratio of the storage module, computing module and training module is dynamically adjusted to ensure the optimal resource allocation during the execution of the task.

[0172] The experimenter's model was trained using an adaptive moment estimation optimizer with a learning rate of 0.001, a mini-batch size of 32, and a training cycle of 100 epochs.

[0173] The experimenters conducted performance evaluation in multiple dimensions including task delay, system energy consumption, resource utilization and task execution time. To further verify the effectiveness of the model, they compared the traditional static resource allocation method with the dynamic collaborative optimization method proposed in this invention.

[0174] Task delay: used to evaluate the time it takes for a task to be scheduled and completed.

[0175] System energy consumption: used to evaluate the energy consumption of the system during task execution.

[0176] Resource utilization: The ratio of storage, computing, and training resources used during task execution.

[0177] Task execution time: Evaluates the total time of a task from start to finish.

[0178] The experimental results are shown in Table 1 below:

[0179] Table 1 Comparison of experimental results between the method of the present invention and the traditional static allocation method on GCD and ACTB datasets

[0180]

[0181] It can be seen from the results in Table 1 that, compared with the traditional static allocation method, the reinforcement learning method of the present invention reduces system energy consumption by about 15%-18%, and shortens task execution time by about 16%-20%. On the GCD dataset, the method of the present invention improves resource utilization by about 20% and reduces task delay by 12.5%; on the ACTB dataset, resource utilization is improved by about 22% and task delay is reduced by 11.8%.

[0182] Specific examples in the GCD data set: Task 1234 in the GCD data set is a highly computationally intensive task. Under the traditional method, 50% of the computing resources and 30% of the storage resources are allocated to it, resulting in insufficient computing resources and waste of storage resources. Under the method of the present invention, the system allocates 70% of the computing resources and 20% of the storage resources to it, and the computing efficiency is improved by 15%, the task completion time is shortened by 20 seconds, and the energy consumption is reduced by 12%. Task 5678 is a low storage occupancy task. The traditional method allocates more storage resources to it, but the actual demand for the task is low, resulting in a waste of resources. The method of the present invention analyzes the task requirements through unsupervised learning, and only allocates 20% of the storage resources to it, and the computing resources are increased to 40%, the task completion time is shortened by 15 seconds, and the energy consumption is reduced by 15%.

[0183] Experimental data show that the traditional static allocation method performs poorly in terms of resource utilization and system energy consumption, especially in the scenario of concurrent execution of multiple tasks. It is unable to effectively coordinate the resource usage of the storage module, computing module and training module, resulting in increased system energy consumption and low task execution efficiency. The method of the present invention dynamically adjusts resource allocation through reinforcement learning, and can optimize the usage ratio of storage, computing and training resources in real time according to the changes in task load, thereby reducing energy consumption while improving task execution efficiency.

[0184] In summary, the present invention has verified its effectiveness in complex task environments through experiments. It not only has obvious advantages in energy consumption management, but also can significantly improve resource utilization and task execution efficiency. The experimental results show that the method of the present invention can better cope with resource allocation and energy consumption optimization problems in multi-task concurrent scenarios compared with traditional methods, and provides a new technical approach for efficient energy consumption management of cloud computing data centers.

[0185] The present invention constructs high-level and low-level reinforcement learning strategy models, and performs hierarchical processing for global energy consumption management and resource optimization of storage modules, computing modules and training modules respectively. In the high-level reinforcement learning strategy model, by dynamically adjusting the resource allocation ratio of storage, computing and training modules, the collaborative optimization among storage modules, computing modules and training modules is realized in combination with global energy consumption feedback data. Compared with the traditional single module optimization strategy, the present invention can adaptively adjust the resource allocation of storage modules, computing modules and training modules according to the real-time load conditions of the system to achieve the optimal configuration of global energy efficiency. The collaborative management method effectively avoids the problem of resource waste caused by independent optimization of storage modules, computing modules and training modules, and significantly improves the overall energy efficiency of the system.

[0186] The present invention performs cluster analysis on tasks through an unsupervised learning model, can automatically identify high-computing-intensive tasks and low-storage-occupancy tasks, dynamically adjust the resource allocation strategy of the high-level policy model according to the different resource requirements of the tasks, preferentially allocate more computing resources to high-computing-intensive tasks, and reduce the use of storage resources for low-storage-occupancy tasks, significantly improving the accuracy of resource allocation and avoiding the problem of excessive or insufficient allocation of resources, thereby maximizing resource utilization and energy consumption management in a complex multi-tasking environment.

[0187] The present invention can monitor the energy consumption status of the system in real time through the dynamic feedback mechanism of the multi-level reinforcement learning model, and dynamically adjust the resource usage ratio of the storage module, computing module and training module according to the load changes of the task, so as to adaptively adjust the resource allocation of the storage module, computing module and training module in a load-changing environment in a timely manner, ensuring that the system can maintain the optimal energy efficiency configuration under different loads.

[0188] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A storage training computing collaborative energy consumption optimization method based on reinforcement learning, characterized in that: The steps include: S1. Initialize the cloud computing environment, set the energy consumption model and resource consumption status of the storage module, computing module and training module, obtain multiple concurrent tasks in the system, and define the resource requirements and load conditions of each task; S2. Introduce an unsupervised learning model to perform cluster analysis on the tasks in the system, automatically classify the task types based on the resource requirements and load conditions of the tasks, and generate high-computation-intensive tasks and low-storage-occupancy task modes; S3. Build a hierarchical reinforcement learning model, define the high-level strategy model as global energy consumption management and resource allocation, and the low-level strategy model as the specific resource usage optimization of each module storage, computing and training; S4. For high computing intensive tasks, during the execution of the high-level policy model, more computing resources are allocated to the computing module first, and the energy consumption budget of the storage module is reduced. The energy consumption management target of the high-level tasks is dynamically adjusted to meet the high demand for computing resources and minimize the computing energy consumption in the global energy consumption. S5. For low-storage tasks, the high-level policy model is adjusted to allocate fewer resources to the storage module through unsupervised learning classification to identify tasks with low storage requirements. At the same time, the allocation of computing resources and training resources is prioritized in the low-level policy model. S6. Execute high-level reinforcement learning strategy models for different task types, optimize global resource allocation for multiple concurrent tasks based on global energy consumption targets and task patterns identified by unsupervised learning, and generate energy consumption budgets and resource allocation strategy models for different modules; S7, for the low-level resource allocation of each concurrent task, dynamically adjust the resource usage ratio of storage, computing and training modules through the low-level reinforcement learning strategy model; S8. Introduce a multi-level energy consumption feedback mechanism to monitor the energy consumption information of the system as a whole and the storage module, computing module, and training module in real time, and pass the energy consumption feedback data to the high-level strategy model and the low-level strategy model: The high-level policy model dynamically adjusts the high-level resource allocation policy model according to real-time global energy consumption data, task load changes, and resource usage, and provides differentiated resource and energy consumption constraints for high-computation-intensive tasks and low-storage-occupancy tasks; The low-level strategy model adaptively adjusts the resource allocation of storage modules, computing modules, and training modules through real-time task energy consumption feedback data, so as to achieve the optimal energy consumption configuration for different types of tasks under dynamic loads.

2. According to the method for optimizing storage training computing collaborative energy consumption based on reinforcement learning in claim 1, it is characterized in that: The S1 comprises the following steps: S11, initialize the cloud computing environment. The cloud computing environment system architecture includes storage module, computing module and training module. Set the energy consumption model of the storage module to E s (t), the energy consumption model of the computing module is E c (t), the energy consumption model of the training module is E t (t), where t represents the time segment of task execution; S12. Set the resource consumption of the storage module to be defined as R s (t), the resource consumption of the computing module is defined as R c (t), the resource consumption of the training module is defined as R t (t); S13, the system monitors the execution of multiple concurrent tasks and obtains the T i The resource requirements of task T i The resource demand vector is: R i =[R s,i ,R c,i ,R t,i ]; Among them, R s,i For task T i Resource requirements on the storage module, R c,i For task T i Resource requirements on the computing module, R t,i For task T i Resource requirements on training modules; S14. Define the task load vector according to the system load: L i =[L s,i ,L c,i ,L t,i ]; Among them, L s,i Represents task T i The load on the storage module, L c,i Represents task T i The load on the computing module, L t,i Represents task T i Load on training modules; S15. Combine the task resource requirement vector and the task load vector to generate the global resource requirement and load matrix of the task: Among them, R i =[R s,i ,R c,i ,R t,i ] is the resource requirement vector of the i-th task, L i =[L s,i ,L c,i ,L t,i ] is the load vector of the i-th task, and n is the total number of concurrent tasks.

3. According to the method for optimizing storage training computing collaborative energy consumption based on reinforcement learning in claim 1, it is characterized in that: The S2 comprises the following steps: S21. Based on the global resource demand and load matrix of the task, an unsupervised learning model U is introduced. The similarity matrix D between tasks is calculated by using the task resource demand vector and the task load vector through the unsupervised learning model U. ij : Among them, D ij Represents task T i and T j The distance between m and α m They represent the importance weight coefficients of task resource requirements and load, R m,max and R m,min represents the maximum and minimum value in the resource demand vector, L m,max and L m,min represents the maximum and minimum values ​​in the load vector; S23. Using the task similarity matrix D ij Perform cluster analysis to generate different task categories C = {C1, C2, ..., C k }, Task T i Category label As shown below: in, Represents task T i The task category C k ;|C k | for category C k The number of tasks in D ij Represents task T i With category C k Medium Task T j similarity; S24. For the classified tasks, define the resource requirement vector of the high computationally intensive tasks: in, Represents task T i The proportion of computing resources, R h and L h are the resource requirements and load average vectors for highly computationally intensive tasks, respectively. S25. Determine the storage resource requirements of low storage occupation tasks based on the clustering results generated by the unsupervised learning model: in, Represents task T i The storage resource ratio, R l and L l are the resource requirements and load average vectors of low storage occupancy tasks respectively.

4. According to the method for optimizing storage training computing collaborative energy consumption based on reinforcement learning in claim 1, it is characterized in that: The S3 comprises the following steps: S31. Construct a hierarchical reinforcement learning model. The hierarchical reinforcement learning model consists of a high-level strategy model P H and the low-level policy model P L The high-level policy model and the low-level policy model correspond to the global energy consumption management and resource utilization optimization of each module respectively; S32. Define high-level policy model P H The state space is the global state S of the system H =[E s ,E c ,E t ,R h ,L h ], where E s 、E c 、E t are the current energy consumption of the storage module, computing module and training module respectively, R h and L h are the resource demand vector and load vector of high computationally intensive tasks respectively; the action space is the high-level resource allocation strategy A H =[A s ,A c ,A t ], A s , A c , A t Represents the resource ratios allocated to the storage module, computing module, and training module respectively; the high-level reward function R H The goal definition is based on minimizing global energy consumption and maximizing resource utilization efficiency: R H J-(w s ·E s +w c ·E c +w t ·E t )+λ·ηH Among them, w s 、w c 、w t is the energy consumption weight coefficient of the storage module, computing module and training module, η represents the global resource utilization, and λ is the weighted coefficient of resource utilization efficiency; S33. Define the low-level policy model P L The state space is the module internal state S L =[R s ,R c ,R t ,L s ,L c ,L t ], where R s , R c , R t They represent the resource requirements of the storage module, computing module, and training module respectively, and L s , L c , L t They represent the load of the storage module, computing module, and training module respectively; the action space is the resource usage adjustment strategy A of each module L =[A s,L ,A c,L ,A t,L ], the low-level reward function R L Defined as minimizing energy consumption and optimizing resource allocation for each module: R L =-(a s ·E s +a c ·E c +a t ·E t ); Among them, α s , α c , α t is the energy consumption weight coefficient of the low-level storage module, computing module and training module; S34, High-level strategy model P H Adjust the resource allocation strategy of storage module, computing module and training module through global energy consumption feedback A H , and is the low-level policy model P L Provide constraints, low-level policy model P L Under the constraints of high-level strategies, specific optimization and adjustment of resource usage are made to the storage module, computing module and training module to dynamically balance the resource allocation and energy consumption of the storage module, computing module and training module.

5. According to the method for optimizing storage training computing collaborative energy consumption based on reinforcement learning in claim 1, it is characterized in that: The S4 comprises the following steps: S41, in the high-level strategy model P H During the execution of , the resource requirement weight γ of the computing module is defined for high computing intensive tasks c , the energy consumption weight of the storage module γ s , where γ c >γ s Indicates the computing resource priority allocation strategy, which enables computing modules to obtain more resources first; S42, based on the resource requirement vector R of the high computationally intensive task h and load vector L h Calculate the computing resource requirements and load of the current task, and dynamically adjust the resource allocation strategy A in the high-level strategy H , so that the resource allocation of the computing module A c Increase, the energy consumption budget E of the storage module s reduce; S43, the adjusted high-level energy consumption management target is obtained by optimizing the function f opt It is expressed as: f opt =min(w s ·HAVE BEEN s +w c ·HAVE BEEN c +w t ·HAVE BEEN t ); Among them, w s 、w c 、w t are the energy consumption weight coefficients of the storage module, computing module, and training module respectively; S44. Through the hierarchical reinforcement learning model, the resource usage of the computing module is adjusted in real time according to the load feedback of the current task, so that the computing module matches the high computing requirements of the task. Through the continuous feedback and adjustment of the high-level strategy model, the highly computing-intensive tasks can achieve the optimal state of computing resource allocation under the global energy consumption management goal, and at the same time minimize the energy consumption of the computing module in the global energy consumption.

6. According to the method for optimizing storage training computing collaborative energy consumption based on reinforcement learning in claim 1, it is characterized in that: The S5 comprises the following steps: S51. Define the resource requirement weight γ′ of the storage module for the low storage occupation tasks identified by unsupervised learning classification. s and the resource requirement weight γ′ of the computing module c , where γ′ s <γ′ c This means that tasks with lower storage requirements are given priority to allocate less storage resources, and more computing resources are given priority to allocate more; S52, based on the resource requirement vector R of the low storage occupation task l and load vector L l Calculate the storage resource requirements and load of the current task, and dynamically adjust the high-level policy model P H The resource allocation strategy A′ in H =[A′ s ,A′ c ,A′ t ],A′ s Indicates that the resource allocation of the storage module is reduced, A′ c Indicates that the resource allocation of the computing module has increased; S53, by optimizing function f opt,l Adjust the high-level energy management goals to reduce the energy consumption of storage modules in low-storage tasks. The optimization function is defined as: f opt,l =min(w′ s ·HAVE BEEN' s +w′ c ·HAVE BEEN' c +w′ t ·HAVE BEEN' t ); Among them, w′ s , w′ c , w′ t are the energy consumption weight coefficients of the storage module, computing module and training module respectively, E′ s , E′ c , E′ t They are the energy consumption of the adjusted storage module, computing module, and training module respectively; S54, in the low-level strategy model P L Prioritize the allocation of computing resources and training resources, and dynamically adjust the resource usage strategy of computing and training modules based on the load feedback of the current task. L =[A′ c,L ,A′ t,L ], so that computing resources and training resources can be optimally allocated in low-storage tasks.

7. The storage training computing collaborative energy consumption optimization method based on reinforcement learning according to claim 1 is characterized in that: The S6 comprises the following steps: S61. For different types of tasks, the tasks are divided into high-computation-intensive tasks and low-storage-occupancy tasks based on the task mode identified by the unsupervised learning model. Different resource weights are assigned to each type of task, and the computing resource weight of high-computation-intensive tasks is defined as γ c,h and the storage resource weight γ of low storage occupancy tasks s,l , where γ c,h >γ s,l , represents the priority allocation strategy of resources of different task types; S62. Execute high-level reinforcement learning strategy model P H Define the state space as the global state S of the system H =[E s ,E c ,E t ,R h ,L h ,R l ,L l ], R h and L h are the resource requirements and loads of high computationally intensive tasks, R l and L l are the resource requirements and loads of low-storage tasks respectively, and the action space is the resource allocation strategy A H =[A s ,A c ,A t ], used to optimize the resource allocation and energy consumption budget of storage modules, computing modules and training modules; S63, according to the global energy consumption target E total , construct energy consumption optimization function f opt,global : Among them, w m Represents the global energy consumption weight E of the storage module, computing module and training module m (t) represents the energy consumption of the storage module, computing module and training module at time t, Represents module resource requirements R m (t) for load L m The rate of change of (t), κ m Indicates the sensitivity of resource requirements to load changes, is the load L m The second derivative of (t) represents the acceleration change of the load with time, is an exponential decay term, which is used to deal with the relationship between resource demand and load. Introduce the resource requirement R of concurrent task i m,i (t) and load L m,i (t), indicating the mutual influence between concurrent tasks, η i The influence coefficient of concurrent tasks on global energy consumption optimization; S64. Using high-level reinforcement learning strategy model P H Combined with the task mode generated by unsupervised learning, the resource requirements and load conditions of multiple concurrent tasks are globally evaluated, and the resource usage of the storage module, computing module, and training module are dynamically adjusted to generate energy consumption budgets and resource allocation strategy models for different modules. S65. During the task execution process, continuously monitor the resource usage and energy consumption changes of the task, and adjust the high-level strategy model P in real time according to the global energy consumption feedback. H resource allocation strategy.

8. The storage training computing collaborative energy consumption optimization method based on reinforcement learning according to claim 1 is characterized in that: The S7 comprises the following steps: S71. Define a low-level reinforcement learning strategy model P for each concurrent task L The state space is the resource usage state S of the storage module, computing module and training module L ; S72, dynamically adjust the resource allocation ratio of the storage module, computing module and training module according to the resource requirements and load feedback of concurrent tasks, and define the low-level action space A L ; S73, by optimizing function f opt,local Dynamically adjust the resource usage of storage modules, computing modules, and training modules: Among them, α m Represents the energy consumption weight coefficient of the storage module, computing module and training module, E m represents the actual energy consumption of the storage module, computing module and training module, β m (t) represents the time decay factor of the energy consumption of the storage module, computing module and training module, λ m Indicates the sensitivity factor of load change, Indicates the load L of the storage module, computing module and training module during task execution m (t) rate of change with respect to time, is an exponential function used to measure resource demand R m (t) and load L m (t) is adjusted by m represents the module's sensitivity to resource utilization, ∈ is a small constant to prevent the denominator from being zero, and γ m The control coefficient representing the resource utilization; S74, low-level reinforcement learning strategy model P L According to the real-time load feedback of each task, the resource usage ratio of the storage module, computing module and training module is dynamically adjusted, and the low-level reinforcement learning strategy model P is used according to the actual execution of concurrent tasks. L Continuously optimize the resource allocation strategies of storage modules, computing modules, and training modules to achieve optimal resource utilization of storage modules, computing modules, and training modules.

Images