In-situ data storage scheduling optimization method in smart power grid scene

Abstract

The invention discloses an in-situ data storage scheduling optimization method in a smart power grid scene. The method comprises the following steps: firstly, establishing a multi-level storage structure to optimize the in-situ data storage efficiency; designing a personalized scheduling queue to orderly record scheduling requirements generated by different media; thirdly, providing an optimization problem of maximizing the in-situ data storage scheduling rate; and finally, a deep reinforcement learning technology is utilized to make a response decision for the scheduling demand of the in-situ data. On the premise that the power and storage space of the smart grid are limited, response differences and delay comparison of all scheduling demand queues are comprehensively considered, and response decision is made by taking reduction of the number of to-be-responded demands and waiting delay as targets, so that the system can perform response decision making while the demand queues continuously generate scheduling demands. The appropriate scheduling demand is coordinated to be processed preferentially, response congestion caused by response difference is reduced, and the scheduling demands generated by different media are responded at a higher speed.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing, and specifically relates to a storage and scheduling method for in-situ data in a power grid scenario. Background Art

[0002] With the development of Internet technology and virtualization technology, cloud computing has shown higher flexibility and economy than local hardware resources. In many industrial systems, the amount of data generated by terminal devices has exploded. The centralized processing method of uploading data to the cloud has gradually exposed problems such as bandwidth pressure, transmission delay, and high energy consumption. Therefore, edge computing sinks part of the computing and storage capabilities from the centralized cloud to the edge of the network close to the terminal device, which not only reduces the delay in data transmission, but also improves the real-time response capability and processing efficiency of the system, and the cloud-edge integrated system is gradually improved. However, due to geographical and resource constraints, some industrial scenarios prefer to process directly at or near the data source to reduce the cost of transmission and improve the response speed and efficiency of the system. The demand for in-situ data processing is gradually emerging.

[0003] In the smart grid environment, wind turbines are deployed in remote locations and there are many factors such as power restrictions. Therefore, an independent server is deployed for each wind turbine, and a complete in-situ server system (In-situ Server Systems, InS) is formed to process the generated in-situ data in a targeted manner. Initially, the in-situ data generated by the terminal devices were mostly numerical sensor data, and the storage and scheduling requirements were not complicated. A single media structure was used for storage, and the corresponding scheduling demand queue was set for sequential response.

[0004] With the continuous progress and development of smart grids, terminal devices are required to be able to record more types of functional data: log files, video files and other data. Various types of data differ in size, storage rate and update frequency. The use of traditional storage and response structures will cause the entire storage and scheduling process to become very cumbersome due to many problems such as access rate mismatch and judgment of demand sources. How to improve the storage structure of in-situ data under smart grids, reasonably optimize the scheduling response process, and achieve comprehensive decision-making that comprehensively considers response differences and delay comparisons has become a challenge that needs to be addressed urgently. Solving this problem is of great significance to promoting the storage and scheduling of in-situ data in smart grid scenarios. Summary of the invention

[0005] The purpose of the present invention is to provide a method for in-situ data storage and scheduling demand selection decision-making that can achieve both efficient storage and timely response in a complex smart grid environment.

[0006] The solution to achieve the purpose of the present invention is: an in-situ data storage scheduling optimization method in a smart grid scenario, comprising the following steps:

[0007] Step 1: Establish a multi-level storage structure model;

[0008] Step 2: Establish a personalized scheduling queue model;

[0009] Step 3: Propose the optimization problem of in-situ data storage scheduling in the smart grid scenario;

[0010] Step 4: Propose a demand queue decision problem based on the Markov decision process, use the deep reinforcement learning algorithm to solve the problem, comprehensively consider the response difference and delay comparison, make decisions on the scheduling demand response of the original data, and pursue efficient response to the scheduling demand.

[0011] Furthermore, the multi-level storage structure model described in step 1 is established as follows:

[0012] Assume that a medium can be represented as a set Med i ={C i , A i , D i}, which encapsulates the various properties of the medium. i Used to indicate the medium Med i Storage capacity. i Indicates that the response medium Med is selected i The corresponding scheduling demand queue is the maximum number of demands that the corresponding demand queue can respond to in a single time slot. In each time slot, all media will generate scheduling demands and use the scheduling demand queue to record them. i The resulting dispatch demand X is distributed by Poisson distribution X~Poisson(D i ) OK, D i represents the mean of the Poisson distribution, e is the base of the natural logarithm, and x! represents the factorial of x. The details are as follows:

[0013]

[0014] In the smart grid environment, in order to store the in-situ data of terminal devices, independent servers are set up near the data source, and all servers constitute the in-situ server system (InS). Traditional storage structures often use a single medium to store in-situ data, which can be represented by a set as Structure traditional schedule = {Que}, where Med{C, A, D}. As the in-situ data types in the scene gradually diversify, the original storage structure is difficult to provide targeted storage services based on data characteristics such as type, size, and update rate.

[0015] The present invention considers three types of media and designs a multi-level storage structure: high-frequency storage medium Med1 = {C1, A1, D1}, medium-frequency storage medium Med2 = {C2, A2, D2}, and low-frequency storage medium Med3 = {C3, A3, D3}. The three types of media meet the following constraints:

[0016] C1<C2<C3, A1>A2>A3, D1>D2>D3 (2)

[0017] High-frequency media will generate more demands, and the number of demands selected for response in each time slot is relatively large, but the storage capacity is relatively small; on the contrary, low-frequency media will generate relatively few demands, and the number of demands selected for response in each time slot is relatively small, but it has a larger storage capacity. multi-level storage = {Med1, Med2, Med3}, and satisfy the following constraints:

[0018] C=C1+C2+C3 (3)

[0019] The multi-level storage structure can achieve targeted storage based on factors such as the access frequency and data specifications of the original data under the premise of limited storage capacity, greatly improving storage efficiency.

[0020] Furthermore, the personalized scheduling queue model described in step 2 is established as follows:

[0021] Set a demand queue can be represented as a set Que i ={M i , R i , N i , L i}, used to record the corresponding medium Med i The scheduling requirements generated by the request also encapsulate the various attributes of the request queue. i Used to represent the demand queue Que i The maximum required storage capacity will be determined based on the corresponding medium Med i Storage capacity C i Dynamically plan the capacity of the queue to avoid wasting resources. i Used to represent the demand queue Que i The remaining required capacity that can be stored. N i Used to represent a queue Que i The number of times the response was selected. i Used to represent the demand queue Que iThe accumulated delay after the demand in is selected for response. The traditional structure uses a single queue to record scheduling requirements, which can be expressed as Structure traditional schedule = {Que}, where Que = {M, R, N, L}. Each time when responding to the scheduling demand generated by the original data, only the quantitative demand generated by a single scheduling demand queue can be responded to. With the introduction of multi-level storage structure, if the traditional structure is still used to record the demand, the overall response process is relatively cumbersome, and there are a series of problems such as low access efficiency and slow response speed.

[0022] This study designs three scheduling demand queues for the multi-level storage structure: high-frequency medium scheduling demand queue Que1 = {M1, R1, N1, L1}, medium-frequency medium scheduling demand queue Que2 = {M2, R2, N2, L2}, and low-frequency medium scheduling demand queue Que3 = {M3, R3, N3, L3}. The three scheduling demand queues meet the following constraints:

[0023] M1<M2<M3 (4)

[0024] The high-frequency medium scheduling demand queue has a relatively small maximum demand capacity because the corresponding medium has a faster response rate; the low-frequency medium scheduling demand queue is relatively large. Use three scheduling demand combinations to design a personalized scheduling structure personalised schedule = {Que1, Que2, Que3}, and satisfy the following constraints:

[0025] M=M1+M2+M3 (5)

[0026] Personalized scheduling queues can record the in-situ data scheduling demands generated by different media in a targeted manner without changing the constraints of the original demand capacity. This eliminates the need to determine the source media of the scheduling demands, greatly simplifying the steps and processes of demand response.

[0027] The demand queue is used to store the scheduling requirements generated by the corresponding medium. A scheduling requirement can be expressed as a set Where Dem j Represents the jth demand in the current scheduling demand queue. Indicates the number of time slots generated by the scheduling demand, which will be initialized in the generated time slot; Indicates the number of time slots that the scheduling demand is responded to. The default value is null and will not be assigned until it is responded to. Indicates the number of time slots that the scheduling requirement has been waiting for in the current time slot.

[0028] Furthermore, the optimization problem of in-situ data storage scheduling in the smart grid scenario described in step 3 is proposed as follows:

[0029] In each time slot, all media will generate scheduling demands according to the Poisson distribution, and the specific quantity meets the following constraints:

[0030] Poisson(D i )≤R i ≤M i (6)

[0031] The required remaining capacity R of the medium i Always less than or equal to the maximum required capacity of the medium M i When the actual number of scheduling requirements generated meets the above constraints, the generated requirements can be loaded into the corresponding demand queue Que i Otherwise, keep the demand generated this time until it meets the conditions and then load it into the demand queue.

[0032] In each time slot, in order to simplify the response process and improve the response rate, select a scheduling demand corresponding to a medium to respond. i After that, you need to use the corresponding medium Med i The maximum number of requests that can be responded to in a single time slot A i , to respond to Que in turn i The former A i Each scheduling requirement needs to be responded to while the delay needs to be accumulated to L i The details are as follows:

[0033]

[0034] Each demand needs to calculate the response time slot when it is responded to. and generate time slots The difference between n The delay after the selected response. The value range of n is [1, A i ], the scheduling demand delay taken out of each time slot needs to be accumulated.

[0035] Furthermore, the demand queue decision problem based on the Markov decision process described in step 4 is proposed. The deep reinforcement learning algorithm is used to comprehensively consider the response difference and delay comparison, and the scheduling demand response of the original data is decided to pursue efficient response to the scheduling demand. The details are as follows:

[0036] In each time slot, all media will generate scheduling requirements. To simplify the response process, only one demand queue can be selected to respond. Therefore, it is necessary to consider the number of demands and response differences of each demand queue, and comprehensively select the appropriate demand queue to respond. The decision process of the demand queue is regarded as a Markov model, and K = (State, Action, Reward) represents the Markov decision process. Among them, State represents the state space, Action represents the action space, and Reward represents the reward function. The environment of Markov decision is the in-situ data storage and scheduling system under the smart grid. The process is defined as follows:

[0037] State: In the in-situ data multi-level storage scheduling environment, the state is a real-time representation of the state of each medium demand queue in the time slot. It is a three-dimensional vector composed of the current average demand margin, average delay, and average demand satisfaction ratio, and is specifically expressed as:

[0038] State = [ARR, AD, DSR] (7)

[0039] ARR represents the average demand margin of all demand queues in each time slot. For the k medium demand queues that form a multi-level structure, the demand margin is calculated separately, that is, the maximum capacity M of the demand queue k and the remaining capacity of the demand queue R k The difference. All medium demand margins are accumulated and averaged, and its specific definition is shown in formula (8). AD represents the average delay of all demand queues in each time slot. The delays of k demand queues are accumulated and averaged, and its specific definition is shown in formula (9). DAR represents the demand satisfaction ratio of all demand queues in each time slot. The number of all responded demands and the total number of generated demands in each queue are accumulated respectively, and the ratio of the two is calculated, and its specific definition is shown in formula (10).

[0040]

[0041] Action: In the context of multi-level storage scheduling of in-situ data, an action is to select a scheduling demand queue to respond in each time slot, which can be defined as:

[0042] Action∈{Que1, Que2, Que3} (11)

[0043] Reward: In the context of multi-level storage scheduling of in-situ data, the reward setting aims to reduce the average demand margin ARR and minimize the average delay AD. α and β are used as adjustment coefficients to balance the weights of demand and delay, which can be defined as:

[0044] Reward=-(α×ARR+β×AD) (12)

[0045] Compared with the prior art, the present invention has the following significant advantages: (1) It improves the traditional storage structure of in-situ data in the smart grid scenario. Under the premise that the storage capacity of the in-situ server remains unchanged, a multi-level storage structure is designed to store different types of in-situ data in a targeted manner, thereby accelerating the access rate. (2) It improves the traditional dispatch demand response structure, designs a personalized demand dispatch queue, and independently records the dispatch demands generated by different media to facilitate timely response. (3) It adopts deep Q learning to make decisions on the response dispatch demand queue, comprehensively considers the response difference and delay comparison, and optimizes the response process of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a diagram of the multi-level storage structure of in-situ data in the smart grid environment of the present invention.

[0047] Figure 2 This is a personalized dispatching structure for in-situ data in a power grid environment in the present invention.

[0048] Figure 3 This is a diagram of the demand queue decision method based on deep reinforcement learning in the present invention. DETAILED DESCRIPTION

[0049] The present invention is described in detail below with reference to the accompanying drawings

[0050] The present invention discloses an in-situ data storage scheduling optimization method in a smart grid scenario, comprising the following steps:

[0051] Step 1: Establish a multi-level storage structure model, as follows:

[0052] Combination Figure 1 , assuming that a medium can be represented as a set Med i ={C i , A i , D i}, which encapsulates the various properties of the medium. i Used to indicate the medium Med i Storage capacity. i Indicates that the response medium Med is selected i The corresponding scheduling demand queue is the maximum number of demands that the corresponding demand queue can respond to in a single time slot. In each time slot, all media will generate scheduling demands and use the scheduling demand queue to record them. i The resulting dispatch demand X is distributed by Poisson distribution X~Poisson(D i ) OK, D irepresents the mean of the Poisson distribution, e is the base of the natural logarithm, and x! represents the factorial of x. The details are as follows:

[0053]

[0054] In the smart grid environment, in order to store the in-situ data of terminal devices, independent servers are set up near the data source, and all servers constitute the in-situ server system (InS). Traditional storage structures often use a single medium to store in-situ data, which can be represented by a set as Structure traditional storage ={Med}, where Med ={C, A, D}. As the in-situ data types in scenarios become increasingly diversified, it is difficult for the original storage structure to provide targeted storage services based on data characteristics such as type, size, and update rate.

[0055] The present invention considers three types of media and designs a multi-level storage structure: high-frequency storage medium Med1 = {C1, A1, D1}, medium-frequency storage medium Med2 = {C2, A2, D2}, and low-frequency storage medium Med3 = {C3, A3, D3}. The three types of media meet the following constraints:

[0056] C1<C2<C3, A1>A2>A3, D1>D2>D3 (14)

[0057] High-frequency media will generate more demands, and the number of demands selected for response in each time slot is relatively large, but the storage capacity is relatively small; on the contrary, low-frequency media will generate relatively few demands, and the number of demands selected for response in each time slot is relatively small, but it has a larger storage capacity. multi-level storage = {Med1, Med2, Med3}, and satisfy the following constraints:

[0058] C=C1+C2+C3 (15)

[0059] The multi-level storage structure can achieve targeted storage based on factors such as the access frequency and data specifications of the original data under the premise of limited storage capacity, greatly improving storage efficiency.

[0060] Step 2: Establish a personalized scheduling queue model, as follows:

[0061] Combination Figure 2 , a demand queue can be represented as a set Que i ={M i , R i , N i , L i}, used to record the corresponding medium Med i The scheduling requirements generated by the request also encapsulate the various attributes of the request queue. i Used to represent the demand queue Que i The maximum required storage capacity will be determined based on the corresponding medium Med i Storage capacity C i Dynamically plan the capacity of the queue to avoid wasting resources. i Used to represent the demand queue Que i The remaining required capacity that can be stored. N i Used to represent a queue Que i The number of times the response was selected. i Used to represent the demand queue Que i The accumulated delay after the demand in is selected for response. The traditional structure uses a single queue to record scheduling requirements, which can be expressed as Structure traditional schedule = {Que}, where Que = {M, R, N, L}. Each time when responding to the scheduling demand generated by the original data, only the quantitative demand generated by a single scheduling demand queue can be responded to. With the introduction of multi-level storage structure, if the traditional structure is still used to record the demand, the overall response process is relatively cumbersome, and there are a series of problems such as low access efficiency and slow response speed.

[0062] This study designs three scheduling demand queues for the multi-level storage structure: high-frequency medium scheduling demand queue Que1 = {M1, R1, N1, L1}, medium-frequency medium scheduling demand queue Que2 = {M2, R2, N2, L2}, and low-frequency medium scheduling demand queue Que3 = {M3, R3, N3, L3}. The three scheduling demand queues meet the following constraints:

[0063] M1<M2<M3 (16)

[0064] The high-frequency medium scheduling demand queue has a relatively small maximum demand capacity because the corresponding medium has a faster response rate; the low-frequency medium scheduling demand queue is relatively large. Use three scheduling demand combinations to design a personalized scheduling structure personalised schedule = {Que1, Que2, Que3}, and satisfy the following constraints:

[0065] M=M1+M2+M3 (17)

[0066] Personalized scheduling queues can record the in-situ data scheduling demands generated by different media in a targeted manner without changing the constraints of the original demand capacity. This eliminates the need to determine the source media of the scheduling demands, greatly simplifying the steps and processes of demand response.

[0067] The demand queue is used to store the scheduling requirements generated by the corresponding medium. A scheduling requirement can be expressed as a set Where Dem j Represents the jth demand in the current scheduling demand queue. Indicates the number of time slots generated by the scheduling demand, which will be initialized in the generated time slot; Indicates the number of time slots that the scheduling demand is responded to. The default value is null and will not be assigned until it is responded to. Indicates the number of time slots that the scheduling requirement has been waiting for in the current time slot.

[0068] Step 3: Propose the optimization problem of in-situ data storage scheduling in the smart grid scenario, as follows:

[0069] In each time slot, all media will generate scheduling demands according to the Poisson distribution, and the specific quantity meets the following constraints:

[0070] Poisson(D i )≤R i ≤M i (18)

[0071] The required remaining capacity R of the medium i Always less than or equal to the maximum required capacity of the medium M i When the actual number of scheduling requirements generated meets the above constraints, the generated requirements can be loaded into the corresponding demand queue Que i Otherwise, keep the demand generated this time until it meets the conditions and then load it into the demand queue.

[0072] In each time slot, in order to simplify the response process and improve the response rate, select a scheduling demand corresponding to a medium to respond. i After that, you need to use the corresponding medium Med i The maximum number of requests that can be responded to in a single time slot A i , to respond to Que in turn i The former A i Each scheduling requirement needs to be responded to while the delay needs to be accumulated to L i The details are as follows:

[0073]

[0074] Each demand needs to calculate the response time slot when it is responded to. and generate time slots The difference between n The delay after the selected response. The value range of n is [1, A i], the scheduling demand delay taken out of each time slot needs to be accumulated.

[0075] Step 4: Propose a demand queue decision problem based on the Markov decision process, use the deep reinforcement learning algorithm to solve the problem, comprehensively consider the response difference and delay comparison, make decisions on the dispatch demand response of the original data, and pursue efficient response to the dispatch demand. The details are as follows:

[0076] Combination Figure 3 , in each time slot, all media will generate scheduling requirements. In order to simplify the response process, only one demand queue can be selected to respond. Therefore, it is necessary to consider the number of demands and response differences of each demand queue, and comprehensively select the appropriate demand queue to respond. The decision process of the demand queue is regarded as a Markov model, and K = (State, Action, Reward) represents the Markov decision process. Among them, State represents the state space, Action represents the action space, and Reward represents the reward function. The environment of Markov decision is the in-situ data storage and scheduling system under the smart grid. The process is defined as follows:

[0077] State: In the in-situ data multi-level storage scheduling environment, the state is a real-time representation of the state of each medium demand queue in the time slot. It is a three-dimensional vector composed of the current average demand margin, average delay, and average demand satisfaction ratio, and is specifically expressed as:

[0078] State = [ARR, AD, DSR] (19)

[0079] ARR represents the average demand margin of all demand queues in each time slot. For the k medium demand queues that form a multi-level structure, the demand margin is calculated separately, that is, the maximum capacity M of the demand queue k and the remaining capacity of the demand queue R k The difference. All medium demand margins are accumulated and averaged, and its specific definition is shown in formula (20). AD represents the average delay of all demand queues in each time slot. The delays of k demand queues are accumulated and averaged, and its specific definition is shown in formula (21). DAR represents the demand satisfaction ratio of all demand queues in each time slot. The number of all responded demands and the total number of demands generated in each queue are accumulated separately, and the ratio of the two is calculated, and its specific definition is shown in formula (22).

[0080]

[0081] Action: In the context of multi-level storage scheduling of in-situ data, an action is to select a scheduling demand queue to respond in each time slot, which can be defined as:

[0082] Action∈{Que1, Que2, Que3} (23)

[0083] Reward: In the context of multi-level storage scheduling of in-situ data, the reward setting aims to reduce the average demand margin ARR and minimize the average delay AD. α and β are used as adjustment coefficients to balance the weights of demand and delay, which can be defined as:

[0084] Reward=-(α×ARR+β×AD) (24)

[0085] The present invention uses the deep Q network (DQN) in deep reinforcement learning to solve the scheduling demand queue decision of the in-situ data in the smart grid environment. The algorithm pseudo code can be defined as:

[0086]

[0087] The above content describes the implementation process and advantages of the present invention. Those skilled in the art should understand that the present invention may have various changes and improvements without departing from the principles of the present invention, and these changes and improvements fall within the scope of the present invention claimed for protection.

Claims

1. An in-situ data storage scheduling optimization method in a smart grid scenario, characterized in that: The following steps are involved: Step 1: Establish a multi-level storage structure model; Step 2: Establish a personalized scheduling queue model; Step 3: Propose the optimization problem of in-situ data storage scheduling in the smart grid scenario; Step 4: Propose a demand queue decision problem based on the Markov decision process, use the deep reinforcement learning algorithm to solve the problem, comprehensively consider the response difference and delay comparison, make decisions on the scheduling demand response of the original data, and pursue efficient response to the scheduling demand.

2. The in-situ data storage scheduling optimization method in the smart grid scenario described in claim 1 is characterized in that: The multi-level storage structure model described in step 1 is as follows: Assume that a medium can be represented as a set Med i ={C i , A i , D i }, which encapsulates the various properties of the medium. i Used to indicate the medium Med i Storage capacity. i Indicates that the response medium Med is selected i The corresponding scheduling demand queue is the maximum number of demands that the corresponding demand queue can respond to in a single time slot. In each time slot, all media will generate scheduling demands and use the scheduling demand queue to record them. i The resulting dispatch demand X is distributed by Poisson distribution X~Poisson(D i ) OK, D i represents the mean of the Poisson distribution, e is the base of the natural logarithm, and x! represents the factorial of x. The details are as follows: In the smart grid environment, in order to store the in-situ data of terminal devices, independent servers are set up near the data source, and all servers constitute the in-situ server system (InS). Traditional storage structures often use a single medium to store in-situ data, which can be represented by a set as Structure traditional storage ={Med}, where Med ={C, A, D}. As the in-situ data types in scenarios become increasingly diversified, it is difficult for the original storage structure to provide targeted storage services based on data characteristics such as type, size, and update rate. The present invention considers three types of media and designs a multi-level storage structure: high-frequency storage medium Med1 = {C1, A1, D1}, medium-frequency storage medium Med2 = {C2, A2, D2}, and low-frequency storage medium Med3 = {C3, A3, D3}. The three types of media meet the following constraints: C1<C2<C3, A1>A2>A3, D1>D2>D3 (2) High-frequency media will generate more demands, and the number of demands selected for response in each time slot is relatively large, but the storage capacity is relatively small; on the contrary, low-frequency media will generate relatively few demands, and the number of demands selected for response in each time slot is relatively small, but it has a larger storage capacity. multi-level storage ={Med1,Med2,Med3}, and satisfy the following constraints: C=C1+C2+C3 (3) The multi-level storage structure can achieve targeted storage based on factors such as the access frequency and data specifications of the original data under the premise of limited storage capacity, thereby greatly improving storage efficiency.

3. The in-situ data storage scheduling optimization method in the smart grid scenario described in claim 1 is characterized in that: The personalized scheduling queue model described in step 2 is established as follows: Set a demand queue can be represented as a set Que i ={M i , R i , N i ,L i }, used to record the corresponding medium Med i The scheduling requirements generated by the request also encapsulate the various attributes of the request queue. i Used to represent the demand queue Que i The maximum required storage capacity will be determined based on the corresponding medium Med i Storage capacity C i Dynamically plan the capacity of the queue to avoid wasting resources. i Used to represent the demand queue Que i The remaining required capacity that can be stored. N i Used to represent a queue Que i The number of times the response was selected. i Used to represent the demand queue Que i The accumulated delay after the demand in is selected for response. The traditional structure uses a single queue to record scheduling requirements, which can be expressed as Structure traditional schedule = {Que}, where Que = {M, R, N, L}. Each time when responding to the scheduling demand generated by the original data, only the quantitative demand generated by a single scheduling demand queue can be responded to. With the introduction of multi-level storage structure, if the traditional structure is still used to record the demand, the overall response process is relatively cumbersome, and there are a series of problems such as low access efficiency and slow response speed. For the multi-level storage structure, this research powder designs three scheduling demand queues: high-frequency medium scheduling demand queue Que1 = {M1, R1, N1, L1}, medium-frequency medium scheduling demand queue Que2 = {M2, R2, N2, L2}, low-frequency medium scheduling demand queue Que3 = {M3, R3, N3, L3}. The three scheduling demand queues meet the following constraints: M1<M2<M3 (4) The high-frequency medium scheduling demand queue has a relatively small maximum demand capacity because the corresponding medium has a faster response rate; the low-frequency medium scheduling demand queue is relatively large. Use three scheduling demand combinations to design a personalized scheduling structure personalised schedule = {Que1, Que2, Que3}, and satisfy the following constraints: M=M1+M2+M3 (5) Personalized scheduling queues can record the in-situ data scheduling demands generated by different media in a targeted manner without changing the constraints of the original demand capacity. This eliminates the need to determine the source media of the scheduling demands, greatly simplifying the steps and processes of demand response. The demand queue is used to store the scheduling requirements generated by the corresponding medium. A scheduling requirement can be expressed as a set Where Dem j Represents the jth demand in the current scheduling demand queue. Indicates the number of time slots generated by the scheduling demand, which will be initialized in the generated time slot; Indicates the number of time slots that the scheduling demand is responded to. The default value is null and will not be assigned until it is responded to. Indicates the number of time slots that the scheduling requirement has been waiting for in the current time slot.

4. The in-situ data storage scheduling optimization method in the smart grid scenario described in claim 1 is characterized in that: The optimization problem of in-situ data storage scheduling in the smart grid scenario described in step 3 is as follows: In each time slot, all media will generate scheduling demands according to the Poisson distribution, and the specific quantity meets the following constraints: Fish(D i )≤R i ≤M i (6) The required remaining capacity R of the medium i Always less than or equal to the maximum required capacity of the medium M i When the actual number of scheduling requirements generated meets the above constraints, the generated requirements can be loaded into the corresponding demand queue Que i Otherwise, keep the demand generated this time until it meets the conditions and then load it into the demand queue. In each time slot, in order to simplify the response process and improve the response rate, select a scheduling demand corresponding to a medium to respond. i After that, you need to use the corresponding medium Med i The maximum number of requests that can be responded to in a single time slot A i , to respond to Que in turn i The former A i Each scheduling requirement needs to be responded to while the delay needs to be accumulated to L i The details are as follows: Each demand needs to calculate the response time slot when it is responded to. and generate time slots The difference between n The delay after the selected response. The value range of n is [1, A i ], the scheduling demand delay taken out of each time slot needs to be accumulated.

5. The in-situ data storage scheduling optimization method in the smart grid scenario described in claim 1 is characterized in that: Step 4 describes the problem of demand queue decision-making based on the Markov decision process. The deep reinforcement learning algorithm is used to comprehensively consider the response difference and delay comparison, and the decision is made on the dispatch demand response of the original data, so as to pursue the efficient response of the dispatch demand. The details are as follows: In each time slot, all media will generate scheduling requirements. To simplify the response process, only one demand queue can be selected to respond. Therefore, it is necessary to consider the number of demands and response differences of each demand queue, and comprehensively select the appropriate demand queue to respond. The decision process of the demand queue is regarded as a Markov model, and K = (State, Action, Reward) represents the Markov decision process. Among them, State represents the state space, Action represents the action space, and Reward represents the reward function. The environment of Markov decision is the in-situ data storage and scheduling system under the smart grid. The process is defined as follows: State: In the in-situ data multi-level storage scheduling environment, the state is a real-time representation of the state of each medium demand queue in the time slot. It is a three-dimensional vector composed of the current average demand margin, average delay, and average demand satisfaction ratio, and is specifically expressed as: State=[ARR,AD,DSR] (7) ARR represents the average demand margin of all demand queues in each time slot. For the k medium demand queues that form a multi-level structure, the demand margin is calculated separately, that is, the maximum capacity M of the demand queue k and the remaining capacity of the demand queue R k The difference. All medium demand margins are accumulated and averaged, and its specific definition is shown in formula (8). AD represents the average delay of all demand queues in each time slot. The delays of k demand queues are accumulated and averaged, and its specific definition is shown in formula (9). DAR represents the demand satisfaction ratio of all demand queues in each time slot. The number of all responded demands and the total number of generated demands in each queue are accumulated respectively, and the ratio of the two is calculated, and its specific definition is shown in formula (10). Action: In the context of multi-level storage scheduling of in-situ data, an action is to select a scheduling demand queue to respond in each time slot, which can be defined as: Action∈{Que1, Que2, Que3} (11) Reward: In the context of multi-level storage scheduling of in-situ data, the reward setting aims to reduce the average demand margin ARR and minimize the average delay AD. α and β are used as adjustment coefficients to balance the weights of demand and delay, which can be defined as: Reward=-(α×ARR+β×AD) (12) The present invention uses the deep Q network (DQN) in deep reinforcement learning to solve the scheduling demand queue decision of the in-situ data in the smart grid environment. The algorithm pseudo code can be defined as:

Images