Power supply scheme determination method and device, computer device, and storage medium

Abstract

The application relates to a power supply scheme determination method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining an initial power supply scheme in a second time period of a target power consumption area according to target power consumption data in a first time period of the target power consumption area and a preset double-depth reinforcement learning network (DDQN) model, the first time period being earlier than the second time period; issuing the initial power supply scheme to a terminal corresponding to the target power consumption area; obtaining real-time power consumption data generated by the target power consumption area in the second time period based on the initial power supply scheme; optimizing the initial power supply scheme according to the real-time power consumption data; and obtaining a power supply scheme of the target power consumption area in a third time period, the second time period being earlier than the third time period. The method can improve the accuracy of the formulated power supply scheme.

Description

Technical Field

[0001] This application relates to the field of power grid control technology, and in particular to a method, apparatus, computer equipment, and storage medium for determining a power supply scheme. Background Technology

[0002] As the number of electricity users increases, the number of users supplied by the power grid also increases. Therefore, it is particularly important for the power grid to formulate a reasonable power supply plan to ensure that it can provide stable power resources to users in order to meet their electricity demand.

[0003] In traditional technologies, the power grid side primarily obtains historical electricity consumption data from users within a designated area and formulates power supply plans based on this data. However, traditional methods suffer from low accuracy in producing such power supply plans. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for determining a power supply scheme that can improve the accuracy of the formulated power supply scheme, in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for determining a power supply scheme. The method includes:

[0006] Based on the target electricity consumption data of the target electricity consumption area in the first time period and the preset dual deep reinforcement learning network (DDQN) model, the initial power supply scheme for the target electricity consumption area in the second time period is obtained; the first time period is earlier than the second time period.

[0007] The initial power supply scheme is sent to the terminal corresponding to the target power consumption area, and the real-time power consumption data generated by the target power consumption area based on the initial power supply scheme during the second time period is obtained;

[0008] The initial power supply scheme is optimized based on the real-time power consumption data to obtain the power supply scheme for the target power consumption area in the third time period; the second time period is earlier than the third time period.

[0009] In one embodiment, optimizing the initial power supply scheme based on the real-time power consumption data to obtain the power supply scheme for the target power consumption area in the third time period includes:

[0010] Based on the real-time power consumption data, the adjustment values ​​of the power supply parameters in the initial power supply scheme are determined; wherein, the power supply parameters include at least one of power supply frequency, power supply peak value, and power supply power;

[0011] The initial power supply scheme is adjusted according to the adjustment value to obtain the power supply scheme for the target power consumption area during the third time period.

[0012] In one embodiment, determining the adjustment value of the power supply parameters in the initial power supply scheme based on the real-time power consumption data includes:

[0013] Based on the real-time electricity consumption data, the load baseline of the target electricity consumption area during the second time period is obtained;

[0014] The adjustment value is determined based on the load baseline.

[0015] In one embodiment, the method further includes:

[0016] Based on the identifier of the target electricity consumption area, the target electricity consumption data for the first time period is obtained from a preset database.

[0017] In one embodiment, the database creation process includes:

[0018] Real-time collection of electricity consumption data from various electricity-consuming areas;

[0019] The electricity consumption data of each of the aforementioned electricity consumption areas are subjected to Tow regression and clustering processing to obtain the processed electricity consumption data of each of the aforementioned electricity consumption areas;

[0020] The database is established based on the correspondence between the identifiers of each electricity consumption area and the processed electricity consumption data of each electricity consumption area.

[0021] In one embodiment, the method further includes:

[0022] Retrieve historical electricity consumption data for the target electricity consumption area from the database;

[0023] The DDQN model is obtained by training a preset initial DDQN model using a reinforcement learning algorithm and historical electricity consumption data of the target electricity consumption area.

[0024] Secondly, this application also provides a device for determining a power supply scheme. The device includes:

[0025] The first acquisition module is used to acquire the initial power supply scheme for the target power consumption area in the second time period based on the target power consumption data in the first time period of the target power consumption area and the preset dual deep reinforcement learning network (DDQN) model; the first time period is earlier than the second time period.

[0026] The second acquisition module is used to send the initial power supply scheme to the terminal corresponding to the target power consumption area, and to acquire the real-time power consumption data generated by the target power consumption area based on the initial power supply scheme during the second time period.

[0027] The third acquisition module is used to optimize the initial power supply scheme based on the real-time power consumption data and determine the power supply scheme for the target power consumption area in the third time period; the second time period is earlier than the third time period.

[0028] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described in the first aspect.

[0029] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the method described in the first aspect.

[0030] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the method described in the first aspect.

[0031] The aforementioned method, apparatus, computer equipment, and storage medium for determining the power supply scheme, based on the target power consumption data of the target power consumption area in the first time period and a preset dual deep reinforcement learning network (DDQN) model, can obtain the initial power supply scheme for the target power consumption area in the second time period. This initial power supply scheme can then be distributed to the terminals corresponding to the target power consumption area, allowing these terminals to adjust the power consumption scheme for the target area in the second time period. Furthermore, real-time power consumption data generated by the target power consumption area based on the initial power supply scheme in the second time period can be obtained. Based on this real-time power consumption data, the initial power supply scheme can be optimized to obtain the power supply scheme for the target area in the third time period. Compared to traditional technologies, this method acquires real-time power consumption data generated based on an initial power supply scheme and optimizes the initial power supply scheme for the target power consumption area based on the acquired real-time power consumption data, thereby obtaining an optimized power supply scheme. This process involves continuous optimization of the power supply scheme, avoiding the problem of power supply schemes determined based on historical power consumption data not conforming to actual power consumption conditions. This ensures that the determined power supply scheme is closer to the actual power consumption conditions of the target power consumption area, improving the accuracy of the determined power supply scheme. Furthermore, since this application determines the initial power supply scheme based on the DDQN model, which has higher output accuracy, it can improve the accuracy of the determined initial power supply scheme, thereby improving the accuracy of the optimized power supply scheme based on the initial power supply scheme. Attached Figure Description

[0032] Figure 1 This is an application environment diagram of a method for determining a power supply scheme in one embodiment;

[0033] Figure 2 This is a flowchart illustrating a method for determining a power supply scheme in one embodiment;

[0034] Figure 3 This is a flowchart illustrating the method for determining the power supply scheme in another embodiment;

[0035] Figure 4 This is a flowchart illustrating the method for determining the power supply scheme in another embodiment;

[0036] Figure 5 This is a flowchart illustrating the method for determining the power supply scheme in another embodiment;

[0037] Figure 6 This is a flowchart illustrating the method for determining the power supply scheme in another embodiment;

[0038] Figure 7 This is a structural block diagram of a device for determining a power supply scheme in one embodiment;

[0039] Figure 8A structural block diagram of the power supply scheme determination device in another embodiment;

[0040] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] The method for determining the power supply scheme provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. Terminal 102 is a user-side terminal device, and server 104 is a power grid-side server device. The data storage system can store the data that server 104 needs to process. The data storage system can be integrated on server 104 or placed on the cloud or other network servers. Terminal 102 can be, but is not limited to, various personal computers, laptops, tablets, and IoT devices, and server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0043] It should be noted that the server in this application is used in an electricity demand response platform. The electricity demand response platform serves as the carrier of the server, and can control and manage the electricity consumption data stored in the server, as well as receive real-time electricity consumption data sent by the electricity consumer.

[0044] In one embodiment, such as Figure 2 As shown, a method for determining a power supply scheme is provided, which is then applied to... Figure 1 Taking the server in the example, the following steps are included:

[0045] S201, based on the target power consumption data of the target power consumption area in the first time period and the preset dual deep reinforcement learning network DDQN model, obtain the initial power supply scheme of the target power consumption area in the second time period; the first time period is earlier than the second time period.

[0046] The target electricity consumption area refers to the area for which a power supply plan needs to be determined based on the electricity consumption data. For example, the target electricity consumption area could be a residential community or a park, etc. Optionally, the first time period can be the day before the current time or three days before the current time, and the second time period can be the current time or the day after the current time. The first time period must be earlier than the second time period and the time difference between them must be short. For example, if the first time period is hours 0-24, then the second time period could be hours 25-48. The target electricity consumption data refers to the electricity consumption data of the target electricity consumption area within the first time period. The preset dual-deep reinforcement learning network model refers to a model obtained by combining reinforcement learning (RL) and deep neural networks (NN) (Double Deep Q-Learning Network, DDQN).

[0047] In this embodiment, the server on the power grid side can communicate with the terminal on the user side to obtain the target power consumption data of the target power consumption area in the first time period. The obtained target power consumption data is input into a preset DDQN model to obtain the output result of the DDQN model. Based on the output result, the power consumption of the target area in the second time period can be determined, and the initial power supply scheme of the target power supply area in the second time period can be obtained based on the determined power consumption.

[0048] S202, the initial power supply plan is sent to the terminal corresponding to the target power consumption area, and the real-time power consumption data generated by the target power consumption area in the second time period based on the initial power supply plan is obtained.

[0049] Among them, the terminal refers to the equipment on the power consumption side used to receive the power supply plan sent by the power grid side and adjust the power consumption plan; the real-time power consumption data of the target power consumption area refers to the power data generated by the actual power consumption after the power consumption side adjusts the power consumption plan according to the received power supply plan.

[0050] In this embodiment, the server on the power grid side can send an initial power supply plan to the terminal corresponding to the target power consumption area. After receiving the initial power supply plan from the power grid side, the terminal can adjust its power consumption plan according to the initial power supply plan. After the power consumption plan is adjusted, it collects real-time power consumption data of the power consumption side within a second time period through acquisition devices such as current sensors and voltage sensors, and sends the collected real-time power consumption data to the server on the power grid side. For example, the initial power supply plan may include parameters such as current value, voltage value, power supply, power supply, and line loss that can be provided by the power grid side. The terminal can adjust the current value, voltage value, power consumption, and other parameters of each power-consuming device on the power consumption side according to the received initial power supply plan, and collect the actual power consumption data after adjustment and send it to the server on the power grid side.

[0051] S203, optimize the initial power supply scheme based on real-time power consumption data to obtain the power supply scheme for the target power consumption area in the third time period; the second time period is earlier than the third time period.

[0052] The third time period refers to the period following the second time period. In this embodiment, the grid-side server can input the acquired real-time electricity consumption data into a preset DDQN model, determine a new power supply scheme based on the model's output, and optimize the initial power supply scheme based on the new scheme. The optimized power supply scheme will then be used as the power supply scheme for the third time period. For example, the grid-side server can input the acquired real-time electricity consumption data from hour 0 to 24 into the preset DDQN model, use the output as the power supply scheme for hour 25 to 48, and determine the power supply scheme for hour 49 to 72 based on the acquired electricity consumption data from hour 25 to 48, thus ensuring the real-time performance of each power supply scheme.

[0053] The aforementioned method for determining the power supply scheme, based on the target power consumption data of the target power consumption area in the first time period and a pre-set dual deep reinforcement learning network (DDQN) model, can obtain the initial power supply scheme for the target power consumption area in the second time period. This initial power supply scheme can then be distributed to the terminals corresponding to the target power consumption area, allowing these terminals to adjust the power consumption scheme for the target power consumption area in the second time period. Furthermore, it can obtain real-time power consumption data generated by the target power consumption area based on the initial power supply scheme in the second time period. Based on this real-time power consumption data, the initial power supply scheme can be optimized to obtain the power supply scheme for the target power consumption area in the third time period. Compared to traditional technologies, this method… This method can acquire real-time power consumption data generated based on the initial power supply scheme, and optimize the initial power supply scheme for the target power consumption area based on the acquired real-time power consumption data, thereby obtaining an optimized power supply scheme. This process involves continuous optimization of the power supply scheme, avoiding the problem that power supply schemes determined based on historical power consumption data do not match actual power consumption conditions. This ensures that the determined power supply scheme is closer to the actual power consumption conditions of the target power consumption area, improving the accuracy of the determined power supply scheme. Furthermore, since this application determines the initial power supply scheme based on the DDQN model, which has higher output accuracy, it can improve the accuracy of the determined initial power supply scheme, thereby improving the accuracy of the optimized power supply scheme based on the initial power supply scheme.

[0054] In the scenario described above, where the power supply plan for the target electricity consumption area during the third time period is obtained, the initial power supply plan can be optimized based on real-time electricity consumption data to obtain the power supply plan for the third time period. In one embodiment, such as Figure 3 As shown, the above S203 includes:

[0055] S301, Based on real-time power consumption data, determine the adjustment values ​​of the power supply parameters in the initial power supply scheme; wherein, the power supply parameters include at least one of power supply frequency, power supply peak value and power supply power.

[0056] Optionally, power supply parameters may include power supply frequency, peak power supply value, power supply power, power supply quantity, line loss, energy storage power, transformer capacity, total load, and load for each time period. The adjustment value of the power supply parameters refers to the value that needs to be increased or decreased for the above power supply parameters.

[0057] In this embodiment, the server on the grid side can calculate the power data required from the grid side based on real-time power consumption data. It then compares the calculated power data with the power data in the initial power supply plan, and determines the parameters that need adjustment in the initial power supply plan, as well as the magnitude of the adjustment, based on the comparison result. For example, if the calculated power supply frequency is inconsistent with the power supply frequency in the initial power supply plan, the power supply frequency in the initial power supply plan needs to be adjusted; or, if the calculated power supply peak value is inconsistent with the power supply peak value in the initial power supply plan, the power supply peak value in the initial power supply plan needs to be adjusted; or, if the calculated power supply power is inconsistent with the power supply power in the initial power supply plan, the power supply power in the initial power supply plan needs to be adjusted.

[0058] In an optional embodiment, the grid-side server can also compare and display the acquired real-time power consumption data with the power data in the initial power supply plan on the power demand response platform, thereby quickly and intuitively determining the power supply parameters that need to be adjusted in the initial power supply plan.

[0059] S302, adjust the initial power supply scheme according to the adjustment value to obtain the power supply scheme for the target power consumption area in the third time period.

[0060] In this embodiment, the initial power supply scheme can be adjusted according to the determined adjustment value, and the adjusted power supply scheme can be used as the power supply scheme for the target power consumption area in the third time period. For example, the peak value of the voltage provided in the initial power supply scheme is 620V. According to the real-time power consumption data, the adjustment value is determined to be -20V, which means that the peak value of the voltage provided by the grid side can be reduced. Therefore, the peak value of the provided voltage is adjusted to 600V according to the adjustment value.

[0061] In this embodiment, the adjustment values ​​of the power supply parameters in the initial power supply scheme can be determined based on real-time power consumption data. The initial power supply scheme can be adjusted according to the adjustment values ​​to obtain the power supply scheme for the target power consumption area in the third time period. Since the adjustment values ​​of the power supply parameters are determined by real-time power consumption data, the determined adjustment values ​​of the power supply parameters are closer to the actual power consumption situation, thereby obtaining more accurate power supply parameters and improving the power supply scheme in the third time period obtained based on more accurate power supply parameters.

[0062] In the scenario described above where the adjustment values ​​for power supply parameters are determined, these values ​​can be determined based on the load baseline of the target power consumption area. In one embodiment, such as... Figure 4 As shown, the above S301 includes:

[0063] S401, based on real-time electricity consumption data, obtains the load baseline of the target electricity consumption area in the second time period.

[0064] The load baseline refers to a load curve estimated based on historical load data from the user side. It should be noted that the average load of the hour with the highest load in a 24-hour period is usually selected as the peak load. The peak load is the sum of the electrical power used by various electrical devices in the power system at a certain moment.

[0065] In this embodiment, the server on the power grid side can statistically analyze electricity metering data based on real-time electricity consumption data to obtain electricity load data and total load data for each time period. Based on this load data, the server can then obtain the load baseline for the target electricity consumption area within the second time period. For example, if the load data from 19:00 to 20:00 within the second time period represents the highest usage load during that period, then the load data for that time period can be used as the load baseline for the target area within the second time period.

[0066] S402, determine the adjustment value based on the load baseline.

[0067] In this embodiment, the load baseline of the acquired real-time power consumption data can be compared with the power data in the initial power supply scheme, and the adjustment value of the power supply parameters that need to be adjusted in the initial power supply scheme can be determined based on the comparison result.

[0068] In this embodiment, the load baseline of the target power consumption area in the second time period can be obtained based on real-time power consumption data. Based on the load baseline, the adjustment value can be determined. Since the adjustment value is determined based on the obtained real-time power consumption data, the accuracy of the determined adjustment value is guaranteed, thereby improving the accuracy of the power supply scheme determined based on the adjustment value.

[0069] In the scenario described above for determining the initial power supply scheme, target power consumption data for the target power consumption area can be obtained from a preset database. In one embodiment, the method further includes: obtaining target power consumption data for a first time period from the preset database based on the identifier of the target power consumption area.

[0070] The identifier for the target electricity consumption area can be either the name or the number of the electricity consumption area. The preset database is a database built based on the electricity consumption data of the electricity consumption area for each time period prior to the current time. The data structure of the database can include the name and number of each electricity consumption area, as well as the electricity consumption data.

[0071] In this embodiment, the electricity consumption data corresponding to the identifier of the target electricity consumption area can be queried in a preset database according to the identifier of the target electricity consumption area, and the target electricity consumption data within the first time period can be obtained from the queried electricity consumption data.

[0072] In this embodiment, based on the identifier of the target power consumption area, the target power consumption data within the first time period can be obtained from the preset database, avoiding the acquisition of power consumption data from other power consumption areas, thereby ensuring the accuracy of the acquired target power consumption data, and thus ensuring the accuracy of the obtained initial power supply scheme.

[0073] In the scenario described above, where target electricity consumption data is obtained from a pre-set database, the database can be established using real-time collected electricity consumption data. In one embodiment, such as... Figure 5 As shown, the process of establishing the above database includes:

[0074] S501 collects electricity consumption data from various electricity consumption areas in real time.

[0075] Optionally, in this embodiment, the server on the power grid side can periodically send instructions to the terminals in each power consumption area on the power consumption side to collect power consumption data, or the terminals in each power consumption area on the power consumption side can actively send real-time power consumption data to the server on the power grid side, thereby obtaining real-time power consumption data for each power consumption area.

[0076] S502, perform lasso regression and clustering processing on the electricity consumption data of each electricity consumption area to obtain the processed electricity consumption data of each electricity consumption area.

[0077] Lasso regression (Least Absolute Shrinkage and Selection Operator, LASSO) is a type of compression estimation that constructs a penalty function to obtain a more refined model, allowing some regression coefficients to be compressed—that is, forcing the sum of the absolute values ​​of the coefficients to be less than a certain fixed value; simultaneously, setting some regression coefficients to zero. It is a biased estimation method for handling data with multicollinearity. Clustering refers to a data processing method that clusters and merges neighboring, similar classification regions.

[0078] In this embodiment, the collected electricity consumption data from each electricity consumption area can be standardized and unified, and then further processed using LASSO regression and clustering to obtain the processed electricity consumption data for each area. For example, the Min-max standardization method can be used to standardize the collected electricity consumption data from each area. The formula for the Min-max standardization method is shown below:

[0079] x ‘ =(x-MinA) / (MaxA-MinA),x ′ ∈[0,1]

[0080] In the formula, x ′These are the standardized electricity consumption data. x is the electricity consumption data, MaxA is the maximum value in the electricity consumption data, and MinA is the minimum value in the electricity consumption data.

[0081] S503, establish a database based on the identification of each power consumption area and the correspondence between the processed power consumption data of each power consumption area.

[0082] Optionally, in this embodiment, each power consumption area can be numbered, and the number can be used as the identifier of each power consumption area. A correspondence between the number of each power consumption area and the processed power consumption data can be established, and a database can be built based on the established correspondence. Alternatively, the name of each power consumption area can be used as the identifier of each power consumption area, and a correspondence between the name of each power consumption area and the processed power consumption data can be established, and a database can be built based on the established correspondence.

[0083] In this embodiment, electricity consumption data of each power consumption area is collected in real time, and lasso regression and clustering processing are performed on the electricity consumption data of each power consumption area to obtain the processed electricity consumption data of each power consumption area. Based on the correspondence between the identifier of each power consumption area and the processed electricity consumption data of each power consumption area, a database is established. The established database can be imported into a preset DDQN model, and then the power supply scheme of each power consumption area can be quickly and accurately determined based on the preset DDQN model and the real-time electricity consumption data of each power consumption area, thereby improving the accuracy of the determined power supply scheme of each power consumption area.

[0084] In the scenario described above where a power supply scheme is obtained based on a preset DDQN model, the DDQN model can be obtained by training the preset initial DDQN model. In one embodiment, such as... Figure 6 As shown, the above method also includes:

[0085] S601, retrieve historical electricity consumption data for the target electricity consumption area from the database.

[0086] Historical electricity consumption data refers to electricity consumption data generated by the user side within a time period earlier than the first time period. In this embodiment, historical electricity consumption data of the target electricity consumption area within a historical time period can be obtained from the database using time and electricity consumption area identifiers as indexes.

[0087] S602, using reinforcement learning algorithm and historical electricity consumption data of the target electricity consumption area, trains the preset initial DDQN model to obtain the DDQN model.

[0088] Reinforcement learning algorithms can be value-based, policy-based, or actor-critic methods. Reinforcement learning is the third type of machine learning method, alongside supervised and unsupervised learning. It possesses the following key elements:

[0089] The first is the state of the environment.

[0090] The second is the action of an individual.

[0091] The third is environmental reward, which is the reward corresponding to the action taken by an individual in a certain state.

[0092] The fourth is the individual's strategy, which represents the basis for an individual to take a certain action. That is, the individual will choose an action based on the strategy. The most common way to express the strategy is a conditional probability distribution, which is the probability of taking a certain action in a certain state. In other words, the more probable the job, the higher the probability that the individual will choose.

[0093] The fifth factor is the value of an individual's actions based on different strategies and states. This value is typically represented by an expected value function. For example, an individual might receive a delayed reward for choosing a certain action, but a high delayed reward at the moment of action does not guarantee a high delayed reward at a later time. Therefore, the value factor needs to consider both the current delayed reward and the subsequent delayed rewards. Value function Q π It can be expressed as follows:

[0094] Q π (S t ,a t )=E[(R t +γR t+1 +γ 2 R t+2 +γ 3 R t+3 +…) / St=s t A t =a t ]

[0095] In the formula, S t As the current state, a t π represents the current action, and π is the policy function.

[0096] The sixth factor is the reward decay factor, which ranges from [0, 1]. If the value is 0, it's a greedy algorithm, meaning the value is determined only by the current delayed reward; if the value is 1, it assumes that the delayed rewards of all subsequent states are the same as the current reward. Typically, the reward decay factor takes a value between 0 and 1, meaning the weight of the current delayed reward is greater than the weight of the delayed rewards of subsequent states.

[0097] The seventh is the environmental state transition model, which can be represented by a probability model, that is, the probability of taking a certain action in a certain state and transitioning to the next state.

[0098] The eighth is the exploration rate. In the training iteration process of reinforcement learning, when selecting the optimal action, there is a certain probability that the action with the highest value in the current iteration will not be selected, but other actions will be selected instead.

[0099] The difference between the DDQN model and other models is that the objective Q-value of other models is obtained through a greedy method. Although this can make the Q-value quickly approach the optimization objective, it is prone to overestimation, that is, the final model has a large bias. Therefore, the DDQN model can eliminate the overestimation problem by decoupling the selection of the objective Q-value action from the objective Q-value.

[0100] Therefore, in this application, the DDQN model is used as a prediction model for predicting electricity consumption data on the electricity consumption side in the power grid resource application scenario, and the initial DDQN model is iteratively optimized based on historical electricity consumption data to obtain a trained DDQN model.

[0101] In this embodiment, historical electricity consumption data of the target electricity consumption area can be used as input data for a preset initial DDQN model. The Q value of reinforcement learning can be determined based on the output of the model, and then the parameters of the initial DDQN model can be adjusted based on the Q value to obtain a trained DDQN model.

[0102] The training process of the DDQN model is as follows:

[0103] S1, select the parameters of the initial DDQN model, such as: number of iterations T, state feature dimension n, action set A, step size α, decay factor γα, exploration rate ε, Q of the current Q network, Q' of the target Q network, number of samples for batch gradient descent, and parameter update frequency c of the target Q network.

[0104] S2, randomly initialize the values ​​corresponding to all states and actions.

[0105] S3, select the number of iteration rounds and perform iterations, wherein the iteration process is as follows:

[0106] S31, initialize the first state of the current state sequence and obtain its feature vector.

[0107] S32, input the electricity consumption data of the first historical time period into the Q network, and obtain the Q value corresponding to all actions of the Q network.

[0108] S33, use a greedy algorithm to select the corresponding action from the current output Q value, and determine whether the termination state is end.

[0109] S34, execute the current action in the new state to obtain the feature vector and reward corresponding to the new state.

[0110] S35, after executing the above step S34, put the parameters {φ(s), A, R, φ(s′), is end} into the experience replay set.

[0111] S36, Sample m samples from the experience replay set to calculate the current target Q value:

[0112] {φ(s), A, R, φ(s′), is end}, j = 1, 2, ..., m, calculate the current target Q value y j :

[0113]

[0114] S37, using the mean squared error loss function All parameters of the Q-network are updated through backpropagation of the gradients of the neural network.

[0115] S38. If it is not in a terminated state, update the target Q network parameters by setting ω' = ω.

[0116] S39: If it is in a terminated state, the current iteration is complete; otherwise, proceed to execute S33 above.

[0117] In this embodiment, historical electricity consumption data of the target electricity consumption area is obtained from the database. A preset initial DDQN model is trained using reinforcement learning algorithm and the historical electricity consumption data of the target electricity consumption area to obtain a DDQN model. Since the initial DDQN model trained with the historical electricity consumption data of the target electricity consumption area can adjust the model parameters more quickly and accurately according to the model output results, a DDQN model that is closer to the electricity consumption situation of the target electricity consumption area can be obtained. Then, the trained DDQN model is used to determine the power supply scheme of the target power supply area, which can further improve the accuracy of the power supply scheme of the target power supply area.

[0118] To facilitate understanding by those skilled in the art, the method for determining the power supply scheme provided in this application will be described in detail below. This method may include:

[0119] S1 collects electricity consumption data from each electricity consumption area in real time, performs LASSO processing and clustering processing on the electricity consumption data from each electricity consumption area, and obtains the processed electricity consumption data.

[0120] S2, establish a database based on the processed electricity consumption data.

[0121] S3: Obtain historical electricity consumption data for the target electricity consumption area from the established data.

[0122] S4. Use the acquired historical electricity consumption data to train the preset initial DDQN model to obtain the trained DDQN model.

[0123] S5: Based on the identifier of the target electricity consumption area, retrieve the target electricity consumption data for the first time period of the target electricity consumption area from the established database.

[0124] S6. Based on the target power consumption data of the target power consumption area in the first time period and the trained DDQN model, obtain the initial power supply scheme for the target power consumption area in the second time period.

[0125] S7 sends the initial power supply plan to the terminal corresponding to the target power consumption area.

[0126] S7, Obtain real-time power consumption data of the target power consumption area generated in the second time period based on the initial power supply scheme.

[0127] S9, based on the acquired real-time power consumption data, determines the adjustment values ​​of the power supply parameters in the initial power supply scheme.

[0128] S10, adjust the initial power supply scheme according to the adjustment value to obtain the power supply scheme for the target power consumption area in the third time period.

[0129] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0130] Based on the same inventive concept, this application also provides a power supply scheme determination apparatus for implementing the power supply scheme determination method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more power supply scheme determination apparatus embodiments provided below can be found in the limitations of the power supply scheme determination method described above, and will not be repeated here.

[0131] In one embodiment, such as Figure 7As shown, a power supply scheme determination device is provided, comprising: a first acquisition module 10, a second acquisition module 11, and a third acquisition module 12, wherein:

[0132] The first acquisition module 10 is used to acquire the initial power supply scheme for the target power consumption area in the second time period based on the target power consumption data in the first time period of the target power consumption area and the preset dual deep reinforcement learning network model; the first time period is earlier than the second time period.

[0133] The second acquisition module 11 is used to send the initial power supply scheme to the terminal corresponding to the target power consumption area, and to acquire the real-time power consumption data generated by the target power consumption area based on the initial power supply scheme in the second time period.

[0134] The third acquisition module 12 is used to optimize the initial power supply scheme based on real-time power consumption data to obtain the power supply scheme for the target power consumption area in the third time period; the second time period is earlier than the third time period.

[0135] The power supply scheme determination device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0136] In one embodiment, such as Figure 8 As shown, the third acquisition module 12 mentioned above includes: a determining unit 121 and an acquisition unit 122, wherein:

[0137] The determining unit 121 is used to determine the adjustment values ​​of the power supply parameters in the initial power supply scheme based on real-time power consumption data; wherein the power supply parameters include at least one of power supply frequency, power supply peak value and power supply power.

[0138] The acquisition unit 122 is used to adjust the initial power supply scheme according to the adjustment value to obtain the power supply scheme of the target power consumption area in the third time period.

[0139] The power supply scheme determination device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0140] In one embodiment, please continue to refer to Figure 8 The aforementioned determining unit 121 is specifically used for:

[0141] Based on real-time electricity consumption data, the load baseline of the target electricity consumption area in the second time period is obtained; and the adjustment value is determined based on the load baseline.

[0142] The power supply scheme determination device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0143] In one embodiment, please continue to refer to Figure 8 The aforementioned device further includes: a fourth acquisition module 13, wherein:

[0144] The fourth acquisition module 13 is used to acquire target electricity consumption data within a first time period from a preset database based on the identifier of the target electricity consumption area.

[0145] The power supply scheme determination device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0146] In one embodiment, please continue to refer to Figure 8 The database establishment process includes: a data acquisition module 14, a fifth acquisition module 15, and a database establishment module 16, wherein:

[0147] The data acquisition module 14 is used to collect electricity consumption data in each electricity consumption area in real time.

[0148] The fifth acquisition module 15 is used to perform lasso regression and clustering processing on the electricity consumption data of each electricity consumption area to obtain the processed electricity consumption data of each electricity consumption area.

[0149] Module 16 is used to establish a database based on the correspondence between the identifiers of each power consumption area and the processed power consumption data of each power consumption area.

[0150] The power supply scheme determination device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0151] In one embodiment, please continue to refer to Figure 8 The aforementioned device further includes: a sixth acquisition module 17 and a seventh acquisition module 18, wherein:

[0152] The sixth acquisition module 17 is used to obtain historical electricity consumption data of the target electricity consumption area from the database.

[0153] The seventh acquisition module 18 is used to train the preset initial DDQN model using reinforcement learning algorithm and historical electricity consumption data of the target electricity consumption area to obtain the DDQN model.

[0154] The power supply scheme determination device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0155] Each module in the aforementioned power supply scheme determination device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0156] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores power data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for determining a power supply scheme.

[0157] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0158] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0159] Based on the target electricity consumption data of the target electricity consumption area in the first time period and the preset dual deep reinforcement learning network DDQN model, the initial power supply scheme of the target electricity consumption area in the second time period is obtained; the first time period is earlier than the second time period.

[0160] The initial power supply plan is sent to the terminal corresponding to the target power consumption area, and the real-time power consumption data generated by the target power consumption area in the second time period based on the initial power supply plan is obtained.

[0161] The initial power supply plan is optimized based on real-time power consumption data to obtain the power supply plan for the target power consumption area in the third time period; the second time period is earlier than the third time period.

[0162] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0163] Based on real-time electricity consumption data, determine the adjustment values ​​of the power supply parameters in the initial power supply plan; wherein, the power supply parameters include at least one of power supply frequency, power supply peak value and power supply power;

[0164] The initial power supply scheme is adjusted based on the adjustment values ​​to obtain the power supply scheme for the target power consumption area in the third time period.

[0165] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0166] Based on real-time electricity consumption data, obtain the load baseline of the target electricity consumption area in the second time period;

[0167] Determine the adjustment value based on the load baseline.

[0168] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining target electricity consumption data for a first time period from a preset database based on the identifier of the target electricity consumption area.

[0169] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0170] Real-time collection of electricity consumption data from various electricity-consuming areas;

[0171] The electricity consumption data of each electricity consumption area are processed by lasso regression and clustering to obtain the processed electricity consumption data of each electricity consumption area;

[0172] A database is established based on the correspondence between the identifiers of each electricity consumption area and the processed electricity consumption data of each electricity consumption area.

[0173] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0174] Retrieve historical electricity consumption data for the target electricity consumption area from the database;

[0175] The initial DDQN model is trained using reinforcement learning algorithms and historical electricity consumption data of the target electricity consumption area to obtain the DDQN model.

[0176] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0177] Based on the target electricity consumption data of the target electricity consumption area in the first time period and the preset dual deep reinforcement learning network DDQN model, the initial power supply scheme of the target electricity consumption area in the second time period is obtained; the first time period is earlier than the second time period.

[0178] The initial power supply plan is sent to the terminal corresponding to the target power consumption area, and the real-time power consumption data generated by the target power consumption area in the second time period based on the initial power supply plan is obtained.

[0179] The initial power supply plan is optimized based on real-time power consumption data to obtain the power supply plan for the target power consumption area in the third time period; the second time period is earlier than the third time period.

[0180] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0181] Based on real-time electricity consumption data, determine the adjustment values ​​of the power supply parameters in the initial power supply plan; wherein, the power supply parameters include at least one of power supply frequency, power supply peak value and power supply power;

[0182] The initial power supply scheme is adjusted based on the adjustment values ​​to obtain the power supply scheme for the target power consumption area in the third time period.

[0183] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0184] Based on real-time electricity consumption data, obtain the load baseline of the target electricity consumption area in the second time period;

[0185] Determine the adjustment value based on the load baseline.

[0186] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: obtaining target electricity consumption data for a first time period from a preset database based on the identifier of the target electricity consumption area.

[0187] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0188] Real-time collection of electricity consumption data from various electricity-consuming areas;

[0189] The electricity consumption data of each electricity consumption area are processed by lasso regression and clustering to obtain the processed electricity consumption data of each electricity consumption area;

[0190] A database is established based on the correspondence between the identifiers of each electricity consumption area and the processed electricity consumption data of each electricity consumption area.

[0191] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0192] Retrieve historical electricity consumption data for the target electricity consumption area from the database;

[0193] The initial DDQN model is trained using reinforcement learning algorithms and historical electricity consumption data of the target electricity consumption area to obtain the DDQN model.

[0194] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0195] Based on the target power consumption data of the target power consumption area in the first time period and the preset dual deep reinforcement learning network model, the initial power supply scheme of the target power consumption area in the second time period is obtained; the first time period is earlier than the second time period.

[0196] The initial power supply plan is sent to the terminal corresponding to the target power consumption area, and the real-time power consumption data generated by the target power consumption area in the second time period based on the initial power supply plan is obtained.

[0197] The initial power supply plan is optimized based on real-time power consumption data to obtain the power supply plan for the target power consumption area in the third time period; the second time period is earlier than the third time period.

[0198] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0199] Based on real-time electricity consumption data, determine the adjustment values ​​of the power supply parameters in the initial power supply plan; wherein, the power supply parameters include at least one of power supply frequency, power supply peak value and power supply power;

[0200] The initial power supply scheme is adjusted based on the adjustment values ​​to obtain the power supply scheme for the target power consumption area in the third time period.

[0201] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0202] Based on real-time electricity consumption data, obtain the load baseline of the target electricity consumption area in the second time period;

[0203] Determine the adjustment value based on the load baseline.

[0204] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0205] Based on the identification of the target electricity consumption area, the target electricity consumption data for the first time period is obtained from the preset database.

[0206] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0207] Real-time collection of electricity consumption data from various electricity-consuming areas;

[0208] The electricity consumption data of each electricity consumption area are processed by lasso regression and clustering to obtain the processed electricity consumption data of each electricity consumption area;

[0209] A database is established based on the correspondence between the identifiers of each electricity consumption area and the processed electricity consumption data of each electricity consumption area.

[0210] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0211] Retrieve historical electricity consumption data for the target electricity consumption area from the database;

[0212] The initial DDQN model is trained using reinforcement learning algorithms and historical electricity consumption data of the target electricity consumption area to obtain the DDQN model.

[0213] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0214] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0215] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0216] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining a power supply scheme, characterized in that, The method includes: Based on the target electricity consumption data of the target electricity consumption area in the first time period and the preset dual deep reinforcement learning network (DDQN) model, the initial power supply scheme for the target electricity consumption area in the second time period is obtained; the first time period is earlier than the second time period. The initial power supply scheme is sent to the terminal corresponding to the target power consumption area, and the real-time power consumption data generated by the target power consumption area based on the initial power supply scheme during the second time period is obtained; Based on the real-time electricity consumption data, the load baseline of the target electricity consumption area in the second time period is obtained. According to the load baseline, the adjustment value of the power supply parameters in the initial power supply scheme is determined. The initial power supply scheme is adjusted according to the adjustment value to obtain the power supply scheme of the target electricity consumption area in the third time period. The second time period is earlier than the third time period. The power supply parameters include at least one of power supply frequency, power supply peak value and power supply power.

2. The method according to claim 1, characterized in that, The method further includes: Based on the identifier of the target electricity consumption area, the target electricity consumption data for the first time period is obtained from a preset database.

3. The method according to claim 2, characterized in that, The process of establishing the database includes: Real-time collection of electricity consumption data from various electricity-consuming areas; The electricity consumption data of each of the aforementioned electricity consumption areas are subjected to Tow regression and clustering processing to obtain the processed electricity consumption data of each of the aforementioned electricity consumption areas; The database is established based on the correspondence between the identifiers of each electricity consumption area and the processed electricity consumption data of each electricity consumption area.

4. The method according to claim 3, characterized in that, The method further includes: Retrieve historical electricity consumption data for the target electricity consumption area from the database; The DDQN model is obtained by training a preset initial DDQN model using a reinforcement learning algorithm and historical electricity consumption data of the target electricity consumption area.

5. The method according to claim 4, characterized in that, The historical electricity consumption data refers to the electricity consumption data generated by the user side within a time period earlier than the first time period.

6. The method according to claim 1, characterized in that, The load baseline is estimated based on historical load data from the user side.

7. A device for determining a power supply scheme, characterized in that, The device includes: The first acquisition module is used to acquire the initial power supply scheme for the target power consumption area in the second time period based on the target power consumption data in the first time period of the target power consumption area and the preset dual deep reinforcement learning network (DDQN) model; the first time period is earlier than the second time period. The second acquisition module is used to send the initial power supply scheme to the terminal corresponding to the target power consumption area, and acquire the real-time power consumption data generated by the target power consumption area based on the initial power supply scheme during the second time period; The third acquisition module is used to acquire the load baseline of the target power consumption area in the second time period based on the real-time power consumption data, determine the adjustment value of the power supply parameters in the initial power supply scheme according to the load baseline, adjust the initial power supply scheme according to the adjustment value, and determine the power supply scheme of the target power consumption area in the third time period; the second time period is earlier than the third time period, and the power supply parameters include at least one of power supply frequency, power supply peak value and power supply power.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images