An energy management method and system based on deep reinforcement learning

Abstract

The application provides an energy management method and system based on deep reinforcement learning, and relates to the technical field of energy management. The method comprises the following steps: obtaining a plurality of energy storage prediction models of a plurality of energy supply units, processing a plurality of preset time zone energy storage influencing factors, and generating a plurality of predicted energy storage amounts; based on the plurality of energy supply units, performing energy storage loss weight distribution on element position numbers to obtain a plurality of element position weight distribution matrices; obtaining a plurality of damage type weight distribution matrices; calculating a plurality of energy supply incentive factors; based on line service time length, line specification parameters and service environment parameters, calculating a plurality of line section aging factors, combining a plurality of line section lengths, and constructing a plurality of line section energy supply incentive factors; and according to the plurality of energy supply incentive factors and the plurality of line section energy supply incentive factors, combining the predicted energy storage amounts, and performing energy management. The technical problem that the prior art cannot accurately predict future time zone energy data and effectively perform prior management is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of energy management, and particularly relates to an energy management method and system based on deep reinforcement learning. BACKGROUND

[0002] In distributed power management, predicting energy data in future time zones can achieve effective proactive management. However, the commonly used prediction methods at present include two kinds. One is to predict data based on artificial historical experience, which has the disadvantage of relying on expert decision-making, low efficiency and high contingency. The other is to use machine learning to fit past data to achieve data prediction, which has the disadvantage of poor model transferability in distributed scenarios, so that the energy data in future time zones cannot be accurately predicted, and effective proactive management cannot be carried out accordingly. SUMMARY

[0003] The present application provides an energy management method and system based on deep reinforcement learning to solve the technical problem that the prior art cannot accurately predict energy data in future time zones and carry out effective proactive management accordingly.

[0004] The technical solution of the present application to solve the above technical problem is as follows:

[0005] In a first aspect, the present application provides an energy management method based on deep reinforcement learning, comprising:

[0006] Obtaining a plurality of energy storage prediction models of a plurality of energy supply units, processing a plurality of preset time zone energy storage influencing factors to generate a plurality of predicted energy storage, the energy storage prediction model is generated by training a plurality of data using deep reinforcement learning, and any one of the plurality of data includes set time step energy storage influencing factor data of a target energy storage unit and a label identifying energy storage;

[0007] Based on the plurality of energy supply units, the element position number is subjected to energy storage loss weight distribution to obtain a plurality of element position number weight distribution matrices;

[0008] Iterating through the element position number set of the plurality of energy supply units, the element damage type is subjected to energy storage loss weight distribution to obtain a plurality of damage type weight distribution matrices;

[0009] Extracting a plurality of element service time, a plurality of element maintenance damage type list and a plurality of element maintenance frequency list of a plurality of energy supply units, combining the plurality of element position number weight distribution matrices and the plurality of damage type weight distribution matrices, and calculating a plurality of energy supply incentive factors, wherein the larger the energy supply incentive factor, the higher the probability of the energy supply unit participating in power distribution;

[0010] Based on line service length, line specification parameters and service environment parameters, aging analysis is performed on a plurality of line sections to obtain a plurality of line section aging factors, and in combination with a plurality of line section lengths, a plurality of line section energy transmission incentive factors are constructed, wherein the larger the energy transmission incentive factor is, the smaller the aging factor is, and the shorter the length of the section is.

[0011] According to the plurality of energy supply incentive factors and the plurality of line section energy transmission incentive factors, in combination with the predicted energy storage amount, energy management is performed.

[0012] In a second aspect, the present application provides an energy management system based on deep reinforcement learning, comprising:

[0013] An energy storage prediction module is configured to obtain a plurality of energy storage amount prediction models of a plurality of energy supply units, process a plurality of preset time zone energy storage influencing factors, and generate a plurality of predicted energy storage amounts, wherein the energy storage amount prediction models are generated by deep reinforcement learning training based on a plurality of sets of data, and any one of the plurality of sets of data includes set time step energy storage influencing factor data of a target energy storage unit and a label identifying the energy storage amount.

[0014] A bit number weight configuration module is configured to perform energy storage loss weight distribution on element bit numbers based on the plurality of energy supply units, and obtain a plurality of element bit number weight distribution matrices.

[0015] A type weight configuration module is configured to traverse the element bit number set of the plurality of energy supply units, perform energy storage loss weight distribution on element damage types, and obtain a plurality of damage type weight distribution matrices.

[0016] An energy supply incentive factor construction module is configured to extract a plurality of element service lengths, a plurality of element maintenance damage type lists and a plurality of element maintenance frequency lists of a plurality of energy supply units, combine the plurality of element bit number weight distribution matrices and the plurality of damage type weight distribution matrices, and calculate a plurality of energy supply incentive factors, wherein the larger the energy supply incentive factor is, the higher the probability of the energy supply unit participating in power distribution is.

[0017] An energy transmission incentive factor construction module is configured to perform aging analysis on a plurality of line sections based on line service length, line specification parameters and service environment parameters, obtain a plurality of line section aging factors, and construct a plurality of line section energy transmission incentive factors in combination with a plurality of line section lengths, wherein the larger the energy transmission incentive factor is, the smaller the aging factor is, and the shorter the length of the section is.

[0018] A fusion execution module is configured to perform energy management according to the plurality of energy supply incentive factors and the plurality of line section energy transmission incentive factors in combination with the predicted energy storage amount.

[0019] The present application has the following beneficial effects:

[0020] Compared with the prior art, firstly, a plurality of energy storage quantity prediction models of a plurality of energy supply units are obtained, a plurality of preset time zone energy storage influencing factors are processed to generate a plurality of predicted energy storage quantities, a plurality of energy storage quantity prediction models are obtained by setting time step energy storage influencing factor data and label of identified energy storage quantity to train and optimize a predefined discrete model architecture set based on deep reinforcement learning, compared with the fixed prediction model of the traditional method, the generalization ability of the model can be improved, the predicted energy storage quantity of the target energy supply unit is predicted and output, and necessary data basis is provided for subsequent energy management. Secondly, based on a plurality of energy supply units, the element position number is subjected to energy storage loss weight distribution to obtain a plurality of element position number weight distribution matrices, the influence degree of each element of the energy supply unit on the energy storage loss is quantified by constructing the element position number weight distribution matrix, and necessary data basis is provided for subsequent energy management. Thirdly, the element position number set of a plurality of energy supply units is traversed, the element damage type is subjected to energy storage loss weight distribution to obtain a plurality of damage type weight distribution matrices, and the influence degree of different damage types on the energy storage loss is quantified by the damage type weight distribution matrix, and necessary data basis is provided for subsequent energy management. Further, a plurality of element service time lengths of a plurality of energy supply units, a plurality of element maintenance damage type lists and a plurality of element maintenance frequency lists are extracted, a plurality of energy supply incentive factors are calculated in combination with a plurality of element position number weight distribution matrices and a plurality of damage type weight distribution matrices, the influence of the multi-dimensional parameters of the element health state and different element positions and damage types on the energy storage loss is dynamically fused, the probability of different energy supply units participating in power distribution is quantified by the energy supply incentive factor, and the units with high health states can be driven to preferentially participate in power distribution accordingly. Further, based on line service time length, line specification parameters and service environment parameters, a plurality of line sections are subjected to aging analysis to obtain a plurality of line section aging factors, and a plurality of line section energy sending incentive factors are constructed in combination with a plurality of line section lengths, the line section energy sending incentive factor combines the line aging degree and length, and in energy scheduling, the line section with a large energy sending incentive factor is preferentially selected, which provides a quantitative basis for the selection of lines in energy management and helps to reduce transmission loss and improve energy transmission efficiency. Finally, according to a plurality of energy supply incentive factors and a plurality of line section energy sending incentive factors, in combination with the predicted energy storage quantity, energy management is performed, the predicted energy storage quantity, energy supply unit state data and line parameters are comprehensively considered, different energy distribution schemes are scored by an adaptability function, an energy distribution scheme is determined, and energy management is performed.

[0021] By the technical solution, the application can construct a plurality of energy storage prediction models of a plurality of energy supply units through deep reinforcement learning, can accurately output a plurality of predicted energy storage according to preset time zone energy storage influencing factors, consider the influence degree of different element position number abnormalities and different damage types on energy storage loss, then quantize the probability of different energy supply units participating in power distribution and line transmission efficiency, finally score different energy distribution schemes through an adaptability function, determine an energy distribution scheme according to adaptability, and perform energy management. In this way, future time zone energy data is accurately predicted, and effective pre-management is performed accordingly. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A flowchart of an energy management method based on deep reinforcement learning provided by the application is shown.

[0023] Figure 2 A structural diagram of an energy management system based on deep reinforcement learning provided by the application is shown.

[0024] In the drawings, the components represented by the numbers are as follows:

[0025] The energy storage prediction module 11, the position number weight configuration module 12, the type weight configuration module 13, the energy supply incentive factor construction module 14, the energy supply incentive factor construction module 15, and the fusion execution module 16. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the application.

[0027] In the description of the application, the terms "first", "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0028] In the description of the present application, the term "for example" is used to indicate "as an example, instance, or illustration". Any embodiment described as "for example" in the present application is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to practice the present application as claimed. In the following description, details are set forth in order to provide a thorough understanding of the present application. It will be apparent to one ordinarily skilled in the art that the present application can be practiced without using these specific details. In other instances, well-known structures and processes have not been elaborated in order to avoid unnecessary detail, which might obscure the description of the present application. Therefore, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed.

[0029] Embodiment one, as shown in the present application, provides a deep reinforcement learning-based energy management method, comprising: Figure 1

[0030] S10: obtaining a plurality of energy storage capacity prediction models of a plurality of energy supply units, processing a plurality of preset time zone energy storage influencing factors, and generating a plurality of predicted energy storage capacities, wherein the energy storage capacity prediction models are generated by training a plurality of sets of data using deep reinforcement learning, and any one of the plurality of sets of data includes set time step energy storage influencing factor data of a target energy storage unit and a label identifying the energy storage capacity;

[0031] In distributed energy management, usually contains a plurality of dispersed energy supply units, for example, photovoltaic, wind power, energy storage battery energy supply units, due to the geographical location of each energy supply unit is dispersed, the factors affecting energy storage are numerous and different, it is difficult to predict the energy storage capacity of the energy supply unit, and accordingly to carry out effective pre-management.

[0032] To solve the above problems, the present application obtains a plurality of energy storage capacity prediction models of a plurality of energy supply units, processes a plurality of preset time zone energy storage influencing factors, and generates a plurality of predicted energy storage capacities.

[0033] Specifically, step S10 in the method comprises:

[0034] From the plurality of energy supply units, a first energy supply unit is extracted as a constraint, set time step energy storage influencing factor data and a label identifying the energy storage capacity are collected;

[0035] Obtaining a plurality of reinforcement learning initial model architectures, wherein the reinforcement learning initial model architectures are randomly selected from a user-predefined discrete model architecture set;

[0036] According to the set time step energy storage influencing factor data and the label identifying the energy storage capacity, a plurality of reinforcement learning initial model architectures are trained respectively to generate a plurality of initial energy storage capacity prediction models, wherein the plurality of initial energy storage capacity prediction models have a plurality of validation losses.​

[0037] When none of the plurality of validation losses satisfies the convergence loss threshold, based on the plurality of validation losses, the plurality of initial energy storage prediction models are sorted from small to large, and the initial energy storage prediction model with the first serial number is selected;

[0038] An upward integer value of a preset ratio multiplied by the total number of serial numbers is calculated, and the reference initial energy storage prediction model satisfying the upward integer value is selected from the tail serial number. The first initial energy storage prediction model is taken as the target, and the model architecture similarity, node hyperparameter similarity adjustment is performed to generate an updated reinforcement learning initial model architecture, and a loop is executed, wherein the preset ratio is at least 0.25 and at most 0.5;

[0039] When any one of the plurality of validation losses satisfies the convergence loss threshold, the corresponding initial energy storage prediction model is set as the first energy supply unit energy storage prediction model, and is added to the plurality of energy storage prediction models.

[0040] In the embodiments of the present application, first, the first energy supply unit is extracted from the plurality of energy supply units as a constraint, and the energy storage influencing factor data and the label identifying the energy storage capacity at the set time step are collected. For example, one energy supply unit is selected from the plurality of energy supply units in the distributed network as the first energy supply unit, for example, the node #5 photovoltaic energy supply unit is selected as the first energy supply unit, and then the energy storage influencing factor data and the label identifying the energy storage capacity are collected according to the preset time step, for example, on June 26, 2025, at 12:00, the temperature of the first energy supply unit is 31℃, the light intensity is 850W / m 2 and so on, and the label identifying the energy storage capacity is 82.1kWh, on June 26, 2025, at 12:15, the temperature of the first energy supply unit is 31.5℃, the light intensity is 855W / m 2 and so on, and the label identifying the energy storage capacity is 82.5kWh, wherein the set time step is to balance the prediction accuracy and the computing power consumption, and if it is set too short, the prediction accuracy can be improved while the computing power consumption is huge, and if it is set too long, the computing power consumption can be saved while the prediction may not be accurate, and the present application recommends that the time step be set to 15 minutes, and a person skilled in the art can adjust it according to the actual situation, the energy storage influencing factor is an influencing factor that can affect the energy storage capacity of the energy supply unit, such as temperature, light intensity, humidity, etc., and the label identifying the energy storage capacity is the actual energy storage capacity of the first energy supply unit, which can be obtained in real time through a smart meter.

[0041] Secondly, a plurality of reinforcement learning initial model architectures are obtained, wherein the reinforcement learning initial model architectures are randomly selected from a user-predefined discrete model architecture set, wherein the user-predefined discrete model architecture set is a user-prebuilt energy storage prediction model, for example, a DQN+LSTM model, a PPO+CNN model, etc. For example, 3 reinforcement learning initial model architectures are randomly obtained from the user-predefined discrete model architecture set, for example, a DQN+LSTM model, a PPO+CNN model, and a SAC+Transformer model.

[0042] Thirdly, a plurality of reinforcement learning initial model architectures are trained based on the set time step energy storage influencing factor data and the label identifying the energy storage, to generate a plurality of initial energy storage prediction models, wherein the plurality of initial energy storage prediction models have a plurality of validation losses. For example, the set time step energy storage influencing factor data and the label identifying the energy storage are used as a data set to independently train a plurality of reinforcement learning initial model architectures, to generate a plurality of initial energy storage prediction models, each of which has a validation loss. For example, the same data set is used to independently train a DQN+LSTM model, a PPO+CNN model, and a SAC+Transformer model, to generate 3 initial energy storage prediction models, and obtain 3 corresponding validation losses: 12.8%, 10.5%, and 15.5%.

[0043] Further, when none of the plurality of validation losses meets the convergence loss threshold, the plurality of initial energy storage prediction models are sorted from small to large based on the plurality of validation losses, and a first sequence number initial energy storage prediction model is selected. Specifically, when there is an initial energy storage prediction model in the plurality of validation losses that meets the convergence loss threshold, it is directly added to the plurality of energy storage prediction models. When none of the plurality of validation losses meets the convergence loss threshold, the plurality of initial energy storage prediction models are sorted from small to large according to the validation losses, and the initial energy storage prediction model ranked first (i.e., with the smallest validation loss) is selected as the first sequence number initial energy storage prediction model. The convergence loss threshold is an index for evaluating whether an initial energy storage prediction model has accurate prediction capability, and the present application recommends setting the convergence loss threshold to 10%, which can be dynamically adjusted by those skilled in the art according to actual conditions. For example, the convergence loss threshold is 10%, the validation losses of the 3 initial energy storage prediction models are 12.8%, 10.5%, and 15.5% in turn, none of the 3 initial energy storage prediction models meets the convergence loss threshold, and the 3 initial energy storage prediction models are sorted from small to large, for example, a PPO+CNN model, a DQN+LSTM model, and a SAC+Transformer model, and the PPO+CNN model ranked first is selected as the first sequence number initial energy storage prediction model.

[0044] Further, the preset ratio is multiplied by the total number of sequence numbers to calculate an upward integer value, and the reference initial energy storage prediction model meeting the upward integer value is selected from the tail sequence number. The first sequence number initial energy storage prediction model is taken as the target to perform model architecture similarity, node hyperparameter similarity adjustment, and an updated reinforcement learning initial model architecture is generated. The loop is executed, wherein the preset ratio is at least 0.25 and at most 0.5, that is, the preset ratio controls the selection of the last 0.25-0.5 of the initial energy storage prediction models. This is because if the preset ratio is too small, the selected model will be too small, which may not be able to obtain an ideal model through optimization. If the preset ratio is too large, the selected model will be too large, and the resource consumption of model training will be too large. Illustratively, the preset ratio is 0.5, the total number of sequence numbers arranged from small to large is 3, the upward integer value is [0.5*3]=2, and two initial energy storage prediction models are selected from the tail sequence number, that is, the DQN+LSTM model and the SAC+Transformer model. Then, the first sequence number initial energy storage prediction model (such as the PPO+CNN model) is taken as the target to perform model architecture similarity, node hyperparameter similarity adjustment, for example, the Transformer self-attention layer of the SAC+Transformer model is removed, the learning rate and batch size are adjusted to be close to the first sequence number initial energy storage prediction model, and an updated reinforcement learning initial model architecture is generated, for example, the SAC+CNN model. The reference initial energy storage prediction model meeting the upward integer value is selected from the tail sequence number, and the model at the tail is optimized because the model at the tail and the first sequence number initial energy storage prediction model have a relatively large gap, and the model optimization space is large. In the optimization process with a target guide, a better solution can be obtained more quickly.

[0045] Finally, when any one of the plurality of validation losses meets the convergence loss threshold, the corresponding initial energy storage prediction model is set as the first energy storage prediction model of the energy supply unit, and is added to the plurality of energy storage prediction models. Illustratively, the SAC+CNN model in the updated reinforcement learning initial model architecture generated by the model architecture similarity, node hyperparameter similarity adjustment has a validation loss of 8.5%, which meets the convergence loss threshold (such as 10%). The SAC+CNN model is set as the first energy storage prediction model of the energy supply unit, and is added to the plurality of energy storage prediction models.

[0046] Further, a plurality of energy storage prediction models of a plurality of energy supply units are obtained, and a plurality of preset time zone energy storage influencing factors are processed to generate a plurality of predicted energy storage. For example, a plurality of energy storage prediction models of a plurality of energy supply units are obtained according to the same construction method of the first energy supply unit energy storage prediction model, and then the energy storage influencing factors of the target energy supply unit in the preset time zone are input into the corresponding energy storage prediction model to output the predicted energy storage of the target energy supply unit. For example, the energy storage influencing factors of the first energy supply unit in the preset time zone, such as temperature 31℃ and light intensity 860W / m 2 , are input into the first energy supply unit energy storage prediction model, and the predicted energy storage of 85.1kWh is obtained.

[0047] In summary, compared with the prior art, the present application obtains a plurality of energy storage prediction models of a plurality of energy supply units, processes a plurality of preset time zone energy storage influencing factors, and generates a plurality of predicted energy storage. In this way, based on deep reinforcement learning, the pre-defined discrete model architecture set is trained and optimized by setting the time step energy storage influencing factor data and the label of the energy storage, a plurality of energy storage prediction models are obtained, compared with the fixed prediction model of the traditional method, the generalization ability of the model can be improved, and the predicted energy storage of the target energy supply unit is output, which provides necessary data basis for subsequent energy management.

[0048] S20: Based on the plurality of energy supply units, the element position number is subjected to energy storage loss weight distribution to obtain a plurality of element position number weight distribution matrices;

[0049] In distributed energy management, an energy supply unit is composed of a plurality of elements, such as battery packs, power converters, temperature sensors, etc. The influence degree of each element abnormality on energy storage loss is significantly different, for example, battery pack abnormality will cause a straight-line rise in energy storage loss, while power converter abnormality will only weakly affect energy storage loss. However, the traditional energy management method is difficult to quantify the influence of single element abnormality on the overall energy storage loss, and lacks dynamic adaptability to environmental factor changes.

[0050] To solve the above problems, the present application distributes the energy storage loss weight of the element position number based on the plurality of energy supply units to obtain a plurality of element position number weight distribution matrices.

[0051] Specifically, step S20 in the method comprises:

[0052] extracting a first energy supply unit model, wherein the first energy supply unit model has a set of element position numbers;

[0053] With the first energy supply unit model and the preset energy storage influencing factor as constants, a first energy storage amount record value of the full health of the element position number set is collected, with the first energy supply unit model and the preset energy storage influencing factor as constants, a second energy storage amount record value of the first element position number anomaly is collected, a first energy storage loss amount of the first energy storage amount record value minus the second energy storage amount record value is calculated, if the first energy storage loss amount is greater than or equal to 0, a plurality of first element position number energy storage loss amounts are added, otherwise, re-collection is performed;

[0054] The preset energy storage influencing factor is updated, the loop is executed for a preset number of times, the mode statistics is performed on the plurality of first element position number energy storage loss amounts, the first element position number energy storage loss feature is obtained, and the element position number energy storage loss feature set is added;

[0055] The ratio of the first element position number energy storage loss feature and the sum of the element position number energy storage loss features is set as a first element position number loss weight, and the first energy supply unit element position number weight distribution matrix is added.

[0056] The first energy supply unit element position number weight distribution matrix is added to the plurality of element position number weight distribution matrices.

[0057] In the embodiment of the application, first, the first energy supply unit model is extracted, wherein the first energy supply unit model has an element position number set. Specifically, the element position number is the unique identity of the plurality of elements in the energy supply unit, for example, each element can be identified by a two-level code of element name + element serial number, such as battery module B1. Illustratively, the extraction of the first energy supply unit model includes the element position number set: [battery module B1, inverter INV-01, temperature sensor T1, cooling fan F1].

[0058] Secondly, with the first energy supply unit model and the preset energy storage influencing factor as constants, a first energy storage amount record value in a full health state of the element position number set is collected, and with the first energy supply unit model and the preset energy storage influencing factor as constants, a second energy storage amount record value in a first element position number abnormality is collected. A first energy storage loss amount, i.e., first energy storage loss amount = first energy storage amount record value - second energy storage amount record value, is calculated by subtracting the second energy storage amount record value from the first energy storage amount record value. If the first energy storage loss amount is greater than or equal to 0, the first element position number energy storage loss amount is added to the plurality of first element position number energy storage loss amounts. Otherwise, the collection is restarted. Specifically, first, with the first energy supply unit model and the preset energy storage influencing factor as constants, a first energy storage amount record value in a full health state of the element position number set is collected. The first energy storage amount record value reflects an ideal energy storage amount of the first energy supply unit under the current energy storage influencing factor. Then, energy storage amounts in different element position number abnormalities are collected respectively as second energy storage amount record values. The second energy storage amount record values reflect energy storage amounts in the first element position number abnormality. In theory, the second energy storage amount record values should be less than or equal to the first energy storage amount record value in the full health state. Therefore, the first energy storage loss amount should be greater than or equal to 0. When the first energy storage loss amount is less than 0, it indicates that the currently collected data is not reliable, i.e., there may be a collection error, and the data needs to be collected again. Exemplarily, with the first energy supply unit model (such as [battery module B1, inverter INV-01, temperature sensor T1, cooling fan F1]) and the preset energy storage influencing factor (such as temperature 25℃, light intensity 800W / m 2 ) as constants, a first energy storage amount record value (such as 70kWh) in a full health state of the element position number set is collected, reflecting an ideal energy storage amount under the current energy storage influencing factor. Then, a second energy storage amount record value (such as 65kWh) in a first element position number abnormality (such as battery module B1 abnormality) is collected, reflecting an energy storage amount when the battery module B1 is abnormal. Then, a first energy storage loss amount = 70kWh - 65kWh = 5kWh is calculated. Since it is greater than 0, it is added to the plurality of first element position number energy storage loss amounts.

[0059] Thirdly, the preset energy storage influencing factor is updated, and the cycle is executed for a preset number of times. For the plurality of first element position number energy storage loss amounts, mode statistics are performed to obtain a first element position number energy storage loss characteristic, which is added to the element position number energy storage loss characteristic set. Exemplarily, the preset energy storage influencing factor is updated, for example, temperature 30℃, light intensity 800W / m 2, the same method as in the foregoing steps, a plurality of first element position number energy storage loss amounts, for example, 5 kWh, 6 kWh, 5 kWh, 4 kWh, 5 kWh, 5 kWh, are calculated, and then the mode statistics are performed to take 5 kWh as the first element position number energy storage loss feature and add it to the element position number energy storage loss feature set, wherein the preset energy storage influencing factor is updated to obtain the first element position number energy storage loss amount under different influencing factors, improve the data accuracy and generalization ability, and the mode statistics are performed to take the most representative data as the first element position number energy storage loss feature.

[0060] Further, the ratio of the first element position number energy storage loss feature to the sum of the element position number energy storage loss features is taken as the first element position number loss weight, which is added to the first energy supply unit element position number weight distribution matrix, wherein the first element position number loss weight = the first element position number energy storage loss feature / the sum of the element position number energy storage loss features. The greater the first element position number energy storage loss feature, the more serious the energy storage loss caused by the first element position number anomaly, and the greater the corresponding first element position number loss weight. Illustratively, the first element position number energy storage loss feature is 5 kWh, the sum of the element position number energy storage loss features is 30 kWh, and the first element position number loss weight = 5 / 30 = 0.167, which is added to the first energy supply unit element position number weight distribution matrix. The first element position number loss weight reflects the influence degree of the first element position number anomaly on the energy storage loss, and the greater the first element position number loss weight, the greater the influence degree.

[0061] Finally, the first energy supply unit element position number weight distribution matrix is added to the plurality of element position number weight distribution matrices, which can reflect the influence degree of different element position number anomalies on the energy storage loss.

[0062] In summary, compared with the prior art, the present application performs energy storage loss weight distribution on the element position number based on the plurality of energy supply units to obtain a plurality of element position number weight distribution matrices. In this way, by constructing the element position number weight distribution matrix, the influence degree of each element of the energy supply unit on the energy storage loss is quantified, providing necessary data basis for subsequent energy management.

[0063] S30: traverse the element position number set of the plurality of energy supply units, perform energy storage loss weight distribution on the element damage type, and obtain a plurality of damage type weight distribution matrices;

[0064] The influence degrees of different damage types on the energy storage loss are significantly different. For example, the plate sulfuration of the battery pack directly leads to the reduction of active substances, causing the energy storage capacity to decay by 15%-30%, and the electrolyte dryness exacerbates the internal impedance rise, causing the charging and discharging efficiency to decrease by 20%-40%; in comparison, the slight wear of the battery pack shell has an influence of less than 5% on the energy storage performance.

[0065] To solve the above problems, the element position number set of the energy supply units is traversed, and the energy storage loss weight distribution of the element damage type is obtained to obtain a plurality of damage type weight distribution matrices.

[0066] Specifically, step S30 in the method comprises:

[0067] Taking the first energy supply unit type and the preset energy storage influencing factor as constants, a third energy storage amount record value of the first element position number in the first damage type is collected, a second energy storage loss amount of the first energy storage amount record value minus the third energy storage amount record value is calculated, if the second energy storage loss amount is greater than or equal to 0, the second energy storage loss amount is added to a plurality of first damage type energy storage loss amounts, otherwise, the collection is re-performed;

[0068] The preset energy storage influencing factor is updated, the loop is performed for a preset number of times, the mode statistics of the plurality of first damage type energy storage loss amounts is performed, the first damage type energy storage loss feature is obtained, and the first damage type energy storage loss feature is added to a plurality of damage type energy storage loss features;

[0069] The ratio of the first damage type energy storage loss feature to the sum of the damage type energy storage loss features is set as the first damage type loss weight, and the first damage type loss weight is added to the first element position number damage type weight distribution matrix;

[0070] The first element position number damage type weight distribution matrix is added to the first energy supply unit damage type weight distribution matrix;

[0071] The first energy supply unit damage type weight distribution matrix is added to the plurality of damage type weight distribution matrices.

[0072] In the embodiments of the present application, first, the first energy supply unit model and the preset energy storage influencing factor are taken as constants, the third energy storage record value of the first component position in the first damage type is collected, the second energy storage loss amount of the first energy storage record value minus the third energy storage record value is calculated, that is, the second energy storage loss amount = the first energy storage record value - the third energy storage record value, if the second energy storage loss amount is greater than or equal to 0, add the plurality of first damage type energy storage loss amounts, otherwise, re-collect, because theoretically, the third energy storage record value should be less than or equal to the first energy storage record value in the full health state, therefore, the second energy storage loss amount should be greater than or equal to 0, when the second energy storage loss amount is less than 0, it means that the current collected data is not reliable, that is, there may be collection errors, and the data needs to be re-collected. Exemplarily, the same method as the foregoing S20 step, the third energy storage record value (such as 50 kWh) of the first component position (such as battery module B1) in the first damage type (such as leakage) is collected, reflecting the energy storage amount of the battery module B1 when it is in leakage, then the second energy storage loss amount = 70 kWh - 50 kWh = 20 kWh is calculated, since it is greater than 0, it is added to the plurality of first damage type energy storage loss amounts.

[0073] Secondly, the preset energy storage influencing factor is updated, and the plurality of first damage type energy storage loss amounts are executed for a plurality of preset times, and the mode statistics are executed to obtain the first damage type energy storage loss characteristic and add it to the plurality of damage type energy storage loss characteristics. Exemplarily, according to the same update of the preset energy storage influencing factor as the foregoing S20 step, a plurality of third energy storage record values under the same energy storage influencing factor are collected, a plurality of first damage type energy storage loss characteristics are calculated and obtained, for example, 20 kWh, 22 kWh, 20 kWh, 18 kWh, 20 kWh, 21 kWh, then the mode statistics are executed, 20 kWh is taken as the first damage type energy storage loss characteristic, and is added to the plurality of damage type energy storage loss characteristics. Wherein, the preset energy storage influencing factor is updated in order to obtain the third energy storage record value under different influencing factors and improve the data accuracy, and the mode statistics are executed in order to take the most representative data as the first damage type energy storage loss characteristic.

[0074] Again, a ratio of the first damage type energy storage loss feature to a sum of damage type energy storage loss features is set as a first damage type loss weight, which is added to the first element position number damage type weight distribution matrix, where the first damage type loss weight = the first damage type energy storage loss feature / the sum of damage type energy storage loss features. The greater the first damage type energy storage loss feature, the more serious the energy storage loss caused by the first damage type, and the greater the corresponding first damage type loss weight. For example, the first damage type energy storage loss feature is 20 kWh, and the sum of damage type energy storage loss features is 100 kWh, so the first damage type loss weight = 20 / 100 = 0.2, which is added to the first element position number damage type weight distribution matrix. The first damage type loss weight reflects the degree of influence of the first damage type on energy storage loss, and the greater the first damage type loss weight, the greater the degree of influence.

[0075] Further, the first element position number damage type weight distribution matrix is added to the first energy supply unit damage type weight distribution matrix, which can reflect the degree of influence of the first element position number damage type on energy storage loss.

[0076] Finally, the first energy supply unit damage type weight distribution matrix is added to the plurality of damage type weight distribution matrices, which can reflect the degree of influence of the plurality of damage types on energy storage loss.

[0077] In summary, compared with the prior art, the present application distributes energy storage loss weights of element damage types for element position number sets of the plurality of energy supply units, obtains a plurality of damage type weight distribution matrices, and quantifies the degree of influence of different damage types on energy storage loss, thereby providing necessary data basis for subsequent energy management.

[0078] S40: Extracting a plurality of element service durations, a plurality of element maintenance damage type lists, and a plurality of element maintenance frequency lists of a plurality of energy supply units, combining the plurality of element position number weight distribution matrices and the plurality of damage type weight distribution matrices, and calculating a plurality of energy supply incentive factors, wherein the greater the energy supply incentive factor, the higher the probability of the energy supply unit participating in power distribution.

[0079] In distributed power management, the service life, maintenance damage type and maintenance frequency of the elements of each power supply unit are significantly different, which directly affects the health status and power distribution reliability of the power supply unit. Specifically, the power supply unit with shorter service time, no high-weight damage type, and lower maintenance frequency usually has higher health level and power supply stability; on the contrary, the unit with long service time, serious damage to key elements, and frequent maintenance has high operation risk and low power distribution reliability. Therefore, based on the above differentiated characteristics, the priority and probability of different power supply units in participating in power distribution can be quantified.

[0080] To solve the above problems, the service life of a plurality of elements of a plurality of power supply units, a plurality of element maintenance damage type lists and a plurality of element maintenance frequency lists are extracted, and a plurality of power supply incentive factors are calculated in combination with the plurality of element position weight distribution matrices and the plurality of damage type weight distribution matrices, wherein the larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution.

[0081] Specifically, step S40 in the method comprises:

[0082] The power supply incentive factor evaluation function is constructed as follows:

[0083]

[0084] wherein, the power supply incentive factor of any one power supply unit, the number of elements, i represents the element serial number, which corresponds to the element position number one by one, the weight of the i-th element position, the service life of the i-th element position, the j-th damage type weight of the i-th element position, the maintenance frequency of the j-th damage type, the number of damage types of the i-th element position, is a natural constant;

[0085] Based on the power supply incentive factor evaluation function, the service life of a plurality of elements of a plurality of power supply units, a plurality of element maintenance damage type lists and a plurality of element maintenance frequency lists are extracted, and a plurality of power supply incentive factors are calculated in combination with the plurality of element position weight distribution matrices and the plurality of damage type weight distribution matrices.

[0086] In the embodiment of the application, the power supply incentive factor evaluation function is first constructed, and a plurality of power supply incentive factors are calculated and obtained accordingly. The larger the power supply incentive factor, the better the health status of the power supply unit, and the higher the probability of the power supply unit participating in power distribution.

[0087] wherein, The energy supply incentive factor of any one energy supply unit, the larger the energy supply incentive factor, the better the health status of the energy supply unit, the more suitable for priority participation in energy supply, the higher the participation probability.

[0088] wherein, characterizing the i-th element position number weight, The larger, the greater the influence of the i-th element position number on the energy loss, the smaller the energy supply incentive factor calculated The smaller.

[0089] wherein, characterizing the i-th element position number weight, The larger, the greater the influence of the i-th element position number on the energy loss, the smaller the energy supply incentive factor calculated The smaller.

[0090] wherein, characterizing the i-th element position number weight, The smaller, The smaller, the larger the energy supply incentive factor The larger, because the shorter service time, the lower the probability of damage to the i-th element position number, often with better health status and power supply reliability.

[0091] wherein, characterizing the i-th element position number weight, The larger, the greater the influence of the i-th element position number on the energy loss, the smaller the energy supply incentive factor calculated The smaller.

[0092] Further, based on the energy supply incentive factor evaluation function, the service time of the elements of the energy supply unit, the element maintenance damage type list and the element maintenance frequency list are extracted. For example, the service time of the battery module B1 can be extracted from the device account or the Internet of Things sensor, for example, the service time of the battery module B1 is 1000h, the element maintenance damage type list and the element maintenance frequency list can be extracted from the historical maintenance record, for example, the maintenance frequency of the battery module B1 is 0.5 times / month.

[0093] Finally, the service life of the elements of the plurality of energy supply units, the list of maintenance damage types of the elements, the list of maintenance frequencies of the elements, the plurality of element position weight distribution matrices and the plurality of damage type weight distribution matrices are substituted into the energy supply incentive factor evaluation function to obtain the plurality of energy supply incentive factors. The service life of the elements of the plurality of energy supply units, the list of maintenance damage types of the elements, the list of maintenance frequencies of the elements dynamically integrate the multi-dimensional parameters of the element health state, and the plurality of element position weight distribution matrices and the plurality of damage type weight distribution matrices quantify the influence of different element positions and damage types on energy storage loss. In this way, the power distribution priority of different energy supply units is quantified by the energy supply incentive factor, and the probability of the energy supply unit participating in power distribution is configured accordingly to drive the selection of energy supply units with high health state to participate in power distribution preferentially.

[0094] In summary, compared with the prior art, the present application extracts the service life of the elements of the plurality of energy supply units, the list of maintenance damage types of the elements, and the list of maintenance frequencies of the elements, and combines the plurality of element position weight distribution matrices and the plurality of damage type weight distribution matrices to calculate the plurality of energy supply incentive factors. The greater the energy supply incentive factor, the higher the probability of the energy supply unit participating in power distribution. In this way, by dynamically integrating the multi-dimensional parameters of the element health state and the influence of different element positions and damage types on energy storage loss, the probability of different energy supply units participating in power distribution is quantified by the energy supply incentive factor, which can be used to drive the selection of energy supply units with high health state to participate in power distribution preferentially.

[0095] S50: based on the line service life, the line specification parameters and the service environment parameters, performing aging analysis on the plurality of line sections to obtain a plurality of line section aging factors, and combining the plurality of line section lengths to construct a plurality of line section energy sending incentive factors, wherein the greater the energy sending incentive factor, the smaller the aging factor, and the shorter the length of the section;

[0096] In the process of energy transmission, due to the differences in aging degree and length of each line section, the energy loss, transmission efficiency and reliability of the line are different, which further causes the priority and use frequency of each line section in participating in energy transmission to also differ.

[0097] To solve the above problems, the present application performs aging analysis on the plurality of line sections based on the line service life, the line specification parameters and the service environment parameters to obtain a plurality of line section aging factors, and combines the plurality of line section lengths to construct a plurality of line section energy sending incentive factors, wherein the greater the energy sending incentive factor, the smaller the aging factor, and the shorter the length of the section.

[0098] Specifically, step S50 in the method comprises:

[0099] collect a similar sample line section set of the first line section, perform mode statistics of unit distance energy transmission loss rate, generate a first line section aging factor, and add the first line section aging factor into the plurality of line section aging factors, with the line service length, the line specification parameter, and the service environment parameter as constraints;

[0100] An energy transmission incentive factor evaluation function is constructed:

[0101] ,

[0102] wherein, the energy transmission incentive factor represents any one line section, the line section length represents a line section length, the line section aging factor represents a line section aging factor, the natural constant is a natural constant;

[0103] The plurality of line section energy transmission incentive factors are constructed according to the energy transmission incentive factor evaluation function.

[0104] In the embodiments of the present application, first, a similar sample line section set of the first line section is collected, mode statistics of unit distance energy transmission loss rate is performed, and a first line section aging factor is generated and added into the plurality of line section aging factors, with the line service length, the line specification parameter, and the service environment parameter as constraints. The line service length is the cumulative total length after the line is put into use, the longer the service time is, the higher the possibility of line aging is, the line specification parameter includes wire diameter, material, insulation layer material, etc., different line specification parameters affect the electrical performance and aging resistance of the line, for example, the copper cable has better electrical conductivity and is more corrosion-resistant than the aluminum cable, the service environment parameter includes temperature, humidity, pH value, electromagnetic environment, etc., high temperature and humidity environment can accelerate the aging of the line insulation layer, and strong electromagnetic environment can affect the transmission performance of the line. The unit distance energy transmission loss rate refers to the energy loss rate per unit distance of transmission, the mode statistics selects the unit distance energy transmission loss rate with the highest frequency as the representative value, and the mode statistics can reduce the interference of abnormal values, so that the generated first line section aging factor is more representative. For example, with the line service length (such as 2 years), the line specification parameter (such as copper core 120 mm 2 ), and the service environment parameter (such as temperate zone 22℃, humidity 45%-55%) as constraints, 5 similar lines are collected, a similar sample line section set of the first line section is obtained, the unit distance energy transmission loss rates of the 5 similar lines are 10%, 8%, 10%, 10%, and 10% in turn, mode statistics of unit distance energy transmission loss rate is performed, a first line section aging factor (such as 10%) is generated, and the first line section aging factor is added into the plurality of line section aging factors.

[0105] Secondly, an energy transmission incentive factor evaluation function is constructed:

[0106] ,

[0107] wherein, a power transmission excitation factor representing any one line section, a line section length, a line section aging factor, is a natural constant, the longer the line section length, the higher the transmission loss rate, the greater the line section aging factor, the more serious the line aging, the lower the transmission efficiency, and the smaller the calculated power transmission excitation factor, indicating that the line is less suitable for energy transmission.

[0108] Finally, according to the power transmission excitation factor evaluation function, the power transmission excitation factors of the line sections are constructed. For example, according to the power transmission excitation factor evaluation function, the length and the aging factor of each line section are substituted into the formula to calculate the power transmission excitation factor of each line section, and the power transmission excitation factors of the line sections are obtained. For example, when the length of a line section is 50 meters and the line section aging factor is 10%, the calculated power transmission excitation factor is Thus, a quantitative basis is provided for the selection of lines in energy management, and the selection of line sections with large power transmission excitation factors helps to reduce transmission loss and improve energy transmission efficiency.

[0109] In summary, compared with the prior art, the present application performs aging analysis on a plurality of line sections based on the service time, line specification parameters and service environment parameters of the line, obtains a plurality of line section aging factors, and constructs a plurality of line section power transmission excitation factors in combination with a plurality of line section lengths. The larger the power transmission excitation factor, the smaller the aging factor, and the shorter the line section length. Thus, the power transmission excitation factor in combination with the aging degree and length of the line enables the selection of line sections with large power transmission excitation factors in energy scheduling, provides a quantitative basis for the selection of lines in energy management, and helps to reduce transmission loss and improve energy transmission efficiency.

[0110] S60: According to the plurality of power supply excitation factors and the plurality of line section power transmission excitation factors, in combination with the predicted energy storage amount, energy management is performed.

[0111] The power supply excitation factor can reflect the element health status of the power supply unit, and the power transmission excitation factor can reflect the transmission efficiency and transmission loss of the line. The energy distribution scheme can be quantitatively scored based on this, and a dynamic energy management strategy that takes into account the health management of the power supply unit and the optimization of line loss can be generated.

[0112] To solve the above problems, the application executes energy management according to the energy supply incentive factors and the line section energy supply incentive factors, in combination with the predicted energy storage amount.

[0113] Specifically, step S60 in the method includes:

[0114] The Delphi method is used to configure a first weight for the energy supply incentive factors and a second weight for the energy supply incentive factors.

[0115] Based on the first weight and the second weight, an adaptability function is constructed.

[0116] Based on the predicted energy storage amount, a target energy distribution amount is taken as a constraint to construct a plurality of energy distribution schemes, and based on the adaptability function, the energy supply incentive factors and the line section energy supply incentive factors, a plurality of adaptability values are calculated.

[0117] The minimum adaptability value of the plurality of adaptability values is selected as an energy distribution scheme, and energy management is executed.

[0118] In the embodiment of the application, first, the Delphi method is used to configure a first weight for the energy supply incentive factors and a second weight for the energy supply incentive factors. The Delphi method is a structured prediction method that realizes group decision consensus through multiple rounds of anonymous expert consultation and feedback convergence. For example, experts in the fields of power system planning, energy storage operation and maintenance, and line engineering can be invited to configure dynamic weights for the energy supply incentive factors and the energy supply incentive factors. And , for example, , .

[0119] Secondly, based on the first weight and the second weight, an adaptability function is constructed, wherein the adaptability function , wherein, represents the energy supply incentive factor of any one energy supply unit, represents the energy supply incentive factor of any one line section, is the first weight, is the second weight, and the adaptability function can score different energy distribution schemes. The smaller the adaptability value, the better the energy distribution scheme.

[0120] Again, based on the predicted energy storage, a number of energy distribution schemes are constructed as constraints of target energy allocation, and a number of fitness values are calculated based on the fitness function, the number of energy supply incentive factors and the number of line section energy supply incentive factors. Exemplarily, as constraints of the predicted energy storage and the target energy allocation of each energy supply unit, a small-scale energy management process can be constructed by enumeration method, all possible combinations of energy supply units and lines are exhausted, a number of energy distribution schemes are constructed, and genetic algorithm, particle swarm optimization and the like are used for large-scale energy management process, and then a number of energy distribution schemes are constructed, and then a number of fitness values are calculated based on the fitness function, the number of energy supply incentive factors and the number of line section energy supply incentive factors.

[0121] Finally, the minimum value of the number of fitness values is selected as the energy distribution scheme, and energy management is performed. Exemplarily, scheme 1 selects 2 energy supply units with high energy supply incentive factors and 3 lines with high energy supply incentive factors, and scheme 2 selects 3 energy supply units with medium energy supply incentive factors and 2 lines with medium energy supply incentive factors. After calculation, the fitness value of scheme 2 is 2.68, which is better than 3.02 of scheme 1. Therefore, scheme 2 is preferentially selected as the energy distribution scheme, and energy management is performed.

[0122] In summary, compared with the prior art, the present application performs energy management according to the number of energy supply incentive factors and the number of line section energy supply incentive factors, combined with the predicted energy storage. In this way, the predicted energy storage, energy supply unit state data and line parameters are comprehensively considered, and different energy distribution schemes can be scored by the fitness function to determine the energy distribution scheme and perform energy management.

[0123] In summary, the embodiments of the present application have at least the following technical effects:

[0124] Compared with the prior art, the present application first obtains a number of energy storage prediction models of a number of energy supply units, processes a number of preset time zone energy storage influencing factors, and generates a number of predicted energy storages. In this way, based on deep reinforcement learning, the pre-defined discrete model architecture set is trained and optimized by setting the time step energy storage influencing factor data and the label of the identified energy storage, and a number of energy storage prediction models are obtained. Compared with the fixed prediction model of the traditional method, the generalization ability of the model can be improved, and the predicted energy storage of the target energy supply unit can be predicted, providing necessary data basis for subsequent energy management.

[0125] Secondly, the present application distributes the energy storage loss weight of the element position number based on the number of energy supply units, and obtains a number of element position weight distribution matrices. In this way, by constructing the element position weight distribution matrix, the influence degree of each element of the energy supply unit on the energy storage loss is quantified, providing necessary data basis for subsequent energy management.

[0126] Again, the application traverses the element number set of the plurality of energy supply units, and obtains a plurality of damage type weight distribution matrices of energy storage loss by storing energy element damage types. The damage type weight distribution matrix quantifies the influence degree of different damage types on energy storage loss, and provides a necessary data basis for subsequent energy management.

[0127] Further, the application extracts a plurality of element service lengths, a plurality of element maintenance damage type lists, and a plurality of element maintenance frequency lists of a plurality of energy supply units, combines the plurality of element number weight distribution matrices and the plurality of damage type weight distribution matrices, and calculates a plurality of energy supply incentive factors. The larger the energy supply incentive factor, the higher the probability of the energy supply unit participating in power distribution. In this way, by dynamically fusing the multi-dimensional parameters of the element health state and the influence of different element numbers and damage types on energy storage loss, the probability of different energy supply units participating in power distribution is quantified by the energy supply incentive factor, which can drive the selection of energy supply units with high health status to participate in power distribution preferentially.

[0128] Further, the application performs aging analysis on a plurality of line sections based on line service length, line specification parameters, and service environment parameters, obtains a plurality of line section aging factors, and constructs a plurality of line section energy sending incentive factors in combination with a plurality of line section lengths. The larger the energy sending incentive factor, the smaller the aging factor, and the shorter the length of the section. In this way, the energy sending incentive factor combines the aging degree and length of the line, and when energy is dispatched, the line section with a large energy sending incentive factor is preferentially selected, providing a quantitative basis for the selection of lines in energy management, and helping to reduce transmission loss and improve energy transmission efficiency.

[0129] Finally, the application performs energy management according to the plurality of energy supply incentive factors and the plurality of line section energy sending incentive factors in combination with the predicted energy storage amount. In this way, the predicted energy storage amount, the energy supply unit state data, and the line parameters are comprehensively considered, and different energy distribution schemes can be scored by the fitness function to determine the energy distribution scheme and perform energy management.

[0130] Through the above technical solutions, the application constructs a plurality of energy storage amount prediction models of a plurality of energy supply units by deep reinforcement learning, can accurately output a plurality of predicted energy storage amounts according to preset time zone energy storage influencing factors, considers the influence degree of different element number abnormalities and different damage types on energy storage loss, and then quantifies the probability of different energy supply units participating in power distribution and the line transmission efficiency. Finally, different energy distribution schemes are scored by the fitness function, and the energy distribution scheme is determined according to the fitness to perform energy management. In this way, the energy data of the future time zone is accurately predicted, and effective proactive management is performed accordingly.

[0131] Embodiment two, as Figure 2As shown, based on the same inventive concept of the energy management method based on deep reinforcement learning provided in Embodiment One, the embodiment of the present application also provides an energy management system based on deep reinforcement learning, comprising:

[0132] An energy storage prediction module 11 is configured to obtain a plurality of energy storage quantity prediction models of a plurality of energy supply units, process a plurality of preset time zone energy storage influencing factors, and generate a plurality of predicted energy storage quantities, wherein the energy storage quantity prediction models are generated by deep reinforcement learning training based on a plurality of groups of data, and any one of the plurality of groups of data includes set time step energy storage influencing factor data of a target energy storage unit and a label identifying an energy storage quantity;

[0133] A bit number weight configuration module 12 is configured to perform energy loss weight distribution on element bit numbers based on the plurality of energy supply units, and obtain a plurality of element bit number weight distribution matrices;

[0134] A type weight configuration module 13 is configured to traverse a set of element bit numbers of the plurality of energy supply units, perform energy loss weight distribution on element damage types, and obtain a plurality of damage type weight distribution matrices;

[0135] An energy supply incentive factor construction module 14 is configured to extract a plurality of element service durations of a plurality of energy supply units, a plurality of element maintenance damage type lists, and a plurality of element maintenance frequency lists, combine the plurality of element bit number weight distribution matrices and the plurality of damage type weight distribution matrices, and calculate a plurality of energy supply incentive factors, wherein the larger the energy supply incentive factor is, the higher the probability of the energy supply unit participating in power distribution is;

[0136] A power transmission incentive factor construction module 15 is configured to perform aging analysis on a plurality of line sections based on line service durations, line specification parameters, and service environment parameters, obtain a plurality of line section aging factors, and combine a plurality of line section lengths to construct a plurality of line section power transmission incentive factors, wherein the larger the power transmission incentive factor is, the smaller the aging factor is, and the shorter the line section length is;

[0137] A fusion execution module 16 is configured to execute energy management according to the plurality of energy supply incentive factors and the plurality of line section power transmission incentive factors in combination with the predicted energy storage quantities.

[0138] The energy storage prediction module 11 is specifically configured to:

[0139] Extract a first energy supply unit as a constraint from the plurality of energy supply units, collect set time step energy storage influencing factor data and a label identifying an energy storage quantity;

[0140] Obtain a plurality of reinforcement learning initial model architectures, wherein the reinforcement learning initial model architectures are randomly selected from a discrete model architecture set predefined by a user;

[0141] According to the setting time step energy storage influencing factor data and the label identifying the energy storage amount, a plurality of reinforcement learning initial model architectures are trained respectively to generate a plurality of initial energy storage amount prediction models, wherein the plurality of initial energy storage amount prediction models have a plurality of validation losses;

[0142] When none of the plurality of validation losses meets the convergence loss threshold, the plurality of initial energy storage amount prediction models are sorted from small to large based on the plurality of validation losses, and a first sequence number initial energy storage amount prediction model is selected;

[0143] An upward integer value of a preset ratio multiplied by a total number of sequence numbers is calculated, and reference initial energy storage amount prediction models meeting the upward integer value are sorted from the tail sequence number forward, and the first sequence number initial energy storage amount prediction model is taken as a target to perform model architecture similarity, node hyperparameter similarity adjustment, to generate an updated reinforcement learning initial model architecture, and a loop is executed, wherein the preset ratio is at least 0.25 and at most 0.5;

[0144] When any one of the plurality of validation losses meets the convergence loss threshold, the corresponding initial energy storage amount prediction model is set as a first energy supply unit energy storage amount prediction model, and is added to the plurality of energy storage amount prediction models.

[0145] The bit number weight configuration module 12 is specifically configured to:

[0146] Extract a first energy supply unit model, wherein the first energy supply unit model has an element bit number set;

[0147] Collect a first energy storage amount record value of the element bit number set in full health with the first energy supply unit model and a preset energy storage influencing factor as constants, collect a second energy storage amount record value of a first element bit number anomaly with the first energy supply unit model and the preset energy storage influencing factor as constants, calculate a first energy loss amount of the first energy storage amount record value minus the second energy storage amount record value, if the first energy loss amount is greater than or equal to 0, add a plurality of first element bit number energy loss amounts, otherwise, re-collect;

[0148] Update the preset energy storage influencing factor, execute a loop for a preset number of times, perform mode statistics on the plurality of first element bit number energy loss amounts, obtain a first element bit number energy loss feature, and add the first element bit number energy loss feature to an element bit number energy loss feature set;

[0149] Calculate a ratio of the first element bit number energy loss feature and an element bit number energy loss feature sum, set as a first element bit number loss weight, and add the first element bit number loss weight to a first energy supply unit element bit number weight distribution matrix;

[0150] Add the first energy supply unit element bit number weight distribution matrix to the plurality of element bit number weight distribution matrices.

[0151] The type weight configuration module 13 is specifically configured to:

[0152] Taking the first energy supply unit model and the preset energy storage influencing factor as constants, a third energy storage record value of the first element position in the first damage type is collected, a second energy storage loss amount of the first energy storage record value minus the third energy storage record value is calculated, if the second energy storage loss amount is greater than or equal to 0, the first damage type energy storage loss amount is added to the plurality of first damage type energy storage loss amounts, otherwise, the collection is re-performed;

[0153] The preset energy storage influencing factor is updated, the plurality of first damage type energy storage loss amounts are subjected to mode statistics for a preset number of cycles, a first damage type energy storage loss characteristic is obtained, and the first damage type energy storage loss characteristic is added to the plurality of damage type energy storage loss characteristics;

[0154] The ratio of the first damage type energy storage loss characteristic and the sum of the damage type energy storage loss characteristics is set as a first damage type loss weight, and the first damage type loss weight is added to the first element position damage type weight distribution matrix;

[0155] The first element position damage type weight distribution matrix is added to the first energy supply unit damage type weight distribution matrix;

[0156] The first energy supply unit damage type weight distribution matrix is added to the plurality of damage type weight distribution matrices.

[0157] The energy supply incentive factor construction module 14 is specifically configured to:

[0158] An energy supply incentive factor evaluation function is constructed:

[0159] ,

[0160] wherein, the energy supply incentive factor of any one energy supply unit, the element quantity, i represents the element serial number, and the element serial number corresponds to the element position one by one, the i-th element position weight, the i-th element position service time, the i-th element position j-th damage type weight, the j-th damage type maintenance frequency, the number of damage types of the i-th element position, is a natural constant;

[0161] Based on the energy supply incentive factor evaluation function, the service time of the elements of the energy supply units, the list of maintenance damage types of the elements and the list of maintenance frequency of the elements are extracted, and the element position weight distribution matrix and the damage type weight distribution matrix are combined to calculate the energy supply incentive factors.

[0162] The energy supply incentive factor construction module 15 is specifically configured to:

[0163] With the line service time, the line specification parameters and the service environment parameters as constraints, a similar sample line segment set of the first line segment is collected, a unit distance energy supply loss rate mode statistics is performed, a first line segment aging factor is generated, and the first line segment aging factor is added to the plurality of line segment aging factors.

[0164] The energy supply incentive factor evaluation function is constructed as:

[0165] ,

[0166] Wherein, The energy supply incentive factor of any one line segment is represented by, The line segment length is represented by, The line segment aging factor is represented by, is a natural constant.

[0167] According to the energy supply incentive factor evaluation function, the plurality of line segment energy supply incentive factors are constructed.

[0168] The fusion execution module 16 is specifically configured to:

[0169] The Delphi method is used to configure a first weight for the energy supply incentive factor and a second weight for the energy supply incentive factor.

[0170] Based on the first weight and the second weight, an adaptability function is constructed.

[0171] Based on the predicted energy storage amount, a target energy distribution amount is configured as a constraint to construct a plurality of energy distribution schemes, and based on the adaptability function, the plurality of energy supply incentive factors and the plurality of line segment energy supply incentive factors, a plurality of adaptability values are calculated.

[0172] The minimum energy distribution scheme of the plurality of adaptability values is selected to perform energy management.

[0173] In summary, the embodiments of the present application have at least the following technical effects:

[0174] Compared with the prior art, firstly, through the energy storage prediction module, a plurality of energy storage quantity prediction models of a plurality of energy supply units are obtained, a plurality of preset time zone energy storage influencing factors are processed, a plurality of predicted energy storage quantities are generated, based on deep reinforcement learning, a set of predefined discrete model architectures is trained and optimized by setting time step energy storage influencing factor data and labels of identified energy storage quantities, a plurality of energy storage quantity prediction models are obtained, compared with the fixed prediction model of the traditional method, the generalization ability of the model can be improved, the predicted energy storage quantity of the target energy supply unit is predicted, and necessary data basis is provided for subsequent energy management. Secondly, through the bit number weight configuration module, based on a plurality of energy supply units, energy loss weight distribution is performed on the element bit numbers to obtain a plurality of element bit number weight distribution matrices. By constructing the element bit number weight distribution matrix, the influence degree of each element of the energy supply unit on the energy loss can be quantified. Thirdly, through the type weight configuration module, the element bit number set of a plurality of energy supply units is traversed, energy loss weight distribution is performed on the element damage types to obtain a plurality of damage type weight distribution matrices. The damage type weight distribution matrix quantifies the influence degree of different damage types on the energy loss, thereby providing necessary data basis for subsequent energy management. Further, through the energy supply incentive factor construction module, the service time of a plurality of elements of a plurality of energy supply units, a plurality of element maintenance damage type lists and a plurality of element maintenance frequency lists are extracted, combined with a plurality of element bit number weight distribution matrices and a plurality of damage type weight distribution matrices, a plurality of energy supply incentive factors are calculated, the multi-dimensional parameters of the element health state and the influence of different element bit numbers and damage types on the energy loss are dynamically fused, and the probability of different energy supply units participating in power distribution is quantified through the energy supply incentive factor. The unit with high health state can be driven to participate in power distribution preferentially. Further, through the energy sending incentive factor construction module, based on the line service time, the line specification parameters and the service environment parameters, the aging analysis of a plurality of line sections is performed to obtain a plurality of line section aging factors, combined with a plurality of line section lengths, a plurality of line section energy sending incentive factors are constructed. The energy sending incentive factor combines the line aging degree and the length, and when energy is dispatched, the line section with a large energy sending incentive factor is preferentially selected, thereby providing a quantitative basis for the selection of lines in energy management, and helping to reduce transmission loss and improve energy transmission efficiency. Finally, through the fusion execution module, the energy management is executed according to the plurality of energy supply incentive factors and the plurality of line section energy sending incentive factors in combination with the predicted energy storage quantity. The predicted energy storage quantity, the energy supply unit state data and the line parameters are comprehensively considered, the different energy distribution schemes are scored through the fitness function, the energy distribution scheme is determined, and the energy management is executed. In this way, the energy data of the future time zone is accurately predicted, and effective pre-management is performed accordingly.

[0175] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0176] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0177] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0178] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0180] Although preferred embodiments of the application have been described, those skilled in the art will appreciate that additional modifications and alterations can be made to the embodiments without departing from the basic inventive concepts.

[0181] It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the application and their equivalent technology.

Claims

1. An energy management method based on deep reinforcement learning, characterized in that, include: Several energy storage prediction models for several energy supply units are obtained, several energy storage influencing factors in preset time zones are processed, and several predicted energy storage is generated. The energy storage prediction models are generated by deep reinforcement learning training through multiple sets of data. Each set of multiple sets of data includes energy storage influencing factor data at a set time step for the target energy storage unit and a label identifying the energy storage. Based on the aforementioned energy supply units, an energy storage loss weight distribution is performed on the component reference numbers to obtain several component reference number weight distribution matrices. Among them, the first energy supply unit model and preset energy storage influencing factors are taken as constants, and the first energy storage record value of the component reference number set is collected when all components are healthy. The second energy storage record value of the first component reference number is collected when the first energy supply unit model and preset energy storage influencing factors are taken as constants, and the first energy storage loss amount is calculated by subtracting the second energy storage record value from the first energy storage record value. Traverse the component reference set of the plurality of power supply units, perform energy storage loss weight distribution on the component damage type, and obtain a plurality of damage type weight distribution matrices. Among them, with the first power supply unit model and preset energy storage influencing factors as constants, collect the third energy storage record value of the first component reference number in the first damage type, and calculate the second energy storage loss amount by subtracting the third energy storage record value from the first energy storage record value. Extract the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies, and combine the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types to calculate several power supply incentive factors. The larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution. Based on the service life of the line, the line specifications and service environment parameters, aging analysis is performed on several line segments to obtain several line segment aging factors. Combined with the length of several line segments, several line segment energy transmission incentive factors are constructed. Among them, the larger the energy transmission incentive factor, the smaller the aging factor and the shorter the line segment length. Energy management is performed based on the aforementioned energy supply incentive factors and the aforementioned line segment energy transmission incentive factors, combined with the predicted energy storage.

2. The method as described in claim 1, characterized in that, Several energy storage prediction models for several energy supply units were obtained, including: From the aforementioned energy supply units, the first energy supply unit is extracted as a constraint, and data on energy storage influencing factors and tags identifying the energy storage capacity are collected at a set time step. Several initial reinforcement learning model architectures are obtained, wherein the initial reinforcement learning model architectures are randomly selected from a set of discrete model architectures predefined by the user; Based on the data of energy storage influencing factors and the labels of the identified energy storage, several initial reinforcement learning model architectures are trained to generate several initial energy storage prediction models, wherein the several initial energy storage prediction models have several validation losses. When none of the verification losses meet the convergence loss threshold, the initial energy storage prediction models are sorted from smallest to largest based on the verification losses, and the first-order initial energy storage prediction model is selected. Calculate the preset ratio multiplied by the total number of serial numbers and round up to the nearest integer. Select the baseline initial energy storage prediction models that satisfy the round up value from the last serial number forward. Using the first serial number initial energy storage prediction model as the target, perform model architecture similarity and node hyperparameter similarity adjustment to generate an updated reinforcement learning initial model architecture. Execute the loop. The preset ratio is a minimum of 0.25 and a maximum of 0.

5. When any one of the verification losses satisfies the convergence loss threshold, the corresponding initial energy storage prediction model is set as the first energy supply unit energy storage prediction model and added to the plurality of energy storage prediction models.

3. The method as described in claim 1, characterized in that, Based on the aforementioned power supply units, an energy storage loss weight distribution is performed on the component reference numbers to obtain several component reference number weight distribution matrices, including: Extract the model number of the first power supply unit, wherein the model number of the first power supply unit has a set of component reference numbers; Using the first energy supply unit model and preset energy storage influencing factors as constants, collect the first energy storage record value of a fully healthy set of component reference numbers. Using the first energy supply unit model and preset energy storage influencing factors as constants, collect the second energy storage record value of an abnormal first component reference number. Calculate the first energy storage loss amount by subtracting the second energy storage record value from the first energy storage record value. If the first energy storage loss amount is greater than or equal to 0, add multiple first component reference number energy loss amounts; otherwise, re-collect. Update the preset energy storage influencing factors, execute the preset number of loops, perform mode statistics on the energy loss of multiple first element reference numbers, obtain the energy loss characteristics of the first element reference numbers, and add them to the element reference number energy loss characteristic set. Calculate the ratio of the energy loss characteristics of the first component reference number to the sum of the energy loss characteristics of the component reference number, set it as the first component reference number loss weight, and add it to the first energy supply unit component reference number weight distribution matrix; Add the first power supply unit component number weight distribution matrix into the plurality of component number weight distribution matrices.

4. The method as described in claim 3, characterized in that, By traversing the component reference set of the aforementioned power supply units, energy storage loss weight distribution is performed on the component damage type to obtain several damage type weight distribution matrices, including: Using the model of the first energy supply unit and the preset energy storage influencing factors as constants, the third energy storage record value of the first element tag number in the first damage type is collected, and the second energy storage loss is calculated by subtracting the third energy storage record value from the first energy storage record value. If the second energy storage loss is greater than or equal to 0, multiple first damage type energy storage loss values ​​are added; otherwise, the data is collected again. Update the preset energy storage influencing factors, execute the preset number of loops, perform mode statistics on the energy storage loss of multiple first damage types, obtain the energy storage loss characteristics of the first damage type, and add the energy storage loss characteristics of multiple damage types. Calculate the ratio of the energy storage loss feature of the first damage type to the sum of the energy storage loss features of the damage types, set it as the first damage type loss weight, and add it to the first component reference number damage type weight distribution matrix; Add the first component reference number damage type weight distribution matrix into the first power supply unit damage type weight distribution matrix; The first power supply unit damage type weight distribution matrix is ​​added to the plurality of damage type weight distribution matrices.

5. The method as described in claim 1, characterized in that, Extract the service life of several components from several power supply units, a list of several component maintenance damage types, and a list of several component maintenance frequencies. Combine these with the component tag number weight distribution matrix and the damage type weight distribution matrix to calculate several power supply excitation factors, including: Construct an evaluation function for energy supply incentive factors: ， in, The energy supply excitation factor characterizing any energy supply unit. The number of components is represented by 'i', and the component number is represented by 'i', which corresponds one-to-one with the component reference number. Characterizes the weight of the i-th element's bit number. Characterizes the service life of the i-th element bit number. The weight representing the damage type of the i-th element bit number is as follows: Characterizing the repair frequency of the j-th damage type, Characterizes the number of damage types for the i-th element bit. It is a natural constant; Based on the power supply incentive factor evaluation function, the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies are extracted. Combined with the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types, several power supply incentive factors are calculated.

6. The method as described in claim 1, characterized in that, Based on the service life, specifications, and environmental parameters of the lines, aging analysis was conducted on several line sections to obtain aging factors. Combined with the lengths of these line sections, power transmission excitation factors were constructed, including: Constrained by the service life of the line, the line specifications and the service environment parameters, a set of similar sample line segments of the first line segment is collected, the mode statistics of the power transmission loss rate per unit distance are performed, the aging factor of the first line segment is generated, and the aging factor of the several line segments is added to it. Construct an evaluation function for energy delivery incentive factors: ， in, The energy delivery excitation factor characterizing any given line segment. Characterizing the length of the line segment, Characterizing the aging factors of the line segment, It is a natural constant; Based on the energy transmission incentive factor evaluation function, the energy transmission incentive factors for the several line segments are constructed.

7. The method as described in claim 1, characterized in that, Based on the aforementioned energy supply incentive factors and the aforementioned line segment energy transmission incentive factors, and in conjunction with the predicted energy storage, energy management is performed, including: The Delphi method is used to assign a first weight to the energy supply incentive factor and a second weight to the energy delivery incentive factor. Based on the first weight and the second weight, a fitness function is constructed; Based on the predicted energy storage, and with the target energy distribution as a constraint, several energy distribution schemes are constructed. Based on the fitness function, the several energy supply incentive factors, and the several line segment energy delivery incentive factors, several fitness values ​​are calculated. Select the energy distribution scheme with the minimum fitness value from the given fitness values ​​and perform energy management.

8. An energy management system based on deep reinforcement learning, characterized in that, For performing the method according to any one of claims 1-7, comprising: The energy storage prediction module is used to obtain several energy storage prediction models for several energy supply units, process several preset time zone energy storage influencing factors, and generate several predicted energy storage. The energy storage prediction model is generated by deep reinforcement learning training through multiple sets of data. Each set of the multiple sets of data includes energy storage influencing factor data at a set time step of the target energy storage unit and a label identifying the energy storage. The tag weight configuration module is used to distribute the energy storage loss weight of the tag numbers based on the plurality of energy supply units, and obtain a plurality of tag weight distribution matrices. The module uses the first energy supply unit model and preset energy storage influencing factors as constants to collect the first energy storage record value of the entire set of tag numbers that are healthy. It also uses the first energy supply unit model and preset energy storage influencing factors as constants to collect the second energy storage record value of the first tag number that is abnormal. The module calculates the first energy storage loss amount by subtracting the second energy storage record value from the first energy storage record value. The type weight configuration module is used to traverse the component reference number set of the plurality of energy supply units, perform energy storage loss weight distribution on the component damage type, and obtain a plurality of damage type weight distribution matrices. Among them, the first energy supply unit model and the preset energy storage influencing factors are constants, the third energy storage record value of the first component reference number in the first damage type is collected, and the second energy storage loss amount is calculated by subtracting the third energy storage record value from the first energy storage record value. The power supply incentive factor construction module is used to extract the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies, and calculate several power supply incentive factors by combining the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types. The larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution. The energy delivery incentive factor construction module is used to perform aging analysis on several line segments based on line service duration, line specification parameters, and service environment parameters, obtain several line segment aging factors, and construct several line segment energy delivery incentive factors in combination with the length of several line segments. Among them, the larger the energy delivery incentive factor, the smaller the aging factor and the shorter the line segment length. The fusion execution module is used to perform energy management based on the plurality of energy supply incentive factors and the plurality of line segment energy delivery incentive factors, combined with the predicted energy storage.

Images