Energy management method and system based on deep reinforcement learning

Abstract

The invention provides an energy management method and system based on deep reinforcement learning, and relates to the technical field of energy management, and the method comprises the steps: obtaining a plurality of energy storage prediction models of a plurality of energy supply units, processing a plurality of preset time zone energy storage influence factors, and generating a plurality of predicted energy storage amounts; based on the plurality of energy supply units, carrying out energy storage loss weight distribution on the element bit numbers to obtain a plurality of element bit number weight distribution matrixes; obtaining a plurality of damage type weight distribution matrixes; calculating a plurality of energy supply excitation factors; based on the line service duration, the line specification parameters and the service environment parameters, a plurality of line section aging factors are obtained through calculation, and a plurality of line section energy transmission excitation factors are constructed by combining the lengths of a plurality of line sections; and executing energy management according to the plurality of energy supply excitation factors and the plurality of line section energy transmission excitation factors in combination with the predicted energy storage amount. The technical problem that in the prior art, energy data of a future time zone cannot be accurately predicted, and effective pre-management cannot be carried out according to the energy data is solved.

Description

Technical Field

[0001] This invention relates to the field of energy management, and in particular to an energy management method and system based on deep reinforcement learning. Background Technology

[0002] In distributed energy management, predicting energy data for future time zones can enable effective proactive management. However, the commonly used prediction methods include two types: one is data prediction based on historical experience, which has the disadvantage of relying heavily on expert decisions, resulting in low efficiency and high randomness; the other is using machine learning to fit past data to achieve data prediction, but the disadvantage is that the transferability of the model is poor in distributed scenarios, thus making it impossible to accurately predict energy data for future time zones and thus fail to carry out effective proactive management. Summary of the Invention

[0003] This invention addresses the technical problem in existing technologies that cannot accurately predict energy data for future time zones and thus conduct effective proactive management, by providing an energy management method and system based on deep reinforcement learning.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0005] In a first aspect, the present invention provides an energy management method based on deep reinforcement learning, comprising:

[0006] Several energy storage prediction models for several energy supply units are obtained, several energy storage influencing factors in preset time zones are processed, and several predicted energy storage is generated. The energy storage prediction models are generated by deep reinforcement learning training through multiple sets of data. Each set of multiple sets of data includes energy storage influencing factor data at a set time step for the target energy storage unit and a label identifying the energy storage.

[0007] Based on the aforementioned energy supply units, an energy storage loss weight distribution is performed on the component reference numbers to obtain a weight distribution matrix of several component reference numbers.

[0008] Traverse the component reference set of the several power supply units, perform energy storage loss weight distribution on the component damage type, and obtain several damage type weight distribution matrices;

[0009] Extract the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies, and combine the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types to calculate several power supply incentive factors. The larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution.

[0010] Based on the service life of the line, the line specifications and service environment parameters, aging analysis is performed on several line segments to obtain several line segment aging factors. Combined with the length of several line segments, several line segment energy transmission incentive factors are constructed. Among them, the larger the energy transmission incentive factor, the smaller the aging factor and the shorter the line segment length.

[0011] Energy management is performed based on the aforementioned energy supply incentive factors and the aforementioned line segment energy transmission incentive factors, combined with the predicted energy storage.

[0012] Secondly, the present invention provides an energy management system based on deep reinforcement learning, comprising:

[0013] The energy storage prediction module is used to obtain several energy storage prediction models for several energy supply units, process several preset time zone energy storage influencing factors, and generate several predicted energy storage. The energy storage prediction model is generated by deep reinforcement learning training through multiple sets of data. Each set of the multiple sets of data includes energy storage influencing factor data at a set time step of the target energy storage unit and a label identifying the energy storage.

[0014] The tag number weight configuration module is used to distribute the energy storage loss weight of the tag number based on the plurality of energy supply units, and obtain a plurality of tag number weight distribution matrices.

[0015] The type weight configuration module is used to traverse the component reference set of the plurality of energy supply units, perform energy storage loss weight distribution on the component damage type, and obtain a plurality of damage type weight distribution matrices.

[0016] The power supply incentive factor construction module is used to extract the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies, and calculate several power supply incentive factors by combining the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types. The larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution.

[0017] The energy delivery incentive factor construction module is used to perform aging analysis on several line segments based on line service duration, line specification parameters, and service environment parameters, obtain several line segment aging factors, and construct several line segment energy delivery incentive factors in combination with the length of several line segments. Among them, the larger the energy delivery incentive factor, the smaller the aging factor and the shorter the line segment length.

[0018] The fusion execution module is used to perform energy management based on the plurality of energy supply incentive factors and the plurality of line segment energy delivery incentive factors, combined with the predicted energy storage.

[0019] The beneficial effects of this invention are:

[0020] Compared to existing technologies, this application first obtains several energy storage prediction models for several energy supply units, processes several energy storage influencing factors in preset time zones, and generates several predicted energy storage values. Based on deep reinforcement learning, it trains and optimizes a predefined set of discrete model architectures by setting time step energy storage influencing factor data and labels identifying energy storage values, thus obtaining several energy storage prediction models. Compared to the fixed prediction models of traditional methods, this improves the generalization ability of the models and predicts the predicted energy storage value of the target energy supply unit, providing a necessary data foundation for subsequent energy management. Second, based on several energy supply units, it performs energy storage loss weight distribution on the component reference numbers, obtaining several component reference number weight distribution matrices. By constructing the component reference number weight distribution matrices, it quantifies the impact of each component of the energy supply unit on energy storage loss, providing a necessary data foundation for subsequent energy management. Third, it traverses the component reference number set of several energy supply units and performs energy storage loss weight distribution on the component damage type, obtaining several damage type weight distribution matrices. The damage type weight distribution matrices quantify the impact of different damage types on energy storage loss, providing a necessary data foundation for subsequent energy management. Furthermore, the service life of several components, a list of component maintenance and damage types, and a list of component maintenance frequencies of several energy supply units are extracted. Combined with the component tag number weight distribution matrix and the damage type weight distribution matrix, several energy supply incentive factors are calculated. By dynamically integrating multi-dimensional parameters of component health status and the impact of different component tag numbers and damage types on energy storage losses, the probability of different energy supply units participating in power distribution is quantified through these energy supply incentive factors. This allows for prioritizing units with high health status in power distribution. Furthermore, based on line service life, line specifications, and service environment parameters, aging analysis is performed on several line segments to obtain several line segment aging factors. Combined with the length of these line segments, several line segment energy delivery incentive factors are constructed. These energy delivery incentive factors, combined with the line aging degree and length, prioritize line segments with high energy delivery incentive factors during energy dispatch. This provides a quantitative basis for line selection in energy management and helps reduce transmission losses and improve energy transmission efficiency. Finally, based on several energy supply incentive factors and several line segment energy delivery incentive factors, combined with predicted energy storage, energy management is implemented. Taking into account predicted energy storage, energy supply unit status data, and line parameters, different energy distribution schemes can be scored through a fitness function to determine the energy distribution scheme and implement energy management.

[0021] Through the above technical solution, this application constructs several energy storage prediction models for several energy supply units using deep reinforcement learning. Based on preset time zone energy storage influencing factors, it accurately outputs several predicted energy storage values. It considers the impact of different component tag number anomalies and different damage types on energy storage losses. Furthermore, it quantifies the probability of different energy supply units participating in power distribution and line transmission efficiency. Finally, it scores different energy distribution schemes using a fitness function, determines the energy distribution scheme based on the fitness, and executes energy management. In this way, it accurately predicts energy data for future time zones and conducts effective proactive management accordingly. Attached Figure Description

[0022] Figure 1 A flowchart illustrating an energy management method based on deep reinforcement learning provided by this invention;

[0023] Figure 2 This is a schematic diagram of an energy management system based on deep reinforcement learning, provided by the present invention.

[0024] In the attached diagram, the components represented by each number are as follows:

[0025] Energy storage prediction module 11, tag number weight configuration module 12, type weight configuration module 13, energy supply incentive factor construction module 14, energy delivery incentive factor construction module 15, and fusion execution module 16. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0029] Example 1, as Figure 1 As shown, this embodiment of the invention provides an energy management method based on deep reinforcement learning, including:

[0030] S10: Obtain several energy storage prediction models for several energy supply units, process several preset time zone energy storage influencing factors, and generate several predicted energy storage. The energy storage prediction model is generated by deep reinforcement learning training through multiple sets of data. Each set of the multiple sets of data includes the set time step energy storage influencing factor data of the target energy storage unit and the label identifying the energy storage.

[0031] In distributed power management, there are usually multiple decentralized power supply units, such as photovoltaic, wind power and energy storage battery power supply units. Because the geographical locations of each power supply unit are scattered and the factors affecting energy storage are numerous and different, it is difficult to predict the energy storage capacity of the power supply unit and carry out effective pre-management accordingly.

[0032] To address the aforementioned issues, this application obtains several energy storage prediction models for several energy supply units, processes several preset time zone energy storage influencing factors, and generates several predicted energy storage values.

[0033] Specifically, step S10 in the method includes:

[0034] From the aforementioned energy supply units, the first energy supply unit is extracted as a constraint, and data on energy storage influencing factors and tags identifying the energy storage capacity are collected at a set time step.

[0035] Several initial reinforcement learning model architectures are obtained, wherein the initial reinforcement learning model architectures are randomly selected from a set of discrete model architectures predefined by the user;

[0036] Based on the data of energy storage influencing factors and the labels of the identified energy storage, several initial reinforcement learning model architectures are trained to generate several initial energy storage prediction models, wherein the several initial energy storage prediction models have several validation losses.

[0037] When none of the verification losses meet the convergence loss threshold, the initial energy storage prediction models are sorted from smallest to largest based on the verification losses, and the first-order initial energy storage prediction model is selected.

[0038] Calculate the preset ratio multiplied by the total number of serial numbers and round up to the nearest integer. Select the baseline initial energy storage prediction models that satisfy the round up value from the last serial number forward. Using the first serial number initial energy storage prediction model as the target, perform model architecture similarity and node hyperparameter similarity adjustment to generate an updated reinforcement learning initial model architecture. Execute the loop. The preset ratio is a minimum of 0.25 and a maximum of 0.5.

[0039] When any one of the verification losses satisfies the convergence loss threshold, the corresponding initial energy storage prediction model is set as the first energy supply unit energy storage prediction model and added to the plurality of energy storage prediction models.

[0040] In this embodiment, firstly, a first energy supply unit is extracted as a constraint from the plurality of energy supply units, and data on energy storage influencing factors and tags identifying the stored energy are collected at a set time step. For example, one energy supply unit is selected from the plurality of energy supply units in the distributed network as the first energy supply unit; for instance, node #5 photovoltaic energy supply unit is selected as the first energy supply unit. Then, data on energy storage influencing factors and tags identifying the stored energy are collected according to a preset time step. For example, at 12:00 on June 26, 2025, the temperature of the first energy supply unit is 31°C and the irradiance is 850W / m². 2 The data also includes information on energy storage capacity of 82.1 kWh, and data collected at 12:15 PM on June 26, 2025, showing a temperature of 31.5℃ and a light intensity of 855 W / m² for the first energy supply unit. 2 The data includes the energy storage capacity of 82.5 kWh. The time step is set to balance prediction accuracy and computing power consumption. Setting it too short can improve prediction accuracy but consume a lot of computing power, while setting it too long can save computing power but may cause inaccurate prediction. This application recommends setting the time step to 15 minutes. Those skilled in the art can adjust it according to the actual situation. Energy storage influencing factors are factors that can affect the energy storage capacity of the energy supply unit, such as temperature, light intensity, humidity, etc. The labeled energy storage capacity is the actual energy storage capacity of the first energy supply unit, which can be obtained in real time through a smart meter.

[0041] Secondly, several initial reinforcement learning model architectures are obtained, wherein the initial reinforcement learning model architectures are randomly selected from a user-predefined set of discrete model architectures. This user-predefined set of discrete model architectures consists of user-built energy storage prediction models, such as DQN+LSTM models, PPO+CNN models, etc. For example, three initial reinforcement learning model architectures are randomly selected from the user-predefined set of discrete model architectures, such as DQN+LSTM models, PPO+CNN models, and SAC+Transformer models.

[0042] Next, based on the data of energy storage influencing factors at the set time step and the labels identifying the energy storage, several initial reinforcement learning model architectures are trained to generate several initial energy storage prediction models. Each of these initial energy storage prediction models has a validation loss. For example, using the data of energy storage influencing factors at the set time step and the labels identifying the energy storage as the dataset, several initial reinforcement learning model architectures are trained independently to generate several initial energy storage prediction models. Each initial energy storage prediction model has a validation loss. For instance, using the same dataset, a DQN+LSTM model, a PPO+CNN model, and a SAC+Transformer model are trained independently to generate three initial energy storage prediction models, obtaining three corresponding validation losses: 12.8%, 10.5%, and 15.5%.

[0043] Furthermore, when none of the aforementioned verification losses meet the convergence loss threshold, the initial energy storage prediction models are sorted from smallest to largest based on the aforementioned verification losses, and the first-order initial energy storage prediction model is selected. Specifically, when an initial energy storage prediction model among the aforementioned verification losses meets the convergence loss threshold, it is directly added to the list of energy storage prediction models. When none of the aforementioned verification losses meet the convergence loss threshold, the initial energy storage prediction models are sorted from smallest to largest according to the verification losses, and the initial energy storage prediction model ranked first (i.e., with the smallest verification loss) is selected as the first-order initial energy storage prediction model. Here, the convergence loss threshold is an indicator for evaluating whether the initial energy storage prediction model has accurate prediction capabilities. This application recommends setting the convergence loss threshold to 10%, but those skilled in the art can dynamically adjust it according to actual conditions. For example, with a convergence loss threshold of 10%, the validation losses of the three initial energy storage prediction models are 12.8%, 10.5%, and 15.5%, respectively. None of the three initial energy storage prediction models meet the convergence loss threshold. The three initial energy storage prediction models are sorted in order of increasing validation loss, for example, PPO+CNN model, DQN+LSTM model, and SAC+Transformer model. The PPO+CNN model, which ranks first, is selected as the first initial energy storage prediction model.

[0044] Furthermore, the preset ratio is calculated by multiplying the total number of serial numbers by the integer value taken upwards. Starting from the last serial number, baseline initial energy storage prediction models that satisfy the integer value are selected. Taking the first serial number initial energy storage prediction model as the target, the model architecture similarity and node hyperparameter similarity are adjusted to generate an updated reinforcement learning initial model architecture. The process is repeated. The preset ratio is a minimum of 0.25 and a maximum of 0.5. That is, the preset ratio controls the selection of the last 0.25 to 0.5 of several initial energy storage prediction models. This is because if the preset ratio is set too small, too few models will be selected, and it may not be possible to obtain an ideal model through optimization. If the preset ratio is set too large, too many models will be selected, and the resource consumption for model training will be too high. For example, with a preset ratio of 0.5 and a total of 3 sequential numbers arranged from smallest to largest, the integer value upwards is [0.5 * 3] = 2. Two initial energy storage prediction models are selected from the last sequential number backwards: a DQN+LSTM model and a SAC+Transformer model. Then, using the first initial energy storage prediction model (e.g., PPO+CNN model) as the target, adjustments are made to the model architecture similarity and node hyperparameter similarity. For example, the SAC+Transformer model is adjusted by removing the Transformer self-attention layer, and its learning rate and batch size are adjusted to be closer to the first initial energy storage prediction model. This generates an updated reinforcement learning initial model architecture, such as the SAC+CNN model. The reason for selecting baseline initial energy storage prediction models that satisfy the integer value upwards from the last sequential number and optimizing the last model is that the last model has a larger relative difference from the first initial energy storage prediction model, resulting in more optimization space. In a target-guided optimization process, a better solution can be obtained more quickly.

[0045] Finally, when any one of the validation losses satisfies the convergence loss threshold, the corresponding initial energy storage prediction model is set as the first energy supply unit energy storage prediction model and added to the plurality of energy storage prediction models. For example, after adjusting for model architecture similarity and node hyperparameter similarity, an updated SAC+CNN model is generated in the initial reinforcement learning model architecture. Its validation loss is 8.5%, satisfying the convergence loss threshold (e.g., 10%). The SAC+CNN model is then set as the first energy supply unit energy storage prediction model and added to the plurality of energy storage prediction models.

[0046] Furthermore, several energy storage prediction models for several energy supply units are obtained, several energy storage influencing factors in preset time zones are processed, and several predicted energy storage values ​​are generated. For example, following the same construction method as the energy storage prediction model for the first energy supply unit, several energy storage prediction models for several energy supply units are generated. Then, the energy storage influencing factors in the preset time zone of the target energy supply unit are input into the corresponding energy storage prediction model, and the predicted energy storage value of the target energy supply unit is output. For example, the energy storage influencing factors in the preset time zone of the first energy supply unit, such as June 26, 2025, 13:00, temperature 31℃, and light intensity 860W / m², are used. 2 Input the energy storage prediction model of the first energy supply unit and output the predicted energy storage, such as 85.1 kWh.

[0047] In summary, compared to existing technologies, this application obtains several energy storage prediction models for several energy supply units, processes several preset time zone energy storage influencing factors, and generates several predicted energy storage values. Thus, based on deep reinforcement learning, by setting time step energy storage influencing factor data and labeling energy storage values, a predefined set of discrete model architectures is trained and optimized to obtain several energy storage prediction models. Compared to the fixed prediction models of traditional methods, this approach improves the model's generalization ability, predicts the predicted energy storage value of the target energy supply unit, and provides a necessary data foundation for subsequent energy management.

[0048] S20: Based on the aforementioned energy supply units, perform energy storage loss weight distribution on the component reference numbers to obtain a weight distribution matrix of several component reference numbers;

[0049] In distributed energy management, the energy supply unit consists of numerous components, such as battery packs, power converters, and temperature sensors. The impact of each component's malfunction on energy storage losses varies significantly. For example, a battery pack malfunction can cause a sharp increase in energy storage losses, while a power converter malfunction may only have a slight impact. However, traditional energy management methods struggle to quantify the impact of a single component's malfunction on overall energy storage losses and lack dynamic adaptability to changes in environmental factors.

[0050] To address the aforementioned issues, this application, based on the aforementioned energy supply units, performs energy storage loss weight distribution on the component reference numbers to obtain a weight distribution matrix for several component reference numbers.

[0051] Specifically, step S20 in the method includes:

[0052] Extract the model number of the first power supply unit, wherein the model number of the first power supply unit has a set of component reference numbers;

[0053] Using the first energy supply unit model and preset energy storage influencing factors as constants, collect the first energy storage record value of a fully healthy set of component reference numbers. Using the first energy supply unit model and preset energy storage influencing factors as constants, collect the second energy storage record value of an abnormal first component reference number. Calculate the first energy storage loss amount by subtracting the second energy storage record value from the first energy storage record value. If the first energy storage loss amount is greater than or equal to 0, add multiple first component reference number energy loss amounts; otherwise, re-collect.

[0054] Update the preset energy storage influencing factors, execute the preset number of loops, perform mode statistics on the energy loss of multiple first element reference numbers, obtain the energy loss characteristics of the first element reference numbers, and add them to the element reference number energy loss characteristic set.

[0055] Calculate the ratio of the energy loss characteristics of the first component reference number to the sum of the energy loss characteristics of the component reference number, set it as the first component reference number loss weight, and add it to the first energy supply unit component reference number weight distribution matrix;

[0056] Add the first power supply unit component number weight distribution matrix into the plurality of component number weight distribution matrices.

[0057] In this embodiment, the model number of the first power supply unit is first extracted, wherein the model number of the first power supply unit has a set of component reference numbers. Specifically, the component reference number is a unique identifier for several components within the power supply unit. For example, each component can be identified by a two-level code consisting of the component name and the component serial number, such as battery module B1. For example, the extracted model number of the first power supply unit includes the set of component reference numbers: [battery module B1, inverter INV-01, temperature sensor T1, cooling fan F1].

[0058] Secondly, using the model of the first energy supply unit and the preset energy storage influencing factors as constants, the first energy storage record value of the entire set of component reference numbers is collected when all components are healthy. Using the model of the first energy supply unit and the preset energy storage influencing factors as constants, the second energy storage record value of the first component reference number is collected when all components are abnormal. The first energy storage loss is calculated by subtracting the second energy storage record value from the first energy storage record value. That is, the first energy storage loss = the first energy storage record value - the second energy storage record value. If the first energy storage loss is greater than or equal to 0, multiple first component reference number energy loss values ​​are added. Otherwise, the data is collected again. Specifically, firstly, using the model of the first energy supply unit and preset energy storage influencing factors as constants, the first energy storage record value is collected when the set of component reference numbers is in a fully healthy state. This first energy storage record value reflects the ideal energy storage of the first energy supply unit under the current energy storage influencing factors. Then, the energy storage when different component reference numbers are abnormal is collected as the second energy storage record value. This second energy storage record value reflects the energy storage when the first component reference number is abnormal. Theoretically, the second energy storage record value should be less than or equal to the first energy storage record value under a fully healthy state. Therefore, the first energy storage loss should be greater than or equal to 0. When the first energy storage loss is less than 0, it indicates that the currently collected data is unreliable, meaning there may be a collection error, and data needs to be collected again. For example, the first energy supply unit model (e.g., [battery module B1, inverter INV-01, temperature sensor T1, cooling fan F1]) and preset energy storage influencing factors (e.g., temperature 25℃, light intensity 800W / m²) are used. 2 The first energy storage record value (e.g., 70kWh) is a constant. It is collected when the first element tag set is in good condition. This reflects the ideal energy storage under the current energy storage influencing factors. Then, the second energy storage record value (e.g., 65kWh) is collected when the first element tag set is abnormal (e.g., battery module B1 is abnormal). This reflects the energy storage when battery module B1 is abnormal. Then, the first energy storage loss is calculated as 70kWh - 65kWh = 5kWh. Since it is greater than 0, it is added to the energy storage loss of multiple first element tag sets.

[0059] Next, update the preset energy storage influencing factors, execute the loop a preset number of times, perform mode statistics on the energy storage loss of multiple first element reference numbers, obtain the energy storage loss characteristics of the first element reference numbers, and add them to the element reference number energy storage loss characteristic set. For example, update the preset energy storage influencing factors, such as temperature 30℃ and light intensity 800W / m². 2Following the same method as in the previous steps, multiple energy storage loss values ​​for the first component reference number are calculated, such as 5kWh, 6kWh, 5kWh, 4kWh, 5kWh, and 5kWh. Then, mode statistics are performed, and 5kWh is used as the energy storage loss feature of the first component reference number and added to the energy storage loss feature set of the component reference number. Updating the preset energy storage influencing factors is to obtain the energy storage loss value of the first component reference number under different influencing factors, thereby improving data accuracy and generalization ability. Performing mode statistics is to use the most representative data as the energy storage loss feature of the first component reference number.

[0060] Further, the ratio of the energy storage loss characteristic of the first component reference number to the sum of the energy storage loss characteristics of the component reference numbers is calculated and set as the first component reference number loss weight. This weight is added to the first power supply unit component reference number weight distribution matrix. The first component reference number loss weight = first component reference number energy storage loss characteristic / sum of component reference number energy storage loss characteristics. The larger the first component reference number energy storage loss characteristic, the more severe the energy storage loss caused by the first component reference number anomaly, and thus the larger its corresponding first component reference number loss weight. For example, if the first component reference number energy storage loss characteristic is 5kWh and the sum of component reference number energy storage loss characteristics is 30kWh, then the first component reference number loss weight = 5 / 30 = 0.167. This weight is added to the first power supply unit component reference number weight distribution matrix. The first component reference number loss weight reflects the degree of impact of the first component reference number anomaly on energy storage loss; the larger the first component reference number loss weight, the greater the impact.

[0061] Finally, the weight distribution matrix of the first power supply unit component number is added to the weight distribution matrix of several component number. The weight distribution matrix of several component number can reflect the degree of impact of different component number anomalies on energy storage loss.

[0062] In summary, compared to existing technologies, this application, based on the aforementioned energy supply units, performs energy storage loss weight distribution on the component reference numbers to obtain several component reference number weight distribution matrices. Thus, by constructing these component reference number weight distribution matrices, the impact of each component in the energy supply unit on energy storage losses is quantified, providing a necessary data foundation for subsequent energy management.

[0063] S30: Traverse the component reference set of the several energy supply units, perform energy storage loss weight distribution on the component damage type, and obtain several damage type weight distribution matrices.

[0064] Different types of damage have significantly different effects on energy storage loss. For example, sulfation of the battery pack plates will directly lead to a reduction in active material, causing a 15%-30% decrease in energy storage capacity, while the drying of the electrolyte will exacerbate the increase in internal impedance, resulting in a 20%-40% reduction in charge and discharge efficiency. In contrast, slight wear on the battery pack casing has less than 5% impact on energy storage performance.

[0065] To address the aforementioned issues, this application traverses the component reference set of the aforementioned power supply units, performs energy storage loss weight distribution on the component damage types, and obtains several damage type weight distribution matrices.

[0066] Specifically, step S30 in the method includes:

[0067] Using the model of the first energy supply unit and the preset energy storage influencing factors as constants, the third energy storage record value of the first element tag number in the first damage type is collected, and the second energy storage loss is calculated by subtracting the third energy storage record value from the first energy storage record value. If the second energy storage loss is greater than or equal to 0, multiple first damage type energy storage loss values ​​are added; otherwise, the data is collected again.

[0068] Update the preset energy storage influencing factors, execute the preset number of loops, perform mode statistics on the energy storage loss of multiple first damage types, obtain the energy storage loss characteristics of the first damage type, and add the energy storage loss characteristics of multiple damage types.

[0069] Calculate the ratio of the energy storage loss feature of the first damage type to the sum of the energy storage loss features of the damage types, set it as the first damage type loss weight, and add it to the first component reference number damage type weight distribution matrix;

[0070] Add the first component reference number damage type weight distribution matrix into the first power supply unit damage type weight distribution matrix;

[0071] The first power supply unit damage type weight distribution matrix is ​​added to the plurality of damage type weight distribution matrices.

[0072] In this embodiment, firstly, using the model of the first energy supply unit and preset energy storage influencing factors as constants, the third energy storage record value of the first component reference number in the first damage type is collected. The second energy storage loss is calculated by subtracting the third energy storage record value from the first energy storage record value, i.e., the second energy storage loss = the first energy storage record value - the third energy storage record value. If the second energy storage loss is greater than or equal to 0, multiple first damage type energy storage loss values ​​are added; otherwise, the data is collected again. This is because, theoretically, the third energy storage record value should be less than or equal to the first energy storage record value in the fully healthy state. Therefore, the second energy storage loss should be greater than or equal to 0. When the second energy storage loss is less than 0, it indicates that the currently collected data is unreliable, i.e., there may be a collection error, and the data needs to be collected again. For example, using the same method as step S20 above, the third energy storage record value (e.g., 50kWh) of the first component reference number (e.g., battery module B1) in the first damage type (e.g., leakage) is collected to reflect the energy storage of battery module B1 when leakage occurs. Then, the second energy storage loss is calculated as 70kWh - 50kWh = 20kWh. Since it is greater than 0, it is added to the multiple first damage type energy storage loss amounts.

[0073] Secondly, the preset energy storage influencing factors are updated, and the loop is executed a preset number of times. For multiple first-damage-type energy storage loss amounts, mode statistics are performed to obtain the first-damage-type energy storage loss features, which are then added to the multiple damage-type energy storage loss features. For example, following the same steps as in step S20 above, the preset energy storage influencing factors are updated, and multiple third-level energy storage record values ​​under the same energy storage influencing factors are collected. Multiple first-damage-type energy storage loss features are calculated, such as 20kWh, 22kWh, 20kWh, 18kWh, 20kWh, and 21kWh. Then, mode statistics are performed, and 20kWh is used as the first-damage-type energy storage loss feature, which is then added to the multiple damage-type energy storage loss features. Here, updating the preset energy storage influencing factors is to obtain third-level energy storage record values ​​under different influencing factors to improve data accuracy, and performing mode statistics is to use the most representative data as the first-damage-type energy storage loss feature.

[0074] Next, the ratio of the energy storage loss characteristic of the first damage type to the sum of the energy storage loss characteristics of the damage types is calculated and set as the first damage type loss weight. This weight is added to the first element reference number damage type weight distribution matrix. The first damage type loss weight = first damage type energy storage loss characteristic / sum of damage type energy storage loss characteristics. The larger the first damage type energy storage loss characteristic, the more severe the energy storage loss caused by the first damage type, and thus the larger its corresponding first damage type loss weight. For example, if the first damage type energy storage loss characteristic is 20 kWh and the sum of damage type energy storage loss characteristics is 100 kWh, then the first damage type loss weight = 20 / 100 = 0.2. This weight is added to the first element reference number damage type weight distribution matrix. The first damage type loss weight reflects the degree of influence of the first damage type on energy storage loss; the larger the first damage type loss weight, the greater the influence.

[0075] Furthermore, the weight distribution matrix of the first component reference number damage type is added to the weight distribution matrix of the first power supply unit damage type. The weight distribution matrix of the first power supply unit damage type can reflect the degree of influence of the first component reference number damage type on energy storage loss.

[0076] Finally, the damage type weight distribution matrix of the first energy supply unit is added to the several damage type weight distribution matrices. The damage type weight distribution matrix can reflect the degree of influence of several damage types on energy storage loss.

[0077] In summary, compared with the prior art, this application traverses the component reference set of the aforementioned energy supply units, performs energy storage loss weight distribution on the component damage types, and obtains several damage type weight distribution matrices. The damage type weight distribution matrices quantify the degree of impact of different damage types on energy storage losses, providing a necessary data foundation for subsequent energy management.

[0078] S40: Extract the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies, and combine the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types to calculate several power supply incentive factors. The larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution.

[0079] In distributed power management, the service life, types of maintenance damage, and maintenance frequency of components in each power supply unit vary significantly. These factors directly affect the health status of the power supply unit and the reliability of power distribution. Specifically, power supply units with shorter service lives, fewer high-weight damage types, and lower maintenance frequencies typically have higher health levels and power supply stability; conversely, units with longer service lives, severe damage to critical components, and frequent maintenance have higher operational risks and lower power distribution reliability. Therefore, based on these differentiated characteristics, the priority and probability of different power supply units participating in power distribution can be quantified.

[0080] To address the aforementioned issues, this application extracts the service life of several components of several power supply units, a list of several component maintenance and damage types, and a list of several component maintenance frequencies. Combining the weight distribution matrix of the component tag number and the weight distribution matrix of the damage types, several power supply incentive factors are calculated. The larger the power supply incentive factor, the higher the probability that the power supply unit will participate in power distribution.

[0081] Specifically, step S40 in the method includes:

[0082] Construct an evaluation function for energy supply incentive factors:

[0083] in, The energy supply excitation factor characterizing any energy supply unit. The number of components is represented by 'i', and the component number is represented by 'i', which corresponds one-to-one with the component reference number. Characterizes the weight of the i-th element's bit number. Characterizes the service life of the i-th element bit number. The weight representing the damage type of the i-th element bit number is as follows: Characterizing the repair frequency of the j-th damage type, Characterizes the number of damage types for the i-th element bit. It is a natural constant;

[0084] Based on the power supply incentive factor evaluation function, the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies are extracted. Combined with the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types, several power supply incentive factors are calculated.

[0085] In this embodiment, an evaluation function for energy supply incentive factors is first constructed, and several energy supply incentive factors are calculated accordingly. The larger the energy supply incentive factor, the better the health status of the energy supply unit, and the higher the probability that the energy supply unit will participate in power distribution.

[0086] in, The energy supply incentive factor characterizes any energy supply unit. The larger the energy supply incentive factor, the better the health status of the energy supply unit, the more suitable it is to participate in energy supply, and the higher the probability of participating in power distribution.

[0087] in, Characterizes the weight of the i-th element's bit number. The larger the value, the greater the impact of an anomaly in the i-th element's reference number on energy storage losses. The calculated energy supply excitation factor... The smaller.

[0088] in, The weight representing the damage type of the i-th element bit number is as follows: The larger the value, the greater the impact of the j-th damage type on the energy storage loss of the i-th element bit number. The calculated energy supply excitation factor... The smaller.

[0089] in, This characterizes the service life of the i-th component reference number; the shorter the service life of the i-th component reference number, the longer the service life. The smaller, The smaller the term, the greater the energy-supplying incentive factor. The larger the value, the lower the probability of damage to the i-th component reference number, which often indicates better health and power supply reliability.

[0090] in, Characterizing the repair frequency of the j-th damage type, The larger the value, the higher the maintenance frequency of the j-th damage type, and the more efficient the calculated energy supply excitation factor. The smaller.

[0091] Furthermore, based on the power supply incentive factor evaluation function, the service life of several components of several power supply units, the list of several component repair damage types, and the list of several component repair frequencies are extracted. For example, the service life of several components of several power supply units can be extracted from the equipment ledger or IoT sensors. For instance, the service life of battery module B1 is 1000 hours. The list of several component repair damage types and the list of several component repair frequencies can be extracted from historical maintenance records. For instance, the repair frequency of leakage in battery module B1 is 0.5 times / month.

[0092] Finally, the service life, damage type list, and maintenance frequency list of several components from several energy supply units, combined with the component tag number weight distribution matrix and the damage type weight distribution matrix, are substituted into the energy supply incentive factor evaluation function to calculate several energy supply incentive factors. The service life, damage type list, and maintenance frequency list of several energy supply units dynamically integrate multi-dimensional parameters of component health status. The component tag number weight distribution matrix and the damage type weight distribution matrix quantify the impact of different component tag numbers and damage types on energy storage losses. Thus, the power distribution priority of different energy supply units is quantified through the energy supply incentive factors, and the probability of energy supply units participating in power distribution is configured accordingly, driving the selection of energy supply units with high health status to participate in power distribution preferentially.

[0093] In summary, compared to existing technologies, this application extracts the service life of several components of several power supply units, a list of several component maintenance and damage types, and a list of several component maintenance frequencies. Combining these with the weight distribution matrices of the component tag numbers and the weight distribution matrices of the damage types, several power supply incentive factors are calculated. The larger the power supply incentive factor, the higher the probability that the power supply unit will participate in power distribution. Thus, by dynamically integrating multi-dimensional parameters of component health status and the impact of different component tag numbers and damage types on energy storage losses, the probability of different power supply units participating in power distribution is quantified through the power supply incentive factors. Based on this, power supply units with high health status can be preferentially selected for power distribution.

[0094] S50: Based on the service life of the line, the line specifications and service environment parameters, aging analysis is performed on several line segments to obtain several line segment aging factors. Combined with the length of several line segments, several line segment energy transmission incentive factors are constructed. Among them, the larger the energy transmission incentive factor, the smaller the aging factor and the shorter the line segment length.

[0095] During energy transmission, the aging degree and length of each line segment vary, resulting in differences in energy loss, transmission efficiency, and reliability. Consequently, the priority and frequency of use of each line segment in energy transmission should also differ.

[0096] To address the aforementioned issues, this application conducts aging analysis on several line segments based on line service duration, line specifications, and service environment parameters, obtaining several line segment aging factors. Combined with the lengths of these line segments, it constructs several line segment energy transmission incentive factors. The larger the energy transmission incentive factor, the smaller the aging factor and the shorter the line segment length.

[0097] Specifically, step S50 in the method includes:

[0098] Constrained by the service life of the line, the line specifications and the service environment parameters, a set of similar sample line segments of the first line segment is collected, the mode statistics of the power transmission loss rate per unit distance are performed, the aging factor of the first line segment is generated, and the aging factor of the several line segments is added to it.

[0099] Construct an evaluation function for energy delivery incentive factors: ,

[0100] in, The energy delivery excitation factor characterizing any given line segment. Characterizing the length of the line segment, Characterizing the aging factors of the line segment, It is a natural constant;

[0101] Based on the energy transmission incentive factor evaluation function, the energy transmission incentive factors for the several line segments are constructed.

[0102] In this embodiment, firstly, constrained by the line service life, line specifications, and service environment parameters, a set of similar sample line segments for the first line segment is collected. Then, mode statistics of the unit distance power loss rate are performed to generate an aging factor for the first line segment. This factor is then added to the aging factors of the aforementioned line segments. The line service life is the cumulative total time since the line was put into use; the longer the service time, the higher the likelihood of line aging. Line specifications include wire diameter, material, and insulation material. Different line specifications affect the electrical performance and aging resistance of the line. For example, copper cables have better conductivity and are more corrosion-resistant than aluminum cables. Service environment parameters include temperature, humidity, pH, and electromagnetic environment. High temperature and high humidity environments accelerate the aging of the line insulation layer, while strong electromagnetic environments may affect the line transmission performance. The unit distance power loss rate refers to the proportion of energy loss per unit distance transmitted. Mode statistics select the unit distance power loss rate with the highest frequency as a representative value. Mode statistics reduce outlier interference, making the generated first line segment aging factor more representative. For example, constraints are imposed by line service duration, line specifications, and service environment parameters. For instance, constraints may be imposed by line service duration (e.g., 2 years) and line specifications (e.g., 120mm copper core). 2 Constrained by service environment parameters (such as temperate 22℃ and humidity 45%-55%), five similar lines were collected to obtain a set of similar sample line segments for the first line segment. The unit distance power transmission loss rate of the five similar lines is 10%, 8%, 10%, 10%, and 10% respectively. The mode statistics of the unit distance power transmission loss rate were performed to generate the aging factor of the first line segment (such as 10%), and several line segment aging factors were added.

[0103] Secondly, construct the evaluation function for energy delivery incentive factors: ,

[0104] in, The energy delivery excitation factor characterizing any given line segment. Characterizing the length of the line segment, Characterizing the aging factors of the line segment, As a natural constant, the longer the line segment, the higher the transmission loss rate; the larger the line segment aging factor, the more severe the line aging and the lower the transmission efficiency; the smaller the calculated energy transmission excitation factor, the less suitable the line is for energy transmission.

[0105] Finally, based on the energy transmission incentive factor evaluation function, energy transmission incentive factors for the several line segments are constructed. For example, based on the above energy transmission incentive factor evaluation function, the lengths corresponding to each line segment are... and aging factors Substitute the values ​​and calculate the energy delivery excitation factor for each line segment. Several line segment energy transmission excitation factors are obtained. For example, when the line segment length is 50 meters and the line segment aging factor is 10%, the following calculations are performed: This provides a quantitative basis for the selection of lines in energy management. When scheduling energy transportation, prioritizing line segments with high energy delivery incentive factors helps reduce transmission losses and improve energy transmission efficiency.

[0106] In summary, compared to existing technologies, this application conducts aging analysis on several line segments based on line service life, line specifications, and service environment parameters to obtain aging factors for these segments. Combined with the lengths of these segments, it constructs energy delivery incentive factors for each segment. A higher energy delivery incentive factor corresponds to a lower aging factor and a shorter segment length. Thus, by combining the energy delivery incentive factor with the degree of line aging and length, energy dispatch prioritizes line segments with higher energy delivery incentive factors, providing a quantitative basis for line selection in energy management and helping to reduce transmission losses and improve energy transmission efficiency.

[0107] S60: Based on the aforementioned energy supply incentive factors and the aforementioned line segment energy transmission incentive factors, and in conjunction with the predicted energy storage, perform energy management.

[0108] Energy supply incentive factors can reflect the health status of components in energy supply units, while energy delivery incentive factors can reflect line transmission efficiency and transmission loss. Based on these, energy distribution schemes can be quantitatively scored, thereby generating dynamic energy management strategies that take into account both the health management of energy supply units and the optimization of line losses.

[0109] To address the aforementioned issues, this application implements energy management based on the aforementioned energy supply incentive factors and the aforementioned line segment energy transmission incentive factors, combined with the predicted energy storage.

[0110] Specifically, step S60 in the method includes:

[0111] The Delphi method is used to assign a first weight to the energy supply incentive factor and a second weight to the energy delivery incentive factor.

[0112] Based on the first weight and the second weight, a fitness function is constructed;

[0113] Based on the predicted energy storage, and with the target energy distribution as a constraint, several energy distribution schemes are constructed. Based on the fitness function, the several energy supply incentive factors, and the several line segment energy delivery incentive factors, several fitness values ​​are calculated.

[0114] Select the energy distribution scheme with the minimum fitness value from the given fitness values ​​and perform energy management.

[0115] In this embodiment, the Delphi method is first used to assign a first weight to the energy supply incentive factor and a second weight to the energy transmission incentive factor. The Delphi method is a structured prediction method that achieves consensus-based group decision-making through multiple rounds of anonymous expert consultation and feedback convergence. For example, the Delphi method can invite experts in power system planning, energy storage operation and maintenance, and transmission line engineering to assign dynamic weights to the energy supply and energy transmission incentive factors. and ,For example, , .

[0116] Secondly, based on the first weight and the second weight, a fitness function is constructed, wherein the fitness function... ,in, The energy supply excitation factor characterizing any energy supply unit. The energy delivery excitation factor characterizing any given line segment. As the first weight, As the second weight, the fitness function can score different energy distribution schemes. The smaller the fitness value, the better the energy distribution scheme.

[0117] Furthermore, based on the predicted energy storage and constrained by the target energy allocation, several energy distribution schemes are constructed. Based on the fitness function, the several energy supply incentive factors, and the several line segment energy delivery incentive factors, several fitness values ​​are calculated. For example, using the predicted energy storage and target energy allocation of each energy supply unit as constraints, several energy distribution schemes can be constructed by enumerating all possible combinations of energy supply units and lines in a small-scale energy management process. Alternatively, genetic algorithms, particle swarm optimization, etc., can be used to construct several energy distribution schemes for a large-scale energy management process. Then, based on the fitness function, the several energy supply incentive factors, and the several line segment energy delivery incentive factors, several fitness values ​​are calculated.

[0118] Finally, the energy distribution scheme with the lowest fitness value is selected for energy management. For example, Scheme 1 selects two energy supply units with high energy supply incentive factors paired with three lines with high energy delivery incentive factors, and Scheme 2 selects three energy supply units with medium energy supply incentive factors paired with two lines with medium energy delivery incentive factors. The calculated fitness value of Scheme 2 is 2.68, which is better than Scheme 1's 3.02. Therefore, Scheme 2 is selected as the preferred energy distribution scheme for energy management.

[0119] In summary, compared to existing technologies, this application performs energy management based on the aforementioned energy supply incentive factors and the aforementioned line segment energy delivery incentive factors, combined with the predicted energy storage capacity. Thus, by comprehensively considering the predicted energy storage capacity, energy supply unit status data, and line parameters, different energy distribution schemes can be scored using a fitness function to determine the appropriate energy distribution scheme and perform energy management.

[0120] In summary, the embodiments of this application have at least the following technical effects: Compared to existing technologies, this application first obtains several energy storage prediction models for several energy supply units, processes several energy storage influencing factors in preset time zones, and generates several predicted energy storage values. Thus, based on deep reinforcement learning, by setting time-step energy storage influencing factor data and labeling energy storage values, a predefined set of discrete model architectures is trained and optimized to obtain several energy storage prediction models. Compared to the fixed prediction models of traditional methods, this approach can improve the generalization ability of the models, predict the predicted energy storage values ​​of the target energy supply units, and provide a necessary data foundation for subsequent energy management.

[0121] Secondly, based on the aforementioned energy supply units, this application performs energy storage loss weight distribution on the component reference numbers to obtain several component reference number weight distribution matrices. Thus, by constructing the component reference number weight distribution matrices, the impact of each component of the energy supply unit on energy storage losses is quantified, providing a necessary data foundation for subsequent energy management.

[0122] Furthermore, this application traverses the component reference set of the aforementioned energy supply units, performs energy storage loss weight distribution on the component damage types, and obtains several damage type weight distribution matrices. The damage type weight distribution matrices quantify the degree of impact of different damage types on energy storage losses, providing a necessary data foundation for subsequent energy management.

[0123] Furthermore, this application extracts the service life of several components, a list of several component maintenance and damage types, and a list of several component maintenance frequencies from several energy supply units. Combining these with the component tag number weight distribution matrix and the damage type weight distribution matrix, several energy supply incentive factors are calculated. The larger the energy supply incentive factor, the higher the probability that the energy supply unit will participate in power distribution. Thus, by dynamically integrating multi-dimensional parameters of component health status and the impact of different component tag numbers and damage types on energy storage losses, the probability of different energy supply units participating in power distribution is quantified through the energy supply incentive factors. Based on this, energy supply units with high health status can be preferentially selected to participate in power distribution.

[0124] Furthermore, this application conducts aging analysis on several line segments based on line service duration, line specifications, and service environment parameters to obtain aging factors for these segments. Combined with the lengths of these segments, it constructs energy delivery incentive factors for each segment. A higher energy delivery incentive factor corresponds to a lower aging factor and a shorter segment length. Thus, by combining the energy delivery incentive factor with the degree of line aging and length, energy dispatch prioritizes line segments with higher energy delivery incentive factors, providing a quantitative basis for line selection in energy management and helping to reduce transmission losses and improve energy transmission efficiency.

[0125] Finally, this application performs energy management based on the aforementioned energy supply incentive factors and the aforementioned line segment energy delivery incentive factors, combined with the predicted energy storage capacity. Thus, by comprehensively considering the predicted energy storage capacity, energy supply unit status data, and line parameters, different energy distribution schemes can be scored using a fitness function to determine the energy distribution scheme and perform energy management.

[0126] Through the above technical solution, this application constructs several energy storage prediction models for several energy supply units using deep reinforcement learning. Based on preset time zone energy storage influencing factors, it accurately outputs several predicted energy storage values. It considers the impact of different component tag number anomalies and different damage types on energy storage losses. Furthermore, it quantifies the probability of different energy supply units participating in power distribution and line transmission efficiency. Finally, it scores different energy distribution schemes using a fitness function, determines the energy distribution scheme based on the fitness, and executes energy management. In this way, it accurately predicts energy data for future time zones and conducts effective proactive management accordingly.

[0127] Example 2, as Figure 2 As shown, based on the same inventive concept as the energy management method based on deep reinforcement learning provided in Embodiment 1, this embodiment of the invention also provides an energy management system based on deep reinforcement learning, including:

[0128] Energy storage prediction module 11 is used to obtain several energy storage prediction models for several energy supply units, process several preset time zone energy storage influencing factors, and generate several predicted energy storage. The energy storage prediction model is generated by deep reinforcement learning training through multiple sets of data. Each set of the multiple sets of data includes the set time step energy storage influencing factor data of the target energy storage unit and the label identifying the energy storage.

[0129] The tag number weight configuration module 12 is used to perform energy storage loss weight distribution on the tag numbers of the components based on the plurality of energy supply units, and obtain a plurality of component tag number weight distribution matrices.

[0130] The type weight configuration module 13 is used to traverse the component reference set of the plurality of energy supply units, perform energy storage loss weight distribution on the component damage type, and obtain a plurality of damage type weight distribution matrices.

[0131] The power supply incentive factor construction module 14 is used to extract the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies, and calculate several power supply incentive factors by combining the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types. The larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution.

[0132] The energy delivery incentive factor construction module 15 is used to perform aging analysis on several line segments based on the line service duration, line specification parameters and service environment parameters, obtain several line segment aging factors, and construct several line segment energy delivery incentive factors in combination with the length of several line segments. Among them, the larger the energy delivery incentive factor, the smaller the aging factor and the shorter the line segment length.

[0133] The fusion execution module 16 is used to perform energy management based on the plurality of energy supply incentive factors and the plurality of line segment energy transmission incentive factors, combined with the predicted energy storage.

[0134] Specifically, the energy storage prediction module 11 is used for:

[0135] From the aforementioned energy supply units, the first energy supply unit is extracted as a constraint, and data on energy storage influencing factors and tags identifying the energy storage capacity are collected at a set time step.

[0136] Several initial reinforcement learning model architectures are obtained, wherein the initial reinforcement learning model architectures are randomly selected from a set of discrete model architectures predefined by the user;

[0137] Based on the data of energy storage influencing factors and the labels of the identified energy storage, several initial reinforcement learning model architectures are trained to generate several initial energy storage prediction models, wherein the several initial energy storage prediction models have several validation losses.

[0138] When none of the verification losses meet the convergence loss threshold, the initial energy storage prediction models are sorted from smallest to largest based on the verification losses, and the first-order initial energy storage prediction model is selected.

[0139] Calculate the preset ratio multiplied by the total number of serial numbers and round up to the nearest integer. Select the baseline initial energy storage prediction models that satisfy the round up value from the last serial number forward. Using the first serial number initial energy storage prediction model as the target, perform model architecture similarity and node hyperparameter similarity adjustment to generate an updated reinforcement learning initial model architecture. Execute the loop. The preset ratio is a minimum of 0.25 and a maximum of 0.5.

[0140] When any one of the verification losses satisfies the convergence loss threshold, the corresponding initial energy storage prediction model is set as the first energy supply unit energy storage prediction model and added to the plurality of energy storage prediction models.

[0141] Specifically, the position weight configuration module 12 is used for:

[0142] Extract the model number of the first power supply unit, wherein the model number of the first power supply unit has a set of component reference numbers;

[0143] Using the first energy supply unit model and preset energy storage influencing factors as constants, collect the first energy storage record value of a fully healthy set of component reference numbers. Using the first energy supply unit model and preset energy storage influencing factors as constants, collect the second energy storage record value of an abnormal first component reference number. Calculate the first energy storage loss amount by subtracting the second energy storage record value from the first energy storage record value. If the first energy storage loss amount is greater than or equal to 0, add multiple first component reference number energy loss amounts; otherwise, re-collect.

[0144] Update the preset energy storage influencing factors, execute the preset number of loops, perform mode statistics on the energy loss of multiple first element reference numbers, obtain the energy loss characteristics of the first element reference numbers, and add them to the element reference number energy loss characteristic set.

[0145] Calculate the ratio of the energy loss characteristics of the first component reference number to the sum of the energy loss characteristics of the component reference number, set it as the first component reference number loss weight, and add it to the first energy supply unit component reference number weight distribution matrix;

[0146] Add the first power supply unit component number weight distribution matrix into the plurality of component number weight distribution matrices.

[0147] Specifically, the type weight configuration module 13 is used for:

[0148] Using the model of the first energy supply unit and the preset energy storage influencing factors as constants, the third energy storage record value of the first element tag number in the first damage type is collected, and the second energy storage loss is calculated by subtracting the third energy storage record value from the first energy storage record value. If the second energy storage loss is greater than or equal to 0, multiple first damage type energy storage loss values ​​are added; otherwise, the data is collected again.

[0149] Update the preset energy storage influencing factors, execute the preset number of loops, perform mode statistics on the energy storage loss of multiple first damage types, obtain the energy storage loss characteristics of the first damage type, and add the energy storage loss characteristics of multiple damage types.

[0150] Calculate the ratio of the energy storage loss feature of the first damage type to the sum of the energy storage loss features of the damage types, set it as the first damage type loss weight, and add it to the first component reference number damage type weight distribution matrix;

[0151] Add the first component reference number damage type weight distribution matrix into the first power supply unit damage type weight distribution matrix;

[0152] The first power supply unit damage type weight distribution matrix is ​​added to the plurality of damage type weight distribution matrices.

[0153] Specifically, the energy supply excitation factor construction module 14 is used for:

[0154] Construct an evaluation function for energy supply incentive factors: ,

[0155] in, The energy supply excitation factor characterizing any energy supply unit. The number of components is represented by 'i', and the component number is represented by 'i', which corresponds one-to-one with the component reference number. Characterizes the weight of the i-th element's bit number. Characterizes the service life of the i-th element bit number. The weight representing the damage type of the i-th element bit number is as follows: Characterizing the repair frequency of the j-th damage type, Characterizes the number of damage types for the i-th element bit. It is a natural constant;

[0156] Based on the power supply incentive factor evaluation function, the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies are extracted. Combined with the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types, several power supply incentive factors are calculated.

[0157] Specifically, the energy delivery excitation factor construction module 15 is used for:

[0158] Constrained by the service life of the line, the line specifications and the service environment parameters, a set of similar sample line segments of the first line segment is collected, the mode statistics of the power transmission loss rate per unit distance are performed, the aging factor of the first line segment is generated, and the aging factor of the several line segments is added to it.

[0159] Construct an evaluation function for energy delivery incentive factors: ,

[0160] in, The energy delivery excitation factor characterizing any given line segment. Characterizing the length of the line segment, Characterizing the aging factors of the line segment, It is a natural constant;

[0161] Based on the energy transmission incentive factor evaluation function, the energy transmission incentive factors for the several line segments are constructed.

[0162] The fusion execution module 16 is specifically used for:

[0163] The Delphi method is used to assign a first weight to the energy supply incentive factor and a second weight to the energy delivery incentive factor.

[0164] Based on the first weight and the second weight, a fitness function is constructed;

[0165] Based on the predicted energy storage, and with the target energy distribution as a constraint, several energy distribution schemes are constructed. Based on the fitness function, the several energy supply incentive factors, and the several line segment energy delivery incentive factors, several fitness values ​​are calculated.

[0166] Select the energy distribution scheme with the minimum fitness value from the given fitness values ​​and perform energy management.

[0167] In summary, the embodiments of this application have at least the following technical effects:

[0168] Compared to existing technologies, this application firstly obtains several energy storage prediction models for several energy supply units through an energy storage prediction module, processes several preset time zone energy storage influencing factors, and generates several predicted energy storage values. Based on deep reinforcement learning, it trains and optimizes a predefined discrete model architecture set by setting time step energy storage influencing factor data and identifying energy storage labels, thus obtaining several energy storage prediction models. Compared to the fixed prediction models of traditional methods, this improves the model's generalization ability and predicts the predicted energy storage of the target energy supply unit, providing a necessary data foundation for subsequent energy management. Secondly, through a tag number weight configuration module, it distributes energy storage loss weights on the tag numbers of several energy supply units, obtaining several tag number weight distribution matrices. By constructing these matrix matrices, the impact of each element in the energy supply unit on energy storage loss can be quantified. Thirdly, through a type weight configuration module, it iterates through the tag number set of several energy supply units, distributes energy storage loss weights on element damage types, and obtains several damage type weight distribution matrices. These matrices quantify the impact of different damage types on energy storage loss, providing a necessary data foundation for subsequent energy management. Furthermore, through the energy supply incentive factor construction module, the service life of several components, the list of maintenance and damage types of several components, and the list of maintenance frequencies of several components for several energy supply units are extracted. Combined with the weight distribution matrix of several component tag numbers and the weight distribution matrix of the aforementioned damage types, several energy supply incentive factors are calculated. By dynamically integrating multi-dimensional parameters of component health status and the impact of different component tag numbers and damage types on energy storage losses, the probability of different energy supply units participating in power distribution is quantified through the energy supply incentive factors. Based on this, units with high health status can be prioritized for power distribution. Furthermore, through the energy delivery incentive factor construction module, based on the line service life, line specifications, and service environment parameters, aging analysis is performed on several line segments to obtain several line segment aging factors. Combined with the length of several line segments, several line segment energy delivery incentive factors are constructed. These energy delivery incentive factors, combined with the degree of line aging and length, prioritize line segments with large energy delivery incentive factors during energy dispatch. This provides a quantitative basis for line selection in energy management and helps reduce transmission losses and improve energy transmission efficiency. Finally, through the fusion execution module, based on several energy supply incentive factors and several line segment energy delivery incentive factors, combined with predicted energy storage, energy management is performed. This comprehensively considers predicted energy storage, energy supply unit status data, and line parameters. A fitness function is used to score different energy distribution schemes, determine the appropriate energy distribution scheme, and execute energy management. In this way, energy data for future time zones is accurately predicted, and effective proactive management is carried out accordingly.

[0169] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0170] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0172] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0174] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0175] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. An energy management method based on deep reinforcement learning, characterized in that, include: Several energy storage prediction models for several energy supply units are obtained, several energy storage influencing factors in preset time zones are processed, and several predicted energy storage is generated. The energy storage prediction models are generated by deep reinforcement learning training through multiple sets of data. Each set of multiple sets of data includes energy storage influencing factor data at a set time step for the target energy storage unit and a label identifying the energy storage. Based on the aforementioned energy supply units, an energy storage loss weight distribution is performed on the component reference numbers to obtain a weight distribution matrix of several component reference numbers. Traverse the component reference set of the several power supply units, perform energy storage loss weight distribution on the component damage type, and obtain several damage type weight distribution matrices; Extract the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies, and combine the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types to calculate several power supply incentive factors. The larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution. Based on the service life of the line, the line specifications and service environment parameters, aging analysis is performed on several line segments to obtain several line segment aging factors. Combined with the length of several line segments, several line segment energy transmission incentive factors are constructed. Among them, the larger the energy transmission incentive factor, the smaller the aging factor and the shorter the line segment length. Energy management is performed based on the aforementioned energy supply incentive factors and the aforementioned line segment energy transmission incentive factors, combined with the predicted energy storage.

2. The method as described in claim 1, characterized in that, Several energy storage prediction models for several energy supply units were obtained, including: From the aforementioned energy supply units, the first energy supply unit is extracted as a constraint, and data on energy storage influencing factors and tags identifying the energy storage capacity are collected at a set time step. Several initial reinforcement learning model architectures are obtained, wherein the initial reinforcement learning model architectures are randomly selected from a set of discrete model architectures predefined by the user; Based on the data of energy storage influencing factors and the labels of the identified energy storage, several initial reinforcement learning model architectures are trained to generate several initial energy storage prediction models, wherein the several initial energy storage prediction models have several validation losses. When none of the verification losses meet the convergence loss threshold, the initial energy storage prediction models are sorted from smallest to largest based on the verification losses, and the first-order initial energy storage prediction model is selected. Calculate the preset ratio multiplied by the total number of serial numbers and round up to the nearest integer. Select the baseline initial energy storage prediction models that satisfy the round up value from the last serial number forward. Using the first serial number initial energy storage prediction model as the target, perform model architecture similarity and node hyperparameter similarity adjustment to generate an updated reinforcement learning initial model architecture. Execute the loop. The preset ratio is a minimum of 0.25 and a maximum of 0.

5. When any one of the verification losses satisfies the convergence loss threshold, the corresponding initial energy storage prediction model is set as the first energy supply unit energy storage prediction model and added to the plurality of energy storage prediction models.

3. The method as described in claim 1, characterized in that, Based on the aforementioned power supply units, an energy storage loss weight distribution is performed on the component reference numbers to obtain several component reference number weight distribution matrices, including: Extract the model number of the first power supply unit, wherein the model number of the first power supply unit has a set of component reference numbers; Using the first energy supply unit model and preset energy storage influencing factors as constants, collect the first energy storage record value of a fully healthy set of component reference numbers. Using the first energy supply unit model and preset energy storage influencing factors as constants, collect the second energy storage record value of an abnormal first component reference number. Calculate the first energy storage loss amount by subtracting the second energy storage record value from the first energy storage record value. If the first energy storage loss amount is greater than or equal to 0, add multiple first component reference number energy loss amounts; otherwise, re-collect. Update the preset energy storage influencing factors, execute the preset number of loops, perform mode statistics on the energy loss of multiple first element reference numbers, obtain the energy loss characteristics of the first element reference numbers, and add them to the element reference number energy loss characteristic set. Calculate the ratio of the energy loss characteristics of the first component reference number to the sum of the energy loss characteristics of the component reference number, set it as the first component reference number loss weight, and add it to the first energy supply unit component reference number weight distribution matrix; Add the first power supply unit component number weight distribution matrix into the plurality of component number weight distribution matrices.

4. The method as described in claim 3, characterized in that, By traversing the component reference set of the aforementioned power supply units, energy storage loss weight distribution is performed on the component damage type to obtain several damage type weight distribution matrices, including: Using the model of the first energy supply unit and the preset energy storage influencing factors as constants, the third energy storage record value of the first element tag number in the first damage type is collected, and the second energy storage loss is calculated by subtracting the third energy storage record value from the first energy storage record value. If the second energy storage loss is greater than or equal to 0, multiple first damage type energy storage loss values ​​are added; otherwise, the data is collected again. Update the preset energy storage influencing factors, execute the preset number of loops, perform mode statistics on the energy storage loss of multiple first damage types, obtain the energy storage loss characteristics of the first damage type, and add the energy storage loss characteristics of multiple damage types. Calculate the ratio of the energy storage loss feature of the first damage type to the sum of the energy storage loss features of the damage types, set it as the first damage type loss weight, and add it to the first component reference number damage type weight distribution matrix; Add the first component reference number damage type weight distribution matrix into the first power supply unit damage type weight distribution matrix; The first power supply unit damage type weight distribution matrix is ​​added to the plurality of damage type weight distribution matrices.

5. The method as described in claim 1, characterized in that, Extract the service life of several components from several power supply units, a list of several component maintenance damage types, and a list of several component maintenance frequencies. Combine these with the component tag number weight distribution matrix and the damage type weight distribution matrix to calculate several power supply excitation factors, including: Construct an evaluation function for energy supply incentive factors: ， in, The energy supply excitation factor characterizing any energy supply unit. The number of components is represented by 'i', and the component number is represented by 'i', which corresponds one-to-one with the component reference number. Characterizes the weight of the i-th element's bit number. Characterizes the service life of the i-th element bit number. The weight representing the damage type of the i-th element bit number is as follows: Characterizing the repair frequency of the j-th damage type, Characterizes the number of damage types for the i-th element bit. It is a natural constant; Based on the power supply incentive factor evaluation function, the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies are extracted. Combined with the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types, several power supply incentive factors are calculated.

6. The method as described in claim 1, characterized in that, Based on the service life, specifications, and environmental parameters of the lines, aging analysis was conducted on several line sections to obtain aging factors. Combined with the lengths of these line sections, power transmission excitation factors were constructed, including: Constrained by the service life of the line, the line specifications and the service environment parameters, a set of similar sample line segments of the first line segment is collected, the mode statistics of the power transmission loss rate per unit distance are performed, the aging factor of the first line segment is generated, and the aging factor of the several line segments is added to it. Construct an evaluation function for energy delivery incentive factors: ， in, The energy delivery excitation factor characterizing any given line segment. Characterizing the length of the line segment, Characterizing the aging factor of the line segment, It is a natural constant; Based on the energy transmission incentive factor evaluation function, the energy transmission incentive factors for the several line segments are constructed.

7. The method as described in claim 1, characterized in that, Based on the aforementioned energy supply incentive factors and the aforementioned line segment energy transmission incentive factors, and in conjunction with the predicted energy storage, energy management is performed, including: The Delphi method is used to assign a first weight to the energy supply incentive factor and a second weight to the energy delivery incentive factor. Based on the first weight and the second weight, a fitness function is constructed; Based on the predicted energy storage, and with the target energy distribution as a constraint, several energy distribution schemes are constructed. Based on the fitness function, the several energy supply incentive factors, and the several line segment energy delivery incentive factors, several fitness values ​​are calculated. Select the energy distribution scheme with the minimum fitness value from the given fitness values ​​and perform energy management.

8. An energy management system based on deep reinforcement learning, characterized in that, For performing the method according to any one of claims 1-7, comprising: The energy storage prediction module is used to obtain several energy storage prediction models for several energy supply units, process several preset time zone energy storage influencing factors, and generate several predicted energy storage. The energy storage prediction model is generated by deep reinforcement learning training through multiple sets of data. Each set of the multiple sets of data includes energy storage influencing factor data at a set time step of the target energy storage unit and a label identifying the energy storage. The tag number weight configuration module is used to distribute the energy storage loss weight of the tag number based on the plurality of energy supply units, and obtain a plurality of tag number weight distribution matrices. The type weight configuration module is used to traverse the component reference set of the plurality of energy supply units, perform energy storage loss weight distribution on the component damage type, and obtain a plurality of damage type weight distribution matrices. The power supply incentive factor construction module is used to extract the service life of several components of several power supply units, the list of several component maintenance damage types and the list of several component maintenance frequencies, and calculate several power supply incentive factors by combining the weight distribution matrix of the several component tag numbers and the weight distribution matrix of the several damage types. The larger the power supply incentive factor, the higher the probability of the power supply unit participating in power distribution. The energy delivery incentive factor construction module is used to perform aging analysis on several line segments based on line service duration, line specification parameters, and service environment parameters, obtain several line segment aging factors, and construct several line segment energy delivery incentive factors in combination with the length of several line segments. Among them, the larger the energy delivery incentive factor, the smaller the aging factor and the shorter the line segment length. The fusion execution module is used to perform energy management based on the plurality of energy supply incentive factors and the plurality of line segment energy delivery incentive factors, combined with the predicted energy storage.

Images