Ship power system fault network dynamic reconstruction method based on deep reinforcement learning

Abstract

The invention discloses a ship power system fault network dynamic reconstruction method based on deep reinforcement learning, and relates to the technical field of power system information. The ship power system fault network dynamic reconstruction method based on deep reinforcement learning mainly comprises the following steps: topologizing a ship power system circuit diagram to obtain a directed graph, constructing an objective function and a Markov decision model, and training a decision dual depth Q network algorithm by using a priority experience playback strategy; and performing fault network dynamic reconstruction on the to-be-analyzed ship power system to obtain a dynamic switch sequence. By implementing the ship power system fault network dynamic reconstruction method based on deep reinforcement learning provided by the invention, the speed and accuracy of dynamic fault reconstruction can be improved.

Description

Technical Field

[0001] The present invention relates to the field of power system information technology, and more specifically, to a method for dynamically reconstructing a fault network of a ship power system based on deep reinforcement learning. Background Art

[0002] With the continuous advancement of marine technology, modern ships are gradually moving towards full electrification and automation. A major challenge facing modern ship electrical systems is how to ensure the survivability and operability of the ship in the event of a fault. Unlike land power systems, the topology of ship power systems is unique. Its power supply lines are short, and the current will increase rapidly in the event of a short circuit, which will quickly affect the stability of the entire ship power system, posing a huge potential threat to the life safety of personnel on board. Therefore, when a fault occurs, the ship electrical system must be able to quickly isolate the fault and reconfigure the system topology in the initial stage of the electrical fault to restore power supply to the greatest extent possible. Therefore, the ship power system urgently needs an effective fault reconstruction strategy to enhance its ability and reliability to respond to emergencies.

[0003] In recent years, with the rapid development of artificial intelligence technology, Deep Reinforcement Learning (DRL) is being applied to different service areas of power systems more and more in an unprecedented manner. As an advanced artificial intelligence method, Deep Reinforcement Learning effectively combines two valuable artificial intelligence technologies: the powerful feature extraction ability of deep learning and the decision optimization ability of reinforcement learning. It does not passively accept knowledge, but actively interacts in a complex and changing environment, constantly draws lessons from environmental feedback, and dynamically learns how to make the most sensible and appropriate decisions in various different situations, providing strong technical support for the efficient operation, stability and reliability of the power system. Therefore, this technology is suitable for the dynamic reconstruction of ship power systems.

[0004] Zhao et al. (2020) proposed a grid reconstruction method based on deep reinforcement learning to reduce the error between the expected offline optimization scheme and the actual load scheduling, and realize a two-level solution for distribution network reconstruction; Di et al. (2020) proposed a grid reconstruction method based on deep reinforcement learning to reduce the error between the expected offline optimization scheme and the actual load scheduling, and realize a two-level solution for distribution network reconstruction; Wang et al. (2021) proposed a reconstruction method based on model-free reinforcement learning, using a deep Q network with a noisy network (NoisyNetDQN) to accelerate the training process and optimize performance; Gautam et al. (2022) proposed a distributed network reconstruction optimization framework based on a deep Q network (DQN) with the objective function of minimizing the average clipping power to improve the network configuration reliability of the power system; Bui and Su (2022) proposed an optimization method based on three-stage deep reinforcement learning to determine the set value of the power supply and the optimal reconstruction of the power grid with the goal of minimizing the operating cost and switching load. However, the studies of the above scholars all use deep reinforcement learning algorithms to solve the fault reconstruction or circuit reconstruction optimization problems of ground power grids. Das et al. (2012) studied a fault reconstruction method for ship power systems based on the Q-Learning method, which is used to obtain a set of optimal switch combinations in the event of a fault to restore load power as much as possible and obtain the dynamic operation sequence of the switches. However, this study only focuses on the switch switching of important loads, but pays less attention to the switch switching of non-important loads. At the same time, it does not effectively deal with repeated or invalid operations of switches. Shang et al. (2023) took the minimum number of switch changes and the maximum weighted load power as the optimization objectives, while considering load priority. At the same time, the article used an action masking mechanism and an invalid work penalty strategy to deal with repeated or invalid operations of switches. And the present invention was carried out and verified in different power area scenarios. However, this study did not take into account some limitations of the deep Q network (DQN) algorithm. At the same time, in multi-regional power systems, as the corresponding action space and state space increase significantly, the number of invalid actions is also increasing, which will cause the algorithm to have a sharp decline in training efficiency in the corresponding scenarios. Summary of the invention

[0005] The purpose of the present invention is to provide a method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning, which can improve the speed and accuracy of dynamic fault reconstruction.

[0006] The present invention provides a method for dynamically reconstructing a fault network of a ship power system based on deep reinforcement learning, comprising the following steps: S1: topologically transforming a circuit diagram of a ship power system to obtain a directed graph of the ship power system; S2: constructing an objective function according to the circuit diagram of the ship power system; S3: constructing a Markov decision model according to the circuit diagram of the ship power system and the objective function; S4: training a duel dual deep Q network algorithm according to the Markov decision model using priority experience replay to obtain a trained duel dual deep Q network algorithm; S5: dynamically reconstructing a fault network of a ship power system to be analyzed using the trained duel dual deep Q network algorithm to obtain a dynamic switch sequence.

[0007] Furthermore, the above step S1 specifically includes: taking the circuit, distribution board and power supply in the ship power system circuit diagram as vertices, taking the load switch in the ship power system circuit diagram as edges, and obtaining a directed graph of the ship power system according to the vertices and the edges.

[0008] Furthermore, the above step S2 specifically includes: constructing an objective function according to the ship power system circuit diagram, such as formula: , , , , in, is the weighted load power, , and denote the weights of important loads, semi-important loads, and non-important loads, respectively; , and They are the load states of important load, semi-important load and non-important load, respectively. , and Represent the power of important load, semi-important load and non-important load respectively; For the switch cost, Indicates the switch status of the power supply path and branch before the fault. Indicates the switch status of the power supply path and branch at the current moment. represents the Euclidean distance; Represents that the objective function maximize, is the weight factor.

[0009] Furthermore, the above-mentioned Markov decision model includes a state space, an action space and a reward function; the state space is used to record the state of the load power supply switch and the tie line switch, and at the same time record the action selected in the current time and the previous time step; the action space is used to record the action at a given time step, and the action is the operation performed by the intelligent agent; the reward function includes an immediate reward, and the immediate reward is the sum of the weighted load power increment and the switch cost increment.

[0010] Furthermore, the instant reward is as follows: , in, For instant rewards, and Represent the changes of F1 and F2 respectively, Indicates the penalty for illegal operation.

[0011] Furthermore, the above step S4 includes initialization, agent-environment interaction, training, sample priority adjustment and sample elimination; the duel dual deep Q network algorithm includes an evaluation network and a target network; the initialization is used to initialize the parameters, experience replay buffer and algorithm parameters of the evaluation network and the target network; the agent interacts with the environment, and uses the agent to select actions using strategies in each cycle, and after executing the selected action, the corresponding samples are stored in the experience replay buffer; the training performs priority experience replay sampling from the experience replay buffer according to the sample priority to obtain samples; the sample priority adjustment calculates the importance sampling weight according to the sampling probability; the importance sampling weight is used to adjust the contribution of samples of different priorities when calculating the loss function to reduce the deviation caused by the priority experience replay; after selecting the corresponding samples, the network parameters and priority parameters are updated according to the loss function; the parameters of the target network are updated; the sample elimination is used to eliminate samples using sample priority and first-in-first-out strategy when the number of samples is greater than the maximum capacity of the sample pool.

[0012] Furthermore, the above sample priority is a time difference error.

[0013] Furthermore, the above-mentioned sample elimination specifically includes: when the number of samples is greater than the maximum capacity of the sample pool, each time a new sample is generated, an old sample is eliminated; according to a preset larger probability, samples are eliminated by way of sample priority; according to a preset smaller probability, samples are eliminated using a first-in-first-out elimination strategy.

[0014] Furthermore, the preset larger probability is 0.9, and the preset smaller probability is 0.1.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning.

[0016] Implementing the method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning provided by the present invention has the following beneficial effects: The present invention simplifies the circuit diagram into a directed graph, and converts the closing and opening operations of the switch into the action selection of the deep reinforcement learning algorithm, and converts the immediate reward into the sum of the weighted load power increment and the switch cost increment; the present invention uses the time difference (TD) error as the priority of the sample, which can distinguish the priority of different samples for training, effectively select samples that are helpful for intelligent agent learning, and use the D3QN algorithm to alleviate the problems of over-estimation of Q value and low efficiency in complex environments; the present invention introduces an elimination mechanism in which sample priority elimination and FIFO elimination coexist, which can retain high-priority samples that are helpful for intelligent agent learning to a certain extent in an environment with a large number of invalid or negative samples, thereby effectively improving the learning ability and convergence efficiency of the algorithm in a high-dimensional action space; in the environment of dynamic fault reconstruction of eight-zone MVDC, the present invention improves the speed and accuracy of reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 It is a flow chart of a method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning provided by the present invention; Figure 2 is a directed graph of an eight-area low-voltage direct current power system provided by the present invention; Figure 3 This is a graph showing the average reward results of the ablation experiment of the duel dual deep Q network algorithm provided by the present invention; Figure 4 It is a graph of average penalty results of ablation experiment on the duel dual deep Q network algorithm provided by the present invention; Figure 5 This is a graph showing the average reward comparison results of the duel dual-depth Q network algorithm provided by the present invention; Figure 6 It is a graph of average penalty comparison results of the duel dual-depth Q network algorithm provided by the present invention; Figure 7 This is a comparison result diagram of average rewards of the dual deep Q network algorithm under different learning rates provided by the present invention. DETAILED DESCRIPTION

[0018] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0019] Figure 1 A schematic diagram of a method for dynamic reconstruction of a faulty network of a ship power system based on deep reinforcement learning in this embodiment is shown. In this embodiment, the method for dynamic reconstruction of a faulty network of a ship power system based on deep reinforcement learning includes the following steps: S1: Topologically transform the circuit diagram of the ship power system to obtain a directed graph of the ship power system; In an exemplary embodiment, step S1 specifically includes: taking the circuit, the switchboard and the power supply in the circuit diagram of the ship power system as vertices, taking the load switch in the circuit diagram of the ship power system as an edge, and obtaining a directed graph of the ship power system according to the vertices and the edges; In an exemplary embodiment, the circuit diagram of the eight-zone medium voltage direct current ship power system is topologically converted into a corresponding directed graph; this embodiment selects an eight-zone medium voltage direct current ship power system (MVDC SPS) as an experimental scenario, and the ship power system (SPS) will be divided into different electrical areas, and power will be transmitted to the loads in the load area through a set of busbars arranged along the port and starboard sides; this embodiment retains the basic network topology of the ship power station system, and divides the load importance into important loads, semi-important loads and non-important loads according to the level; the load area is powered by two busbars, and the important loads and semi-important loads have two power supply paths, normal and standby, while the non-important loads have only one power supply path; Please refer to FIG. 2 for the directed graph of the actual scenario of this embodiment; the present invention uses a directed graph to represent the topological structure of the ship power station system; the directed graph includes a vertex V and an edge E, wherein the vertex V represents a circuit, a distribution board or a power supply, and the edge E represents a load switch; S2: constructing an objective function according to the ship power system circuit diagram; In an exemplary embodiment, step S2 specifically includes: constructing an objective function according to the ship power system circuit diagram, such as formula: , , , , in, is the weighted load power, , and denote the weights of important loads, semi-important loads, and non-important loads, respectively; , and They are the load states of important load, semi-important load and non-important load, respectively. , and Represent the power of important load, semi-important load and non-important load respectively; For the switch cost, Indicates the switch status of the power supply path and branch before the fault. Indicates the switch status of the power supply path and branch at the current moment. represents the Euclidean distance; Represents that the objective function maximize, is the weight factor; Specifically, in step S2, the circuit dynamic fault reconstruction problem is described as an optimal control problem with the goal of maximizing the weighted load and minimizing the switching action; for the single objective function optimization problem of dynamic reconstruction of the fault network of the ship power system, this embodiment, on the basis of fully considering the load priority, expresses the dynamic fault reconstruction problem as an optimal control problem with the goal of maximizing the weighted load and minimizing the switching action; by linearly weighting the objective function of maximizing the weighted load and the objective function of minimizing the switching action, and selecting an appropriate weight ratio, the final single objective optimization model is constructed; It should be noted that the loads of the ship power system can be divided into three categories, namely, important loads, semi-important loads and non-important loads; in different mission scenarios of the ship, the same load may have different priority changes; the first optimization target of this embodiment is the weighted load power F1, which is an important indicator for quantifying the load recovery level after a fault, as shown in Formula 1: (1) (2) (3) In the formula, , and denote the weights of important loads, semi-important loads, and non-important loads, respectively; , and are the load states of important load, semi-important load and non-important load respectively, where 0 means the load is powered off and 1 means the load is powered on, as shown in Equation 2 and Equation 3; It should be noted that the second optimization target of this embodiment is the switch cost F2, which is an important indicator for measuring the effectiveness and recovery speed of the fault reconstruction solution, as shown in Formula 4: (4) In the formula, Indicates the switch status of the power supply path and branch before the fault. Indicates the switch status of the power supply path and branch at the current moment, and the minimum Euclidean distance between the two is solved as the scalar value of the switch cost; It should be noted that, in the process of fault reconstruction, how to restore high-priority loads to the greatest extent is crucial; in addition, the number of operations of the branch power supply switch must also be considered; the above reconstruction problem involves two conflicting objective functions F1 and F2; based on these criteria, the goal of fault reconstruction is to maximize the weighted power using as few switching actions as possible, so the two objective functions are linearly weighted to form a single objective function optimization problem, as shown in Equation 5: (5) In the formula, is a weight factor, ranging from 0 to 1, used to weigh F1 and F2. Since the priority of restoring important loads is higher in this embodiment, Set to 0.8; It should be noted that in the target normalization process, this embodiment adopts the maximum-minimum normalization method (Min-Max Normalization), and the principle is shown in the following formula: (6) is the normalized data, is the original data, is the minimum value in the data set, is the maximum value in the data set; S3: constructing a Markov decision model according to the ship power system circuit diagram and the objective function; In an exemplary embodiment, the Markov decision model includes a state space, an action space and a reward function; the state space is used to record the state of the load power supply switch and the tie line switch, and the action selected in the current time and the previous time step; the action space is used to record the action at a given time step, and the action is the operation performed by the agent; the reward function includes an instant reward, and the instant reward is the sum of the weighted load power increment and the switch cost increment; In an exemplary embodiment, the instant reward is as follows: , in, For instant rewards, and Represent the changes of F1 and F2 respectively, Indicates the penalty for illegal operation; Specifically, in step S3, a Markov decision model is constructed, which mainly includes the following parts: State space: All factors that affect power failure reconstruction should be included in the state information used by the intelligent agent for real-time decision-making; the state space is used to record the state of the load power supply switch and the tie line switch, and also record the current time and the action chosen in the previous time step ; Action space: at a given time Step length, the operation performed by the agent is defined as an action; in this embodiment, each action is limited to changing only one load power switch / tie line switch; therefore, according to the actual scenario of this embodiment, an action space of size 76 can be designed; Reward function: The immediate reward is the sum of the weighted load power increment and the switch cost increment, as shown in Equation 7: (7) In the formula, and Represent the changes of F1 and F2 respectively, Indicates the penalty for illegal operation; It should be noted that It is the key guarantee to ensure that the agent complies with the system operation constraints in the process of action exploration; it is extremely inefficient to train the agent only by relying on the simple data interaction between the agent and the environment; in the fault reconstruction scenario of this embodiment, the set penalty mainly covers two aspects: one is the penalty for violating the operation constraints; the other is the penalty for the action that causes the reward increment to be negative; if it is not necessary for the agent to cut off the load of normal power supply, that is, the next agent action cannot increase the reward increment of the agent, then it is also necessary to impose a penalty on the reward; the specific penalty value is usually set to a constant value (it can also be set to a variable value, such as related to the size of the limit); generally speaking, the maximum single-step reward increment of the agent is calculated first, and then the penalty value is set to one third of the maximum single-step reward increment; S4: According to the Markov decision model, using priority experience replay, training the duel dual deep Q network algorithm to obtain a trained duel dual deep Q network algorithm; In an exemplary embodiment, step S4 includes initialization, agent-environment interaction, training, sample priority adjustment, and sample elimination; The duel dual deep Q network algorithm includes an evaluation network and a target network; The initialization is used to initialize the parameters of the evaluation network and the target network, the experience playback buffer and the algorithm parameters; The agent interacts with the environment, using the agent to select actions using the strategy in each cycle, and after performing the selected action, storing the corresponding sample in the experience replay buffer; The training includes performing priority experience replay sampling from the experience replay buffer according to the sample priority to obtain samples; The sample priority adjustment is to calculate the importance sampling weight according to the sampling probability; the importance sampling weight is used to adjust the contribution of samples of different priorities when calculating the loss function to reduce the deviation caused by the priority experience playback; after selecting the corresponding samples, the parameters and priority parameters of the network are updated according to the loss function; the parameters of the target network are updated; The sample elimination is used to eliminate samples using sample priority and first-in-first-out strategy when the number of samples is greater than the maximum capacity of the sample pool; In an exemplary embodiment, the sample priority is a time difference error; In an exemplary embodiment, the sample elimination specifically includes: when the number of samples is greater than the maximum capacity of the sample pool, each time a new sample is generated, an old sample is eliminated; according to a preset greater probability, samples are eliminated by sample priority; according to a preset smaller probability, samples are eliminated by a first-in-first-out elimination strategy; In an exemplary embodiment, the preset greater probability is 0.9, and the preset lesser probability is 0.1; Specifically, in step S4, the Dueling Double Deep Q Net (D3QN) algorithm is combined with the Prioritized Experience Replay (PER) algorithm. Replay); in the initialization stage, the relevant parameters of the two networks, the experience replay buffer D, and the relevant parameters of the algorithm such as the discount factor are initialized; in the stage of interaction between the agent and the environment, a specific strategy is adopted in each cycle to select actions; after executing the selected action, the corresponding sample is stored in the experience replay buffer D; in the training stage, priority experience replay sampling is performed from the experience replay buffer D according to the priority, so as to obtain a batch of samples; the priority experience replay selects more valuable samples for training according to the priority of the samples; for each acquired sample, the present embodiment calculates the time difference error (TD-Error) and uses it as the priority of the sample for relevant processing; in the sample priority adjustment stage, the importance sampling weight is calculated according to the sampling probability; the importance sampling weight is used to adjust the contribution of samples of different priorities in calculating the loss function to reduce the deviation caused by the priority experience replay; after selecting the corresponding sample, the parameters and priority parameters of the evaluation network are updated according to the loss function; then, the parameters of the target network are updated at a certain time interval so that the target network gradually keeps up with the update pace of the evaluation network; Specifically, when the sample capacity of the algorithm is greater than the maximum capacity of the sample pool, samples are eliminated by combining sample priority elimination with the First-In First-Out (FIFO) strategy; Specifically, when the sample pool is full, the algorithm will enter the sample elimination phase, that is, every time a new sample is generated, an old sample will be eliminated. There are two ways to eliminate the sample: (This example selects 0.9) Samples are selected and eliminated by sample priority, that is, samples with higher sample priority are less likely to be eliminated, and samples with lower sample priority are more likely to be eliminated; if the first option is not selected, then the sample with a smaller probability Select the first-in-first-out elimination strategy to ensure that the training process is not prone to entering the local optimal solution; S5: using the trained duel dual deep Q network algorithm to dynamically reconstruct the fault network of the ship power system to be analyzed, and obtaining a dynamic switch sequence; In an exemplary embodiment, the trained duel dual deep Q network algorithm is placed in an eight-zone medium voltage DC ship power system scenario for algorithm training to obtain a dynamic switching sequence. In this embodiment, corresponding tests are performed in a typical eight-area medium voltage direct current (MVDC) power system, such as Figure 2 The figure shows a directed graph of an eight-area high-voltage DC power system to verify the effectiveness of the proposed method. The dual-depth Q network algorithm in this embodiment is implemented on a laptop equipped with a 64-bit Windows 10 operating system. The computer is based on an AMD Ryzen 75800H processor with Radeon Graphics, a main frequency of 3.20GHz, and a memory capacity of 16GB. The algorithm language used is Python, and the Pytorch deep learning framework is used to model and iteratively train the neural network. In terms of algorithm parameters in this embodiment, the number of training episodes is 10000, the learning rate lr is 0.001, the ratio of the used data set to the test set is 8:2, and the main fault forms are divided into three types: single main generator failure, main and auxiliary generator failure, and dual main generator failure; In this embodiment, after the algorithm is trained for 10,000 rounds, the algorithm is tested with the test set scenario, and the corresponding switch dynamic sequence, the load recovered and the time taken are recorded, and then compared with the expected results to verify the actual efficiency of the recovery; like Figure 3 The figure shows the average reward results of the ablation experiment of the D3QN, D3QN_PER, and D3QN_PER_PU algorithms; Figure 4 The figure shows the average penalty result of the ablation experiment of D3QN, D3QN_PER, and D3QN_PER_PU algorithms; Figure 5 The figure shows the average reward comparison results of D3QN_PER_PU, DQN, DDQN, and SAC algorithms; Figure 6 The figure shows the average penalty comparison results of D3QN_PER_PU, DQN, DDQN, and SAC algorithms; Figure 7 The figure shows the average reward comparison results of the D3QN_PER_PU algorithm under different learning rates; it can be seen that the dynamic reconstruction method of the fault network of the ship power system based on deep reinforcement learning of the present invention performs better.

[0020] This embodiment provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning.

[0021] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A method for dynamic reconstruction of fault network of ship power system based on deep reinforcement learning, characterized in that: The following steps are involved: S1: Topologically transform the circuit diagram of the ship power system to obtain a directed graph of the ship power system; S2: constructing an objective function according to the circuit diagram of the ship power system; S3: constructing a Markov decision model according to the ship power system circuit diagram and the objective function; S4: According to the Markov decision model, using priority experience replay, training the duel dual deep Q network algorithm to obtain a trained duel dual deep Q network algorithm; S5: Using the trained duel dual deep Q network algorithm, dynamically reconstruct the fault network of the ship power system to be analyzed to obtain a dynamic switch sequence.

2. According to the method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning in claim 1, it is characterized in that: Step S1 specifically includes: taking the circuit, switchboard and power supply in the ship power system circuit diagram as vertices, taking the load switch in the ship power system circuit diagram as edges, and obtaining a directed graph of the ship power system according to the vertices and the edges.

3. The method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning according to claim 1 is characterized in that: Step S2 specifically includes: constructing an objective function according to the ship power system circuit diagram, such as formula: ， ， ， ， in, is the weighted load power, , and denote the weights of important loads, semi-important loads, and non-important loads, respectively; , and They are the load states of important load, semi-important load and non-important load, respectively. , and Represent the power of important load, semi-important load and non-important load respectively; For the switch cost, Indicates the switch status of the power supply path and branch before the fault. Indicates the switch status of the power supply path and branch at the current moment. represents the Euclidean distance; Represents that the objective function maximize, is the weight factor.

4. The method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning according to claim 1 is characterized in that: The Markov decision model includes a state space, an action space and a reward function; the state space is used to record the state of the load power supply switch and the tie line switch, and at the same time record the actions selected in the current time and the previous time step; the action space is used to record the actions at a given time step, and the actions are the operations performed by the intelligent agent; the reward function includes an immediate reward, and the immediate reward is the sum of the weighted load power increment and the switch cost increment.

5. The method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning according to claim 4 is characterized in that: The instant reward is as follows: ， in, For instant rewards, and Represent the changes of F1 and F2 respectively, Indicates the penalty for illegal operation.

6. The method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning according to claim 1 is characterized in that: Step S4 includes initialization, agent-environment interaction, training, sample priority adjustment, and sample elimination; The duel dual deep Q network algorithm includes an evaluation network and a target network; The initialization is used to initialize the parameters of the evaluation network and the target network, the experience playback buffer and the algorithm parameters; The agent interacts with the environment, using the agent to select actions using the strategy in each cycle, and after performing the selected action, storing the corresponding sample in the experience replay buffer; The training includes performing priority experience replay sampling from the experience replay buffer according to the sample priority to obtain samples; The sample priority adjustment is to calculate the importance sampling weight according to the sampling probability; the importance sampling weight is used to adjust the contribution of samples of different priorities when calculating the loss function to reduce the deviation caused by the priority experience playback; after selecting the corresponding samples, the parameters and priority parameters of the network are updated according to the loss function; the parameters of the target network are updated; The sample elimination is used to eliminate samples using sample priority and a first-in-first-out strategy when the number of samples is greater than the maximum capacity of the sample pool.

7. The method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning according to claim 6 is characterized in that: The sample priority is the time difference error.

8. The method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning according to claim 6 is characterized in that: The sample elimination specifically includes: when the number of samples is greater than the maximum capacity of the sample pool, each time a new sample is generated, an old sample is eliminated; according to a preset larger probability, samples are eliminated by way of sample priority; according to a preset smaller probability, samples are eliminated using a first-in-first-out elimination strategy.

9. The method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning according to claim 8 is characterized in that: The preset larger probability is 0.9, and the preset smaller probability is 0.

1.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for dynamic reconstruction of a fault network of a ship power system based on deep reinforcement learning as described in any one of claims 1 to 9 are implemented.

Images