An integrated ac-dc shore power supply control method and system
By using an integrated AC/DC shore power control method combined with a Q-network based on deep reinforcement learning, flexible and efficient power supply to AC and DC ships has been achieved. This solves the control stability and self-learning problems of traditional shore power systems under complex operating conditions, and improves the power supply stability and equipment lifespan of port shore power systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU UNIV OF SCI & TECH
- Filing Date
- 2026-03-30
- Publication Date
- 2026-06-16
AI Technical Summary
Traditional single AC or DC shore power systems are insufficient to meet the diverse needs of ships. Existing control technologies have poor control performance under complex operating conditions and lack self-learning capabilities and stability.
An integrated AC/DC shore power supply control method is adopted. Through voltage and current dual closed-loop control of DC/DC rectifier and DC/AC inverter, combined with a deep reinforcement learning Q network, a DQN agent is constructed to optimize charging efficiency and power supply stability. A reward function and state space are designed to achieve adaptive control.
It enables flexible and efficient power supply to AC and DC ships, reduces voltage fluctuations and current surges, improves power supply stability and reliability, extends equipment life, adapts to complex operating conditions, and enhances the compatibility and operational efficiency of port shore power systems.
Smart Images
Figure CN122225591A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of shore power technology, specifically to a method and system for controlling AC / DC integrated shore power supply based on deep reinforcement learning. Background Technology
[0002] With the diversification of modern ship power systems, in addition to the large number of ships using AC power distribution systems, the number of ships using DC power distribution systems is increasing. At the same time, the development of the electric ship industry has also placed new demands on shore power technology in ports. Against this backdrop, traditional single AC or DC shore power systems are no longer sufficient to meet the diverse needs of various berthing vessels. Integrated AC / DC shore power supplies represent the development direction of the next generation of shore power technology. They can flexibly and efficiently output stable AC and DC power, achieving "one machine for multiple uses, one cable connection," breaking the constraints of power supply systems on shore power applications. This is of great significance for comprehensively improving the compatibility, operational efficiency, and energy conservation and emission reduction benefits of ports.
[0003] Secondly, in the field of AC / DC integrated shore power supply control, current common control technologies have shortcomings. Traditional PID control methods are difficult to tune parameters and struggle to adapt to the complex and ever-changing operating conditions of shore power systems. When the system's operating state changes, the control performance deteriorates, failing to effectively cope with unexpected situations. While fuzzy control methods can handle uncertainty and nonlinear problems, they lack self-learning capabilities, and the formulation of control rules relies on expert experience, making it difficult to achieve optimal control. Model predictive control methods require establishing accurate system models, but shore power system models are complex and contain uncertainties; model errors can affect control performance. Neural network-based control methods require a large amount of sample data during training and are prone to getting trapped in local optima, making it difficult to guarantee control stability and reliability in practical applications. Summary of the Invention
[0004] The purpose of this invention is to provide an AC / DC integrated shore power supply control method and system to solve the technical problem that traditional single AC or DC shore power supplies cannot meet the diverse needs of various types of berthing vessels.
[0005] To achieve the above objectives, in a first aspect, the present invention provides a control method for an integrated AC / DC shore power supply, characterized in that the integrated AC / DC shore power supply includes a DC / DC rectifier and a DC / AC inverter, both of which employ voltage and current dual closed-loop control; the method includes: Define the state space based on the real-time collected AC / DC integrated shore power supply operation data; The action space is defined by PI control parameters based on voltage and current dual closed-loop control; A reward function is designed with the goal of optimizing charging efficiency, improving power supply stability, and minimizing power loss. A Q-network with a three-layer network structure is built based on state space, action space, and reward function; Based on the Q-network and preset algorithm parameters, a trainable DQN agent is constructed. The system continuously collects steady-state operation data of the AC / DC integrated shore power supply under typical operating conditions, and uses the steady-state operation data to conduct offline training of the DQN agent; the DQN agent, which has been trained offline, controls the AC / DC integrated shore power supply.
[0006] According to the AC / DC integrated shore power supply control method provided by the present invention, the state variables in the state space include: DC bus voltage amplitude, DC voltage deviation, load power, AC total harmonic distortion, DC ripple, grid frequency, and sampling interval time.
[0007] According to the AC / DC integrated shore power supply control method provided by the present invention, the state space is as follows:
[0008] in, These represent the normalized DC bus voltage amplitude, DC voltage deviation, load power, AC total harmonic distortion, DC ripple, grid frequency, and sampling interval, respectively. According to the AC / DC integrated shore power supply control method provided by the present invention, the action space is: , in, This is the voltage outer loop proportionality coefficient. The voltage outer loop integral coefficient, This is the proportionality coefficient for the inner current loop. The integral coefficient of the inner current loop. This represents the bandwidth of the inner current loop.
[0009] According to the AC / DC integrated shore power supply control method provided by the present invention, the reward function is:
[0010] in, Let be the instantaneous reward value for the t-th action; It is the normalized power efficiency change. For output power, For input power, Rated power; It is DC voltage deviation. This represents the DC bus voltage amplitude. This is the voltage reference value; This is the actual energy loss value of the AC / DC integrated shore power supply. The maximum permissible energy loss threshold for AC / DC integrated shore power supply; , , All are positive hyperparameters.
[0011] According to the AC / DC integrated shore power supply control method provided by the present invention, the Q network includes an input layer, a hidden layer, and an output layer; the input layer corresponds to a 7-dimensional state vector in the state space; there are two hidden layers, referred to as the first hidden layer and the second hidden layer, respectively, with 56 to 70 neurons in the first hidden layer and half the number of neurons in the second hidden layer; the output layer corresponds to a 5-dimensional action in the action space and outputs the Q value of each action. According to the AC / DC integrated shore power supply control method provided by the present invention, the Q network function is:
[0012] in, Represents the weight parameters of a neural network; Indicates the expected value; It is a discount factor used to evaluate actions. In state The long-term benefits. According to the present invention, an AC / DC integrated shore power supply control method is provided, wherein the algorithm parameters include learning rate, discount factor, empirical replay buffer capacity, and target network update frequency.
[0013] According to the present invention, an AC / DC integrated shore power supply control method continuously collects steady-state operation data of the AC / DC integrated shore power supply under typical operating conditions, and uses the steady-state operation data to perform offline training on a DQN agent, including: Real-time monitoring time when AC / DC integrated shore power supply supplies power externally Output measurement value The range of fluctuations; when For all At the time of its establishment, the AC / DC integrated shore power supply was considered to be in a steady state, among which, To preset the steady-state value, It is the allowable error threshold. To achieve a stable time; The current state, actions, rewards, and next-moment state data of the AC / DC integrated shore power supply during steady-state operation are stored as tuples. This serves as an experience pool; during training, a batch of datasets is sampled from the experience pool and input into the DQN agent, updating the Q-network function as follows:
[0014] in, It is the learning rate. Secondly, this invention provides an integrated AC / DC shore power supply control system, comprising a DC / DC rectifier and a DC / AC inverter, both of which employ dual closed-loop voltage and current control; the system includes: The state observer is used to define the state space based on the real-time collected AC / DC integrated shore power supply operation data; The DQN intelligent control module is used to define the action space based on PI control parameters of voltage and current dual closed-loop control; to design a reward function with the goal of optimizing charging efficiency, improving power supply stability and minimizing power loss; to build a three-layer Q network based on the state space, action space and reward function; and to construct a trainable DQN agent based on the Q network and preset algorithm parameters. The training and update module is used to continuously collect steady-state operation data of the AC / DC integrated shore power supply under typical operating conditions, and to use the steady-state operation data to perform offline training on the DQN agent; the DQN agent completed through offline training controls the AC / DC integrated shore power supply.
[0015] Compared with the prior art, the present invention has at least the following technical effects: This invention provides a deep reinforcement learning-based AC / DC integrated shore power supply control method and system. It can operate in both AC and DC output modes according to the ship's load requirements. Through coordinated control of the DC / AC inverter and DC / DC rectifier, it achieves flexible and efficient power supply and charging for different types of ships, including AC-powered ships, DC-powered pure electric ships, and DC-powered hybrid ships. Furthermore, it significantly reduces voltage fluctuations and current surges during AC / DC power supply in shore power systems, improving power supply stability and reliability, reducing damage to electrical equipment, and extending equipment lifespan. In real-world scenarios, it can be applied to port shore power systems to ensure ship power safety, reduce impact on the power grid, and improve energy efficiency. Simultaneously, by combining deep reinforcement learning with power electronics systems, it overcomes the limitations of traditional "model-dependent control," providing a replicable solution for intelligent control of multi-port energy systems such as ship power distribution and microgrids. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0017] In the attached diagram: Figure 1 This is a schematic diagram of the AC / DC integrated shore power supply control system based on deep reinforcement learning, according to an embodiment of the present invention. Figure 2 This is a topology diagram of the DC / AC inverter according to an embodiment of the present invention; Figure 3 This is a topology diagram of the DC / DC rectifier according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the voltage and current dual closed-loop control model of the DC / AC inverter according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the voltage and current dual closed-loop control model of the DC / DC rectifier according to an embodiment of the present invention; Figure 6 This is a flowchart illustrating how a deep learning network is integrated into an AC / DC integrated shore power supply according to an embodiment of the present invention. Detailed Implementation
[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0019] The following detailed description of some embodiments of the present invention will be provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0020] This invention combines the decision-making advantages of reinforcement learning with the multi-mode power supply and dynamic disturbance rejection requirements of shore power systems. It optimizes voltage and current dual-loop control parameters or energy allocation strategies using the DQN algorithm, improving the system's adaptability and stability under complex operating conditions. When using the AC / DC integrated shore power supply control method and system based on deep reinforcement learning disclosed in this invention, voltage fluctuations and current surges during AC / DC power supply can be significantly reduced, improving power supply stability and reliability, reducing damage to electrical equipment, and extending equipment lifespan. In real-world scenarios, it can be applied to port shore power systems to ensure ship power safety, reduce impact on the power grid, and improve energy efficiency. Simultaneously, by combining deep reinforcement learning with power electronics systems, it overcomes the limitations of traditional "model-dependent control," providing a replicable solution for intelligent control of multi-port energy systems such as ship power distribution and microgrids. This invention was developed to meet this real-world need.
[0021] Please see Figure 1The AC / DC integrated shore power supply provided in this embodiment of the invention includes a mains power grid input, an AC / DC bidirectional rectifier, a DC bus, a DC / AC inverter, a DC / DC rectifier, and a voltage and current dual closed-loop control module. The mains power is used as the primary power input, forming a unified DC bus through the AC / DC bidirectional rectifier. On one hand, the DC bus can convert DC power into high-quality AC power through the DC / AC inverter, providing stable AC power for AC ship loads. On the other hand, the DC bus can also be adjusted to different voltage levels through the DC / DC rectifier, providing stable DC power for DC-electric ship loads. The voltage and current dual closed-loop control module precisely regulates the voltage and current parameters of the DC / AC inverter and DC / DC rectifier in real time, ensuring coordinated operation of all modules of the AC / DC integrated shore power supply and achieving stability, reliability, and adaptability in the power conversion and supply process.
[0022] like Figure 2 As shown, the DC and AC sides of the DC / AC inverter are connected and disconnected via contactors, and are protected by surge protectors. The DC side is protected and current-limited via fuses, circuit breakers, and a pre-charge circuit, and interference is suppressed by DC EMI filtering. The AC side, after LC filtering, AC EMI filtering, and surge protection, outputs a three-phase four-wire power supply via fuses and circuit breakers, suitable for AC marine loads. During system operation, the DC power is pulse-width modulated and converted into harmonic-rich AC power by the DC / AC inverter. High-frequency harmonics are filtered out by the filter reactor to obtain the power frequency sinusoidal component, which is then output as AC power to supply the AC marine load.
[0023] like Figure 3 As shown, the DC / DC rectifier adopts a typical H-bridge structure and outputs a two-level PWM waveform to realize the conversion and transmission of DC power. The four IGBT switches on the left invert the DC input into a high-frequency square wave voltage, which is then transferred through L1 and L2. By adjusting the conduction state of the four IGBT switches on the right, precise control of the DC voltage is achieved, thereby providing stable DC power to the DC pure electric ship load.
[0024] Specifically, the voltage and current dual closed-loop control includes: Voltage outer loop control. The DC bus voltage is collected, and the difference between it and the voltage reference value is input to the PI controller. The output result is used as the current reference value for the current inner loop. Current inner loop control. The converter output current is acquired, converted into dq-axis current via Clark / Park conversion, and the difference between this current and the reference current value is input to the PI controller, which outputs a voltage modulation signal. The current reference value output from the outer voltage loop is connected to the inner current loop, and the voltage modulation signal output from the inner current loop is connected to the PWM module to complete the construction of the dual closed-loop control link.
[0025] In one specific embodiment, the mains grid input, AC / DC bidirectional rectifier, DC bus, DC / AC inverter, DC / DC rectifier, and voltage-current dual closed-loop control are modeled in Simulink to construct a simulation model of an integrated AC / DC shore power supply that can simultaneously meet the diverse power supply needs of AC-powered and DC-powered pure electric ships. The core elements of the deep reinforcement learning (DQN) algorithm are defined in a MATLAB script, and its interaction with the Simulink model is achieved through an interface, thereby embedding the DQN algorithm into the control of the integrated AC / DC shore power supply to achieve optimized control.
[0026] like Figure 4 As shown, the specific steps of the voltage and current dual closed-loop control of the DC / AC inverter in a specific embodiment of the present invention include: The outer voltage loop takes the deviation between the bus reference voltage and the measured voltage as input, and outputs the dq-axis current reference value through a PI regulator to achieve stable control of the DC bus voltage. The inner current loop takes the deviation between the current reference value and the measured dq-axis current as input, and outputs the dq-axis modulated voltage through a PI regulator to quickly track the current command and suppress disturbances. After the modulated voltage undergoes an inverse dq / abc conversion, it generates the switching transistor drive pulses for the DC / AC inverter through a PWM module, completing the energy conversion from DC to three-phase AC and providing stable AC power for the AC ship.
[0027] like Figure 5 As shown, the specific steps of the voltage and current dual closed-loop control of the DC / DC rectifier in a specific embodiment of the present invention include: The outer voltage loop takes the deviation between the bus reference voltage and the measured voltage as input, and generates an inductor current reference value through a PI regulator to achieve stable control of the DC bus voltage. The inner current loop takes the deviation between the inductor current reference value and the measured value as input, and generates a PWM duty cycle modulation signal through a PI regulator to quickly track the current command and suppress load disturbances. The modulation signal generates a drive pulse through the PWM module to control the switching of the DC / DC rectifier transistors, completing the efficient energy conversion and precise voltage regulation from the DC bus to the ship's DC load, providing stable DC power for loads such as DC pure electric ships.
[0028] Please see Figure 6 This invention provides an integrated AC / DC shore power supply control method, comprising: Step 1: Define the state space based on the real-time collected AC / DC integrated shore power supply operation data; Specifically, based on the AC / DC control requirements of the shore power supply, the state space of the Q network is defined. The design of the state space is based on the real-time operating data collected by the integrated AC / DC shore power supply. The state variables include the DC bus voltage amplitude. (Unit: V) DC voltage deviation (Unit: V, representing voltage stability), load power (Unit: W, real-time system power demand), Total Harmonic Distortion (THD) (characterizing AC power quality), DC ripple (Characterizing DC power quality), system frequency (Unit: Hz, grid or inverter output frequency), and timestamp. (Unit: s, sampling interval time).
[0029] Furthermore, the raw data is preprocessed to remove noise and outliers. A low-pass Butterworth filter is applied to remove high-frequency interference; the filtering formula is as follows: ,in It is a smoothing coefficient (optimized based on empirical values). The original measurement value at time t, This is the filtered output. Next, normalization is performed, scaling each variable to the [0,1] interval to improve model convergence speed. The normalization formula is: ,in This is the average of historical data. Standard deviation, It represents any primitive variable (such as voltage or current). These are normalized values. The historical statistics window uses 30 minutes of data and is dynamically updated. and To adapt to load fluctuations.
[0030] A multidimensional state vector is constructed as input for deep reinforcement learning. The preprocessed variables are combined: normalized bus voltage. :
[0031] Normalized voltage deviation Normalized load power Normalized frequency Normalized communication Normalized DC ripple Normalized frequency and normalized timestamps These variables constitute a multidimensional state vector, and the state space of the Q-network is defined as follows:
[0032] Where t is the discrete time step, and the state dimension ensures the capture of transient system behavior, such as voltage fluctuations and load changes. This vector has a dimension of 7, covering key system dynamics and ensuring that the DQN control strategy aligns with the actual engineering requirements of shore power supplies.
[0033] Step 2: Define the action space based on PI control parameters for voltage and current dual closed-loop control; Furthermore, the action space of the Q-network is defined, the PI control parameters of the voltage-current dual closed-loop control are discretized, and five key action variables are selected, including: Voltage outer loop proportionality coefficient Adjust the dynamic response speed of the bus voltage to solve the problem of delayed voltage recovery when the load changes suddenly; Voltage outer loop integral coefficient Eliminate steady-state deviation of bus voltage to ensure voltage accuracy during long-term system operation; Current inner loop proportionality coefficient Improve current tracking speed and disturbance rejection capability, and reduce AC THD and DC ripple; Inner loop integral coefficient of current Eliminating current steady-state deviation ensures the control accuracy of charging current and voltage; Current inner loop bandwidth It adapts to the suppression of disturbances at different frequencies, improving the robustness of the system under complex operating conditions.
[0034] Define the action space of the Q network as follows:
[0035] This vector has a dimension of 5, covering key system dynamics and ensuring that the DQN control strategy aligns with the actual engineering needs of the shore power system.
[0036] Step 3: Design a reward function with the goals of optimizing charging efficiency, improving power supply stability, and minimizing power loss; Specifically, the reward function is determined with the objectives of optimizing charging efficiency, improving system stability, and minimizing system energy loss. The reward function is designed as follows:
[0037] in: This is the instantaneous reward value for the t-th action (a dimensionless value between 0 and 1). It is the normalized power efficiency change ( For output power, For input power, (Rated power) It is DC voltage deviation ( (For reference values, such as 400V). It is the actual power loss value of the AC / DC integrated shore power system (unit: W, the real-time cumulative power loss value of the entire power supply link from the grid side to the ship load side). This represents the maximum permissible energy loss threshold for an integrated AC / DC shore power system. (Weight) , , Positive hyperparameters need to be adjusted through reinforcement learning training, for example... Emphasizing power supply efficiency, Ensure voltage stability. Reduce energy loss, reward value Normalize to [-1,1] to accelerate convergence.
[0038] Step 4: Construct a three-layer Q-network based on the state space, action space, and reward function; Specifically, building a three-layer Q network includes: Input layer: Corresponds to the 7-dimensional state space defined in the Q network, serving as the network input; Hidden Layers: To balance the fitting ability and overfitting risk of the network, two hidden layers are set. If only a single hidden layer is set, its expressive power is insufficient and it is difficult to capture complex nonlinear relationships in the shore power system, such as load changes and charging phase switching; while setting three or more hidden layers will easily lead to overfitting (i.e., the Q value prediction is accurate during training, but fails in actual operation), and the training time will increase exponentially, which is not necessary for the real-time control of the power system. Therefore, two hidden layers are chosen as a reasonable compromise. The number of neurons in the first hidden layer is selected to be 8 to 10 times the input dimension (7) to fully extract the basic nonlinear features (such as the combination features of bus voltage and load power); the number of neurons in the second hidden layer is set to half that of the first hidden layer, which aims to reduce the dimensionality and refine the extracted features to avoid redundancy. Output layer: Corresponds to the 5-dimensional action space defined in the Q network, outputting the Q value of each action.
[0039] Secondly, Q-networks cannot directly fit the nonlinear relationships of shore power systems. Therefore, it is necessary to introduce nonlinearity through the core activation function ReLU to fit the nonlinear relationships of shore power systems. ReLU is defined as:
[0040] in This is the input to the neuron.
[0041] Furthermore, define the Q-network function:
[0042] in These are the weight parameters of the neural network. Indicates the expected value. For instant rewards, It is a discount factor (value 0.99), balancing "immediate gains" and "long-term gains," preventing agents from sacrificing long-term system stability for short-term high rewards, and is used to evaluate actions. In state The long-term benefits.
[0043] Step 5: Construct a trainable DQN agent based on the Q-network and preset algorithm parameters; Specifically, the Simulink model is encapsulated into a reinforcement learning environment using the rlSimulationEnv function in MATLAB, enabling real-time interaction between state, action, and reward signals; the designed Q-network structure and algorithm parameters (including learning rate and discount factor) are then implemented. The system integrates the experience replay buffer capacity, target network update frequency, etc., as well as the predefined state space (7-dimensional) and action space (5-dimensional) through scripts to finally construct a trainable DQN agent.
[0044] By following the steps above, a DQN agent with preliminary decision-making capabilities can be obtained, which can output the optimal control action based on the real-time status of the system to achieve control objectives such as voltage stability and power balance.
[0045] Step 6: Continuously collect steady-state operation data of the AC / DC integrated shore power supply under typical operating conditions, and use the steady-state operation data to conduct offline training of the DQN agent; use the DQN agent completed by offline training to control the AC / DC integrated shore power supply.
[0046] The specific process of offline training includes: Real-time monitoring time when the AC / DC integrated shore power system charges a DC pure electric ship or supplies power to an AC ship. Output measurement value The fluctuation range; when For all At the time of its establishment, the AC / DC integrated shore power supply was considered to be in a steady state, among which, To preset the steady-state value, It is the allowable error threshold. To achieve a stable time.
[0047] The current state, actions, rewards, and next-moment state data of the AC / DC integrated shore power supply during steady-state operation are stored as tuples. This serves as an experience pool; key parameters are encapsulated as experience tuples and stored in the experience pool data structure. The experience pool employs a circular buffer design, with each entry storing a tuple of current state-action-reward-next time step state. Tuples are written via the database API, ensuring indexing and query efficiency.
[0048] These experience pool data are used for offline training of deep reinforcement learning models. During training, the dataset is sampled in batches from the experience pool, input into the DQN agent, and the Q-network function is updated as follows:
[0049] in, It is the learning rate (e.g., 0.001). The discount factor (e.g., 0.99) is used, and the max operation selects the optimal action. The entire process is implemented through automated scripts to ensure data integrity and training convergence.
[0050] Based on the same inventive concept, another embodiment of the present invention provides an integrated AC / DC shore power supply control system, comprising: The state observer is used to define the state space based on the real-time collected AC / DC integrated shore power supply operation data; The DQN intelligent control module is used to define the action space based on PI control parameters of voltage and current dual closed-loop control; to design a reward function with the goal of optimizing charging efficiency, improving power supply stability and minimizing power loss; to build a three-layer Q network based on the state space, action space and reward function; and to construct a trainable DQN agent based on the Q network and preset algorithm parameters. The training and update module is used to continuously collect steady-state operation data of the AC / DC integrated shore power supply under typical operating conditions, and to use the steady-state operation data to perform offline training on the DQN agent; the DQN agent completed through offline training controls the AC / DC integrated shore power supply.
[0051] In some embodiments, the AC / DC integrated shore power supply simulation model built in Simulink is encapsulated into a reinforcement learning environment using the rlSimulationEnv function in MATLAB, enabling real-time interaction between state, action, and reward signals. Based on this, the designed Q-network structure, algorithm parameters, and defined state and action spaces are integrated through scripts to ultimately construct a trainable DQN agent, laying the foundation for subsequent co-simulation and online optimization control with the Simulink model.
[0052] This invention constructs an integrated AC / DC shore power system architecture in Simulink that can simultaneously meet the diverse power supply needs of both AC and DC-electric ships, enabling simple, fast, and intuitive simulation of circuit conditions. Furthermore, by defining the core elements of Deep Learning Network (DQN) and linking MATLAB scripts with the Simulink model, it integrates deep learning networks into the design of the integrated AC / DC shore power control system, moving from "single control" to "multi-objective optimization" while accommodating various performance requirements.
[0053] In summary, the AC / DC integrated shore power supply control method and system based on deep reinforcement learning provided by this invention can operate in both AC and DC output modes according to the ship's load requirements. Through the coordinated control of the DC / AC inverter and DC / DC rectifier, it achieves flexible and efficient power supply and charging for different types of ships, including AC ships, DC pure electric ships, and DC hybrid ships. Furthermore, by integrating deep reinforcement learning with AC / DC integrated shore power supply, it solves the problems of existing shore power control technologies lacking adaptive, self-learning, and precise model-dependent methods for adapting to complex operating conditions.
[0054] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. It should be understood that the invention is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. A method for controlling an integrated AC / DC shore power supply, characterized in that, The AC / DC integrated shore power supply includes a DC / DC rectifier and a DC / AC inverter, both of which employ dual closed-loop control of voltage and current; the method includes: Define the state space based on the real-time collected AC / DC integrated shore power supply operation data; The action space is defined based on the PI control parameters of the aforementioned voltage and current dual closed-loop control; A reward function is designed with the goal of optimizing charging efficiency, improving power supply stability, and minimizing power loss. A three-layer Q-network is constructed based on the state space, action space, and reward function. Based on the Q-network and preset algorithm parameters, a trainable DQN agent is constructed; The system continuously collects steady-state operation data of the AC / DC integrated shore power supply under typical operating conditions, and uses the steady-state operation data to perform offline training on the DQN agent; the DQN agent, which has been trained offline, controls the AC / DC integrated shore power supply.
2. The AC / DC integrated shore power supply control method according to claim 1, characterized in that, The state variables in the state space include: DC bus voltage amplitude, DC voltage deviation, load power, AC total harmonic distortion, DC ripple, grid frequency, and sampling interval time.
3. The AC / DC integrated shore power supply control method according to claim 2, characterized in that, The state space is as follows: in, These represent the normalized DC bus voltage amplitude, DC voltage deviation, load power, AC total harmonic distortion, DC ripple, grid frequency, and sampling interval, respectively.
4. The AC / DC integrated shore power supply control method according to claim 3, characterized in that, The action space is: ， in, This is the voltage outer loop proportionality coefficient. The voltage outer loop integral coefficient, This is the proportionality coefficient for the inner current loop. The integral coefficient of the inner current loop. This represents the bandwidth of the inner current loop.
5. The AC / DC integrated shore power supply control method according to claim 4, characterized in that, The reward function is: in, Let be the instantaneous reward value for the t-th action; It is the normalized power efficiency change. For output power, For input power, Rated power; It is DC voltage deviation. This represents the DC bus voltage amplitude. This is the voltage reference value; This is the actual energy loss value of the AC / DC integrated shore power supply. The maximum permissible energy loss threshold for AC / DC integrated shore power supply; , , All are positive hyperparameters.
6. The AC / DC integrated shore power supply control method according to claim 5, characterized in that, The Q-network includes an input layer, a hidden layer, and an output layer; the input layer corresponds to the 7-dimensional state vector of the state space; there are two hidden layers, referred to as the first hidden layer and the second hidden layer, with the first hidden layer having 56 to 70 neurons and the second hidden layer having half the number of neurons in the first hidden layer; the output layer corresponds to the 5-dimensional actions of the action space and outputs the Q-value of each action.
7. The AC / DC integrated shore power supply control method according to claim 6, characterized in that, The Q network function is: in, Represents the weight parameters of a neural network; Indicates the expected value; It is a discount factor used to evaluate actions. In state The long-term benefits.
8. The AC / DC integrated shore power supply control method according to claim 7, characterized in that, The algorithm parameters include the learning rate, discount factor, experience replay buffer capacity, and target network update frequency.
9. The AC / DC integrated shore power supply control method according to claim 8, characterized in that, Continuously collect steady-state operation data of the AC / DC integrated shore power supply under typical operating conditions, and use the steady-state operation data to perform offline training on the DQN agent, including: Real-time monitoring time when AC / DC integrated shore power supply supplies power externally Output measurement value The fluctuation range; when For all At the time of its establishment, the AC / DC integrated shore power supply was considered to be in a steady state, among which, To preset the steady-state value, It is the allowable error threshold. To achieve a stable time; The current state, actions, rewards, and next-moment state data of the AC / DC integrated shore power supply during steady-state operation are stored as tuples. This serves as an experience pool; during training, a batch of datasets is sampled from the experience pool and input into the DQN agent, updating the Q-network function as follows: in, It is the learning rate.
10. An integrated AC / DC shore power supply control system, characterized in that, The integrated AC / DC shore power supply includes a DC / DC rectifier and a DC / AC inverter, both of which employ dual closed-loop control of voltage and current; the system includes: The state observer is used to define the state space based on the real-time collected AC / DC integrated shore power supply operation data; The DQN intelligent control module is used to define the action space based on the PI control parameters of the voltage and current dual closed-loop control; design a reward function with the goal of optimizing charging efficiency, improving power supply stability and minimizing power loss; build a three-layer Q network based on the state space, action space and reward function; and construct a trainable DQN agent according to the Q network and preset algorithm parameters. The training and update module is used to continuously collect steady-state operation data of the AC / DC integrated shore power supply under typical operating conditions, and to use the steady-state operation data to perform offline training on the DQN agent; the DQN agent completed through offline training controls the AC / DC integrated shore power supply.