Island micro-grid load frequency control method and system based on deep Q learning

Abstract

The application discloses a kind of island micro-grid load frequency control method and system based on deep Q learning, for the randomness of user charging behavior of electric vehicle charging station, establish the electric vehicle frequency control model considering SOC constraint, to build the micro-grid load frequency control model containing photovoltaic, wind power, micro gas turbine, electric vehicle and its random power increment constraint, design the frequency controller structure based on DQN, sequentially complete the definition of state space, action space and reward function, and obtain the optimal hyperparameter by adjusting, before the application of DQN controller, accept a random trial-and-error learning process, obtain the optimal value function Q network, thus guarantee enough good convergence characteristics, can complete the frequency control under actual condition.The application has the ability of online learning and experience playback, can more effectively deal with the problem of micro-grid LFC with strong randomness, and can also better adapt to the complex operating conditions of system network topology parameter change.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electric energy storage systems in the power industry, and more particularly relates to an island micro-grid load frequency control terminal and method. BACKGROUND

[0002] Micro-grid can solve the problem of flexible and efficient grid connection of various forms of distributed power supply, realize high reliable supply of various energy forms of load, and is an effective way to realize active distribution network.

[0003] The micro-grid can operate in grid-connected mode or island mode; in island mode, its frequency stability is the key to guarantee the safe operation of the micro-grid. The energy storage module is an important part of the micro-grid load frequency control model, and the electric vehicle has become a new type of distributed energy storage unit due to its energy saving, environmental protection and flexibility. Through the vehicle-to-grid technology, i.e. V2G system (Vehicle-to-grid), when the electric vehicle is not in use, the power of the vehicle-mounted battery is sold to the power grid system. The electric vehicle can provide power support for the frequency of the island micro-grid and improve its operation flexibility. However, the existing research has little consideration of the randomness of user travel demand, and has not refined the modeling of random output power increment from the perspective of individual and cluster of electric vehicles. In actual situation, the power increment constraint of the charging station is affected by the randomness of user charging behavior and the characteristics of electric vehicle cluster.

[0004] In addition, with the access of high proportion of distributed new energy, the control performance of the traditional controller needs to be further improved in the face of complex working conditions such as a large number of random disturbances, system parameters and structure changes in the island micro-grid.

[0005] Moreover, the existing micro-grid frequency intelligent control method ignores the access of electric vehicles, and does not take the output power increment of electric vehicles as the state space, i.e. the design idea, convergence characteristics and dynamic performance of the control method also have further improvement space.

[0006] According to the search of Chinese patent document library, no relevant patents or solutions are found to solve the frequency control problem of island micro-grid containing electric vehicles.

[0007] The problem existing in the prior art is that the randomness influence, the influence of the output power of electric vehicles, etc. are not considered in the frequency control of island micro-grid containing electric vehicles, thereby affecting the quality of the micro-grid. SUMMARY

[0008] In view of the deficiencies in the prior art, the purpose of the present application is to reduce the influence of the randomness of electric vehicles on the island micro-grid, and to solve the problem of micro-grid load control quality.

[0009] In order to achieve the above object, the present application provides the following technical solutions:

[0010] A load frequency control method for an island micro-grid based on deep Q learning, comprising the following steps:

[0011] S1: Based on the random charging behavior of users in an electric vehicle charging station, an electric vehicle frequency control model is established, which includes SOC constraints.

[0012] Q-learning is a computer science and technology term, which refers to a model-independent reinforcement learning algorithm that directly optimizes an iterative Q function. SOC is the State of Charge, battery state of charge, which refers to the ratio of the remaining dischargeable capacity of the battery after a period of use or long-term storage to the capacity of the fully charged state.

[0013] S2: A micro-grid load frequency control model is established, which includes photovoltaic power disturbance power parameter constraints, wind power disturbance power parameter constraints, load random power increment parameter constraints, gas turbine output power increment amplitude constraints and electric vehicle output power increment amplitude constraints.

[0014] Load frequency control, also known as LFC, is the full name of Load Frequency Control, which refers to adjusting the frequency of the system to the rated value or / and maintaining the planned value of the regional tie-line exchange power. In order to ensure power quality, the load frequency control system maintains the system frequency at the nominal value and minimizes the unplanned tie-line exchange power between control areas. Wind power and photovoltaic power are used as uncontrollable random power sources together with loads to input disturbance power to the system, while micro gas turbines and electric vehicle charging stations (discharging stations) are used as frequency modulation units of the micro-grid.

[0015] Wind power and photovoltaic power are used as uncontrollable (random) power sources together with loads to input disturbance power to the system, while micro gas turbines and electric vehicle charging and discharging stations are used as frequency modulation units of the micro-grid, and ΔP L is the load disturbance power, ΔP w is the wind power disturbance power, ΔP pv is the photovoltaic disturbance power, ΔP w and ΔP pv comprise the random power disturbance power ΔP S , and ΔP S and ΔP L comprise the total disturbance power ΔP D , ΔP MT is the output power increment of the micro gas turbine, ΔP E is the output power increment of the EV charging station, 2H tis an inertia constant of the microgrid.

[0016] S3: based on the DQN frequency controller, defining a state set as a real-time frequency deviation of the microgrid and an upper and lower limit constraint of the charging power, defining an action set as an output instruction set of the controller, considering an adjustment dead zone, designing a reward function, performing random trial and error learning training, and obtaining a super parameter value.

[0017] DQN refers to neural network reinforcement learning. The action set is used to control the output change of each frequency regulation unit. According to the evaluation standard of the microgrid frequency and considering the adjustment dead zone, a reward function is designed.

[0018] S4: according to the output power change data of the electric vehicle charging station, determining a constraint function with random changes over time, obtaining an optimal value function Q network, and importing the optimal value function Q network into the DQN frequency controller.

[0019] Through random trial and error learning training, and setting a constraint function, an optimal value function Q is obtained to ensure good convergence characteristics, so as to complete the frequency control in actual situation.

[0020] S5: the DQN frequency controller is connected with the information collection terminal of the microgrid to obtain the state information of the microgrid, find out the action of maximizing the benefit of the microgrid, and realize the transmission power through the energy management system to realize the frequency control of the microgrid.

[0021] The DQN frequency controller is in communication connection with the information collection terminal in the microgrid, so that the DQN frequency controller obtains the state information from the microgrid and finds out the action of maximizing the benefit of the system, and then realizes the frequency control of the microgrid through power transmission.

[0022] Further, the SOC constraint in the S1 step includes EV single charging and discharging power constraint and EV cluster charging and discharging power constraint. EV refers to the charging state of electric vehicle.

[0023] The three factors affecting the equivalent energy storage capacity of the charging station are the randomness of electric vehicle charging behavior, the charging state of the vehicle battery, and the number of vehicles in the charging station. The battery performance of the vehicle will affect the charging time and thus affect the controllable power of the system. However, considering that the brand and number of vehicles in the charging station at a fixed geographical location are relatively fixed within a certain time, the average value can be used to simplify the complexity of the calculation process, so the average capacity C of the battery and the average initial charging state SOC i can be obtained first, and then the average rated charging power of the charging station is obtained, so that the average charging time T of the single EV in the charging station is obtained av .

[0024] For the state of charge of the vehicle battery, the EV charging and discharging constraint model is established, the SOC of the electric vehicle in the station is in the range of [SOC min , SOC max ], and there is enough SOC m to ensure the driving range of the electric vehicle after leaving the charging station, and the EV charging and discharging constraint boundary is obtained.

[0025] The charging boundary represents that the electric vehicle is in a normal charging state, the discharging boundary represents that the electric vehicle is delivering power to the microgrid, and the forced charging boundary represents that the electric vehicle is being forced to charge to ensure that it has enough SOC m to complete the driving range after pulling out the power supply.

[0026] Further, the rated charging power of the single EV in the access charging station period is set to , and the rated discharging power is set to

[0027] Therefore, the relationship between the single EV charging power and the charging and discharging state is as follows: when SOC i >SOC max , the single EV can discharge, that is, output a positive power increment When SOC i <SOC min , the single EV can only charge, that is, only output a negative power increment When SOC min ≤SOC i ≤SOC max , the single EV can both charge and discharge, and the power increment satisfies

[0028] As described above, the upper and lower limit constraints of the single EV charging and discharging power are as follows:

[0029]

[0030]

[0031] Finally, for the number of vehicles in the charging station, the maximum capacity of the charging station is set to n EV , and when a single EV satisfies , it is located in the charging station, and when it does not satisfy , it is not in the charging station.

[0032] Therefore, the charging power P EV of the cluster EV and its upper and lower limit constraints are as follows:

[0033]

[0034] Further, considering the MT and EV output power increment limit constraint, the MT is a gas turbine, the DQN frequency controller first determines the real-time upper and lower limit constraints of the frequency deviation Δf and the charging power according to the frequency deviation Δf and the charging power And The real-time LFC signal Δu is provided to the frequency control layer, and the output power of the MT and the EV is controlled to quickly suppress the system frequency oscillation.

[0035] The state set of the micro-grid load frequency control system is the real-time frequency deviation ΔF(t) and the upper and lower limit constraints of the charging power And Therefore, the state space can be defined as:

[0036] And the joint action set A of the DQN frequency controller, that is, the output of the controller, should be the real-time set of the joint scheduling instruction (Δu MT ,Δu EV ), so the action space can be defined as: A=[ΔU MT (t),ΔU EV (t)]

[0037] Further, since the Q-learning algorithm cannot process continuous signals, the action space needs to be discretized, and to prevent the "dimension disaster" problem caused by too high discretization degree and the frequency quality reduction caused by insufficient discretization degree, the discretization degree of the state space discrete set S and the control action set A should be reasonably arranged.

[0038] The present application selects the relatively conservative electric power safety work procedure principle, that is, the frequency of the power system in the normal operating state should be within the range of 50±0.2Hz

[18] , and on this basis, a certain adjustment dead zone is considered, that is, the discrete set of the real-time frequency deviation ΔF(t) can be set as (-∞,-0.2), [-0.2,-0.15), [-0.15,-0.10), [-0.10,-0.05), [-0.05,0.05], (0.05,0.10], (0.10,0.15], (0.15,0.2], (0.2,+∞), unit Hz.

[0039] In addition, according to the maximum power increment limit of the electric vehicle ±0.16pu, and according to the actual situation, the real-time upper and lower limit constraints of the charging power And The discrete set of the micro gas turbine is divided into: [0, 0.4), [0.4, 0.8), [0.8, 0.12), [0.12, 0.16], and [-0.16, -0.12], (-0.12, -0.08], (-0.08, -0.04], (-0.04, 0], unit pu.

[0040] Further, A MT and A EV are the discrete output action sets of the micro gas turbine and the electric vehicle charging station respectively, and let A MT = A EV = (-0.01, -0.005, -0.003, -0.001, 0, 0.001, 0.003, 0.005, 0.01), unit pu.

[0041] According to the above evaluation criteria of the microgrid frequency, the reward function r i (k) can be designed as:

[0042]

[0043] When |Δf| is in the regulation dead zone [-0.05, 0.05] Hz, the frequency meets the minimum error requirement of normal operation, so the DQN controller is given the maximum reward value 0 at this time; when |Δf| is in the normal control zone (0.05, 0.10] and (0.10, 0.15] Hz, the auxiliary control zone (0.15, 0.2] Hz, and the emergency control zone (0.2, +∞) Hz respectively, the controller will obtain the corresponding negative reward, that is, the penalty value. and are the weights corresponding to the reward functions of each control area. When determining the reward function, it should be noted that too large reward value will affect the convergence speed of learning, therefore, through a large number of simulation research, the values of and are respectively taken as 1, 5, 10 and 20. Further, under the premise of ensuring the convergence of the agent training, through measurement and verification, the discount factor γ is selected as 0.9, the learning rate α is selected as 0.001, the iteration number is set as 500 times, each time 500 steps, and the full connection layer network structure of h=5 and u=50.

[0044] Further, the random trial-and-error learning process in the S3 sets the load disturbance superimposed by different amplitudes and different types of functions to train the DQN frequency controller.

[0045] The random trial and error learning process, referred to as a pre-learning stage, in the initial stage of pre-learning, the DQN frequency controller does not have intelligent control capability, only after accepting various state actions, the optimal value function Q network can be obtained. The DQN frequency controller can adapt to strong random interference and network topology parameter changes in the island micro-grid and other complex working conditions.

[0046] The application also provides an island micro-grid load frequency control system based on deep Q learning, the load frequency control system comprises a micro-grid layer, a data layer and a decision layer, data transmission and communication are carried out between the layers, the micro-grid layer comprises photovoltaic power generation, wind power generation, gas turbine, generator set and electric vehicle, the data layer comprises an energy management system and a data acquisition device, the decision layer is provided with an artificial intelligence (AI) computing management platform, the artificial intelligence (AI) computing management platform runs and comprises the DQN frequency controller, and the DQN frequency controller is used for deep Q learning action on the electric vehicle frequency control model.

[0047] Before the application of the DQN frequency controller, a random trial and error learning process is set, referred to as a pre-learning stage. In the initial stage of pre-learning, the controller has not accumulated any experience, and does not have intelligent control capability, only after accepting various state actions, the optimal value function Q network can be obtained. Therefore, a load disturbance formed by superposition of different amplitudes and different types of functions can be set to train the controller, and a group of constraint functions with random changes in probability over time are set according to the output power increment change data of a certain electric vehicle charging station. After the end of the pre-learning stage, the controller has high online learning ability and good convergence characteristics to complete the frequency control under the two scenes of strong random disturbance and system parameter structure change.

[0048] Further, the data acquisition device and the micro-grid layer are in contact through wireless communication 5G for power transmission, state information acquisition and instruction action.

[0049] Considering the storage information and transmission communication in the micro-grid system database, and based on the AI computing management platform, a model of electric vehicle user randomness prediction based on DQN is designed, and thus a network structure of the load frequency control of the island micro-grid containing electric vehicles based on DQN is obtained. Further, the storage method of system data and model is determined, the unified training standard of the micro-grid is determined, the simulation application of the load frequency control of the micro-grid containing electric vehicles based on DQN is applied to the terminal, and support for the electric vehicle participating in the grid regulation is realized. In the regulation process, the controller and the information acquisition terminal in the micro-grid can be in contact through wired transmission or wireless communication 5G, so as to realize that the DQN frequency controller obtains state information from the micro-grid and finds out the action of maximizing the system benefit, and then realizes the micro-grid frequency control through power transmission.

[0050] Further, the DQN frequency controller comprises a coordination control layer and a frequency control layer, the coordination control layer comprises a state space, an action space, a reward function space, a hyperparameter space and a deep Q network space, respectively storing the state set, the action set, the reward function, the hyperparameter and the trial and error learning data, the coordination control layer controls the output power of the electric vehicle and the gas turbine of the frequency control layer according to the real-time upper limit constraint and the lower limit constraint of the frequency deviation and the charging power, and suppresses the micro-grid frequency oscillation.

[0051] Compared with the prior art, the application has the beneficial effects that: the island micro-grid load frequency control method based on deep Q learning provided by the application can effectively improve the ability of the island micro-grid containing electric vehicles to cope with strong random disturbance and system network topology parameter change through the deep Q learning step, thereby solving the factor consideration problem of the island micro-grid caused by the random access of electric vehicles, effectively solving the problem of affecting the quality of micro-grid load control, and improving the quality of the micro-grid.

[0052] The application further provides an island micro-grid load frequency control system based on deep Q learning, which finds the maximum benefit of the micro-grid system through the deep Q learning action, and then realizes the micro-grid frequency control through power transmission, thereby solving the influence caused by the random access of electric vehicles. DETAILED DESCRIPTION

[0053] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the principles of the application. The terminology in the detailed description is used for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0054] Figure 1 The flow chart of the load frequency control method of the application.

[0055] Figure 2 The relationship diagram of the load frequency control system of the application.

[0056] Figure 3 The SOC model schematic diagram of the application.

[0057] Figure 4 The upper and lower limit constraint schematic diagram of the single EV charging power of the application.

[0058] Figure 5 The LFC controller structure based on deep Q learning of the application.

[0059] Figure 6 The pre-learning schematic diagram of the controller of the application.

[0060] Legend: W1 - microgrid layer, W2 - data layer, W3 - decision layer, D1 - coordinated control layer, D2 - frequency control layer. DETAILED DESCRIPTION

[0061] The application will be further described in conjunction with the accompanying drawings and specific embodiments, so as to be more clear and intuitive.

[0062] The application combines convolutional neural network and Q learning algorithm through deep Q learning, adopts experience replay mechanism, fixes target Q value network, and narrows down reward value range, and can well cope with the load frequency control problem of island microgrid containing electric vehicles.

[0063] Embodiment 1

[0064] The embodiment provides an island microgrid load frequency control method based on deep Q learning, as shown in Figure 1 , comprising the following steps:

[0065] Step 1, by formulating the range of electric vehicle SOC in the station [SOC min , SOC max ], wherein SOC min is the recommended minimum state of charge of the electric vehicle, SOC max is the recommended maximum state of charge of the electric vehicle, and the range of SOC can improve the service life of the electric vehicle. And select a suitable SOC m value to ensure the driving range of the electric vehicle after leaving the charging station, as shown in Figure 3 , SOC0 is the initial state of charge of the electric vehicle when entering the charging station, the left solid line represents the charging boundary; the outer dotted line represents the discharging boundary; and the right solid line represents the forced charging boundary, that is, the electric vehicle will be forced to charge before leaving the charging station to ensure that it has enough SOC m to complete the driving range after unplugging the power supply. Thus, the distribution of the controller instructions in the charging station can be obtained, and the upper and lower limit constraints of the single EV charging power can be obtained through the above classification mode as shown in Figure 4 .

[0066] Step 2, considering the output power increment amplitude constraint of the micro gas turbine and the electric vehicle, a deep Q learning-based LFC controller structure is proposed, as shown in Figure 5 . The controller consists of two layers of coordinated control layer D1 and frequency control layer D2. The coordinated control layer D1 includes state space, action space, reward function space and hyperparameter space. First, the system calculates the real-time upper and lower limit constraints of the frequency deviation Δf and the electric vehicle charging power and The real-time LFC signal Δu (i.e., the action space signal) is provided to the frequency control layer, which in turn controls the output power of the MT and EV to quickly suppress system frequency oscillation.

[0067] Step 3: Select the power safety work procedure principle according to the actual situation, i.e., the frequency of the power system in the normal operating state should be within the range of 50 ± 0.2 Hz, and consider a certain adjustment dead zone, set the real-time frequency deviation ΔF(t) interval as (-∞, -0.2), [-0.2, -0.16), [-0.16, -0.12), [-0.12, -0.08), [-0.08, -0.03), [-0.03, 0.03], (0.03, 0.08], (0.08, 0.12], (0.12, 0.16], (0.16, 0.2], (0.2, +∞) in Hz. Thus, the reward function space of the controller can be designed. The hyperparameter space is a parameter that needs to be set before training, so a set of optimal hyperparameters needs to be provided to the agent after a large number of experimental tests to improve the performance and effect of learning.

[0068] Step 4: Before the controllers in steps 2 and 3 are put into use, they need to undergo a random trial-and-error learning process, called the pre-learning phase. In the early stage of pre-learning, the controller has not accumulated any experience and does not have intelligent control capabilities. Only after accepting various state actions can the optimal value function Q network be obtained. Therefore, this paper sets up load disturbances of different amplitudes and different types to train the controller; at the same time, a set of constraint functions with random changes in probability over time is set according to the output power increment change data of a certain electric vehicle charging station. The pre-learning process is shown in Figure 6 , where the vertical axis represents the reward value and the horizontal axis represents the number of training rounds. The circles represent real-time reward values, and the asterisks represent average reward values.

[0069] Step 5: Finally, the control terminal runs the microgrid frequency control test scheme as follows:

[0070] First, the electric vehicle microgrid frequency control simulation test. Based on the actual microgrid model composition, the frequency control models of the micro gas turbine, electric vehicle charging station, and distributed power source are built into a simulation model based on Simulink, and simulation verification is performed to verify the control rationality.

[0071] Second, the simulation model is embedded in a virtual machine for running tests. After determining the control logic through simulation verification, the simulation model constructed above is converted into code and loaded into a virtual machine for running. Through programming and Modbus communication, the microgrid model and the controller can exchange signals and perform frequency control tests.

[0072] Finally, the transmission power is performed through the energy management system to realize the micro-grid frequency control.

[0073] Embodiment 2

[0074] The embodiment provides an island micro-grid load frequency control system based on deep Q learning, which comprises a micro-grid layer W1, a data layer W2 and a decision layer W3. Figure 2 As shown in the figure, the micro-grid layer W1, the data layer W2 and the decision layer W3 are considered in the light of the information stored in the micro-grid system database and the transmission communication, and a model of electric vehicle user randomness prediction based on DQN is designed based on an AI computing management platform, so that a network structure of the island micro-grid load frequency control containing electric vehicles based on DQN is obtained. Further, the storage method of system data and the model is determined, the unified training standard of the micro-grid is determined, the simulation of the micro-grid load frequency control containing electric vehicles based on DQN is applied to the terminal, and support for the electric vehicles participating in the power grid regulation and control is realized. In the regulation and control process, the controller and the information acquisition terminal in the micro-grid can be connected in a wired transmission or wireless communication 5G mode, so that the DQN frequency controller obtains the state information from the micro-grid and finds out the action maximizing the system benefit, and then realizes the micro-grid frequency control through power transmission.

[0075] The embodiment first considers the randomness of the user charging behavior in the modeling process of the micro-grid, that is, the performance parameters of the vehicle battery, the charging state of the vehicle battery and the number of vehicles in the charging station, constructs an SOC model of the electric vehicle by analyzing the charging and discharging constraint boundary of the electric vehicle, divides the charging states of the single EV, considers the battery performance of the single EV and the number of EVs in the charging station and other parameters, designs an electric vehicle frequency control model under the constraint of random output power increment, and thus establishes an island micro-grid LFC model containing various distributed power sources, electric vehicles and the constraint conditions of the random output power increment of the electric vehicles. Meanwhile, the deep Q learning algorithm formed by combining Q learning and deep learning is used, a frequency controller based on DQN is designed, the definition of the state space, the action space and the reward function is sequentially completed, and the optimal hyperparameters are obtained through adjustment. The controller has the abilities of online learning and experience playback, has good convergence characteristics and model adaptability, and can adapt to complex working conditions such as strong randomness interference and network topology parameter change in the island micro-grid.

Claims

1. A deep Q-learning-based island microgrid load frequency control method, characterized in that, The method comprises: S1: based on the user random charging behavior in the electric vehicle charging station, an electric vehicle frequency control model is established, and the electric vehicle frequency control model comprises an SOC constraint; S2: a micro-grid load frequency control model is established, and the micro-grid load frequency control model comprises a photovoltaic power generation disturbance power parameter constraint, a wind power generation disturbance power parameter constraint, a load random power increment parameter constraint, a gas turbine output power increment amplitude constraint and an electric vehicle output power increment amplitude constraint; S3: based on the DQN frequency controller, the state set is defined as the real-time frequency deviation and the upper and lower limit constraints of the charging power of the micro-grid, the action set is defined as the output instruction set of the controller, the adjustment dead zone is considered, the reward function is designed, the random trial and error learning process is carried out, and the super parameter value is obtained; S4: according to the electric vehicle charging station output power change data, a constraint function with random changes over time is determined, an optimal value function Q network is obtained, and the optimal value function Q network is introduced into the DQN frequency controller; S5: the DQN frequency controller is connected with the information collection terminal of the micro-grid, the state information of the micro-grid is obtained, the action of maximizing the benefit of the micro-grid is found out, the transmission power is carried out through the energy management system, and the micro-grid frequency control is realized.

2. The island micro-grid load frequency control method based on deep Q learning according to claim 1, wherein: The state set in the S3 step is the real-time frequency deviation ΔF(t) of the microgrid and the upper limit constraint of the charging power and the lower limit constraint The state set defines the state space as:

3. The island micro-grid load frequency control method based on deep Q learning according to claim 1, wherein: The adjustment dead zone in the S3 step is a discrete set of real-time frequency deviation ΔF(t); and the reward function is:

4. The island micro-grid load frequency control method based on deep Q learning according to claim 1, wherein: In the S3 step, the random trial and error learning process is set by superimposing load disturbances of different amplitudes and different types to train the DQN frequency controller.

5. A deep Q-learning based islanded microgrid load frequency control system, characterized in that: The load frequency control system for the deep Q learning island micro-grid load frequency control method according to any one of claims 1 to 4 comprises a micro-grid layer, a data layer and a decision layer, data transmission and communication are carried out between the layers, the micro-grid layer comprises photovoltaic power generation, wind power generation, a gas turbine, a generator set and an electric vehicle, the data layer comprises an energy management system and a data collection device, the decision layer is provided with an artificial intelligence computing management platform, the artificial intelligence computing management platform runs the DQN frequency controller, and the DQN frequency controller is based on the DQN frequency controller to perform deep Q learning action on the electric vehicle frequency control model.

6. The deep Q-learning based islanded microgrid load frequency control system of claim 5, wherein: The data collection device and the micro-grid layer are connected through wireless communication 5G to perform power transmission, collect state information and perform instruction action.

7. The deep Q-learning based islanded microgrid load frequency control system of claim 5, wherein: The DQN frequency controller comprises a coordination control layer and a frequency control layer, the coordination control layer comprises a state space, an action space, a reward function space, a hyperparameter space and a deep Q network space, which respectively store a state set, an action set, a reward function, a hyperparameter and trial-and-error learning data, and the coordination control layer controls the output power of the electric vehicles and the gas turbine of the frequency control layer according to the real-time upper limit constraint and lower limit constraint of the frequency deviation and the charging power, and suppresses the micro-grid frequency oscillation.

Images