Island micro-grid electrolytic cell adaptive configuration method based on deep reinforcement learning

Abstract

The present application relates to the technical field of micro-grid control, and discloses a sea-island micro-grid electrolytic cell adaptive configuration method based on deep reinforcement learning. In the discrete time step, seven-state physical working condition analysis and decay integral accumulation are carried out on the proton exchange membrane electrolytic cell, the variable hydrogen production efficiency is read by using two-dimensional lookup table fixed-point displacement addressing, the environment tensor containing real-time decay, energy storage capacity and wind-solar difference is constructed, the deep reinforcement learning double network processing output device installation boundary and shunting parameters are input, and the power surge is preferentially guided to the storage battery in cooperation with the hardware action space mask and the life reset trigger. The present application suppresses the high-frequency voltage decay of the electrode from the bottom electrochemical level, and greatly reduces the comprehensive energy consumption and reset cost in the whole life cycle.

Description

Technical Field

[0001] This invention relates to the field of microgrid control technology, and in particular to an adaptive configuration method for electrolyzers in island microgrids based on deep reinforcement learning. Background Technology

[0002] In microgrid power supply systems in remote areas such as islands, proton exchange membrane electrolyzers (PEMELs) have become a core energy storage and conversion carrier for long-term absorption of wind and solar renewable energy power and solving the mismatch between supply and demand due to their characteristics such as fast start-up and rapid fluctuation response. However, the output of wind and solar power and other renewable energy sources in the extremely harsh environment of islands is highly intermittent and fluctuates widely. This high-frequency and drastic change in input power can cause drastic transient shifts in temperature and potential inside the electrolyzer, leading to local overheating and high potential damage to the diaphragm plates. This, in turn, results in electrochemical activation and an increase in ohmic resistance, causing irreversible plate voltage decay and significantly reducing the service life of the equipment.

[0003] Currently, in research on microgrid optimization scheduling and capacity allocation considering electrolyzer degradation, most traditional solutions tend to use static heuristic algorithms (such as particle swarm optimization combined with linear programming solvers) to perform two-level optimization of the entire lifecycle cost. However, when complex electrochemical dynamic characteristics such as multi-scale variable hydrogen production efficiency degradation, frequent transitions in operating conditions, and nonlinear product terms are integrated into the model, traditional methods are prone to getting trapped in local optima due to slow computational convergence and high floating-point operation time. Furthermore, the original algorithms decouple capacity allocation from power scheduling, lacking dynamic boundary response capabilities during online operation. The optimization results often lead to accelerated overload aging of electrolyzers under actual operating conditions and a surge in expected replacement costs, posing significant industrial implementation risks and control delays to the operation and maintenance system. Summary of the Invention

[0004] This invention provides an adaptive configuration method for electrolyzers in island microgrids based on deep reinforcement learning. This addresses the challenges faced by existing island microgrids when receiving intermittent, high-frequency fluctuating renewable energy power. These challenges include plate overload and overheating of proton exchange membrane electrolyzers due to rapid changes in operating conditions, severe and continuous degradation of variable hydrogen production efficiency, and a significant reduction in physical lifespan. Furthermore, existing microgrid capacity configuration methods largely ignore the degradation of underlying electrochemical efficiency or rely solely on traditional heuristic algorithms. These methods suffer from slow floating-point convergence and processing congestion when dealing with multi-scale nonlinear transformations and continuous product terms. Additionally, there is a severe disconnect between static capacity planning and online real-time control, ultimately leading to frequent overload failures of electrolyzers under actual operating conditions, a surge in lifecycle replacement costs, and high latency in underlying collaborative control.

[0005] In view of the above problems, the present invention provides an adaptive configuration method for electrolyzers in island microgrids based on deep reinforcement learning, executed by a computer device, the method comprising: Collect renewable energy output time-series data and multi-energy load demand data of island microgrids, divide the continuous operation time into discrete time control steps, and initialize the cumulative voltage decay register corresponding to the proton exchange membrane electrolyzer in the first storage area to zero value; The physical operating state of the proton exchange membrane electrolyzer is detected, and the physical operating state is mapped into a condition representation vector containing seven discrete operating conditions. The arithmetic logic unit reads the condition representation vector, extracts the voltage decay rate coefficient of the corresponding operating condition preset in the system firmware, calculates the single-step voltage decay within the current step size, and accumulates the single-step voltage decay into the cumulative voltage decay register to obtain the real-time cumulative voltage decay. The real-time input power and the real-time cumulative voltage decay are input into a two-dimensional lookup table of variable hydrogen production efficiency pre-constructed in a continuous physical memory space. The corresponding electro-hydrogen conversion efficiency and target output hydrogen power are read through fixed-point quantization and displacement addressing. An environmental state vector containing the real-time cumulative voltage decay, the remaining capacity of the energy storage device, and the fluctuation value of renewable energy output is input into a deep reinforcement learning policy network. Matrix multiplication and activation function operations are performed in the tensor processing unit to output an action tensor, which is then parsed into device capacity configuration instructions and real-time power allocation parameters. The hardware installation boundary between the proton exchange membrane electrolyzer and the energy storage device is determined according to the equipment capacity configuration instruction, and the microgrid power electronic inverter and hydrogen pipeline control valve are driven by the real-time power distribution parameters to control the coordinated power splitting of the proton exchange membrane electrolyzer and the energy storage device.

[0006] The technical solution provided in this application has at least the following technical effects: This application employs a tensor processing unit to run a deep reinforcement learning policy network and embeds an action space truncation mask before the inverter control commands to prioritize driving the battery to absorb excess energy during input power surges. This mechanism avoids the risk of being subjected to fluctuations due to blindly increasing the installed capacity of electrolyzers, successfully confining the electrolyzers to a steady-state, high-efficiency range alternating between high load and standby, significantly reducing electrode voltage decay by 10% to 20%, and extending the physical lifespan of the electrolyzers from the expected reduced state to more than 5.8 years.

[0007] This application integrates the reduced losses due to the efficiency degradation of the underlying electrolyzer and the trigger for resetting the limit degradation threshold into the reinforcement learning reward scalar, thereby achieving a negative cost buffer for model evolution. Under the influence of multiple material degradation factors, this method can dynamically optimize the capacity boundaries of the battery and electrolyzer, effectively reducing the replacement costs caused by frequent equipment scrapping, and lowering the actual total lifecycle cost by 1% to 4%, enabling the system to maintain a high hydrogen production efficiency of 69.23% even in the 6th year of operation.

[0008] This application utilizes McCormick physical register envelopes and piecewise linearization to eliminate computational congestion caused by polynomial nonlinear differentiation. Simultaneously, it embeds the hydrogen production polynomial into SRAM during the operating system boot process to construct a static two-dimensional lookup table. In real-time control, it invokes fixed-point truncation and single-clock-cycle left shift instructions, replacing floating-point function calculations with hardware shift addressing. This achieves nanometer-level ultra-fast energy mapping and zero-latency collaborative control of the microgrid at the microprocessor instruction cycle level. Attached Figure Description

[0009] Figure 1 A three-dimensional physical relationship diagram of output hydrogen energy, input power, and voltage attenuation of a proton exchange membrane electrolyzer provided in an embodiment of the present invention; Figure 2 This is a physical architecture topology diagram of an island microgrid system provided in an embodiment of the present invention; Figure 3 This is a diagram of the two-layer capacity configuration and adaptive control model architecture provided in an embodiment of the present invention; Figure 4 The flowchart illustrates the execution of the adaptive configuration method based on deep reinforcement learning provided in this embodiment of the invention. Figure 5 This is a graph showing the evolution of discrete operating conditions of the electrolytic cell over four consecutive typical days in the comparative experiment of this invention; Figure 6 This is a graph showing the cumulative electrode voltage decay in each comparative scenario during a typical day of operation in the comparative experiment of this invention; Figure 7 This is a graph showing the decline in hydrogen production efficiency of the electrolyzer under long-term operation in various comparative scenarios in the comparative experiments of this invention. Detailed Implementation

[0010] The above technical solutions will now be described in detail with reference to the accompanying drawings and specific embodiments to provide a better understanding of them. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be understood that the present invention is not limited to the exemplary embodiments used only to explain the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. Furthermore, it should be noted that, for ease of description, only the parts related to the present invention are shown in the drawings, not all of them.

[0011] Example: The technical solution provided by this invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This invention establishes an adaptive configuration method that integrates the electrochemical degradation characteristics of underlying materials with the hardware collaboration of edge computing power. Through microprocessor state register monitoring and tensor inference, adaptive capacity planning and current shunting suppression are performed on the proton exchange membrane electrolyzer (PEMEL) of the island microgrid under the wide fluctuation of renewable energy conditions, thereby curbing electrode aging and voltage decay caused by high-frequency power fluctuations from the physical source.

[0012] Please see Figure 2 , Figure 2 This is a topological diagram of the physical architecture of an island microgrid system provided in an embodiment of the present invention. The physical architecture of the island microgrid is divided into an energy execution plane and an intelligent control plane. The energy execution plane consists of a photovoltaic array (PV), a wind turbine (WT), a proton exchange membrane electrolyzer (PEMEL), a battery bank (BAT), a high-pressure hydrogen storage tank (HS), and a hydrogen fuel cell (FC). The photovoltaic array and wind turbine provide input electrical energy. When the power generation exceeds the island's electrical load demand, the excess power is fed into the proton exchange membrane electrolyzer via a DC bus to electrolyze water to produce hydrogen, which is then supplied to hydrogen loads such as hydrogen-powered vehicles and hydrogen-fueled ships. The energy storage system includes a battery bank and a hydrogen storage tank. The battery bank is used to smooth out transient fluctuations in wind and solar power output, while the hydrogen storage tank converts renewable energy into chemical energy for long-term storage. When the electrical load peaks and wind and solar power output is insufficient, the hydrogen storage tank releases hydrogen to drive the fuel cell to generate electricity and feed it back to the AC microgrid.

[0013] The core of the intelligent control plane is an industrial-grade edge tensor computing server deployed on the island. Internally, the edge computing server is connected via a system bus to a multi-core central processing unit (CPU), an arithmetic logic unit (ALU), static random access memory (SRAM), high-bandwidth video memory, and a tensor processing unit (TPU). A sensor network collects real-time data on wind and solar power generation voltage levels, load current parameters, and equipment plate converter signals, submitting these data to the edge computing server's memory registers via the fieldbus, serving as the basic physical input for the underlying adaptive control.

[0014] Please see Figure 3 and Figure 4 , Figure 3 This is a diagram of the dual-layer capacity configuration and adaptive control model architecture of the present invention. Figure 4 This is a flowchart illustrating the execution of an adaptive configuration method based on deep reinforcement learning. The main adaptive configuration and control process covers the discrete-time partitioning stage, the physical condition analysis and decay accumulation stage, the fixed-point lookup table addressing stage, and the reinforcement learning tensor modulation stage. The microprocessor divides the continuous operation cycle of the system into discrete-time control steps. The first storage area is allocated in the physical contiguous memory area and bound to the electrolytic cell cumulative voltage decay register. The register level is cleared to zero during the initial stage of system power-on.

[0015] During online operation, the microprocessor reads the voltage at the electrolyzer converter terminals in real time, parses it into binary discrete operating conditions using a logic gate array, and performs integral calculations based on the corresponding voltage attenuation coefficient according to different operating conditions, updating the value in the cumulative voltage attenuation register in real time. Subsequently, the arithmetic logic unit calls a memory truncation instruction to convert the input power and attenuation floating-point numbers into integer indices, and uses a single-clock-cycle left shift instruction to superimpose the base address bias, directly reading the output hydrogen power and electro-hydrogen conversion efficiency from a static two-dimensional lookup table. The central processing unit encapsulates the attenuation data, battery state of charge (SOC), and wind-solar differential levels into state vectors and submits them to the tensor processing unit. The policy network infers and outputs an action tensor containing the equipment installation boundary and real-time power distribution ratio. The hardware execution module drives the microgrid power electronic inverter chopper and hydrogen valve based on the action tensor, achieving the optimization of equipment physical lifespan and global energy cost.

[0016] The conversion efficiency of a proton exchange membrane electrolyzer to hydrogen energy varies under different operating conditions. The variable hydrogen production efficiency is physically defined as the product of voltage efficiency and current efficiency, expressed as: , The electro-hydrogen conversion efficiency of the electrolyzer; For voltage efficiency; This refers to the current efficiency.

[0017] The voltage efficiency of an electrolytic cell is a isomorphic function of the overvoltages in different parts of the cell, expressed as: In the formula: This is the actual operating voltage of the electrolytic cell; It is the thermal neutral voltage; To activate overvoltage; For ohmic overvoltage; This is concentration overvoltage. The voltages of each component are electrochemically coupled with the operating temperature, Faraday constant, number of electrons transferred in the reaction, anolyte exchange current density, gas constant, and proton exchange membrane ohmic resistance.

[0018] Given that the macroscopic physical phenomenon of aging of various components in an electrolyzer manifests as requiring more electrical energy to produce the same amount of hydrogen (i.e., a continuous increase in operating voltage or a decrease in current density), this application quantifies the degradation of PEMEL as an overall voltage decay. This allows for seamless integration into the voltage efficiency model: .

[0019] When the electrolyzer receives fluctuating power input, the drastic changes in operating conditions will accelerate the aging of the proton exchange membrane. The microprocessor reads the voltage level of the plate converter and, through the underlying logic circuit, resolves the real-time operating status of PEMEL into seven mutually exclusive binary state variables: shutdown state. Standby mode Low load condition High load condition Overload condition and cold start status and hot start status Specifically, the transition from a stopped state to a working state triggers a cold start Boolean bit set to 1, and the transition from a standby state to a working state triggers a warm start Boolean bit set to 1.

[0020] The voltage decay generated by the electrolytic cell within the current clock step t is obtained by the arithmetic logic unit determining the current operating condition and calling the corresponding voltage decay rate. The cumulative recursive relationship is defined as follows: In the formula, The cumulative voltage decay stored in the register at time t; The register value is the old value from the previous moment; This represents the Boolean state bit under condition s at time t; The voltage decay rate corresponding to the S-condition is fixed in the system firmware, and the parameter calibration covers all levels from standby to hot start. This is for discrete-time control step size. When the arithmetic logic unit detects the stop Boolean state bit ( When =1), the clock controller directly blocks the multiplication instructions of the floating-point arithmetic unit, keeps the value of the cumulative decay register unchanged, and eliminates invalid computing power loss.

[0021] Please see Figure 1 , Figure 1 A three-dimensional physical relationship diagram of output hydrogen energy, input power, and voltage decay in a proton exchange membrane electrolyzer. The diagram shows the cumulative voltage decay... Increase hydrogen energy output and input power The efficiency of the electro-hydrogen conversion between the two continuously declines. To avoid consuming CPU instruction cycles for online calculations of complex polynomials, the system initiates a space-for-time optimization mechanism during the operating system boot phase. The initialization daemon process iterates through the numerical combinations of input power and cumulative decay within the discrete floating-point nominal boundaries, and calls the floating-point arithmetic unit to deduce the hydrogen production power based on a quadratic fitting formula. In the formula, - are the coefficients of the polynomial.

[0022] The daemon process stores the calculated floating-point hydrogen power results one by one into consecutive physical addresses in static random access memory (SRAM), generating a static two-dimensional lookup table.

[0023] After the microgrid enters operational status, the arithmetic logic unit (ALU) performs fixed-point acceleration. The microprocessor calls a type conversion instruction to truncate the real-time collected floating-point values ​​of input power and cumulative voltage decay into integer index vectors. Subsequently, the ALU extracts the valid data bits of the power integer index and executes a left shift instruction, simultaneously extracting the valid data bits of the decay integer index and executing a left shift instruction. The state shift output is input to a bitwise OR gate, superimposed with a preset base address memory bias mask, and outputs an integer memory address pointer. Based on this pointer, the central processing unit directly extracts the target hydrogen production power from the SRAM static two-dimensional lookup table using a single physical memory load instruction, reducing the execution latency to within the nanometer-level clock span of modern physical memory.

[0024] Addressing the nonlinear efficiency conversion and continuous variable product terms in the capacity configuration and operation optimization model of microgrid systems This application performs low-level hardware-level convex envelope reconstruction through the arithmetic logic unit of the microprocessor. The arithmetic logic unit calls the linear comparison instruction to read the upper and lower bound values ​​of the physical register of the cumulative voltage decay, and generates four closed boundary register constraints according to McCormick envelope logic, which strictly clamps the effective fluctuation range of the nonlinear product term within the quadrilateral polygon space formed by the connection of vertices; In the formula, yes upper and lower boundaries, and These represent the current input power and the cumulative voltage decay, respectively. and The physical lower and upper bounds of the input electrical power to the electrolytic cell; and These represent the physical lower and upper bounds of the cumulative voltage decay.

[0025] Simultaneously, the piecewise linearization logic unit is invoked to divide the second power operation of the input electrical power into a weighted sum of discrete trapezoidal intervals, completely eliminating computational congestion when processing high-order polynomial derivatives.

[0026] After completing the nonlinear spatial clamping, the central processing unit assembles the real-time cumulative voltage decay, the battery state of charge (SOC) register value, the high-pressure hydrogen storage tank gas pressure conversion capacity, and the renewable energy output differential data into a one-dimensional continuous floating-point environmental state vector via the high-speed bus, and sends it to the high-bandwidth video memory of the tensor processing unit through the direct memory access channel.

[0027] The tensor processing unit invokes the Actor-Critic dual-network inference engine embedded in video memory. The policy network (Actor) reads the environment state vector and invokes the Single Instruction Multiple Data (SIMD) instruction set to perform high-dimensional matrix multiplication and modified linear unit activation operations in the tensor processing array, directly outputting the physical action tensor. The action tensor contains two dimensions: the first dimension is the equipment installed capacity limit, which locks the hardware physical upper limit of the rated power of the proton exchange membrane electrolyzer, the rated capacity of the battery, the capacity of the hydrogen storage tank, and the power of the fuel cell; the second dimension is the real-time operation allocation ratio, which determines the current adjustment parameters of the microgrid redundant power between the battery charging interface and the electrolyzer input interface at the current step size.

[0028] Before the action tensor is converted into the underlying inverter pulse control command, the underlying current interception is performed by the action truncation mask constructed in the programmable logic circuit. When the sensor detects a transient surge in wind power or photovoltaic output, and the total input power monitored by the bus exceeds the upper limit threshold of the electrolyzer's nominal high load condition, the action mask hardware interrupt logic is triggered and set.

[0029] The Boolean mask directly blocks the control signal from the current controller to inject all remaining power into the electrolyzer. Based on the feedback level of the battery's state of charge register, it prioritizes generating a duty cycle adjustment signal to drive battery charging, diverting high-frequency power surges to the battery cells. Only when the bus monitoring detects that the battery's SOC has reached its physical maximum load limit (e.g., 90% capacity limit) does the action mask circuit release the electrolyzer overload latch register, allowing the power electronic chopper to briefly drive the electrolyzer into overload operation. Through this hardware current diversion logic, the electrolyzer's operating time is strictly limited to the efficient standby and high-load periods, suppressing high-frequency overload and load fluctuations that cause a surge in degradation from the physical source.

[0030] The sole metric for evaluating decision quality in deep reinforcement learning value networks (Critic) is the negative of the comprehensive cost scalar, calculated in real-time by the arithmetic logic unit. The central processing unit calculates the total lifecycle cost of the system over the initial planning year. Mapped to the negative optimization objective of agent evolution, the addition logic gate performs a summation of the following four core physical costs within the sampling period: In the formula, The initial one-time investment cost of the equipment is obtained by multiplying the rated configuration capacity of each piece of hardware by the nominal installation unit price. This represents the cost difference between equipment recovery and replacement within the cycle. Annual operating and maintenance costs and penalties for abandoned power lines; Costs are calculated based on the reduction in efficiency of the electrolyzer.

[0031] To quantify the decrease in hydrogen production efficiency due to electrode voltage decay, the system sets a unit hydrogen production loss reduction factor, and the arithmetic logic unit performs the following algebraic calculation on the hydrogen production loss over the entire project lifecycle: In the formula, Costs are calculated based on the reduction in electrolytic cell efficiency; The unit hydrogen production loss price conversion factor; T is the total number of control steps throughout the entire life cycle; HHV represents the actual output hydrogen energy power at time t; HHV is the nominal higher calorific value constant of hydrogen. This represents the cumulative voltage decay at time t. The microprocessor's subtraction register directly inverts this value to generate the immediate reward scalar that feeds back the current control step size to the policy network. .

[0032] The expected upper limit of the physical lifespan of the proton exchange membrane electrolyzer during the complete project lifecycle is quantified to the 20% drop threshold of the rated reversible operating voltage due to cumulative voltage decay. The comparator circuit within the arithmetic logic unit compares the current cumulative voltage decay register value in the first memory area with the system's fixed voltage decay threshold in real time. When the comparator determines that the value has reached the upper limit, it sends a trigger pulse to the interrupt control chip. The microprocessor skips the current regular control clock and executes two synchronous physical actions: it adds the single-cell update and reset cost to the comprehensive cost accumulation register in a single operation; simultaneously, it calls a write-back instruction to clear all the cumulative voltage decay register spaces of the corresponding group of electrolyzers to 0V, increments the total replacement count by 1 in the system status table, and simulates the hardware replacement and performance restoration of the aging electrolyzer from a physical level.

[0033] At each control step clock transition, the value network extracts the real-time environment state vector, the current step feedback reward scalar, and the next step long state vector. The arithmetic logic unit calculates the timing difference error in the register according to the Bellman equation: In the formula, The time-series difference error scalar calculated at time t; The immediate reward scalar value is the environmental feedback after the action is performed at time t. A future reward discount scalar stored in read-only memory; A functional representation for estimating state values ​​in a value network; and These are the environmental state vectors collected at time t+1 and time t, respectively; Let be the array of synaptic weights in the memory matrix of the value network at time t.

[0034] Based on the timing differential error, the tensor processing unit calls the underlying chained calculation circuit and backpropagation instruction set to update the transistor level weights in the memory arrays of the policy network and value network. When the monitoring timer detects that the arithmetic sum of the squares of the differential error is continuously below the hardened convergence limit within a preset clock span, the bus controller cuts off the memory write permission and redirects the frozen high-bandwidth memory weights to the static inference code area of ​​the non-volatile flash memory. The microgrid controller then completes the underlying switch from the exploration training mode to the static physical control mode.

[0035] To verify the effectiveness of this application in considering the efficiency decay of the proton exchange membrane electrolyzer and the hardware adaptive current diversion configuration, a rigorous comparison was conducted between simulation calculations and actual operating performance for the following three scenarios based on measured meteorological and load data of the island microgrid: Scenario 1: Capacity configuration and operation scheduling without considering any attenuation characteristics of the electrolyzer; Scenario 2: Ignore the variable hydrogen production efficiency decay of the electrolyzer, and only configure capacity and schedule operation according to the fixed lifetime decay curve; Scenario 3: The adaptive model proposed in this application is adopted, and efficiency decay and lifetime decay are integrated into the control strategy for planning and operation.

[0036] Under actual operational conditions (i.e., the electrolyzer faces reduced actual lifespan and decreased hydrogen production efficiency due to renewable energy fluctuations), the actual lifecycle cost of the system varies significantly. Specific simulation data statistics are shown in Table 2.

[0037] Scenario 1, due to the failure to consider attenuation during planning, resulted in an optimization scheme that unilaterally expanded the electrolyzer capacity to directly absorb high-frequency power fluctuations. This led to a rapid reduction in the electrolyzer's physical lifespan from the expected 6 years to the actual 4.71 years, causing the equipment replacement cost to surge to 2.766 × The actual total lifecycle cost climbed to 8.789 × yuan. Yuan; the actual total lifecycle cost after optimization in Scenario 2 is 8.527 × Yuan; while in scenario 3, which adopts the processing method of this application, the actual total life cycle cost is the lowest, reduced to 8.434× Yuan. Compared to Scenario 1 and Scenario 2, the total cost of the proposed solution is reduced by 4% and 1% respectively, and the cost of efficiency degradation converted into losses is reduced by approximately 17% compared to Scenario 1.

[0038] Analysis of the underlying equipment installation boundary results shown in Table 3:

[0039] The optimized electrolytic cells in Scenario 1 and Scenario 2 have rated installed power as high as 1720.4kW and 1774.3kW, respectively. However, Scenario 3 of this application, through collaborative space exploration using reinforcement learning, reduces the upper limit of the electrolytic cell installed power to 1664.9kW, while simultaneously expanding the battery configuration capacity significantly to 1357.4kWh (approximately twice the battery configuration capacity of Scenario 1 and Scenario 2). This reconstruction of physical boundaries confirms the core control logic of this application: instead of blindly enlarging the electrolytic cell to directly bear the scouring effect of fluctuations, it strengthens the battery as a physical buffer medium, using a large-capacity battery to share and smooth transient power peaks and valleys, creating a steady-state working environment for the electrolytic cell.

[0040] Please see Figure 5 , Figure 5 This demonstrates the discrete evolution trajectory of the binary operating condition machine of the electrolyzer over time over four consecutive typical operating days. On typical days 1 and 2, the electrolyzers in scenarios 1 and 2 frequently experienced overload operation during the 0-5 time frame. ) and low-load operation from 18:00 to 23:00 ( In contrast, scenario 3 of this application, under the control of the policy network, significantly shortens the time of high-current overload and low-current underload that drastically cause plate aging, and greatly extends the relatively stable high-load operating period. During the peak photovoltaic output period from 11:00 to 15:00 on typical day 2, the action mask circuit forces the battery to absorb excess power. On typical day 3 at 18:00, the battery is instructed to discharge in advance to suppress the load peak, thus enabling the electrolytic cell to successfully avoid entering the overvoltage attenuation range.

[0041] Please see Figure 6 and Figure 7 , Figure 6 This is a comparison chart showing the cumulative electrode voltage decay in various scenarios during a typical day of operation. Figure 7 This chart compares the efficiency degradation of electrolyzers during long-term operation in various comparative scenarios. Figure 6 A comparison of the recorded real-time voltage decay of various typical electrolytic cells shows that after four typical working days of physical operation, the cumulative plate voltage drop of the electrolytic cell in Scenario 1 reached as high as 0.085V, while that in Scenario 2 was 0.076V. However, the cumulative voltage drop of the electrolytic cell using the control strategy of Scenario 3 in this application was strictly suppressed to 0.068V. Compared to conventional control methods that do not consider decay, this application significantly reduces the electrode decay caused by charge surges by 20%; compared to methods that only consider static lifespan aging, the decay is further reduced by 10%. Figure 7The presented long-cycle efficiency curve of the electrolyzer confirms that by the 6th year of operation, the average annual hydrogen production efficiency of the Scenario 1 electrolyzer had plummeted to 66.93% (a decrease of 13.87% from the initial stage), while Scenario 3 of this application still maintained a relatively high and stable hydrogen production efficiency of 69.23% in the 6th year, ensuring the efficient utilization and reliable power supply of renewable energy hydrogen production in the integrated energy island microgrid throughout its entire life cycle.

[0042] In engineering applications, the software model of this invention is fully compiled and executed on the controller's underlying hardware. The computer device handling the method is instantiated as a high-concurrency heterogeneous edge server, whose motherboard integrates a general-purpose microprocessor and a dedicated tensor computing chip (such as a GPU or TPU module). Multiple cyclic matrix multiplications, backpropagation of timing difference errors, and two-dimensional lookup table fixed-point addressing logic are converted by the compiler into a machine code instruction set specific to the underlying machine architecture and loaded into the physical memory code segment space.

[0043] The general-purpose microprocessor reads assembly operation instructions and performs parallel multiplication and addition operations on multiple operating condition feature vectors by calling the Single Instruction Stream Multiple Data (SIMD) extended instruction set. When performing status bit matching and logic clamping, the level of the bipolar transistor in the arithmetic logic unit quickly flips, directly changing the value of the duty cycle register driving the chopper.

[0044] The carrier storing the underlying machine code instruction set of the adaptive control algorithm is instantiated as an industrial-grade non-volatile computer-readable storage medium. This computer-readable storage medium includes solid-state drives, flash memory chips, and read-only flash memory space built into the embedded controller. The machine code instruction set is embedded within the semiconductor floating-gate charge capture unit in the form of high and low level physical voltage characteristics. When the microgrid control cluster is powered on and a reset interrupt is triggered, the motherboard's basic input / output system directly reads the boot code from the storage medium, loads the online adaptive inference execution strategy into memory, and submits it to the processor for scheduling and execution.

[0045] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive configuration method for electrolyzers in island microgrids based on deep reinforcement learning, characterized in that, Performed by a computer device, the method includes: Collect renewable energy output time-series data and multi-energy load demand data of island microgrids, divide the continuous operation time into discrete time control steps, and initialize the cumulative voltage decay register corresponding to the proton exchange membrane electrolyzer in the first storage area to zero value; The physical operating state of the proton exchange membrane electrolyzer is detected, and the physical operating state is mapped into a condition representation vector containing seven discrete operating conditions. The arithmetic logic unit reads the condition representation vector, extracts the voltage decay rate coefficient of the corresponding operating condition preset in the system firmware, calculates the single-step voltage decay within the current step size, and accumulates the single-step voltage decay into the cumulative voltage decay register to obtain the real-time cumulative voltage decay. The real-time input power and the real-time cumulative voltage decay are input into a two-dimensional lookup table of variable hydrogen production efficiency pre-constructed in a continuous physical memory space. The corresponding electro-hydrogen conversion efficiency and target output hydrogen power are read through fixed-point quantization and displacement addressing. An environmental state vector containing the real-time cumulative voltage decay, the remaining capacity of the energy storage device, and the fluctuation value of renewable energy output is input into a deep reinforcement learning policy network. Matrix multiplication and activation function operations are performed in the tensor processing unit to output an action tensor, which is then parsed into device capacity configuration instructions and real-time power allocation parameters. The hardware installation boundary between the proton exchange membrane electrolyzer and the energy storage device is determined according to the equipment capacity configuration instruction, and the microgrid power electronic inverter and hydrogen pipeline control valve are driven by the real-time power distribution parameters to control the coordinated power splitting of the proton exchange membrane electrolyzer and the energy storage device.

2. The method according to claim 1, characterized in that, The process of mapping the physical operating state into a condition representation vector containing seven discrete operating conditions includes: By reading the voltage converter level status at both ends of the proton exchange membrane electrolyzer, Boolean state bits corresponding to seven mutually exclusive physical states—shutdown, standby, low load, high load, overload, cold start, and hot start—are generated. In the step of accumulating the single-step voltage attenuation amount into the cumulative voltage attenuation register to obtain the real-time cumulative voltage attenuation amount, the arithmetic logic unit uses the following recursive relationship for calculation: in, The cumulative voltage decay stored in the register at time t; The register value is the old value from the previous moment; This represents the Boolean state bit under the s-th condition at time t; The voltage attenuation rate corresponding to the S-condition fixed in the system firmware; The discrete-time control step size is defined as follows: when the Boolean state bit is in the shutdown condition, the clock multiplication instruction is skipped, and the real-time cumulative voltage decay remains unchanged.

3. The method according to claim 1, characterized in that, The step of inputting the real-time input electrical power and the real-time cumulative voltage decay into a two-dimensional lookup table of variable hydrogen production efficiency pre-constructed in a continuous physical memory space, and reading the corresponding electro-hydrogen conversion efficiency and target output hydrogen power through fixed-point quantization and displacement addressing includes: During the system initialization phase, based on the offline constructed binary quadratic polynomial fitting relationship, the discrete floating-point input combinations of input power and cumulative voltage decay within the physical boundary are traversed, the corresponding output hydrogen power values ​​are calculated and written one by one into the continuous address space of the static random access memory to generate the variable hydrogen production efficiency two-dimensional lookup table. During the online operation phase, the arithmetic logic unit calls conversion instructions to convert the real-time input power and the real-time cumulative voltage decay in floating-point format into fixed-point truncation into an integer index vector; The integer index vector is converted into a physical memory address pointer by superimposing a memory base address offset mask with a left shift instruction, and the target output hydrogen power stored in the corresponding address is read from the variable hydrogen production efficiency two-dimensional lookup table by a physical memory load instruction.

4. The method according to claim 1, characterized in that, In the process of evaluating the environmental state vector to generate the action tensor, when processing the product term of continuous variables caused by variable hydrogen production efficiency, the process further includes: The arithmetic logic unit reads the physical upper limit and physical lower limit of the real-time cumulative voltage decay through comparison logic gates; it calls the linear envelope operation instruction to construct a quadrilateral boundary, clamping the continuous fluctuation value of the continuous variable product term within the convex envelope boundary register determined by four linear inequalities; By using piecewise linearization logic units, the quadratic operation of input power and cumulative voltage decay is converted into a weighted sum of discrete piecewise linear functions, thus avoiding the floating-point instruction cycle consumption generated when the microprocessor performs high-order nonlinear operations.

5. The method according to claim 1, characterized in that, The deep reinforcement learning policy network is obtained through iterative training using backpropagation based on a reward function in a tensor processing unit. The construction and processing of the reward function includes: The subtraction arithmetic logic unit subtracts the comprehensive cost scalar within the system's operating cycle from the zero value to obtain the reward value for the current step. The comprehensive cost scalar includes the initial discounted cost of hardware equipment, the renewable energy curtailment penalty cost, the energy supply cut-off penalty cost, and the discounted cost of electrolytic cell voltage decay. After each state transition calculation, the comparison logic gate compares the real-time cumulative voltage decay with a preset device lifespan threshold. When the real-time cumulative voltage decay is greater than or equal to the device lifespan threshold, the interrupt controller is triggered to generate a device reset instruction, which adds the single device replacement cost to the comprehensive cost scalar and calls the memory write-back instruction to clear the cumulative voltage decay register in the first storage area.

6. The method according to claim 5, characterized in that, The process of iterative training using back gradient propagation based on the reward function further includes: The synaptic weight matrices of the policy network and the value network are stored in high-bandwidth video memory. The value network extracts the environmental state vector and the actual reward value, and calculates the time difference error through the arithmetic logic unit. The tensor processing unit calls a single instruction stream multiple data stream instruction set to perform chained differentiation operations based on the timing difference error, and updates the synaptic weight matrix of the policy network and the value network in the video memory space. When the sequential difference error is lower than the preset convergence limit within the preset evaluation window, the synaptic weight matrix is ​​frozen, and the policy network is converted into static inference code and written into non-volatile memory.

7. The method according to claim 1, characterized in that, The step of parsing the action tensor into device capacity configuration instructions and real-time power allocation parameters to control the coordinated power splitting between the proton exchange membrane electrolyzer and the energy storage device further includes: When the real-time input power exceeds the high-load operation limit threshold of the proton exchange membrane electrolyzer, the logic gate circuit triggers the action space truncation mask. The action space truncation mask intercepts the instruction to send all remaining power to the proton exchange membrane electrolyzer, and generates a power shunt instruction to drive the battery charging based on the state of charge register value of the battery energy storage device. When the value of the charge state register reaches the physical full load limit, the overload operation limit command for the proton exchange membrane electrolyzer is released, and the proton exchange membrane electrolyzer is controlled to enter the overload operation state.

8. An adaptive configuration device for electrolyzers in island microgrids based on deep reinforcement learning, characterized in that, include: The environmental sensing and register initialization unit is used to collect renewable energy output time-series data and multi-energy load demand data of the island microgrid, divide the continuous operating time into discrete time control steps, and initialize the cumulative voltage decay register corresponding to the proton exchange membrane electrolyzer to zero value. Discrete operating condition and attenuation accumulation unit is used to detect the physical operating state of the proton exchange membrane electrolyzer, map it into operating condition characterization vectors of seven discrete operating conditions, extract the corresponding voltage attenuation rate coefficients and accumulate them to calculate the real-time cumulative voltage attenuation. The fixed-point efficiency lookup table mapping unit is used to quantize and shift the real-time input power and the real-time cumulative voltage decay, and read the corresponding electro-hydrogen conversion efficiency and target output hydrogen power from the preset variable hydrogen production efficiency two-dimensional lookup table. Tensor processing and decision unit is used to input the environmental state vector containing the real-time cumulative voltage decay, the remaining capacity of the energy storage device and the power output fluctuation value of renewable energy into the deep reinforcement learning policy network, calculate and output action tensor, and parse it into device capacity configuration instructions and real-time power allocation parameters. The collaborative control execution unit is used to determine the equipment hardware installation boundary according to the equipment capacity configuration instruction, and to control the collaborative power splitting of the proton exchange membrane electrolyzer and the energy storage device using the real-time power allocation parameters.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is a non-volatile solid-state storage device that stores a computer underlying machine code instruction set. When the computer underlying machine code instruction set is loaded and executed by a microprocessor or tensor arithmetic unit, it implements the method described in any one of claims 1 to 7.

Images