Three-port converter output voltage modulation method, system, device, medium and product

Abstract

The invention discloses a three-port converter output voltage modulation method, system and device, a medium and a product. The method comprises the steps that digital signals output by a three-port converter are collected in real time; determining a current state space of a first output port and a current state space of a second output port by taking the bus voltage as a shared observation variable and combining the inductive current, the first voltage and the second voltage; determining a first phase shift angle and a second phase shift angle based on the trained agent A and the trained agent B according to the current state space of the first output port and the current state space of the second output port; and according to the first phase shift angle and the second phase shift angle, modulating the operation state of the three-port converter based on a single phase shift modulation module, and determining the output voltage of the three-port converter. According to the invention, the dynamic coordination control and global optimization capability of the multi-port converter under a complex working condition can be improved, and then stable voltage is output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic power, in particular to a three-port converter output voltage modulation method, system, device, medium and product. BACKGROUND

[0002] As the core device of multi-port energy management, three-port converters are widely used in DC power grids. Currently, traditional control strategies (such as single-phase shift control and PI control) or single-agent reinforcement learning algorithms are mainly used. However, there are the following shortcomings:

[0003] 1. High computational complexity: optimization algorithms based on mathematical models are difficult to handle multi-port coupling problems in real time.

[0004] 2. Poor dynamic response: PI control has large overshoot and long adjustment time, which cannot adapt to load mutation scenarios.

[0005] 3. Local optimal trap: single-agent algorithms are easily affected by state space coupling and are difficult to globally optimize.

[0006] Therefore, based on the problem of insufficient dynamic coordination control and global optimization ability of multi-port converters under complex working conditions, there is an urgent need for a method that can break through the above technical bottlenecks and modulate the output voltage of three-port converters. SUMMARY

[0007] The purpose of the present application is to provide a three-port converter output voltage modulation method, system, device, medium and product, which can improve the dynamic coordination control and global optimization ability of multi-port converters under complex working conditions, and then output stable voltage.

[0008] To achieve the above purpose, the present application provides the following solutions:

[0009] In a first aspect, the present application provides a three-port converter output voltage modulation method, comprising:

[0010] Real-time acquisition of digital signals output by the three-port converter; the digital signals include output voltages, inductor currents and bus voltages of each port of the three-port converter; the output voltages include first and second voltages output by first and second output ports of the three-port converter, respectively;

[0011] The bus voltage is taken as a shared observation variable, and the inductor current, the first voltage and the second voltage are combined to determine the current state space of the first output port and the current state space of the second output port;

[0012] determine the first phase shift angle and the second phase shift angle based on the trained agent A and the trained agent B according to the current state space of the first output port and the current state space of the second output port; the trained agent A and the trained agent B are determined based on the DDQN algorithm;

[0013] modulate the operating state of the three-port converter based on the single-phase modulation module according to the first phase shift angle and the second phase shift angle to determine the three-port converter output voltage.

[0014] In a second aspect, the present application provides a three-port converter output voltage modulation system, comprising:

[0015] a state perception module configured to collect digital signals output by the three-port converter in real time; the digital signals include output voltages, inductor currents and bus voltages of each port of the three-port converter; the output voltages include first and second voltages output by the first and second output ports of the three-port converter, respectively;

[0016] a task division module configured to determine a current state space of the first output port and a current state space of the second output port by taking the bus voltage as a shared observation variable and combining the inductor currents, the first voltage and the second voltage;

[0017] an action decision module configured to determine the first phase shift angle and the second phase shift angle based on the trained agent A and the trained agent B according to the current state space of the first output port and the current state space of the second output port; the trained agent A and the trained agent B are determined based on the DDQN algorithm;

[0018] a dynamic adjustment module configured to modulate the operating state of the three-port converter based on the single-phase modulation module according to the first phase shift angle and the second phase shift angle to determine the three-port converter output voltage.

[0019] In a third aspect, the present application provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the three-port converter output voltage modulation method described in the above.

[0020] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the three-port converter output voltage modulation method described in the above.

[0021] In a fifth aspect, the present application provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the three-port converter output voltage modulation method described in the above.

[0022] According to the specific embodiments provided in the present application, the present application discloses the following technical effects:

[0023] The present application first acquires the digital signals output by the three-port converter in real time; wherein the digital signals include the output voltage, inductor current and bus voltage of each port of the three-port converter; the output voltage includes the first voltage and the second voltage output by the first output port and the second output port of the three-port converter respectively. Then the bus voltage is taken as a shared observation variable, and the current state space of the first output port and the current state space of the second output port are determined in combination with the inductor current, the first voltage and the second voltage. That is, the present application divides the different voltages, inductor currents and shared observation variables of the multi-port output of the three-port converter into tasks in view of the difficulty of the optimization algorithm based on the mathematical model in the conventional technology to process the multi-port coupling problem in real time, and obtains the current state space of the first output port and the current state space of the second output port as two sub-tasks to work together. Further, the first phase shift angle and the second phase shift angle are determined based on the trained intelligent agent A and the trained intelligent agent B respectively. In this way, each intelligent agent focuses on processing a specific part, reducing the computational burden of a single intelligent agent, while based on the DDQN algorithm, reducing the computational redundancy, improving the computational efficiency and the rapid response to the dynamic changes of the system. Furthermore, it also solves the difficulty of the single intelligent agent algorithm being easily affected by the state space coupling and being difficult to globally optimize. Finally, the running state of the three-port converter is modulated based on the single phase shift modulation module according to the first phase shift angle and the second phase shift angle, and the output voltage of the three-port converter is determined. The present application not only can improve the stability of the output voltage of the three-port converter, but also can improve the dynamic coordination control and global optimization ability of the multi-port converter under complex working conditions. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creating any inventive labor.

[0025] Figure 1 A flowchart of a three-port converter output voltage modulation method provided by an embodiment of the present application;

[0026] Figure 2 A TAB converter topology provided by an embodiment of the present application;

[0027] Figure 3 A double-intelligent-agent DDQN architecture diagram provided by an embodiment of the present application;

[0028] Figure 4 A MATLAB / Simulink simulation platform architecture diagram provided for an embodiment of the present application is shown in FIG. 1.

[0029] Figure 5 A DDQN algorithm flowchart provided for an embodiment of the present application is shown in FIG. 2.

[0030] Figure 6 A voltage / current response waveform comparison diagram when a load mutates provided for an embodiment of the present application is shown in FIG. 3; wherein, Figure 6 (a) in FIG. 3 is the output voltage of port 1; Figure 6 (b) in FIG. 3 is the load current of port 1; Figure 6 (c) in FIG. 3 is the output voltage of port 2; Figure 1 (d) in FIG. 3 is the load current of port 2. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0032] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0033] As shown in FIG. 4, the present application provides a modulation method of output voltage of a three-port converter, including steps 11-13. Figure 2

[0034] Step 11: Real-time acquisition of digital signals output by the three-port converter; the digital signals include output voltages of each port of the three-port converter, inductor currents and bus voltages; the output voltages include first and second voltages respectively output by first and second output ports of the three-port converter.

[0035] Step 12: Taking the bus voltage as a shared observation variable, and combining the inductor currents, the first voltage and the second voltage, to determine the current state space of the first output port and the current state space of the second output port.

[0036] In some embodiments, step 12 specifically includes steps 21-25.

[0037] Step 21: Based on the first voltage and a first expected voltage, determining a first voltage tracking error.

[0038] ​Step 22: determining a second voltage tracking error based on the second voltage and a second expected voltage.

[0039] Step 23: the inductor current includes a first inductor current and a second inductor current output by the first output port and the second output port of the three-port converter respectively.

[0040] Step 24: taking the first voltage tracking error, the first inductor current and the bus voltage as the current state space of the first output port.

[0041] Step 25: taking the second voltage tracking error, the second inductor current and the bus voltage as the current state space of the second output port.

[0042] Step 13: determining the first phase shift angle and the second phase shift angle based on the trained agent A and the trained agent B according to the current state space of the first output port and the current state space of the second output port; the trained agent A and the trained agent B are determined based on the DDQN algorithm.

[0043] The DDQN (Double Deep Q-Network) algorithm is an improved reinforcement learning method based on the classic DQN (Deep Q-Network) algorithm, aiming to solve the problem of inaccurate value function estimation caused by the maximization bias in DQN. By separating the action selection and value evaluation processes, DDQN can more accurately estimate Q values, thereby improving learning efficiency and stability.

[0044] In some embodiments, step 13 specifically includes: determining the first phase shift angle according to the current state space of the first output port and the trained agent A; determining the second phase shift angle according to the current state space of the second output port and the trained agent B.

[0045] In some embodiments, before step 13, the method further comprises: training the agent A and the agent B to determine the trained agent A and the trained agent B; the network structures of the two agents are the same; the training process of any agent comprises: the agent is a double Q network; the double Q network comprises a main Q network and a target network; based on the main Q network, the Q value corresponding to the historical current action of the historical current state space of any output port of the three-port converter is determined; the historical current action is a historical phase shift angle; the historical current state space of the output port is determined based on the bus voltage, the output voltage of the output port and the inductor current; the historical current action is performed to obtain the historical next time state space corresponding to the output port, and the target Q value is determined based on the target network; the trained main Q network and the target network are determined based on the Q value and the target Q value; and the trained main Q network and the target network are used as the trained agent.

[0046] In some embodiments, the trained main Q network and the target network are determined based on the Q value and the target Q value, specifically comprising: updating the main Q network to minimize the difference between the Q value and the target Q value to determine the trained main Q network; and determining the trained target network based on the trained main Q network.

[0047] Step 14: based on the single-phase modulation module, the operating state of the three-port converter is modulated according to the first phase shift angle and the second phase shift angle to determine the output voltage of the three-port converter.

[0048] In practical applications, the application provides a three-port converter modulation method based on a double-agent DDQN algorithm, comprising the following steps:

[0049] Step 1: Obtain the converter state data. Sensors are closely connected to each port of the three-port converter. Specifically, the sensors are connected to two output ports and a bus of the three-port converter, respectively, to collect the first output voltage and the second output voltage (V out1 / V out2 ), the first inductor current and the second inductor current (I L1 / I L2 ), and the bus voltage (V bus ) in real time. These sensors are voltage sensors and current sensors, which can convert the electrical signals of the converter into digital signals that can be processed, providing accurate data basis for subsequent control strategies.

[0050] Step 2: Based on the data obtained in Step 1, construct the observation space of the dual-agent. The operation of the three-port converter under complex conditions requires more precise control, so two agents are adopted. Agent A and Agent B are responsible for the control of different output ports. Through precise regulation of the ports they are responsible for and in the design of the observation space, the bus voltage V bus As a shared observation variable, the dual-agent handles the coupling problem in the three-port converter circuit by adjusting the phase shift angles D of each other, so that both output voltages can accurately reach the expected value, achieving efficient and stable operation of the entire three-port converter. The observation space includes voltage tracking error (ΔV out ), historical phase shift angle, and coupling variable (V bus ), where the voltage tracking error reflects the deviation of the output voltage from the expected voltage, the historical phase shift angle can be used as a control reference, and the bus voltage as a coupling variable reflects the relationship between the ports.

[0051] Step 3: Each agent estimates the maximum expected cumulative reward Q value of all possible actions under the current observation space state through an independent deep neural network, and then makes action decisions through a greedy strategy. The main network here is part of the deep neural network, which is composed of multiple layers of neurons and is used to output the phase shift angle based on the input observation space state S. Agent A / B outputs the phase shift angle through the main network to adjust the voltage of the output port. The parameters of the main network will be updated during the training process to better adapt to different working conditions. At the same time, the target network is used to calculate the optimal Q' value, which satisfies the Bellman optimal equation:

[0052] y t =r t +γmax a' Q'(s t+1 ,a';θ')

[0053] Where Q' is the target network, the parameter is θ', max a' Q'(s t+1 ,a';θ') is the Q' value of all possible actions of the next state s t+1 taking the maximum value represents the estimate of future returns, and γ is the discount factor for balancing immediate and future rewards. The main Q network outputs the Q value of the action pair in the current state: Q(s t ,a t ;θ'), which is calculated by the loss function and optimized, and the loss function is the mean square error error

[0054]

[0055] Update the main network parameters θ through gradient descent to make the Q value of the main Q network approach the target Q value yt ; E[.] is expectation.

[0056] The target network uses a soft update method, and the target network parameters are updated smoothly at each training step:

[0057] θ'←τ·θ+(1-τ)·θ'

[0058] where τ is a coefficient, which is a small coefficient (such as 0.001) in practice, indicating the proportion of the main Q network parameters. The target network parameters change slowly, avoiding the parameter mutation that may be caused by hard update, and the training is more stable.

[0059] Step 4: Evaluate the action effect through the reward function (R A1 and R B1 are the reward functions of agent A and agent B respectively) The reward function is designed according to the control target and operation requirements of the transformer, and its purpose is to guide the agent to make action decisions that are more conducive to achieving stable output voltage. Specifically, the reward function at a certain time can be defined as:

[0060] R A1 = ω1×(-|ΔV out1 |) + ω2×ΔP A

[0061] R B1 = ω1×(-|ΔV out2 |) + ω2×ΔP B

[0062] where ΔV out1 and ΔV out2 are the voltage tracking errors of the two output ports corresponding to agent A and agent B respectively, ΔP A and ΔP B are the power changes of the three-port transformer under the control of the two agents, and ω1 and ω2 are weight coefficients. The role of the weight coefficient is to balance the influence of voltage tracking error and power change on the reward value, and by dynamically adjusting the weight coefficients ω1 and ω2, the collaborative control between the ports can be optimized. For example, when the load changes, |ΔV out1 | and |ΔV out2 | increase, resulting in a decrease in the reward value, triggering the agent to adjust and change the action to make the reward value return to a high value.

[0063] At different times, R A1 and R B1 reward functions are closely related and dynamic. Based on the real-time state of the system (voltage fluctuation, power change), R A1 , R B1is the current state mapping, the load mutation, power disturbance and other events at the historical moment will make the voltage fluctuation continue, if the system response is lagging, the voltage fluctuation at the next moment is still affected, so that the voltage term is associated with the rewards at different times; at the same time, due to the continuity of energy flow, the power adjustment at the previous moment is not completed, which will make the power term at the next moment associate the corresponding reward. In addition, R A1 , R B1 As the input of the control algorithm, the control strategy triggered by the reward at the previous moment will change the system state at the subsequent moment, and then affect the subsequent reward. The control among multiple modules also interacts with each other, so that the rewards at different times and different modules are indirectly associated through the control action. The whole system is connected through the system state inheritance and the control closed loop, and the time interlaced relationship is formed in the dynamic, which is the embodiment of the dynamic closed loop of "past disturbance continuation" and "current control result".

[0064] Step 5: Repeat steps 1 to 4 to train the double agent through the experience replay and target network delay update mechanism until the output voltage is stabilized at the expected voltage (V ref1 / V ref2 ). The experience replay mechanism stores the historical experience of the agent in the experience pool, and randomly extracts samples from the experience pool for learning during training, which can break the correlation between samples and improve the training efficiency. The target network delay update mechanism periodically copies the parameters of the main network to the target network, so that the parameter update of the target network is relatively slow, thereby ensuring the stability of the training.

[0065] Step 6: Output the final modulation strategy to realize the efficient and stable operation of the three-port converter under complex working conditions. The three ports of the three-port converter have different functions: the first port is usually connected to the power supply to provide input energy for the converter; the second port and the third port are output ports, which provide stable voltage output for different loads to meet the power demand of different loads.

[0066] The application also provides a modulation system for output voltage of a three-port converter, comprising:

[0067] A state perception module is configured to collect digital signals output by the three-port converter in real time; the digital signals include output voltages, inductor currents and bus voltages of each port of the three-port converter.

[0068] An action decision module is configured to determine a phase shift angle based on the digital signals and the expected voltage, and based on a trained double agent.

[0069] A dynamic adjustment module is configured to modulate the running state of the three-port converter based on a single phase shift modulation module according to the phase shift angle, and determine the output voltage of the three-port converter.

[0070] In some embodiments, the state perception module includes a three-port converter and a plurality of sensors;

[0071] The plurality of sensors are connected to each port of the three-port converter; the plurality of sensors are used to convert the collected electrical signals of the three-port converter into digital signals; each port includes a first output port, a second output port, and a bus port; the action decision module includes two agents; the two agents are connected to the first output port and the second output port, respectively; the dynamic adjustment module includes a single-phase shift control module; the single-phase shift control module is connected to the two agents and the three-port converter, respectively.

[0072] In practical applications, the modulation system of the output voltage of the three-port converter is verified based on the MATLAB / Simulink simulation platform, and the matlab simulation platform includes a simulation and training platform and a modulation system of the output voltage of the three-port converter.

[0073] Specifically, the state perception module includes a plurality of sensors and a TAB converter (Triple Active Bridge Converter, TAB). The TAB converter is a specific type of three-port converter, as shown in Figure 2 The TAB converter is composed of three H-bridges (i.e. Figure 4 full-bridge 1, full-bridge 2, and full-bridge 3) in 12 respectively, are insulated gate bipolar transistors (IGBTs), which are core switching devices in power electronic circuits, V1-V3 represent the direct current input or output voltage of the three circuit parts, respectively, V1 is the direct current input voltage of full-bridge 1, V2 is the direct current output voltage of full-bridge 2, and V3 is the direct current output voltage of full-bridge 3, which is used to provide power to the circuit or to represent the direct current level output by the circuit. C1, C2, and C3 represent the filtering capacitors on the direct current side, which smooth the direct current voltage, reduce voltage fluctuations, and make the input or output direct current voltage more stable. V2 is the first output port, V3 is the second output port, V1 is the direct current input bus, and the alternating current side connected to the transformer is the high-frequency alternating current bus, which realizes the bidirectional energy flow among the three ports and supports functions such as voltage boosting / boosting, energy recovery, etc. The three ports of this module have different functions. The first port is usually connected to the main power supply and is responsible for providing stable input energy to the entire three-port converter system, which is the energy source end. The second port can be connected to energy storage devices such as batteries, etc., to realize energy storage and release. When the system has excess energy, the excess energy can be stored in the energy storage device; when the system is energy deficient, the energy storage device can release energy to the system. The third port is connected to the load to provide stable power output to the load to meet the power demand of the load. This module can realize bidirectional energy flow among the three ports and support functions such as voltage boosting / boosting, energy recovery, etc.

[0074] Wherein, the dynamic decision module, namely the double-agent DDQN control module, contains two independent agents (agent A and agent B), a reward function module and a target network, agent A and agent B are responsible for the control of different output ports of the TAB transformer respectively, the double agents realize policy coordination through sharing the hidden layer network, and meanwhile, the independent Q value network is used to avoid the dimension explosion of the action space. Secondly, in the design of the observation space, the non-related variables are shielded to reduce the policy coupling from the data source. V bus As a shared observation variable, the dynamic characteristics of the common coupling path are forced to be learned by the agent, the observation noise is reduced through the feature orthogonalization processing to reduce the interference of the policy, and the optimal phase shift angle β 12 / β 13 , namely the first phase shift angle and the second phase shift angle, adaptive control is realized, and the port coupling problem is solved.

[0075] As Figure 6 shown, the simulation and training platform is a simulation platform based on MATLAB / Simulink, integrates the DDQN algorithm and the real-time feedback mechanism, supports the rapid policy optimization under the load mutation scene, provides the algorithm training environment, and verifies the effectiveness of the control strategy under the complex working conditions such as load mutation.

[0076] The beneficial effects of the present application are as follows.

[0077] 1. Efficient decoupling control: through the design of double-agent independent decision and shared coupling variable (namely shared observation variable V bus ), the strong coupling problem between the ports in the traditional method is effectively solved, and the flexibility and precision of the multi-port energy distribution are improved.

[0078] 2. Strong robustness: Under load mutation or input disturbance, the DDQN algorithm can quickly adjust the phase-shifting angle through online learning. Specifically, the agent makes action decisions based on the maximum expected cumulative reward Q value estimated by the main network and outputs the corresponding phase-shifting angle. If the output voltage does not reach the desired state, the reward value will decrease accordingly. The agent adjusts the parameters of the main network based on the reward value feedback. The target network updates the parameters at a slower frequency to provide a relatively stable learning goal for the agent. In this way, the agent can quickly adapt to load mutation or input disturbance and adjust the phase-shifting angle in time. In this application, the control effect is compared with PI control. PI control is a classical control algorithm widely used in industrial control. The proportional element can output control quantity in proportion to the current error to make the system respond quickly to the error. The integral element is used to eliminate the steady-state error of the system by integrating the error and adjusting the control quantity. Through a large number of simulations and experimental verifications, under the same load mutation or input disturbance, the output voltage overshoot of the three-port converter controlled by the DDQN algorithm tends to zero, which is reduced by 15V compared with the traditional PI control, and the dynamic response time is shortened by more than 50%, fully embodying the superiority of the control strategy in dealing with complex working conditions.

[0079] 3. Intelligent adaptation: Combined with experience replay and target network mechanism, the agent can optimize the strategy and adapt to the nonlinear and time-varying conditions in the coal mine underground, and the system efficiency is improved by 12%, providing a new paradigm for intelligent control of power electronic converters. The double-agent DDQN algorithm realizes control through the following steps:

[0080] 1. State perception: Real-time acquisition of port output voltage including first output voltage and second output voltage, inductor current and bus voltage.

[0081] 2. Action decision: Agent A / B outputs phase-shifting angle β12 / β13, and calculates the optimal Q value through the target network;

[0082] 3. Dynamic adjustment: According to the load mutation (such as resistance from 100Ω→50Ω), the reward function is updated in real time to quickly converge to a stable state, as shown in Figure 3 Port 1 is the first output port; port 2 is the second output port.

[0083] 4. Anti-interference optimization: Introduce noise parameters to improve the robustness of the neural network and suppress voltage oscillation.

[0084] As shown in Figure 5 V1 and V2 represent the first output voltage and the second output voltage of the two output ports, respectively, and the first expected voltage V ref1 and the second expected voltage Vref2 The resulting voltage deviation signal (voltage tracking error) is used as input information for agents A and B to construct the agent's observation space, enabling it to perceive the difference between the system's current output and the desired output.

[0085] Outputs of Agents A and B: Agents A and B make decisions based on the input information and output a control signal of the phase shift angle (phase shift ratio), which is then transmitted to the single-phase-shift modulation module. The single-phase-shift modulation module modulates the three-port converter based on the received phase shift angle, thereby adjusting the operating state of the three-port converter and controlling the output voltage, forming a complete closed loop of "input-decision-output-control".

[0086] Reference ​ S t The system state at time t (i.e., the state of the observation space at the current time), such as the state space composed of the current output voltage, inductor current, bus voltage, and other data in a three-port converter.

[0087] The main Q network is used to estimate S. t Q value of all possible actions Q(s) t a n This provides a basis for the agent's action decisions.

[0088] Q(s t a t ): In S t Next, execute action a t The expected cumulative reward value. For example, in three-port converter control, if a t This indicates the action of adjusting the phase shift angle, Q(s) t a t This reflects the contribution of the action to the stability of the output voltage; where a n This represents the nth action at this moment; a t This represents a specific action involving adjusting the phase shift angle at time t. n Contains multiple a t .

[0089] S t+1 : The system state at time t+1, that is, the state of the observation space after the action is performed at time t and the state is transferred to the next time.

[0090] r t+1 : The current reward obtained at time t+1 after performing an action at state t. If the output voltage approaches the desired value, r t+1 If positive, then negative; conversely, if the voltage deviates, r t+1 It is negative.

[0091] gamma: discount factor, value range [0, 1], used to balance the current reward r t+1 and the weight of the reward Q(s t+1 , a n' ) of the next moment, reflecting the trade-off between short-term and long-term benefits; a n' represents the nth action at the next moment.

[0092] Target network, i.e., Target Q Network, has the same structure as the main Q network, but the parameter update is lagging. It is used to calculate the target Q value Q'(s t+1 , a n' ), providing a stable reference for training the main Q network.

[0093] In the actual loop process, taking the output voltage control of the agent A and one output port of the three-port converter as an example, it is assumed that the state S t at time t is that the output voltage is 230V, the expected voltage is 240V, and the bus voltage is 400V. The main Q network calculates Q(s t , a1) and Q(s t , a2), where a1 represents increasing the phase shift angle (a possible current action) and a2 represents decreasing the phase shift angle (another possible current action). The agent A selects the current action a1, and after execution, enters S t+1 , at which time the output voltage is 235V, and the reward r t+1 obtained is equal to 3 (the voltage approaches the target), and it is assumed that gamma = 0.9, the main Q network calculates Q(s t+1 , a 1' ) = 10. Then the target Q value is Q'(s t+1 , a 1' ) = r t+1 + gamma Q(s t+1 , a 1' ) = 3 + 0.9 x 10 = 12, which is used to correct the parameters of Q(s t , a1) in the main Q network, and complete a loop. This process is iterated continuously, so that the agent learns the optimal phase shift angle (the first phase shift angle) adjustment strategy, and gradually stabilizes the output voltage to 240V.

[0094] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above method.

[0095] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, which is executed by a processor to implement the above method.

[0096] In one example embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the above method.

[0097] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0098] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. Any reference to a memory, database or other medium used in the embodiments provided by the present application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (ReadOnly Memory, ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive memory (ReRdM), a magnetoresistive memory (MRdM), a ferroelectric memory (FRdM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RdM) or an external cache memory, etc. As an illustration but not limitation, the RdM can be in various forms, such as a static random access memory (SRdM) or a dynamic random access memory (DRdM), etc.

[0099] The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0100] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, it should be understood that the application encompasses all possible combinations of the technical features unless such a combination is not technically possible.

[0101] The principles and implementation manners of the present application are described herein by using specific examples, and the above embodiments are only used to help understand the method of the present application and its core idea; meanwhile, according to the idea of the present application, the specific implementation manners and application scopes will be changed by those skilled in the art. In conclusion, the content of the present specification should not be understood as a limitation of the present application.

Claims

1. A method for modulating the output voltage of a three-port converter, characterized in that, include: Real-time acquisition of digital signals output from a three-port converter; the digital signals include the output voltage, inductor current, and bus voltage of each port of the three-port converter. The output voltage includes the first voltage and the second voltage output from the first output port and the second output port of the three-port converter, respectively. The bus voltage is used as a shared observation variable, and combined with the inductor current, the first voltage, and the second voltage, the current state space of the first output port and the current state space of the second output port are determined. Based on the current state space of the first output port and the current state space of the second output port, the first phase shift angle and the second phase shift angle are determined based on the trained agent A and the trained agent B; the trained agent A and the trained agent B are determined based on the DDQN algorithm. Based on the first phase shift angle and the second phase shift angle, and using the single-phase-shift modulation module, the operating state of the three-port converter is modulated to determine the output voltage of the three-port converter.

2. The modulation method for the output voltage of a three-port converter according to claim 1, characterized in that, Using the bus voltage as a shared observation variable, and combining it with the inductor current, the first voltage, and the second voltage, the current state space of the first output port and the current state space of the second output port are determined, specifically including: The first voltage tracking error is determined based on the first voltage and the first expected voltage. The second voltage tracking error is determined based on the second voltage and the second expected voltage. The inductor current includes the first inductor current and the second inductor current output from the first output port and the second output port of the three-port converter, respectively. The first voltage tracking error, the first inductor current, and the bus voltage are used as the current state space of the first output port; The second voltage tracking error, the second inductor current, and the bus voltage are used as the current state space of the second output port.

3. The modulation method for the output voltage of a three-port converter according to claim 2, characterized in that, Based on the current state space of the first output port and the current state space of the second output port, and based on the trained agent A and the trained agent B, the first phase shift angle and the second phase shift angle are determined, specifically including: The first phase shift angle is determined based on the current state space of the first output port and the trained agent A. The second phase shift angle is determined based on the current state space of the second output port and the trained agent B.

4. The modulation method for the output voltage of a three-port converter according to claim 1, characterized in that, Based on the current state space of the first output port and the current state space of the second output port, and based on the trained agent A and the trained agent B, before determining the first phase shift angle and the second phase shift angle, the process further includes: Train agents A and B to determine the trained agents A and B; the network structures of the two agents are identical. The training process for any agent specifically includes: The agent is a dual-Q network; the dual-Q network includes a master-Q network and a target network; Based on the historical current state space of any output port of the three-port converter, and using the main Q network, the Q value corresponding to the historical current action in the historical current state space is determined; the historical current action is the historical phase shift angle; the historical current state space of the output port is determined based on the bus voltage, the output voltage of the output port, and the inductor current. Execute the historical current action, obtain the historical next moment state space corresponding to the output port, and determine the target Q value based on the target network; Based on the Q-value and the target Q-value, determine the trained main Q-network and the target network; The trained master Q-network and target network are used as the trained agent.

5. The modulation method for the output voltage of a three-port converter according to claim 1, characterized in that, Based on the Q-value and the target Q-value, the trained main Q-network and target network are determined, specifically including: The main Q-network is updated with the goal of minimizing the difference between the Q-value and the target Q-value, and the trained main Q-network is determined. Based on the trained master Q network, the trained target network is determined.

6. A modulation system for the output voltage of a three-port converter, characterized in that, include: The state sensing module is used to acquire the digital signals output by the three-port converter in real time; the digital signals include the output voltage, inductor current and bus voltage of each port of the three-port converter; the output voltage includes the first voltage and the second voltage output by the first output port and the second output port of the three-port converter, respectively. The task partitioning module is used to use the bus voltage as a shared observation variable, and combine it with the inductor current, the first voltage and the second voltage to determine the current state space of the first output port and the current state space of the second output port. The action decision module is used to determine the first phase shift angle and the second phase shift angle based on the current state space of the first output port and the current state space of the second output port, and based on the trained agent A and the trained agent B; the trained agent A and the trained agent B are determined based on the DDQN algorithm. The dynamic adjustment module is used to determine the output voltage of the three-port converter by modulating the operating state of the three-port converter based on the first phase shift angle and the second phase shift angle and the single phase shift modulation module.

7. The modulation system for the output voltage of a three-port converter according to claim 6, characterized in that, The state sensing module includes a three-port converter and multiple sensors; the multiple sensors are connected to each port of the three-port converter; the multiple sensors are used to convert the collected electrical signals of the three-port converter into digital signals; each port includes a first output port, a second output port, and a bus port; The action decision module includes agent A and agent B; agent A and agent B are respectively connected to the first output port and the second output port. The dynamic adjustment module includes a single-phase shift control module; the single-phase shift control module is connected to agent A, agent B, and a three-port converter, respectively.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement a modulation method for the output voltage of a three-port converter according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the modulation method for the output voltage of the three-port converter as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the modulation method for the output voltage of the three-port converter as described in any one of claims 1-6.

Images