Intelligent control methods, systems, media, and devices for converters based on reinforcement learning

Abstract

This invention relates to the field of intelligent control technology for converters, and provides a method, system, medium, and device for intelligent control of converters based on reinforcement learning. The method includes: training an agent using a reinforcement learning algorithm; wherein the agent takes the environmental state of a three-phase dual active bridge converter as input and control parameters as output, and sends the control parameters to a simulation environment to receive a reward; optimizing the agent's network parameters based on the reward, the reward including the effective current value, soft-switching implementation, and transmission power normalization error; using the agent as a teacher model, obtaining a student model through knowledge distillation; acquiring the environmental state of the three-phase dual active bridge converter, and obtaining the control parameters of the three-phase dual active bridge converter through the student model. This method helps improve the overall operating efficiency of the three-phase DAB converter and enhances the model's versatility and adaptability in different devices and scenarios, reducing the reliance on large amounts of data and complex manual derivation for model training.

Description

Technical Field

[0001] This invention belongs to the field of intelligent control technology for converters, and particularly relates to intelligent control methods, systems, media, and devices for converters based on reinforcement learning. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Single-phase shift (SPS) control was once a basic and simple control strategy for three-phase dual active bridge (DAB) converters. However, with the continuous improvement of performance standards in modern technology, the complexity of control has increased significantly, the number of control degrees of freedom has increased, and consequently the order of the converter model has increased accordingly. Under these circumstances, obtaining an accurate and practically applicable mathematical model has become extremely difficult. At the same time, in real-world applications, the variety of operating conditions encountered makes optimal control strategies intricate and complex. Moreover, there are also challenging situations involving nonlinear multi-constraint optimization that are difficult to derive analytical solutions from.

[0004] Existing technologies employ grid-based solving and fitting methods to address related optimization problems. Specifically, the solution area is first divided into numerous smaller regions using a grid, each corresponding to a grid point. Then, for each grid point, an intelligent algorithm is applied to calculate relevant parameter values ​​under a specific intelligent algorithm, attempting to obtain optimization information across the entire solution area. After completing the intelligent algorithm solution for the grid points, a neural network is further used for fitting. Leveraging the powerful learning and function approximation capabilities of neural networks, the input data from the grid points and the corresponding intelligent algorithm solutions are used as training samples to train the neural network. The goal is for it to learn the complex mapping relationship between input and output, thereby fitting the entire solution area to obtain a more continuous and complete optimization result model, and ultimately attempting to construct a suitable control strategy.

[0005] However, this approach has many obvious drawbacks:

[0006] (1) Regarding the training prerequisites: If a data model is used, a large amount of data needs to be collected to support the training of the neural network. This process is extremely time-consuming, and the collection, organization, and preprocessing of data also require a lot of human and material resources. If a mathematical model is used, complex derivation work is required first, which is undoubtedly very laborious. Moreover, the training of such a mathematical model involves multiple stages and requires an initial mathematical model as a foundation. Otherwise, it is difficult to carry out subsequent training work, which further increases the difficulty and complexity of mathematical model construction.

[0007] (2) Regarding training results: Since neural networks are trained based on specific system parameters, their training results are often only applicable to specific device models. Once the device model changes or the application scenario changes, their versatility will be severely insufficient, making it difficult to apply them effectively in different devices or scenarios, thus limiting their promotion and use in a wider range.

[0008] (3) The contradiction between computational efficiency and accuracy: In the scheme of using grid solving, fitting and neural network training, if the accuracy is to be improved, a higher grid density is required, which will lead to a large number of grid points needing to be solved by intelligent algorithms, which will increase the computational burden. While reducing the grid density can reduce the amount of computation, the accuracy is difficult to guarantee, and it is difficult to balance the relationship between the two. Summary of the Invention

[0009] To address the technical problems mentioned above, this invention provides a converter intelligent control method, system, medium, and device based on reinforcement learning. The reward system used in the training of the agent optimizes the effective value of the current and the transmission power while achieving soft switching, which helps to improve the overall operating efficiency of the three-phase DAB converter. Furthermore, by utilizing reinforcement learning, the agent can autonomously explore and learn in the complex operating environment of the three-phase DAB converter to achieve optimized control. The model is compressed through knowledge distillation technology, which enhances the model's versatility and adaptability in different devices and scenarios, and reduces the reliance on large amounts of data and complex manual derivation for model training.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] The first aspect of the present invention provides a reinforcement learning-based intelligent control method for converters, comprising:

[0012] A reinforcement learning algorithm is used to train the agent. The agent takes the environmental state of the three-phase dual active bridge converter as input and the control parameters as output. The agent sends the control parameters to the simulation environment to obtain a reward. The network parameters of the agent are optimized based on the reward. The reward includes the current effective value, the soft switching implementation, and the transmission power normalization error.

[0013] By using the agent as the teacher model and knowledge distillation, the student model is obtained.

[0014] The environmental state of the three-phase dual active bridge converter is obtained, and the control parameters of the three-phase dual active bridge converter are obtained through the student model.

[0015] Furthermore, the knowledge distillation step includes: sampling the input and output of the teacher model to form an expert system, and based on the expert system, transferring the knowledge of the teacher model to the student model through imitation learning.

[0016] Furthermore, the teacher model employs a neural network, while the student model employs a lightweight gradient boosting machine.

[0017] Furthermore, the control parameters include the phase shift angle, primary duty cycle, and secondary duty cycle of the three-phase dual active bridge converter.

[0018] Furthermore, the environmental conditions include a specified normalized transmission power value and voltage ratio for the three-phase dual active bridge converter.

[0019] Furthermore, the reward calculation steps include: after setting the environmental state and executing the action, sampling the current and the instantaneous phase voltage of the secondary A phase within a switching cycle, calculating the normalized value of the current effective value and the normalized value of the transmission power, and calculating the reward based on the normalized value of the current effective value and the normalized value of the transmission power, wherein the action consists of control parameters.

[0020] Furthermore, the goal of the student model is to minimize the mean square error between the student model output and the teacher output.

[0021] A second aspect of the present invention provides a reinforcement learning-based intelligent control system for a converter, comprising:

[0022] The reinforcement learning module is configured to: train an agent using a reinforcement learning algorithm; wherein the agent takes the environmental state of the three-phase dual active bridge converter as input and the control parameters as output, and sends the control parameters to the simulation environment to obtain a reward, and optimizes the network parameters of the agent based on the reward, wherein the reward includes the effective value of the current, the soft switching implementation, and the transmission power normalization error;

[0023] The knowledge distillation module is configured to: use the agent as a teacher model and obtain a student model through knowledge distillation;

[0024] The intelligent control module is configured to: acquire the environmental status of the three-phase dual active bridge converter and obtain the control parameters of the three-phase dual active bridge converter through the student model.

[0025] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the reinforcement learning-based intelligent control method for converters as described above.

[0026] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of the reinforcement learning-based intelligent control method for converters as described above.

[0027] Compared with the prior art, the beneficial effects of the present invention are:

[0028] The reward system used in the training of the intelligent agent in this invention optimizes the effective value of the current and the transmission power while achieving soft switching, which helps to improve the overall operating efficiency of the three-phase DAB converter.

[0029] This invention utilizes reinforcement learning to enable an intelligent agent to autonomously explore and learn in the complex operating environment of a three-phase DAB converter, thereby achieving optimized control and reducing the reliance on large amounts of data and complex manual derivation for model training.

[0030] This invention uses knowledge distillation technology to compress models, enhancing their versatility and adaptability across different devices and scenarios. Attached Figure Description

[0031] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0032] Figure 1 This is a flowchart of the virtual power control process according to Embodiment 1 of the present invention;

[0033] Figure 2 This is a topology diagram of the 3p-DAB converter according to Embodiment 1 of the present invention;

[0034] Figure 3 This is a waveform diagram of duty cycle modulation in Embodiment 1 of the present invention;

[0035] Figure 4 This is a block diagram of the data distillation structure of Embodiment 1 of the present invention;

[0036] Figure 5 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0038] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0039] Terminology Explanation:

[0040] LightGBM: A gradient boosting framework (a method to improve the accuracy of weak classification algorithms) that uses decision trees based on learning algorithms; it is distributed and efficient.

[0041] Example 1

[0042] This embodiment provides a converter intelligent control method based on reinforcement learning.

[0043] The intelligent control method for converters based on reinforcement learning provided in this embodiment focuses on three-phase DAB (dual active bridge) converters and aims to overcome many thorny problems encountered by existing technologies in improving operating efficiency, such as the high complexity of the controlled model, the difficulty in obtaining analytical solutions for multivariate and multi-order optimization caused by the increase in control degrees of freedom, the irreconcilable contradiction between computational burden and accuracy, and the poor model versatility.

[0044] This embodiment provides a converter intelligent control method based on reinforcement learning. Utilizing deep reinforcement learning in artificial intelligence, the method enables an intelligent agent to autonomously explore and learn in the complex operating environment of a three-phase DAB converter, achieving optimized control. Knowledge distillation technology is used for model compression, ensuring model effectiveness while achieving lightweight deployment. This effectively solves problems such as the lack of analytical solutions in multivariate and multi-order optimization caused by the complexity of the controlled model and the increased degrees of control freedom. While achieving soft switching, the method optimizes the effective current value, improves the overall operating efficiency of the three-phase DAB converter, and enhances the model's versatility and adaptability across different devices and scenarios, reducing the reliance on large amounts of data and complex manual derivation for model training.

[0045] The intelligent control method for converters based on reinforcement learning provided in this embodiment includes the following steps:

[0046] Step 1: Three-phase DAB analysis.

[0047] like Figure 2 As shown, the topology of a three-phase DAB converter consists of three key components: an inverter (primary-side bridge arm) unit, a high-frequency transformer, and a rectifier (secondary-side bridge arm) unit. In both the primary and secondary bridge arms, each arm exhibits complementary conduction characteristics between its upper and lower transistors, with the three arms on a single side operating alternately at a 120° phase difference. The high-frequency transformer, as the core component connecting the primary and secondary sides, provides electrical isolation, effectively ensuring safe isolation between circuits of different voltage levels. It also plays a crucial role in voltage conversion; through a reasonable turns ratio design, it can flexibly adapt to the needs of different input and output voltage levels, meeting diverse application scenarios.

[0048] The operating waveform of a three-phase DAB converter, such as Figure 3 As shown, Q p1 ~Qp6 For the primary-side switching transistor and its drive signal, Q s1 ~Q s6 For the secondary-side switching transistor and its drive signals, D0, D1, and D2 are control parameters, and T... s For the switching period, V pA V is the secondary phase A voltage, n is the transformer turns ratio, and V sA Let i be the primary phase A voltage. LA This is the inductor current of phase A.

[0049] Based on the inductor voltage-current relationship, the operating current of the three-phase DAB converter is as follows: = = = = ;in, v pa ( t k () represents the instantaneous phase voltage of phase A on the secondary side. v sa ( t k () represents the instantaneous phase voltage of phase A on the primary side. L The equivalent inductance of the transformer, S pa ( t k () represents the switching state of the primary side. S sa ( t k ) represents the switching state of the secondary side, Δ t k = t k+1 - t k = T k T s , T k The physical meaning is two current moments (t) k and t k+1 The proportion of the time difference to the switching cycle. V p This is the power supply voltage. V s For output voltage, k This represents the voltage ratio.

[0050] Since the average current over one cycle is 0, that is... The operating current at time t0 can be obtained as follows: ;in, S k+1For from t0 to t k+1 In this context, the equivalent duration of voltage action on inductor current; S k For from t0 to t k In the equation, the equivalent time of voltage action on inductor current is as follows: .

[0051] The magnitude of the current at each moment can then be obtained by applying the inductor current transformation law: = = ,in, i a ( t k (t) represents the current time. k The magnitude of the current, k is the time label.

[0052] The magnitude of the transmitted power can be obtained by integrating the product of the inductor voltage and current. .

[0053] It can be observed that after normalization, i.e., selecting the reference value of the current... Select the reference value for power. We can obtain the normalized t. k Current magnitude at any moment Then the normalized value of the effective value of the inductor current Normalized transmission power This proves that the normalized current magnitude and transmission power are both related to the system parameters (switching period T). s Inductance value L, input voltage V i (Irrelevant)

[0054] Step 2: Application of deep reinforcement learning algorithms.

[0055] By precisely adjusting the phase shift angle and duty cycle between the corresponding bridge arms of the primary and secondary sides, such as Figure 3 As shown, precise control of output voltage or transmitted power can be achieved. The adjustment of the phase shift angle directly affects the phase relationship of energy transmission between the primary and secondary sides, thus determining the magnitude and direction of the transmitted power; the change of duty cycle further refines the time allocation of energy transmission in each cycle. The two work together to provide a flexible and precise control means for the efficient and stable operation of the three-phase DAB converter under different operating conditions, enabling it to dynamically adjust the output characteristics according to actual needs.

[0056] The analysis in step 1 shows that the normalized current and normalized transmission power of the three-phase DAB converter are independent of the system parameters. Therefore, under the same voltage ratio k and a set of identical D0, D1, and D2 parameters, the same effect can be achieved under different systems (different inductor parameters / switching frequencies) (the normalized transmission power p is the same, and the normalized inductor current I is the same). rms The sameness is the premise of universalization.

[0057] In traditional strategies, calculations need to be performed separately for different partitions D0, D1, and D2. The result was that it was time-consuming and labor-intensive.

[0058] In this embodiment, a simulation software environment is used to collect the current of a single switching cycle, which is almost equivalent to the result of the formula derivation, providing a basis for simplifying the operation.

[0059] Step 3: Deep reinforcement learning algorithm.

[0060] (1) Definition of state space.

[0061] The environmental state space of a three-phase DAB converter includes the converter's specified transmission power normalization value. p ref The voltage ratio k and two other parameters outline the operating status of the three-phase DAB converter. Based on this status information, the intelligent agent formulates precise control strategies, laying a solid foundation for reinforcement learning.

[0062] (2) Definition of action space.

[0063] The action space defines the operational range of the agent: for phase shift angle D1, the action space is set to [0, 0.5]; for primary and secondary duty cycles D1 and D2, the action space is set to [0, 1]. At each decision moment, the agent selects an appropriate action from this action space range based on the current state, and adjusts the phase shift angle or duty cycle value according to the power error, current RMS value, and soft switching implementation to optimize converter performance and gradually explore the optimal control strategy.

[0064] (3) Reward function design.

[0065] The reward function is a core element guiding the learning of an agent. For a three-phase DAB converter, it is constructed by integrating multiple key performance indicators:

[0066] Improved operating efficiency (normalized value of RMS current I) rms Rewards are given to the agent for reducing current and achieving soft switching; reducing the effective value of the current helps reduce on-state losses, which corresponds to a positive reward; successful soft switching helps reduce switching losses, which also results in a reward.

[0067] In addition, it is necessary to limit the transmission power normalization value p from the specified transmission power normalization value. p ref The error will be rewarded according to its magnitude.

[0068] In this way, the agent learns to adjust control variables through the reward mechanism while interacting with the simulation environment, pursues the maximization of long-term cumulative rewards, and masters the optimal strategy to optimize the effective value of current and improve the operating efficiency of the converter under soft-switching conditions.

[0069] The reward function is set to Where α represents the penalty factor for power error, β represents the penalty factor for conduction loss, and γ represents the penalty factor for switching loss; set α=1, β=1; when soft switching is satisfied, γ=0, otherwise γ=1; p ref This represents the current environmental state, i.e., the normalized value of the specified transmission power, ranging from 0 to 1; p is the transmission power of the converter, but it has been normalized, i.e., its range has been changed to 0-1; k is the ratio of the output voltage to the power supply voltage after being normalized to the same side of the transformer, and its calculation method is: k = nV s / V p ;p and I rms It is the output result of the simulation environment after the actions are performed.

[0070] The general process can be summarized as follows: Set the initial environment variable S t = (k, p) ref ); Execute action a t =(D1,D2,D3); The environment is based on the action feedback current. i a ( t Instantaneous phase voltage of secondary side A phase v pa ( t (obtained by sampling), then calculate p and I. rms ; Calculate the reward R using the reward function t Reset the environment; repeat this process until convergence or the maximum number of rounds is reached.

[0071] (4) Network design of deep reinforcement learning model.

[0072] The network's input layer contains two nodes, k and p. ref The hidden layers are set to 2 layers, with the first layer containing 256 neurons and the second layer containing 128 neurons; the output layer has 3 nodes, corresponding to D1, D2 and D3 respectively.

[0073] (5) Training of deep reinforcement learning models.

[0074] Construct a deep neural network to simulate the value function or policy function of reinforcement learning; use the state vector of a three-phase DAB converter as input.

[0075] During training, the agent interacts extensively with the simulation environment: the agent first acquires state information, calculates actions (control parameters D1, D2, D3) through the neural network, and sends them to the simulation environment; after the simulation environment executes the actions, it updates the state and feeds back the new state and reward value; the agent collects data, trains the neural network using the backpropagation algorithm, optimizes the network parameters to make it approximate the optimal policy or value function, and improves the converter performance.

[0076] (6) Deep reinforcement learning algorithm process.

[0077] The entire training algorithm is executed on two platforms: the agent side and the simulation environment side. The agent side is responsible for simulating the agent's training process, while the simulation environment side is responsible for simulating environmental changes. The two interact to allow the agent to find the optimal strategy in the simulated environment.

[0078] The steps for training an agent include:

[0079] Input: Environment variables k and p ref ;

[0080] Output: Control parameters D1, D2, D3;

[0081] Initialize network parameters, experience replay pool, and total training rounds;

[0082] For each training round, execute: randomly set the environment state S. t = (k, p) ref ); for the current state S t Select Action a t = (D1, D2, D3); Output status S t With action a t Enter the simulation environment; read the reward feedback from PLECS. R t ; the sample ( S t , a t , R t Store the data in the experience replay pool R; update the network parameters.

[0083] Execution in the simulation environment:

[0084] Input: Environment variables k and p ref Control parameters D1, D2, D3;

[0085] Output: Reward R t ;

[0086] For each training round, execute: Read the state sent by the agent. S t With action a t Set the environmental state and execute actions, sampling one switching cycle T. s current i a ( t Instantaneous phase voltage of secondary side A phase v pa ( t Then perform the following calculations: , To obtain p and I rms The reward function is used to calculate the reward for the current state and action. R t ,Right now .

[0087] Step 4: Knowledge distillation technique based on imitation learning.

[0088] Currently, neural networks are primarily used in deep reinforcement learning frameworks to fit rewards because: neural networks can approximate arbitrarily complex functions, thus modeling the complex "state-action-value" relationships in reinforcement learning; neural networks can directly output continuous actions, making them suitable for continuous control tasks; and neural networks can generalize to unseen states, which is crucial for exploration in reinforcement learning. However, the current deep reinforcement learning training process requires a large amount of experience trajectory data from interaction processes, resulting in a large network architecture during training, even though such a large architecture is not needed for decision-making after actual training. Decision trees are currently mainly used for fitting data, offering speed and low computational burden. This embodiment combines the advantages of both, constructing an expert database after obtaining the final agent (neural network), and then using LightGBM to fit the expert database to achieve model compression distillation.

[0089] Knowledge distillation is an advanced model compression technique whose main purpose is to efficiently transfer knowledge from complex teacher models to simpler student models. Traditional model distillation architectures are NN (Neural Network)-NN, and are generally used for classification problems. However, they often fail to balance accuracy and lightweight design for regression problems. To address this issue, we propose a distillation technique based on imitation learning, allowing student models to learn the behavioral patterns of teacher models and reducing the difficulty of deploying regression models. Specifically, this is the NN-LightGBM distillation algorithm.

[0090] NN refers to the neural network model obtained after training the agent, i.e., the teacher neural network. LightGBM (Lightweight Gradient Boosting Machine) is a distributed gradient boosting framework based on the decision tree algorithm, which generates the student network. LightGBM cannot be directly applied to data distillation techniques. Therefore, we propose an imitation learning architecture, which first builds an expert system and then performs data imitation learning, simultaneously achieving the requirements of lightweight model and accuracy.

[0091] Teacher neural network is ,in, θ NN For teacher model parameters, x Indicates the input, y NN This represents the output of the teacher model. f NN This represents the teacher model.

[0092] Expert models generate the input space (i.e., environment variables k and p) through grid sampling. ref sample set And record the output of the teacher model: , This indicates that the teacher model is for the i-th input x. i The output, where N is the number of samples sampled, represents this dataset. D expert This is known as an expert system.

[0093] The student model is a LightGBM regressor. ,in, θ T For student model parameters, y T This represents the output of the student model. f T This represents the student model.

[0094] The goal of the student model is to minimize the mean squared error between the student model output and the teacher output. = ,in, This indicates that the student model is for the i-th input x. i The output, For lightweight constraints, among which Represents the number of leaves. λ This represents the severity of the penalty for lightweight requirements; the higher the requirement, the larger this value. In this embodiment... λ Set to 0.001.

[0095] like Figure 4As shown, specifically, the output of the teacher model NN is sampled using a grid brute-force sampling method to form an expert system; the student network is based on the LightGBM model to imitate the expert system learning the NN output. During the training process, it strives to approximate the output of the teacher model and minimize the difference between the two (using the mean squared error loss function).

[0096] LightGBM iteratively optimizes the objective function using gradient boosting decision trees (GBDT).

[0097] The steps of imitation learning include:

[0098] Input: Environment variables k and p ref That is, the input sampled in the expert system;

[0099] Output: Control parameters D1, D2, and D3, which are the outputs sampled in the expert system;

[0100] For each training epoch, perform the following: Calculate the negative gradient (pseudo residual) of the current student model: Fit a new tree Eliminating approximate pseudo residuals Update student model ( (Learning rate): .in, For the student model parameters in the (t-1)th round, η As weight.

[0101] This method transfers knowledge from neural networks (NNs) to LightGBM for regression tasks by mimicking learning frameworks. It utilizes grid sampling to build an expert system and achieves lightweighting through MSE loss and regularization. The student model significantly reduces the number of parameters and computational complexity while maintaining certain performance levels. This lightweight model can run efficiently on resource-constrained devices.

[0102] Step 5, Actual Deployment: Virtual Power Control Strategy.

[0103] like Figure 1 As shown, virtual power control is a key aspect of actual deployment, and its core lies in using a reference voltage V. ref and actual voltage V s The PI (proportional-integral) controller output is used to construct a virtual transmission power reference value p. v Virtual transmission power reference value p v The equivalent of the specified normalized transmission power value p input by the agent ref ; via V p / n Combine the power supply voltage and the actual output voltage V s The voltage is then transferred to the same side of the transformer and the actual output voltage V is used. s With the reduced power supply voltage Vp Dividing by n phases yields the voltage ratio k; this enables the three-phase DAB (3p-DAB) converter to adjust according to actual conditions and system requirements (reference voltage V). ref The agent dynamically adjusts the transmission power using the actual voltage ratio k and the virtual transmission power reference value p. v The output control quantities D1, D2, and D3 minimize the effective value of the current while achieving soft switching, thereby optimizing efficiency.

[0104] In summary, this invention integrates intelligent agents and simulation environments, and leverages deep reinforcement learning and knowledge distillation techniques to solve many challenges of three-phase DAB converters. It achieves soft switching while simultaneously optimizing the effective current value and operating efficiency, improves the model's versatility and lightweight design, and provides an innovative and efficient solution for three-phase DAB converter control. In practical deployments, it ensures stable and efficient system operation through virtual power control strategies.

[0105] Example 2

[0106] This embodiment provides a converter intelligent control system based on reinforcement learning, which specifically includes:

[0107] The reinforcement learning module is configured to: train an agent using a reinforcement learning algorithm; wherein the agent takes the environmental state of the three-phase dual active bridge converter as input and the control parameters as output, and sends the control parameters to the simulation environment to obtain a reward, and optimizes the network parameters of the agent based on the reward, wherein the reward includes the effective value of the current, the soft switching implementation, and the transmission power normalization error;

[0108] The knowledge distillation module is configured to: use the agent as a teacher model and obtain a student model through knowledge distillation;

[0109] The intelligent control module is configured to: acquire the environmental status of the three-phase dual active bridge converter and obtain the control parameters of the three-phase dual active bridge converter through the student model.

[0110] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0111] Example 3

[0112] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the reinforcement learning-based intelligent control method for converters as described in Embodiment 1 above.

[0113] Example 4

[0114] This embodiment provides a computer device, such as... Figure 5As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and send data. When the processor 1001 executes the program, it implements the steps in the reinforcement learning-based intelligent control method for converters as described in Embodiment 1 above.

[0115] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A transformer intelligent control method based on reinforcement learning, characterized in that, The method comprises the following steps: training the agent by using a reinforcement learning algorithm; wherein the agent takes an environment state of the three-phase dual active bridge converter as input, the environment state comprising a specified transmission power normalization value and a voltage ratio of the three-phase dual active bridge converter; the control parameters comprising a phase shift angle, a primary side duty cycle and a secondary side duty cycle of the three-phase dual active bridge converter; the agent sends the control parameters to a simulation environment to obtain a reward, and optimizes network parameters of the agent based on the reward, the reward comprising a current effective value condition, a soft switching implementation condition and a transmission power normalization value error; using the agent as a teacher model, the teacher model using a neural network, obtaining a student model through knowledge distillation, the student model using a light gradient boosting machine; obtaining the environment state of the three-phase dual active bridge converter, and obtaining the control parameters of the three-phase dual active bridge converter through the student model. 2.The reinforcement learning based transformer intelligent control method of claim 1, wherein, The step of knowledge distillation comprises: sampling the input and output of the teacher model to form an expert system, and migrating the knowledge of the teacher model to the student model through imitation learning based on the expert system. 3.The reinforcement learning based transformer intelligent control method of claim 1, wherein, The reward calculation step comprises: sampling the current and secondary side A-phase instantaneous phase voltage in a switching cycle after setting the environment state and performing an action, calculating the current effective value normalization value and the transmission power normalization value, and calculating the reward based on the current effective value normalization value and the transmission power normalization value, the action being composed of the control parameters. 4.The method of claim 1, wherein, The target of the student model is to minimize the mean square error between the output of the student model and the output of the teacher.

5. The transformer intelligent control system based on reinforcement learning, characterized in that, The method comprises the following steps: a reinforcement learning module configured to train the agent by using a reinforcement learning algorithm; wherein the agent takes an environment state of the three-phase dual active bridge converter as input, the environment state comprising a specified transmission power normalization value and a voltage ratio of the three-phase dual active bridge converter; the control parameters comprising a phase shift angle, a primary side duty cycle and a secondary side duty cycle of the three-phase dual active bridge converter; the agent sends the control parameters to a simulation environment to obtain a reward, and optimizes network parameters of the agent based on the reward, the reward comprising a current effective value condition, a soft switching implementation condition and a transmission power normalization value error; a knowledge distillation module configured to use the agent as a teacher model, the teacher model using a neural network, obtaining a student model through knowledge distillation, the student model using a light gradient boosting machine; an intelligent control module configured to obtain the environment state of the three-phase dual active bridge converter, and obtain the control parameters of the three-phase dual active bridge converter through the student model.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the steps in the reinforcement learning-based converter intelligent control method of any one of claims 1-4.

7. A computer device comprising a computer readable storage medium, a processor, and a computer program stored on the computer readable storage medium and executable on the processor, characterized in that, The processor executes the program to realize the steps in the reinforcement learning-based converter intelligent control method of any one of claims 1-4.

Images