A large-current constant current source control method based on reinforcement learning algorithm

Abstract

The application discloses a large-current constant current source control method based on a reinforcement learning algorithm, and belongs to the technical field of intelligent control of power electronic devices. In view of the problems of low control precision, slow dynamic response and poor load adaptability of an existing large-current constant current source, the method initializes a Q table through pulse calibration data to reduce invalid exploration of reinforcement learning; an improved reward and punishment return function and a load feedforward compensation mechanism are introduced to realize load disturbance prediction and early inhibition; an epsilon-greedy method is used to optimize PID parameters and the optimal parameters are selected through comprehensive scoring; and a double-current loop topology is constructed to realize fast response by using transformer leakage inductance to replace LC filtering. The application makes the current precision of the constant current source greater than or equal to 98%, the total harmonic distortion less than or equal to 1%, can adapt to a wide range of loads, and the stability is significantly improved, and is suitable for high-precision large-current scenes such as industrial electrolysis, new energy battery testing and high-power motor driving.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for power electronic devices, specifically to a high-current constant current source control method based on reinforcement learning algorithms. Background Technology

[0002] High-current constant current sources play a crucial role in numerous industrial, scientific research, and power electronics applications. In the fast-charging stage of electric vehicle batteries, high-current constant current sources must provide a stable and precise high current to ensure an efficient and safe charging process. In semiconductor manufacturing, high-current constant current sources are used in processes such as ion implantation and photolithography, and their current control accuracy directly affects chip performance and yield. In high-power motor drive systems, high-current constant current sources ensure stable current during motor startup and operation, achieving smooth and efficient operation.

[0003] However, existing high-current constant current source control technology has three major bottlenecks, making it difficult to meet the stringent requirements of high-end applications:

[0004] 1. Insufficient Current Control Accuracy: As output current increases to the kiloampere level, traditional control methods are affected by factors such as differences in the characteristics of power devices (such as IGBTs), uneven heat dissipation, and imbalanced current distribution among multiple devices connected in parallel, resulting in a mismatch between the actual output current and the set value. For example, in circuits with multiple power devices connected in parallel to amplify current, the on-state voltage drop and current distribution of each power device are inconsistent, leading to a large deviation in current distribution, which is difficult to meet the stringent requirements of current accuracy in applications such as high-precision electroplating and power supply for precision instruments.

[0005] 2. Poor Dynamic Response and Adaptability to Operating Conditions: Existing constant current source control methods mostly rely on simple voltage / current feedback or basic PID control. While current negative feedback circuits built using operational amplifiers can stabilize the current to some extent, they exhibit poor adaptability under complex operating conditions. Especially when the sampled voltage feedback is used as the control loop, when the load carries a large current of several thousand amps, the load heats up, causing abrupt changes in characteristics. Insufficient phase margin in the control loop leads to "loop collapse." Furthermore, the slow dynamic response speed makes it unable to promptly track changes in current demand, limiting the application of constant current sources in emerging high-speed, high-dynamic application scenarios.

[0006] 3. Weak load adaptability: The load characteristics vary significantly across different application scenarios. For example, industrial electrolysis is a low-resistive load, motor drives are inductive loads, and capacitor testing is a capacitive load. Existing constant current sources struggle to adapt flexibly. When the load abruptly changes from resistive to inductive or capacitive, the constant current source cannot maintain a stable current output, resulting in current overshoot and oscillation. Even with slow start-up via PID control, it is impossible to balance rapid response and stable output, limiting its application in complex load scenarios.

[0007] Therefore, there is an urgent need for a high-current constant current source control method that can simultaneously improve control accuracy, accelerate dynamic response, and enhance load adaptability, so as to break through the existing technical bottlenecks and support the development of high-end equipment manufacturing and new energy industries. Summary of the Invention

[0008] The purpose of this invention is to provide a high-current constant current source control method based on reinforcement learning algorithm. By optimizing the Q-Learning algorithm and innovating the dual current loop topology, the accuracy, response speed and load adaptability are improved simultaneously.

[0009] To achieve the above objectives, the present invention employs the following technical solutions.

[0010] A method for controlling a high-current constant current source based on a reinforcement learning algorithm includes the following steps:

[0011] S1. Initialize the Q-table using pulse test data before sinusoidal wave startup, establish the correlation between load characteristics and initial control parameters, and enable the Q-Learning algorithm to iterate from near-optimal solutions to reduce invalid exploration; initialize all network parameters of the Q-Learning algorithm; clear the experience replay set D used to store historical interaction data; complete data state modeling, and define the action space A and state space S;

[0012] S2. Calculate the obtained eigenvalues The reward and punishment function is improved by assigning weights to the reward decay factor to achieve adaptive adjustment. The constant current source output load fluctuation and load pre-sampling are introduced into the load feedforward compensation mechanism. The load pre-sampling is to sample the load impedance in real time through a high-frequency pulse signal, construct a load model through the Q-Learning algorithm, calculate the predicted load impedance value for the next control cycle based on the load dynamic model, calculate the feedforward compensation control quantity and superimpose it on the output of the PID controller to deal with load disturbances.

[0013] S3, Adopt ε- The greedy method selects the action A generated by the corresponding state space S in the current Q-value output to obtain the feature value corresponding to the new state S', and uses the Q-Learning algorithm to perform coarse PID tuning; through real-time interaction between the pre-trained model and the constant current source, PID coefficients that meet the current accuracy requirements are generated and stored in the Q table; the weight ratio of distortion and current accuracy is set, and the PID coefficients in the Q table are retrieved according to the weight ratio, the comprehensive score F of the candidate PID coefficients is calculated, and the PID coefficient with the largest comprehensive score F is selected as the optimal PID coefficient;

[0014] S4. Construct a dual current loop topology with an inner loop and an outer loop, and use the transformer leakage inductance for current filtering to build a fast response loop without LC filtering; the constant current source is input through three phases, and after rectification by the rectifier bridge, it provides inverter input to the IGBT full bridge. The drive signal of the IGBT full bridge is a loop signal composed of the transformer primary side current, the transformer secondary side current and the load feedforward compensation. The IGBT full bridge output is sent to the transformer primary side, and after transformer isolation, it is output from the secondary side.

[0015] S5. Determine whether the new state S' is a termination state. If so, repeat steps S2 to S4 until the desired state or the preset termination condition is reached.

[0016] Furthermore, step S1 initializes the Q-table using pulse test data prior to the sine wave start-up, including the following steps:

[0017] The pulse response accuracy P is calculated based on pulse test data. The formula for defining the pulse response accuracy P is:

[0018] ,

[0019] in, Given a current, This is the actual steady-state value of the output current. The impulse response time, The maximum allowable response time;

[0020] Calculate the initial value of the Q table based on the impulse response accuracy P and the initial PID parameters. :

[0021] ,

[0022] in, This is the initial state, the initial state Includes load impedance Ambient temperature T, This is the initial action, the initial action Corresponding to the initial PID parameter adjustment, , , These are the initial PID parameters obtained from pulse calibration. , , To meet The PID parameter weights are given by k, which is a proportional coefficient set according to the system range.

[0023] Furthermore, the definition formula for the state space S in step S1 is:

[0024] ,

[0025] in, For the outer loop output current deviation, The outer loop current deviation rate is... For inner loop current deviation, This represents the rate of change of the inner loop current deviation. Where T is the load impedance and T is the temperature of the power device;

[0026] The formula for defining action space A is:

[0027] ,

[0028] in, For the outer loop PID parameters, For the inner loop PID parameters, This is the load feedforward coefficient.

[0029] Furthermore, the formula for the improved reward / penalty function in step S2 is as follows:

[0030] ,

[0031] in, For accuracy-related bonus items, The deviation between the outer loop output current and the given current. The accuracy reward weighting coefficient; For response speed bonus items, The settling time of the system from the occurrence of a disturbance to the recovery of current stability. A weighting coefficient is assigned to reward response speed; As a stability bonus item, The standard deviation of the current deviation The stability reward weighting coefficient.

[0032] Furthermore, the formula for the adaptive adjustment of the reward decay factor in step S2 is as follows:

[0033] ,

[0034] in, The initial decay factor, The maximum attenuation factor, The difference in reward between adjacent control cycles, where t is a time variable. This refers to the sigmod function.

[0035] Furthermore, the feedforward compensation control quantity of the load feedforward compensation mechanism in step S2 The calculation formula is:

[0036] ,

[0037] in, This refers to the load impedance fluctuation. For load resistor The rate of change of load impedance. To control the period, t is a time variable. , These are the feedforward coefficients obtained by optimizing the Q-Learning algorithm.

[0038] Furthermore, in step S3, ε - Initial exploration probability of the greedy algorithm ,and ε The current output accuracy decreases over time; the number of iterations for coarse PID tuning is 1000-2000, and the current output accuracy after coarse tuning is not less than 90% of the preset threshold.

[0039] The formula for calculating the current accuracy is:

[0040] ,

[0041] in, Given a current, This is the actual steady-state value of the output current;

[0042] When selecting the optimal PID coefficients, the formula for the comprehensive scoring function F is:

[0043] ,

[0044] in, For precision weights, The distortion weight is denoted by THD, which represents total harmonic distortion.

[0045] Furthermore, , The optimal PID coefficient H after screening is:

[0046] ,

[0047] in, , , These are the core parameters of a PID controller: proportional coefficient, integral coefficient, and derivative coefficient; and H corresponds to the current accuracy. Furthermore, the distortion THD is ≤1%.

[0048] Furthermore, in step S4, the inner loop uses the primary current of the transformer. For feedback, the goal is to suppress current ripple within a preset range; the outer loop uses the transformer secondary side output current. For feedback, sampling is performed through a current transformer, with the goal of achieving a steady-state accuracy of the output current of <±0.2%;

[0049] The impedance formula for the leakage inductance of the transformer is:

[0050] ,

[0051] in, This refers to the switching frequency of the IGBT full-bridge. This is due to the inherent leakage inductance of the transformer.

[0052] The transfer function of the fast response loop without LC filtering is:

[0053] ,

[0054] in, This is the load resistance.

[0055] Furthermore, the preset termination conditions in step S5 include one or more of the following: the constant current source output current accuracy reaches ≥98% and the distortion ≤1%, the loss function value is less than a preset threshold, the load model converges, and the maximum number of iterations is reached.

[0056] The advantages of this invention are:

[0057] 1. This invention employs the Q-Learning algorithm in reinforcement learning, which can significantly improve the control accuracy and dynamic response performance of a high-current constant current source. Through a dual-current-loop topology design, the inner loop ensures the system responds quickly to load changes, while the outer loop precisely controls the output current accuracy. Combined with the optimized Q-Learning algorithm to construct a load model and feedforward compensation mechanism, load disturbances can be predicted and suppressed in advance. After secondary screening of optimized PID parameters, the current output accuracy is greatly improved, and it can quickly adjust when the load changes abruptly, effectively reducing distortion and ensuring that the output is stable and closely matches the set value.

[0058] 2. This invention uses pulse calibration data to initialize the Q-table, reducing invalid algorithm exploration and accelerating convergence speed; the improved reward and penalty function adaptively adjusts the reward decay factor, incorporates load fluctuations and pre-sampling into the assessment, enhances the adaptability to complex load conditions, enables the system to maintain stable and efficient operation over a wide load range, reduces manual debugging costs, enhances industrial application value, and enables high-current constant current sources to output quickly and accurately over a wide load range. Attached Figure Description

[0059] Figure 1 This is a flowchart of the high-current constant current source control process;

[0060] Figure 2 This is the flowchart for optimizing the Q-Learning algorithm;

[0061] Figure 3This is a flowchart of the optimized Q-Learning algorithm for debugging PID;

[0062] Figure 4 This is a circuit topology diagram of a high-current constant current source. Detailed Implementation

[0063] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0064] Example 1

[0065] Please refer to Figure 1 The flowchart shown illustrates a high-current constant current source control method based on the Q-Learning algorithm in reinforcement learning, which includes the following steps:

[0066] Step S1: Initialize the Q-table using pulse calibration data, optimize the initial state of the Q-Learning algorithm, reduce invalid exploration, initialize all network parameters, clear the set D of experience replays, complete data state modeling, and define the action space and state space.

[0067] Step S2: Process the obtained feature values The calculation is performed to improve the reward and punishment function, and the reward decay factor is weighted to achieve adaptive adjustment. The constant current source output load fluctuation and pre-sampling are introduced into the load feedforward compensation mechanism. The load model is constructed by Q-Learning algorithm to predict load changes and adjust the control strategy in advance to deal with disturbances and optimize load disturbance interference.

[0068] Step S3: Use ε - Greedy algorithm selects the corresponding state based on the current Q-value output. S Action A results in a new state. S' The corresponding feature values ​​are used to perform coarse PID tuning using the Q-Learning algorithm; the pre-trained model interacts with the constant current source in real time to generate PID coefficients that meet the current accuracy and store them in the Q table; the weight ratio of distortion and current accuracy is set, the PID coefficients in the Q table are retrieved according to the set weight values, and the optimal PID coefficients are selected to ensure that the constant current source outputs according to the set values.

[0069] Step S4: Construct an inner loop plus an outer loop dual current loop topology, use the transformer leakage inductance for current filtering, and construct a fast response loop without LC filtering; the constant current source is input through three phases, rectified by the rectifier bridge and then output to the IGBT full-bridge inverter. The drive signal is the loop signal output composed of the transformer primary and secondary currents and load feedforward compensation. The IGBT output is to the transformer primary side, and then isolated to the secondary side output through the transformer.

[0070] Step S5: The new state S' is a state variable. The model will generate action A based on this state. If S' does not reach the final required state, S2~S4 will be repeated until the preset number of iterations is reached (e.g., 1500 iterations).

[0071] In one embodiment, step S1 above includes:

[0072] Step S11: Using the pulse test data before the sine wave start, the load characteristics are directly correlated with the initial control parameters, so that the algorithm starts iterating from the "near-optimal solution" and the Q table is initialized with pulse calibration data;

[0073] Impulse response accuracy calculation model:

[0074] In this embodiment of the pulse test, the given current is 2000A, the actual steady-state value of the output current is 1996A, the pulse response time is 3.5ms, and the maximum allowable response time is 5ms. Then, the pulse response accuracy... P for:

[0075]

[0076] The calculated value is P = 99.46%. A larger value indicates a better response.

[0077] Q Table initial values The calculation formula is:

[0078]

[0079] in: This is the initial state (including a load impedance of 5mΩ and an ambient temperature of 30℃). This is the initial action (corresponding to the initial PID parameter adjustment). , , The initial PID parameters obtained from pulse calibration are 2.5, 0.6, and 0.15. , , The PID parameter weights are set to 0.5, 0.3, and 0.2, which satisfies... ; k With a proportionality constant of 15, the calculation yields... It is 21.78.

[0080] Step S12: Initialize all network parameters, including the core parameters of the Q-Learning algorithm:

[0081] Set the learning rate (controlQ Value update range), discount factor (Weighing immediate rewards versus future rewards), exploration rate (Initial exploration probability) ensures that the algorithm has both exploratory and convergent aspects in the initial stage.

[0082] Experience replay collection D Used to store historical interaction data Clear it during initialization. After each subsequent round of interaction, new data is stored according to the "first-in, first-out" principle to avoid data redundancy affecting iteration efficiency.

[0083] Based on pulse calibration data and system physical characteristics, the operating state, control actions, and feedback signals of the high-current constant source are transformed into quantifiable discrete characteristic quantities. The current deviation is graded into ±0.5A, ±1A, and ±2A, and the temperature is discretized into the ranges of 25℃~35℃ and 35℃~45℃, eliminating the limitation of continuous state on Q-Learning iteration.

[0084] state space S It includes key state variables that reflect the system's operation, as shown in the following formula:

[0085]

[0086] in: This is the deviation of the outer loop output current (i.e., the difference between the given value and the actual output current). The outer loop current deviation rate is... For inner loop current deviation, This represents the rate of change of the inner loop current deviation. For load impedance, This refers to the temperature of the power device.

[0087] Action space A The set of adjustment parameters for the dual current loop PID controller with load feedforward:

[0088]

[0089] in: For the outer loop PID parameters, For the inner loop PID parameters, This is the load feedforward coefficient.

[0090] Step S13: Initialize the state S For the first state in the current state sequence, obtain its feature vector. ;

[0091] When a high-current constant current source operates, it interacts with the load and the power grid to form a continuous state space. Pulse calibration data is used to assist in extracting state feature vectors. Because the continuous state encompasses multiple dimensions such as current deviation and load impedance, manually designing features can easily miss key information. Therefore, the state feature representation is automatically learned using the impulse response law. Specifically, by fitting and analyzing the current waveform and load parameters obtained from pulse calibration, a feature vector reflecting the system characteristics is obtained as the state representation. Simultaneously, to cope with situations exceeding the scope of conventional training data, such as sudden load changes and power grid fluctuations, a parameter perturbation strategy is adopted to make small-scale adjustments to the current setpoint and load impedance in the original state, expanding the effective dataset to improve adaptability.

[0092] Example 2

[0093] Please refer to Figure 2 The flowchart of the optimized Q-Learning algorithm is shown below. Step S2 above includes:

[0094] Step S21: Calculate The judgment obtained Q If the matrix converges, return to the previous step if it does not converge; otherwise, exit step S21 and complete the process. Q Continue with the study of matrices, proceeding to step S22;

[0095] Each output from the high-current constant current source generates a corresponding state. Based on the current state, an action is performed. The system scores this action, and a reward value is awarded upon output. Finally, upon achieving the target task, the rewards for each step are added together to obtain the final target reward. Every action and state obtained during this process is recorded on a [database / system]. Q In the table.

[0096] Table 1 Storage Q Value Q surface

[0097]

[0098] During the training of the high-current constant current source model, each step generates an action based on the current state, resulting in a series of states and actions. Q Value composition Q surface, Q The table is shown in Table 1:

[0099] in, S 1. S 2. S 3. S 4 represents the state of the constant current source at each step. A 1. A 2. A 3.A 4 represents the corresponding action generated by each state. The final judgment yields... Q If the matrix converges, check if it does not converge. If it does converge, return to the previous step. If it does converge, exit and complete. Q Learning about matrices.

[0100] Step S22: Improve the reward / penalty function;

[0101] Based on the multiple requirements of high-current constant current sources in industrial scenarios regarding current accuracy, dynamic response speed, and operational stability, a composite reward and penalty function integrating multi-dimensional indicators is designed to enable the Q-Learning algorithm to converge towards the optimal control strategy. This function quantifies system performance through three core indicators, as shown in the following formula:

[0102]

[0103] in: For accuracy-related bonus items, This represents the deviation between the outer loop output current and the given current. The larger the absolute value, the smaller the reward value for this item. (This is determined by a weighting coefficient.) Strengthen the assessment of current tracking accuracy to ensure that the output current matches the set value;

[0104] For response speed bonus items, The weighting coefficient represents the settling time of the system from the occurrence of a disturbance to the recovery of current stability. This is used to emphasize rapid response capabilities and avoid prolonged fluctuations during sudden load changes;

[0105] As a stability bonus item, The standard deviation of the current deviation reflects the stability of the output current, with a weighting factor. By amplifying the reward in a steady state using an exponential function, excessive current ripple can be suppressed.

[0106] For example, during operation, the measured current deviation was 0.3A, the disturbance recovery time was 2.5ms, and the standard deviation of the current deviation was 0.15A.

[0107] Set weight coefficients , , The values ​​are 0.5, 0.3, and 0.2 respectively.

[0108] Substituting into the formula, the total reward is calculated to be R≈-0.70.

[0109] Step S23: Assign weights to the reward decay factor to achieve adaptive adjustment;

[0110] To address the issue of traditional fixed reward decay factor γTo address the issue of insufficient adaptability under complex operating conditions, γ is dynamically adjusted through weight assignment. This allows the algorithm to prioritize long-term benefit accumulation when the load is stable, and to focus on immediate control effects when the load changes abruptly. The formula is as follows:

[0111]

[0112] in: This is the initial attenuation factor (set to 0.8 in this embodiment). The maximum attenuation factor (set to 0.95) is used to limit the adjustment range and avoid... Excessive fluctuations can affect algorithm convergence.

[0113] The reward difference between adjacent control cycles directly reflects the degree of change in system operating conditions; when the load is stable, the current deviation and settling time change little. As the sigmoid function approaches 0, its output is close to 0. near The algorithm focuses more on the cumulative rewards from long-term stable operation. When the load changes abruptly, As the value increases significantly, the output of the sigmoid function approaches 1. Down to Nearby, the algorithm prioritizes optimizing immediate control effects to quickly suppress disturbances.

[0114] Through this mechanism, the reward decay factor can automatically switch between "foresight" and "nearsight" modes based on the system's dynamics, balancing the relationship between exploration and utilization.

[0115] Step S24: Introduce the constant current source output load fluctuation and pre-sampling into the load feedforward compensation mechanism;

[0116] To suppress the interference of load impedance changes on the output current in advance, the load dynamic characteristics are obtained through pre-sampling, and a load feedforward compensation mechanism is constructed to work in conjunction with the dual current loop PID control, as shown in the following formula:

[0117]

[0118] in: It is a feedforward compensation control quantity, which is directly superimposed on the output of the PID controller to counteract the disturbance of load changes; The load impedance fluctuation is obtained in real time through high-frequency pulse pre-sampling to ensure the capture of instantaneous changes. The formula for the load impedance change rate is as follows:

[0119]

[0120] The rate of change of load impedance reflects the severity of load fluctuations. To control the cycle;

[0121] and The feedforward coefficients are obtained by the Q-Learning algorithm based on historical load-current response data; the algorithm iteratively learns the optimal compensation magnitude under different load fluctuations, making... Adapt to static load changes, Adapting to dynamic load changes, ultimately achieving pre-disturbance response. The compensation is initiated in each control cycle, which significantly reduces the peak value of the current deviation.

[0122] Example 3

[0123] Please refer to Figure 3 The optimized Q-Learning algorithm debugging PID flowchart, step S3 above includes:

[0124] Step S31: ε - The greedy algorithm, by setting an initial random exploration probability of 10% and decreasing it over time, will eventually reach the optimal result with an increasing number of training iterations. Q Value. At this point, exploration will be reduced, and... ε The value of is reduced, and the control quantity is determined according to the Q function, so that the high current constant current source can output a stable and accurate current.

[0125] Step S32: Adjust the 7 PID parameters of the loop according to the selected action, calculate the reward value in combination with the reward and penalty function, and update it through the Q-Learning iterative formula. Q The table shows that after 1000-2000 iterations, coarse adjustment is completed, bringing the current output accuracy to over 90% of the preset threshold.

[0126] Based on the current state and the selected action, through Q The value iteration update rule adjusts the PID parameters of the dual current loop, achieving initial parameter optimization and laying the foundation for subsequent fine-tuning; Action A Including the outer ring With the inner ring and There are a total of 7 PID parameters to adjust, depending on the action. A The existing PID parameters are updated using the following formula:

[0127]

[0128] After the parameters are updated, the system outputs the new status. The reward value is calculated based on the reward and punishment function designed in step S22. R (Comprehensive current accuracy, response speed, and stability). Updated according to the core iterative formula of Q-Learning. Q The table and formula are shown below:

[0129]

[0130] in, (Learning rate, control) Q (Value update range) (Discount factor, reflecting the weight of future rewards). Through 1000-2000 iterations, coarse adjustment of PID parameters is achieved, so that the current output accuracy reaches more than 90% of the preset threshold.

[0131] Step S33: The pre-trained model outputs the initial PID parameter range. After coarse tuning, the parameters are applied to the constant current source to calculate the current accuracy and compare it with the threshold. The parameter combinations that meet the criteria are associated with the state and action and stored. Q surface;

[0132] By interacting in real time with the constant current source through a pre-trained model, PID coefficient combinations that meet the basic current accuracy requirements are selected and stored. Q The table serves as a candidate strategy, reducing the storage and retrieval costs of invalid parameters. The pre-trained model is based on historical pulse calibration data (covering the range of milliohms to tens of ohms resistive loads) and 100 sets of typical operating condition samples, outputting the initial PID parameter range to avoid parameter adjustments exceeding the physically feasible domain.

[0133] The coarsely tuned PID parameters Applied to a constant current source system, the deviation between the output current and the given value is collected in real time. The formula for calculating the current accuracy is as follows:

[0134]

[0135] Set precision threshold ,when When the parameter combination is deemed valid, it is compared with the corresponding state. S ,action A Store after association Q surface:

[0136]

[0137] This step involves precision screening to ensure... Q The table retains only parameters with basic control performance, reducing the number of candidate parameters in the subsequent screening stage by more than 60% and improving algorithm efficiency.

[0138] Step S34: A high-current constant current source not only requires the output current to accurately track the set value, but also needs to ensure low distortion of the current waveform (especially under high-frequency conditions). This embodiment defines a comprehensive scoring function. F Fusion current accuracy With total harmonic distortion THD (Reflecting waveform quality):

[0139]

[0140] in, For precision weights, The distortion weights are adjusted according to the needs of industrial scenarios. , .

[0141] Retrieve Q All candidate PID coefficients in the table are applied to constant current sources, and data is collected. and THD Value and calculate F Value, final choice F The largest combination of parameters is the optimal solution:

[0142]

[0143] H To obtain the optimal parameters, this step ensures that the selected PID parameters guarantee current accuracy. It can also control THD to within 1%, ensuring that the constant current source can stably output high-quality current under complex loads.

[0144] Example 4

[0145] Please refer to Figure 4 The high-current constant current source circuit topology diagram, step S4 above includes:

[0146] Step S41: Construct a dual current loop topology with load feedforward to achieve coordinated optimization of disturbance prediction and hierarchical control;

[0147] A load feedforward compensation mechanism is introduced into the outer loop control, combined with a load model constructed using Q-Learning to predict disturbances. This, along with hierarchical control of the dual current loop, addresses the accuracy fluctuation problem caused by sudden load changes in high-current scenarios. The inner loop uses the transformer primary current... For feedback, current ripple is captured through high-frequency sampling (sampling rate = 100kHz) and processed by an inner-loop PID controller. Adjusting the IGBT switching state aims to suppress current ripple within a certain range. Within this range; its core function is to quickly offset the high-frequency disturbances caused by IGBT switching noise and grid voltage fluctuations.

[0148] The outer loop outputs current from the secondary side of the transformer. For feedback, the current is sampled by a high-precision current transformer (current transformer ratio 10000:10, accuracy <0.1%) and compared with the set current. By comparison, the outer loop current deviation is obtained. This deviation is controlled by the outer loop PID controller. After processing, an inner loop current is generated to ensure that the steady-state accuracy of the output current is <±0.2%.

[0149] Load feedforward compensation signal In the outer loop, it works in conjunction with the PID output, specifically by monitoring the load impedance in real time through a pulse calibration module. Calculate its rate of change:

[0150]

[0151] Feedforward signal generation, based on a load model constructed using the Q-Learning algorithm, outputs feedforward compensation. ,

[0152] feed forward signal With outer loop PID output Superimposed, forming the final inner loop. :

[0153] The following formula shows the basic adjustment value of the outer loop PID:

[0154]

[0155] Step S42: Construct a complete signal link from the three-phase input to the load output.

[0156] Power input and rectification stage: Three-phase 380V AC power is rectified into pulsating DC power (580V) by a three-phase uncontrolled rectifier bridge (composed of 6 diodes, 1200V / 200A), and then filtered by electrolytic capacitors (capacity ≥1000uF) to obtain a stable DC bus voltage. It provides power to the IGBT full bridge.

[0157] IGBT Full Bridge and Drive Signal Generation: The IGBT full bridge adopts an H-bridge topology (4 IGBTs, model FF450R12KT4, withstand voltage 1200V, current 450A), and its drive signal is generated by the fusion of "feedback adjustment signal + feedforward compensation signal".

[0158] Feedback adjustment signal: from the outputs of the primary side current inner loop and the secondary side current outer loop. The inner loop adjusts according to the primary side current deviation. Output the base duty cycle; the outer loop modifies the inner loop's given value. Indirectly adjust the duty cycle, ultimately forming a feedback adjustment mechanism for the duty cycle. ;

[0159] Feedforward compensation signal: Based on the load model constructed in step S24 using Q-Learning, according to the load impedance change rate. Calculate the feedforward duty cycle compensation amount (formula An optimized feedforward coefficient is used to offset the impact of load changes on the output current in advance.

[0160] The high-frequency AC input to the primary side of the IGBT full-bridge output transformer uses a high-permeability ferrite core and is designed with a turns ratio of... The AC output from the secondary side is rectified by a fast recovery diode (for DC output scenarios) or directly output (for AC scenarios) to the load, while the secondary side current... The data is fed back to the outer loop via the sensor, forming a complete closed loop.

[0161] Step S43: Replace the traditional LC filter with the inherent leakage inductance of the transformer, and suppress current ripple through the high-frequency impedance characteristics of the leakage inductance.

[0162] Transformer leakage inductance This leakage inductance is an inherent inductance resulting from incomplete winding coupling. In high-frequency switching conditions, the leakage inductance exhibits a characteristic where the impedance increases with higher frequency. The impedance formula is:

[0163]

[0164] in, For impedance, For switching frequency, The inherent inductance and leakage inductance can effectively suppress the high-frequency ripple generated by IGBT switching.

[0165] Loop model construction without LC filtering: The equivalent loop model without LC filtering is "IGBT full bridge + leakage inductance". +load The transfer function of the RL series circuit is:

[0166]

[0167] Compared to the second-order system of traditional LC filtering, this model is a first-order system, with no risk of resonance and a cutoff frequency. It dynamically adjusts according to changes in load.

[0168] The absence of LC filtering can improve system response speed, eliminate capacitor energy storage delay, and shorten current recovery time during load changes; it enhances stability, eliminates resonance risk in first-order systems, provides higher phase margin, and avoids high-frequency oscillation; by eliminating LC components, the system size is reduced, costs are lowered, and parameter drift caused by component aging is reduced.

[0169] The leakage inductance parameters are pre-stored in the system through pulse calibration (step S1), and the Q-Learning algorithm can then be used to... Dynamically adjust PID parameters to ensure a balance between filtering effect and response speed.

[0170] In step S5, the state S' If the process is in a terminated state or the loss function is too large to make the model converge, then repeat steps S2 to S4. If it is not in a terminated state, then the calculation and operation are complete. Finally, the optimal strategy is obtained by comparing the shortest scheduling time under different requirements.

[0171] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A large current constant current source control method based on a reinforcement learning algorithm, characterized in that, The method comprises the following steps: S1, initializing the Q table with pulse test data before sine wave starting, establishing the association between load characteristics and initial control parameters, making the Q-Learning algorithm start iteration from a near-optimal solution to reduce invalid exploration, initializing all network parameters of the Q-Learning algorithm, emptying the experience replay set D for storing historical interaction data, completing data state modeling, defining the action space A and the state space S; S2, calculating the obtained characteristic value , the improved reward and punishment reward function, the reward decay factor is weighted to realize adaptive adjustment, the load fluctuation of constant current source output and load pre-sampling are introduced into the load feedforward compensation mechanism, the load pre-sampling is real-time sampling of load impedance through high-frequency pulse signal, the load model is constructed through Q-Learning algorithm, the predicted value of load impedance of next control period is calculated based on the load dynamic model, the feedforward compensation control amount is calculated and superimposed to the output end of PID controller to cope with load disturbance; S3、adopting ε- The greedy method selects the action A corresponding to the state space S in the current Q value output, obtains the characteristic value corresponding to the new state S', and performs PID coarse adjustment by using a Q-Learning algorithm; the pre-trained model is interacted with a constant current source in real time, PID coefficients meeting current accuracy requirements are generated and stored in a Q table; the weight proportions of distortion degree and current accuracy are set, the PID coefficients in the Q table are recalled according to the weight proportions, the comprehensive scores F of the candidate PID coefficients are calculated, and the optimal PID coefficient is selected as the one with the maximum comprehensive score F. S4, constructing a double-current loop topology with an inner loop and an outer loop, filtering the current by using the leakage inductance of the transformer, and constructing a fast response loop without LC filtering; the constant current source is input through three-phase input, and after rectification by the rectifier bridge, the IGBT full bridge is provided with inverter input; the driving signal of the IGBT full bridge is the loop signal composed of the primary side current of the transformer, the secondary side current of the transformer and the load feedforward compensation; the IGBT full bridge outputs to the primary side of the transformer, and after isolation by the transformer, it is output from the secondary side; S5, judging whether the new state S' is a terminal state, if yes, repeating steps S2-S4 until the required state is reached or the preset ending condition is reached.

2. The method of claim 1, wherein the method is based on a reinforcement learning algorithm. The initialization of the Q table with pulse test data before sine wave starting in step S1 comprises the following steps: The pulse response accuracy P is calculated based on the pulse test data, and the definition formula of the pulse response accuracy P is: ， wherein, is the given current, is the actual output current steady state value, is the pulse response time, is the allowed maximum response time; Based on the impulse response accuracy P and the initial PID parameter calculation Q table initial value : ， Wherein, is an initial state, the initial state contains a load impedance , ambient temperature T, is an initial action, the initial action corresponding initial PID parameter adjustment amount, , , is the initial PID parameter obtained by pulse calibration, , , is the PID parameter weight that meets k is the proportional coefficient set according to the system range.

3. The method of claim 1, wherein the method is based on a reinforcement learning algorithm. The definition formula of the state space S in step S1 is: ， wherein, is the outer loop output current error, is the outer loop current error rate of change, is the inner loop current error, is the inner loop current error rate of change, is the load impedance, T is the power device temperature; The definition formula of the action space A is: ， wherein, is an outer loop PID parameter, is an inner loop PID parameter, is a load feed forward coefficient.

4. The method of claim 1, wherein the method is based on a reinforcement learning algorithm. The formula of the improved reward and punishment return function in step S2 is: ， wherein, is a precision reward term, is a deviation of the outer loop output current from a given current, is a precision reward weight coefficient; is a response speed reward term, is a regulation time of the system from the occurrence of a disturbance to the recovery of the current to stable, is a response speed reward weight coefficient; is a stability reward term, is a standard deviation of the current deviation, is a stability reward weight coefficient.

5. The method of claim 1, wherein the method is based on a reinforcement learning algorithm. The formula of the adaptive adjustment of the reward decay factor in step S2 is: ， wherein, is an initial decay factor, is a maximum decay factor, is a reward difference of a neighboring control period, t is a time variable, is a sigmod function.

6. The method of claim 1, wherein the method is based on a reinforcement learning algorithm. The feedforward compensation control quantity of the load feedforward compensation mechanism in step S2 The calculation formula is: ， wherein, is a load impedance fluctuation amount, is a load resistance is a load impedance change rate, is a control period, t is a time variable, 、 is a feedforward coefficient obtained by optimizing the Q-Learning algorithm.

7. The method of claim 1, wherein the method is based on a reinforcement learning algorithm. In step S3, the ε - initial exploration probability of the greedy method , and ε decreases over time; the number of iterations of the PID coarse adjustment is 1000-2000 times, and the current output accuracy after coarse adjustment is not less than 90% of the preset threshold. The calculation formula of the current accuracy is: ， wherein, for a given current, is the actual output current steady state value; The formula of the comprehensive score function F when screening the optimal PID coefficient is: ， wherein, is a precision weight, is a distortion weight, THD is total harmonic distortion.

8. The method of claim 7, wherein the method further comprises: , , the optimal PID coefficient H after screening is: ， Wherein, , , PID controller core parameters: proportional coefficient, integral coefficient, differential coefficient, H corresponds to the current accuracy and distortion THD≤1%.

9. The method of claim 1, wherein the method is based on a reinforcement learning algorithm. The inner ring in step S4 is the transformer primary side current For feedback, the target is to suppress the current ripple within a preset range; the outer ring is the transformer secondary side output current For feedback, sampled by current transformer, the target is to make the output current steady-state accuracy <±0.2%; The impedance formula of the leakage inductance of the transformer is: ， wherein, is the switching frequency of the IGBT full-bridge, is the inherent leakage inductance of the transformer; The transfer function of the fast response loop without LC filtering is: ， wherein is a load resistance.

10. The method of claim 1, wherein the method is based on a reinforcement learning algorithm. The preset ending condition in step S5 comprises one or more of the following: the constant current source output current accuracy reaches ≥98% and the distortion degree is ≤1%, the loss function value is less than the preset threshold, the load model converges, and the maximum iteration number is reached.

Images