Permanent magnet synchronous generator accurate model construction method based on deep reinforcement learning

Abstract

The invention provides a permanent magnet synchronous generator accurate model construction method based on deep reinforcement learning. According to the method, an electromagnetic field analytic equation, finite element simulation data and a fast convergence Actor-Critic reinforcement learning framework are firstly established, deep reinforcement learning is adopted, and reasonable state space, network structure and the like are designed, so that the limitation of a traditional method technology can be overcome, and a more efficient and more accurate PMSG parameter identification effect is achieved. And continuously feeding back the identified PMSG parameters to the estimation model until the signal difference between the output current of the reference model and the output current of the estimation model is almost zero, thereby obtaining the accurate model of the permanent magnet synchronous generator. Through interactive autonomous learning with the environment, the method has dynamic optimization capability and strong anti-interference performance, and can adapt to variable working condition scenes and realize high-precision control requirements.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of generator control and deep reinforcement learning, and is especially suitable for high-precision modeling and real-time parameter state monitoring of a permanent magnet synchronous generator (PMSG). BACKGROUND

[0002] In order to accurately control the power required by the PMSG to drive the load, accurate real-time monitoring of parameter changes is the key to realizing high-performance control. The traditional PMSG parameter identification method relies on offline testing or mathematical model calculation, but the above methods have obvious shortcomings. The offline method is measured in a static environment and cannot reflect the dynamic parameter changes of the actual PMSG in operation; online parameter identification strategies such as model reference adaptive and Kalman filtering rely on accurate motor mathematical models, are difficult to adapt to the nonlinear and coupled characteristics of the PMSG, and have poor parameter tracking ability under sudden changes. Intelligent algorithms such as genetic algorithm, particle swarm algorithm and ant colony algorithm lack online learning ability and are easy to fall into local optimum. Deep reinforcement learning can overcome the limitations of traditional methods and achieve more efficient and accurate PMSG parameter identification by designing a reasonable state space, reward function and network structure. The identified PMSG parameters are continuously fed back to the estimation model until the difference between the output current of the reference model and the output current of the estimation model is almost zero, and the accurate model of the permanent magnet synchronous generator is obtained.

[0003] The application has dynamic optimization ability and strong anti-interference ability through self-learning interaction with the environment, can adapt to variable working condition scenes and realize high-precision control. SUMMARY

[0004] The application provides a permanent magnet synchronous generator parameter identification method based on deep reinforcement learning, which uses the advantages of deep reinforcement learning to establish a dynamic PMSG parameter updating mechanism and realize accurate model construction. The system includes constructing a finite element model, analyzing the input-output relationship, designing a running condition, and obtaining a training data set; based on the Actor-Critic framework, a deep learning generator model is formed, the generator parameters are calculated using the obtained model input, and the offline parameter and online parameter error are continuously fed back to the deep learning model training process until the model accuracy reaches the required threshold.

[0005] The technical solutions of the application will be described in detail below with reference to the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 The application provides a permanent magnet synchronous generator parameter identification method based on deep reinforcement learning, which uses the advantages of deep reinforcement learning to establish a dynamic PMSG parameter updating mechanism and realize accurate model construction. The system includes constructing a finite element model, analyzing the input-output relationship, designing a running condition, and obtaining a training data set; based on the Actor-Critic framework, a deep learning generator model is formed, the generator parameters are calculated using the obtained model input, and the offline parameter and online parameter error are continuously fed back to the deep learning model training process until the model accuracy reaches the required threshold.

[0007] Figure 2An Actor model and a Critic model structure based on a neural network.

[0008] Figure 3 A PMSG parameter identification schematic diagram based on a deep reinforcement learning algorithm. DETAILED DESCRIPTION

[0009] The application discloses a permanent magnet synchronous generator accurate model construction method based on deep reinforcement learning, and in combination with the drawings, the specific implementation and implementation effects of the embodiment are carefully described.

[0010] The specific process includes the following steps: Step 1: the application discloses a permanent magnet synchronous generator accurate model construction method based on deep reinforcement learning as shown in Figure 1 The method according to claim 1. First, a permanent magnet synchronous generator finite element model is constructed, and important electrical parameters of the generator are obtained d-q Shaft inductance L d , q Permanent magnet flux linkage Ψ f Stator resistance R s and the like; under a continuous working state, the PMSG obtains signals such as phase current output signals i a,b,c , phase voltage output U a,b,c , generator angular velocity ω e , inverter bridge arm switch state S i (i=1,…,6) and the like.

[0011] Step 2: according to the circuit Kirchhoff law, the voltage equation of the PWM rectifier can be obtained as formula: (1) In the formula, u iN = S i U dc , S i is the switch state of each bridge arm. S i =1 indicates that the upper bridge arm of the first i phase is turned on, and the lower bridge arm is turned off, at this time u iN = U dc ; S i =0 indicates that the upper bridge arm of the second iWhen the upper arm of the phase is turned off and the lower arm is turned on, at this time... u iN =0. Therefore, equation (1) can be reformulated as follows: (2) Step 3: Based on the above parameter analysis, the input-output relationship characteristics need to be determined, and the back electromotive force needs to be calculated. e d and e q as follows: (3) Step 4: The voltage calculation method described in the claims of this invention is as follows: (4) Step 5: As Figure 2 As shown, the signal from the PMSG acquired in step 1 is used as the input to the neural network to obtain the corresponding output signal. .

[0012] Step 6: Design operating conditions according to actual needs. One scenario input corresponds to a set of inputs and outputs, which are continuously acquired. n Input and output under group scenarios. Under the above multi-parameter, multi-combination conditions, when the generator speed is... n At 1 r / min, current and voltage signals are obtained. u d1 , u q1 , i d1 , i q1 The generator speed is obtained by analogy. n r r / min, to obtain u dr , u qr , i dr , i qr This allows for the acquisition of a large amount of training data for use in training. Figure 3 Training process: The collected observation signals are first sent to the data pool, and then data is randomly extracted in batches from the data pool for model training.

[0013] Step 7: Before identifying generator parameters, first clear the data pool and initial observations to zero, and then initialize all parameters in E-net and IterC-Net to zero.

[0014] Step 8: When the generator is running, it is necessary to collect motor data. d-q Voltage and current are used as observed values ​​to represent the current initial values. d-q Shaft inductance L d,q Permanent magnet flux linkage Ψ f Stator resistance R s As an action value into the reinforcement learning model.

[0015] Step 9: According to the requirements of step 6, at a given speed n and current i d 、i q Next, a reinforcement learning parameter identification algorithm based on the Actor-Critic framework is designed, and the model structure is as shown in Figure 3 . Among them, R is the current reward, using the TD time difference algorithm, the input is ω e - ω ref , and the output is the TD_target current target value; S is the observed generator speed, current, voltage signal result; T is the training process, using the Actor model, the input is the inverter bridge arm switch state S i , the output is lg πζ ( S t , A t ), ζ represents the hyperparameter of the Actor network; lg πζ ( S t , A t ) represents the logarithm of the output value of the Actor network under the premise that the current observation is S t , the action is A t , the hyperparameter is ζ , and the learning strategy is π . TD_est is the value function estimate value of the time difference algorithm, TD_target is the value function target value of the time difference algorithm, and TD_error is the training error of the Critic network.

[0016] Step 10: According to the requirements of claim 3, inject excitation signals on the shaft to obtain the PMSG voltage vector equation (5) under different signals; by solving the equation under given voltage and current, the parameters are matched. d

[0017] (5)​ In the above formula, is d the shaft excitation voltage, is q the shaft excitation voltage, is d the shaft excitation current signal, is q the shaft excitation current signal, is i d = 0 when d the shaft voltage, is i q = 0 when q the shaft voltage.

[0018] Step 11: The optimal PMSG parameters can be obtained by solving the model formula (6) by observing the voltage signal; wherein the PMSG parameter identification model is set as: (6) wherein the identified d- shaft inductance, q- shaft inductance, permanent magnet flux linkage, stator resistance are as follows, .

[0019] Step 12: The output current signal of the reference model i d and i q is subtracted from the output current signal of the estimated model i d ^ and i q ^ The PMSG parameters obtained by the PMSG parameter identification of the deep reinforcement learning algorithm are fed back to the estimated model until the difference between the output current of the reference model and the output current signal of the estimated model is almost zero, and the accurate model of the permanent magnet synchronous generator described in the present application can be obtained.

[0020] The PMSG reference model equation is represented as: (7) The PMSG estimated model equation is represented as: (8) wherein the symbol “^” represents the parameters or signals that need to be identified.

Claims

1. This invention discloses a method for constructing an accurate model of a permanent magnet synchronous generator based on deep reinforcement learning, as shown in Figure 1. It includes the following steps: Step 1-1: According to the requirements of claim 1, construct a finite element model of the permanent magnet synchronous generator and obtain the important electrical parameters of the generator. dq Shaft inductor L d , q Permanent magnet chain ψ f Stator resistance R s etc.; In continuous operation, the PMSG obtains the phase current output signal from the sensor. i a,b,c Phase voltage output U a,b,c generator angular velocity ω e Inverter bridge arm switch status S i (i=1,…,6) and other signals; Steps 1-2: Based on Kirchhoff's laws for circuits, the voltage equation of the PWM rectifier can be obtained as shown in the formula. （1）， In the formula, u iN = S i U dc S i The switching state of each bridge arm. S i =1 indicates the first i When the upper arm of the phase is on and the lower arm is off, at this time... u iN = U dc ; S i =0 indicates the first i When the upper arm of the phase is turned off and the lower arm is turned on, at this time... u iN =0. Therefore, equation (1) can be reformulated as follows: （2）， Steps 1-3: Based on the above parameter analysis, the input-output relationship characteristics need to be determined. According to the method described in Steps 1-2, the back electromotive force needs to be calculated. e d and e q The calculation method is as follows: （3）， Steps 1-4: Transform Steps 1-3 into the following equations for calculating the required control voltage: （4）。 2. The method for constructing an accurate model of a permanent magnet synchronous generator based on deep reinforcement learning according to claim 1 utilizes the advantages of deep reinforcement learning to establish a dynamic PMSG parameter update mechanism, thereby achieving accurate model construction. The specific steps are as follows: Step 2-1: The structure of the Actor and Critic models based on neural networks is shown in Figure 2. Among them, The Actor-Critic framework consists of two parts: the Actor and the Critic. It employs a multi-layer backpropagation (BP) neural network structure, containing 3 input layer neurons, 5 hidden layer neurons, and 1 output layer neuron. S 1,2 , S 3,4 , S 5,6 This represents the input to the neural network. W ij , b ij (ij=11,12, 13, 14, 15) represent the weights and biases of the neural network, respectively. This is the output of the neural network; Step 2-2: Obtain the generator speed, current and voltage and other operating status outputs and continuously apply them to the reinforcement learning model. Calculate the error value to search for the optimal output value, and iteratively calculate the optimal reward value to match the motor parameter values. Steps 2-3: Under the above multi-parameter and multi-combination conditions, obtain a large amount of training set data. When the generator speed... n 1 r / min, obtained u d1 , u q1 , i d1 , i q1 The generator speed is obtained by analogy. n r r / min, to obtain u dr , u qr , i dr , i qr This is used for training in step 2-1. The collected observation signals are first sent to the data pool, and then data is randomly batch-sampled from the data pool for model training; as shown in Figure 3, where, R This is the current reward; T For the training process; S These are the observation results. The operating conditions are designed according to actual needs, with one set of inputs and outputs corresponding to one scenario input, and data is continuously acquired. n Input and output in a group scenario.

3. The method for constructing an accurate model of a permanent magnet synchronous generator based on deep reinforcement learning as described in claim 2 requires initialization and training, and the specific steps are as follows: Step 3-1: Before identifying generator parameters, first clear the data pool and initial observations to zero, and then initialize all parameters in E-net and IterC-Net to zero; Step 3-2: When the generator is running, it is necessary to collect motor data. dq Voltage and current are used as observed values ​​to represent the current initial values. dq Shaft inductor L d,q Permanent magnet chain ψ f Stator resistance R s It is fed into the reinforcement learning model as an action value; Step 3-3: According to the requirement of claim 2, at a given rotational speed n and current i d 、i q Below, a reinforcement learning parameter identification algorithm based on the Actor-Critic framework is designed, and its model structure is shown in Figure 3. Where ζ represents the hyperparameters of the Actor network; lg πζ ( S t , A t ) indicates that the current observation is S t Actions A t Hyperparameters are ζ Learning strategies are π Under the premise that, the logarithm of the Actor network output value; TD represents the time difference algorithm; TD_est represents the value function estimate of the time difference algorithm; TD_target represents the target value of the value function of the time difference algorithm; TD_error represents the training error of the Critic network.

4. The method for constructing an accurate model of a permanent magnet synchronous generator based on deep reinforcement learning as described in claim 3 requires parameter calculation and model accuracy evaluation. The specific steps are as follows: Step 4-1: According to the requirements of claim 3, in d An excitation signal is injected into the shaft to obtain the PMSG voltage vector equation (5) under different signals; parameter matching is achieved by solving the equations under given voltage and current. （5）， In the above formula (5), for d Shaft excitation voltage, for q Shaft excitation voltage, for d Shaft excitation current signal, for q Shaft excitation current signal, for i d = 0 o'clock d shaft voltage, for i q = 0 o'clock q Shaft voltage; Step 4-2: By observing the voltage signal, the optimal PMSG parameters can be obtained by solving model equation (6); where, The PMSG parameter identification model is set as follows: （6）， Among them, the identification d- Shaft inductance, q- The shaft inductance, permanent magnet flux linkage, and stator resistance are as follows. ; Step 4-3: Output current signal after passing through the reference model i d and i q , and the output current signal of the estimated model i d ^ and i q ^ Make the difference, and, as described in claim 3, identify the parameters. Feedback is fed into the estimation model until the difference between the output current of the reference model and the output current of the estimation model is almost zero, and then the accurate model of the permanent magnet synchronous generator described in this invention can be obtained. The PMSG reference model equations are expressed as follows: （7）， The PMSG estimation model equation is expressed as follows: （8）， The symbol "^" represents a parameter or signal that needs to be identified. It should be noted that although the embodiments disclosed in this invention are as described above, the content is merely for the purpose of facilitating understanding of this invention and is not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope disclosed in this invention; however, the scope of patent protection of this invention shall still be determined by the scope defined in the appended claims.

Images