A double-motor safe coordination control method under FDI network attack based on reinforcement learning

Abstract

The application discloses a kind of under the FDI network attack of reinforcement learning dual-motor safety coordination control method, comprising the following steps: (1) the double-time scale mathematical model of flexible coupling dual permanent magnet synchronous motor system is established;(2) the FDI network attack model of double-time scale mathematical model is established;(3) design safety coordination controller based on LQR theory;(4) according to the FDI network attack model of step (2) established, design model-free reinforcement learning, the safety coordination controller of step (3) is iteratively solved;The method avoids the influence caused by FDI network attack to system, using reinforcement learning algorithm can not depend on model data, avoid the influence of motor parameter change to system;For improving the coordination control performance of networked dual permanent magnet synchronous motor coordination control system and guarantee network security has important significance.

Description

Technical Field

[0001] This invention belongs to the fields of dual-motor coordinated control and reinforcement learning, and specifically relates to a dual-motor secure coordinated control method under FDI network attacks based on reinforcement learning. Background Technology

[0002] Since modern times, power electronics technology has made tremendous progress in the field of control. Semiconductor devices have undergone a series of changes, such as enhanced switching capabilities, higher power handling capacity, and faster frequency response, leading to a steady improvement in circuit control performance. In the context of rapid industrial development, motor control occupies a central position in industrial automation control, and multi-motor coordinated control is an indispensable part of motor control and even industrial production automation control. The purpose of coordinated control is to enable the controlled object to achieve high-precision, high-stability, and fast-response synchronous or proportional control. In modern production, the requirements for multi-motor coordinated control are increasingly stringent, making the study of multi-motor system coordinated control of significant theoretical and practical value.

[0003] However, adopting a multi-motor coordinated drive method also brings a series of problems. In actual control systems, various uncertainties exist (such as uncertain model parameters) and external disturbances (such as applied loads). These problems directly and adversely affect the performance of the multi-motor system, which is unacceptable for production and personal safety reasons. To meet the high-precision and high-stability control requirements of multi-motor systems, an effective controller needs to be designed. Given the complexity of modern intelligent control mathematical models, the difficulty in acquiring empirical knowledge, and the lack of online learning capabilities, more advanced control algorithms are needed to optimize the multi-motor coordinated control system in order to achieve better adaptive capabilities and superior performance.

[0004] Reinforcement learning (RL), as a data-driven intelligent method independent of system models, can design optimal controllers using system input-output data. In the control field, RL is also known as adaptive dynamic programming. It iteratively searches for solutions to the Hamilton-Jacobi-Bellman equations through policy evaluation and improvement. For example, some researchers have proposed current balancing and voltage recovery control strategies based on RL algorithms for the microgrid field; and a data-based RL method has been developed to solve the optimal consensus tracking control problem of discrete-time multi-agent systems with multiple time delays.

[0005] Networked Control Systems (NCS) are a popular technology in industrial automation. They are closed-loop control systems composed of four main components: controllers, sensors, actuators, and communication networks. The biggest advantage of NCS is its effective integration of network and physical entities, allowing for resource and information sharing among components. In recent years, with the continuous development of industry and manufacturing, the control tasks of multi-motor systems have become increasingly complex, and the distribution area of ​​controlled objects has expanded. Synchronous and coordinated control of multiple motors within a limited physical area is no longer sufficient; there is a need for controlled objects to achieve synchronous and coordinated actions regardless of geographical limitations. Therefore, multi-motor systems employing NCS technology, using a shared network for signal transmission, can overcome the drawbacks of traditional point-to-point control requiring extensive wiring, achieving less wiring, lower maintenance costs, more real-time signal transmission, and faster system response. However, with the rapid development of information technology, the explosive growth of data has made data and network security issues complex and difficult. On June 1, 2017, the "Cybersecurity Law of the People's Republic of China" officially came into effect. On June 1, 2020, the "Measures for Cybersecurity Review" officially came into effect. The introduction of regulations and measures has not only become a powerful weapon for protecting network security, but also highlighted the importance of network security. Currently, with the development of internet technology, all industries rely on its support, meaning that network security systems will face greater challenges. Therefore, solving network security issues is a fundamental requirement for the development of NCS technology. Thus, ensuring the network security of multi-motor systems is a prerequisite for the networked development of multi-motor systems, possessing significant research value and far-reaching impact. Summary of the Invention

[0006] To address the aforementioned technical shortcomings, the purpose of this invention is to provide a dual-motor safe coordinated control method under FDI network attacks using reinforcement learning. This method does not rely on model parameters and can perform safe coordinated control of a dual-motor system under FDI network attacks.

[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0008] This invention provides a method for safe coordinated control of dual motors under FDI network attacks using reinforcement learning, comprising the following steps:

[0009] (1) Establish a dual-time-scale mathematical model for a flexible connection dual permanent magnet synchronous motor system;

[0010] (2) Establish an FDI network attack model targeting the dual-timescale mathematical model in step (1);

[0011] (3) Design a safety coordination controller based on LQR theory;

[0012] (4) Based on the FDI network attack model established in step (2), design a model-free reinforcement learning method and iteratively solve the security coordination controller designed in step (3).

[0013] Preferably, step (1) specifically includes:

[0014] (1-1) Establish the electrical equations of the dual permanent magnet synchronous motor in the dq coordinate axis:

[0015]

[0016]

[0017]

[0018]

[0019] Among them, i q1 i is the q-axis current of motor 1; q2 i is the q-axis current of motor 2; d1 i is the d-axis current of motor 1; d2 L is the d-axis current of motor 2; q1 L is the q-axis stator inductance of motor 1; q2 L is the q-axis stator inductance of motor 2; d1 L is the stator inductance along the d-axis of motor 1; d2 R1 is the stator inductance of motor 2 along the d-axis; R2 is the stator resistance of motor 1; R3 is the stator resistance of motor 2; φ1 is the permanent magnet flux of motor 1; φ2 is the permanent magnet flux of motor 2; u q1 u is the q-axis voltage of motor 1; q2 u is the q-axis voltage of motor 2; d1 The voltage across the d-axis of motor 1; u d2 n is the d-axis voltage of motor 2; p1 n is the number of pole pairs of motor 1; p2 Let be the number of pole pairs of motor 2;

[0020] (1-2) For a dual-motor system connected by an elastic belt, the formula used to analyze the interaction force generated by the belt being stretched is:

[0021]

[0022] Where f1 and f2 are the interaction forces generated by the belt being stretched when the speeds of the two motors are in error, k is the stiffness coefficient of the elastic belt, r1 and r2 are the radii of roller 1 and roller 2, and ω1 and ω2 are the angular velocities of motor 1 and motor 2.

[0023] (1-3) If the interaction force generated by the belt is considered as an additional load on the two motors, then the formula for the actual load on the two motors is:

[0024]

[0025] Among them, T l '1 and T l '2 is the external load for motor 1 and motor 2;

[0026] (1-4) Based on the models in steps (1-1)-(1-3), the mathematical model of the flexible connection dual permanent magnet synchronous motor system can be obtained, and the formula used is:

[0027]

[0028] (1-5) Selecting a cross-coupling control structure, the state-space form of the mathematical model of the flexible connection dual permanent magnet synchronous motor system is obtained by rearranging the formulas used:

[0029]

[0030] in:

[0031] x = [i q1 i q2 ω1 ω2 p1 p2 ω d1 ω d2 ] T

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039] (1-6) Considering the dual-time-scale characteristics of the motor system, the time-scale parameter ξ = L is extracted. q1 The system is established as a dual-timescale system, and the formula used is:

[0040]

[0041] in:

[0042]

[0043] A ξ11 =-R1-k cp1 K1,

[0044]

[0045] x = [i q1 i q2 ω1 ω2 p1 p2 ω d1 ω d2 ] T

[0046]

[0047]

[0048]

[0049] Preferably, step (2) specifically includes:

[0050] (2-1) The system state obtained by the controller can be expressed by the following formula:

[0051]

[0052] (2-2) According to step (2-1), there exists a real matrix П such that x d This can be expressed by the formula:

[0053] x d =∏x

[0054] (2-3) Based on steps (2-1) and (2-2), the flexible connection dual-motor system under FDI network attacks can be described by the following formula:

[0055]

[0056] Preferably, step (3) specifically includes:

[0057] (3-1) Based on the LQR algorithm, a performance index is proposed, and the formula used is:

[0058]

[0059] Where: Q = П T C T CП and R d >0 represents the weight matrix;

[0060] (3-2) According to step (3-1), the state feedback control input minimizes the performance index of step (3-1), and the formula used is:

[0061]

[0062] Where: u d = [v1 v2], where v1 and v2 are the voltage compensations for the two motors;

[0063] (3-3) Solve for P ξd That is, the solution to the Riccati equation, the formula used for the Riccati equation is:

[0064]

[0065] (3-4) will Substituting into the formula in step (3-3), the Riccati equation can be expressed as:

[0066]

[0067] in:

[0068]

[0069]

[0070] Preferably, step (4) specifically includes:

[0071] (4-1) Rewrite the formula for a flexible coupled dual-motor system under FDI network attacks. The formula used is:

[0072]

[0073] Among them, A k =A-BK k ;

[0074] (4-2) According to the RL algorithm, the following formula can be used to iteratively solve the equation without relying on the system model. The formula used is:

[0075]

[0076] Among them, Q d =C T C;

[0077] (4-3) Definition The equation in step (4-2) can be expressed as a formula:

[0078]

[0079] in,

[0080] (4-4) Based on the Kronecker product, the equation in step (4-3) can be expressed as follows:

[0081]

[0082] in:

[0083]

[0084]

[0085]

[0086]

[0087]

[0088]

[0089]

[0090] (4-5) Using the least squares method, solve the equation in step (4-3) using the following formula:

[0091]

[0092] Among them, t k Sampling time;

[0093] N≥n(n+1) / 2+nm

[0094] α k =[Θ k (t1),...,…,Θk(tNΘ k (t N )] T

[0095] β k =[T k (t1),…,…,Tk(tNT k (t N )] T

[0096] (4-6) According to the formula From the formula in step (4-3), we can obtain K. d =KΠ -1 K d =KΠ -1 Therefore, the state feedback control input can be obtained, and the formula used is:

[0097] u d =-K d x d=-KΠ -1 Πx=-Kx

[0098] The beneficial effects of this invention are as follows:

[0099] The method of this invention improves the tracking and synchronization performance of a flexible coupled dual-motor system with an unknown model and avoids the impact of FDI network attacks on the dual-motor system. Based on LQR optimal control theory, a safe coordination controller is designed. In addition, based on the characteristics of FDI network attacks, a new RL algorithm is proposed to iteratively solve the optimal control law using tampered data. Compared with traditional cross-coupling control, the proposed coordination control method can avoid the impact of FDI network attacks and has superior transient performance. Attached Figure Description

[0100] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0101] Figure 1 This is a schematic diagram of the flexible connection dual-motor system structure of the present invention;

[0102] Figure 2 This is a diagram of the coordinated control structure of the flexible dual-motor system of the present invention;

[0103] Figure 3 This is a schematic diagram of an FDI network attack according to the present invention;

[0104] Figure 4 This is a flowchart of the reinforcement learning algorithm proposed in this invention;

[0105] Figure 5 This is a flowchart illustrating the method of the present invention. Detailed Implementation

[0106] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0107] like Figures 1 to 5 As shown in the figure, this embodiment provides a dual-motor safety coordination control method under FDI network attacks based on reinforcement learning, which specifically includes the following steps:

[0108] Step 1: Combining Figure 1 and Figure 2 A dual-time-scale mathematical model of a flexible-connected dual permanent magnet synchronous motor system was established.

[0109] Step 1 establishes the mathematical model of the dual-motor system, specifically including:

[0110] Step 1-1: Establish the electrical equations of the dual permanent magnet synchronous motor in the dq coordinate axis:

[0111]

[0112]

[0113]

[0114]

[0115] Among them, i q1 i is the q-axis current of motor 1; q2 i is the q-axis current of motor 2; d1 i is the d-axis current of motor 1; d2 L is the d-axis current of motor 2; q1 L is the q-axis stator inductance of motor 1; q2 L is the q-axis stator inductance of motor 2; d1 L is the stator inductance along the d-axis of motor 1; d2 R1 is the stator inductance of motor 2 along the d-axis; R2 is the stator resistance of motor 1; R3 is the stator resistance of motor 2; φ1 is the permanent magnet flux of motor 1; φ2 is the permanent magnet flux of motor 2; u q1 u is the q-axis voltage of motor 1; q2 u is the q-axis voltage of motor 2; d1 The voltage across the d-axis of motor 1; u d2 n is the d-axis voltage of motor 2; p1 n is the number of pole pairs of motor 1; p2 Let be the number of pole pairs of motor 2;

[0116] Step 1-2: For a dual-motor system connected by an elastic belt, analyze the interaction forces generated by the belt being stretched. The formula used is:

[0117]

[0118] Where f1 and f2 are the interaction forces generated by the belt being stretched when the speeds of the two motors are in error, k is the stiffness coefficient of the elastic belt, r1 and r2 are the radii of roller 1 and roller 2, and ω1 and ω2 are the angular velocities of motor 1 and motor 2.

[0119] Steps 1-3: Treating the interaction force generated by the belt as an additional load on both motors, the formula for the actual load on the two motors is:

[0120]

[0121] Among them, T l '1 and T l '2 is the external load for motor 1 and motor 2;

[0122] Steps 1-4: Based on the models in steps (1-1)-(1-3), the mathematical model of the flexible-connected dual permanent magnet synchronous motor system can be obtained, and the formula used is:

[0123]

[0124] Steps 1-5: Select the cross-coupling control structure and simplify to obtain the state-space form of the mathematical model of the flexible connection dual permanent magnet synchronous motor system. The formula used is:

[0125]

[0126] in:

[0127] x = [i q1 i q2 ω1 ω2 p1 p2 ω d1 ω d2 ] T

[0128]

[0129]

[0130]

[0131]

[0132]

[0133]

[0134]

[0135] Steps 1-6: Considering the dual-time-scale characteristics of the motor system, extract the time-scale parameter ξ = L. q1 The system is established as a dual-timescale system, and the formula used is:

[0136]

[0137] in:

[0138]

[0139]

[0140]

[0141] x = [i q1 i q2 ω1 ω2 p1 p2 ω d1 ω d2 ] T

[0142]

[0143]

[0144]

[0145] Step 2: Combining Figure 3 Establish an FDI network attack model, specifically including:

[0146] Step 2-1: The sensor-controller (SC) channel is subjected to an FDI network attack. The system state obtained by the controller can be expressed by the formula:

[0147]

[0148] Step 2-2: According to Step 2-1, there exists a real matrix П such that x d Expressed as a formula:

[0149] x d =Πx

[0150] Step 2-3: Based on Step 2-1 and Step 2-2, the flexible connection dual-motor system under FDI network attacks can be described by the following formula:

[0151]

[0152] Step 3: Design a safety coordination controller based on LQR theory, specifically including:

[0153] Step 3-1: Based on the LQR algorithm, propose performance metrics using the following formula:

[0154]

[0155] Where Q = П T C T CП and R d >0 represents the weight matrix;

[0156] Step 3-2: Based on Step 3-1, the state feedback control input minimizes the performance index of Step 3-1. The formula used is:

[0157]

[0158] Where u d = [v1 v2], where v1 and v2 are the voltage compensations for the two motors;

[0159] Step 3-3: P in Step 3-2 ξd This is a solution to the Riccati equation, and the formula used in the Riccati equation is:

[0160]

[0161] Steps 3-4: Through The Riccati equation in step 3-3 can be expressed as:

[0162]

[0163] in:

[0164]

[0165]

[0166] Step 4: Based on the FDI network attack model established in Step 2, design a model-free reinforcement learning algorithm to iteratively solve the security coordination controller designed in Step 3. Figure 4 This is the flowchart of the proposed reinforcement learning algorithm, which includes:

[0167] Step 4-1: Due to the design requirements of the RL algorithm, rewrite the system in Step 2-3, using the following formula:

[0168]

[0169] Among them, A k =A-BK k ;

[0170] Step 4-2: According to the RL algorithm, the equations in Step 3-4 are solved iteratively without relying on the system model using the following formula:

[0171]

[0172] Among them, Q d =C T C;

[0173] Step 4-3: Definition The equation in step 4-2 can be expressed by the formula:

[0174]

[0175] in,

[0176] Step 4-4: Based on the Kronecker product, the equation of Step 4-3 can be expressed as follows:

[0177]

[0178] in:

[0179]

[0180]

[0181]

[0182]

[0183]

[0184]

[0185]

[0186] Step 4-5: According to the least squares method, the solution to the equation in step 4-3 can be obtained by the following formula:

[0187]

[0188] Among them, t k Sampling time,

[0189] N≥n(n+1) / 2+nm

[0190] α k =[Θ k (t1),...,Θ k (t N )] T

[0191] β k =[T k (t1),...,T k (t N )] T

[0192] Step 4-6: Based on the formulas in Step 3-2 and Step 4-3, we can obtain K. d =KΠ -1 K d =KΠ -1Therefore, the state feedback control input can be obtained, and the formula used is:

[0193] u d =-K d x d =-KΠ -1 Πx=-Kx

[0194] Based on the above, the designed algorithm utilizes the tampered system state data x d The results of the iterative solution are consistent with those obtained using the original system data, indicating that the designed algorithm can avoid the impact of FDI network attacks on the system.

[0195] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A dual-motor safety coordination control method based on reinforcement learning under FDI network attacks, characterized in that, Includes the following steps: (1) Establish a dual-time-scale mathematical model for a flexible-connected dual permanent magnet synchronous motor system; specifically including: (1-1) Establishing a dual permanent magnet synchronous motor in dq Electrical equations in coordinate system: in, i q1 For motor 1 q shaft current; i q2 Let q be the q-axis current of motor 2; i d1 For motor 1 d shaft current; i d2 For motor 2 d shaft current; L q1 Let q be the stator inductance of motor 1; L q2 For motor 2 q Shaft stator inductance; L d1 Let d be the stator inductance of motor 1; L d2 For motor 2 d Shaft stator inductance; R 1 represents the stator resistance of motor 1; R 2 represents the stator resistance of motor 2; The permanent magnet flux of motor 1; The permanent magnet flux of motor 2; u q1 Let q be the voltage across motor 1; u q2 For motor 2 q Shaft voltage; u d1 The voltage across the d-axis of motor 1; u d2 For motor 2 d Shaft voltage; n p1 Let be the number of pole pairs of motor 1; n p2 Let be the number of pole pairs of motor 2; (1-2) For a dual-motor system connected by an elastic belt, the formula used to analyze the interaction force generated by the belt being stretched is: in, and It is the interaction force generated when the belt is stretched due to the difference in speed between the two motors. This represents the stiffness coefficient of the elastic belt. and These are the radii of roller 1 and roller 2. and These are the angular velocities of motor 1 and motor 2; (1-3) If the interaction force generated by the belt is considered as an additional load on the two motors, then the formula for the actual load on the two motors is: in, and It is the external load for motor 1 and motor 2; (1-4) Based on the models in steps (1-1) to (1-3), the mathematical model of the flexible-connected dual permanent magnet synchronous motor system can be obtained, and the formula used is: (1-5) Selecting a cross-coupling control structure, the state-space form of the mathematical model of the flexible connection dual permanent magnet synchronous motor system is obtained. The formula used is: in: ， ， ， ， ， ， ， (1-6) Considering the dual time-scale characteristics of the motor system, extract the time-scale parameters. L q1 The system is established as a dual-timescale system, and the formula used is: in: ， ， ， ， ； (2) Establish an FDI network attack model targeting the dual-timescale mathematical model of step (1); (3) Design a safety coordination controller based on LQR theory; (4) Based on the FDI network attack model established in step (2), design a model-free reinforcement learning method to iteratively solve the security coordination controller designed in step (3).

2. The dual-motor safety coordination control method based on reinforcement learning under FDI network attacks as described in claim 1, characterized in that, Step (2) specifically includes: (2-1) The system state obtained by the controller can be expressed by the following formula: (2-2) According to step (2-1), there exists a real matrix П such that x d This can be expressed by the formula: (2-3) Based on steps (2-1) and (2-2), the flexible connection dual-motor system under FDI network attack can be described by the following formula: 。 3. The dual-motor safety coordination control method based on reinforcement learning under FDI network attacks as described in claim 1, characterized in that, Step (3) specifically includes: (3-1) Based on the LQR algorithm, a performance index is proposed, and the formula used is: in: Q= П T C T C П and R d >0 represents the weight matrix; (3-2) According to step (3-1), the state feedback control input minimizes the performance index of step (3-1), and the formula used is: in: u d =[ v 1 v 2 ], v 1 and v 2 It is voltage compensation for two motors; (3-3) Solution P ξd That is, the solution to the Riccati equation, the formula used for the Riccati equation is: (3-4) will Substituting into the formula in step (3-3), the Riccati equation can be expressed as: in: 。 4. The dual-motor safety coordination control method based on reinforcement learning under FDI network attacks as described in claim 1, characterized in that, Step (4) specifically includes: (4-1) Rewrite the formula for a flexible dual-motor system under FDI network attacks. The formula used is: in, ; (4-2) According to the RL algorithm, the following formula can be used to iteratively solve the equation without relying on the system model. The formula used is: in, Q d =C T C ; (4-3) Definition The equation in step (4-2) can be expressed as follows: in, ; (4-4) Based on the Kronecker product, the equation in step (4-3) can be expressed as: in: (4-5) Using the least squares method, solve the equation in step (4-3) using the following formula: in, t k Sampling time; (4-6) According to the formula From the formula in step (4-3), we can obtain , Therefore, the state feedback control input can be obtained, and the formula used is: 。

Images