Controller design method and system based on improved structured neural network

Abstract

The application relates to a controller design method and system based on an improved structured neural network. The design method comprises the following steps: sampling the initial state of a controlled object in a data collection module and storing the sampling data in a database; extracting the sampling data in the database and using the sampling data to perform training in a data training module using PyTorch and an Adam optimizer; extracting the trained data in the data training module as the input of a secondary deep neural network controller; taking the output of the secondary deep neural network controller as the weight of a primary deep neural network controller and performing optimization of the controller in a controller optimization module. The design system comprises a data collection module, a data training module, a secondary deep neural network controller module, a primary deep neural network controller module and a controller optimization module. The controller based on the improved structured neural network improves the stability and adaptability of the controller.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic control, and particularly relates to a controller design method and system based on an improved structured neural network. BACKGROUND

[0002] In the process of engineering practice, engineering technicians often need to control complex nonlinear systems such as robots and aircraft. For the control of nonlinear systems, there are two categories, which are traditional nonlinear control methods and neural network-based control methods. Traditional nonlinear control methods, such as sliding mode control and robust adaptive control, are not suitable for situations where there are state and input constraints. Neural network-based control, such as deep neural network (DNN), has strong approximation ability, but due to its black box nature, it is difficult to provide stability guarantees. In order to guarantee stability, the structure of the dynamics is currently used and control theory is used to design a structured DNN controller, and then the structured DNN controller is trained to achieve optimal performance, but the optimal controller of the nonlinear control problem usually depends on the initial state, and when the initial state changes, it needs to be retrained, which increases the computational complexity; there are also structured DNN controllers designed for multiple initial states, but they cannot guarantee optimal performance at a specific initial state. SUMMARY

[0003] In view of some or all of the problems in the prior art, the present application provides a controller design method based on an improved structured neural network, which comprises the following steps:

[0004] The initial state of the controlled object is sampled in the data collection module and stored in the database as sampling data;

[0005] The sampling data in the database is extracted and used to perform training in the data training module using PyTorch and Adam optimizer;

[0006] The trained data in the data training module is extracted as the input of the secondary deep neural network controller; and

[0007] The output of the secondary deep neural network controller is used as the weight of the primary deep neural network controller, and the optimization of the controller is performed in the controller optimization module.

[0008] Further, the sampling of the initial state of the controlled object in the data collection module comprises:

[0009] For the controlled object, the sampling method involves uniform sampling or random sampling in a certain initial state interval to obtain the required initial state x(0).

[0010] Further, the training performed in the data training module using PyTorch and Adam optimizer comprises:

[0011] For the controlled object, the neural network is set with epoch and learning rate, the gradient descent is used to solve the optimization problem of epoch, and the training is performed by itself.

[0012] Further, the data trained in the data training module is extracted as the input of the secondary deep neural network controller.

[0013] The data trained in different initial states is used as the input of the secondary deep neural network controller, and the secondary deep neural network controller learns the initial state in the training process, thereby adapting to different initial states.

[0014] Further, the output of the secondary deep neural network controller is used as the weight of the primary deep neural network controller.

[0015] The Lyapunov function of the primary deep neural network controller is constructed,

[0016] The primary deep neural network controller is set as u(t) = φ(x(t); θ),

[0017] Where x(t) represents the state quantity at t time, θ is provided by the output of the secondary deep neural network controller W(x(0); γ), x(0) represents the initial state, and γ represents the parameter of the secondary deep neural network controller.

[0018] Further, the optimization of the controller performed in the controller optimization module comprises:

[0019] The dynamic model is constructed as Where x(t) represents the state quantity at t time, and u(t) represents the primary deep neural network controller;

[0020] The optimization equation is

[0021]

[0022]

[0023] u i (t) = φ(x i (t); θ), t ∈ [0, T]

[0024] θ = W(x 0,i ; γ),

[0025] initial state x 0,i , i = 1,..., N.

[0026] Where min represents the minimum value of the function, N represents the number of initial states, T represents the training time domain, and l t (x i (t), u i (t) represents the loss function, x i (t) represents the state variable, u i (t) represents the control input, θ represents the parameters of the main neural network controller, and x 0，i γ represents the initial state, and γ represents the parameters of the sub-neural network controller.

[0027] Furthermore, the controller design method also includes:

[0028] The initial state of the controlled object is collected in a loop, and the optimization equation is solved in a loop to achieve the optimal controller design.

[0029] This invention also provides a controller design system based on an improved structured neural network, the system comprising the following modules:

[0030] The data collection module is configured to sample the initial state of the controlled object and store the sampled data in the database;

[0031] The data training module is configured to extract sampled data from the database and perform training using PyTorch and the Adam optimizer.

[0032] The sub-deep neural network controller module is configured to include a sub-deep neural network controller.

[0033] The main deep neural network controller module is configured to include a main deep neural network controller; and

[0034] The controller optimization module is configured to perform controller optimization within it.

[0035] The present invention also provides a computer system, comprising:

[0036] A processor, configured to execute machine-readable instructions;

[0037] Graphics cards with artificial intelligence chips are configured to train neural networks; and

[0038] A memory configured to store machine-readable instructions that, when executed by a processor and / or graphics card, perform the steps of the design method described herein.

[0039] The present invention also provides a computer-readable storage medium having machine-readable instructions stored thereon, which, when executed by a processor, perform the steps of the design method described herein.

[0040] Compared with the prior art, the application has the beneficial effects that the sub-depth neural network controller takes the initial state of the controlled object as input, can be optimized, and enhances the adaptability of the sub-depth neural network controller to the initial state change. The stability of the main depth neural network is ensured by constructing a Lyapunov function, which is the control premise of the improved structured neural network controller. By using this method, the main neural network can ensure stability, and the sub-neural network can improve the performance of the controller by adapting to the changing initial state, thereby improving the response speed of the sub-neural network. BRIEF DESCRIPTION OF DRAWINGS

[0041] To further clarify the above and other advantages and features of the embodiments of the present application, a more particular description of embodiments of the application will be rendered by reference to specific embodiments thereof which are illustrated in the drawings. It is appreciated that these drawings depict only typical embodiments of the application and are therefore not to be considered limiting of its scope. The same or corresponding elements in the drawings are denoted by the same or similar reference signs.

[0042] Figure 1 a flowchart of a design method of an improved structured neural network-based controller for an embodiment of the present application;

[0043] Figure 2 a principle diagram of an improved structured neural network-based controller for an embodiment of the present application;

[0044] Figure 3 a diagram of the difference between the control parameters of different controllers and the optimal control parameters for an embodiment of the present application;

[0045] Figure 4 a diagram of the difference between the cost functions of different controllers and the optimal cost for an embodiment of the present application;

[0046] Figure 5 a diagram of the difference between the costs of different controllers based on a Lagrange system for an embodiment of the present application; and

[0047] Figure 6 a design system diagram of an improved structured neural network-based controller for an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, reference is made to the accompanying drawings which form a part hereof, and which are shown by way of illustration of the embodiments of the present application. However, it is understood that embodiments of the present application can be practiced without many of the specific details that are set forth in the following description. In other instances, well-known structures and operational aspects of the devices or methodologies have not been shown or described in order to avoid obscuring the application.

[0049] In this specification, reference to "one embodiment" or "the embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0050] It is to be understood that the embodiments of the present application are to be performed in a particular order, however, this is only for the purpose of illustrating the specific embodiments, and does not limit the order of the steps. On the contrary, the order of the steps can be adjusted according to the actual needs of adjustment in different embodiments of the present application.

[0051] In the present application, each module of the system according to the present application can be implemented using software, hardware, firmware or a combination thereof. When a module is implemented using software, the function of the module can be implemented by a computer program flow, for example, the module can be implemented by a code segment (such as a code segment in C, C++, etc.) stored in a storage device (such as a hard disk, a memory, etc.), wherein when the code segment is executed by a processor, the corresponding function of the module can be implemented. When a module is implemented using hardware, the function of the module can be implemented by setting a corresponding hardware structure, for example, the function of the module can be implemented by hardware programming on a programmable device such as a field programmable logic gate array (FPGA), or the function of the module can be implemented by designing an application specific integrated circuit (ASIC) including a plurality of transistors, resistors and capacitors, etc. electronic devices. When a module is implemented using firmware, the function of the module can be written in the form of program code in the read-only memory such as EPROM or EEPROM of the device, and when the program code is executed by the processor, the corresponding function of the module can be implemented. In addition, some functions of the module can need to be implemented by a separate hardware or cooperated with the hardware, for example, the detection function is implemented by a corresponding sensor (such as a proximity sensor, an acceleration sensor, a gyroscope, etc.), the signal transmitting function is implemented by a corresponding communication device (such as a Bluetooth device, an infrared communication device, a baseband communication device, a Wi-Fi communication device, etc.), the output function is implemented by a corresponding output device (such as a display, a speaker, etc.), and so on.

[0052] The existing control strategy of a deep neural network (DNN) only has one controller to control a nonlinear system, while the controller based on an improved structured neural network, namely, an NNtoNN controller, of the application includes two controllers, a primary DNN controller and a secondary DNN controller.

[0053] The technical solutions in the embodiments of the application will be clearly and completely described in the following with reference to the drawings in the embodiments of the application.

[0054] Figure 1 A flowchart of a controller design method based on an improved structured neural network of one embodiment of the application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the controller design method based on the improved structured neural network includes the following steps:

[0055] First, the initial state of the controlled object is sampled in the data collection module and stored in the database as sampling data. For the controlled object, the sampling method involves uniform sampling or random sampling in a certain initial state interval to obtain the required initial state x(0).

[0056] Next, the sampling data in the database is extracted and used to perform training in the data training module using PyTorch and the Adam optimizer. For the controlled object, the neural network is set with epoch and learning rate, and the gradient descent is used to solve the optimization problem of epoch for self-training.

[0057] Next, the trained data in the data training module is extracted as the input of the secondary deep neural network controller. The trained data of different initial states is used as the input of the secondary deep neural network controller, which continuously learns the initial state in the training process to adapt to different initial states.

[0058] Finally, the output of the secondary deep neural network controller is used as the weight of the primary deep neural network controller, and the optimization of the controller is performed in the controller optimization module. Figure 2 A principle diagram of a controller based on an improved structured neural network of one embodiment of the application is shown in FIG. 2. Figure 2 As shown in FIG. 2, a Lyapunov function of the primary deep neural network controller is constructed, and the primary deep neural network controller is set as u(t) = φ(x(t); θ), where x(t) represents the state quantity at time t, θ is provided by the output of the secondary deep neural network controller W(x(0); γ), x(0) represents the initial state, and γ represents the parameter of the secondary deep neural network controller.

[0059] The dynamic model is constructed as Wherein, x(t) represents state quantity at t moment, u(t) represents main deep neural network controller;

[0060] The optimization equation is,

[0061]

[0062]

[0063] u i (t)=φ(x i (t);θ),t∈∈[0,T]

[0064] θ=W(x 0,i ;γ),

[0065] initial state x 0,i , i=1,...,N.

[0066] Wherein, min represents minimum value of function, N represents quantity of initial state, T represents training time domain, l t (x i (t), u i (t)) represents loss function, x i (t) represents state quantity, u i (t) represents control quantity, θ represents parameter of main neural network controller, x 0，i represents initial state, and γ represents parameter of secondary neural network controller.

[0067] In one embodiment of the application, the controller design method further comprises: cyclically collecting initial states of the controlled object, and cyclically solving the optimization equation, so that the optimal controller design is achieved.The number of cycles is determined according to the size of the collected data and the optimization target of the controller.

[0068] The effect of the application can be further illustrated by the following experimental results.

[0069] The first experimental object is a simple discrete-time nonlinear system, and the initial state of the discrete-time nonlinear system is assumed to be x0, which can be expressed as x t+1 = sin(x t )+u t , wherein x t is the system state at t moment, and u t is the control quantity.The structured controller is regarded as a linear controller with control parameter k, that is, u t =kx t . The optimal parameter k * is needed to be found to minimize the control performance.The cost function J(x0) is set as where x0 is the initial state. By simple calculation, the optimal control gain is k * = -sin(x0) / 2x0. From this equation, it can be seen that the optimal control parameter k * is explicitly dependent on the initial state. Numerical simulations are used to detect the performance of three control methods: the improved structured neural network-based controller of the present application, i.e. the NNtoNN controller; the linear controller trained with a single initial state; the linear controller trained with multiple initial states.

[0070] Figure 3 A diagram showing the difference between the control parameters of different controllers and the optimal control parameter for an embodiment of the present application. Figure 4 A diagram showing the difference between the cost functions of different controllers and the optimal cost for an embodiment of the present application. From the diagram, it can be seen that the difference between the learning control gain and the cost of the NNtoNN controller and the optimal controller is almost zero, while the difference between the two linear controllers and the optimal controller is large. Therefore, the NNtoNN controller can effectively learn the optimal linear control gain k*(x0) and achieve a cost J*(x0) close to the optimal cost.

[0071] A second experiment is to use the NNtoNN controller on a traditional structured DNN controller.

[0072] The original Lagrangian system uses a deep energy shaping strategy (DES) to control a two-link robot model, which can guarantee its stability. The inventors use the NNtoNN controller to modify the original DES controller. Figure 5 A diagram showing the difference between the costs of different controllers based on the Lagrangian system for an embodiment of the present application. Figure 5 J DES is the cost of the deep energy shaping strategy (DES) controller of the original Lagrangian system, J NNtoNN is the cost of the NNtoNN controller. As shown in Figure 5 J DES J NNtoNN are all greater than 0. It can be seen that the cost of the NNtoNN controller is much smaller than that of the DES controller, which indicates that the NNtoNN controller improves the optimization performance of the DES controller.

[0073] The original Hamiltonian system's distributed neural network controller (DeepDisCoPH) controls mobile robots to form ranks within a given time. Robots can only avoid collisions by relying on information from neighboring robots, and their trajectories minimize a given loss function. Maintaining the same structure, the inventors modified the original controller with an NNtoNN controller. Table 1 compares the collision counts of the distributed neural network control (DeepDisCoPH) and the NNtoNN controller. As shown in Table 1, with an increasing number of training epochs, the NNtoNN controller proves to be more effective than the DeepDisCoPH controller in reducing both the number of collisions and the cost function Lx.

[0074] Table 1. Comparison of collision counts between DeepDisCoPH and NNtoNN controllers.

[0075]

[0076] This invention addresses the control optimization problem of nonlinear systems by proposing an improved structured neural network-based controller, namely the NNtoNN controller. The main neural network ensures stability, while the secondary neural network can improve the controller's performance by adapting to changing initial states, thereby enhancing the controller's stability and adaptability.

[0077] This invention also provides a controller design system based on an improved structured neural network. Figure 6 This is a schematic diagram of a controller design system based on an improved structured neural network, according to an embodiment of the present invention. Figure 6 As shown, the system includes the following modules:

[0078] The data collection module is configured to sample the initial state of the controlled object and store the sampled data in the database;

[0079] The data training module is configured to extract sampled data from the database and perform training using PyTorch and the Adam optimizer.

[0080] The sub-deep neural network controller module is configured to include a sub-deep neural network controller.

[0081] The main deep neural network controller module is configured to include a main deep neural network controller; and

[0082] The controller optimization module is configured to perform controller optimization within it.

[0083] In one embodiment of the present application, the present application also provides a computer system comprising a processor, a graphics card and a memory, the memory is configured to store machine readable instructions, the graphics card is configured to train a neural network, and the processor is configured to execute the machine readable instructions. When the processor and / or the graphics card execute the machine readable instructions, the following processing steps are implemented: sampling the initial state of the controlled object in the data collection module and storing it in the database as sampling data; extracting the sampling data in the database and using it to perform training in the data training module using PyTorch and Adam optimizer; extracting the trained data in the data training module as the input of the secondary deep neural network controller; taking the output of the secondary deep neural network controller as the weight of the primary deep neural network controller, and performing optimization of the controller in the controller optimization module.

[0084] The graphics card can preferably be a GPU algorithm power higher than 5.0 model graphics card. Since the amount of data required for training is large, providing a graphics card configuration can significantly improve the training speed.

[0085] The memory includes various media that can store machine readable instructions, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, etc.

[0086] It can be understood that, in addition to the above-mentioned memory and processor, the above-mentioned computer system also includes other software and hardware components not listed in the present specification, which can be determined according to the specific data processing equipment model in different application scenarios, and the present specification will not be listed in detail.

[0087] In one embodiment, a computer readable storage medium is also provided, which stores machine readable instructions, and the machine readable instructions, when executed by a processor, implement the following processing steps: sampling the initial state of the controlled object in the data collection module and storing it in the database as sampling data; extracting the sampling data in the database and using it to perform training in the data training module using PyTorch and Adam optimizer; extracting the trained data in the data training module as the input of the secondary deep neural network controller; taking the output of the secondary deep neural network controller as the weight of the primary deep neural network controller, and performing optimization of the controller in the controller optimization module.

[0088] While the forgoing is a complete description of the embodiments of the application, it is to be understood that many alternatives, modifications, and variations thereof will be apparent to persons skilled in the art in light of the foregoing description. It is therefore, contemplated that the breadth and scope of the application should not be limited by that which has been specifically disclosed, but should be defined solely by the claims and their equivalents.

Claims

1. A method for controller design based on improved structured neural networks, characterized in that, Comprising the following steps: sampling initial states of the controlled object in a data collection module and storing the sampling data in a database; extracting the sampling data in the database and using it to perform training in a data training module using PyTorch and Adam optimizer; extracting the trained data in the data training module as input of the secondary deep neural network controller; and using the output of the secondary deep neural network controller as the weight of the primary deep neural network controller and performing optimization of the controller in a controller optimization module; wherein using the output of the secondary deep neural network controller as the weight of the primary deep neural network controller comprises: constructing a Lyapunov function of the primary deep neural network controller, setting the primary deep neural network controller as u(t) = ϕ(x(t); θ), wherein x(t) represents the state quantity at time t, θ is provided by the output of the secondary deep neural network controller W(x(0); γ), x(0) represents the initial state, and γ represents the parameters of the secondary deep neural network controller; wherein performing optimization of the controller in the controller optimization module comprises: The kinetic model is constructed as where x(t) represents the state quantity at time t, and u(t) represents the main deep neural network controller. the optimization equation is wherein min denotes a function minimum value, N denotes a number of initial states, T denotes a training time domain, denotes a loss function, denotes a state quantity, denotes a control quantity, and θ denotes a parameter of a main neural network controller, denotes an initial state, and γ denotes a parameter of a secondary neural network controller.

2. The improved structured neural network based controller design method of claim 1, wherein, the sampling of the initial states of the controlled object in the data collection module comprises: for the controlled object, the sampling method involves uniform sampling or random sampling in a certain initial state interval to obtain the required initial state x(0).

3. The improved structured neural network based controller design method of claim 1, wherein, the training in the data training module using PyTorch and Adam optimizer comprises: for the controlled object, the epoch and learning rate of the neural network are set, the gradient descent is used to solve the optimization problem of the epoch, and the training is performed automatically.

4. The improved structured neural network based controller design method of claim 1, wherein, the extraction of the trained data in the data training module as the input of the secondary deep neural network controller comprises: the trained data of different initial states is used as the input of the secondary deep neural network controller, and the secondary deep neural network controller continuously learns the initial state in the training process, thereby adapting to different initial states.

5. The improved structured neural network based controller design method of claim 1, wherein, the controller design method further comprises: the initial states of the controlled object are collected cyclically, and the optimization equation is solved cyclically, thereby achieving optimal controller design.

6. A controller design system based on an improved structured neural network for the method of any one of claims 1-5, characterized by, comprising the following modules: a data collection module configured to sample initial states of the controlled object and store the sampling data in a database; a data training module configured to extract the sampling data in the database and perform training therein using PyTorch and Adam optimizer; a secondary deep neural network controller module configured to contain a secondary deep neural network controller; a primary deep neural network controller module configured to contain a primary deep neural network controller; and a controller optimization module configured to perform optimization of the controller therein. comprising:

7. A computer system, characterized by a processor configured to execute machine-readable instructions; a graphics card with an artificial intelligence chip configured to train a neural network; and a memory configured to store machine-readable instructions, which, when executed by the processor and / or the graphics card, perform the steps of the method according to any one of claims 1-5. ​ 8. A computer-readable storage medium, characterized in that, having machine readable instructions stored thereon that, when executed by a processor, perform steps of the method of any of claims 1-5.

Images