A method and system for designing a neural network guidance controller based on a visual servoing model of a multi-copter

Abstract

The application discloses a kind of neural network guidance controller design method and system based on multi-rotor aircraft vision servo model.There is including constructing vision servo model, design Lyapunov function and calculate differential, according to differential and solve quadratic programming synthesis stable data set to establish optimization target, and utilize the data set training neural network controller;System is by model construction, Lyapunov analysis, data set and training module composition is made up of.The application will Lyapunov stability condition into data generation, so that the neural network controller of training can ensure that image error exponential convergence, with theoretical stability and adaptability to uncertainty.

Description

Technical Field

[0001] This invention belongs to the field of aircraft guidance and control design technology, and in particular relates to a neural network guidance controller design method and system based on a visual servo model of a multi-rotor aircraft. Background Technology

[0002] Current guidance and control design methods heavily rely on the designer's personal experience and mastery of control design theory. Therefore, a guidance controller design method for visual servo models of multirotor aircraft has been proposed to reduce the requirements for designer experience and control theory knowledge in the guidance and control design of multirotor aircraft.

[0003] In multi-rotor aircraft guidance and control applications, there are two main control design methods based on neural network learning. One is reinforcement learning, which trains the neural network through trial and error. This method relies on a reward function to ensure the training results meet user needs. However, reinforcement training suffers from slow convergence and requires repeated tuning of training parameters, leading to reduced training efficiency. Furthermore, for multi-rotor aircraft guidance and control, an unstable neural network may introduce safety issues such as crashes or uncontrollable collisions, significantly reducing the efficiency of this design method. The other type of control design method is based on Lyapunov stability constraints. This method utilizes Lyapunov stability constraints to ensure the designed controller conforms to stability constraints, enabling the neural network controller to possess stability and safety constraints. This method requires a large amount of control system operational data for effective training. However, for guidance and control problems, obtaining control data first requires a stable and usable controller. Therefore, without a usable controller, it is impossible to use Lyapunov stability constraint-based learning methods for controller design. This type of method is only suitable for improving existing controllers. Moreover, a stable and usable controller needs to be designed before using this method, which makes this type of method lack sufficient application value. Summary of the Invention

[0004] The purpose of this invention is to provide a design method and system for a neural network guidance controller based on a visual servo model of a multi-rotor aircraft. This invention aims to solve the problems of poor adaptability and instability of traditional visual servo controllers in complex environments, as well as the lack of theoretical stability guarantees for learning-based controllers. The goal is to achieve a guidance controller that combines data-driven flexibility with Lyapunov stability guarantees.

[0005] To achieve the objective of this invention, a method for designing a neural network guidance controller based on a visual servo model of a multi-rotor aircraft is provided, comprising the following steps:

[0006] Step 1: Construct a visual servo mathematical model for a multi-rotor aircraft;

[0007] Step 2: Design the Lyapunov function based on the visual servo mathematical model of the multi-rotor aircraft and calculate the differential of the Lyapunov function;

[0008] Step 3: Determine the optimization objective based on the derivative of the Lyapunov function, and synthesize a dataset that meets the stability condition of the optimization objective;

[0009] Step 4: Train the neural network controller based on the dataset.

[0010] Step 1 specifically includes the following steps:

[0011] Step 1-1: Collect motion parameters of the multi-rotor aircraft using sensors, and define the camera motion state vector based on these parameters; the motion parameters include motion speed and rotational angular velocity;

[0012] Steps 1-2: Establish the normalized coordinates of the target point in the gimbal camera image, and construct a transformation relationship matrix based on the camera motion state vector and the normalized coordinates;

[0013] Steps 1-3: Based on the camera motion state vector and the transformation relationship matrix, construct the visual servo mathematical model of the multi-rotor aircraft.

[0014] Step 3 specifically includes the following steps:

[0015] Step 3-1: Utilize the differential of the Lyapunov function Establish an optimization objective function;

[0016] Step 3-2: Define the input vector sample, set the optimization objective function to 0, and substitute the differential of the Lyapunov function. The non-homogeneous equations are obtained by rearranging them, and a quadratic programming equation is established.

[0017] Step 3-3: Define state vector samples, and construct a dataset that meets the target by using the state vector samples and the input vector samples through the quadratic programming equation.

[0018] The optimization objective function in step 3-1 is shown in the following formula:

[0019] ;

[0020] in, This indicates the speed of the gimbal camera along the horizontal axis of the image coordinate system. and the speed of the gimbal camera along the vertical axis of the image coordinate system The maximum absolute value; Indicates the exponential convergence speed. The horizontal axis coordinate represents the normalized coordinates of the target point in the gimbal camera image; Indicates the target point is on the gimbal.

[0021] Step 3-2 specifically includes:

[0022] Set the objective function to 0 and substitute it into the differential of the Lyapunov function. The specific formula is as follows:

[0023] ;

[0024] in, This indicates the speed at which the gimbal camera moves along the camera's optical axis. This indicates the distance between the gimbal camera and the target along the camera's optical axis. This indicates the angular velocity of the gimbal camera rotating around its longitudinal axis;

[0025] The resulting nonhomogeneous equation after simplification is shown in the following equation:

[0026] ;

[0027] in, Represents the input vector sample Like terms matrix, Represents the input vector sample Irrelevant scalar terms of the same kind, Represents an input vector sample;

[0028] According to the like terms matrix scalars of the same kind The quadratic programming equation is established as follows:

[0029] ;

[0030] in, Indicates the data fitting weights. Indicates the control quantity weight. Represents the identity matrix.

[0031] Define the objective function used in step 4 to train the neural network controller. The specific formula is as follows:

[0032] ;

[0033] in, Indicates the number of training rounds. Indicates the process The neural network after rounds of training, i.e., the target controller. Indicates that the neural network has passed through The set of network parameters after one round of training.

[0034] On the other hand, the present invention also provides a system for implementing the above-described neural network guidance controller design method based on a multi-rotor aircraft visual servo model, comprising the following modules:

[0035] The model building module is used to build a visual servo mathematical model for a multi-rotor aircraft.

[0036] The Lyapunov analysis module is used to design Lyapunov functions based on the mathematical model and calculate their differentials.

[0037] The dataset synthesis module is used to determine the optimization objective based on the derivative of the Lyapunov function and synthesize a dataset that satisfies the stability condition.

[0038] The training module is used to train the neural network controller based on the dataset.

[0039] Compared with the prior art, the significant progress of the present invention is as follows: (1) The present invention provides a theoretical basis for stability by constructing a mathematical model and designing a Lyapunov function; (2) The present invention transforms the stability condition into a dataset synthesis criterion by optimizing the objective function, and the generated training data naturally satisfies the exponential stability condition, so that the trained neural network controller inherits stability; (3) The present invention generates a dataset by using a quadratic programming method, which can flexibly adjust the weights and balance the fitting accuracy and control energy; (4) The present invention directly matches the stable mapping of the dataset with the training objective of the neural network controller, avoiding the trial and error and stability verification problems in traditional reinforcement learning, and improving design efficiency and reliability.

[0040] To more clearly illustrate the functional characteristics and structural parameters of the present invention, further explanation is provided below in conjunction with the accompanying drawings and specific embodiments. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0042] Figure 1 This is a flowchart of the steps of the present invention;

[0043] Figure 2 This is the curve showing the coordinate change of the target point in the camera image according to an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] This invention discloses a design method for a neural network guidance controller based on a visual servo model of a multi-rotor aircraft, combined with... Figure 1 This includes the following steps:

[0046] Step 1: Construct a visual servo mathematical model for a multi-rotor aircraft;

[0047] Specifically, the following steps are included:

[0048] Step 1-1: Collect the motion parameters of the multi-rotor aircraft using sensors, and define the camera motion state vector based on these parameters, as shown in the following formula:

[0049] ;

[0050] in, Represents the camera motion state vector. This indicates the speed at which the gimbal camera moves along the horizontal axis of the image coordinate system, in meters per second. This indicates the speed at which the gimbal camera moves along the vertical axis of the image coordinate system, in meters per second. This indicates the speed at which the gimbal camera moves along the camera's optical axis, measured in meters per second. This indicates the angular velocity of the gimbal camera rotating around its longitudinal axis, expressed in radians per second.

[0051] Steps 1-2: Establish the normalized coordinates of the target point in the gimbal camera image, and construct a transformation relationship matrix based on the camera motion state vector and the normalized coordinates;

[0052] The normalized coordinates for: ;in, The horizontal axis coordinate represents the normalized coordinates of the target point in the gimbal camera image; The vertical axis coordinate represents the normalized coordinates of the target point in the gimbal camera image;

[0053] The transformation relationship matrix is ​​shown in the following formula:

[0054] ;

[0055] in, This indicates the distance between the gimbal camera and the target along the camera's optical axis; the origin of the camera image coordinate system is located at the center of the image, with the positive direction of the horizontal axis to the right of the origin and the positive direction of the vertical axis downwards from the origin.

[0056] Steps 1-3: Based on the camera motion state vector and the transformation relationship matrix, construct the visual servo mathematical model of the multi-rotor aircraft. : .

[0057] Step 2: Design the Lyapunov function based on the visual servo mathematical model of the multi-rotor aircraft and calculate the differential of the Lyapunov function;

[0058] The Lyapunov function The specific formula is as follows:

[0059] ;

[0060] The differential of the Lyapunov function The specific formula is as follows:

[0061] .

[0062] Further expand the differential of the Lyapunov function The specific formula is as follows:

[0063] .

[0064] Step 3: Determine the optimization objective based on the derivative of the Lyapunov function, and synthesize a dataset that meets the stability condition.

[0065] Specifically, the following steps are included:

[0066] Step 3-1: Utilize the differential of the Lyapunov function The objective function is established as follows:

[0067] ;

[0068] in, This indicates the speed of the gimbal camera along the horizontal axis of the image coordinate system. and the speed of the gimbal camera along the vertical axis of the image coordinate system The maximum absolute value is used to limit the gimbal's flight speed along the horizontal and vertical axes of the camera coordinate system. Since the gimbal is mounted on a multi-rotor aircraft, and the aircraft is constrained by physical laws and cannot reach arbitrary flight speed values, this constraint is to limit the range of speed commands generated by the controller from exceeding the aircraft's flight speed capability boundary. Indicates the exponential convergence rate;

[0069] Step 3-2: Define the input vector sample for: Set the objective function to 0 and substitute it with the differential of the Lyapunov function. After rearranging, we obtain a non-homogeneous equation;

[0070] Set the objective function to 0 and substitute it into the differential of the Lyapunov function. The specific formula is as follows:

[0071] ;

[0072] The resulting nonhomogeneous equation after simplification is shown in the following equation:

[0073] ;

[0074] in, Represents the input vector sample The like terms matrix is ​​specifically represented as:

[0075] ;

[0076] Represents the input vector sample Unrelated like terms scalars are specifically represented as:

[0077] ;

[0078] According to the like terms matrix scalars of the same kind The quadratic programming equation is established as follows:

[0079] ;

[0080] in, Indicates the data fitting weights. Indicates the control quantity weight. Represents the identity matrix;

[0081] Step 3-3: Define state vector samples Specifically, it is expressed as Define a dataset consisting of state vector samples and input samples. ;

[0082] Choose according to actual needs. Different state vector samples For each given state vector sample Substituting into the quadratic programming equation, and using known methods for solving quadratic programming equations, we obtain... Input Samples Thus obtaining A dataset of samples .

[0083] Step 4: Train the neural network controller based on the dataset;

[0084] Define the objective function used to train the neural network controller. The specific formula is as follows:

[0085] ;

[0086] in, Indicates the number of training rounds. Indicates the process The neural network after rounds of training, i.e., the target controller. Indicates that the neural network has passed through The network parameter set after rounds of training. Using this dataset, according to the stated objective function... Training to obtain controller .

[0087] A system for implementing the above-described design method of a neural network guidance controller based on a visual servo model of a multi-rotor aircraft includes the following modules:

[0088] The model building module is used to build a visual servo mathematical model for a multi-rotor aircraft.

[0089] The Lyapunov analysis module is used to design Lyapunov functions based on the mathematical model and calculate their differentials.

[0090] The dataset synthesis module is used to determine the optimization objective based on the derivative of the Lyapunov function and synthesize a dataset that satisfies the stability condition.

[0091] The training module is used to train the neural network controller based on the dataset.

[0092] The specific implementation steps of this system are the same as those described above, and will not be repeated here.

[0093] Example

[0094] This embodiment calculates the parameters used in the simulation dataset based on steps 1-4 above.

[0095] Table 1. Synthesis Parameters of Visual Servo Guidance and Control Dataset

[0096]

[0097] Next, we need to select the state combination range. For the image coordinate range, due to physical constraints, its absolute value will not exceed 1, so we directly select the full state range. For the distance between the camera and the target, we set a maximum range of 50 meters and a minimum value of 0.5 to avoid singularities. For the camera angular velocity, we set a maximum of 1 radian. For the maximum guidance velocity... The appropriate value should be selected based on the needs; the maximum setting here is 20 m / s. If a higher speed is required, it can be increased further. Table 2 details the sampling range, sampling interval, and final number of samples used during dataset synthesis. An excessively large sampling step size may lead to poor training results, while an excessively small sampling step size will result in slow training speed. Therefore, it is necessary to experiment multiple times in practice to determine the appropriate value.

[0098] Table 2. State sampling distribution of the visual servoing dataset

[0099]

[0100] Based on the state combinations in Table 2, assuming that each state combination has a feasible solution, then the number of samples in this dataset is...

[0101]

[0102] For the state of each sample, a quadratic programming solver is used to solve the objective cost function to find a suitable control variable, that is... and After solving, the composition of each sample is shown in Table 3:

[0103] Table 3 Visual Servo Sample Format

[0104]

[0105] By combining the number of samples and the sample format, the size of the dataset can be roughly calculated. In this example, there are 74,132,100 samples, each sample contains 8 data points, and assuming that each data point is stored in single-precision format and occupies four bytes, the resulting dataset size is approximately 2.21 GB.

[0106] Step 4: Based on the obtained dataset, randomly initialize a feedforward neural network and train it. The input layer of this neural network has 5 neurons, used to input the system state vector. The output layer has 2 neurons, used to output the control vector. In addition, the neural network contains three fully connected layers, each with 16 neurons; the hyperbolic tangent function is used as the activation function between adjacent layers, and the number of training epochs is... .

[0107] The trained neural network controller is imported into the simulation software and used as the controller, while simultaneously employing the formula...

[0108]

[0109] As a controlled model, it is used for simulation. Here, for simplicity, it is directly set to... The forward flight speed of the multi-rotor aircraft is set to... , and It comes from the output of the neural network controller.

[0110] Simulation results are as follows Figure 2 As shown, in the initial state of the simulation, the target point is not at the origin of the image coordinate system. After the simulation starts, the aircraft flies at a given forward speed, and under the action of the trained controller, the position of the target point in the image tends towards the center of the image. This means that in the simulation, the aircraft always flies towards and approaches the target point.

[0111] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0112] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A design method for a neural network guidance controller based on a visual servo model of a multi-rotor aircraft, characterized in that, Includes the following steps: Step 1: Construct a visual servo mathematical model for a multi-rotor aircraft; Step 2: Design the Lyapunov function based on the visual servo mathematical model of the multi-rotor aircraft and calculate the differential of the Lyapunov function; Step 3: Determine the optimization objective based on the derivative of the Lyapunov function, and synthesize a dataset that meets the stability condition of the optimization objective; Step 4: Train the neural network controller based on the dataset.

2. The design method of a neural network guidance controller based on a visual servo model of a multi-rotor aircraft according to claim 1, characterized in that, Step 1 specifically includes the following steps: Step 1-1: Collect motion parameters of the multi-rotor aircraft using sensors, and define the camera motion state vector based on these parameters; the motion parameters include motion speed and rotational angular velocity; Steps 1-2: Establish the normalized coordinates of the target point in the gimbal camera image, and construct a transformation relationship matrix based on the camera motion state vector and the normalized coordinates; Steps 1-3: Based on the camera motion state vector and the transformation relationship matrix, construct the visual servo mathematical model of the multi-rotor aircraft.

3. The design method of a neural network guidance controller based on a visual servo model of a multi-rotor aircraft according to claim 2, characterized in that, Step 3 specifically includes the following steps: Step 3-1: Utilize the differential of the Lyapunov function Establish an optimization objective function; Step 3-2: Define the input vector sample, set the optimization objective function to 0, and substitute the differential of the Lyapunov function. The non-homogeneous equations are obtained by rearranging them, and a quadratic programming equation is established. Step 3-3: Define state vector samples, and construct a dataset that meets the target by using the state vector samples and the input vector samples through the quadratic programming equation.

4. The design method of a neural network guidance controller based on a visual servo model of a multi-rotor aircraft according to claim 3, characterized in that, The optimization objective function in step 3-1 is shown in the following formula: ； in, This indicates the speed of the gimbal camera along the horizontal axis of the image coordinate system. and the speed of the gimbal camera along the vertical axis of the image coordinate system The maximum absolute value; Indicates the exponential convergence speed. The horizontal axis coordinate represents the normalized coordinates of the target point in the gimbal camera image; The vertical axis coordinate represents the normalized coordinates of the target point in the gimbal camera image.

5. The design method of a neural network guidance controller based on a visual servo model of a multi-rotor aircraft according to claim 4, characterized in that, Step 3-2 specifically includes: Set the objective function to 0 and substitute it into the differential of the Lyapunov function. The specific formula is as follows: ； in, This indicates the speed at which the gimbal camera moves along the camera's optical axis. This indicates the distance between the gimbal camera and the target along the camera's optical axis. This indicates the angular velocity of the gimbal camera rotating around its longitudinal axis; The resulting nonhomogeneous equation after simplification is shown in the following equation: ； in, Represents the input vector sample Like terms matrix, Represents the input vector sample Irrelevant scalar terms of the same kind Represents an input vector sample; According to the like terms matrix scalars of the same kind The quadratic programming equation is established as follows: ； in, Indicates the data fitting weights. Indicates the control quantity weight. Represents the identity matrix.

6. The design method of a neural network guidance controller based on a visual servo model of a multi-rotor aircraft according to claim 5, characterized in that, Define the objective function used in step 4 to train the neural network controller. The specific formula is as follows: ； in, Indicates the number of training rounds. Indicates the process The neural network after rounds of training, i.e., the target controller. Indicates that the neural network has passed through The set of network parameters after one round of training.

7. A system for implementing the neural network guidance controller design method based on a visual servo model of a multi-rotor aircraft as described in any one of claims 1-6, characterized in that, Includes the following modules: The model building module is used to build a visual servo mathematical model for a multi-rotor aircraft. The Lyapunov analysis module is used to design Lyapunov functions based on the mathematical model and calculate their differentials. The dataset synthesis module is used to determine the optimization objective based on the derivative of the Lyapunov function and synthesize a dataset that satisfies the stability condition. The training module is used to train the neural network controller based on the dataset.

Images