A visual landing guidance method for carrier-based aircraft

Abstract

The present invention belongs to the field of aviation technology, and specifically discloses a carrier-based aircraft visual landing guidance method, comprising the following steps: taking a screenshot of an aircraft carrier model in a visual software to obtain an aircraft carrier data set, and using the aircraft carrier data set to train a YOLOv8 model to obtain a trained YOLOv8 model; determining a first position coordinate or a second position coordinate of the aircraft carrier in a world coordinate system; calculating the relative distance between the aircraft carrier and a drone according to the first position coordinate or the second position coordinate, and determining a pitch angle command and a roll angle command of the drone; determining a pitch angle command difference and a roll angle command difference, and using a preset control law to convert the pitch angle command difference and the roll angle command difference into a rudder deflection angle, and guiding the drone to land. The present invention can realize the autonomous landing mission of the drone and improve the flexibility of the drone when landing. It solves the problem of low flexibility of the drone landing in the prior art.

Description

Technical Field

[0001] The present invention belongs to the field of aviation technology, and in particular relates to a carrier-based aircraft visual landing guidance method. Background Art

[0002] In recent years, with the rapid development of autonomous control technology and mission payloads, more and more drones have good mission autonomy capabilities. In the process of executing the mission, the landing stage has the greatest safety risk. Poor navigation and positioning accuracy and personnel decision-making errors are the main reasons for the frequent accidents during the landing process. In order to meet the needs of high-frequency and large-scale use of drones and ensure the safety and efficiency of the entire process, autonomous and safe landing technology of drones has become a focus of attention in the civil and military fields. In the military field, due to the complex and changeable battlefield environment, the demand for autonomous take-off and landing of manned and unmanned aircraft on ships is more urgent, showing the characteristics of complex electromagnetic environment and high dynamic movement of the platform. In addition, considering the large number of unmanned equipment on board in the future, the existing number of personnel cannot meet the high-frequency take-off and landing needs of drones. Therefore, providing drones with a systematic guidance landing solution with high automation and reliable navigation and positioning accuracy will help further cope with the complex and changeable battlefield environment and reduce the operating burden of personnel.

[0003] With the continuous development of visual perception and visual navigation technology, the use of UAV onboard vision systems to guide landing has received widespread attention. The use of visual systems to guide UAV landing has the following advantages: (1) It does not require the coordination of ground-side software and hardware, and has low requirements for the hardware sensors of the aircraft carrier itself, which reduces costs while improving the flexibility of UAV landing; (2) It can simplify the complex landing guidance process and reduce the burden on operators; (3) Vision-based state estimation uses a passive measurement mechanism, which does not require the transmission and reception of external signals and is therefore not easily interfered with. Due to the above advantages of the visual guidance landing system, if the UAV vision system can be used to accurately locate the UAV and complete the guided landing, the cost of UAV landing will be greatly reduced while ensuring safe landing, and the autonomous capability of the UAV will be improved. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a carrier-based aircraft visual landing guidance method, which solves the problem of low flexibility when unmanned aerial vehicles land in the prior art.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a carrier-based aircraft visual landing guidance method, comprising the following steps:

[0006] S1. Take a screenshot of the aircraft carrier model in the visual software to obtain an aircraft carrier data set, and use the aircraft carrier data set to train the YOLOv8 model to obtain a trained YOLOv8 model;

[0007] S2, obtaining image data of the target aircraft carrier through a drone equipped with a camera;

[0008] S3. When the target aircraft carrier image data is taken by the drone at a distance greater than a preset distance threshold from the aircraft carrier, the trained YOLOv8 model is used to identify the target aircraft carrier image data, obtain the coordinate position of the aircraft carrier in the target aircraft carrier image data, and obtain the first position coordinate of the aircraft carrier in the world coordinate system based on the conversion formula;

[0009] When the target aircraft carrier image data is taken by the UAV at a distance less than or equal to a preset distance threshold from the aircraft carrier, the runway line in the target aircraft carrier image data is identified, and the runway line is positioned using the constant deformation method to obtain the second position coordinates of the UAV in the runway coordinate system;

[0010] S4. Calculate the relative distance between the aircraft carrier and the UAV according to the first position coordinate and the second position coordinate, and determine the pitch angle command and the roll angle command of the UAV;

[0011] S5, obtaining state feedback of the drone, calculating the difference between the state feedback and the pitch angle command and the roll angle command, and determining the pitch angle command difference and the roll angle command difference;

[0012] S6. Use the preset control law to convert the pitch angle command difference and the roll angle command difference into the rudder surface deflection angle, and guide the UAV to land through the rudder surface deflection angle.

[0013] The beneficial effect of the above scheme is as follows: the present invention establishes an aircraft carrier data set, targets the target tracking requirements in the landing scenario, trains a convolutional neural network to realize aircraft carrier target detection at a long distance, and after realizing the target detection of the aircraft carrier in the image, converts the pixel coordinates of the aircraft carrier in the image into coordinates in the world system, so as to guide the approach by the landing control law, and when the distance is close, uses the deck runway detection algorithm to perform line detection on the deck runway, realizes the position solution of the UAV in the runway coordinate system according to the geometric relationship identity deformation method, and embeds the position solution result into the landing control law after filtering and smoothing, finally realizing the autonomous landing mission of the UAV and improving the flexibility of the UAV during landing.

[0014] Furthermore, before S3, the method further includes:

[0015] Obtain the altitude of the drone based on its sensors and calculate the image depth of the pictures taken by the drone’s camera;

[0016] The calculation formula used to calculate the image depth of the picture taken by the camera on the drone is:

[0017]

[0018] in, Indicates the image depth of the picture taken by the camera on the drone. Indicates the vertical coordinate of the aircraft carrier in the world coordinate system, Indicates the vertical coordinate of the drone in the world coordinate system, Represents the rotation matrix from the drone coordinate system to the world coordinate system, Represents the coordinate position of the camera in the world coordinate system, represents the homogeneous coordinates in the pixel coordinate system, where , Represents the rotation matrix from the world coordinate system to the camera coordinate system, Represents the camera intrinsic parameter matrix.

[0019] Furthermore, in S3, the conversion formula is:

[0020]

[0021] in, Indicates the position coordinates of the aircraft carrier in the world coordinate system. Indicates the position coordinates of the drone in the world coordinate system, Represents the coordinate position of the camera in the world coordinate system, Represents the rotation matrix from the drone coordinate system to the world coordinate system, Represents the rotation matrix from the camera coordinate system to the drone coordinate system, Represents the camera internal parameter matrix, which is a constant coefficient matrix. Indicates the image depth of the picture taken by the camera on the drone. Represents homogeneous coordinates in the pixel coordinate system.

[0022] Furthermore, before S4, the method further includes:

[0023] S3-A, obtaining a first state vector of the aircraft carrier through a first inertial sensor, and obtaining a second state vector of the UAV through a second inertial sensor;

[0024] S3-B, based on the Kalman filter algorithm, the first state vector and the first position coordinate are fused to obtain the first corrected position coordinate of the aircraft carrier in the world coordinate system;

[0025] Alternatively, based on an extended Kalman filter algorithm, the second state vector and the second position coordinate are fused to obtain a second corrected position coordinate of the aircraft carrier in the world coordinate system;

[0026] S4, specifically includes:

[0027] According to the first corrected position coordinates and the second corrected position coordinates, the relative distance between the aircraft carrier and the UAV is calculated, and the pitch angle command and the roll angle command of the UAV are determined.

[0028] Further, in S3-A, obtaining a second state vector of the drone through a second inertial sensor specifically includes:

[0029] The attitude quaternion of the UAV, the velocity vector of the UAV in the runway coordinate system, and the position vector of the UAV in the runway coordinate system are respectively acquired by the second inertial sensor and are spliced ​​to obtain a second state vector of the UAV;

[0030] In S3-B, based on the extended Kalman filter algorithm, the second state vector and the second position coordinate are fused to obtain the second corrected position coordinate of the aircraft carrier in the world coordinate system, which specifically includes:

[0031] S3-B1, determine the one-step state transition function;

[0032] S3-B2, obtaining a state estimation result according to the second state vector and the one-step state transfer function;

[0033] S3-B3, determining the second position coordinate as a measurement vector;

[0034] S3-B4, determine the measurement function, the process noise covariance matrix, the process noise shaping matrix, the approximately linearized measurement matrix and the measurement noise covariance matrix;

[0035] S3-B5, perform partial differentiation on the one-step state transfer function to obtain an approximate linearized state transfer matrix;

[0036] S3-B6. Use the extended Kalman filter algorithm to fuse the state estimation results and the measurement vector based on the measurement function, process noise covariance matrix, process noise shaping matrix, approximate linearized measurement matrix, measurement noise covariance matrix and approximate linearized state transfer matrix to obtain the second corrected position coordinate.

[0037] Furthermore, in S3-B1, the one-step state transition function is:

[0038]

[0039] in, represents the state transition function, represents quaternion multiplication, represents the process noise covariance matrix, Indicates that the gyroscope is Three-axis angle increment output within the time interval, express The modulus value, represents the speed of the aircraft in the runway coordinate system at time k-1, represents the measurement noise covariance matrix, The gyroscope is extrapolated The three-axis velocity increment output in the drone coordinate system within the time interval, Indicates the position of the aircraft in the runway coordinate system at the last moment. represents the speed of the aircraft in the runway coordinate system at time k, represents the time increment, , represents the k moment, represents k-1 moment;

[0040] In S3-B4, the measurement function is:

[0041]

[0042] in, represents the measurement function, Indicates the position of the aircraft in the runway coordinate system obtained by visual solution;

[0043] In S3-B5, the approximate linearized state transfer matrix is:

[0044]

[0045] in, Table approximates the linearized state transfer matrix, represents the state matrix, represents the transpose of the state matrix, represents a quaternion, represents the time increment, , represents the k moment, represents the k-1 moment, represents the identity matrix;

[0046] In S3-B4, the process noise covariance matrix is:

[0047]

[0048] in, represents the k-1 time process noise covariance matrix, , represents the k-time process noise covariance matrix;

[0049] In S3-B4, the process noise shaping matrix is:

[0050]

[0051] in, represents the k-1 moment process noise shaping matrix, represents the k-1 time process noise covariance matrix, represents the angle calculated by the gyroscope, represents the identity matrix;

[0052] In S3-B4, the approximate linearized measurement matrix is:

[0053]

[0054] in, represents the approximately linearized measurement matrix, represents an approximately linearized quantity measurement, represents the transpose of the state matrix, represents the identity matrix;

[0055] In S3-B4, the measurement noise covariance matrix is:

[0056]

[0057] in, represents the measurement noise covariance matrix.

[0058] Furthermore, in S4, the relative distance between the aircraft carrier and the UAV is calculated, and the pitch angle command and roll angle command of the UAV are determined using the following calculation formula:

[0059]

[0060] in, and Indicates the position of the aircraft carrier in the world coordinate system , Direction coordinates, and Indicates the coordinates of the drone in the world coordinate system. represents the glide angle of the drone, Indicates altitude control instructions. Indicates the pitch angle control command, Indicates the roll angle control command, represents the height scale parameter, Indicates the height feedback value, represents the height integral parameter, represents the highly differential parameter, represents the differential of height change, represents the side slip ratio parameter, represents the yaw ratio parameter, Indicates the yaw angle.

[0061] Furthermore, in S6, the calculation formula of the preset control law is:

[0062]

[0063] in, Indicates the elevator increment command, represents the pitch angular velocity ratio parameter, represents the pitch angular velocity, represents the pitch angle ratio parameter, represents the pitch angle, Indicates the throttle increment command, represents the speed ratio parameter, Indicates speed command, Indicates speed feedback, represents the integral proportional parameter, represents the aileron increment command, represents the roll angular velocity proportional parameter, is the rolling angular velocity, represents the roll angle ratio parameter, represents the roll angle feedback, Indicates the roll angle command.

[0064] The beneficial effect of the above further solution is that the autonomous landing of the aircraft can be effectively achieved by using visual guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 The figure is a flowchart of a carrier-based aircraft visual landing guidance method.

[0066] Figure 2 A diagram of the label file.

[0067] Figure 3 Schematic diagram of the position relationship between the UAV and the aircraft carrier in the world coordinate system.

[0068] Figure 4 This is the recognition effect diagram of the runway line in the target aircraft carrier image data.

[0069] Figure 5 This is a schematic diagram of the detection and positioning effect during the aircraft approach process.

[0070] Figure 6 This is a schematic diagram of the recognition results of the lower computer running the visual detection and positioning algorithm.

[0071] Figure 7 This is another schematic diagram of the recognition results of the lower computer running the visual detection and positioning algorithm.

[0072] Figure 8 The communication diagram of the simulation verification platform.

[0073] Fig. 9 Schematic diagram of the trajectory of the UAV and aircraft carrier in the visual closed-loop landing simulation.

[0074] Fig.10 Schematic diagram of the trajectory response of the UAV and aircraft carrier in the visual closed-loop landing simulation.

[0075] Fig.11 Another schematic diagram of the trajectory response of the UAV and the aircraft carrier in the visual closed-loop landing simulation. DETAILED DESCRIPTION

[0076] Now, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the embodiments shown and described in the accompanying drawings are only exemplary and are intended to explain the principles and spirit of the present invention, rather than to limit the scope of the present invention.

[0077] Example 1

[0078] like Figure 1 As shown, a carrier-based aircraft visual landing guidance method includes the following steps:

[0079] S1. Take a screenshot of the aircraft carrier model in the vision software to obtain an aircraft carrier data set, and use the aircraft carrier data set to train the YOLOv8 model to obtain a trained YOLOv8 model.

[0080] For example, when the drone is far away from the aircraft carrier, the aircraft carrier target is small and does not have sufficient pixel features to be detected by traditional methods. As the drone approaches, the size of the aircraft carrier in the image captured by the onboard camera continues to increase, and its pixel features continue to change.

[0081] Therefore, we can manually take screenshots of the aircraft carrier model in the visual software and create an aircraft carrier data set to train the YOLOv8 model. During the data collection phase, we can let the aircraft gradually approach along the standard glide angle, randomly change the aircraft's attitude angle at each position, and expand the distribution range of the data set as much as possible. After collecting enough images, we can use labelme software to implement data annotation. Figure 2 As shown, Figure 2 This is a diagram of the annotation of the label file. The software can provide a rectangular box annotation method, saving the coordinates of the upper left vertex and the lower right vertex of the annotated rectangular box, as well as the label to which the rectangular box belongs, to a json file with the same name as the image.

[0082] After the aircraft carrier data set is labeled, the labels can be further processed. The label files used by the YOLO series of algorithms are generally in .txt format. Each line in each txt file represents a label, and each line has five columns of data, namely the category, the normalized value of the x-axis coordinate of the rectangular box, the normalized value of the y-axis coordinate of the rectangular box, the normalized value of the width of the rectangular box, and the normalized value of the height of the rectangular box. After the label data is processed, it is stored in the image folder and label folder under the training set and verification set folders respectively, and the corresponding label path txt file is generated to realize the creation of the aircraft carrier data set.

[0083] Use the aircraft carrier dataset to train the YOLOv8 model. After training, you can choose to convert the model to ONNX type first, so that you can call the trained YOLOv8 model under Windows or Ubuntu.

[0084] S2. Obtain image data of the target aircraft carrier through a drone equipped with a camera.

[0085] Exemplarily, a drone equipped with a camera can be a virtual visual sensor for the simulation platform. The virtual visual sensor mainly provides the image information required by the visual detection algorithm, and is composed of Vega Prime visual simulation software based on the MFC framework. Among them, the visual simulation software receives the status information of the aircraft model in Simulink through UDP communication, can complete the visual presentation of the flight process, and collect the required image information. The virtual visual sensor part saves the airport runway view during the aircraft landing as an image sequence, and transmits the image, posture and timestamp information to the visual navigation terminal for solution.

[0086] S3. When the target aircraft carrier image data is taken by the drone at a distance greater than a preset distance threshold from the aircraft carrier, the trained YOLOv8 model is used to identify the target aircraft carrier image data, obtain the coordinate position of the aircraft carrier in the target aircraft carrier image data, and obtain the first position coordinate of the aircraft carrier in the world coordinate system based on the conversion formula;

[0087] When the target aircraft carrier image data is taken by the UAV at a distance less than or equal to a preset distance threshold from the aircraft carrier, the runway line in the target aircraft carrier image data is identified, and the runway line is positioned using the constant deformation method to obtain the second position coordinates of the UAV in the runway coordinate system.

[0088] In this embodiment, before S3, the method further includes:

[0089] Obtain the altitude of the drone based on its sensors and calculate the image depth of the pictures taken by the drone’s camera;

[0090] The calculation formula used to calculate the image depth of the picture taken by the camera on the drone is:

[0091]

[0092] in, Indicates the image depth of the picture taken by the camera on the drone. Indicates the vertical coordinate of the aircraft carrier in the world coordinate system, Indicates the vertical coordinate of the drone in the world coordinate system, Represents the rotation matrix from the drone coordinate system to the world coordinate system, Represents the coordinate position of the camera in the world coordinate system, represents the homogeneous coordinates in the pixel coordinate system, where , Represents the rotation matrix from the world coordinate system to the camera coordinate system, Represents the camera intrinsic parameter matrix.

[0093] In this embodiment, in S3, the conversion formula is:

[0094]

[0095] in, Indicates the position coordinates of the aircraft carrier in the world coordinate system. Indicates the position coordinates of the drone in the world coordinate system, Represents the coordinate position of the camera in the world coordinate system, Represents the rotation matrix from the drone coordinate system to the world coordinate system, Represents the rotation matrix from the camera coordinate system to the drone coordinate system, Represents the camera internal parameter matrix, which is a constant coefficient matrix. Indicates the image depth of the picture taken by the camera on the drone. Represents homogeneous coordinates in the pixel coordinate system.

[0096] For example, the world coordinate system is the north-east coordinate system, the drone coordinate system is a coordinate system established with the drone as the origin, the camera coordinate system is a coordinate system established with the camera as the origin, and the runway coordinate system is a right-handed coordinate system established with the center of the runway starting point as the origin, the longitudinal runway as the positive direction of the x-axis, and the transverse runway as the positive direction of the y-axis. The pixel coordinate system is a coordinate system established with the upper left corner of the image taken by the camera as the coordinate origin, the horizontal right as the positive direction of the x-axis, and the vertical downward as the positive direction of the y-axis.

[0097] like Figure 3 As shown, Figure 3 This is a schematic diagram of the position relationship between the UAV and the aircraft carrier in the world coordinate system. Figure 3 In the world coordinate system, the position coordinates of the aircraft carrier in the world coordinate system It can be obtained from the following relationship:

[0098]

[0099] in, Indicates the position coordinates of the drone in the world coordinate system, Represents the projection of the camera's position coordinates in the drone's coordinate system to the world coordinate system. Represents the projection of the aircraft carrier's position coordinates in the camera coordinate system to the world coordinate system.

[0100] The calculation method of is as follows:

[0101]

[0102] in, Indicates the position coordinates of the camera in the drone coordinate system. Represents the rotation matrix from the drone coordinate system to the world coordinate system.

[0103] The calculation method of is as follows:

[0104]

[0105] in, Indicates the position coordinates of the aircraft carrier in the camera coordinate system. Represents the rotation matrix from the camera coordinate system to the drone coordinate system.

[0106] The calculation formula used to calculate the image depth of the picture taken by the camera on the drone is: ,in, Represents the rotation matrix from the drone coordinate system to the world coordinate system, Represents the rotation matrix from the world coordinate system to the camera coordinate system, Represents the camera intrinsic parameter matrix.

[0107] From this, we can deduce the first position coordinates of the aircraft carrier in the world coordinate system:

[0108]

[0109] in, Indicates the position coordinates of the aircraft carrier in the world coordinate system. Indicates the position coordinates of the drone in the world coordinate system, Represents the coordinate position of the camera in the world coordinate system, Represents the rotation matrix from the drone coordinate system to the world coordinate system, represents the homogeneous coordinates in the pixel coordinate system, where , Represents the rotation matrix from the world coordinate system to the camera coordinate system, Represents the camera intrinsic parameter matrix.

[0110] Optionally, the first position coordinates of the aircraft carrier in the world coordinate system are valid only when the target aircraft carrier image data is taken by the drone at a distance greater than a preset distance threshold from the aircraft carrier. If the distance is too close, due to the large size of the aircraft carrier, the center point of the detection result will keep changing, and the center point can no longer represent the position of the aircraft carrier. When the target aircraft carrier image data is taken by the drone at a distance less than, or equal to, the preset distance threshold from the aircraft carrier, the coordinates of the runway are required instead of the coordinates of the aircraft carrier, and simply detecting the aircraft carrier can no longer provide sufficient navigation information.

[0111] Exemplarily, a method for identifying runway lines in target aircraft carrier image data may be as follows: using traditional methods to detect runway lines, first converting the RGB three-channel image into a single-channel grayscale image, which can reduce the amount of calculation and facilitate subsequent processing. Then, Gaussian filtering is performed on the grayscale image to remove impurities and noise in the image. Then, based on Gaussian filtering, the image is binarized by setting a threshold, and a series of subsequent image processing is performed on the binarized image. By setting dynamic masks and static masks, interference can be reduced, the scope of image processing can be reduced, and the processing speed can be accelerated. Adding white lines on both sides of the dynamic mask and dilation corrosion to remove cracks in the image can make the non-runway part form a connected area, which is convenient for the subsequent removal of this part of interference by filling the connected area. After a series of noise reduction and interference removal on the image, the edge of the runway line can be detected by the Sobel operator, and then the runway line can be extracted by Hough transform. At this time, many runway lines may be detected, and then the extracted runway lines need to be filtered through a series of constraints to finally obtain the runway line. Figure 4 As shown, Figure 4 This is the recognition effect diagram of the runway line in the target aircraft carrier image data.

[0112] S4. Calculate the relative distance between the aircraft carrier and the UAV according to the first position coordinate and the second position coordinate, and determine the pitch angle command and the roll angle command of the UAV.

[0113] In this embodiment, before S4, the method further includes:

[0114] S3-A, obtaining a first state vector of the aircraft carrier through a first inertial sensor, and obtaining a second state vector of the UAV through a second inertial sensor;

[0115] S3-B, based on the Kalman filter algorithm, the first state vector and the first position coordinate are fused to obtain the first corrected position coordinate of the aircraft carrier in the world coordinate system;

[0116] Alternatively, based on an extended Kalman filter algorithm, the second state vector and the second position coordinate are fused to obtain a second corrected position coordinate of the aircraft carrier in the world coordinate system;

[0117] S4, specifically includes:

[0118] According to the first corrected position coordinates and the second corrected position coordinates, the relative distance between the aircraft carrier and the UAV is calculated, and the pitch angle command and the roll angle command of the UAV are determined.

[0119] In this embodiment, in S3-A, obtaining the second state vector of the drone through the second inertial sensor specifically includes:

[0120] The attitude quaternion of the UAV, the velocity vector of the UAV in the runway coordinate system, and the position vector of the UAV in the runway coordinate system are respectively acquired by the second inertial sensor and are spliced ​​to obtain a second state vector of the UAV;

[0121] In S3-B, based on the extended Kalman filter algorithm, the second state vector and the second position coordinate are fused to obtain the second corrected position coordinate of the aircraft carrier in the world coordinate system, which specifically includes:

[0122] S3-B1, determine the one-step state transition function;

[0123] S3-B2, obtaining a state estimation result according to the second state vector and the one-step state transfer function;

[0124] S3-B3, determining the second position coordinate as a measurement vector;

[0125] S3-B4, determine the measurement function, the process noise covariance matrix, the process noise shaping matrix, the approximately linearized measurement matrix and the measurement noise covariance matrix;

[0126] S3-B5, perform partial differentiation on the one-step state transfer function to obtain an approximate linearized state transfer matrix;

[0127] S3-B6. Use the extended Kalman filter algorithm to fuse the state estimation results and the measurement vector based on the measurement function, process noise covariance matrix, process noise shaping matrix, approximate linearized measurement matrix, measurement noise covariance matrix and approximate linearized state transfer matrix to obtain the second corrected position coordinate.

[0128] Optionally, the execution process of the Kalman filter algorithm is similar to that of the extended Kalman filter algorithm. Therefore, in S3-B, based on the Kalman filter algorithm, the first state vector and the first position coordinate are fused to obtain the first corrected position coordinate of the aircraft carrier in the world coordinate system, which specifically includes:

[0129] S3-B11, determine the one-step state transfer matrix;

[0130] S3-B12, according to the first state vector and the one-step state transfer matrix, obtain the state estimation result,

[0131] S3-B13, determining the first position coordinate as a measurement vector;

[0132] S3-B14, determine the measurement function matrix, the process noise covariance matrix, the process noise shaping matrix and the measurement noise covariance matrix;

[0133] S3-B15. Use the Kalman filter algorithm to fuse the state estimation results and the measurement vector based on the measurement function matrix, process noise covariance matrix, process noise shaping matrix, one-step state transfer matrix and measurement noise covariance matrix to obtain the first corrected position coordinates of the UAV in the runway coordinate system.

[0134] Optionally, a state estimation result is obtained based on the second state vector and the one-step state transfer function, and the state estimation result is obtained based on the first state vector and the one-step state transfer matrix by multiplying the second state vector and the one-step state transfer function, or by multiplying the first state vector and the one-step state transfer matrix, to obtain the state estimation result; or, a neural network can be used to process the second state vector and the one-step state transfer function, or to process the first state vector and the one-step state transfer matrix to obtain the state estimation result, which is not specifically limited here.

[0135] In this embodiment, in S3-B1, the one-step state transfer function is:

[0136]

[0137] in, represents the state transition function, represents quaternion multiplication, represents the process noise covariance matrix, Indicates that the gyroscope is Three-axis angle increment output within the time interval, express The modulus value, represents the speed of the aircraft in the runway coordinate system at time k-1, represents the measurement noise covariance matrix, The gyroscope is extrapolated The three-axis velocity increment output in the drone coordinate system within the time interval, Indicates the position of the aircraft in the runway coordinate system at the last moment. represents the speed of the aircraft in the runway coordinate system at time k, represents the time increment, , represents the k moment, represents the k-1 moment;

[0138] In S3-B4, the measurement function is:

[0139]

[0140] in, represents the measurement function, Indicates the position of the aircraft in the runway coordinate system obtained by visual solution;

[0141] In S3-B5, the approximate linearized state transfer matrix is:

[0142]

[0143] in, Table approximates the linearized state transfer matrix, represents the state matrix, represents the transpose of the state matrix, represents a quaternion, represents the time increment, , represents the k moment, represents the k-1 moment, represents the identity matrix;

[0144] In S3-B4, the process noise covariance matrix is:

[0145]

[0146] in, represents the k-1 time process noise covariance matrix, , represents the k-time process noise covariance matrix;

[0147] In S3-B4, the process noise shaping matrix is:

[0148]

[0149] in, represents the k-1 moment process noise shaping matrix, represents the k-1 time process noise covariance matrix, represents the angle calculated by the gyroscope, represents the identity matrix;

[0150] In S3-B4, the approximate linearized measurement matrix is:

[0151]

[0152] in, represents the approximately linearized measurement matrix, represents an approximately linearized quantity measurement, represents the transpose of the state matrix, represents the identity matrix;

[0153] In S3-B4, the measurement noise covariance matrix is:

[0154]

[0155] in, represents the measurement noise covariance matrix.

[0156] In this embodiment, in S4, the relative distance between the aircraft carrier and the UAV is calculated, and the pitch angle command and roll angle command of the UAV are determined using the following calculation formula:

[0157]

[0158] in, and Indicates the position of the aircraft carrier in the world coordinate system , Direction coordinates, and Indicates the coordinates of the drone in the world coordinate system. represents the glide angle of the drone, Indicates altitude control instructions. Indicates the pitch angle control command, Indicates the roll angle control command, represents the height scale parameter, Indicates the height feedback value, represents the height integral parameter, represents the highly differential parameter, represents the differential of height change, represents the side slip ratio parameter, represents the yaw ratio parameter, Indicates the yaw angle.

[0159] For example, since there are some defects when relying on a single visual sensor, it is necessary to fuse the state estimation results from different sources based on the Kalman filter algorithm to improve the accuracy and robustness of the positioning data solution. The state estimation results from different sources include the visual positioning results and the state estimation results of the inertial sensor, that is, the first position coordinates calculated above and the state estimation results of the aircraft carrier obtained by the inertial sensor.

[0160] For example, take the aircraft carrier as the object to be filtered, and take the position and speed of the aircraft carrier as the filter object. As the state vector, the position of the aircraft carrier As the measurement vector. Then the state transfer matrix and measurement matrix in the filtering equation are:

[0161]

[0162] in, represents the state transfer matrix, Represents the measurement matrix.

[0163] like Figure 5 As shown, Figure 5 This is a schematic diagram of the detection and positioning effect during the aircraft approach process. Figure 5 In the image, as the aircraft approaches, the size of the aircraft carrier in the image keeps changing, but the detection algorithm can adapt well to this change and continuously and stably detect the position of the aircraft carrier in the image.

[0164] For example, in S3-C1, the attitude quaternion of the drone, the velocity vector of the drone in the runway coordinate system, and the position vector of the drone in the runway coordinate system are concatenated to obtain the state vector: First, the attitude quaternion of the aircraft is selected: , the velocity vector of the UAV in the runway coordinate system and the position vector of the UAV in the runway coordinate system , concatenate to get the state vector .

[0165] Exemplarily, the relevant parameter form in the measurement function is:

[0166]

[0167] in, represents the measurement noise matrix at time k, It represents unknown and unknowable parameters that can only be estimated. It is only used here to indicate that the measurement noise matrix is ​​a 3x1 position matrix.

[0168] Optionally, and These are two key adjustable parameters of the Kalman filter, and their specific values ​​are for reference only.

[0169] S5. Obtain state feedback of the drone, calculate the difference between the state feedback and the pitch angle command and the roll angle command, and determine the pitch angle command difference and the roll angle command difference.

[0170] S6. Use the preset control law to convert the pitch angle command difference and the roll angle command difference into the rudder surface deflection angle, and guide the UAV to land through the rudder surface deflection angle.

[0171] In this embodiment, in S6, the calculation formula of the preset control law is:

[0172]

[0173] in, Indicates the elevator increment command, represents the pitch angular velocity ratio parameter, represents the pitch angular velocity, represents the pitch angle ratio parameter, represents the pitch angle, Indicates the throttle increment command, represents the speed ratio parameter, Indicates speed command, Indicates speed feedback, represents the integral proportional parameter, represents the aileron increment command, represents the roll angular velocity ratio parameter, is the rolling angular velocity, represents the roll angle ratio parameter, represents the roll angle feedback, Indicates the roll angle command.

[0174] For example, after obtaining the coordinate position of the aircraft carrier in the world coordinate system, the longitudinal pitch angle command and the lateral roll angle command can be calculated by calculating the relative distance between the aircraft carrier and the UAV, and the commands are calculated to control the aircraft to perform an automatic approach and landing.

[0175] Example 2

[0176] This embodiment provides a carrier aircraft visual landing guidance simulation system, and its construction and design can be described as follows:

[0177] 1) Software interface design

[0178] In the host computer, the main task is to design a Vega Prime display and control interface based on MFC, while in the slave computer, the main task is to run a carrier-based aircraft visual landing guidance method in Example 1, that is, to run a visual detection and positioning algorithm.

[0179] The virtual scene display part and the lower computer visual positioning screen part can be designed as follows:

[0180] Virtual scene display part: The virtual scene display part mainly uses Vega Prime software to realize the vivid and intuitive display of the real-time model of the drone. You can continue to refer to Figure 5 ,The images captured by the camera carried by the UAV will be sent to the lower computer in real time for visual positioning.

[0181] The lower computer visual positioning screen part: Figure 6 and Figure 7 As shown, Figure 6 This is a schematic diagram of the recognition results of the lower computer running the visual detection and positioning algorithm. Figure 7 This is another schematic diagram of the recognition results of the lower computer running the visual detection and positioning algorithm.

[0182] exist Figure 4 , Figure 6 and Figure 7 In the figure, roll, pitch, and yaw are the roll angle, pitch angle, and yaw angle of the aircraft sent by the host computer, respectively. Pos_x, pox_y, and pos_z are the three-axis coordinates (x-axis, y-axis, and z-axis) of the aircraft carrier in the world coordinate system calculated by the visual detection and positioning algorithm.

[0183] 2) Model building

[0184] The aircraft model and related functional modules are built in the Matlab software in the host computer, and the system model is built using the Matlab / Simulink module.

[0185] 3) Communication module design

[0186] The communication of the entire simulation verification platform is divided into three parts: the host computer Simulink sends to the host computer MFC, the host computer MFC sends to the lower computer vision platform, and the lower computer vision platform sends to the host computer Simulink. The communication of these three parts all uses the UDP mechanism for data exchange. The data packet design, host computer UDP communication module, and lower computer UDP communication module during data exchange are shown below:

[0187] like Figure 8 As shown, Figure 8 The communication diagram of the simulation verification platform. Under the MFC framework, the opencv library function is called to capture the displayed scene screen. On the one hand, it is saved in the specified folder of the upper computer, and on the other hand, it is transmitted to the lower computer through UDP communication for calculation and processing. When the lower computer receives the image sent by the upper computer MFC, it runs the visual positioning algorithm in real time to solve the current position of the aircraft. The solved position is then encoded and sent directly to the upper computer Simulink through UDP communication. After decoding in the upper computer Simulink, the visually positioned aircraft position is obtained. Finally, Simulink sends the visual positioning position to the upper computer MFC through UDP communication, thus forming a closed loop. Fig. 9 , Fig.10 and Fig.11 As shown, Fig. 9 Schematic diagram of the trajectory of the UAV and aircraft carrier in the visual closed-loop landing simulation. Fig.10Schematic diagram of the trajectory response of the UAV and aircraft carrier in the visual closed-loop landing simulation. Fig.11 Another schematic diagram of the trajectory response of the UAV and the aircraft carrier in the visual closed-loop landing simulation.

[0188] In addition, image transmission adopts packet transmission. Since the size of a data packet sent by UDP communication is limited to 65507 bytes at a time, and the memory occupied by an image is generally much larger than 65507 bytes, the image pixel information is first encapsulated into multiple data packets, and then transmitted in sequence through UDP communication, and finally the multiple data packets are combined into the information of an image at the receiving end.

[0189] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A carrier-based aircraft visual landing guidance method, characterized in that: The method comprises: S1. Take a screenshot of the aircraft carrier model in the visual software to obtain an aircraft carrier data set, and use the aircraft carrier data set to train the YOLOv8 model to obtain a trained YOLOv8 model; S2, obtaining image data of the target aircraft carrier through a drone equipped with a camera; S3. When the target aircraft carrier image data is taken by the drone at a distance greater than a preset distance threshold from the aircraft carrier, the trained YOLOv8 model is used to identify the target aircraft carrier image data, obtain the coordinate position of the aircraft carrier in the target aircraft carrier image data, and obtain the first position coordinates of the aircraft carrier in the world coordinate system based on a conversion formula; When the target aircraft carrier image data is taken by the UAV at a distance less than or equal to a preset distance threshold from the aircraft carrier, a runway line in the target aircraft carrier image data is identified, and the runway line is positioned by using an identical deformation method to obtain a second position coordinate of the UAV in the runway coordinate system; S4. Calculate the relative distance between the aircraft carrier and the UAV according to the first position coordinates and the second position coordinates, and determine the pitch angle command and the roll angle command of the UAV; The calculation formula used to calculate the relative distance between the aircraft carrier and the UAV and determine the pitch angle command and roll angle command of the UAV is: in, and Indicates the position of the aircraft carrier in the world coordinate system , Direction coordinates, and Indicates the coordinates of the drone in the world coordinate system. represents the glide angle of the drone, Indicates altitude control instructions. Indicates the pitch angle control command, Indicates the roll angle control command, represents the height scale parameter, Indicates the height feedback value, represents the height integral parameter, represents the highly differential parameter, represents the differential of height change, represents the side slip ratio parameter, represents the yaw ratio parameter, represents the yaw angle; S5, obtaining state feedback of the drone, calculating the difference between the state feedback and the pitch angle command and the roll angle command, and determining the pitch angle command difference and the roll angle command difference; S6. Using a preset control law, converting the pitch angle command difference and the roll angle command difference into a rudder surface deflection angle, and guiding the UAV to land by using the rudder surface deflection angle; The calculation formula of the preset control law is: in, Indicates the elevator increment command, represents the pitch angular velocity ratio parameter, represents the pitch angular velocity, represents the pitch angle ratio parameter, represents the pitch angle, Indicates the throttle increment command, represents the speed ratio parameter, Indicates speed command, Indicates speed feedback, represents the integral proportional parameter, represents the aileron increment command, represents the roll angular velocity proportional parameter, is the rolling angular velocity, represents the roll angle ratio parameter, Indicates roll angle feedback; Before S3, the method further includes: Acquire the height of the drone according to the sensor of the drone, and calculate the image depth of the picture taken by the camera carried by the drone; The calculation formula used to calculate the image depth of the picture taken by the camera carried by the drone is: in, Indicates the image depth of the picture taken by the camera on the drone. Indicates the vertical coordinate of the aircraft carrier in the world coordinate system, Indicates the vertical coordinate of the drone in the world coordinate system, Represents the rotation matrix from the drone coordinate system to the world coordinate system, Represents the coordinate position of the camera in the world coordinate system, represents the homogeneous coordinates in the pixel coordinate system, where , Represents the rotation matrix from the world coordinate system to the camera coordinate system, Represents the camera intrinsic parameter matrix; In S3, the conversion formula is: in, Indicates the position coordinates of the aircraft carrier in the world coordinate system. Indicates the position coordinates of the drone in the world coordinate system, Represents the rotation matrix from the camera coordinate system to the drone coordinate system.

2. The method according to claim 1, characterized in that Before S4, the method further includes: S3-A, obtaining a first state vector of the aircraft carrier through a first inertial sensor, and obtaining a second state vector of the UAV through a second inertial sensor; S3-B, based on the Kalman filter algorithm, fusing the first state vector and the first position coordinate to obtain a first corrected position coordinate of the aircraft carrier in the world coordinate system; Alternatively, based on an extended Kalman filter algorithm, the second state vector and the second position coordinate are fused to obtain a second corrected position coordinate of the aircraft carrier in a world coordinate system; The S4 specifically includes: According to the first corrected position coordinates and the second corrected position coordinates, the relative distance between the aircraft carrier and the UAV is calculated, and the pitch angle command and the roll angle command of the UAV are determined.

3. The method according to claim 2, characterized in that In S3-A, obtaining a second state vector of the drone by using a second inertial sensor specifically includes: The attitude quaternion of the UAV, the velocity vector of the UAV in the runway coordinate system, and the position vector of the UAV in the runway coordinate system are respectively acquired by a second inertial sensor and are spliced ​​to obtain a second state vector of the UAV; In S3-B, based on the extended Kalman filter algorithm, the second state vector and the second position coordinate are fused to obtain the second corrected position coordinate of the aircraft carrier in the world coordinate system, which specifically includes: S3-B1, determine the one-step state transition function; S3-B2, obtaining a state estimation result according to the second state vector and the one-step state transfer function; S3-B3, determining the second position coordinate as a measurement vector; S3-B4, determine the measurement function, the process noise covariance matrix, the process noise shaping matrix, the approximately linearized measurement matrix and the measurement noise covariance matrix; S3-B5, performing partial differentiation on the one-step state transfer function to obtain an approximate linearized state transfer matrix; S3-B6. Use the extended Kalman filter algorithm to fuse the state estimation result and the measurement vector based on the measurement function, the process noise covariance matrix, the process noise shaping matrix, the approximate linearized measurement matrix, the measurement noise covariance matrix and the approximate linearized state transfer matrix to obtain the second corrected position coordinate.

4. The method according to claim 3, characterized in that In S3-B1, the one-step state transfer function is: in, represents the state transition function, represents quaternion multiplication, represents the process noise covariance matrix, Indicates that the gyroscope is Three-axis angle increment output within the time interval, express The modulus value, represents the speed of the aircraft in the runway coordinate system at time k-1, represents the measurement noise covariance matrix, The gyroscope is extrapolated The three-axis velocity increment output in the drone coordinate system within the time interval, Indicates the position of the aircraft in the runway coordinate system at the last moment. represents the speed of the aircraft in the runway coordinate system at time k, represents the time increment, , represents the k moment, represents the k-1 moment; In S3-B4, the measurement function is: in, represents the measurement function, Indicates the position of the aircraft in the runway coordinate system obtained by visual solution; In S3-B5, the approximate linearized state transfer matrix is: in, Table approximates the linearized state transfer matrix, represents the state matrix, represents the transpose of the state matrix, represents a quaternion, represents the identity matrix; In S3-B4, the process noise covariance matrix is: in, represents the k-1 time process noise covariance matrix, , represents the k-time process noise covariance matrix; In S3-B4, the process noise shaping matrix is: in, represents the k-1 moment process noise shaping matrix, represents the k-1 time process noise covariance matrix, Indicates the angle calculated by the gyroscope; In S3-B4, the approximate linearized measurement matrix is: in, represents the approximately linearized measurement matrix, represents an approximately linearized quantity measurement, Represents the transpose of the state matrix; In S3-B4, the measurement noise covariance matrix is: in, represents the measurement noise covariance matrix.

Images