Unmanned aerial vehicle flight attitude and load automatic control method based on target identification and tracking

Abstract

The invention provides an unmanned aerial vehicle flight attitude and load automatic control method based on target identification and tracking. The method comprises the following steps: 1, planning inspection of a monitoring area reached by an unmanned aerial vehicle flight route and setting flight height and camera visual angle; 2, national standard cascading video streaming is carried out; step 3, carrying out target selection on the identified object by a single target tracking algorithm; and 4, tracking the target and continuously identifying a target frame. And 5, calculating a direction included angle and a linear distance from the unmanned aerial vehicle to the target according to a projective geometry principle. Sixthly, the flight direction and the flight speed of the unmanned aerial vehicle are automatically adjusted. And 7, calculating the flight direction and speed, and adjusting the flight attitude and load of the unmanned aerial vehicle in real time to control the flight of the tracking target. The method is high in accuracy, high in practicability and high in commercial value.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and computer vision technology, and specifically is a method for automatic control of the flight attitude and load of a UAV based on target recognition and tracking. Background Art

[0002] As drones are increasingly used and can adapt to various dangerous and complex environments, drones can reach difficult or dangerous areas and complete scenarios that other tools cannot perform. Drone technology can replace humans to reach dangerous areas and complete some important tasks, reducing the risk of harm to human life.

[0003] The UAV tracking single target flight mainly uses the target real-time tracking algorithm to track the single target in the image. After obtaining the center position of the tracked target in the image, the distance and angle between the tracked object and the UAV are obtained through the projective geometry principle technology, so as to control the flight navigation and flight speed of the UAV and effectively control the focal length of the load. Experiments have proved that this algorithm design can effectively enable the UAV to track the target in real time. Summary of the invention

[0004] This method proposes a method for automatic control of the flight attitude and load of UAV based on target recognition and tracking. The algorithm is used to monitor the center coordinate position of the image of the tracked object in real time, and the projective geometry principle is used to calculate the distance and angle between the real tracked object and the UAV, and the flight navigation and flight speed of the UAV are changed in real time to facilitate the purpose of timely tracking the target.

[0005] In order to achieve the above-mentioned purpose, the present invention provides a method for automatic control of UAV flight attitude and load based on target recognition and tracking, comprising the following steps:

[0006] S1. Plan the flight route of the drone from the airport to the preset monitoring area for inspection, and set the flight altitude and camera angle of the flight route setting parameters in the monitoring area for inspection;

[0007] S2. The cascaded video gateway decodes the video stream to obtain continuous image frames;

[0008] S3. Use the single target tracking algorithm Siamese Mask to select the target of the bounding box of the identified object in the target area of ​​the decoded image frame;

[0009] S4. Tracking the selected target appearing in the continuous decoded images and continuously identifying the target frame;

[0010] S5. Calculate the center area of ​​the target frame, and calculate the direction angle and straight-line distance from the drone to the target based on the principle of projective geometry;

[0011] S6. Automatically adjust the flight direction and speed of the drone based on the angle and distance calculated, and set the image size of the target in the picture by controlling the focal length of the camera;

[0012] S7. After the UAV flies to the target area, the algorithm identifies the tracking result, calculates the flight direction and speed based on the principle of projective geometry, and adjusts the flight attitude and load control of the UAV in real time, so that the UAV can accurately track the target and fly;

[0013] S8. The UAV automatically tracks the target and the flight mission ends.

[0014] Beneficial effects: The method for automatic control of UAV flight attitude and load based on target recognition and tracking described in the present invention takes advantage of the UAV's basically obstacle-free flight in the airspace, and can fly to areas that are difficult for people to reach or unwilling to reach to perform work tasks. After discovering the tracked target, the target flight tracking mechanism is triggered to perform UAV tracking flight, and the background command personnel can arrange to handle the corresponding load device at an appropriate height distance. This patent has great practical and commercial value. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a flow chart of the method steps of the present invention; DETAILED DESCRIPTION

[0016] In order to more clearly describe the technical solution in the example of the present invention, the present invention will be described in detail below in conjunction with the drawings in the embodiment. The specific embodiment described in the present invention, as well as other embodiments based on the present invention, all belong to the protection scope of the present invention.

[0017] The purpose of the present invention is to provide a method for automatic control of the flight attitude and load of a UAV based on target recognition and tracking. The method mainly includes 7 modules, which are respectively the flight of the UAV to the inspection area, the decoding of the cascaded video stream into an image sequence, the single target tracking algorithm Siamese Mask to select the target of the bounding box of the identified object, the continuous image sequence to track the target rectangular frame, the calculation of the actual route angle and distance between the tracked object and the UAV, and the control of the flight direction and speed according to the angle distance between the real-time tracked object and the UAV.

[0018] like Figure 1 Shown is a flowchart of the steps of a method for automatic control of UAV flight attitude and load based on target recognition and tracking according to an example of the present invention.

[0019] An embodiment of the present invention provides a method for automatically controlling the flight attitude and load of a UAV based on target recognition and tracking, comprising the following steps:

[0020] Step S1. Plan the flight route of the drone from the airport to the preset monitoring area for inspection, and set the flight altitude and camera viewing angle of the flight route setting parameters for inspection in the monitoring area;

[0021] Step S2. The cascaded video gateway decodes the video stream to obtain a continuous sequence of image frames;

[0022] Step S3. Use the single target tracking algorithm Siamese Mask to select the target of the bounding box of the identified object in the target area of ​​the decoded image frame;

[0023] Step S4. Tracking the selected targets appearing in the continuous decoded images and continuously marking the target frames;

[0024] Step S5. Calculate the center area of ​​the target frame, and calculate the direction angle and straight-line distance from the drone to the target based on the principle of projective geometry;

[0025] Step S6. Automatically adjust the flight direction and speed of the drone based on the angle and distance calculated, and set the image size of the target in the picture by controlling the focal length of the camera;

[0026] S61. According to the principles of projective geometry, the way a point in three-dimensional space is imaged on a camera can be constrained by equations. The specific constraints are as follows:

[0027] p i =λHP c (1)

[0028] Among them, p i is the coordinate point on the imaging plane, p i =(p ix p iy 1) T , the three-dimensional coordinates in real space are P = (P cX P cY P cZ 1) T , this notation is the representation of homogeneous coordinates. Homogeneous coordinates are a method of representing an n-dimensional vector as an n+1-dimensional vector, mainly used in projective geometry. This method solves problems that cannot be handled in Euclidean geometry by adding an extra dimension, such as the intersection of parallel lines at infinity. H is the projection matrix;

[0029]

[0030] S62. The homography matrix H is constrained by its own camera intrinsic parameters and extrinsic parameters, where the intrinsic parameter is the matrix K and the extrinsic parameters are the matrices R and T. The specific representation is as follows:

[0031]

[0032] The matrix of the camera external parameters R is expressed as:

[0033]

[0034] T=(t x t y t z 1) T (5)

[0035] Among them, the angles between the real space coordinate system and the camera coordinate system in the three directions of X, Y, and Z are α, β, and γ respectively, and t is the distance vector between the origins of the coordinate systems. x ,t y ,t z ;

[0036] S64. Then the homography matrix H is expressed as follows:

[0037]

[0038] Then, substituting formula (6) into (1) yields the following:

[0039]

[0040] S65. By p i The coordinate point information of the imaging plane is p i =(p ix p iy 1) T , then substitute it into formula (7) to obtain:

[0041]

[0042] S66. Assuming that the 3D space coordinate system and the camera coordinate system are the same coordinate system, then the angles between the three coordinate systems α, β, and γ are all 0°, and the angle between the origins of the other two coordinate systems is t x ,t y ,t z The translation size is also 0. Then

[0043]

[0044] Then the three-dimensional space point imaging is only related to the internal parameters and the flight altitude:

[0045]

[0046] S67. Generally, the distortion s=0 is determined, then:

[0047]

[0048] Then λ=1 / P cZ , when setting c x and c y is the coordinate of the midpoint of the image, then c x =c y = 0. Then

[0049]

[0050] Among them, P cZ is the information from the imaging point to the origin of the coordinate system of the drone’s camera, which can be considered as the height information of the drone, f x and f y is the internal parameter of the camera.

[0051] S68.f represents the focal length, in mm, where f x and f y The calculation formula is as follows:

[0052]

[0053] Among them, 1 / dx and 1 / dy represent the number of pixels per mm in the x and y directions of the image respectively. If the size of the photosensitive material is L in length and W in width, the pixel resolution in the direction is R x , R y , then 1 / dx=L / Rx, calculate dy in the same way;

[0054] S69. Substituting formula (13) into formula (11) yields:

[0055]

[0056] Then, after obtaining the accurate length and height of the horizontal and vertical coordinates, the straight-line distance and direction between the drone and the tracked object can be accurately calculated;

[0057] Step S7. After the UAV flies to the target area, the algorithm identifies the tracking result, calculates the flight direction and speed based on the principle of projective geometry, and adjusts the flight attitude and load control of the UAV in real time, so that the UAV can accurately track the target and fly;

[0058] Step S8: The UAV automatically tracks the target and the flight mission ends.

Claims

1. A video concentration method for multi-target tracking, comprising the following steps: (1) Plan the flight route of the drone from the airport to the preset monitoring area for inspection, and set the flight altitude and camera angle of the flight route setting parameters in the monitoring area for inspection; (2) The cascaded video gateway decodes the video stream to obtain continuous image frames; (3) Using the single target tracking algorithm Siamese Mask to select the target in the target area of ​​the decoded image frame and the bounding box of the identified object; (4) Tracking the selected targets appearing in the continuous decoded images and continuously marking the target frames; (5) Calculate the center area of ​​the target frame and calculate the direction angle and straight-line distance from the UAV to the target based on the principles of projective geometry; (6) Automatically adjust the flight direction and speed of the drone based on the angle and distance calculated, and set the image size of the target in the picture by controlling the focal length of the camera; (7) After the UAV flies to the target area, the algorithm identifies the tracking results and calculates the flight direction and speed based on the principle of projective geometry to adjust the flight attitude and load control of the UAV in real time, so that the UAV can accurately track the target and fly; (8) The UAV automatically tracks the target and the flight mission ends.

Images