Multi-rotor unmanned aerial vehicle target identification tracking method and platform

Abstract

The invention relates to the technical field of unmanned aerial vehicles, and discloses a multi-rotor unmanned aerial vehicle target recognition tracking method and platform, and the method comprises the following steps: constructing a data set of a target to be recognized, and training a preset target to be recognized; the flight route of the multi-rotor unmanned aerial vehicle is planned, so that the multi-rotor unmanned aerial vehicle flies to a designated position and collects images; a ground operator selects a target needing to be tracked and sends out an identification tracking instruction, the multi-rotor unmanned aerial vehicle transmits a collected video image to a tracking algorithm board card, and the target is identified and selected according to the selected target needing to be identified and a target identification algorithm; the target recognition algorithm transmits target recognition frame information to the tracking algorithm board card, and the tracking algorithm board card controls the two-axis dual-light lens and the multi-rotor unmanned aerial vehicle to track the target. Through the two-axis dual-light lens, the problem that the target tracking effect is poor at night or when the illumination intensity is underground is solved, the adaptability is high, and the method is used for miniature and small-sized multi-rotor unmanned aerial vehicles.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles (UAVs), and in particular to a target identification and tracking method and platform for a multi-rotor UAV. Background Art

[0002] As an important branch of low-altitude unmanned systems, multi-rotor drones have been widely used in civil surveying and mapping, logistics and transportation, emergency rescue and other fields due to their core advantages such as vertical take-off and landing, hovering operation, and three-dimensional maneuverability.

[0003] With the widespread adoption of machine vision-based technologies such as target identification and tracking, combined with the advantage of drones' wide aerial field of view, they can better perform tasks such as patrolling, identification, aerial photography, and target identification and strike. Furthermore, drone reconnaissance and target identification and tracking have become a popular task. Existing technologies have enabled drones to autonomously identify and track targets based on the identified targets. For example, patent CN116430893A provides a drone target identification and tracking method, system, and computer device. However, target identification and tracking present the following practical challenges:

[0004] (1) The existing technology process is complicated and requires manual selection of tracking operations after identifying the target. When multiple targets appear, there is no effective target selection strategy, and fully autonomous identification and tracking is not achieved. In addition, it is difficult to identify the target at night or in dim scenes.

[0005] (2) Existing drone identification and tracking platforms are often adapted to specific drones and cannot be widely applied to different drones.

[0006] (3) During the target tracking process, the lens gimbal rotates with the target, but the UAV's motion trajectory is inconsistent with the lens, resulting in occlusion of UAV components and affecting target tracking. Summary of the Invention

[0007] In order to solve the above problems, the technical solution adopted by the present invention is:

[0008] A multi-rotor UAV target identification and tracking platform comprises a multi-rotor UAV: ​​used for flying and tracking a target;

[0009] Two-axis dual-light lens: fixedly installed on the multi-rotor drone, used to collect image information around the multi-rotor drone;

[0010] Flight control module: installed in the multi-rotor drone to control the movement of the multi-rotor drone;

[0011] Tracking algorithm board: The two-axis dual-optical lens and the flight control module are electrically connected to the tracking algorithm board through the serial port. The tracking algorithm board is used to control the flight control module and the two-axis dual-optical lens, allowing the multi-rotor drone to track the target in real time;

[0012] The image transmission module sky unit is electrically connected to the tracking algorithm board via the serial port, and is used to receive image information from the two-axis bifocal lens and send image information to the image transmission module ground unit.

[0013] Image transmission module ground terminal: receives image information sent by the image transmission module sky terminal through wireless transmission and sends commands to the image transmission module sky terminal;

[0014] Ground station: The ground station is connected to the ground end of the image transmission module through a serial port. It is used to display the received image information in real time and send commands to the ground end of the image transmission module.

[0015] A multi-rotor UAV target identification and tracking platform provides a multi-rotor UAV target identification and tracking method, comprising the following steps:

[0016] S1. Build the target dataset required for recognition and train the preset targets required for recognition;

[0017] S2. Plan the flight route of the multi-rotor UAV, make the multi-rotor UAV fly to the designated location and collect images;

[0018] S3: The ground operator selects the target to be tracked and issues an identification and tracking command. The multi-rotor drone transmits the collected video images to the tracking algorithm board. Based on the target to be identified, the target recognition algorithm identifies and selects the target.

[0019] S4. The target recognition algorithm transmits the target recognition frame information to the tracking algorithm board, and the tracking algorithm board controls the two-axis dual-light lens and the multi-rotor drone to track the target.

[0020] Furthermore, the step S1 includes the following sub-steps:

[0021] S1.1. Collect a video set of the tracked target using a multi-rotor drone, using a visible light lens and an infrared imager lens of a two-axis bifocal lens;

[0022] S1.2. Based on the images collected in S1.1, create visible light and infrared datasets respectively, label the targets and use them to train the target recognition algorithm.

[0023] Furthermore, step S2 includes the following sub-steps:

[0024] S2.1. The ground station plans the route and unlocks the multi-rotor drone, which then automatically flies to the designated location and hovers or patrols.

[0025] S2.2. The multi-rotor drone collects images and transmits them to the ground station via an image transmission link. The ground station obtains the images collected by the multi-rotor drone in real time and selects visible light imaging or infrared imaging as needed.

[0026] Furthermore, step S3 includes the following sub-steps:

[0027] S3.1. The ground station selects the target to be identified and issues an identification and tracking command. The two-axis bifocal lens transmits the captured video stream to the tracking algorithm board. The algorithm identifies the target, selects it, and tracks it.

[0028] S3.2. For S3.1, if you need to modify the target to be identified, to identify a new target, just check the target; to cancel the identification of a target, just uncheck the target.

[0029] Furthermore, step S4 includes the following sub-steps:

[0030] S4.1. The target recognition algorithm inputs the target recognition frame coordinate information into the tracking algorithm board;

[0031] S4.2. For S4.1, when there are multiple targets to be identified, input the coordinate information of the target identification frame closest to the image center into the tracking algorithm board;

[0032] S4.3. The tracking algorithm board continuously updates the tracking frame based on the features and transmits the target miss distance information to the servo motor for the two-axis bifocal lens. The servo motor controls the two-axis bifocal lens to always keep the target in the center of the image.

[0033] S4.4. The servo motor of the two-axis bifocal lens inputs the azimuth angle information and azimuth velocity information to the tracking algorithm board. The tracking algorithm board adjusts the azimuth heading of the multi-rotor drone through the flight control module to align the multi-rotor drone's heading with the two-axis bifocal lens.

[0034] S4.5. When the target is lost or disappears, the target recognition algorithm is activated, waiting for the target to appear in the field of view and repeating S4.1;

[0035] S4.6. For S4.4, if the multi-rotor drone has a support leg, to prevent the two-axis bifocal lens from being blocked by the support leg during target tracking, thus affecting target tracking, the multi-rotor drone should be aligned with the two-axis bifocal lens to prevent the support leg from appearing in the tracking lens.

[0036] S4.7. When you need to exit tracking mode, select Exit Identification Tracking on the ground station. The two-axis dual-light lens and multi-rotor drone will stop tracking. If you need to track again, repeat S3.1.

[0037] Beneficial effects of the present invention:

[0038] 1. The dual-axis dual-light lens uses both visible light and infrared imaging modes to improve the problem of poor target tracking at night or in low light conditions.

[0039] 2. Suitable for micro and small multi-rotor drones, its high adaptability improves the problem that existing identification and tracking platforms are only applicable to specific drones.

[0040] 3. Improved the problem that the existing recognition and tracking platform is easily blocked by drone components. The camera is controlled to rotate while adjusting the drone to prevent the drone's own components from affecting tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the invention.

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 It is a structural schematic diagram of the present invention;

[0044] Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0045] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0046] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by people with ordinary skills in the field to which this disclosure belongs. The words "first", "second" and similar words used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0047] A multi-rotor UAV target identification and tracking platform comprises a multi-rotor UAV: ​​used for flying and tracking a target;

[0048] Two-axis dual-light lens: fixedly installed on the multi-rotor drone, used to collect image information around the multi-rotor drone;

[0049] Flight control module: installed in the multi-rotor drone to control the movement of the multi-rotor drone;

[0050] Tracking algorithm board: The two-axis dual-optical lens and the flight control module are electrically connected to the tracking algorithm board through the serial port. The tracking algorithm board is used to control the flight control module and the two-axis dual-optical lens, allowing the multi-rotor drone to track the target in real time;

[0051] The image transmission module sky unit is electrically connected to the tracking algorithm board via the serial port, and is used to receive image information from the two-axis bifocal lens and send image information to the image transmission module ground unit.

[0052] Image transmission module ground terminal: receives image information sent by the image transmission module sky terminal through wireless transmission and sends commands to the image transmission module sky terminal;

[0053] Ground station: The ground station is connected to the ground end of the image transmission module through a serial port. It is used to display the received image information in real time and send commands to the ground end of the image transmission module.

[0054] A multi-rotor UAV target identification and tracking platform provides a multi-rotor UAV target identification and tracking method, comprising the following steps:

[0055] S1. Build the target dataset required for recognition and train the preset targets required for recognition;

[0056] S2. Plan the flight route of the multi-rotor UAV, make the multi-rotor UAV fly to the designated location and collect images;

[0057] S3: The ground operator selects the target to be tracked and issues an identification and tracking command. The multi-rotor drone transmits the collected video images to the tracking algorithm board. Based on the target to be identified, the target recognition algorithm identifies and selects the target.

[0058] S4. The target recognition algorithm transmits the target recognition frame information to the tracking algorithm board, and the tracking algorithm board controls the two-axis dual-light lens and the multi-rotor drone to track the target.

[0059] Furthermore, the step S1 includes the following sub-steps:

[0060] S1.1. Collect a video set of the tracked target using a multi-rotor drone, using a visible light lens and an infrared imager lens of a two-axis bifocal lens;

[0061] S1.2. Based on the images collected in S1.1, create visible light and infrared datasets respectively, label the targets and use them to train the target recognition algorithm.

[0062] Furthermore, step S2 includes the following sub-steps:

[0063] Specifically, a set of targets to be identified is selected, a video set of the targets to be identified is obtained through aerial photography, and the video set is divided into image data sets at intervals of 3 seconds. The targets are labeled using labeling software. The dataset labeling software used in this example is LabelImg. The targets to be tracked include people, vehicles, trees, and houses. The drone used is a four-axis multi-rotor drone with a wheelbase of 80 cm and a maximum mounting weight of 500 grams. The communication protocol is the Mavlink2.0 protocol. The target recognition algorithm used is YOLOv5. YOLOv5 has the advantages of high recognition accuracy and fast recognition speed, and can meet the requirements of real-time recognition. The YOLOv5 target recognition algorithm is trained through the labeled dataset, and the model is deployed to the tracking algorithm board.

[0064] S2.1. The ground station plans the route and unlocks the multi-rotor drone, which then automatically flies to the designated location and hovers or patrols.

[0065] S2.2. The multi-rotor drone collects images and transmits them to the ground station via an image transmission link. The ground station obtains the images collected by the multi-rotor drone in real time and selects visible light imaging or infrared imaging as needed.

[0066] Specifically, the drone is connected through a ground station, and waypoints are planned on the ground station to ensure that the desired tracking target can be observed at the location the drone flies to. The ground station unlocks the drone, and the drone flies to the destination along the route. In this example, the drone flies to a position 10 meters away from the roadside and hovers at an altitude of 50 meters. The ground station obtains the images collected by the drone through the image transmission link.

[0067] Furthermore, step S3 includes the following sub-steps:

[0068] S3.1. The ground station selects the target to be identified and issues an identification and tracking command. The two-axis bifocal lens transmits the captured video stream to the tracking algorithm board. The algorithm identifies the target, selects it, and tracks it.

[0069] S3.2: For S3.1, if you need to modify the target to be identified, you only need to check the target to identify a new target, and you only need to uncheck the target to cancel the identification of a target;

[0070] Specifically, in this example, people and vehicles are identified as targets. Select people and vehicles and uncheck trees and houses. After selecting, use the visible light lens and click Identify and Track. The drone begins target recognition, identifies the target, and transmits the target information closest to the center of the two-axis dual-light lens to the tracking algorithm board.

[0071] Furthermore, step S4 includes the following sub-steps:

[0072] S4.1. The target recognition algorithm inputs the target recognition frame coordinate information into the tracking algorithm board;

[0073] S4.2. For S4.1, when there are multiple targets to be identified, input the coordinate information of the target identification frame closest to the image center into the tracking algorithm board;

[0074] S4.3. The tracking algorithm board continuously updates the tracking frame based on the features and transmits the target miss distance information to the servo motor for the two-axis bifocal lens. The servo motor controls the two-axis bifocal lens to always keep the target in the center of the image.

[0075] S4.4. The servo motor of the two-axis bifocal lens inputs the azimuth angle information and azimuth velocity information to the tracking algorithm board. The tracking algorithm board adjusts the azimuth heading of the multi-rotor drone through the flight control module to align the multi-rotor drone's heading with the two-axis bifocal lens.

[0076] S4.5. When the target is lost or disappears, the target recognition algorithm is activated, waiting for the target to appear in the field of view and repeating S4.1;

[0077] S4.6. For S4.4, if the multi-rotor drone has a support leg, to prevent the two-axis bifocal lens from being blocked by the support leg during target tracking, thus affecting target tracking, the multi-rotor drone should be aligned with the two-axis bifocal lens to prevent the support leg from appearing in the tracking lens.

[0078] S4.7. When you need to exit tracking mode, select Exit Identification Tracking on the ground station. The two-axis dual-light lens and multi-rotor drone will stop tracking. If you need to track again, repeat S3.1.

[0079] Specifically, the tracking algorithm board obtains the recognition frame features according to the recognition frame information input by the recognition algorithm, and continuously updates the tracking frame information of the next frame image, and continuously transmits the miss amount to the servo motor of the two-axis dual-optical lens. The servo motor of the two-axis dual-optical lens adjusts the position of the two-axis dual-optical lens according to the target miss amount, so that the two-axis dual-optical lens is always aimed at the target, and transmits the azimuth information and azimuth velocity back to the tracking algorithm board. The tracking algorithm board transmits the azimuth velocity information to the deployed drone control program. After receiving the azimuth velocity information, the control program uses the MAVSDK library function written in C++ language to send instructions to the flight control module to control the azimuth angle of the drone, keep the drone movement consistent with the lens, and the drone tracks the target vehicle until the vehicle disappears from the field of view. The drone loses the target and keeps hovering. When the next required tracking target appears in the field of view, S3.1 is repeated.

[0080] (1) Unless otherwise defined, in the embodiments of the present disclosure and the accompanying drawings, the same reference numerals represent the same meanings.

[0081] (2) In the drawings of the embodiments of the present disclosure, only the structures related to the embodiments of the present disclosure are involved, and other structures can refer to the general design.

[0082] (3) For the sake of clarity, components or regions are exaggerated in the drawings used to describe embodiments of the present disclosure. It is understood that when an element is referred to as being “on” or “under” another element, the element may be “directly on” or “under” the other element, or intervening elements may be present.

[0083] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A multi-rotor UAV target recognition and tracking platform, characterized by: Including multi-rotor drones: used for flying and tracking targets; Two-axis dual-light lens: fixedly installed on the multi-rotor drone, used to collect image information around the multi-rotor drone; Flight control module: installed in the multi-rotor drone to control the movement of the multi-rotor drone; Tracking algorithm board: The two-axis dual-optical lens and the flight control module are electrically connected to the tracking algorithm board through the serial port. The tracking algorithm board is used to control the flight control module and the two-axis dual-optical lens, allowing the multi-rotor drone to track the target in real time; The image transmission module sky unit is electrically connected to the tracking algorithm board via the serial port, and is used to receive image information from the two-axis bifocal lens and send image information to the image transmission module ground unit. Image transmission module ground terminal: receives image information sent by the image transmission module sky terminal through wireless transmission and sends commands to the image transmission module sky terminal; Ground station: The ground station is connected to the ground end of the image transmission module through a serial port. It is used to display the received image information in real time and send commands to the ground end of the image transmission module.

2. A multi-rotor UAV target recognition and tracking platform according to claim 1, providing a multi-rotor UAV target recognition and tracking method, characterized in that: The following steps are involved: S1. Build the target dataset required for recognition and train the preset targets required for recognition; S2. Plan the flight route of the multi-rotor UAV, make the multi-rotor UAV fly to the designated location and collect images; S3: The ground operator selects the target to be tracked and issues an identification and tracking command. The multi-rotor drone transmits the collected video images to the tracking algorithm board. Based on the target to be identified, the target recognition algorithm identifies and selects the target. S4. The target recognition algorithm transmits the target recognition frame information to the tracking algorithm board, and the tracking algorithm board controls the two-axis dual-light lens and the multi-rotor drone to track the target.

3. The multi-rotor UAV target recognition and tracking method according to claim 2, characterized in that: The step S1 includes the following sub-steps: S1.

1. Collect a video set of the tracked target using a multi-rotor drone, using a visible light lens and an infrared imager lens of a two-axis bifocal lens; S1.

2. Based on the images collected in S1.1, create visible light and infrared datasets respectively, label the targets and use them to train the target recognition algorithm.

4. The multi-rotor UAV target recognition and tracking method according to claim 2, characterized in that: The step S2 includes the following sub-steps: S2.

1. The ground station plans the route and unlocks the multi-rotor drone, which then automatically flies to the designated location and hovers or patrols. S2.

2. The multi-rotor drone collects images and transmits them to the ground station via an image transmission link. The ground station obtains the images collected by the multi-rotor drone in real time and selects visible light imaging or infrared imaging as needed.

5. The multi-rotor UAV target recognition and tracking method according to claim 2, characterized in that: The step S3 includes the following sub-steps: S3.

1. The ground station selects the target to be identified and issues an identification and tracking command. The two-axis bifocal lens transmits the captured video stream to the tracking algorithm board. The algorithm identifies the target, selects it, and tracks it. S3.

2. For S3.1, if you need to modify the target to be identified, to identify a new target, just check the target; to cancel the identification of a target, just uncheck the target.

6. The multi-rotor UAV target recognition and tracking method according to claim 5, characterized in that: The step S4 includes the following sub-steps: S4.

1. The target recognition algorithm inputs the target recognition frame coordinate information into the tracking algorithm board; S4.

2. For S4.1, when there are multiple targets to be identified, input the coordinate information of the target identification frame closest to the image center into the tracking algorithm board; S4.

3. The tracking algorithm board continuously updates the tracking frame based on the features and transmits the target miss distance information to the servo motor for the two-axis bifocal lens. The servo motor controls the two-axis bifocal lens to always keep the target in the center of the image. S4.

4. The servo motor of the two-axis bifocal lens inputs the azimuth angle information and azimuth velocity information to the tracking algorithm board. The tracking algorithm board adjusts the azimuth heading of the multi-rotor drone through the flight control module to align the multi-rotor drone's heading with the two-axis bifocal lens. S4.

5. When the target is lost or disappears, the target recognition algorithm is activated, waiting for the target to appear in the field of view and repeating S4.1; S4.

6. For S4.4, if the multi-rotor drone has a support leg, to prevent the two-axis bifocal lens from being blocked by the support leg during target tracking, thus affecting target tracking, the multi-rotor drone should be aligned with the two-axis bifocal lens to prevent the support leg from appearing in the tracking lens. S4.

7. When you need to exit tracking mode, select Exit Identification Tracking on the ground station. The two-axis dual-light lens and multi-rotor drone will stop tracking. If you need to track again, repeat S3.1.

Images