Real-time single-target tracking method for unmanned aerial vehicle carrying image-data integrated module

Abstract

The invention provides a real-time single-target tracking method for an unmanned aerial vehicle carrying a graph-digital integrated module, and the method comprises the steps: enabling a transmitter at a sky end of the graph-digital integrated machine to be in communication connection with the unmanned aerial vehicle, and enabling a receiver at a ground end of the graph-digital integrated machine to be in communication connection with edge intelligent equipment; the image and digital integrated module transmits images in real time and remotely, and converts video streams transmitted back by the unmanned aerial vehicle into encodable video streams and images; a deep learning model built in the edge intelligent device pre-loads nine threads, discovers a target and obtains target pixel coordinate information; coordinates of a world coordinate system are obtained, and the unmanned aerial vehicle deduces the optimal shooting position of the tracking target; target world coordinate information obtained through calculation in the edge intelligent equipment is sent to the ground end receiver, the ground end receiver sends a control instruction to the unmanned aerial vehicle, and the unmanned aerial vehicle moves according to the instruction to track the target in real time and continuously repeats coordinate backstepping tracking. The problem of low identification accuracy of low-altitude long-distance monitoring and tracking targets is effectively solved, and the targets are tracked in real time.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicles, and in particular relates to a method for real-time tracking a single target by a unmanned aerial vehicle equipped with an integrated image and data module. Background Art

[0002] In the field of smart cities, drones equipped with edge intelligent devices mainly rely on digital image transmission to obtain video streams transmitted by drones in image processing and analysis. Since the price of WiFi image transmission modules in digital image transmission is generally low, most drones are equipped with WiFi image transmission modules for image transmission. However, existing WiFi image transmission technology has problems such as limited transmission range, large transmission delay, and low security. At the same time, drones transmit images and videos, mainly relying on specific image transmission modules to establish connections, encode and modulate video information at the signal transmitter on the sky side, and use the ground receiver to receive signals for demodulation. The demodulated information is stored and analyzed on the back-end server, and the artificial intelligence deep learning model is used to detect and locate targets on the computer or server side. Flight instructions are issued to the drone flight control system based on the real-time target position information to achieve intelligent decision-making and autonomous flight of drones.

[0003] Realizing the long-distance, low-altitude, fast, real-time tracking of a single target by drones has made significant progress in drone intelligent navigation, but there are still some problems:

[0004] 1. Limited by signal range: The transmission distance of existing image transmission modules is limited by the transmission range of the signal, which is usually between several hundred meters and several thousand meters. It is greatly affected by environmental factors such as terrain and buildings, and is prone to signal interruption or loss.

[0005] 2. Susceptible to interference: Due to the popularity and widespread use of WiFi signals, there may be a large number of WiFi signal sources around, which may easily cause signal interference and affect the stability and reliability of the image transmission module.

[0006] 3. Large transmission delay: Compared with some professional communication modules, such as optical fiber transmission or dedicated data links, WiFi image transmission modules have a certain transmission delay, which may affect some application scenarios with high real-time requirements.

[0007] 4. Poor security: The transmission signal is relatively easy to be eavesdropped and attacked, and there are certain security risks. For some application scenarios with higher security requirements, additional data encryption and security protection measures need to be strengthened. Summary of the invention

[0008] The purpose of the present invention is to provide a method for real-time tracking of a single target by a drone equipped with an integrated image and data module, aiming to solve the problems of low timeliness, unstable transmission and small transmission range in the process of ground-end edge intelligent devices acquiring long-distance image transmission data of drones when identifying and locating long-distance targets in the prior art.

[0009] The present invention is implemented as follows: a method for real-time tracking of a single target by a drone equipped with an integrated image and data module, the method comprising the following steps:

[0010] Step S1: Connect the transmitter of the sky terminal of the integrated image and data machine to the UAV, connect the receiver of the ground terminal of the integrated image and data machine to the edge intelligent device, and unify the IP addresses of the sky terminal module and the ground terminal module;

[0011] Step S2: Based on the real-time long-distance image transmission between the edge intelligent device and the drone image-data integrated module, the video stream returned by the drone is connected to the edge intelligent device through the USB data interface and converted into an encodable video stream and image;

[0012] Step S3: The deep learning model built into the edge smart device preloads nine threads, and detects nine images in parallel to find the target and obtain the pixel coordinate information of the target;

[0013] Step S4: The coordinates of the tracking target in the world coordinate system are obtained through the algorithm of inversely inferring the 2D coordinates to the 3D coordinates. The drone infers the optimal shooting position of the tracking target through the coordinates of the world coordinate system.

[0014] Step S5: The target world coordinate information calculated in the edge intelligent device is sent to the ground receiver of the image-data integration module through the USB data interface, and then the ground receiver sends a control command to the drone. The drone moves here according to the command, tracks the target in real time, and continuously repeats the coordinate inversion method for tracking.

[0015] A further technical solution of the present invention is: the step S1 comprises the following steps:

[0016] Step S11: Connect the image-data integration module to the drone and the edge intelligent device respectively to unify the IP addresses of the module's air end and ground end;

[0017] Step S12: Connecting the sky-end module and the ground-end module to the drone and the edge smart device in turn;

[0018] Step S13: establishing a communication network mechanism through the modulation mode of the integrated digital-graphics model and the frequency band of data transmission, thereby increasing the signal transmission distance;

[0019] Step S14: Install a high-power antenna on the integrated image and data module transmitter to increase the signal transmission speed and ensure the timeliness of data acquisition by edge smart devices;

[0020] Step S15: The edge intelligent device processes the video stream in real time and runs a coordinate conversion algorithm to process the frame image.

[0021] A further technical solution of the present invention is: in step S2, the drone image transmission and video stream encoding and decoding process includes:

[0022] Video stream compression: The image-data integration module compresses the real-time video stream acquired by the drone into video stream data;

[0023] Receiving compressed data: The edge computing device receives the video stream data sent by the image and data integration device;

[0024] Frame processing: The decoder separates the compressed video stream data into independent frames;

[0025] Quantization: Mapping continuous pixel values ​​to discrete values ​​to reduce the amount of data;

[0026] Entropy coding: Use entropy coding techniques to further compress data;

[0027] Format conversion: Organize the encoded data into a specific bit stream format for easy storage and transmission.

[0028] A further technical solution of the present invention is: in step S3, the deep learning model adopts the yolov8n model.

[0029] A further technical solution of the present invention is: the step S3 comprises the following steps:

[0030] Step S31: Acquire the intrinsic and extrinsic parameters of the camera in advance, wherein the intrinsic parameters are the parameters set by the camera before leaving the factory, and the extrinsic parameters are the position and posture of the camera, including the rotation matrix and the translation matrix.

[0031] Step S32: The target pixel coordinates in the frame image are inverted into the world coordinate system coordinates through the pre-processed coordinate conversion algorithm in the edge intelligent device. The tracking target is located on the ground, and the Z direction in the world coordinate system is w The value is 0, u, v represent pixel coordinates, x, y represent image coordinates, Xc, Yc, Zc represent camera coordinates, Oc represents the origin of the camera coordinate system, Xw, Yw, Zw represent the world coordinate system, and the affine transformation formula from pixel coordinates (u, v) to image coordinates (x, y) is:

[0032]

[0033] p x is the x-coordinate value of the projection point p, f is the focal length of the camera, Zc is the camera coordinates Zc of the space point P, X c is the camera coordinate X of the space point P c , Xc / Zc=px / f, Yc / Zc=px / f; the relationship equation of camera coordinates is as follows:

[0034]

[0035] Extract Z c , the following relationship is derived:

[0036]

[0037] Combined with the pixel coordinate conversion to image coordinate, we can deduce:

[0038]

[0039] The camera internal parameters are expressed as:

[0040]

[0041] The final formula is as follows:

[0042]

[0043] The world coordinates of the target are obtained by inverse calculation:

[0044]

[0045] Since the tracking target is on the ground, the default Z c =0.

[0046] A further technical solution of the present invention is: the step S5 comprises the following steps:

[0047] Step S51: taking the acquired world coordinates as input, and calculating the best shooting coordinates for target tracking when the target at this position is tracked;

[0048] Step S52: the position is notified to the drone, and the drone receives the calculated movement signal of the shooting position, moves to a better shooting position, and adjusts the corresponding camera posture;

[0049] Step S53: Considering the long-distance image transmission, there is a large delay for the edge intelligent device to obtain the target information. The two-way data link enhancement technology of the integrated image and data device is used to enable the drone to obtain the video stream and import it into the edge intelligent device for processing in real time, predict the target's future movement route, and adjust the optimal detection position to be transmitted to the drone.

[0050] A further technical solution of the present invention is: in step S5, after obtaining a stable target video stream, target detection and real-time tracking of the target is started, based on the real-time image transmission of the image-data integrated module and the real-time processing and detection of the target by the yolov8n model, the background of the edge intelligent device continuously runs the coordinate conversion algorithm to obtain the world coordinates of the tracking target, and the drone moves accordingly to track the target in real time.

[0051] The beneficial effects of the present invention are: effectively solving the problem of low accuracy in target recognition in low-altitude long-distance monitoring and tracking in a certain area of ​​the existing city, based on the long-distance video stream stable transmission technology of the integrated image and data module and the built-in algorithms and models of the edge intelligent computing device, the target can be accurately identified and tracked in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is the main flow chart of the method of the present invention;

[0053] Figure 2 It is a schematic diagram of the image transmission and video stream encoding and decoding process of the drone of the present invention;

[0054] Figure 3 It is a histogram comparing the transmission delays of different integrated image transmission modules of the present invention;

[0055] Figure 4 It is a comparative bar chart of target coordinate detection accuracy of different image-data integrated image transmission modules of the present invention;

[0056] Figure 5 It is a transformation relationship diagram from the world coordinate system to the pixel coordinate system of the present invention;

[0057] Figure 6 The present invention is a flowchart of low-altitude and long-distance target detection, identification and tracking of unmanned aerial vehicles. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0059] The present invention provides a method for real-time tracking of a single target by a drone equipped with an integrated image and data module, the method comprising the following steps:

[0060] Step S1: Connect the transmitter of the sky terminal of the integrated image and data machine to the UAV, connect the receiver of the ground terminal of the integrated image and data machine to the edge intelligent device, and unify the IP addresses of the sky terminal module and the ground terminal module;

[0061] Step S2: Based on the real-time long-distance image transmission between the edge intelligent device and the drone image-data integrated module, the video stream returned by the drone is connected to the edge intelligent device through the USB data interface and converted into an encodable video stream and image;

[0062] Step S3: The deep learning model built into the edge smart device preloads nine threads, and detects nine images in parallel to find the target and obtain the pixel coordinate information of the target;

[0063] Step S4: The coordinates of the tracking target in the world coordinate system are obtained through the algorithm of inversely inferring the 2D coordinates to the 3D coordinates. The drone infers the optimal shooting position of the tracking target through the coordinates of the world coordinate system.

[0064] Step S5: The target world coordinate information calculated in the edge intelligent device is sent to the ground receiver of the image-data integration module through the USB data interface, and then the ground receiver sends a control command to the drone. The drone moves here according to the command, tracks the target in real time, and continuously repeats the coordinate inversion method for tracking.

[0065] Preferably, the step S1 comprises the following steps:

[0066] Step S11: Connect the image-data integration module to the drone and the edge intelligent device respectively to unify the IP addresses of the module's air end and ground end;

[0067] Step S12: Connecting the sky-end module and the ground-end module to the drone and the edge smart device in turn;

[0068] Step S13: establishing a communication network mechanism through the modulation mode of the integrated digital-graphics model and the frequency band of data transmission, thereby increasing the signal transmission distance;

[0069] Step S14: Install a high-power antenna on the integrated image and data module transmitter to increase the signal transmission speed and ensure the timeliness of data acquisition by edge smart devices;

[0070] Step S15: The edge intelligent device processes the video stream in real time and runs a coordinate conversion algorithm to process the frame image.

[0071] Preferably, in step S2, the drone image transmission and video stream encoding and decoding process includes:

[0072] Video stream compression: The image-data integration module compresses the real-time video stream acquired by the drone into video stream data;

[0073] Receiving compressed data: The edge computing device receives the video stream data sent by the image and data integration device;

[0074] Frame processing: The decoder separates the compressed video stream data into independent frames;

[0075] Quantization: Mapping continuous pixel values ​​to discrete values ​​to reduce the amount of data;

[0076] Entropy coding: Use entropy coding techniques to further compress data;

[0077] Format conversion: Organize the encoded data into a specific bit stream format for easy storage and transmission.

[0078] Preferably, in step S3, the deep learning model adopts the yolov8n model.

[0079] Preferably, step S3 comprises the following steps:

[0080] Step S31: Acquire the intrinsic and extrinsic parameters of the camera in advance, wherein the intrinsic parameters are the parameters set by the camera before leaving the factory, and the extrinsic parameters are the position and posture of the camera, including the rotation matrix and the translation matrix.

[0081] Step S32: The target pixel coordinates in the frame image are inverted into the world coordinate system coordinates through the pre-processed coordinate conversion algorithm in the edge intelligent device. The tracking target is located on the ground, and the Z direction in the world coordinate system is w The value is 0, u, v represent pixel coordinates, x, y represent image coordinates, Xc, Yc, Zc represent camera coordinates, Oc represents the origin of the camera coordinate system, Xw, Yw, Zw represent the world coordinate system, and the affine transformation formula from pixel coordinates (u, v) to image coordinates (x, y) is:

[0082]

[0083] p x is the x-coordinate value of the projection point p, f is the focal length of the camera, Z c is the camera coordinates Zc of the space point P, X c is the camera coordinate X of the space point P c , Xc / Zc=px / f, Yc / Zc=px / f; the relationship equation of camera coordinates is as follows:

[0084]

[0085] Extract Z c , the following relationship is derived:

[0086]

[0087] Combined with the pixel coordinate conversion to image coordinate, we can deduce:

[0088]

[0089] The camera internal parameters are expressed as:

[0090]

[0091] The final formula is as follows:

[0092]

[0093] The world coordinates of the target are obtained by inverse calculation:

[0094]

[0095] Since the tracking target is on the ground, the default Z c =0.

[0096] Preferably, step S5 comprises the following steps:

[0097] Step S51: taking the acquired world coordinates as input, and calculating the best shooting coordinates for target tracking when the target at this position is tracked;

[0098] Step S52: the position is notified to the drone, and the drone receives the calculated movement signal of the shooting position, moves to a better shooting position, and adjusts the corresponding camera posture;

[0099] Step S53: Considering the long-distance image transmission, there is a large delay for the edge intelligent device to obtain the target information. The two-way data link enhancement technology of the integrated image and data device is used to enable the drone to obtain the video stream and import it into the edge intelligent device for processing in real time, predict the target's future movement route, and adjust the optimal detection position to be transmitted to the drone.

[0100] Preferably, in step S5, after obtaining a stable target video stream, target detection and real-time tracking of the target is started. Based on the real-time image transmission of the image-data integrated module and the real-time processing and detection of the target by the yolov8n model, the background of the edge intelligent device continuously runs the coordinate conversion algorithm to obtain the world coordinates of the tracked target, and the drone moves accordingly to track the target in real time.

[0101] The purpose of the present invention is to overcome the shortcomings of existing edge devices on drones in the process of long-distance image transmission in image data transmission technology, and to provide a method for edge-side drones to detect target image data in real time. The edge intelligent devices on the sky side and the ground side of the drone are respectively connected to the image transmission module of the image-data integrated machine, and the module parameters are uniformly set on the edge intelligent devices. Based on the characteristics of edge intelligent devices that can have built-in code, small size and light weight, and the characteristics of the drones, Figure 1The long-distance real-time image transmission function provided by the edge device converts the drone video stream into an OpenCV encodable video stream through the network interface. Multi-threaded pre-loaded yolov8 is used to perform target detection on the image information, and the detected target pixel coordinates and orientation are used as input for the next step. The target coordinate information processed by the edge intelligent device is sent to the image-digital integrated module through the USB interface, and then sent to the flight control system of the drone by the image-digital integrated module, so that the drone moves to the target position and tracks the target in real time. Repeat the above operations to enable the drone to quickly track the target in real time under long-distance and low-altitude conditions. The present invention effectively solves the problem of low accuracy in target recognition in low-altitude long-distance monitoring and tracking in a certain area of ​​the existing city. Based on the long-distance video stream stable transmission technology of the image-digital integrated module and the built-in algorithm and model of the edge intelligent computing device, the target is accurately identified and tracked in real time. The specific steps are as follows

[0102] 1. Lightweight the yolov8 model so that it can achieve faster reasoning efficiency on edge smart devices;

[0103] 2. Based on the real-time long-distance image transmission function of edge intelligent devices and drone image-data integrated modules, the video stream sent back by the drone is connected to the edge intelligent device through the USB data interface and converted into an encodable video stream and image;

[0104] 3. Preload nine threads into the lightweight yolov8 model, detect the nine small images in parallel, find the target, and obtain the pixel coordinate information of the target;

[0105] 4. Through the algorithm of inverting 2D coordinates to 3D coordinates, the coordinates of the tracking target in the world coordinate system are obtained. The drone uses this coordinate to infer the optimal shooting position for tracking the target next.

[0106] 5. The target world coordinate information calculated in the edge intelligent device is sent to the ground receiver of the integrated image and data module through the USB data interface, and then the ground receiver sends control instructions to the UAV flight control system. The UAV moves here according to the instructions, tracks the target in real time, and continuously repeats the coordinate inversion method for tracking.

[0107] The use of drone real-time image transmission interface and image-data integrated module can ensure the timeliness of the drone's long-distance video stream back to the edge intelligent device for calculation and processing. At the same time, multiple yolov8 models are preloaded to process different areas of a frame image in parallel. The image transmission method of the monitoring process is redesigned, and the image-data integrated module is used to achieve the stability and timeliness of the drone's long-distance video stream back; the tracking uses the yolov8n model for target detection and recognition. The target pixel coordinate information obtained is used to quickly derive the position of the target in the world coordinate system through the coordinate conversion algorithm. The flight command is sent to the drone flight control system by using the receiver on the ground end of the image-data integrated module. The drone is transferred here according to the flight mission and begins to track the target at low altitude.

[0108] like Figure 2 As shown in the figure, the UAV is equipped with an integrated image and data module to realize the UAV long-distance image transmission and the transmission of mission information such as waypoints and flight attitude. The UAV is equipped with an integrated image and data module to realize the long-distance image transmission process.

[0109] The main structure of the UAV image transmission module is composed of two parts: the transmitter on the sky side and the receiver on the ground side. The signal transmitter uses QPSK / QAM16 / QAM64 automatic adaptation to modulate the signal to encode and modulate the video stream obtained by the drone. The signal receiver on the ground side demodulates the signal and transmits the video stream to the edge smart device through the USB data interface. The signal antenna can also be used to further enhance the image transmission signal. The image transmission module realizes the enhancement process of the UAV low-altitude long-distance image transmission signal as follows:

[0110] Parameter setting of the integrated image and data module: connect the integrated image and data module to the drone and edge intelligent device respectively, and unify the IP addresses of the module's air end and ground end;

[0111] Connection of integrated image and data devices: connect the sky module and ground module to the drone and edge smart device in turn;

[0112] Establishment of self-organizing network protocol: Establish a communication network mechanism covering a large area through the modulation method and data transmission frequency band unique to the integrated graphics and data module, thereby increasing the signal transmission distance;

[0113] Connection of high-power antenna: By installing a high-power antenna on the integrated image and data module transmitter, the image transmission signal is enhanced, the signal transmission speed is increased, and the timeliness of data acquisition by edge smart devices is ensured;

[0114] Receive stable video streams: Edge smart devices process video streams in real time and run coordinate conversion algorithms to process frame images.

[0115] The integrated image transmission module improves the efficiency of long-distance image transmission. The latency comparison of using front and rear edge smart devices to obtain video streams at different transmission distances is shown in the figure. Figure 3 As shown, the DJIOcuSync Air Unit digital image transmission module and the SKW77 WiFi digital image transmission module are the original models, and the YD-TS0080-WK integrated image and digital module is the used model. In a set of bar charts in the figure, the one on the far left is the YD-TS0080-WK integrated image and digital module, the one in the middle is the SKW77 WiFi digital image transmission module, and the one on the far right is the DJIOcuSync Air Unit digital image transmission module.

[0116] On the basis of obtaining real-time video streams, due to the requirements of edge computing on computing power, and based on the computing power evaluation and the actual application scenario, there is no high-precision requirement for the initial positioning of the detection target, so we choose to preload the yolov8 detection model in the edge intelligent all-in-one machine with multiple threads. Due to the use of yolov8's small model and multi-threaded processing, the edge intelligent device can perform real-time frame processing in the acquired video stream and obtain the pixel coordinates of the target.

[0117] Due to the different image data transmission delays of different image transmission modules, the edge intelligent device will have different network delays in obtaining the video stream, which will further affect the running rate of the yolov8n model and coordinate dimension-upgrading algorithm in the edge intelligent device, resulting in errors between the processed target longitude and latitude coordinates and the actual coordinates. The comparison diagram is as follows: Figure 4 As shown in the group of bar charts in the figure, the one on the far left is the YD-TS0080-WK integrated image and data module, the one in the middle is the SKW77 WiFi digital image transmission module, and the one on the far right is the DJIOcuSync Air Unit digital image transmission module.

[0118] Input the coordinate conversion algorithm into the edge intelligent device in advance. The image-data integration module transmits the acquired target information to the edge intelligent device and starts the coordinate conversion algorithm to convert the target object based on the two-dimensional coordinates in the captured image into three-dimensional world coordinates, thereby obtaining the real-world coordinates of the target object. The coordinate conversion algorithm flow chart and detailed implementation steps are as follows:

[0119] Obtain the camera's internal parameters (factory default) and external parameters (camera posture: rotation matrix, translation matrix) in advance; use the coordinate transformation algorithm preprocessed in the edge smart device (such as Figure 5 As shown in Figure 2, the target pixel coordinates in the frame image can be inferred to the world coordinate system coordinates. The tracking target is located on the ground, so Z w The value of is 0, only X w and Y w The target's two-dimensional to three-dimensional coordinate conversion can be completed.

[0120] Through the world coordinate system coordinates of the target, the optimal tracking position is obtained and the drone is moved to this position. The specific steps are as follows: obtain the obtained (Xw, Yw, Zw) as input, calculate the optimal shooting coordinates for target tracking when the target at this position is tracked; inform the drone of this position, the drone receives the calculated movement signal of this shooting position, moves to a better shooting position, and adjusts the corresponding camera posture; considering the long-distance image transmission, there is a large delay in the edge intelligent device obtaining target information. The two-way data link enhancement technology of the integrated image and data device is used to enable the drone to obtain the video stream and import it into the edge intelligent device for processing in real time, predict the future movement route of the target, and slightly adjust the optimal detection position to be transmitted to the drone.

[0121] After obtaining a stable target video stream, target detection and real-time tracking of the target begins. Based on the real-time image transmission of the integrated image and data device and the real-time processing and detection of the target by the yolov8n model, the coordinate conversion algorithm is continuously run in the background of the edge computing device to obtain the world coordinates of the tracked target. The drone moves accordingly to track the target in real time.

[0122] The following table shows the edge intelligent computing device algorithm environment configuration:

[0123]

[0124]

[0125] The following table shows the relevant software and hardware configuration information of the present invention:

[0126]

[0127] This invention proposes a new processing flow for the detection, identification and tracking of long-distance low-altitude targets. This processing flow separates "image transmission" and "tracking". During the long-distance image transmission process, an integrated image and data module is used to transmit the image of the target information and obtain a stable video stream in real time. During real-time tracking, a model suitable for low-altitude target detection is used to convert the world coordinates of the target. These steps are integrated into the edge intelligent device on the ground. The key points of this process are as follows:

[0128] Fast return of long-distance image data: Reinstall the drone's onboard image transmission module, install the sky-side module of the image-data integration module based on the drone's own serial port protocol, and install CH340 (USB serial port driver) and network port driver on the edge smart device for the ground-side module, and connect under the premise that the USB serial port of the ground-side image-data integration module can be recognized. After the module is installed, the edge computing communication protocol and the self-organizing network protocol of the image-data integration module are negotiated, and data is transmitted based on the self-organizing network communication protocol of the image-data integration module to achieve high-speed, stable and reliable image data transmission;

[0129] Fast real-time tracking: Based on the tracking method optimized by artificial intelligence algorithms, under the premise of real-time feedback of the target detection video stream of the image-data integrated module, the detection model only needs to obtain an imprecise positioning of the target to quickly calculate the WGS84 geographic coordinates of the target, so that the drone can track it quickly. Since the tracking of the target only needs to obtain the initial positioning of the target, the yolov8 small model is used for detection to speed up the calculation efficiency, and a multi-threaded approach is used in the processing process to improve data processing capabilities. The tracking algorithm model adopts a frame extraction method for the returned video stream, and only one frame of the picture is needed to identify and locate the target, which increases the timeliness of data processing. By using four technologies of small models, multi-threaded detection and frame extraction recognition, as well as edge computing processing, the drone can realize rapid identification and real-time tracking of tracking targets under long-distance and low-altitude conditions;

[0130] Processing of the tracking module: real-time conversion of two-dimensional coordinates and three-dimensional coordinates, conversion based on WGS84 geographic coordinates and drone video pixel coordinates, building a set of customized drone position tracking algorithm models, which can realize the real-time update strategy of the world coordinates of the flight control module in the ground station according to the real-time detection and position coordinate conversion and tracking algorithm model of the tracking target in the video. After the drone receives the corresponding flight control command task, it moves to the tracking position and starts to track the target clearly. Repeating the above operations can also complete the long-distance low-altitude tracking of the target.

[0131] The present invention can implement a device for long-distance single-target detection of a UAV by replacing the UAV image data transmission signal enhancement device, and replace the code container "edge computing device" with other small-volume intelligent computing devices.

[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for real-time tracking of a single target by a drone equipped with an integrated image and data module, characterized in that: The method comprises the following steps: Step S1: Connect the transmitter of the sky terminal of the integrated image and data machine to the UAV, connect the receiver of the ground terminal of the integrated image and data machine to the edge intelligent device, and unify the IP addresses of the sky terminal module and the ground terminal module; Step S2: Based on the real-time long-distance image transmission between the edge intelligent device and the drone image-data integrated module, the video stream returned by the drone is connected to the edge intelligent device through the USB data interface and converted into an encodable video stream and image; Step S3: The deep learning model built into the edge smart device preloads nine threads, and detects nine images in parallel to find the target and obtain the pixel coordinate information of the target; Step S4: The coordinates of the tracking target in the world coordinate system are obtained through the algorithm of inversely inferring the 2D coordinates to the 3D coordinates. The drone infers the optimal shooting position of the tracking target through the coordinates of the world coordinate system. Step S5: The target world coordinate information calculated in the edge intelligent device is sent to the ground receiver of the image-data integration module through the USB data interface, and then the ground receiver sends a control command to the drone. The drone moves here according to the command, tracks the target in real time, and continuously repeats the coordinate inversion method for tracking.

2. The method for real-time tracking a single target by a drone equipped with an integrated image and data module according to claim 1, characterized in that: The step S1 comprises the following steps: Step S11: Connect the image-data integration module to the drone and the edge intelligent device respectively to unify the IP addresses of the module's air end and ground end; Step S12: Connecting the sky-end module and the ground-end module to the drone and the edge smart device in turn; Step S13: establishing a communication network mechanism through the modulation mode of the integrated digital-graphics model and the frequency band of data transmission, thereby increasing the signal transmission distance; Step S14: Install a high-power antenna on the integrated image and data module transmitter to increase the signal transmission speed and ensure the timeliness of data acquisition by edge smart devices; Step S15: The edge intelligent device processes the video stream in real time and runs a coordinate conversion algorithm to process the frame image.

3. The method for real-time tracking a single target by a drone equipped with an integrated image and data module according to claim 1, characterized in that: In step S2, the drone image transmission and video stream encoding and decoding process includes: Video stream compression: The image-data integration module compresses the real-time video stream acquired by the drone into video stream data; Receiving compressed data: The edge computing device receives the video stream data sent by the image and data integration device; Frame processing: The decoder separates the compressed video stream data into independent frames; Quantization: Mapping continuous pixel values ​​to discrete values ​​to reduce the amount of data; Entropy coding: Use entropy coding techniques to further compress data; Format conversion: Organize the encoded data into a specific bit stream format for easy storage and transmission.

4. The method for real-time tracking a single target by a drone equipped with an integrated image and data module according to claim 1, characterized in that: In step S3, the deep learning model adopts the yolov8n model.

5. The method for real-time tracking a single target by a drone equipped with an integrated image and data module according to claim 1, characterized in that: The step S3 comprises the following steps: Step S31: Acquire the intrinsic and extrinsic parameters of the camera in advance, wherein the intrinsic parameters are the parameters set by the camera before leaving the factory, and the extrinsic parameters are the position and posture of the camera, including the rotation matrix and the translation matrix. Step S32: The target pixel coordinates in the frame image are inverted into the world coordinate system coordinates through the pre-processed coordinate conversion algorithm in the edge intelligent device. The tracking target is located on the ground, and the Z direction in the world coordinate system is w The value is 0, u, v represent pixel coordinates, x, y represent image coordinates, Xc, Yc, Zc represent camera coordinates, Oc represents the origin of the camera coordinate system, Xw, Yw, Zw represent the world coordinate system, and the affine transformation formula from pixel coordinates (u, v) to image coordinates (x, y) is: p x is the x-coordinate value of the projection point p, f is the focal length of the camera, Z c is the camera coordinates Zc of the space point P, X c is the camera coordinate X of the space point P c , Xc / Zc=px / f, Yc / Zc=px / f; the relationship equation of camera coordinates is as follows: Extract Z c , the following relationship is derived: Combined with the pixel coordinate conversion to image coordinate, we can deduce: The camera internal parameters are expressed as: The final formula is as follows: The world coordinates of the target are obtained by inverse calculation: Since the tracking target is on the ground, the default Z c =0.

6. The method for real-time tracking a single target by a drone equipped with an integrated image and data module according to claim 1, characterized in that: The step S5 comprises the following steps: Step S51: taking the acquired world coordinates as input, and calculating the best shooting coordinates for target tracking when the target at this position is tracked; Step S52: the position is notified to the drone, and the drone receives the calculated movement signal of the shooting position, moves to a better shooting position, and adjusts the corresponding camera posture; Step S53: Considering the long-distance image transmission, there is a large delay for the edge intelligent device to obtain the target information. The two-way data link enhancement technology of the integrated image and data device is used to enable the drone to obtain the video stream and import it into the edge intelligent device for processing in real time, predict the target's future movement route, and adjust the optimal detection position to be transmitted to the drone.

7. The method for real-time tracking a single target by a drone equipped with an integrated image and data module according to claim 4, characterized in that: In step S5, after obtaining a stable target video stream, target detection and real-time tracking of the target are started. Based on the real-time image transmission of the image-data integration module and the real-time processing and detection of the target by the yolov8n model, the background of the edge intelligent device continuously runs the coordinate conversion algorithm to obtain the world coordinates of the tracking target. The drone moves accordingly to track the target in real time.

Images