Cloud-side collaborative unmanned aerial vehicle target detection tracking and trajectory prediction device

Abstract

The invention discloses a cloud-side collaborative unmanned aerial vehicle target detection tracking and trajectory prediction device, and belongs to the technical field of Internet of Things, artificial intelligence and unmanned aerial vehicles. The device comprises a device hardware module, an unmanned aerial vehicle target detection module, an unmanned aerial vehicle target tracking module, an unmanned aerial vehicle trajectory prediction module and a cloud edge cooperation module. Aiming at the problems that a traditional method is slow in response speed and low in detection precision and a system is difficult to adapt to a complex environment, based on a hardware device, a lightweight artificial intelligence algorithm is introduced to carry out high-precision target tracking, detection and trajectory prediction on a potential threat unmanned aerial vehicle; real-time information collaboration is realized based on an efficient cloud edge collaboration technology; according to the method, the trajectory of the potential threatening unmanned aerial vehicle can be quickly identified, tracked and predicted in a complex environment.

Description

Technical Field

[0001] The present invention belongs to the fields of Internet of Things, artificial intelligence and drone technology, and in particular relates to a cloud-edge collaborative drone target detection, tracking and trajectory prediction device. Background Art

[0002] With the rapid development of drone technology and the continuous expansion of its application areas, the use of drones in both civilian and military fields is increasing. However, the growing number of drones also brings new challenges to airspace security. For example, unauthorized drone activities may pose a threat to national security, public safety, and personal privacy. To address these potential threats, it is necessary to develop a device that can detect, track, and predict the behavior of drones, capable of rapid response to ensure airspace security.

[0003] In recent years, with the development of the Internet of Things (IoT) and artificial intelligence (AI) technologies, several IoT-based methods for drone target detection, tracking, and trajectory prediction have emerged. Advanced AI and machine learning algorithms, including deep learning and reinforcement learning, have significantly improved the accuracy of target detection, recognition, and classification. Convolutional neural networks, recurrent neural networks, and their variants have been particularly effective in image recognition and video analysis. Furthermore, the cloud-edge collaboration enabled by IoT technologies enables real-time data transmission and remote control, enabling drones to be better integrated into applications such as smart cities and intelligent transportation, significantly enhancing the overall performance and scope of the systems.

[0004] However, traditional methods have some significant shortcomings when performing drone target detection, tracking and trajectory prediction: Slow response speed: Traditional methods rely on ground stations or a single centralized processing unit, which may lead to data transmission delays and time consumption in the decision-making process, thereby reducing the real-time and response efficiency of the system. Low detection accuracy: Due to the influence of environmental factors (such as weather conditions and lighting changes), traditional methods have difficulty maintaining a stable high-precision detection rate, especially in complex and changing environments. Poor adaptability: Existing systems are usually designed for specific scenarios. When encountering new or unexpected situations, their performance may drop significantly and cannot be flexibly adjusted to adapt to different operating environments. Limited information processing capabilities: Most traditional drone monitoring systems do not have strong data analysis capabilities and intelligent decision-making support functions, making it difficult to achieve effective management of large-scale, high-speed moving targets.

[0005] To address these issues, this paper proposes a cloud-edge collaborative drone target detection, tracking, and trajectory prediction device. This device integrates hardware and software, combined with advanced artificial intelligence algorithms, to improve detection accuracy and response speed, while enhancing the system's adaptability and stability.

[0006] In summary, the present invention aims to overcome the limitations of the existing technology and provide a more intelligent, fast-responding and adaptable hardware and software integrated device to improve the ability to detect, track and predict the trajectory of drone targets and enhance airspace safety. Summary of the Invention

[0007] The purpose of the present invention is to address the deficiencies of the existing technology and provide a cloud-edge collaborative UAV target detection, tracking and trajectory prediction device.

[0008] The object of the present invention is achieved through the following technical solutions: a cloud-edge collaborative UAV target detection, tracking and trajectory prediction device, the device comprising a device hardware module, a UAV target detection module, a UAV target tracking module, a UAV trajectory prediction module and a cloud-edge collaboration module; The device hardware module is used to collect images of the target drone and transmit them to the drone target detection module; The drone target detection module is used to detect the image of the target drone, obtain an image stream marked with the category and bounding box coordinates of the target drone, and transmit it to the drone trajectory prediction module; The UAV target tracking module is used to track the target based on the image stream marked with the category and bounding box coordinates of the target UAV based on the lightweight Deepsort algorithm, obtain the motion trajectory set of all target UAVs and transmit it to the UAV trajectory prediction module; The UAV trajectory prediction module is used to predict the final motion prediction trajectory set of all target UAVs through a generative adversarial network based on the motion trajectory set of all target UAVs and transmit it to the cloud-edge collaboration module; The cloud-edge collaboration module is used to transmit the motion prediction trajectory set of all target drones to the cloud.

[0009] Furthermore, the device hardware module includes a storage device, a high-performance embedded computing unit, an image acquisition module and a power supply module; The storage device is used to provide a large-capacity flash memory or SSD; The high-performance embedded computing unit is used to provide high-computing power chips for the operation of the drone target detection module, drone target tracking module, drone trajectory prediction module and cloud-edge collaboration module respectively; The image acquisition module is used to acquire images of the target drone using a high-resolution camera, and transmit the acquired images of the target drone to the drone target detection module; The power module is used to provide battery and intelligent power monitoring.

[0010] Furthermore, the cloud-edge collaboration module includes a communication module and an edge computing service module; The communication module is used to achieve full-duplex communication between the cloud and the edge of the drone based on network socket and message queue telemetry transmission; The edge computing service module is used to provide edge computing service functions at the edge end.

[0011] Furthermore, the drone target detection module detects the image of the target drone, obtains an image stream marked with the category and bounding box coordinates of the target drone, and transmits it to the drone trajectory prediction module, which specifically includes the following sub-steps: (a.1) First, the image of the target drone collected by the drone target detection module Perform preprocessing to obtain the preprocessed image ; The preprocessing includes size adjustment, normalization and channel alignment; (a.2) Then the preprocessed image Input into the lightweight MobileNetv3 backbone network for feature extraction to obtain the feature map The MobileNetv3 backbone network includes a depthwise separable convolutional layer, a channel attention module, and a Hard-Swish activation function. (a.3) Feature maps based on the Coordinate Attention mechanism Perform attention enhancement to obtain the feature map after attention enhancement ; (a.4) Feature map after attention is added The input is sent to the decoupled detection head, which outputs an image stream marked with the category and bounding box coordinates of the target drone and outputs it to the drone target tracking module.

[0012] Furthermore, the UAV target tracking module performs target tracking based on the image stream marked with the category and bounding box coordinates of the target UAV based on the lightweight Deepsort algorithm, obtains the motion trajectory set of all target UAVs and transmits it to the UAV trajectory prediction module, specifically including the following sub-steps: (b.1) Based on the image stream marked with the target drone’s category and bounding box coordinates, the DeepSort algorithm is used to extract the appearance features and motion features in the bounding boxes of all target drones; (b.2) Using the Kalman filter to predict the position of each target drone in the next frame based on its appearance and motion features, the Hungarian algorithm and cascade matching algorithm are used to calculate the matching degree between all two target drones in the previous and next frames. When the matching degree exceeds the matching threshold, the two corresponding target drones are assigned the same ID. (b.3) Finally, the positions of target drones with the same ID are connected in sequence to form a set of motion trajectories of all target drones and transmitted to the drone trajectory prediction module.

[0013] Furthermore, the UAV trajectory prediction module obtains the final motion prediction trajectory set of all target UAVs through the generative adversarial network according to the motion trajectory set of all target UAVs and transmits it to the cloud-edge collaboration module, which specifically includes the following sub-steps: (c.1) First, the motion trajectory set of all target drones is predicted through the encoder-decoder structure of the Autoformer to obtain the initial motion prediction trajectory set; (c.2) Then, based on the initial motion prediction trajectory set, the final motion prediction trajectory set of all target drones is obtained through the generative adversarial network and transmitted to the cloud-edge collaboration module.

[0014] Furthermore, the sub-step (c.1) specifically includes the following sub-steps: (c.1.1) The set of motion trajectories of all target drones ,in, Represents the motion trajectory of any target drone, represents the number of target drones, ; The motion trajectory of each target drone Input to the encoder-decoder structure of Autoformer, first encode, with encoder layers, The overall equation of the layer encoder is ,in, represents the encoder layer; Indicates the The output of the layer encoder, Indicates the The output of the layer encoder, ; through After encoding through the encoder layers, the final encoded output is obtained ; (c.1.2) Then, for each final encoded output To decode, decoder layer, The overall equation of the layer decoder is , in, represents the decoder layer, Indicates the The output of the layer decoder, Indicates the The output of the layer decoder, ; go through After decoding through the decoder layers, the final decoded output is obtained And as the initial motion prediction trajectory of the corresponding target UAV, the initial motion prediction trajectory set is obtained : .

[0015] Furthermore, the Output of layer decoder The specific steps include the following: (d.1) First, Output of layer decoder Apply the Auto-Correlation mechanism and then auto-correlate the results And add output Perform time series decomposition operation and decompose it into the first seasonal part result and the first trend-cyclical part results ; (d.2) Then the first seasonal part result and the final encoded output Apply the Auto-Correlation mechanism and then auto-correlate the results And add the first seasonal part results Perform time series decomposition operation and decompose it into the second seasonal part results and the second trend-cyclical part of the results ; (d.3) Then the second seasonal part result Apply a feedforward network layer and then pass the output of the feedforward network layer to And add the second seasonal part results Perform time series decomposition operation and decompose it into the third seasonal part results and the third trend - cyclical part results ; Finally, the third seasonal part results As the first Output of layer decoder .

[0016] Furthermore, the sub-step (c.2) is specifically as follows: The initial motion prediction trajectory set The initial motion prediction trajectory of each target UAV The input is fed into a generative adversarial network, which includes a generator and a discriminator. The generator and the discriminator are alternately optimized in combination with a diversity loss function until the generative adversarial network converges, and the network parameters are fixed to obtain an optimized generative adversarial network. Finally, the initial motion prediction trajectory set is optimized through the generative adversarial network Perform trajectory prediction to obtain the final motion prediction trajectory set of all target drones and transmit it to the cloud-edge collaboration module.

[0017] The beneficial effects of the present invention are: 1) Based on the device hardware module and cloud-edge collaboration module, it supports real-time data processing and cloud-edge communication, can adapt to complex network environments, and ensure data security and system maintainability; 2) Based on the UAV target detection module and the UAV target tracking module, high-precision detection and real-time stable tracking of small target UAVs are achieved, effectively dealing with fast-moving targets; 3) Based on the UAV trajectory prediction module, the Autoformer model is used to capture long-term flight dependencies, and combined with GAN diversity loss to generate multimodal trajectories, improving the ability to predict complex flight behaviors and enhancing the countermeasure effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a cloud-edge collaborative UAV target detection, tracking, and trajectory prediction device; Figure 2 is a schematic diagram of the device hardware module; Figure 3 This is a schematic diagram of the cloud-edge collaboration module; Figure 4 Flowchart for detecting target drone images; Figure 5 Flowchart for target tracking based on a lightweight Deepsort algorithm based on an image stream labeled with the target drone's category and bounding box coordinates. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to illustrate the present invention, rather than to represent all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] Example 1: Figure 1As shown, the present invention provides a cloud-edge collaborative UAV target detection, tracking and trajectory prediction device, which includes a device hardware module, a UAV target detection module, a UAV target tracking module, a UAV trajectory prediction module and a cloud-edge collaboration module; the device hardware module is respectively connected to the UAV target detection module, the UAV target tracking module, the UAV trajectory prediction module and the cloud-edge collaboration module.

[0021] 1. Device Hardware Module The device hardware module is used to collect images of the target drone and transmit them to the drone target detection module.

[0022] like Figure 2 As shown, the device hardware module is mounted on the drone of the user, including a storage device, a high-performance embedded computing unit, an image acquisition module and a power module.

[0023] The storage device is used to provide a large-capacity flash memory or SSD, providing ample storage space and supporting fast read and write operations; supporting hot plugging and redundant backup to facilitate maintenance and replacement while ensuring that data is not lost; supporting data compression and encryption to reduce storage occupancy and protect sensitive information.

[0024] The high-performance embedded computing unit is used to provide high-computing power chips for the operation of the drone target detection module, drone target tracking module, drone trajectory prediction module and cloud-edge collaboration module, ensuring real-time processing capabilities.

[0025] The image acquisition module is used to acquire images of the target UAV through a high-resolution camera, and transmit the acquired images of the target UAV to the UAV target detection module. The high-resolution camera has functions such as optical zoom and night vision.

[0026] The power module is used to provide battery and intelligent power monitoring, monitor battery status in real time, and automatically adjust the working mode.

[0027] 2. Cloud-Edge Collaboration Module The cloud-edge collaboration module is used to transmit the trajectory prediction results output by the drone trajectory prediction module from the edge end of the drone to the cloud.

[0028] like Figure 3 As shown, the cloud-edge collaboration module includes a communication module and an edge computing business module.

[0029] The communication module realizes full-duplex communication between the cloud and the edge of the drone based on Websocket (network socket) and MQTT (message queue telemetry transmission), realizes the coordination of the trajectory prediction results of the edge and the cloud information, and the cloud collects, circulates and analyzes the trajectory prediction results in real time to facilitate collaborative decision-making on the cloud; supports the cloud-edge integrated system framework to authenticate and authorize edge devices and implement effective management of edge devices; supports cloud-edge information collaboration, remote operation and maintenance, and offline autonomy.

[0030] The edge computing business module provides edge computing business functions at the edge, including edge container management, edge node management, and edge computing system. The edge computing business module provides a cloud-edge collaborative console visual interface for the cloud to facilitate operation, including edge device instance management, edge device access management, edge application management, and edge driver management.

[0031] 3. UAV Target Detection Module The drone target detection module is based on the YOLOX algorithm, uses a lightweight MobileNetv3 backbone network as a feature extractor, and introduces the Coordinate Attention mechanism to enhance coordinate-aware attention. It is used to adopt a high-precision drone target detection algorithm to perform high-precision target detection on potentially threatening drones.

[0032] The UAV target detection module is used to detect the image of the target UAV, obtain the image stream marked with the category and bounding box coordinates of the target UAV and transmit it to the UAV trajectory prediction module, such as Figure 4 As shown, it specifically includes the following sub-steps: (a.1) Image preprocessing: First, the image of the target drone collected by the drone target detection module is processed. Perform preprocessing to obtain the preprocessed image ; The preprocessing includes size adjustment, normalization and channel alignment.

[0033] (a.2) Lightweight feature extraction: The preprocessed image is then Input into the lightweight MobileNetv3 backbone network for feature extraction to obtain the feature map .

[0034] The sub-step (a.2) is specifically: The MobileNetv3 backbone network contains depthwise separable convolutional layers, channel attention (Squeeze-and-Excitation) modules, and Hard-Swish activation functions.

[0035] Preprocessed image First, the depth-wise convolution and point-wise convolution are performed in sequence through the depth-wise separable convolution layer, and then input into the channel attention module for feature map enhancement, and finally input into the Hard-Swish activation function to obtain the feature map .

[0036] (a.3) Coordinate-aware attention enhancement: According to the Coordinate Attention mechanism, the feature map Perform attention enhancement to obtain the feature map after attention enhancement .

[0037] After the output of the backbone network, the Coordinate Attention mechanism is introduced to enhance the attention of coordinate perception.

[0038] The sub-step (a.3) is specifically: (a.3.1) Feature map for ,in, Representation feature map Contains channels, Indicates the The sub-feature map of each channel, .

[0039] The sub-feature map for ,in, Indicates the The eigenvalues ​​at height h and width w in the sub-feature graph of each channel, , , Represents the height of each sub-feature map, Indicates the width of each sub-feature map.

[0040] (a.3.2) First, the feature map The feature values ​​at height h in each channel are encoded horizontally to obtain the feature set at height h. : ,in, represents the horizontal global average pooling result of the c-th channel at height h, .

[0041] Then the feature map The feature values ​​at width w in each channel are encoded vertically to obtain the feature set at width w. : ,in, Represents the vertical global average pooling result of the c-th channel at width w, .

[0042] Then the feature set and feature set Splicing is performed, and 1x1 convolution operation, batch normalization operation, and linear activation function operation are performed in sequence to obtain the intermediate feature set , the calculation formula is as follows: ; in, Representing a feature set and feature set The splicing, represents a 1x1 convolution operation, represents the batch normalization operation, Represents a linear activation function operation.

[0043] Then the intermediate feature set Split the intermediate feature set at height h along the spatial dimension and the intermediate feature set at width w , and based on the intermediate feature set and intermediate feature sets After performing 1x1 convolution operation and Sigmoid activation function processing respectively, the attention weight map at the corresponding height h is obtained. and attention weight map at width w , the calculation formula is as follows: ; ; in, Represents the Sigmoid activation function.

[0044] The attention weight map at the height h for ,in, represents the attention weight at height h of the c-th channel.

[0045] The attention weight map at the width w for ,in, represents the attention weight at the width w of the c-th channel.

[0046] Finally, the attention weight at the height h of the cth channel is used and the attention weight at width w of the cth channel For eigenvalues Perform feature rescaling to obtain The enhanced eigenvalues ​​at the height h and width w of the channel .

[0047] (a.3.3) Repeat step (a.3.2) for the feature map Each eigenvalue in is rescaled to obtain the feature map after attention increase : ,in, Indicates the The sub-feature map after the attention of the channels is increased.

[0048] The sub-feature map after the attention is increased for .

[0049] (a.4) Detection result output: Feature map after attention is added The input is sent to the decoupled detection head, which outputs an image stream marked with the category and bounding box coordinates of the target drone and outputs it to the drone target tracking module.

[0050] The drone target detection module is based on the YOLOX algorithm and uses a lightweight MobileNetv3 backbone network as a feature extractor. It can significantly reduce model parameters and computational complexity while maintaining high accuracy, thereby accelerating the feature extraction process and supporting efficient real-time detection on mobile devices.

[0051] 4. UAV target tracking module The UAV target tracking module is used to adopt a lightweight UAV target tracking algorithm, and can perform high-precision target tracking on potentially threatening UAVs on hardware devices with limited computing resources.

[0052] The UAV target tracking module is used to track the target based on the image stream marked with the category and bounding box coordinates of the target UAV based on the lightweight Deepsort algorithm, obtain the motion trajectory set of all target UAVs and transmit it to the UAV trajectory prediction module, such as Figure 5 As shown, it specifically includes the following sub-steps: (b.1) Feature extraction: Based on the image stream labeled with the target drone’s category and bounding box coordinates, the DeepSort algorithm is used to extract the surface features and motion features in the bounding boxes of all target drones. These features will be used for subsequent target matching and tracking.

[0053] (b.2) Target Matching: A Kalman filter is used to predict the position of each target drone in the next frame based on its appearance and motion characteristics. The Hungarian algorithm and cascade matching algorithm are then used to calculate the degree of match between all pairwise target drones in the two frames. When the degree of match exceeds a matching threshold, the two corresponding target drones are assigned the same ID. This step identifies the correspondence between identical target drones in each frame and assigns a unique ID to each tracked target drone. In this example, the matching threshold is 0.7.

[0054] (b.3) Target tracking: Finally, the positions of target drones with the same ID are connected in sequence to form a set of motion trajectories of all target drones and transmitted to the drone trajectory prediction module to achieve drone target tracking.

[0055] 5. UAV trajectory prediction module After acquiring data from the drone trajectory tracking module, the drone trajectory prediction module performs high-precision, real-time position sequence prediction of the drone target trajectory, thereby improving air safety and the ability to counter illegal or malicious drones. The drone trajectory prediction module is modeled based on long-term trajectory dependency capture and randomness simulation.

[0056] The UAV trajectory prediction module obtains the final motion prediction trajectory set of all target UAVs through the generative adversarial network according to the motion trajectory set of all target UAVs and transmits it to the cloud-edge collaboration module. Specifically, it includes the following sub-steps: (c.1) First, the motion trajectory set of all target drones is predicted through the encoder-decoder structure of the Autoformer to obtain the motion prediction trajectory set.

[0057] The sub-step (c.1) specifically includes the following sub-steps: (c.1.1) The set of motion trajectories of all target drones ,in, Represents the motion trajectory of any target drone, represents the number of target drones, .

[0058] The motion trajectory of each target drone Input to the encoder-decoder structure of Autoformer, first encode, with encoder layers, The overall equation of the layer encoder is ,in, represents the encoder layer; Indicates the The output of the layer encoder, Indicates the The output of the layer encoder, ;go through After encoding through the encoder layers, the final encoded output is obtained .

[0059] (c.1.2) is then decoded, with decoder layer, The overall equation of the layer decoder is , in, represents the decoder layer, Indicates the The output of the layer decoder, Indicates the The output of the layer decoder, ;go through After decoding through the decoder layers, the final decoded output is obtained , and the final decoded output As the motion prediction trajectory of the corresponding target UAV, the motion prediction trajectory set is obtained : .

[0060] The said Output of layer decoder The specific calculation process is: (d.1) First, Output of layer decoder Apply the Auto-Correlation mechanism and then auto-correlate the results And add output Perform time series decomposition operation and decompose it into the first seasonal part result and the first trend-cyclical part results , the calculation formula is: ; in, represents the autocorrelation operation; Represents a time series decomposition operation.

[0061] (d.2) Then the first seasonal part result and the final encoded output Apply the Auto-Correlation mechanism and then auto-correlate the results And add the first seasonal part results Perform time series decomposition operation and decompose it into the second seasonal part results and the second trend-cyclical part of the results , the calculation formula is: .

[0062] (d.3) Then the second seasonal part result Apply a feedforward network layer and then pass the output of the feedforward network layer to And add the second seasonal part results Perform time series decomposition operation and decompose it into the third seasonal part results and the third trend - cyclical part results , the calculation formula is: ; in, Represents a feedforward network layer.

[0063] Finally, the third seasonal part results As the first Output of layer decoder ,Right now .

[0064] (c.2) Then, based on the motion prediction trajectory set, the final motion prediction trajectory set of all target drones is obtained through the generative adversarial network and transmitted to the cloud-edge collaboration module.

[0065] The sub-step (c.2) is specifically: The initial motion prediction trajectory set The initial motion prediction trajectory of each target UAV The input is fed into a generative adversarial network, which includes a generator and a discriminator. The generator and the discriminator are alternately optimized in combination with a diversity loss function until the generative adversarial network converges, and the network parameters are fixed to obtain an optimized generative adversarial network.

[0066] Finally, the initial motion prediction trajectory set is optimized through the generative adversarial network Perform trajectory prediction to obtain the final motion prediction trajectory set of all target drones and transmit it to the cloud-edge collaboration module.

[0067] Example 2: The present invention provides a cloud-edge collaborative drone target detection, tracking, and trajectory prediction device. The drone target detection module and drone target tracking module are particularly suitable for resource-constrained environments and are used in a cloud-edge collaborative drone target detection, tracking, and trajectory prediction device of Example 1 above. Specifically, The drone target detection module utilizes an optimized YOLOX algorithm, combined with the lightweight MobileNetv3 convolutional neural network and the Coordinate Attention mechanism. This significantly reduces computational resource requirements while maintaining high accuracy, enabling fast, real-time target detection on mobile devices. By replacing the larger CSPDarknet53 feature extractor with the more compact MobileNetv3 and introducing a CoordinateAttention layer that prioritizes target location information, the module significantly improves detection performance for small targets such as drones. It also simplifies training and reduces the workload of hyperparameter tuning.

[0068] In order to ensure that the detected target can be stably tracked, this embodiment also integrates a drone target tracking module based on the DeepSort algorithm. The drone target tracking module solves the problem of ID reallocation caused by occlusion in the traditional Sort algorithm, and improves the accuracy of target identity matching. DeepSort effectively manages the trajectory life cycle by introducing two types of trajectory state management: confirmed and unconfirmed, and using a feature re-identification network to extract target surface features and measure similarity. In addition, the use of Kalman filters for motion feature matching calculations further enhances the accuracy and stability of tracking. Overall, the two modules complement each other and together provide a complete drone monitoring system, significantly improving the ability of air security prevention.

[0069] Finally, based on the UAV trajectory prediction module, the trajectory of potential threat UAVs is predicted in real time with high precision, taking into account the properties of low-altitude object interactions and external complex factors, thereby improving air safety and the ability to deal with illegal or malicious UAVs.

[0070] Example 3: This example provides a cloud-edge-coordinated drone target detection, tracking, and trajectory prediction device, which can be used in drone security systems for large-scale event venues. To ensure safety during major sporting events, concerts, and other large-scale public events, a cloud-edge-coordinated drone target detection, tracking, and trajectory prediction device designed specifically for these scenarios is deployed to prevent potential aerial threats.

[0071] To address the high density of people and complex radio signal environments expected at event sites, the device employed a dedicated, low-latency, high-bandwidth communication link to ensure unimpeded data transmission. Furthermore, leveraging edge computing, a temporary data center was established near the site to rapidly process data from each drone and enable timely decision-making.

[0072] The Drone Target Detection Module and the Drone Target Tracking Module provide rapid target detection and stable tracking. This combined solution is particularly well-suited for rapidly changing dynamic scenarios, enabling rapid lock-on and tracking of any suspicious drone.

[0073] Taking into account factors such as the possible flow of people and changes in air flow during large-scale events, the drone trajectory prediction module aims to improve the accuracy of future path predictions, thereby better assisting security personnel in formulating emergency plans and ensuring the smooth progress of the event.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A cloud-edge collaborative drone target detection, tracking, and trajectory prediction device, characterized by: The device includes a device hardware module, a drone target detection module, a drone target tracking module, a drone trajectory prediction module and a cloud-edge collaboration module; The device hardware module is used to collect images of the target drone and transmit them to the drone target detection module; The drone target detection module is used to detect the image of the target drone, obtain an image stream marked with the category and bounding box coordinates of the target drone, and transmit it to the drone trajectory prediction module; The UAV target tracking module is used to track the target based on the image stream marked with the category and bounding box coordinates of the target UAV based on the lightweight Deepsort algorithm, obtain the motion trajectory set of all target UAVs and transmit it to the UAV trajectory prediction module; The UAV trajectory prediction module is used to predict the final motion prediction trajectory set of all target UAVs through a generative adversarial network based on the motion trajectory set of all target UAVs and transmit it to the cloud-edge collaboration module; The cloud-edge collaboration module is used to transmit the motion prediction trajectory set of all target drones to the cloud.

2. The cloud-edge collaborative drone target detection, tracking, and trajectory prediction device according to claim 1, characterized in that: The device hardware modules include a storage device, a high-performance embedded computing unit, an image acquisition module and a power supply module; The storage device is used to provide a large-capacity flash memory or SSD; The high-performance embedded computing unit is used to provide high-computing power chips for the operation of the drone target detection module, drone target tracking module, drone trajectory prediction module and cloud-edge collaboration module respectively; The image acquisition module is used to acquire images of the target drone using a high-resolution camera, and transmit the acquired images of the target drone to the drone target detection module; The power module is used to provide battery and intelligent power monitoring.

3. The cloud-edge collaborative drone target detection, tracking, and trajectory prediction device according to claim 1, characterized in that: The cloud-edge collaboration module includes a communication module and an edge computing service module; The communication module is used to achieve full-duplex communication between the cloud and the edge of the drone based on network socket and message queue telemetry transmission; The edge computing service module is used to provide edge computing service functions at the edge end.

4. The cloud-edge collaborative drone target detection, tracking, and trajectory prediction device according to claim 1, characterized in that: The drone target detection module detects the target drone image, obtains an image stream marked with the target drone category and bounding box coordinates, and transmits it to the drone trajectory prediction module. Specifically, it includes the following sub-steps: (a.1) First, the image of the target drone collected by the drone target detection module Perform preprocessing to obtain the preprocessed image ; The preprocessing includes size adjustment, normalization and channel alignment; (a.2) Then the preprocessed image Input into the lightweight MobileNetv3 backbone network for feature extraction to obtain the feature map The MobileNetv3 backbone network includes a depthwise separable convolutional layer, a channel attention module, and a Hard-Swish activation function. (a.3) Feature maps based on the Coordinate Attention mechanism Perform attention enhancement to obtain the feature map after attention enhancement ; (a.4) Feature map after attention is added The input is sent to the decoupled detection head, which outputs an image stream marked with the category and bounding box coordinates of the target drone and outputs it to the drone target tracking module.

5. The cloud-edge collaborative drone target detection, tracking, and trajectory prediction device according to claim 1, characterized in that: The UAV target tracking module performs target tracking based on the image stream marked with the category and bounding box coordinates of the target UAV based on the lightweight Deepsort algorithm, obtains the motion trajectory set of all target UAVs and transmits it to the UAV trajectory prediction module, specifically including the following sub-steps: (b.1) Based on the image stream marked with the target drone’s category and bounding box coordinates, the DeepSort algorithm is used to extract the appearance features and motion features in the bounding boxes of all target drones; (b.2) Using the Kalman filter to predict the position of each target drone in the next frame based on its appearance and motion features, the Hungarian algorithm and cascade matching algorithm are used to calculate the matching degree between all two target drones in the previous and next frames. When the matching degree exceeds the matching threshold, the two corresponding target drones are assigned the same ID. (b.3) Finally, the positions of target drones with the same ID are connected in sequence to form a set of motion trajectories of all target drones and transmitted to the drone trajectory prediction module.

6. The cloud-edge collaborative UAV target detection, tracking, and trajectory prediction device according to claim 1, characterized in that: The UAV trajectory prediction module obtains the final motion prediction trajectory set of all target UAVs based on the motion trajectory set of all target UAVs through the generative adversarial network and transmits it to the cloud-edge collaboration module. Specifically, it includes the following sub-steps: (c.1) First, the motion trajectory set of all target UAVs is predicted through the encoder-decoder structure of the Autoformer to obtain the initial motion prediction trajectory set; (c.2) Then, based on the initial motion prediction trajectory set, the final motion prediction trajectory set of all target drones is obtained through the generative adversarial network and transmitted to the cloud-edge collaboration module.

7. The cloud-edge collaborative UAV target detection, tracking, and trajectory prediction device according to claim 6, characterized in that: The sub-step (c.1) specifically includes the following sub-steps: (c.1.1) The set of motion trajectories of all target drones ,in, Represents the motion trajectory of any target drone, represents the number of target drones, ; The motion trajectory of each target drone Input to the encoder-decoder structure of Autoformer, first encode, with encoder layers, The overall equation of the layer encoder is ,in, represents the encoder layer; Indicates the The output of the layer encoder, Indicates the The output of the layer encoder, ; through After encoding through the encoder layers, the final encoded output is obtained ; (c.1.2) Then, for each final encoded output To decode, decoder layer, The overall equation of the layer decoder is , in, represents the decoder layer, Indicates the The output of the layer decoder, Indicates the The output of the layer decoder, ; go through After decoding through the decoder layers, the final decoded output is obtained And as the initial motion prediction trajectory of the corresponding target UAV, the initial motion prediction trajectory set is obtained : .

8. The cloud-edge collaborative drone target detection, tracking, and trajectory prediction device according to claim 7, characterized in that: The said Output of layer decoder The specific steps include the following: (d.1) First, Output of layer decoder Apply the Auto-Correlation mechanism and then auto-correlate the results And add output Perform time series decomposition operation and decompose it into the first seasonal part result and the first trend-cyclical part results ; (d.2) Then the first seasonal part result and the final encoded output Apply the Auto-Correlation mechanism and then auto-correlate the results And add the first seasonal part results Perform time series decomposition operation and decompose it into the second seasonal part results and the second trend-cyclical part of the results ; (d.3) Then the second seasonal part result Apply a feedforward network layer and then pass the output of the feedforward network layer to And add the second seasonal part results Perform time series decomposition operation and decompose it into the third seasonal part results and the third trend - cyclical part results ; Finally, the third seasonal part results As the first Output of layer decoder .

9. The cloud-edge collaborative UAV target detection, tracking, and trajectory prediction device according to claim 6, characterized in that: The sub-step (c.2) is specifically: The initial motion prediction trajectory set The initial motion prediction trajectory of each target UAV The input is fed into a generative adversarial network, which includes a generator and a discriminator. The generator and the discriminator are alternately optimized in combination with a diversity loss function until the generative adversarial network converges, and the network parameters are fixed to obtain an optimized generative adversarial network. Finally, the initial motion prediction trajectory set is optimized through the generative adversarial network Perform trajectory prediction to obtain the final motion prediction trajectory set of all target drones and transmit it to the cloud-edge collaboration module.

Images