A cloud-edge collaborative unmanned aerial vehicle target detection tracking and trajectory prediction device

Abstract

The application discloses a kind of cloud edge coordination's unmanned plane target detection tracking and trajectory prediction device, belong to Internet of Things, artificial intelligence and unmanned plane technical field.The device includes device hardware module, unmanned plane target detection module, unmanned plane target tracking module, unmanned plane trajectory prediction module and cloud edge coordination module.For the problems such as slow response speed, low detection accuracy and system difficult to adapt to complex environment of traditional method, based on hardware device, introduce lightweight artificial intelligence algorithm to carry out high-precision target tracking, detection and trajectory prediction to potential threat unmanned plane;Real-time information coordination is realized based on efficient cloud edge coordination technology;The application can rapidly identify, track and predict the trajectory of potential threat unmanned plane in complex environment.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of Internet of Things, Artificial Intelligence and Unmanned Aerial Vehicle, and particularly relates to a cloud-edge collaborative unmanned aerial vehicle target detection, tracking and trajectory prediction device. BACKGROUND

[0002] With the rapid development of unmanned aerial vehicle technology and the continuous expansion of its application fields, the use of unmanned aerial vehicles in civilian and military fields is increasing. However, the increase in the number of unmanned aerial vehicles also brings new challenges to airspace safety, such as unauthorized unmanned aerial vehicle activities that may pose a threat to national security, public safety and personal privacy. In order to cope with these potential threats, it is necessary to develop a device for detecting, tracking and predicting the behavior trajectory of unmanned aerial vehicles, which can quickly respond to ensure the safety of the airspace.

[0003] In recent years, with the development of Internet of Things and Artificial Intelligence technology, some unmanned aerial vehicle target detection, tracking and trajectory prediction methods based on Internet of Things have emerged. Advanced artificial intelligence and machine learning algorithms, including deep learning and reinforcement learning, have significantly improved the accuracy of target detection, recognition and classification, especially in image recognition and video analysis, where convolutional neural networks and recurrent neural networks and their variants have performed particularly well. In addition, with the help of cloud-edge collaboration provided by Internet of Things technology, real-time data transmission and remote control are realized, making unmanned aerial vehicles better integrate into smart city, intelligent transportation and other application scenarios, thereby greatly enhancing the overall performance and application range of the system.

[0004] However, traditional methods have some significant shortcomings in unmanned aerial vehicle target detection, tracking and trajectory prediction: slow response speed: traditional methods rely on ground stations or single centralized processing units, which may cause data transmission delays and time consumption in the decision-making process, thereby reducing the real-time performance and response efficiency of the system. Low detection accuracy: due to environmental factors such as weather conditions and changes in lighting, traditional methods have difficulty maintaining stable high-precision detection rates, especially in complex and variable environments. Poor adaptability: existing systems are usually designed for specific scenarios, and their performance may drop significantly when encountering new or unexpected situations, making it difficult to adjust flexibly to different operating environments. Limited information processing capability: most traditional unmanned aerial vehicle monitoring systems do not have powerful data analysis capabilities and intelligent decision support functions, making it difficult to effectively manage large-scale, high-speed moving targets.

[0005] To solve the above problems, the present application proposes a cloud-edge collaborative unmanned aerial vehicle target detection, tracking and trajectory prediction device. Based on the soft and hard integrated device, combined with advanced artificial intelligence algorithms, the detection accuracy, response speed, adaptability and stability of the system are improved.

[0006] In summary, the present application aims to overcome the limitations of the prior art, and provide a more intelligent, fast response and adaptive soft and hard integrated device to improve the ability of unmanned aerial vehicle target detection tracking and trajectory prediction, and enhance airspace safety. SUMMARY

[0007] The present application aims to overcome the limitations of the prior art, and provide a more intelligent, fast response and adaptive soft and hard integrated device to improve the ability of unmanned aerial vehicle target detection tracking and trajectory prediction, and enhance airspace safety.

[0008] The present application aims to overcome the limitations of the prior art, and provide a more intelligent, fast response and adaptive soft and hard integrated device to improve the ability of unmanned aerial vehicle target detection tracking and trajectory prediction, and enhance airspace safety.

[0009] The device hardware module is used to collect the image of the target unmanned aerial vehicle and transmit it to the unmanned aerial vehicle target detection module;

[0010] The unmanned aerial vehicle target detection module is used to detect the image of the target unmanned aerial vehicle, obtain the image stream marked with the category and bounding box coordinates of the target unmanned aerial vehicle, and transmit it to the unmanned aerial vehicle trajectory prediction module;

[0011] The unmanned aerial vehicle target tracking module is used to track the target based on the lightweight Deepsort algorithm according to the image stream marked with the category and bounding box coordinates of the target unmanned aerial vehicle, obtain the motion trajectory set of all target unmanned aerial vehicles, and transmit it to the unmanned aerial vehicle trajectory prediction module;

[0012] The unmanned aerial vehicle trajectory prediction module is used to predict the final motion trajectory set of all target unmanned aerial vehicles through a generative adversarial network based on the motion trajectory set of all target unmanned aerial vehicles, and transmit it to the cloud edge collaboration module;

[0013] The cloud edge collaboration module is used to transmit the motion trajectory set of all target unmanned aerial vehicles to the cloud.

[0014] Further, the device hardware module includes a storage device, a high-performance embedded computing unit, an image acquisition module and a power module;

[0015] The storage device is used to provide a large-capacity flash memory or SSD;

[0016] The high-performance embedded computing unit is used to provide high-performance chips for the operation of the unmanned aerial vehicle target detection module, the unmanned aerial vehicle target tracking module, the unmanned aerial vehicle trajectory prediction module and the cloud edge collaboration module;

[0017] The image acquisition module is used to acquire images of the target unmanned aerial vehicle through a high-resolution camera, and transmit the acquired images of the target unmanned aerial vehicle to the unmanned aerial vehicle target detection module.

[0018] The power module is used to provide battery and intelligent power monitoring.

[0019] Furthermore, the cloud-edge collaboration module includes a communication module and an edge computing service module;

[0020] The communication module is used to achieve full-duplex communication between the cloud and the edge of the drone based on network sockets and message queue telemetry transmission;

[0021] The edge computing service module is used to provide edge computing service functions at the edge.

[0022] Furthermore, the UAV target detection module detects the image of the target UAV, obtains an image stream labeled with the category and bounding box coordinates of the target UAV, and transmits it to the UAV trajectory prediction module, specifically including the following sub-steps:

[0023] (a.1) First, the image of the target UAV acquired by the UAV target detection module. Preprocessing is performed to obtain the preprocessed image. The preprocessing includes size adjustment, normalization, and channel alignment.

[0024] (a.2) The preprocessed image is then... The input is fed into the lightweight MobileNetv3 backbone network for feature extraction, resulting in feature maps. The MobileNetv3 backbone network includes depthwise separable convolutional layers, channel attention modules, and a Hard-Swish activation function.

[0025] (a.3) Based on the Coordinate Attention mechanism, the feature map is processed. Attention enhancement is performed to obtain the feature map after attention enhancement. ;

[0026] (a.4) Feature map after attention is added The input is fed into the decoupled detection head, and the output is an image stream labeled with the category and bounding box coordinates of the target UAV, which is then sent to the UAV target tracking module.

[0027] Furthermore, the UAV target tracking module performs target tracking based on a lightweight Deepsort algorithm using an image stream labeled with the category and bounding box coordinates of the target UAV, obtaining a set of motion trajectories for all target UAVs and transmitting it to the UAV trajectory prediction module. This process specifically includes the following sub-steps:

[0028] (b.1) extracting the appearance features and motion features in the bounding boxes of all target UAVs based on the DeepSort algorithm according to the image stream marked with the category and bounding box coordinates of the target UAV;

[0029] (b.2) predicting the positions of each target UAV in the next frame using a Kalman filter according to the appearance features and motion features, and calculating the matching degree between all pairs of target UAVs in the previous and subsequent frames using a Hungarian algorithm and a cascade matching algorithm, and assigning the same ID to the corresponding two target UAVs when the matching degree is greater than the matching threshold;

[0030] (b.3) finally connecting the positions of target UAVs with the same ID in sequence to form a set of motion trajectories of all target UAVs and transmitting them to the UAV trajectory prediction module.

[0031] Further, the UAV trajectory prediction module predicts a final set of motion prediction trajectories of all target UAVs through a generative adversarial network according to the set of motion trajectories of all target UAVs and transmits them to the cloud-edge collaboration module, specifically including the following sub-steps:

[0032] (c.1) first predicting the set of motion prediction trajectories through the encoder-decoder structure of Autoformer to obtain an initial set of motion prediction trajectories;

[0033] (c.2) then obtaining the final set of motion prediction trajectories of all target UAVs through a generative adversarial network according to the initial set of motion prediction trajectories and transmitting them to the cloud-edge collaboration module.

[0034] Further, the sub-step (c.1) specifically includes the following sub-steps:

[0035] (c.1.1) the set of motion trajectories of all target UAVs , wherein, represents the motion trajectory of any one target UAV, represents the number of target UAVs, ;

[0036] inputting the motion trajectory of each target UAV into the encoder-decoder structure of Autoformer, first encoding, assuming encoder layers, the overall equation of the th encoder layer is , wherein, represents the encoder layer; represents the output of the th encoder layer, represents the output of the th encoder layer, ;

[0037] After encoding by the encoder layers, the final encoding outputs ;

[0038] (c.1.2) then decoding each of the final encoding outputs respectively, with decoder layers, the overall equation of the layer decoder is , where represents the decoder layer, represents the output of the layer decoder, represents the output of the ;

[0039] After decoding by the decoder layers, the final decoding outputs are obtained and used as the initial motion prediction trajectories of the corresponding target UAVs, and a set of initial motion prediction trajectories is obtained: .

[0040] Further, the output of the layer decoder specifically includes the following sub-steps:

[0041] (d.1) first applying an Auto-Correlation mechanism to the output of the layer decoder, then performing a time series decomposition operation on the Auto-Correlation result and adding the output to obtain a first seasonal part result and a first trend-cyclical part result ;

[0042] (d.2) then applying an Auto-Correlation mechanism to the first seasonal part result and the final encoding output , then performing a time series decomposition operation on the Auto-Correlation result and adding the first seasonal part result to obtain a second seasonal part result and a second trend-cyclical part result ;

[0043] (d.3) then adding the second seasonal part result Apply a feedforward network layer, and then use the output of the feedforward network layer. And add the results of the second seasonal component. Perform time series decomposition to separate the results into a third seasonal component. And the third trend - cyclical partial results Finally, the results of the third seasonal component will be presented. As the first The output of the layer decoder .

[0044] Furthermore, the sub-step (c.2) specifically includes:

[0045] The initial set of predicted motion trajectories Initial motion prediction trajectory for each target UAV The input is fed into a generative adversarial network, which includes a generator and a discriminator; and the generator and discriminator are alternately optimized by combining a diversity loss function until the generative adversarial network converges. The network parameters are then fixed to obtain the optimized generative adversarial network.

[0046] Finally, the optimized generative adversarial network is used to predict the initial motion trajectory set. Trajectory prediction is performed to obtain the final set of predicted motion trajectories for all target drones and transmit them to the cloud-edge collaboration module.

[0047] The beneficial effects of this invention are:

[0048] 1) Based on the device hardware module and cloud-edge collaboration module, it supports real-time data processing and cloud-edge communication, can adapt to complex network environments, and ensures data security and system maintainability;

[0049] 2) Based on the UAV target detection module and UAV target tracking module, high-precision detection and real-time stable tracking of small target UAVs are achieved, effectively dealing with fast-moving targets;

[0050] 3) Based on the UAV trajectory prediction module, the Autoformer model is used to capture long-term flight dependencies, and multimodal trajectories are generated by combining GAN diversity loss, which improves the ability to predict complex flight behaviors and enhances the countermeasure effect. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of a cloud-edge collaborative drone target detection, tracking, and trajectory prediction device.

[0052] Figure 2 This is a schematic diagram of the device's hardware modules;

[0053] Figure 3 This is a schematic diagram of the cloud-edge collaboration module;

[0054] Figure 4 A flowchart for detecting images of a target drone;

[0055] Figure 5 This is a flowchart illustrating target tracking based on a lightweight Deepsort algorithm using an image stream labeled with the target UAV's category and bounding box coordinates. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0057] Example 1: As Figure 1 As shown, the present invention provides a cloud-edge collaborative drone target detection, tracking and trajectory prediction device. The device includes a device hardware module, a drone target detection module, a drone target tracking module, a drone trajectory prediction module and a cloud-edge collaborative module. The device hardware module is connected to the drone target detection module, the drone target tracking module, the drone trajectory prediction module and the cloud-edge collaborative module respectively.

[0058] I. Device Hardware Modules

[0059] The device hardware module is used to acquire images of the target UAV and transmit them to the UAV target detection module.

[0060] like Figure 2 As shown, the hardware module of the device is mounted on the user's own UAV and includes a storage device, a high-performance embedded computing unit, an image acquisition module, and a power supply module.

[0061] The storage device is used to provide large-capacity flash memory or SSD, providing ample storage space and supporting fast read and write operations; it supports hot-swapping and redundant backup, facilitating maintenance and replacement while ensuring no data loss; it supports data compression and encryption, reducing storage usage and protecting sensitive information.

[0062] The high-performance embedded computing unit provides high-computing-power chips for the operation of the UAV target detection module, UAV target tracking module, UAV trajectory prediction module, and cloud-edge collaboration module, ensuring real-time processing capabilities.

[0063] The image acquisition module is configured to acquire images of the target UAV through a high-resolution camera, and transmit the acquired images to the UAV target detection module.

[0064] The power module is configured to provide a battery and intelligent power monitoring, monitor the battery state in real time, and automatically adjust the working mode.

[0065] Second, the cloud edge collaboration module

[0066] The cloud edge collaboration module is configured to transmit the trajectory prediction result output by the UAV trajectory prediction module from the edge end of the UAV to the cloud end.

[0067] As shown in Figure 3 The cloud edge collaboration module includes a communication module and an edge computing service module.

[0068] The communication module realizes full-duplex communication between the cloud end and the edge end of the UAV based on Websocket (network socket) and MQTT (message queue telemetry transfer), realizes collaboration between the trajectory prediction result of the edge end and the information of the cloud end, and collects, circulates and analyzes the trajectory prediction result in real time to facilitate collaborative decision-making of the cloud end; supports authentication and authorization of the cloud edge integrated system framework to the edge end device and effective management of the edge end device; supports information collaboration, remote operation and maintenance, and offline autonomy of the cloud edge.

[0069] The edge computing service module provides edge computing service functions at the edge end, including edge container management, edge node management and edge computing system. The edge computing service module provides a cloud edge collaboration console visual interface for the cloud end for easy operation, including edge device instance management, edge device access management, edge application management and edge drive management.

[0070] Third, the UAV target detection module

[0071] The UAV target detection module is based on the YOLOX algorithm, uses a lightweight MobileNetv3 backbone network as a feature extractor, and introduces a Coordinate Attention mechanism for coordinate-aware attention enhancement. The UAV target detection module is configured to use a high-precision UAV target detection algorithm to detect potentially threatening UAVs with high precision.

[0072] The UAV target detection module is configured to detect the image of the target UAV, obtain an image stream labeled with the category and bounding box coordinates of the target UAV, and transmit the image stream to the UAV trajectory prediction module. As shown in Figure 4 The UAV target detection module includes the following sub-steps:

[0073] (a.1) Image preprocessing: first, the image of the target UAV collected by the UAV target detection module is preprocessed to obtain a preprocessed image The preprocessing includes size adjustment, normalization, and channel alignment.

[0074] (a.2) Lightweight feature extraction: the preprocessed image is then input into a lightweight MobileNetv3 backbone network for feature extraction to obtain a feature map .

[0075] The sub-step (a.2) is specifically:

[0076] The MobileNetv3 backbone network includes a depth separable convolution layer, a channel attention (Squeeze-and-Excitation) module, and a Hard-Swish activation function.

[0077] The preprocessed image is first subjected to depth convolution and point-wise convolution in sequence via the depth separable convolution layer, then input into the channel attention module for feature map enhancement, and finally input into the Hard-Swish activation function to obtain a feature map .

[0078] (a.3) Coordinate-aware attention enhancement: the feature map is subjected to attention enhancement according to the Coordinate Attention mechanism to obtain an attention-enhanced feature map . .

[0079] After the output of the backbone network, the Coordinate Attention mechanism is introduced for coordinate-aware attention enhancement.

[0080] The sub-step (a.3) is specifically:

[0081] (a.3.1) The feature map is , where represents that the feature map contains channels, represents a sub-feature map of the th channel, .

[0082] The sub-feature map is , where represents a feature value at a height h and a width w in the sub-feature map of the th channel,​​ , , represents the height of each sub-feature map, represents the width of each sub-feature map.

[0083] (a.3.2) Firstly, the feature values at height h in each channel of the feature map are respectively encoded in the horizontal direction to obtain the feature set at height h: wherein, represents the horizontal direction global average pooling result of the cth channel at height h, .

[0084] Subsequently, the feature values at width w in each channel of the feature map are respectively encoded in the vertical direction to obtain the feature set at width w: wherein, represents the vertical direction global average pooling result of the cth channel at width w, .

[0085] Subsequently, the feature set and the feature set are spliced, and are sequentially subjected to 1x1 convolution operation, batch normalization operation and linear activation function operation to obtain the intermediate feature set , and the calculation formula is as follows:

[0086] ;

[0087] wherein, represents the splicing of the feature set and the feature set , represents the 1x1 convolution operation, represents the batch normalization operation, represents the linear activation function operation.

[0088] Subsequently, the intermediate feature set is split along the spatial dimension to obtain the intermediate feature set at height h and the intermediate feature set at width w, and according to the intermediate feature set and the intermediate feature set , the corresponding attention weight map at height h and the attention weight map at width w are obtained after being sequentially subjected to 1x1 convolution operation and Sigmoid activation function processing, and the calculation formula is as follows:

[0089] ;

[0090] ;

[0091] wherein, represents a Sigmoid activation function.

[0092] the attention weight map at the height h is wherein, represents the attention weight at the height h of the cth channel.

[0093] the attention weight map at the width w is wherein, represents the attention weight at the width w of the cth channel.

[0094] the feature value is re-labeled using the attention weight at the height h of the cth channel and the attention weight at the width w of the cth channel to obtain the enhanced feature value at the height h and the width w of the cth channel .

[0095] (a.3.3) repeating step (a.3.2) to re-label each feature value in the feature map to obtain the attention-increased feature map : wherein, represents the attention-increased sub-feature map of the cth channel.

[0096] the attention-increased sub-feature map is .

[0097] (a.4) detection result output: the attention-increased feature map is then input to a decoupled detection head to output an image stream labeled with the category and bounding box coordinates of the target UAV and output to a UAV target tracking module.

[0098] The UAV target detection module is based on the YOLOX algorithm, uses a lightweight MobileNetv3 backbone network as a feature extractor, can significantly reduce the model parameters and computational complexity while maintaining high accuracy, thereby accelerating the feature extraction process and supporting efficient real-time detection on mobile devices.

[0099] IV. UAV target tracking module ​​

[0100] The UAV target tracking module is configured to use a lightweight UAV target tracking algorithm to perform high-precision target tracking on potentially threatening UAVs on a hardware device with limited computing resources.

[0101] The UAV target tracking module is configured to perform target tracking based on a lightweight Deepsort algorithm according to an image stream marked with the categories and bounding box coordinates of target UAVs, to obtain a set of motion trajectories of all target UAVs and transmit the set to the UAV trajectory prediction module, as shown in Figure 5

[0102] (b.1) Feature extraction: based on the DeepSort algorithm, the image stream marked with the categories and bounding box coordinates of target UAVs is used to extract the appearance features and motion features in the bounding boxes of all target UAVs, which will be used for subsequent target matching and tracking.

[0103] (b.2) Target matching: based on the appearance features and motion features, a Kalman filter is used to predict the position of each target UAV in the next frame, and a Hungarian algorithm and a cascade matching algorithm are used to calculate the matching degree between all pairs of target UAVs in the previous and subsequent frames. When the matching degree is greater than the matching threshold, the corresponding two target UAVs are assigned the same ID. This step is to find the corresponding relationship of the same target UAV in each frame and assign a unique ID to each tracked target UAV. In this embodiment, the matching threshold is 0.7.

[0104] (b.3) Target tracking: finally, the positions of target UAVs with the same ID are connected in sequence to form a set of motion trajectories of all target UAVs and transmit the set to the UAV trajectory prediction module, thereby realizing UAV target tracking.

[0105] V. UAV trajectory prediction module

[0106] After obtaining the data from the UAV trajectory tracking module, the UAV trajectory prediction module performs high-precision real-time position sequence prediction on the UAV target trajectory, thereby improving the air safety and the ability to deal with illegal or malicious UAVs. The UAV trajectory prediction module is modeled based on long-term dependence capture and randomness simulation of trajectories.

[0107] The UAV trajectory prediction module predicts the final motion prediction trajectory set of all target UAVs based on the set of motion trajectories of all target UAVs through a generative adversarial network and transmits the set to the cloud-edge collaboration module, specifically including the following sub-steps:

[0108] ​(c.1) Firstly, the motion trajectory set of all target UAVs is predicted by the encoder-decoder structure of Autoformer to obtain a motion prediction trajectory set.

[0109] The sub-step (c.1) specifically comprises the following sub-steps:

[0110] (c.1.1) the motion trajectory set of all target UAVs , wherein, represents the motion trajectory of any one target UAV, represents the number of target UAVs, .

[0111] The motion trajectory of each target UAV is input into the encoder-decoder structure of Autoformer, and is firstly encoded, which is provided with encoder layers, and the overall equation of the th encoder layer is , wherein, represents the encoder layer; represents the output of the th encoder layer, represents the output of the th encoder layer, ; after being encoded by encoder layers, the final encoding output is obtained.

[0112] (c.1.2) Then, decoding is performed, which is provided with decoder layers, and the overall equation of the th decoder layer is , wherein, represents the decoder layer, represents the output of the th decoder layer, represents the output of the th decoder layer, ; after being decoded by decoder layers, the final decoding output is obtained, and the final decoding output is taken as the motion prediction trajectory of the corresponding target UAV to obtain a motion prediction trajectory set : .

[0113] The specific calculation process of the output of the th decoder layer is as follows:

[0114] (d.1) Firstly, the output of the th decoder layer application of the Auto-Correlation mechanism, followed by the auto-correlation result and adding the output a time series decomposition operation to decompose into a first seasonal component result and a first trend-cyclical component result , the calculation formula is: ;

[0115] wherein, denotes the auto-correlation operation; denotes the time series decomposition operation.

[0116] (d.2) followed by the first seasonal component result and the final encoded output application of the Auto-Correlation mechanism, followed by the auto-correlation result and adding the first seasonal component result a time series decomposition operation to decompose into a second seasonal component result and a second trend-cyclical component result , the calculation formula is:

[0117] .

[0118] (d.3) then the second seasonal component result application of the feedforward network layer, followed by the output of the feedforward network layer and adding the second seasonal component result a time series decomposition operation to decompose into a third seasonal component result and a third trend-cyclical component result , the calculation formula is:

[0119] ;

[0120] wherein, denotes the feedforward network layer.

[0121] finally, the third seasonal component result is taken as the output of the first layer decoder , that is .

[0122] (c.2) followed by the final motion prediction trajectory set of all target UAVs obtained by the generative adversarial network according to the motion prediction trajectory set and transmitted to the cloud-edge collaboration module.

[0123] The sub-step (c.2) is specifically:

[0124] a set of initial motion prediction trajectories of each target UAV a set of initial motion prediction trajectories of each target UAV into a generative adversarial network comprising a generator and a discriminator, and alternately optimizing the generator and the discriminator in combination with a diversity loss function until the generative adversarial network converges, fixing the network parameters to obtain an optimized generative adversarial network.

[0125] a set of initial motion prediction trajectories of each target UAV performing trajectory prediction to obtain a set of final motion prediction trajectories of all target UAVs and transmitting the set to a cloud-edge collaboration module.

[0126] Embodiment 2: The present application provides a cloud-edge collaborative UAV target detection and tracking and trajectory prediction device, which is particularly suitable for resource-limited environments and is used in the cloud-edge collaborative UAV target detection and tracking and trajectory prediction device of Embodiment 1, specifically comprising:

[0127] The UAV target detection module uses an optimized YOLOX algorithm combined with a lightweight convolutional neural network MobileNetv3 and a Coordinate Attention mechanism, which can significantly reduce the computational resource requirement while maintaining high accuracy, supporting fast and real-time target detection on mobile devices. By replacing the original large feature extractor CSPDarknet53 with a more compact MobileNetv3 and introducing a CoordinateAttention layer that enhances the focus on target position information, the detection performance for small targets such as UAVs is greatly improved, while simplifying the training process and reducing the workload of hyperparameter adjustment.

[0128] To ensure that the detected targets can be stably tracked, the UAV target tracking module based on the DeepSort algorithm is integrated in this embodiment. The UAV target tracking module solves the problem of ID reassignment caused by occlusion in traditional Sort algorithms, improving the accuracy of target identity matching. DeepSort manages the trajectory life cycle by introducing two trajectory state management states: confirmed and unconfirmed, and using a feature re-identification network to extract target appearance features and measure similarity. In addition, Kalman filter is used for motion feature matching calculation, further enhancing the accuracy and stability of tracking. Overall, these two modules complement each other and together provide a complete UAV monitoring system, significantly improving the ability of air safety prevention.

[0129] Finally, for the properties of low-altitude object interaction and external complex factors, based on the UAV trajectory prediction module, the trajectory of potential threat UAVs is predicted with high-precision real-time position sequence, thereby improving the air safety and the ability to deal with illegal or malicious UAVs.

[0130] Embodiment 3: This embodiment provides a cloud-edge collaborative UAV target detection and tracking and trajectory prediction device, which can be used in a large-scale event site UAV security system. To ensure safety during major sports events, concerts or other large public events, a cloud-edge collaborative UAV target detection and tracking and trajectory prediction device specially designed for such scenarios is deployed to prevent potential air threats.

[0131] In view of the high-density crowd and complex radio signal environment that may exist on the event site, the device uses a low-latency, high-bandwidth dedicated communication link to ensure smooth data transmission. At the same time, taking advantage of edge computing, a temporary data center is set up near the site to quickly process data from each UAV and make timely decisions.

[0132] The UAV target detection module and the UAV target tracking module perform rapid target detection to achieve stable tracking. This combined solution is particularly suitable for responding to dynamic scenarios that change rapidly, and can quickly lock and track any suspicious UAV.

[0133] Considering factors such as crowd surges and airflow changes that may exist in large events, the UAV trajectory prediction module aims to improve the accuracy of future path prediction, thereby better assisting security personnel in developing emergency plans and ensuring the smooth progress of the event.

[0134] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A cloud-edge collaborative unmanned aerial vehicle (UAV) target detection, tracking and trajectory prediction apparatus, characterized in that, The device comprises a device hardware module, a UAV target detection module, a UAV target tracking module, a UAV trajectory prediction module, and a cloud-edge collaboration module; The device hardware module is configured to collect images of target UAVs and transmit the images to the UAV target detection module; The UAV target detection module is configured to detect the images of the target UAVs, obtain an image stream marked with the categories and bounding box coordinates of the target UAVs, and transmit the image stream to the UAV trajectory prediction module; The UAV target tracking module is configured to perform target tracking based on a lightweight Deepsort algorithm according to the image stream marked with the categories and bounding box coordinates of the target UAVs, obtain a motion trajectory set of all target UAVs, and transmit the motion trajectory set to the UAV trajectory prediction module; The UAV trajectory prediction module is configured to predict a final motion prediction trajectory set of all target UAVs according to the motion trajectory set of all target UAVs by using a generative adversarial network, and transmit the final motion prediction trajectory set to the cloud-edge collaboration module; The cloud-edge collaboration module is configured to transmit the motion prediction trajectory set of all target UAVs to the cloud. 2.The cloud-edge collaborative unmanned aerial vehicle target detection, tracking and trajectory prediction apparatus of claim 1, wherein, The device hardware module comprises a storage device, a high-performance embedded computing unit, an image acquisition module, and a power module; The storage device is configured to provide a large-capacity flash memory or an SSD; The high-performance embedded computing unit is configured to provide high-performance chips for the operation of the UAV target detection module, the UAV target tracking module, the UAV trajectory prediction module, and the cloud-edge collaboration module, respectively; The image acquisition module is configured to acquire images of target UAVs by using a high-resolution camera, and transmit the images to the UAV target detection module; The power module is configured to provide a battery and intelligent power monitoring. 3.The cloud-edge collaborative unmanned aerial vehicle target detection, tracking and trajectory prediction apparatus of claim 1, wherein, The cloud-edge collaboration module comprises a communication module and an edge computing service module; The communication module is configured to realize full-duplex communication between the cloud and the edge of the UAV based on network sockets and message queue telemetry transmission; The edge computing service module is configured to provide edge computing service functions at the edge. 4.The cloud-edge collaborative unmanned aerial vehicle target detection, tracking and trajectory prediction apparatus of claim 1, wherein, The UAV target detection module detects the images of the target UAVs, obtains an image stream marked with the categories and bounding box coordinates of the target UAVs, and transmits the image stream to the UAV trajectory prediction module, specifically including the following sub-steps: (a.1) first, the image of the target UAV collected by the UAV target detection module is preprocessed to obtain a preprocessed image is preprocessed to obtain a preprocessed image ; the preprocessing includes size adjustment, normalization and channel alignment; (a.2) the pre-processed image is then inputted into a lightweight MobileNetv3 backbone network for feature extraction to obtain a feature map Input into a lightweight MobileNetv3 backbone network for feature extraction to obtain a feature map The MobileNetv3 backbone network comprises a depth separable convolution layer, a channel attention module and a Hard-Swish activation function (a.3) performing attention enhancement on the feature map according to the Coordinate Attention mechanism to obtain an attention-increased feature map ;​ (a.4) the attention-increased feature map The input is decoupled detection head, output image stream marked with the category and boundary box coordinates of the target UAV and output to the UAV target tracking module. 5.The cloud-edge collaborative unmanned aerial vehicle target detection, tracking and trajectory prediction apparatus of claim 1, wherein, The UAV target tracking module performs target tracking based on a lightweight Deepsort algorithm according to the image stream marked with the categories and bounding box coordinates of the target UAVs, obtains a motion trajectory set of all target UAVs, and transmits the motion trajectory set to the UAV trajectory prediction module, specifically including the following sub-steps: (b.1) Extracting the appearance features and motion features in the bounding boxes of all target UAVs based on the DeepSort algorithm according to the image stream marked with the categories and bounding box coordinates of the target UAVs; (b.2) Using a Kalman filter to predict the positions of each target UAV in the next frame according to the appearance features and motion features, and using a Hungarian algorithm and a cascade matching algorithm to calculate the matching degree between all pairs of target UAVs in the two frames, and assigning the same ID to the corresponding two target UAVs when the matching degree is greater than a matching threshold. (b.3) Finally, the positions of the target drones with the same ID are sequentially connected to form a set of motion trajectories of all target drones and transmitted to the drone trajectory prediction module. 6.The cloud-edge collaborative unmanned aerial vehicle target detection, tracking and trajectory prediction apparatus of claim 1, wherein, The drone trajectory prediction module predicts a set of final motion prediction trajectories of all target drones through a generative adversarial network according to the set of motion trajectories of all target drones and transmits the set of final motion prediction trajectories to the cloud-edge collaboration module, and specifically includes the following sub-steps: (c.1) First, the set of motion trajectories of all target drones is predicted through the encoder-decoder structure of the Autoformer to obtain a set of initial motion prediction trajectories; (c.2) Then, the set of final motion prediction trajectories of all target drones is obtained through a generative adversarial network according to the set of initial motion prediction trajectories and transmitted to the cloud-edge collaboration module.

7. The cloud-edge collaborative unmanned aerial vehicle target detection, tracking and trajectory prediction apparatus according to claim 6, wherein, The sub-step (c.1) specifically includes the following sub-steps: (c.1.1) a set of motion trajectories of all target drones wherein, denotes a motion trajectory of any one of the target drones, denotes the number of target drones, ; The motion trajectory of each target UAV is input into an encoder-decoder structure of the Autoformer, and first encoding is performed The encoder-decoder structure of the Autoformer is shown in FIG. 2, and the encoding process is shown in FIG.

3. The Autoformer has K encoder layers, and the first encoder layer is denoted as E1. The overall equation of the Kth encoder layer is as follows. Wherein, E represents the Kth encoder layer. E1 represents the output of the first encoder layer. E2 represents the output of the second encoder layer. E3 represents the output of the third encoder layer. EK represents the output of the Kth encoder layer. EK-1 represents the output of the (K-1)th encoder layer. ​ through After encoding by each encoder layer, the final encoded output is obtained. ; (c.1.2) Subsequently, each of the final encoded outputs is decoded, provided with a decoder layer, the overall equation for the first layer decoder is wherein represents the decoder layer, represents the first layer decoder, represents the output of the first layer decoder, represents the output of the first layer decoder, ; and ; and ; and go through After decoding by each decoder layer, the final decoded output is obtained. This serves as the initial motion prediction trajectory for the corresponding target UAV, resulting in a set of initial motion prediction trajectories. : . 8.The cloud-edge collaborative UAV target detection, tracking and trajectory prediction apparatus of claim 7, wherein, The first Output of the layer decoder comprises the following sub-steps: (d.1) First, the output of the layer decoder is decoded The Auto-Correlation mechanism is applied, and the Auto-Correlation result is added to the output A time series decomposition operation is performed to decompose into a first seasonal component result and a first trend-cyclical component result ;​ (d.2) the first seasonal component result is then added and the final encoded output applying an Auto-Correlation mechanism, the Auto-Correlation result is then added and the first seasonal component result performing a time series decomposition operation to decompose into a second seasonal component result and a second trend-cyclical component result ; (d.3) then adding the second seasonal partial result applying a feedforward network layer, followed by adding the output of the feedforward network layer and adding the second seasonal partial result performing a time series decomposition operation to decompose into a third seasonal partial result and a third trend-cyclical partial result ; Finally the third seasonal part result As a result Output of the layer decoder . 9.The cloud-edge collaborative UAV target detection, tracking and trajectory prediction apparatus of claim 6, wherein, The sub-step (c.2) is specifically: initial motion prediction trajectory set of each target UAV initial motion prediction trajectory of each target UAV into a generative adversarial network, the generative adversarial network comprising a generator and a discriminator; and combining a diversity loss function, alternately optimizing the generator and the discriminator until the generative adversarial network converges, fixing network parameters to obtain an optimized generative adversarial network; Finally, the initial motion prediction trajectory set is optimized by the optimized generative adversarial network Trajectory prediction is performed to obtain the final motion prediction trajectory set of all target UAVs and transmit to the cloud-edge collaboration module.

Images