Black flight crossing machine track detection and deployment and control security and protection system

Abstract

The invention provides a black flight crossing machine track detection and deployment control security system, which comprises a photoelectric recognition module, an image processing module, a target detection module, an image processing module, an image processing module, an image processing module, an image processing module, a target detection module and a security control module, wherein a dual-spectrum sensor array collects and fuses visible light and infrared images; the image processing module is used for enhancing an infrared target based on an IHBF algorithm and an HB-WTLLCM algorithm; based on an improved YOLOv8 neural network in combination with a channel / space attention mechanism, a posture determination module, a three-dimensional model for a SuperPoint network to extract feature points and match, a multi-sensor fusion module, a visual positioning and tracking module for improving positioning precision based on EKF fusion vision and GPS data, and a module for realizing trajectory tracking and counter instruction based on a binocular recognition algorithm and a YOLOv8 + KCF tracking algorithm. And triggering the defense terminal based on the MQTT protocol. According to the invention, the accuracy and countering response speed of unmanned aerial vehicle detection in a complex environment can be improved.

Description

Technical Field

[0001] This invention relates to the field of drone detection and countermeasures technology, and more specifically, to a security system for detecting and controlling the trajectory of unauthorized drones flying over unauthorized areas. Background Technology

[0002] Against the backdrop of rapid development in the low-altitude economy, the application scenarios and numbers of drones are constantly increasing, greatly promoting the development of related fields. However, the widespread use of drones has also brought a series of challenges, especially the frequent occurrence of unauthorized drone flights, which seriously threaten public safety and privacy. In the field of public security, the detection capabilities for unauthorized drone flights are relatively lagging behind, usually only able to handle the situation after it occurs or rely on visual observation, making it difficult to achieve effective management and real-time monitoring.

[0003] To track and detect unauthorized drone flights and take appropriate countermeasures, drone detection and countermeasure technologies are constantly evolving. Drone defense technology is a multi-process integrated technology, mainly including drone detection and drone countermeasures. Among them, drone detection is the prerequisite for drone countermeasures, and different business departments have different drone defense procedures. After a target is detected, necessary decisions are made, and if the decision is approved, further countermeasures are taken.

[0004] In the field of defense against low-altitude, slow-moving, and small unmanned aerial vehicles (UAVs), various detection technologies based on different principles have emerged. Depending on the form of UAV detection, these technologies can be categorized into radar detection, photoelectric identification and tracking, radio detection, sound detection, and protocol cracking, among others. Despite the numerous technical categories, actual public security applications face complex environments and many uncertainties, requiring rapid response and effective reconnaissance and strike capabilities—the primary first step in achieving integrated investigation and strike capabilities.

[0005] In implementing the embodiments of the present invention, the inventors discovered at least the following problems or defects in the prior art: existing UAV detection technologies lack adaptability in complex environments and are difficult to effectively cope with the diverse characteristics and rapidly changing flight states of unauthorized UAVs. For example, radar detection suffers from low detection accuracy and high false alarm rate when facing small, low-speed UAVs; while photoelectric identification and tracking technologies can provide intuitive image information, their accuracy and stability in target detection and tracking are poor in complex backgrounds and low-light conditions; radio detection and sound detection are easily affected by environmental interference and have limited detection range for UAVs. Furthermore, most existing detection technologies can only provide single-dimensional information, such as position or speed, lacking comprehensive analysis of UAV attitude and trajectory, making it difficult to meet the practical needs of countermeasure systems for continuous tracking and precise strikes against unauthorized UAVs. Summary of the Invention

[0006] This invention provides a security system for detecting and controlling the trajectory of unauthorized flying drones, comprising:

[0007] The photoelectric recognition module is used to acquire visible light and infrared images through a dual-spectrum sensor array and perform real-time registration and fusion.

[0008] The image processing module is used for preprocessing infrared images and target enhancement.

[0009] The target detection module uses an improved YOLOv8 neural network to detect drones.

[0010] The attitude determination module uses feature point matching and 3D model to calculate the UAV's flight attitude.

[0011] A multi-sensor fusion module is used to fuse visual data and GPS data to optimize 3D positioning;

[0012] The visual positioning and tracking module uses a binocular recognition algorithm and a target tracking algorithm to track the trajectory of the drone.

[0013] The countermeasure command module is used to trigger a response from the defense terminal based on the tracking results.

[0014] Furthermore, the image processing module includes:

[0015] Infrared images are preprocessed using an improved high-boost filter (IHBF), and the calculation formula is as follows:

[0016] I IHBF =I o *(l·max{I o -I m ,0})

[0017] Among them, I IHBF To improve the infrared image data after high-voltage filtering,

[0018] I o For raw infrared image data, I m The image data is after mean filtering, l is the enhancement coefficient constant, and * represents the Hadamard product;

[0019] Local contrast enhancement is performed based on the weighted three-layer window object detection algorithm HB-WTLLCM, and the calculation formula is as follows:

[0020]

[0021] Among them, TLLCM is the enhanced result of the three-layer window local contrast detection algorithm, I C For the core layer Gaussian filtering result, I S It is the average of the maximum pixel values ​​in the surrounding layers.

[0022] Furthermore, the calculation formula for the core layer Gaussian filter is as follows:

[0023]

[0024] Where (i, j) are the current pixel coordinates, G(p, q) are the weight coefficients of the two-dimensional Gaussian filter template, and I is the original infrared image data.

[0025] Furthermore, the target detection module employs a YOLOv8 neural network, including:

[0026] Channel attention mechanism enhances target feature channels by assigning different weights;

[0027] Spatial attention mechanism, which focuses on the spatial distribution of the target by assigning different weights.

[0028] Furthermore, the attitude determination module extracts image feature points through a SuperPoint network and matches them with a pre-built UAV 3D model, with a feature point matching rate of no less than 92.3%.

[0029] The SuperPoint network includes a shared encoder, a keypoint detection decoder, a descriptor generation decoder, and a self-supervised training module.

[0030] Furthermore, the multi-sensor fusion module uses an extended Kalman filter (EKF) to fuse 5Hz GPS data with visual attitude output, and the three-dimensional positioning error does not exceed 0.45mRMSE.

[0031] Single-frame attitude calculation is optimized using the PnP algorithm and bundle adjustment, with a yaw angle accuracy of 0.8° and a pitch angle accuracy of 1.2°.

[0032] Furthermore, the visual positioning and tracking module employs a binocular recognition algorithm, solving for the target's spatial position by solving a system of simultaneous equations, including:

[0033] AO 2 =m 2 +BO 2 -2m·BO·cosβ2

[0034]

[0035] Where AO is the distance from the first capture device to the target, BO is the distance from the second capture device to the target, m is the distance between the two cameras, α2 is the measurement angle of the first capture device, and β2 is the measurement angle of the second capture device.

[0036] Furthermore, the target tracking algorithm is a fusion method of YOLOv8 and kernelized correlation filter (KCF), and its kernel correlation response calculation formula is as follows:

[0037]

[0038] Where f(z) is the kernel-related response function, F -1 This is the inverse Fourier transform.

[0039] K{xz} represents the kernel function of the candidate region features and the target features in the frequency domain, α is the filter coefficient, and ⊙ is the Hadamard product.

[0040] Furthermore, the system hardware includes:

[0041] Dual-degree-of-freedom pan-tilt camera, model number 2-AxisPan-TiltCameraModule;

[0042] Infrared thermal imaging equipment with a resolution of 384×288, supporting temperature difference detection of 0.04℃;

[0043] The main control module adopts a collaborative architecture of ESP32-WROOM-32 and Raspberry 5.

[0044] Furthermore, the countermeasure instruction module sends JSON instructions to the defense terminal via the MQTT protocol. The instructions include the target ID, three-dimensional coordinates, and threat level.

[0045] The system adopts a layered architecture in the cloud. Data is transmitted to AWS EC2 or Alibaba Cloud ECS servers via EMQX or Mosquitto, and time-series data and structured logs are stored via InfluxDB and MongoDB.

[0046] The embodiments of the present invention have at least the following beneficial effects:

[0047] 1. By using dual-spectral collaborative detection and an improved high-voltage filtering algorithm, the ability to extract target features in complex environments is effectively enhanced, the sensitivity to identify low-altitude, slow-moving, small targets is improved, and the problem of insufficient detection capability of traditional optoelectronic systems under complex background interference is solved.

[0048] 2. A multi-level recognition architecture combining deep neural networks and 3D model matching is adopted to achieve accurate analysis of the UAV's flight attitude, thus solving the technical bottleneck of existing technologies that make it difficult to accurately determine the target's motion state.

[0049] 3. By fusing information from multiple sensor sources and using adaptive tracking algorithms, a stable and reliable target motion trajectory prediction model is established to solve the technical problem of target loss during continuous tracking of a single sensor system. Attached Figure Description

[0050] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the invention are illustrated in the drawings by way of example and not limitation, wherein:

[0051] Figure 1 This is a functional flowchart of a security system for detecting and controlling the trajectory of unauthorized flying drones, provided in an embodiment of the present invention.

[0052] Figure 2 This is a YOLOv8 network framework diagram of a security system for detecting and controlling the trajectory of unauthorized flying drones, provided in an embodiment of the present invention.

[0053] Figure 3 This is a target detection effect diagram of a black-flying Flying Drone trajectory detection and control security system provided in an embodiment of the present invention;

[0054] Figure 4 This is a diagram of a triangulation algorithm module for a black-flying Flying Drone trajectory detection and control security system provided in an embodiment of the present invention;

[0055] Figure 5 This is a flowchart of an infrared drone detection and tracking system for a black-flying Flying Drone trajectory detection and control security system, provided in an embodiment of the present invention.

[0056] Figure 6 This is a diagram of an infrared detection module for a black-flying Flying Jet trajectory detection and control security system provided in an embodiment of the present invention. Detailed Implementation

[0057] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make the invention more thorough and complete, and to fully convey the scope of the invention to those skilled in the art.

[0058] Those skilled in the art will understand that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present invention can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0059] It should be noted that the number of any elements in the accompanying drawings is for illustrative purposes only and not as a limitation, and any naming is for distinction only and has no limiting meaning.

[0060] The following is for reference. Figure 1 , Figure 1 This is a functional flowchart of a security system for detecting and controlling the trajectory of unauthorized flying vehicles, provided in an embodiment of the present invention. Figure 1 As shown, the functional flow of a black-flying racing drone trajectory detection and control security system includes:

[0061] The photoelectric recognition module is used to acquire visible light and infrared images through a dual-spectrum sensor array and perform real-time registration and fusion.

[0062] The image processing module is used for preprocessing infrared images and target enhancement.

[0063] The target detection module uses an improved YOLOv8 neural network to detect drones.

[0064] The attitude determination module uses feature point matching and 3D model to calculate the UAV's flight attitude.

[0065] A multi-sensor fusion module is used to fuse visual data and GPS data to optimize 3D positioning;

[0066] The visual positioning and tracking module uses a binocular recognition algorithm and a target tracking algorithm to track the trajectory of the drone.

[0067] The countermeasure command module is used to trigger a response from the defense terminal based on the tracking results.

[0068] It should be noted that the proposed security system for detecting and controlling unauthorized drone flight trajectories in this invention aims to achieve efficient detection and tracking of unauthorized drones through the collaborative work of multiple modules. The photoelectric recognition module is used to acquire visible light and infrared images through a dual-spectral sensor array and perform real-time registration and fusion. Here, the dual-spectral sensor array refers to a combination of sensors capable of simultaneously capturing visible light and infrared spectra. This approach fully utilizes the high resolution of visible light and the sensitivity of infrared light to heat sources, thereby more comprehensively capturing the thermal radiation characteristics and optical images of the drone's propulsion system. Simultaneously, the image processing module preprocesses and enhances the acquired infrared images to improve the detectability of targets in the images, providing a clearer image foundation for subsequent target detection and tracking. The target detection module uses an improved YOLOv8 neural network to detect drones. YOLOv8 is an advanced target detection algorithm that can quickly and accurately identify drone targets in images through a deep learning model. Furthermore, the attitude determination module uses feature point matching and a 3D model to calculate the drone's flight attitude. By matching feature points in the image with a pre-constructed 3D model of the drone, the attitude information of the drone can be accurately determined. The multi-sensor fusion module combines visual and GPS data to optimize 3D positioning. By combining data from multiple sensors, the position and trajectory of the drone can be determined more accurately. Finally, the countermeasure command module triggers a response from the defense terminal based on the tracking results, achieving effective countermeasures against unauthorized drone flights.

[0069] Specifically, the dual-spectral sensor array in the photoelectric recognition module refers to a system composed of visible light sensors and infrared sensors. The visible light sensor is used to capture high-resolution optical images, while the infrared sensor is used to capture the thermal radiation characteristics of the UAV's propulsion system. In the image processing module, the purpose of preprocessing and target enhancement of the infrared image is to improve image quality for better target identification. Preprocessing may include noise reduction and contrast adjustment, while target enhancement uses specific algorithms to highlight target areas in the image, making them easier to detect. In the target detection module, an improved YOLOv8 neural network detects UAVs in images using deep learning. This network learns UAV features through extensive training data, enabling accurate identification of UAVs in complex backgrounds. Feature point matching in the attitude determination module involves extracting key feature points from the image and matching them with a pre-built 3D UAV model to determine the UAV's attitude. This 3D model refers to a 3D geometric model built based on the actual structure of the UAV, used to assist in determining the UAV's attitude information. In the multi-sensor fusion module, visual data and GPS data are fused to improve positioning accuracy and reliability. Visual data provides the relative position information of the drone, while GPS data provides absolute position information. Combining the two allows for a more accurate determination of the drone's three-dimensional position. Finally, the countermeasure command module triggers a defense terminal response based on the tracking results. This defense terminal can be a jamming device, interceptor, etc., used to effectively counter unauthorized drone flights.

[0070] Preferably, the preprocessing and target enhancement processes in the image processing module employ an improved high-boost filter (IHBF) to preprocess the infrared image. This filter enhances the high-frequency components of the image to highlight the target region while retaining low-frequency components to reduce background interference. Specifically, the IHBF algorithm first calculates the difference between the original image and the mean-filtered image, suppressing the background region to zero and retaining only the positive difference value of the target region. Then, this result is multiplied with the original image using a Hadamard product to enhance the target region. In the target detection module, an improved YOLOv8 neural network enhances detection performance by introducing channel attention and spatial attention mechanisms. The channel attention mechanism enhances the ability to express target features by assigning weights to different feature channels; the spatial attention mechanism focuses on the spatial distribution of the target object by assigning weights to different spatial locations in the feature map. In the attitude determination module, the SuperPoint network is used to extract image feature points and match them with a pre-constructed UAV 3D model. This network can stably extract 120 effective feature points per frame, with a feature point matching rate of no less than 92.3%. The SuperPoint network comprises a shared encoder, a keypoint detection decoder, a descriptor generation decoder, and a self-supervised training module. Through the collaborative work of these modules, it accurately extracts keypoints from images and generates corresponding descriptors, thereby achieving precise calculation of the UAV's attitude. In the multi-sensor fusion module, an extended Kalman filter (EKF) is used to fuse 5Hz GPS data with visual attitude output. The PnP algorithm and bundle adjustment are used to optimize single-frame attitude calculation, ensuring that the 3D positioning error does not exceed 0.45 meters (Root Mean Square Error, RMSE). In the visual positioning and tracking module, a binocular recognition algorithm is used to solve for the target's spatial position by solving a system of simultaneous equations. This algorithm utilizes the distance between the two capture devices and the measurement angles to calculate the target's height and position through geometric relationships, thus achieving precise tracking of the UAV's trajectory.

[0071] like Figure 6 As shown, in some embodiments, the image processing module includes:

[0072] Infrared images are preprocessed using an improved high-boost filter (IHBF), and the calculation formula is as follows:

[0073] I IHBF =I o *(l·max{I o -I m ,0})

[0074] Among them, I IHBF To improve the infrared image data after high-voltage filtering,

[0075] I o For raw infrared image data, I mThe image data is after mean filtering, l is the enhancement coefficient constant, and * represents the Hadamard product;

[0076] Local contrast enhancement is performed based on the weighted three-layer window object detection algorithm HB-WTLLCM, and the calculation formula is as follows:

[0077]

[0078] Among them, TLLCM is the enhanced result of the three-layer window local contrast detection algorithm.

[0079] I C For the core layer Gaussian filtering result, I S It is the average of the maximum pixel values ​​in the surrounding layers.

[0080] It should be noted that the image processing module in this invention is used for preprocessing and target enhancement of infrared images, aiming to improve the detectability and clarity of UAV targets in the images. The improved high-boost filter (IHBF) is an enhancement algorithm specifically designed for infrared images. It enhances the high-frequency components of the image to highlight the target region while suppressing background noise. The core of the IHBF algorithm lies in processing the original infrared image. By calculating the difference between the original image and the mean-filtered image, and adjusting it in conjunction with the enhancement coefficient constant, the target region is enhanced. The weighted three-layer window target detection algorithm (HB-WTLLCM) is a local contrast enhancement algorithm. It constructs a multi-scale three-layer window structure to locally enhance the contrast of the target region in the image, further improving the target's detectability. The combined use of these two algorithms enables the system to more accurately identify UAV targets in complex backgrounds.

[0081] Specifically, the calculation process of the improved high-boost filter IHBF involves processing the original infrared image data. First, the difference between the original image and the image after mean filtering is calculated. Mean filtering is a common image smoothing technique used to remove noise from images. By calculating the difference, high-frequency components in the image, i.e., the target region, can be highlighted. Then, negative or small parts of the difference are zeroed out, retaining the positive difference, which is then multiplied element-wise with the original image to enhance the target region. The enhancement coefficient constant is a parameter used to adjust the degree of enhancement and can be optimized according to the actual application scenario. The calculation process of the weighted three-layer window target detection algorithm HB-WTLLCM involves multi-scale analysis of the image. By constructing three nested windows, the image data within each window is processed to obtain the result of local contrast enhancement. The core layer Gaussian filtering result is obtained by applying Gaussian filtering to the image. Gaussian filtering is a commonly used smoothing filtering method used to highlight targets in the central region of the image. The mean value of the maximum pixel value in the surrounding layers is obtained by statistically processing the pixel values ​​in the surrounding layers to reflect background information. Through this multi-scale analysis and contrast enhancement, the algorithm can effectively improve the detectability of targets in complex backgrounds.

[0082] Preferably, when implementing the improved high-boost filter (IHBF), the enhancement coefficient constant can be dynamically adjusted. An appropriate enhancement coefficient constant value is selected based on the target size and background complexity to achieve the best enhancement effect. For example, when the target is small and the background is complex, the enhancement coefficient constant value can be appropriately increased to enhance the contrast of the target area. In the weighted three-layer window target detection algorithm HB-WTLLCM, the calculation process of the core layer Gaussian filter can be achieved by selecting appropriate Gaussian filter template weight coefficients. These weight coefficients determine the degree of weighting of pixels at different locations in the image, thus affecting the target enhancement effect. Specifically, the weight coefficients of the Gaussian filter template can be calculated using a two-dimensional Gaussian distribution formula, where the standard deviation parameter can be adjusted according to the target size and image resolution. Through these optimization measures, the image processing module's ability and adaptability to detect UAV targets can be further improved.

[0083] like Figure 6 As shown, in some embodiments, the calculation formula for the core layer Gaussian filter is:

[0084]

[0085] Where (i, j) are the current pixel coordinates, G(p, q) are the weight coefficients of the two-dimensional Gaussian filter template, and I is the original infrared image data.

[0086] It should be noted that core-layer Gaussian filtering is a crucial step in the image processing module. Its purpose is to highlight the features of the target region by smoothing the image. Gaussian filtering is a smoothing filtering method based on the Gaussian function. It uses the weights of the Gaussian function to perform a weighted average of the pixels in the image, thereby reducing noise and details in the image and making the target region more prominent. In this invention, core-layer Gaussian filtering is achieved by weighted summation of each pixel in the image. This method can effectively smooth the image while preserving important target features, providing a clearer image foundation for subsequent target detection and tracking.

[0087] Specifically, the calculation process of the core layer Gaussian filtering involves processing each pixel in the image. First, the coordinates (i, j) of the current pixel are determined. Then, the weight coefficients G of the two-dimensional Gaussian filter template are used to weight the surrounding pixels. These weight coefficients G are calculated based on the Gaussian function and determine the degree of influence of the surrounding pixels on the current pixel. By multiplying the weight coefficients of the Gaussian filter template by the values ​​of the surrounding pixels and summing the results, the filtering result for the current pixel can be obtained. This process essentially utilizes the smoothing properties of the Gaussian function to reduce high-frequency noise in the image while preserving the low-frequency features of the target region. In practical applications, the size and standard deviation of the Gaussian filter template are two important parameters. The size of the Gaussian filter template determines the filtering range, while the standard deviation determines the filtering strength. By appropriately setting these two parameters, effective image smoothing can be achieved.

[0088] Preferably, when implementing Gaussian filtering in the core layer, the size and standard deviation of the Gaussian filter template can be optimized. For example, a 3×3 or 5×5 Gaussian filter template can be selected, with the specific size depending on the target size and image resolution. The choice of standard deviation needs to be determined based on the characteristics of the target and the complexity of the background. Generally, a larger standard deviation leads to a stronger smoothing effect but may blur the details of the target; while a smaller standard deviation can retain more details but may not be effective in removing noise. In practical applications, the optimal standard deviation value can be determined experimentally.

[0089] Furthermore, efficient algorithms can be employed to improve computational speed when calculating the Gaussian filtering results of the core layer. For example, the Fast Fourier Transform (FFT) can be used to quickly calculate the Gaussian filter. These optimizations can further enhance the performance of the image processing module, providing higher-quality image data for subsequent target detection and tracking.

[0090] like Figure 2 , Figure 3 As shown, in some embodiments, the target detection module employs a YOLOv8 neural network, including:

[0091] Channel attention mechanism enhances target feature channels by assigning different weights;

[0092] Spatial attention mechanism, which focuses on the spatial distribution of the target by assigning different weights.

[0093] It should be noted that the target detection module in this invention employs an improved YOLOv8 neural network for drone detection. YOLOv8 is an advanced target detection algorithm that uses a deep learning model to quickly and accurately identify drone targets in images. The improved YOLOv8 neural network further enhances the model's drone target detection performance by introducing channel attention and spatial attention mechanisms. The channel attention mechanism enhances the expressive power of target features by assigning weights to different feature channels; the spatial attention mechanism focuses on the spatial distribution of target objects by assigning weights to different spatial locations in the feature map. The combination of these two attention mechanisms enables the model to more effectively handle drone detection tasks in complex backgrounds.

[0094] Specifically, the core of the channel attention mechanism lies in assigning weights to different feature channels. In deep learning models, feature channels refer to channels in a neural network used to extract specific features, and each channel can extract different feature information. By assigning different weights to each channel, the model can more effectively highlight those feature channels that are more helpful for object detection. For example, if a certain channel performs well in extracting the rotor features of a drone, then a higher weight can be assigned to that channel. The spatial attention mechanism, on the other hand, is implemented by assigning weights to different spatial locations in the feature map. The feature map is a two-dimensional data structure in a neural network used to represent image features. By assigning weights to different locations in the feature map, the spatial attention mechanism allows the model to focus more on key regions of the target object. For example, if a drone appears at a specific location in an image, the model can assign a higher weight to that location through the spatial attention mechanism, thereby detecting the drone more accurately. The combination of these two attention mechanisms enables the improved YOLOv8 neural network to handle drone detection tasks in complex backgrounds more effectively, improving detection accuracy and robustness.

[0095] Preferably, when constructing the improved YOLOv8 neural network, the channel attention mechanism and spatial attention mechanism can be further optimized. For example, in the channel attention mechanism, the weights of each channel can be dynamically adjusted through learning, rather than fixed weights. This method can automatically adjust the weight allocation based on data during training, thus better adapting to different detection tasks. In the spatial attention mechanism, positional encoding can be introduced to enhance the model's ability to perceive the target's location. Positional encoding is a technique that embeds positional information into feature maps, allowing the model to better understand the positional relationships of targets within an image.

[0096] Furthermore, when training the improved YOLOv8 neural network, a large amount of labeled data can be used to optimize the model's parameters. This labeled data includes images of the drone under different backgrounds and lighting conditions. By training with this data, the model can learn various features of the drone, thereby improving the accuracy and robustness of detection.

[0097] like Figure 6 As shown, in some embodiments, the attitude determination module extracts image feature points through a SuperPoint network and matches them with a pre-built UAV 3D model, with a feature point matching rate of not less than 92.3%.

[0098] The SuperPoint network includes a shared encoder, a keypoint detection decoder, a descriptor generation decoder, and a self-supervised training module.

[0099] It should be noted that the attitude determination module in this invention extracts image feature points using a SuperPoint network and matches them with a pre-built 3D model of the UAV to determine its flight attitude. The SuperPoint network is an advanced computer vision algorithm capable of simultaneously detecting keypoints and generating corresponding descriptors. Its core lies in achieving efficient extraction and matching of feature points in images through a shared encoder, keypoint detection decoder, descriptor generation decoder, and a self-supervised training module. This module achieves a feature point matching rate of no less than 92.3%, indicating high accuracy and reliability in UAV attitude determination. By matching the extracted feature points with the pre-built 3D model, the system can accurately calculate the UAV's attitude information in 3D space, thus providing crucial information for subsequent tracking and countermeasures.

[0100] Specifically, the SuperPoint network structure includes the following key components: a shared encoder, a keypoint detection decoder, a descriptor generation decoder, and a self-supervised training module. The shared encoder, upon receiving the input image, progressively downsamples through a convolutional neural network to extract multi-scale features, providing a unified basic feature representation for subsequent tasks. The keypoint detection decoder upsamples the feature map output by the encoder, outputs a probability score map for each location as a keypoint through convolution operations, and uses non-maximum suppression to select the points with the highest scores, ultimately outputting the keypoint coordinates. The descriptor generation decoder, based on the encoder's feature map, generates high-dimensional descriptors through cross-channel convolution. Each keypoint obtains a corresponding descriptor vector through bilinear interpolation and undergoes L2 normalization to enhance matching robustness. The self-supervised training module trains the initial model on a virtual shape dataset and uses the model's own prediction results as pseudo-labels for iterative optimization, improving the model's generalization ability. In practical applications, the feature point matching rate is an important indicator of SuperPoint network performance; it represents the proportion of extracted feature points that successfully match the pre-built 3D model. By optimizing the network structure and training strategy, the feature point matching rate can be further improved, thereby enhancing the accuracy of UAV attitude determination.

[0101] Preferably, the performance of the SuperPoint network can be further optimized through the following steps when constructing it. First, for the shared encoder, a more efficient convolutional neural network architecture, such as a VGG-style network structure, can be selected to extract richer multi-scale features. Second, in the keypoint detection decoder, the detection accuracy of keypoints can be optimized by adjusting the size and number of convolutional kernels. For example, using 1×1 convolutions can output keypoint score maps more efficiently. In the descriptor generation decoder, the discriminative power of descriptors can be improved by increasing their dimensions, for example, by setting the descriptor dimensions to 256 dimensions.

[0102] Furthermore, during the self-supervised training phase, the model's generalization ability can be enhanced by introducing more virtual shape data and more complex homography transformations. In practical applications, the feature point extraction effect can be further optimized by adjusting the network's input parameters, such as image resolution and feature extraction scale. For example, for high-resolution image input, the feature extraction scale can be appropriately increased to better adapt to variations in the size of the UAV target. Through these optimization measures, the performance of the SuperPoint network in UAV attitude determination can be further improved, enabling it to more accurately match feature points and calculate the UAV's flight attitude.

[0103] like Figure 5As shown, in some embodiments, the multi-sensor fusion module uses an extended Kalman filter (EKF) to fuse 5Hz GPS data with visual attitude output, and the three-dimensional positioning error does not exceed 0.45mRMSE.

[0104] Single-frame attitude calculation is optimized using the PnP algorithm and bundle adjustment, with a yaw angle accuracy of 0.8° and a pitch angle accuracy of 1.2°.

[0105] It should be noted that the multi-sensor fusion module in this invention is used to fuse visual data and GPS data to optimize the 3D positioning accuracy of the UAV. Visual data refers to the UAV's position and attitude information acquired through the image processing module, while GPS data provides the UAV's absolute position information. By fusing these two types of data, the shortcomings of a single sensor can be effectively compensated for, improving the accuracy and reliability of positioning. This module uses the Extended Kalman Filter (EKF) algorithm for data fusion, which can process sensor data in real time and control the 3D positioning error within 0.45 meters (Root Mean Square Error, RMSE). Furthermore, single-frame attitude calculation is optimized using the PnP algorithm and bundle adjustment, ensuring that the yaw and pitch angle accuracies reach 0.8° and 1.2° respectively, thus providing high-precision positioning support for UAV trajectory tracking.

[0106] Specifically, visual data and GPS data are two important inputs in the multi-sensor fusion module. Visual data typically includes information such as the UAV's position and attitude in the image, obtained through image processing and feature extraction. GPS data provides the UAV's absolute position information, including longitude, latitude, and altitude. The Extended Kalman Filter (EKF) is a commonly used nonlinear filtering algorithm that fuses visual and GPS data through prediction and update steps. During the fusion process, the EKF estimates the UAV's state and corrects it based on new measurement data. The Point-in-Pack (PnP) algorithm is used to estimate camera attitude from known 3D points and corresponding 2D image points, while bundle adjustment optimization is used to optimize camera attitude and point cloud positions in 3D reconstruction. Combining these algorithms enables high-precision estimation of UAV attitude and position. Yaw and pitch angles are important parameters of UAV attitude; yaw represents the UAV's rotation angle around the vertical axis, while pitch represents the UAV's rotation angle around the horizontal axis. Optimizing the estimation of these parameters can improve the accuracy of UAV attitude.

[0107] Preferably, when implementing the multi-sensor fusion module, the parameters of the Extended Kalman Filter (EKF) can be optimized. For example, the state transition matrix and observation matrix of the filter can be adjusted according to the dynamic characteristics of the UAV. The state transition matrix describes the change of the UAV's state over time, while the observation matrix describes the relationship between sensor measurements and the UAV's state. Furthermore, the noise covariance matrix of the filter can be determined experimentally to improve filtering accuracy. In the PnP algorithm, different solution methods, such as Direct Linear Transform (DLT) or iterative optimization methods, can be used to improve the accuracy of attitude estimation. For bundle adjustment optimization, the optimization effect can be improved by adding constraints or using more efficient optimization algorithms. For example, smoothing constraints can be introduced to reduce trajectory jitter, or gradient descent-based optimization algorithms can be used to accelerate convergence. Through these optimization measures, the performance of the multi-sensor fusion module can be further improved, achieving more accurate 3D localization and attitude estimation.

[0108] like Figure 4 , Figure 5 As shown, in some embodiments, the visual positioning and tracking module employs a binocular recognition algorithm to solve for the target's spatial position by solving a system of simultaneous equations, including:

[0109] AO 2 =m 2 +BO 2 -2m·BO·cosβ2

[0110]

[0111] Where AO is the distance from the first capture device to the target, BO is the distance from the second capture device to the target, m is the distance between the two cameras, α2 is the measurement angle of the first capture device, and β2 is the measurement angle of the second capture device.

[0112] It should be noted that the visual positioning and tracking module in this invention achieves drone trajectory tracking through a binocular recognition algorithm and a target tracking algorithm. The binocular recognition algorithm utilizes the distance and measurement angle between the two capture devices to solve for the target's spatial position by solving a system of simultaneous equations. This method is similar to the principle of human binocular vision, acquiring images from two cameras at different locations and calculating the target's position in space. The target tracking algorithm combines YOLOv8 and kernelized correlation filter (KCF) methods, determining the target position through the kernel correlation response function and dynamically updating the tracking model. This fusion method can track drones quickly and accurately in complex backgrounds while maintaining high real-time performance.

[0113] Specifically, the core of the binocular recognition algorithm lies in using the geometric relationship between two capture devices to determine the target's position. The distance *m* between the capture devices is a fixed physical parameter representing the distance between the two cameras. The measured angles α² and β² are the target direction angles obtained through the cameras, which can be calculated from the positional relationship between the camera sensors and the target. By solving a system of equations, the distances from the target to the two cameras, AO and BO, can be obtained, thus determining the target's position in three-dimensional space. The kernel correlation response function (KCF) in the target tracking algorithm is a frequency domain-based tracking algorithm that determines the target's position by calculating the kernel function representation of candidate region features and target features. The kernel correlation response function is crucial to the KCF algorithm; it uses an inverse Fourier transform to convert the kernel function representation in the frequency domain into a response in the spatial domain, thereby determining the target's position. The filter coefficients α are dynamically updated through an optimization algorithm to adapt to changes in the target's motion.

[0114] Preferably, when implementing the visual positioning and tracking module, the binocular recognition algorithm and the target tracking algorithm can be further optimized. For the binocular recognition algorithm, measurement accuracy can be improved by precisely calibrating the relative position and angle of the two capture devices. For example, the intrinsic and extrinsic parameters of the camera can be calibrated using a calibration board to ensure the accuracy of the distance between the capture devices and the measurement angle. In the target tracking algorithm, the robustness and accuracy of tracking can be improved by adjusting the parameters of the kernel correlation filter, such as the learning rate η of the filter coefficients and the parameters of the scale pyramid. For example, the learning rate can be set to a smaller value to ensure more stable updates of the filter coefficients. At the same time, multi-scale candidate regions can be constructed to cope with changes in the target's scale, improving the adaptability of tracking. In addition, a detection-tracking cooperation mechanism can be introduced to re-trigger YOLOv8 detection when the tracking confidence falls below a certain threshold, ensuring the continuity and accuracy of target tracking.

[0115] like Figure 5 As shown, in some embodiments, the target tracking algorithm is a fusion method of YOLOv8 and kernelized correlation filter (KCF), and its kernel correlation response calculation formula is as follows:

[0116]

[0117] Where f(z) is the kernel-related response function, F -1 This is the inverse Fourier transform.

[0118] K{xz} represents the kernel function of the candidate region features and the target features in the frequency domain, α is the filter coefficient, and ⊙ is the Hadamard product.

[0119] It should be noted that the target tracking algorithm in this invention employs a fusion of YOLOv8 and Kernel Correlation Filter (KCF) to achieve efficient tracking of UAVs. YOLOv8 is a deep learning-based target detection algorithm capable of quickly and accurately detecting the position and features of UAVs, while KCF is a highly efficient tracking algorithm that, through kernel ridge regression optimization in the frequency domain, can quickly respond to changes in the target's position. The combination of the two leverages both the high detection accuracy of YOLOv8 and the high-speed tracking capability of KCF, making it particularly suitable for fast-moving and small targets like UAVs. The kernel correlation response function is the core of this algorithm. It determines the target's position in the image by calculating the kernel function representation of candidate region features and target features, and dynamically updates the filter coefficients to adapt to changes in target motion, thereby achieving continuous tracking of the UAV.

[0120] Specifically, the YOLOv8 part of the target tracking algorithm is responsible for initial target detection. It uses input image data and improved neural network structures, such as the CSPDarknet53 backbone, PANet feature pyramid, and decoupled detection head, to perform multi-scale target detection, outputting a predicted bounding box containing the target's location, size, class probability, and confidence score. For the kernel correlation filter (KCF) part, it transforms the tracking problem into a kernel ridge regression optimization problem in the frequency domain. During the tracking phase, the system determines the target location by calculating the kernel correlation response between candidate region features and the model, and dynamically updates the model parameters using a learning rate. Here, the kernel correlation response function converts the kernel function representation in the frequency domain into a response in the spatial domain using an inverse Fourier transform, thereby determining the target's location. The filter coefficients are dynamically updated through an optimization algorithm to adapt to changes in target motion. The scale pyramid is constructed to handle scale variations of UAV targets, improving tracking robustness by tracking at different scales. The confidence score is calculated as the ratio of the maximum to the average value of the kernel correlation response. When the confidence score falls below a set threshold, YOLOv8 detection is retried to ensure tracking accuracy.

[0121] Preferably, when implementing the target tracking algorithm, the parameters of YOLOv8 and KCF can be optimized. For YOLOv8, the detection capability for small targets such as drones can be enhanced by adjusting the resolution of the input image and the adaptive anchor strategy. For example, using a high-resolution input image can improve detection accuracy, while the adaptive anchor strategy can adjust the shape and size of the detection box according to the size distribution of the drone. For KCF, the update speed of the filter coefficients can be controlled by adjusting the learning rate η. Setting the learning rate to a smaller value can ensure more stable updates of the filter coefficients, thereby improving the robustness of tracking. At the same time, the scale pyramid parameters, such as the scale factor and scale level index, can be optimized to better handle the scale changes of the drone. For example, choosing an appropriate scale factor can ensure accurate tracking of the drone at different distances. In addition, the accuracy and real-time performance of tracking can be further improved by introducing a motion prediction mechanism that combines the target's historical motion information to predict its future position.

[0122] like Figure 6 As shown, in some embodiments, the system hardware includes:

[0123] Dual-degree-of-freedom pan-tilt camera, model number 2-AxisPan-TiltCameraModule;

[0124] Infrared thermal imaging equipment with a resolution of 384×288, supporting temperature difference detection of 0.04℃;

[0125] The main control module adopts a collaborative architecture of ESP32-WROOM-32 and Raspberry 5.

[0126] It should be noted that the hardware system of this invention includes a dual-DOF gimbal camera, an infrared thermal imaging device, and a main control module. The dual-DOF gimbal camera enables omnidirectional monitoring and tracking, the infrared thermal imaging device is used to capture the thermal radiation characteristics of the drone, and the main control module is responsible for processing and coordinating data interaction between the various hardware devices. The collaborative work of these hardware components provides a fundamental guarantee for the efficient operation of the system, ensuring accurate detection and tracking of unauthorized drones.

[0127] Specifically, the dual-degree-of-freedom pan-tilt camera, model 2-Axis Pan-Tilt Camera Module, allows for free rotation in both horizontal and vertical directions, thus covering a wider monitoring area. The infrared thermal imaging device has a resolution of 384×288 and can detect temperature differences as small as 0.04℃, enabling it to effectively capture the thermal radiation characteristics of the drone's propulsion system even at night or in complex weather conditions. The main control module adopts a collaborative architecture of ESP32-WROOM-32 and Raspberry 5, with the ESP32-WROOM-32 serving as the core controller responsible for data acquisition and initial processing, while the Raspberry 5 acts as the core computing unit, handling complex image processing and algorithm calculations. This collaborative architecture not only improves the system's computational efficiency but also enhances its stability and reliability.

[0128] Preferably, during the hardware system construction process, the rotation range and speed of the dual-DOF gimbal camera can be optimized to ensure rapid response and target tracking. For example, the motor parameters of the gimbal can be adjusted to enable it to complete a large range of rotations in a short time. For infrared thermal imaging devices, the detection capability for distant targets can be improved by optimizing their focal length and field of view. Simultaneously, to improve the overall system performance, more sensor data fusion algorithms can be introduced into the main control module, such as fusing infrared thermal imaging data with visible light image data, to further enhance the accuracy and reliability of target detection. Furthermore, optimizing the communication protocol between the ESP32 and the Raspberry 5 can reduce data transmission latency and improve the system's real-time performance.

[0129] like Figure 6 As shown, in some embodiments, the countermeasure instruction module sends JSON instructions to the defense terminal via the MQTT protocol. The instructions include the target ID, three-dimensional coordinates, and threat level.

[0130] The system adopts a layered architecture in the cloud. Data is transmitted to AWS EC2 or Alibaba Cloud ECS servers via EMQX or Mosquitto, and time-series data and structured logs are stored via InfluxDB and MongoDB.

[0131] It should be noted that the countermeasure instruction module of this invention sends JSON instructions to the defense terminal via the MQTT protocol. These instructions contain key information such as target ID, 3D coordinates, and threat level. MQTT is a lightweight messaging protocol widely used in the Internet of Things (IoT) field, enabling efficient communication between devices. After receiving the instructions, the defense terminal can take corresponding countermeasures based on the instruction content. Furthermore, the system adopts a layered architecture in the cloud, with data transmitted to AWS EC2 or Alibaba Cloud ECS servers via EMQX or Mosquitto, and time-series data and structured logs stored via InfluxDB and MongoDB. This architecture not only improves the system's scalability and flexibility but also ensures data security and reliability.

[0132] Specifically, the MQTT protocol is a publish / subscribe messaging protocol that allows devices to communicate via topics. In this invention, the countermeasure command module sends JSON commands containing target information to the defense terminal via the MQTT protocol. JSON is a lightweight data exchange format that is easy to read and parse, and can efficiently transmit information such as target ID, 3D coordinates, and threat level. After receiving these commands, the defense terminal can take corresponding measures based on the threat level, such as issuing an alarm, activating jamming devices, or intercepting data. The layered architecture of the system in the cloud includes a data transmission layer, a processing layer, and a storage layer. The data transmission layer uses EMQX or Mosquitto as the MQTT Broker, responsible for receiving data from various devices; the processing layer runs on AWS EC2 or Alibaba Cloud ECS servers, responsible for processing and analyzing the data; and the storage layer uses InfluxDB to store time-series data and MongoDB to store structured logs. This architecture can efficiently process and store large amounts of data, while supporting system expansion and maintenance.

[0133] Preferably, the communication process of the MQTT protocol can be optimized when implementing the countermeasure command module. For example, reliable command transmission can be ensured by setting a reasonable QoS (Quality of Service) level. The QoS level can be adjusted according to the importance and real-time requirements of the command; for example, a higher QoS level can be set for commands targeting high-threat targets. Simultaneously, the structure of JSON commands can be optimized to reduce data redundancy and improve transmission efficiency. In the cloud system, the throughput and stability of data transmission can be improved by optimizing the configuration of EMQX or Mosquitto. For example, performance can be optimized by adjusting the size and timeout of message queues. For AWS EC2 or Alibaba Cloud ECS servers, resource configuration can be dynamically adjusted according to the system load to ensure efficient operation of the processing layer. Furthermore, performance optimization of InfluxDB and MongoDB can improve data read / write speed and query efficiency. For example, storage layer performance can be improved through index optimization and data sharding. These optimization measures can further improve the system's response speed and reliability, ensuring effective countermeasures against unauthorized drone flights.

[0134] The above embodiments of the present invention have the following beneficial effects:

[0135] 1. By using dual-spectral collaborative detection and an improved high-boost filtering algorithm, the ability to extract target features in complex environments can be effectively enhanced, significantly improving the sensitivity to identify low-altitude, slow-moving, small targets, thus solving the problem of insufficient detection capability of traditional optoelectronic systems under complex background interference.

[0136] 2. A multi-level recognition architecture combining deep neural networks and 3D model matching can be adopted to achieve accurate analysis of the UAV's flight attitude, thus solving the technical bottleneck of existing technologies that make it difficult to accurately determine the target's motion state.

[0137] 3. By using multi-source sensor information fusion and adaptive tracking algorithms, a stable and reliable target motion trajectory prediction model can be established, solving the technical problem that single sensor systems are prone to losing targets during continuous tracking.

[0138] Furthermore, the storage medium in the embodiments of this application stores program instructions capable of implementing all the above methods. These program instructions can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0139] The above description is merely a selection of preferred embodiments of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention as described in the embodiments is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A security system for detecting and controlling the trajectory of unauthorized flying drones, characterized in that, Includes the following modules: The photoelectric recognition module is used to acquire visible light and infrared images through a dual-spectrum sensor array and perform real-time registration and fusion. The image processing module is used for preprocessing infrared images and target enhancement. The target detection module uses an improved YOLOv8 neural network to detect drones. The attitude determination module uses feature point matching and 3D model to calculate the UAV's flight attitude. A multi-sensor fusion module is used to fuse visual data and GPS data to optimize 3D positioning; The visual positioning and tracking module uses a binocular recognition algorithm and a target tracking algorithm to track the trajectory of the drone. The countermeasure command module is used to trigger a response from the defense terminal based on the tracking results.

2. The system according to claim 1, characterized in that, The image processing module includes: Infrared images are preprocessed using an improved high-boost filter, and the calculation formula is as follows: I IHBF =I o *(l·max{I o ―I m ,0}) Among them, I IHBF To improve the infrared image data after high-voltage filtering, I o For raw infrared image data, I m The image data is after mean filtering, l is the enhancement coefficient constant, * represents the Hadamard product, and max is the maximum value function; Local contrast enhancement is performed based on a weighted three-layer window object detection algorithm, and the calculation formula is as follows: Among them, TLLCM is the enhanced result of the three-layer window local contrast detection algorithm, I C For the core layer Gaussian filtering result, I S It is the average of the maximum pixel values ​​in the surrounding layers.

3. The system according to claim 2, characterized in that, The formula for calculating the Gaussian filter in the core layer is as follows: Where (i,j) are the current pixel coordinates, G(p,q) are the weight coefficients of the two-dimensional Gaussian filter template, and I is the original infrared image data. C (i,j) represents the pixel value at coordinate (i,j), I(i+p,j+q) represents the pixel value at coordinate (i+p,j+q) in the original infrared image, and p and q are the template offsets.

4. The system according to claim 1, characterized in that, The target detection module employs a YOLOv8 neural network and includes: Channel attention mechanism enhances target feature channels by assigning different weights; Spatial attention mechanism, which focuses on the spatial distribution of the target by assigning different weights.

5. The system according to claim 1, characterized in that, The attitude determination module extracts image feature points through the SuperPoint network and matches them with a pre-constructed UAV 3D model, with a feature point matching rate of no less than 92.3%. The SuperPoint network includes a shared encoder, a keypoint detection decoder, a descriptor generation decoder, and a self-supervised training module.

6. The system according to claim 1, characterized in that, The multi-sensor fusion module uses extended Kalman filtering to fuse 5Hz GPS data with visual attitude output, and the three-dimensional positioning error does not exceed 0.45mRMSE. Single-frame attitude calculation is optimized using the PnP algorithm and bundle adjustment, with a yaw angle accuracy of 0.8° and a pitch angle accuracy of 1.2°.

7. The system according to claim 1, characterized in that, The visual positioning and tracking module employs a binocular recognition algorithm, which solves for the target's spatial position by solving a system of simultaneous equations. The system of equations includes: AO 2 =m 2 +BO 2 ―2m·BO·cosβ2 Where AO is the distance from the first capture device to the target, BO is the distance from the second capture device to the target, m is the distance between the two cameras, α2 is the measurement angle of the first capture device, and β2 is the measurement angle of the second capture device.

8. The system according to claim 1, characterized in that, The target tracking algorithm is a fusion method of YOLOv8 and kernelized correlation filtering (KCF), and its kernel correlation response calculation formula is as follows: Where f(z) is the kernel-related response function, F ―1 This is the inverse Fourier transform. This represents the kernel function of candidate region features and target features in the frequency domain. denoted as filter coefficients, and ⊙ as the Hadamard product.

9. The system according to claim 1, characterized in that, The system hardware includes: Dual-degree-of-freedom pan-tilt camera, model number 2-AxisPan-TiltCameraModule; Infrared thermal imaging equipment with a resolution of 384×288, supporting temperature difference detection of 0.04℃; The main control module adopts a collaborative architecture of ESP32-WROOM-32 and Raspberry 5.

10. The system according to claim 1, characterized in that, The countermeasure command module sends JSON commands to the defense terminal via the MQTT protocol. The commands include the target ID, three-dimensional coordinates, and threat level. The system adopts a layered architecture in the cloud. Data is transmitted to AWS EC2 or Alibaba Cloud ECS servers via EMQX or Mosquitto, and time-series data and structured logs are stored via InfluxDB and MongoDB.

Images