Building-to-building unmanned aerial vehicle monitoring method and device based on detection tracking, equipment and medium

Abstract

The application discloses a building-to-building unmanned aerial vehicle monitoring method and device based on detection tracking, equipment and medium, relates to the unmanned aerial vehicle detection technical field, and includes: collecting target point cloud data in real time through a 4D millimeter wave radar to determine the spatial position information of the target unmanned aerial vehicle; adjusting the sensing direction of the visual sensing device according to the spatial position information, so as to obtain the visual recognition result of the target unmanned aerial vehicle through the visual sensing device and the pre-trained unmanned aerial vehicle recognition model; based on the spatial position information and the visual recognition result, the motion trajectory of the unmanned aerial vehicle is predicted, and the sensing direction is adjusted in real time according to the prediction result, so that the target unmanned aerial vehicle is in the image center position in the real-time image shot by the visual sensing device, and the real-time video stream of the unmanned aerial vehicle is obtained; through the cross-platform video stream transmission component, the real-time video stream of the unmanned aerial vehicle is forwarded and processed to be pushed to the target unmanned aerial vehicle monitoring platform, so that the target unmanned aerial vehicle monitoring platform provides the monitoring service of the target unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of drone detection technology, and in particular to a method, apparatus, equipment and medium for monitoring drones between buildings based on detection and tracking. Background Technology

[0002] With the popularization of drone technology and the expansion of its application scope, drone detection and tracking technologies are becoming increasingly important in fields such as security monitoring, airspace management, and countermeasures systems. Existing drone detection technologies, including radar detection methods, can effectively measure the distance and speed of targets, but they have certain limitations. First, in terms of sensors, traditional 3D (Three Dimensions) millimeter-wave radar cannot detect the vertical angle of obstacles, resulting in inaccurate obstacle detection and localization. Second, at the algorithm level, drones may have similar features to the background under complex background interference, and due to the small size of drones, it is difficult to obtain accurate features for differentiation, leading to a high rate of false detections and false negatives. In addition, most existing systems are limited to single-platform operation, resulting in a limited application scope. In other words, existing drone detection and tracking methods have poor detection accuracy and efficiency in scenarios where drones are passing through buildings, leading to less than ideal drone monitoring results.

[0003] In conclusion, improving the accuracy and efficiency of drone detection in scenarios where drones traverse buildings to optimize drone monitoring is a pressing issue that needs to be addressed. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, equipment, and medium for monitoring unmanned aerial vehicles (UAVs) between buildings based on detection and tracking, which can improve the accuracy and efficiency of UAV detection in scenarios where UAVs traverse buildings, thereby optimizing the UAV monitoring effect. The specific solution is as follows: Firstly, this application provides a detection-tracking-based method for monitoring unmanned aerial vehicles (UAVs) between buildings, applied to a UAV monitoring system. The UAV monitoring system includes a 4D millimeter-wave radar, a visual sensing device, and a cross-platform video streaming component. The 4D millimeter-wave radar and the visual sensing device are respectively deployed in target areas between target buildings. The method includes: The target point cloud data is collected in real time by the 4D millimeter-wave radar to determine the spatial location information of the target UAV based on the target point cloud data; The perception direction of the visual sensing device is adjusted according to the spatial location information so as to obtain the visual recognition result of the target drone through the visual sensing device and the pre-trained drone recognition model. Based on the spatial location information and the visual recognition results, the motion trajectory of the target UAV is predicted, and the perception direction of the visual sensing device is adjusted in real time according to the prediction results, so as to control the target UAV to be in the center position of the real-time image captured by the visual sensing device, and obtain the corresponding UAV real-time video stream. The cross-platform video stream transmission component forwards the real-time video stream of the drone to push it to the target drone monitoring platform, so that the target drone monitoring platform can provide monitoring services for the target drone.

[0005] Optionally, the 4D millimeter-wave radar is composed of multiple millimeter-wave radar units cascaded together. The transmitting antennas in the 4D millimeter-wave radar are arranged in an equally spaced array along the first direction and staggered along the second direction. The UAV recognition model is a model obtained by training a preset UAV recognition network, which is based on a target YOLO11 model. The detection layer of the target YOLO11 model includes a preset small target detection layer and adopts a feature-sharing detection head that meets preset lightweight conditions. Feature extraction is performed using wavelet transformation in the backbone network and neck network of the target YOLO11 model.

[0006] Optionally, the step of acquiring target point cloud data in real time using the 4D millimeter-wave radar to determine the spatial location information of the target UAV based on the target point cloud data includes: The target point cloud data is acquired in real time by the 4D millimeter-wave radar. The target point cloud data is processed using signal processing algorithms to determine the spatial position information of the target UAV; the spatial position information includes first distance information, velocity information, azimuth information, and pitch information.

[0007] Optionally, the spatial location information may also include the latitude and longitude coordinates of the target UAV; Accordingly, the step of acquiring target point cloud data in real time using the 4D millimeter-wave radar to determine the spatial location information of the target UAV based on the target point cloud data further includes: The second distance information of the target UAV is obtained by a coaxial laser sensor; the coaxial laser sensor is deployed in the target area between the target buildings. The latitude and longitude coordinates of the coaxial laser sensor are determined, and the latitude and longitude coordinates of the target UAV are determined based on the latitude and longitude coordinates of the sensor and the second distance information.

[0008] Optionally, obtaining the visual recognition result of the target drone through the visual perception device and the pre-trained drone recognition model includes: Acquire the target image currently captured by the visual sensing device; The target image is then used to perform drone identification and detection using a pre-trained drone identification model to obtain the corresponding drone identification results. The target drone is identified from the target image using a multi-target tracking algorithm and based on the drone identification results. A corresponding drone identifier is assigned to the target drone, thus obtaining the visual recognition result of the target drone.

[0009] Optionally, the step of predicting the trajectory of the target UAV based on the spatial location information and the visual recognition result, and adjusting the perception direction of the visual sensing device in real time according to the obtained prediction result, includes: The spatial location information and the visual recognition result are fused to obtain the corresponding fused data; The motion trajectory of the target UAV is predicted using the fused data and the Kalman filter algorithm to obtain the corresponding prediction result; Based on the prediction results, the sensing direction of the visual sensing device is adjusted in real time using a PID controller.

[0010] Optionally, the step of forwarding the real-time video stream of the UAV through the cross-platform video stream transmission component to push the real-time video stream of the UAV to the target UAV monitoring platform includes: The real-time video stream of the UAV is obtained through a real-time messaging protocol. If the encoding format of the UAV real-time video stream is the target encoding format, then the UAV real-time video stream is forwarded directly, and the protocol of the UAV real-time video stream is converted through a preset video stream transmission architecture to obtain the converted video stream; the preset video stream transmission architecture is an architecture that separates the signaling layer and the media layer. The converted video stream is pushed to the target drone monitoring platform via a preset API interface.

[0011] Secondly, this application provides a detection-tracking-based inter-building drone monitoring device, applied to a drone monitoring system. The drone monitoring system includes a 4D millimeter-wave radar, a visual sensing device, and a cross-platform video streaming component. The 4D millimeter-wave radar and the visual sensing device are respectively deployed in target areas between target buildings. The device includes: The data acquisition module is used to acquire target point cloud data in real time through the 4D millimeter-wave radar, so as to determine the spatial position information of the target UAV based on the target point cloud data; The orientation adjustment module is used to adjust the perception orientation of the visual perception device according to the spatial position information, so as to obtain the visual recognition result of the target drone through the visual perception device and the pre-trained drone recognition model. The trajectory prediction module is used to predict the motion trajectory of the target UAV based on the spatial location information and the visual recognition result, and adjust the perception direction of the visual sensing device in real time according to the obtained prediction result, so as to control the target UAV to be in the center position of the real-time image captured by the visual sensing device, and obtain the corresponding UAV real-time video stream. The forwarding processing module is used to forward the real-time video stream of the UAV through the cross-platform video stream transmission component, so as to push the real-time video stream of the UAV to the target UAV monitoring platform, so that the target UAV monitoring platform can provide monitoring services for the target UAV.

[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned detection-tracking-based inter-building drone monitoring method.

[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned detection-tracking-based inter-building drone monitoring method.

[0014] In this application, target point cloud data is acquired in real time by the 4D millimeter-wave radar to determine the spatial position information of the target UAV based on the target point cloud data; the perception direction of the visual sensing device is adjusted according to the spatial position information to obtain the visual recognition result of the target UAV through the visual sensing device and a pre-trained UAV recognition model; the motion trajectory of the target UAV is predicted based on the spatial position information and the visual recognition result, and the perception direction of the visual sensing device is adjusted in real time according to the obtained prediction result to control the target UAV to be in the center position of the real-time image captured by the visual sensing device, thereby obtaining the corresponding real-time video stream of the UAV; the real-time video stream of the UAV is forwarded through the cross-platform video stream transmission component to push the real-time video stream of the UAV to the target UAV monitoring platform so that the target UAV monitoring platform can provide monitoring services for the target UAV. As can be seen from the above, this application first acquires target point cloud data in real time using 4D millimeter-wave radar to determine the spatial position information of the target UAV. Then, based on the spatial position information, it adjusts the perception direction of the visual perception device and combines it with a pre-trained UAV recognition model to obtain the visual recognition result of the target UAV. Subsequently, based on the spatial position information and the visual recognition result, it predicts the UAV's trajectory and continuously adjusts the perception direction of the visual perception device to keep the target UAV in the center of the captured image, forming a stable real-time UAV video stream. Finally, it forwards the real-time UAV video stream to the target UAV monitoring platform through a cross-platform video stream transmission component, providing support for monitoring services. In this way, through the above process of this application, the fusion of 4D millimeter-wave radar and YOLO11 visual recognition algorithm achieves multi-source information complementarity, improves UAV detection accuracy, and enhances the stability and accuracy of UAV positioning and tracking. At the same time, the video stream is pushed through the cross-platform video stream transmission component to ensure smooth real-time transmission, achieving cross-platform compatibility, enhancing the practicality and reliability of the monitoring system, and thus improving the accuracy and efficiency of UAV detection in scenarios where UAVs pass through buildings, thereby optimizing the UAV monitoring effect. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This application discloses a flowchart of a building-to-building drone monitoring method based on detection and tracking. Figure 2This is a schematic diagram of the structure of a building-to-building drone monitoring device based on detection and tracking disclosed in this application; Figure 3 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Existing drone detection technologies, including radar detection methods, can effectively measure the distance and speed of targets, but they have certain limitations. First, in terms of sensors, traditional 3D millimeter-wave radar cannot detect the vertical angle of obstacles, leading to inaccurate obstacle detection and localization. Second, at the algorithm level, in complex background interference, drones may have similar features to the background; given the small size of drones, it is difficult to obtain precise features for differentiation, resulting in high false positive and false negative rates. Furthermore, most existing systems are limited to single-platform operation, limiting their application scope. In other words, existing drone detection and tracking methods have poor accuracy and efficiency in scenarios where drones are moving through buildings, resulting in less than ideal drone monitoring performance.

[0019] To overcome the aforementioned technical problems, this application provides a detection-tracking-based method for monitoring unmanned aerial vehicles (UAVs) between buildings, which can improve the accuracy and efficiency of UAV detection in scenarios where UAVs pass through buildings, thereby optimizing the UAV monitoring effect.

[0020] See Figure 1 As shown, this invention discloses a detection-tracking-based method for monitoring unmanned aerial vehicles (UAVs) between buildings, applied to a UAV monitoring system. The UAV monitoring system includes a 4D millimeter-wave radar, a visual sensing device, and a cross-platform video streaming component. The 4D millimeter-wave radar and the visual sensing device are respectively deployed in target areas between target buildings. The method includes: Step S11: Collect target point cloud data in real time using the 4D millimeter-wave radar to determine the spatial location information of the target UAV based on the target point cloud data.

[0021] In this embodiment, the 4D millimeter-wave radar is used to collect target point cloud data in real time, and the target object category is identified based on the target point cloud data. The target point cloud data is then filtered according to the target object category to determine the point cloud data of the target UAV, thereby determining the spatial location information of the target UAV.

[0022] It should be noted that the UAV monitoring system includes a 4D millimeter-wave radar detection module, a visual recognition and tracking module, a cross-platform video stream transmission module, and a central processing and control module. Specifically, the 4D millimeter-wave radar detection module is used to detect the spatial location information of the target UAV using the 4D millimeter-wave radar; the visual recognition and tracking module is used to achieve UAV identification and real-time tracking based on the YOLO11 (You Only Look Once 11, a real-time target detection model); the cross-platform video stream transmission module, also known as the cross-platform video stream transmission component, is used to achieve low-latency cross-platform transmission of video stream data; and the central processing and control module coordinates the work of each module to achieve multi-source data fusion processing. Table 1 below shows the module information table of a UAV monitoring system provided in this application.

[0023] Table 1 Module Information Table of UAV Monitoring System

[0024] It should be further noted that the 4D millimeter-wave radar is composed of multiple cascaded millimeter-wave radar units. The transmitting antennas in the 4D millimeter-wave radar are arranged in an equally spaced array along a first direction and staggered along a second direction. That is, the 4D millimeter-wave radar is composed of multiple cascaded millimeter-wave radar units, with each transmitting antenna arranged in an equally spaced array along the first direction and staggered along the second direction. Specifically, the 4D millimeter-wave radar detection module adopts a cascaded configuration of at least four units, 12 transmitting antennas and 16 receiving antennas, forming a millimeter-wave radar device with 192 virtual channels. The transmitting antennas are spaced apart along the first direction and staggered along the second direction, enabling the acquisition of distance, speed, azimuth, and pitch angle information of the target UAV, thus achieving 4D detection of the target UAV. It is understood that the hardware platform of this application system mainly includes a 4D millimeter-wave radar sensor, a visual perception device, a computing unit, and a transmission module. The 4D millimeter-wave radar sensor features a design with two transmitting antennas and three receiving antennas. The radar measures 60mm × 55mm × 5mm and weighs 20g, making it suitable for various drone platforms. For visual perception devices, such as high-definition cameras, industrial-grade cameras supporting at least 1080P resolution and 30fps (Frames Per Second) are selected to ensure high-quality image acquisition. The computing unit utilizes an embedded GPU (Graphics Processing Unit) to provide sufficient computing power to run the YOLO11 model. The system's software environment includes Ubuntu 18.04 (a Linux operating system primarily for desktop applications), the PyTorch deep learning framework, the OpenCV computer vision library, and a video streaming framework based on GStreamer (a multimedia framework).

[0025] Specifically, the 4D millimeter-wave radar collects target point cloud data in real time; signal processing algorithms are used to process the target point cloud data to determine the spatial position information of the target UAV; the spatial position information includes first distance information, velocity information, azimuth angle information, and pitch angle information. That is, in the implementation of the 4D millimeter-wave radar detection module, the radar is configured to operate at a frequency of 77GHz, with a horizontal detection range of -38° to +38° and a vertical detection range of -17° to +17°. The 4D millimeter-wave radar collects target point cloud data in real time, and the signal processing algorithm calculates the spatial position information of the target UAV, including first distance, velocity, azimuth angle, and pitch angle information, which is then transmitted to the central processing and control module via a high-speed data interface.

[0026] It should be noted that the spatial location information also includes the latitude and longitude coordinates of the target UAV; correspondingly, the process of determining the spatial location information also includes determining the latitude and longitude coordinates, and the processing flow is as follows: A second distance information of the target UAV is obtained through a coaxial laser sensor; the coaxial laser sensor is deployed in the target area between the target buildings; the latitude and longitude coordinates of the coaxial laser sensor are determined, and the latitude and longitude coordinates of the target UAV are determined based on the sensor's latitude and longitude coordinates and the second distance information. That is, the spatial location information also includes the latitude and longitude coordinates of the target UAV; the second distance information of the target UAV is obtained through a coaxial laser sensor deployed between the target buildings; the latitude and longitude coordinates of the target UAV are calculated by combining the sensor's own latitude and longitude coordinates with the second distance information and inversely calculating the azimuth angle. In this way, this embodiment acquires the spatial position information of the target UAV through 4D millimeter-wave radar, providing accurate spatial coordinate information (including pitch angle). It can stably acquire accurate spatial positioning of the UAV in complex environments, providing a reliable positional basis for subsequent tracking and identification. The 4D millimeter-wave radar, composed of multiple cascaded millimeter-wave radar units with transmitting antennas arranged in an equally spaced array along the first direction and staggered along the second direction, can effectively improve the radar's angular resolution and detection coverage, and enhance the integrity of point cloud data and spatial perception accuracy. The latitude and longitude coordinate information of the target UAV is acquired through a coaxial laser sensor, complementing the millimeter-wave radar data to achieve accurate positioning, effectively improving positioning accuracy and scene adaptability, and providing a more reliable spatial reference for UAV tracking.

[0027] Step S12: Adjust the perception direction of the visual perception device according to the spatial location information so as to obtain the visual recognition result of the target drone through the visual perception device and the pre-trained drone recognition model.

[0028] In this embodiment, the perception direction of the visual perception device is adjusted according to the spatial location information, and then the visual recognition result of the target drone is obtained through the visual perception device and the pre-trained drone recognition model.

[0029] It should be noted that the drone recognition model is a model obtained by training a preset drone recognition network, which is based on a target YOLO11 model. The detection layer of the target YOLO11 model includes a preset small target detection layer and uses a feature-sharing detection head that meets preset lightweight conditions. Wavelet transforms are used for feature extraction in the backbone and neck networks of the target YOLO11 model. In other words, the implementation of the visual recognition and tracking module includes the training and deployment of the YOLO11 model. The drone recognition model is obtained by training a preset drone recognition network based on the target YOLO11 model using a preset training dataset. The preset training dataset is a large-scale dataset containing multiple scenarios, consisting of preprocessed drone image data from different background environments. The detection layer of the target YOLO11 model adds a preset small target detection layer, and the detection head uses a feature-sharing detection head that meets preset lightweight conditions to reduce the number of network parameters. Wavelet transforms are used in the backbone and neck networks instead of traditional convolution operations to extract multi-scale features using low-frequency and high-frequency components.

[0030] It should be noted that the processing flow for obtaining the visual recognition result of the target drone is as follows: First, acquire the target image currently captured by the visual sensing device; second, use a pre-trained drone recognition model to perform drone recognition and detection on the target image to obtain the corresponding drone recognition result; third, use a multi-target tracking algorithm and based on the drone recognition result to determine the target drone from the target image and assign a corresponding drone identifier to the target drone, thus obtaining the visual recognition result of the target drone. In other words, first, acquire the target image currently captured by the visual sensing device; then, use a pre-trained drone recognition model to perform drone recognition and detection; finally, combine this with a multi-target tracking algorithm to determine and lock onto the target drone, assign a unique drone identifier to the target drone, and obtain the visual recognition result of the target drone. In this way, this embodiment adjusts the perception direction of the visual perception device according to the spatial location information of the UAV, realizing the coordinated cooperation between radar positioning and visual recognition, achieving higher detection accuracy, and improving the accuracy and real-time performance of target capture and recognition. A preset small target detection layer is added to the detection layer of the preset UAV recognition network, significantly improving the system's performance in complex backgrounds and small target detection scenarios. The YOLO11 algorithm uses shared lightweight detection heads and wavelet transform techniques to achieve model lightweighting, improve the effectiveness of feature extraction, reduce computational complexity and network parameter count, increase detection speed, and adapt to real-time recognition scenario requirements. A multi-target tracking algorithm identifies and locks onto the target UAV, assigning it a UAV identifier, achieving stable, accurate, and continuous tracking and differentiation of the target UAV.

[0031] Step S13: Based on the spatial location information and the visual recognition result, predict the motion trajectory of the target UAV, and adjust the perception direction of the visual sensing device in real time according to the prediction result, so as to control the target UAV to be in the center position of the real-time image captured by the visual sensing device, and obtain the corresponding UAV real-time video stream.

[0032] In this embodiment, the spatial location information and the visual recognition result are fused to predict the motion trajectory of the target drone. Based on the obtained prediction result, the direction of the visual sensing device is adjusted in real time to stabilize the target drone in the center position of the real-time image captured by the visual sensing device, thereby outputting a stable and clear real-time video stream of the drone.

[0033] It should be noted that the process of predicting the trajectory of the target UAV and adjusting the perception direction of the visual sensing device in real time based on the prediction results is as follows: The spatial location information and the visual recognition results are fused to obtain fused data; the fused data is used to predict the trajectory of the target UAV using a Kalman filter algorithm to obtain a prediction result; and the perception direction of the visual sensing device is adjusted in real time using a PID controller based on the prediction result. That is, a multi-source data fusion algorithm is used to fuse the spatial location information and the visual recognition results, then a Kalman filter algorithm is used to predict the trajectory of the target UAV, and finally, based on the prediction result, a PID controller (Proportional-Integral-Derivative controller) is used to adjust the perception direction of the visual sensing device in real time, adaptively adjusting the tracking strategy. In this way, this embodiment integrates multi-source data to adjust the perception direction of the visual sensing device in real time, controlling the target UAV to be at the center of the image, achieving accurate positioning and tracking of the UAV, and improving the smoothness and accuracy of tracking.

[0034] Step S14: The real-time video stream of the UAV is forwarded through the cross-platform video stream transmission component to push the real-time video stream of the UAV to the target UAV monitoring platform so that the target UAV monitoring platform can provide monitoring services for the target UAV.

[0035] In this embodiment, the cross-platform video stream transmission component is used to forward and process the real-time video stream of the drone and push it to the target drone monitoring platform at the other end, so that the target drone monitoring platform can provide monitoring services for the target drone.

[0036] It should be noted that the process of pushing the real-time video stream of the UAV to the target UAV monitoring platform through the cross-platform video stream transmission component is as follows: The real-time video stream of the UAV is obtained through a real-time message transmission protocol; if the encoding format of the real-time video stream is the target encoding format, the real-time video stream is forwarded directly; the real-time video stream is then converted using a preset video stream transmission architecture to obtain the converted video stream; the preset video stream transmission architecture is an architecture that separates the signaling layer and the media layer; the converted video stream is then pushed to the target UAV monitoring platform through a preset API interface. In other words, the Real-Time Messaging Protocol (RTMP) is used to pull streaming media to obtain the real-time video stream of the UAV. Video streams that conform to the target encoding format, such as H.264 / H.265 encoding format, are forwarded directly without re-encoding to maintain the original video quality, reducing processing latency and computing resource consumption. The protocol conversion is completed based on the transmission architecture that separates the signaling layer and the media layer, and then pushed to the target UAV monitoring platform through the preset API (Application Programming Interface), that is, the unified API interface of multiple platforms.

[0037] Understandably, the cross-platform video streaming module uses the RTMP protocol for streaming media retrieval, supporting pass-through forwarding of H.264 / H.265 encoding formats. This maintains the original video quality without re-encoding, reducing processing latency and computational resource consumption. The module can efficiently push audio and video frames to a designated RTMP server, ensuring the real-time nature and continuity of the video stream. It also provides a robust event callback mechanism, providing real-time feedback on streaming speed, push status, and connection status metrics. Built-in reconnection and weak network adaptive mechanisms automatically adjust transmission strategies in fluctuating network environments to maintain business continuity. Furthermore, a unified API interface across multiple platforms, including Windows, Ubuntu, and Linux, ensures consistent functionality and interface calling methods across different operating system environments. The cross-platform video streaming module adopts a signaling layer and media layer separation architecture to achieve multi-protocol conversion and transmission. The signaling layer supports multiple protocols such as RTSP (Real-Time Streaming Protocol) and RTMP as signaling plugins, receiving and processing various signaling information to achieve signaling conversion between different protocols. The media layer supports streaming media transmission protocols such as RTP (Real-time Transport Protocol), SRT (Secure Reliable Transport), and HLS (HTTP Live Streaming) as media streaming plugins, receiving and processing audio and video data, and then forwarding it. This module includes the audio and video processing module construction process, signaling negotiation process, media transmission process, and signaling exit negotiation process, and can be deployed at the service layer and / or access layer, providing good scalability and flexibility. In this way, the cross-platform video streaming component in this embodiment optimizes video streaming efficiency, provides the system with stable, compatible, and low-latency video transmission, ensures the real-time performance of the system, and guarantees the reliable operation of monitoring services. Through the architecture of separating the signaling layer and the media layer, it supports the access of multiple protocols in the form of plug-ins, realizing cross-platform compatibility, enabling the system to adapt to different hardware platforms and application scenarios, and achieving low-loss, highly compatible, stable and reliable cross-platform video transmission. The deep integration of 4D millimeter-wave radar, visual recognition, tracking algorithms and cross-platform transmission technology forms a highly integrated system solution, reducing dependence on external devices and reducing system complexity.

[0038] As can be seen from the above, this embodiment first acquires target point cloud data in real time using 4D millimeter-wave radar to determine the spatial position information of the target drone. Then, based on the spatial position information, it adjusts the perception direction of the visual perception device and combines it with a pre-trained drone recognition model to obtain the visual recognition result of the target drone. Subsequently, based on the spatial position information and the visual recognition result, it predicts the drone's trajectory and continuously adjusts the perception direction of the visual perception device to keep the target drone in the center of the captured image, forming a stable real-time drone video stream. Finally, it forwards the real-time drone video stream to the target drone monitoring platform through a cross-platform video stream transmission component, providing support for monitoring services. In this way, through the above process of this embodiment, the fusion of 4D millimeter-wave radar and YOLO11 visual recognition algorithm achieves multi-source information complementarity, improves drone detection accuracy, and enhances the stability and accuracy of drone positioning and tracking. At the same time, the video stream is pushed through the cross-platform video stream transmission component to ensure smooth real-time transmission, achieving cross-platform compatibility, enhancing the practicality and reliability of the monitoring system, and thus improving the accuracy and efficiency of drone detection in scenarios where drones pass through buildings, thereby optimizing the drone monitoring effect.

[0039] Accordingly, see Figure 2 As shown, this application embodiment also provides a detection-tracking-based inter-building drone monitoring device, applied to a drone monitoring system. The drone monitoring system includes a 4D millimeter-wave radar, a visual sensing device, and a cross-platform video streaming component. The 4D millimeter-wave radar and the visual sensing device are respectively deployed in the target area between target buildings. The device includes: Data acquisition module 11 is used to acquire target point cloud data in real time through the 4D millimeter-wave radar, so as to determine the spatial position information of the target UAV based on the target point cloud data; The orientation adjustment module 12 is used to adjust the perception orientation of the visual perception device according to the spatial position information, so as to obtain the visual recognition result of the target drone through the visual perception device and the pre-trained drone recognition model. The trajectory prediction module 13 is used to predict the motion trajectory of the target UAV based on the spatial location information and the visual recognition result, and adjust the perception direction of the visual sensing device in real time according to the obtained prediction result, so as to control the target UAV to be in the center position of the real-time image captured by the visual sensing device, and obtain the corresponding UAV real-time video stream. The forwarding processing module 14 is used to forward the real-time video stream of the UAV through the cross-platform video stream transmission component, so as to push the real-time video stream of the UAV to the target UAV monitoring platform, so that the target UAV monitoring platform can provide monitoring services for the target UAV.

[0040] In some specific embodiments, the 4D millimeter-wave radar is composed of multiple millimeter-wave radar units cascaded together. The transmitting antennas in the 4D millimeter-wave radar are arranged in an equally spaced array along a first direction and staggered along a second direction. The UAV recognition model is a model obtained by training a preset UAV recognition network, which is based on a target YOLO11 model. The detection layer of the target YOLO11 model includes a preset small target detection layer and uses a feature-sharing detection head that meets preset lightweight conditions. Feature extraction is performed using wavelet transformation in the backbone and neck networks of the target YOLO11 model.

[0041] In some specific embodiments, the data acquisition module 11 may specifically include: The data acquisition unit is used to acquire target point cloud data in real time through the 4D millimeter-wave radar; The data processing unit is used to process the target point cloud data using signal processing algorithms to determine the spatial position information of the target UAV; the spatial position information includes first distance information, velocity information, azimuth information, and pitch information.

[0042] In some specific embodiments, the spatial location information also includes the latitude and longitude coordinates of the target UAV; Accordingly, the data acquisition module 11 may further include: An information acquisition unit is used to acquire second distance information of the target drone through a coaxial laser sensor; the coaxial laser sensor is deployed in the target area between the target buildings. The information determination unit is used to determine the latitude and longitude coordinates of the coaxial laser sensor, and to determine the latitude and longitude coordinates of the target UAV based on the latitude and longitude coordinates of the sensor and the second distance information.

[0043] In some specific embodiments, the direction adjustment module 12 may specifically include: The image acquisition unit is used to acquire the target image currently captured by the visual sensing device; The identification and detection unit is used to perform drone identification and detection on the target image using a pre-trained drone identification model to obtain the corresponding drone identification result; The identifier allocation unit is used to determine the target drone from the target image through a multi-target tracking algorithm and based on the drone recognition result, and to assign a corresponding drone identifier to the target drone to obtain the visual recognition result of the target drone.

[0044] In some specific embodiments, the trajectory prediction module 13 may specifically include: The data fusion unit is used to fuse the spatial location information and the visual recognition result to obtain the corresponding fused data; The trajectory prediction unit is used to predict the motion trajectory of the target UAV using the fused data and a Kalman filter algorithm to obtain the corresponding prediction result. An orientation adjustment unit is used to adjust the perception orientation of the visual perception device in real time based on the prediction result and through a PID controller.

[0045] In some specific embodiments, the forwarding processing module 14 may specifically include: The video stream acquisition unit is used to acquire the real-time video stream of the UAV via a real-time message transmission protocol. The protocol conversion unit is used to forward the real-time video stream of the UAV directly if the encoding format of the UAV real-time video stream is the target encoding format, and to convert the protocol of the real-time video stream of the UAV through a preset video stream transmission architecture to obtain the converted video stream; the preset video stream transmission architecture is an architecture that separates the signaling layer and the media layer. The video stream push unit is used to push the converted video stream to the target drone monitoring platform through a preset API interface.

[0046] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the detection-tracking-based inter-building drone monitoring method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.

[0047] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0048] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0049] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the detection-tracking-based inter-building drone monitoring method disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0050] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned detection-tracking-based inter-building drone monitoring method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0051] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0052] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0053] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0054] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0055] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for monitoring unmanned aerial vehicles (UAVs) between buildings based on detection and tracking, characterized in that, An application is made to a drone monitoring system, the drone monitoring system including a 4D millimeter-wave radar, a visual sensing device, and a cross-platform video streaming component, wherein the 4D millimeter-wave radar and the visual sensing device are respectively deployed in a target area between target buildings; wherein, the method includes: The target point cloud data is collected in real time by the 4D millimeter-wave radar to determine the spatial location information of the target UAV based on the target point cloud data; The perception direction of the visual sensing device is adjusted according to the spatial location information so as to obtain the visual recognition result of the target drone through the visual sensing device and the pre-trained drone recognition model. Based on the spatial location information and the visual recognition results, the motion trajectory of the target UAV is predicted, and the perception direction of the visual sensing device is adjusted in real time according to the prediction results, so as to control the target UAV to be in the center position of the real-time image captured by the visual sensing device, thereby obtaining the corresponding UAV real-time video stream. The cross-platform video stream transmission component forwards the real-time video stream of the drone to push it to the target drone monitoring platform, so that the target drone monitoring platform can provide monitoring services for the target drone.

2. The method for monitoring unmanned aerial vehicles (UAVs) between buildings based on detection and tracking according to claim 1, characterized in that, The 4D millimeter-wave radar is composed of multiple cascaded millimeter-wave radar units. The transmitting antennas in the 4D millimeter-wave radar are arranged in an equally spaced array along a first direction and staggered along a second direction. The UAV recognition model is a model obtained by training a preset UAV recognition network, which is based on a target YOLO11 model. The detection layer of the target YOLO11 model includes a preset small target detection layer and uses a feature-sharing detection head that meets preset lightweight conditions. Feature extraction is performed using wavelet transforms in the backbone and neck networks of the target YOLO11 model.

3. The method for monitoring unmanned aerial vehicles (UAVs) between buildings based on detection and tracking according to claim 1, characterized in that, The step of acquiring target point cloud data in real time using the 4D millimeter-wave radar to determine the spatial location information of the target UAV based on the target point cloud data includes: The target point cloud data is acquired in real time by the 4D millimeter-wave radar. The target point cloud data is processed using signal processing algorithms to determine the spatial position information of the target UAV; the spatial position information includes first distance information, velocity information, azimuth information, and pitch information.

4. The method for monitoring unmanned aerial vehicles (UAVs) between buildings based on detection and tracking according to claim 1, characterized in that, The spatial location information also includes the latitude and longitude coordinates of the target UAV; Accordingly, the step of acquiring target point cloud data in real time using the 4D millimeter-wave radar to determine the spatial location information of the target UAV based on the target point cloud data further includes: The second distance information of the target UAV is obtained by a coaxial laser sensor; the coaxial laser sensor is deployed in the target area between the target buildings. The latitude and longitude coordinates of the coaxial laser sensor are determined, and the latitude and longitude coordinates of the target UAV are determined based on the latitude and longitude coordinates of the sensor and the second distance information.

5. The method for monitoring unmanned aerial vehicles (UAVs) between buildings based on detection and tracking according to claim 1, characterized in that, The step of obtaining the visual recognition result of the target drone through the visual perception device and the pre-trained drone recognition model includes: Acquire the target image currently captured by the visual sensing device; The target image is then used to perform drone identification and detection using a pre-trained drone identification model to obtain the corresponding drone identification results. The target drone is identified from the target image using a multi-target tracking algorithm and based on the drone identification results. A corresponding drone identifier is assigned to the target drone, thus obtaining the visual recognition result of the target drone.

6. The method for monitoring unmanned aerial vehicles (UAVs) between buildings based on detection and tracking according to claim 1, characterized in that, The step of predicting the trajectory of the target UAV based on the spatial location information and the visual recognition result, and adjusting the perception direction of the visual sensing device in real time according to the obtained prediction result, includes: The spatial location information and the visual recognition result are fused to obtain the corresponding fused data; Using the fused data and a Kalman filter algorithm, the motion trajectory of the target UAV is predicted to obtain the corresponding prediction result; Based on the prediction results, the sensing direction of the visual sensing device is adjusted in real time using a PID controller.

7. The method for monitoring unmanned aerial vehicles (UAVs) between buildings based on detection and tracking according to any one of claims 1 to 6, characterized in that, The step of forwarding the real-time video stream of the drone through the cross-platform video stream transmission component to push the real-time video stream of the drone to the target drone monitoring platform includes: The real-time video stream of the UAV is obtained through a real-time messaging protocol. If the encoding format of the UAV real-time video stream is the target encoding format, then the UAV real-time video stream is forwarded directly, and the protocol of the UAV real-time video stream is converted through a preset video stream transmission architecture to obtain the converted video stream; the preset video stream transmission architecture is an architecture that separates the signaling layer and the media layer. The converted video stream is pushed to the target drone monitoring platform via a preset API interface.

8. A building-to-building drone monitoring device based on detection and tracking, characterized in that, An application is made in a drone monitoring system, which includes a 4D millimeter-wave radar, a visual sensing device, and a cross-platform video streaming component. The 4D millimeter-wave radar and the visual sensing device are deployed in a target area between target buildings. The device includes: The data acquisition module is used to acquire target point cloud data in real time through the 4D millimeter-wave radar, so as to determine the spatial position information of the target UAV based on the target point cloud data; The orientation adjustment module is used to adjust the perception orientation of the visual perception device according to the spatial position information, so as to obtain the visual recognition result of the target drone through the visual perception device and the pre-trained drone recognition model. The trajectory prediction module is used to predict the motion trajectory of the target UAV based on the spatial location information and the visual recognition result, and adjust the perception direction of the visual sensing device in real time according to the obtained prediction result, so as to control the target UAV to be in the center position of the real-time image captured by the visual sensing device, and obtain the corresponding UAV real-time video stream. The forwarding processing module is used to forward the real-time video stream of the UAV through the cross-platform video stream transmission component, so as to push the real-time video stream of the UAV to the target UAV monitoring platform, so that the target UAV monitoring platform can provide monitoring services for the target UAV.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the detection-tracking-based inter-building drone monitoring method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the detection-tracking-based inter-building drone monitoring method as described in any one of claims 1 to 7.

Images