Unmanned aerial vehicle target detection method and system for complex region based on multi-device fusion

Abstract

The present disclosure provides a multi-device fusion-based complex region unmanned aerial vehicle target detection method and system, relating to the technical field of unmanned aerial vehicle detection, comprising: acquiring camera target detection data and multi-laser radar device target detection data, respectively performing target recognition on the target detection data of the camera device and the laser radar device, and outputting recognition classification information; performing coordinate system joint calibration based on the recognition classification information, constructing the relationship between coordinates, and respectively obtaining the target detection time and GPS coordinates of each device detection; based on each coordinate information, setting a target similarity judgment threshold, judging whether the detection data of each device is from the same target unmanned aerial vehicle according to a target unmanned aerial vehicle fusion judgment method, if it is judged that the detection data is from the same target unmanned aerial vehicle, then fusing the detection data of different devices, and updating the target unmanned aerial vehicle real-time database, to realize the fusion identification of target detection data.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of unmanned aerial vehicle detection, and in particular to an unmanned aerial vehicle target detection method and system based on multi-device fusion in a complex region. BACKGROUND

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute the prior art.

[0003] In recent years, unmanned aerial vehicle technology has developed rapidly, and has broken through the pattern in the fields of military reconnaissance, logistics distribution, urban security, agricultural monitoring, etc. In particular, in the military field, unmanned aerial vehicles, with their small size and low-speed and low-altitude characteristics, can play an important role in reconnaissance, surveillance and exercise tasks, and have become an indispensable tool in some scenarios. However, with the popularization and expansion of application scenarios of unmanned aerial vehicle technology, the identification and tracking technology of unmanned aerial vehicles is facing severe challenges. The existing unmanned aerial vehicle identification technology has obvious limitations in complex environments. Small unmanned aerial vehicles are difficult to be effectively detected and tracked by traditional radars due to their small size and small radar cross-section area; their low-altitude and slow flight mode enables them to evade the detection range of conventional air defense systems, thereby increasing the difficulty of tracking. These technical bottlenecks highlight the need to develop efficient and reliable unmanned aerial vehicle identification technology.

[0004] Existing UAV detection methods include an optical and visual detection method. Although the optical and visual detection method is low in cost and intuitive, its detection accuracy is extremely susceptible to adverse weather (such as rain and fog) and dramatic light changes (strong light during the day and darkness at night), resulting in a significant decline in performance. In a complex background, the visual features of small UAVs are easily overwhelmed, increasing the difficulty of target extraction. Another acoustic detection method has the advantages of low cost and concealment, but its core noise analysis is extremely susceptible to other environmental sounds (such as vehicles, human voices, and wind), resulting in high false negative and false positive rates, and the overall accuracy is severely limited. Radio frequency (RF) detection can monitor communication signals, but for UAVs that use autonomous flight, encryption, or complex coding protocols, the identification difficulty increases significantly. At the same time, it is highly dependent on signal strength and propagation environment, making it difficult to effectively deal with modern UAVs with strong anti-interference capabilities. In addition, there is a radar detection method. Although this method has a long detection distance and is less affected by weather, when detecting small UAVs, its small radar cross section (RCS) results in insufficient resolution and too low signal-to-noise ratio. The detection of low-altitude targets is also easily disturbed by ground clutter, and there are deficiencies in the number of identifications and visualization. Other infrared thermal imaging detection methods can provide advantages at night, but the high cost of equipment is a threshold for application. In hot climates, the temperature difference between the UAV and the high background temperature decreases, and the detection effect is greatly reduced. Secondly, the multi-spectral or hyperspectral detection method can provide more discriminative information by analyzing spectral information, but its technical maturity and application universality are relatively low, and the equipment cost is also high. SUMMARY

[0005] To solve the above problems, the present disclosure proposes a UAV target detection method and system in complex areas based on multi-device fusion. Based on the multi-sensor fusion technology of video, lidar, and UAV platform, video recognition combined with lidar can achieve high-precision target detection in complex backgrounds and provide three-dimensional position and motion information, realizing accurate positioning and tracking. Combined with the maneuverability of the UAV platform, the system flexibility is enhanced, ensuring the real-time and stability of tracking in complex scenes.

[0006] According to some embodiments, the present disclosure adopts the following technical solutions:

[0007] The UAV target detection method in complex areas based on multi-device fusion includes:

[0008] Obtaining camera target detection data and multi-lidar device target detection data, and constructing a target UAV real-time database;

[0009] Extracting image frames from the camera target detection data, performing target recognition on the image frames based on a target detection model, and outputting recognition classification information;

[0010] The point cloud data in each laser radar equipment target detection data is extracted, and after feature extraction, the point cloud data is input into a target detection model for classification and recognition, and identification classification information is respectively output.

[0011] Coordinate system joint calibration is performed based on the identification classification information output by the camera target detection and the identification classification information output by the multi-laser radar equipment target detection, a relationship between coordinates is constructed, and target phase position coordinates, detection time and GPS coordinates are obtained.

[0012] Based on the various coordinate information, a target similarity judgment threshold is set, and whether the detection data of each device is from the same target unmanned aerial vehicle is judged according to a target unmanned aerial vehicle fusion judgment method. If it is judged that the detection data is from the same target unmanned aerial vehicle, the detection data of different devices is fused, and the target unmanned aerial vehicle real-time database is updated, so as to realize the fusion identification of target detection data.

[0013] According to some embodiments, the present disclosure adopts the technical scheme as follows:

[0014] The unmanned aerial vehicle target detection system in a complex area based on multi-device fusion comprises:

[0015] A data acquisition module is configured to acquire camera target detection data and multi-laser radar equipment target detection data, and construct a target unmanned aerial vehicle real-time database.

[0016] A target recognition module is configured to extract image frames in the camera target detection data, perform target recognition on the image frames based on a target detection model, and output identification classification information. The point cloud data in each laser radar equipment target detection data is extracted, and after feature extraction, the point cloud data is input into a target detection model for classification and recognition, and identification classification information is respectively output.

[0017] A joint calibration module is configured to perform coordinate system joint calibration based on the identification classification information output by the camera target detection and the identification classification information output by the multi-laser radar equipment target detection, construct a relationship between coordinates, and obtain target phase position coordinates, detection time and GPS coordinates.

[0018] A fusion judgment module is configured to set a target similarity judgment threshold based on various coordinate information, judge whether the detection data of each device is from the same target unmanned aerial vehicle according to a target unmanned aerial vehicle fusion judgment method, fuse the detection data of different devices if it is judged that the detection data is from the same target unmanned aerial vehicle, update the target unmanned aerial vehicle real-time database, and realize the fusion identification of target detection data.

[0019] According to some embodiments, the present disclosure adopts the technical scheme as follows:

[0020] The computer program product comprises a computer program, and the computer program is executed by a processor to implement the multi-device fusion-based complex region unmanned aerial vehicle target detection method.

[0021] According to some embodiments, the present disclosure adopts the technical scheme as follows:

[0022] A non-transitory computer-readable storage medium is used to store computer instructions, and the computer instructions are executed by a processor to implement the multi-device fusion-based complex region unmanned aerial vehicle target detection method.

[0023] According to some embodiments, the present disclosure adopts the technical scheme as follows:

[0024] An electronic device comprises a processor, a memory and a computer program; the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device implements the multi-device fusion-based complex region unmanned aerial vehicle target detection method.

[0025] Compared with the prior art, the present disclosure has the beneficial effects as follows:

[0026] The multi-device fusion-based complex region unmanned aerial vehicle target detection method of the present disclosure is based on video, laser radar point cloud data and multi-sensor fusion technology transmitted by an unmanned aerial vehicle platform; video recognition is good at target appearance feature extraction, laser radar point cloud data has positioning and speed measurement capability in complex weather, and the unmanned aerial vehicle platform provides flexible monitoring viewing angle. The technology combines the advantages of the three, significantly improves the accuracy and reliability of target recognition. The combination of video recognition and laser radar point cloud data can realize high-precision target detection in complex background, provide three-dimensional position and motion information, and realize accurate positioning and tracking. The maneuverability of the unmanned aerial vehicle platform enhances the flexibility of the system, ensuring the real-time performance and stability of tracking in complex scenes. The multi-technology fusion scheme effectively solves the performance bottleneck of existing unmanned aerial vehicle recognition technology in complex scenes, and provides strong support for the wide application of unmanned aerial vehicles.

[0027] The multi-device fusion-based complex region UAV target detection method of the present disclosure significantly improves the detection performance in complex regions through precise coordinate conversion and joint calibration technology; geometric consistency and uniformity of multi-sensor (video, laser radar) data are achieved, through camera intrinsic parameter calibration, joint extrinsic parameter calibration between devices, and finally unified conversion to the GPS coordinate system, ensuring the spatial alignment of all sensor data and obtaining the unified and accurate three-dimensional position of the target. This greatly improves the target positioning accuracy and robustness in complex backgrounds, effectively combines the precise three-dimensional measurement of the laser radar with the video information, and can achieve high-precision detection and tracking in adverse weather or occluded environments. In addition, precise calibration and coordinate conversion enhance the support of the system for the maneuverability and global positioning capability of the UAV platform, ensuring that the UAV can achieve continuous, stable and real-time target tracking based on global unified target information in complex dynamic environments, overcoming the performance bottleneck of existing technologies in complex scenes.

[0028] The multi-device fusion-based complex region UAV target detection method of the present disclosure significantly improves the reliability and consistency of complex region UAV target detection through a fine data fusion identification process. The target similarity judgment method uses dynamic programming technology to accurately calculate the similarity of targets detected by different devices in GPS coordinates and time, and sets appropriate threshold values. This fine comparison mechanism effectively solves the multiple determination problem caused by sensor errors, synchronization deviations or independent detection, ensuring that only detection data from the same UAV is fused. Therefore, invalid or incorrect fusion is avoided, data redundancy is reduced, the accuracy of target identification is improved, and the independent database further realizes the ordered storage and management of target detection data, laying a solid foundation for subsequent precise tracking and situation awareness, and demonstrating its technical advancement and practical value in data fusion identification. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings, which form a part of this disclosure, are intended to provide further understanding of the present disclosure and are incorporated herein for illustrative purposes. The schematic embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0030] Figure 1 A flowchart of the multi-device fusion-based complex region UAV target detection method of the present disclosure embodiment;

[0031] Figure 2 A coordinate system conversion diagram of the present disclosure embodiment. DETAILED DESCRIPTION

[0032] The present disclosure will be further described below in conjunction with the drawings and embodiments.

[0033] It should be noted that the following detailed description is exemplary in nature and is intended to provide further description of the present disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this present disclosure belongs.

[0034] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0035] Embodiment 1

[0036] In an embodiment of the present disclosure, a multi-device fusion-based complex region unmanned aerial vehicle target detection method is provided, and the method steps include:

[0037] Step 1: Obtain camera target detection data and multi-laser radar device target detection data, and construct a target unmanned aerial vehicle real-time database;

[0038] Step 2: Extract the image frame in the camera target detection data, perform target recognition on the image frame based on a target detection model, and output the recognition classification information;

[0039] Step 3: Extract the point cloud data in each laser radar device target detection data, input the feature extraction to the target detection model for classification and recognition, and output the recognition classification information respectively;

[0040] Step 4: Based on the recognition classification information output by the camera target detection and the recognition classification information output by the multi-laser radar device target detection, perform coordinate system joint calibration, construct the relationship between the coordinates, and obtain the target phase position coordinates, detection time, and GPS coordinates;

[0041] Step 5: Based on each coordinate information, set a target similarity judgment threshold, judge whether the detection data of each device is from the same target unmanned aerial vehicle according to a target unmanned aerial vehicle fusion judgment method, if the detection data is from the same target unmanned aerial vehicle, fuse the detection data of different devices, and update the target unmanned aerial vehicle real-time database, to realize the fusion identification of the target detection data.

[0042] As an embodiment, the multi-device fusion-based complex region unmanned aerial vehicle target detection method of the present disclosure utilizes the advantages of precise positioning and identification of camera devices, combines the complex positioning accuracy of laser radar point cloud data and the maneuverability of unmanned aerial vehicle devices, realizes precise detection of targets in complex regions, and improves the detection accuracy of unmanned aerial vehicles in complex regions. The specific implementation process is as follows:

[0043] Step 1: Obtain camera target detection data and multi-laser radar device target detection data, and construct a target unmanned aerial vehicle real-time database.

[0044] Among them, obtaining camera target detection data and multi-laser radar device target detection data includes: using camera devices, first laser radar devices and second laser radar devices to collect their respective target detection data, the camera and the first laser radar device are fixedly installed at a detection area stop point, and the second laser radar device is installed on a detection unmanned aerial vehicle. The second laser radar device detects complex regions that cannot be detected by the camera device and the first laser radar device.

[0045] Specifically, the camera device is fixedly installed on the intersection gantry, which can completely cover the aerial target area. The first laser radar is installed on the intersection gantry, which can completely cover the aerial target area on the road. The second laser radar is loaded on the detection unmanned aerial vehicle, and the region stop point is set. When there is a task, the on-board laser radar will detect the complex regions that cannot be detected by the camera.

[0046] Obtain the real-time position and state data of the unmanned aerial vehicle identified by the camera device and the unmanned aerial vehicle device, and establish an unmanned aerial vehicle real-time database; the unmanned aerial vehicle real-time database includes a camera-detected unmanned aerial vehicle information library, a first laser radar device-detected unmanned aerial vehicle information library, and a second laser radar device-detected unmanned aerial vehicle information library. The camera device's unmanned aerial vehicle information library includes the number of unmanned aerial vehicles, target types, pixel coordinates, relative positions, GPS coordinates, and detection times. The laser radar device-detected unmanned aerial vehicle information library includes the number of unmanned aerial vehicles, target types, relative positions, GPS coordinates, and detection times. The unmanned aerial vehicle real-time database can be updated by subsequent steps.

[0047] The camera device collects video stream data of the target unmanned aerial vehicle, and the laser radar device collects point cloud data of the target unmanned aerial vehicle. The detection data collected by the camera device and the laser radar device is transmitted to the processor for target detection and identification processing.

[0048] Step 2: Extract the image frames in the camera target detection data, perform target identification on the image frames based on a target detection model, and output the identification classification information;

[0049] Specifically, after obtaining the video stream data of the target unmanned aerial vehicle collected by the camera device, a data set is constructed, and the target boundary box and category are labeled using a labeling tool. The data set is divided into training, validation and test sets to facilitate model training.

[0050] The target detection model is selected, and the YOLOV11 model can be selected. The hyperparameters are set for model training. The test set is used to evaluate the final model, and the average mAP, precision, recall and other indicators are calculated. The evaluation results are analyzed to identify the shortcomings of the model and continuously optimize the model.

[0051] As an embodiment, the specific process of target recognition based on the target detection model for image frames includes:

[0052] First, before target recognition, the image frames in the video stream are extracted in real time, and the video stream is converted into image frames.

[0053] Each image frame is input into the trained YOLOV11 model for target recognition, and the object category, confidence score and boundary box coordinates are output to obtain the recognition classification information, and the target recognition of the camera device is realized.

[0054] Step 3: Extract the point cloud data in each laser radar device target detection data, and input the extracted features into the target detection model for classification and recognition, and output the recognition classification information respectively;

[0055] Specifically, the point cloud data in the target detection data of the first laser radar device and the second laser radar device is preprocessed, including: denoising (statistics, radius, voxel filtering), downsampling (voxel, random, FPS), segmentation (clustering, model fitting, terrain segmentation), and synchronization and registration. The purpose is to remove noise and optimize data for subsequent processing.

[0056] Further, the point cloud data is converted into voxel form by PointPillars, and the PointNet method is used for feature extraction to extract differentiated information from the point cloud data, including point, voxel and object level features. The features are input into the target detection model to output the detected target category and target position respectively.

[0057] As an embodiment, the target detection model can be a YOLOV11 model.

[0058] Step 4: Based on the recognition classification information output by the camera target detection and the recognition classification information output by the multi-laser radar device target detection, the coordinate system is jointly calibrated, the relationship between the coordinates is constructed, and the target phase position coordinates, detection time and GPS coordinates are obtained;

[0059] Specifically, after the camera device detects and identifies the target drone, camera recognition classification information is obtained, which includes the pixel coordinates (u, v) of the target, the detection time T c , the GPS coordinates (X c , Y c , Z c ) of the camera;

[0060] After the first and second laser radar devices detect and identify the target, laser radar recognition classification information is obtained, which includes the relative position coordinates (X L1 , Y L1 , Z L1 ) of each target of the first laser radar device, the detection time T L1 of the first laser radar device, the GPS coordinates (x L1 , y L1 , z L1 ) of the first laser radar device, the relative position coordinates (X L2 , Y L2 , Z L2 ) of each target of the second laser radar device, the detection time T L2 of the second laser radar device, and the GPS coordinates (x L2 , y L2 , z L2 ) of the second laser radar device;

[0061] Further, based on the recognition classification information output by the camera target detection and the recognition classification information output by the multi-laser radar device target detection, coordinate system joint calibration is performed, including:

[0062] (1) Coordinate system calibration of the camera and the first laser radar device;

[0063] 1. Camera intrinsic parameter calibration

[0064] The conversion from the pixel coordinate system to the image coordinate system is as follows:

[0065] (1)

[0066] Wherein, the pixel coordinates of the target are (u, v), the origin of the pixel coordinates of the target is (0, 0), (x, y) is the coordinate of the image coordinate system, u is the horizontal coordinate, v is the vertical coordinate. x y x y

[0067] ​​​​A coordinate transformation method based on the pinhole imaging principle is used to transform the camera coordinate system to the image coordinate system. The formula is as follows:

[0068] (2)

[0069] The GPS coordinates of the camera are (X... c ,Y c Z c ), X c ,Y c These represent the offsets of the point in the camera coordinate system along the horizontal (X-axis) and vertical (Y-axis), respectively. Z c The depth from that point to the optical center of the camera. f The physical focal length is the distance from the optical center of the lens to the image plane.

[0070] The formula for transforming from pixel coordinates to camera coordinates is as follows:

[0071] (3)

[0072] Wherein, the origin of the target's pixel coordinates is ( u 0, v 0), the pixel coordinates of the target are ( u , v ), f x Focal length f In the image width direction x Pixel equivalent values ​​on the axis f y Focal length f In the image height direction y Pixel equivalent values ​​on the axis.

[0073] Perform camera internal parameters K The formula conversion is as follows:

[0074] (4)

[0075] 2. Camera extrinsic parameter calibration;

[0076] The formula for transforming the camera coordinate system to the lidar coordinate system is as follows:

[0077] (5)

[0078] in, R Let be a rotation matrix. T This is the homogeneous transformation matrix from camera to lidar.

[0079] Then camera external parameters WThe formula conversion is as follows:

[0080] (6)

[0081] 3. Coordinate system conversion of the camera and the first lidar device;

[0082] The pixel coordinates are converted to the lidar coordinate system, and the formula is as follows:

[0083] (7)

[0084] When the current detection time T L1= T c , the pixel coordinates (x u , v ) of the target obtained by the camera and the relative position coordinates (X L1 ,Y L1 ,Z L1 ) of the target obtained by the first lidar device are extracted, the relationship between the coordinates is constructed, the internal and external parameter calibration between the coordinates is realized, and the relative position coordinates (X C ,Y C ,Z C ) of the camera are calculated.

[0085] (2) Coordinate system calibration between devices

[0086] The relative position coordinates of each device are converted to the GPS coordinates of the target. Taking the coordinate system conversion of the camera as an example, the specific formula is as follows.

[0087] (8)

[0088] Where (x1, y1, z1) represents the GPS coordinates of a target, (X C ,Y C ,Z C ) is the relative position coordinates of the camera target, (x c ,y c ,z c ) is the GPS coordinates of the camera, R E is the radius of the earth, is the dimension of the device.

[0089] Similarly, after coordinate system conversion, the first lidar device (laser radar 1) and the second lidar device (laser radar 2) obtain the GPS coordinates of their respective targets.

[0090] Step 5: Based on the respective coordinate information, a target similarity judgment threshold is set, and whether the detection data of each device is from the same target UAV is judged according to the target UAV fusion judgment method. If it is judged that the detection data is from the same target UAV, the detection data of different devices is fused, and the target UAV real-time database is updated, so as to realize the fusion identification of target detection data.

[0091] Specifically, the relative position coordinates, GPS coordinate data and detection time data of each obtained UAV within a set time are obtained, and the data values are target GPS coordinates converted through step 4 above; by judging the similarity of the two GPS coordinate positions and detection time, if it exceeds the set target similarity judgment threshold, it is considered that the data of the two UAV trajectories comes from the same target, and the fusion identification is completed. The target UAV fusion judgment method is as follows:

[0092] Step 51: Create a matrix:

[0093] First, according to the GPS coordinates and detection time of the target in each device, the distance and time between each target data point of two devices are calculated, and a matrix is constructed.

[0094] Specifically, first, a detection time similarity judgment table is constructed, as shown in Table 1.

[0095] Table 1 Detection time similarity judgment table

[0096]

[0097] Secondly, a table is constructed to judge the distance similarity of each target identified by similar time, as shown in Table 2.

[0098] Table 2 Target distance similarity judgment table

[0099]

[0100] Then, the targets corresponding to the single target that meet the distance and time similarity are one-to-one corresponding, and a target similarity judgment matrix table is constructed, as shown in Table 3.

[0101] Table 3 Target similarity judgment matrix

[0102]

[0103] Step 52: Construct a dynamic programming table:

[0104] Two dynamic programming tables with the same size as the matrix are created. The first table records the detection time data from the starting point to the current point. The second table records the distance between the two targets of the current point.

[0105] Step 53: update the dynamic planning table

[0106] The data in the dynamic planning table is updated step by step from the starting point. The time data in the first table is updated according to the camera detection time data, and the distance data between the two targets when reaching each point is calculated by combining the time data in the first table. The updated method is to select the GPS coordinate data corresponding to the detection time point of the distributed optical fiber closest to the detection time of the first table, and calculate the distance.

[0107] Specifically, first, according to the detection time of the camera and the laser radar in Table 1, by comparing, the corresponding detection time less than the set threshold is found.

[0108] Secondly, for the GPS coordinates of each target detected by the laser radar and image recognition with similar detection time, the corresponding distance calculation is carried out by combining the Haversine formula, and the coordinates less than the set threshold are found. It is judged that the two coordinates are the same target.

[0109] Step 54: calculate the similarity

[0110] Then, according to the data of the two tables, the time and distance interpolation between each data point of the two targets is calculated. The similarity threshold is designed. If it is greater than the set threshold, it is considered that the position information of the two targets comes from the same target unmanned aerial vehicle;

[0111] Step 6: update the target unmanned aerial vehicle real-time database to realize the fusion identification of target detection data, which specifically includes:

[0112] 1. Constructing a camera realization database

[0113] The target unmanned aerial vehicle detected by the camera is assigned a unique identification number in the database as a piece of data. This piece of data has the following fields:

[0114] (1) Target number: This field represents the identification number of the target unmanned aerial vehicle detected by the camera.

[0115] (2) Target type: This field represents the detection and identification of different unmanned aerial vehicle models by existing deep learning algorithm technology.

[0116] (3) Pixel coordinates: This field represents the pixel coordinates of the target in the image coordinate system.

[0117] (4) Relative position: This field represents the coordinates of the target in the camera coordinate system.

[0118] (5) Detection time: This field represents the time of target detection.

[0119] (6) Camera GPS coordinates: This field represents the GPS coordinates of the detection camera.

[0120] (6) GPS coordinates: This field represents the GPS coordinates of the target.

[0121] 2. Constructing the target drone real-time database of the first laser radar

[0122] For the target drone detected by the first laser radar, a unique identification number is assigned in the database as a piece of data, which has the following fields:

[0123] (1) Target number: This field represents the identification number of the detected target drone.

[0124] (2) Target type: This field represents the type estimate of the target drone by the existing algorithm.

[0125] (3) Relative position: This field represents the coordinates of the target in the coordinate system.

[0126] (4) Detection time: This field represents the time when the target is detected.

[0127] (5) Radar GPS coordinates: This field represents the GPS coordinates of the detection radar.

[0128] (6) GPS coordinates: This field represents the GPS coordinates of the target.

[0129] 3. Constructing the target drone real-time database of the second laser radar

[0130] For the target drone detected by the second laser radar, a unique identification number is assigned in the database as a piece of data, which has the following fields:

[0131] (1) Target number: This field represents the identification number of the detected target drone.

[0132] (2) Target type: This field represents the type estimate of the target drone by the existing algorithm.

[0133] (3) Relative position: This field represents the coordinates of the target in the coordinate system.

[0134] (4) Detection time: This field represents the time when the target is detected.

[0135] (5) Radar GPS coordinates: This field represents the GPS coordinates of the detection radar.

[0136] (6) GPS coordinates: This field represents the GPS coordinates of the target.

[0137] (5) Updating the target drone real-time database.

[0138] The GPS coordinate positions from the two sensors are fused to determine the same, and the type of the unmanned aerial vehicle is determined to be the same. In the target unmanned aerial vehicle state database, the data is merged into one.

[0139] The real-time database is updated: the unique number, target type and detection time information are kept as the values when the camera device is monitored; the GPS coordinate positions are recorded as the detection values of the laser radar device (according to the last detected device).

[0140] Embodiment 2

[0141] In an embodiment of the present disclosure, an unmanned aerial vehicle target detection system based on multi-device fusion in a complex area is provided, comprising:

[0142] A data acquisition module is configured to acquire camera target detection data and multi-laser radar device target detection data, and construct a target unmanned aerial vehicle real-time database.

[0143] A target recognition module is configured to extract an image frame in the camera target detection data, perform target recognition on the image frame based on a target detection model, and output recognition classification information; extract point cloud data in each laser radar device target detection data, input the point cloud data after feature extraction into a target detection model for classification recognition, and respectively output recognition classification information.

[0144] A joint calibration module is configured to perform coordinate system joint calibration based on the recognition classification information output by the camera target detection and the recognition classification information output by the multi-laser radar device target detection, construct a relationship between coordinates, and obtain target phase position coordinates, detection time and GPS coordinates.

[0145] A fusion judgment module is configured to set a target similarity judgment threshold based on each coordinate information, judge whether the detection data of each device is from the same target unmanned aerial vehicle according to a target unmanned aerial vehicle fusion judgment method, fuse the detection data of different devices if it is judged that the detection data is from the same target unmanned aerial vehicle, and update the target unmanned aerial vehicle real-time database to realize fusion determination of target detection data.

[0146] Embodiment 3

[0147] In an embodiment of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the unmanned aerial vehicle target detection method based on multi-device fusion in a complex area.

[0148] Embodiment 4

[0149] In an embodiment of the present disclosure, a non-transitory computer readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the method for detecting a target of a UAV in a complex region based on multi-device fusion.

[0150] Embodiment 5

[0151] In an embodiment of the present disclosure, an electronic device is provided, comprising a processor, a memory, and a computer program; wherein the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device implements the method for detecting a target of a UAV in a complex region based on multi-device fusion.

[0152] The present disclosure is described with reference to the flowcharts and / or block diagrams of the method, device (system), and computer program product according to embodiments of the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in the flowcharts and / or block diagrams.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable data processing device to generate a computer-implemented process, so that the instructions executed by the computer or other programmable data processing device provide a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 one or more flows and / or blocks

[0154] Although the specific embodiments of the present disclosure are described above with reference to the accompanying drawings, the present disclosure is not limited thereto, and various modifications or changes can be made to the technical solutions of the present disclosure without departing from the scope of the present disclosure.

Claims

1. A method for detecting unmanned aerial vehicle (UAV) targets in complex areas based on multi-device fusion, characterized in that, include: Acquire target detection data from cameras and multiple lidar devices, and build a real-time database of target UAVs; Extract image frames from the camera target detection data, perform target recognition on the image frames based on the target detection model, and output the recognition and classification information; Extract point cloud data from the target detection data of each LiDAR device, extract its features, input it into the target detection model for classification and recognition, and output the recognition and classification information respectively. Based on the recognition and classification information output by the camera target detection and the recognition and classification information output by multiple lidar devices, the coordinate system is jointly calibrated, the relationship between the coordinates is constructed, and the target detection time and GPS coordinates of each device are obtained respectively. Based on various coordinate information, a target similarity judgment threshold is set. According to the target UAV fusion judgment method, it is determined whether the detection data of each device comes from the same target UAV. This includes: calculating the distance and time between each target data point between the two devices based on GPS coordinate data and detection time data, and constructing a matrix. Two dynamic programming tables of the same size as the matrix are created. The first table records the detection time data from the starting point to the current point, and the second table records the distance between the two targets at the current point. The dynamic programming tables are updated. The update process includes: starting from the starting point, the data in the dynamic programming tables are updated step by step. The time data of the first table is updated according to the camera detection time data. The distance data between the two targets at each point is updated by combining the time data of the first table. The update method is to select the GPS coordinate data corresponding to the detection time point of the distributed optical fiber that is closest to the detection time of the first table and calculate the distance. Based on the data of the two tables, the time and distance interpolation between each data point between the two targets is calculated. A similarity threshold is designed. If it is greater than the set threshold, it is considered that the two target position information come from the same target UAV. The detection data of different devices are fused and the target UAV real-time database is updated to realize the fusion and identification of target detection data.

2. The UAV target detection method for complex areas based on multi-device fusion as described in claim 1, characterized in that, Acquiring target detection data from cameras and multiple lidar devices includes: using a camera device, a first lidar device, and a second lidar device to collect their respective target detection data. The camera and the first lidar device are fixedly installed at a docking point in the detection area, and the second lidar device is installed on a detection drone. The second lidar device will detect complex areas that cannot be detected by the camera device and the first lidar device.

3. The UAV target detection method for complex areas based on multi-device fusion as described in claim 1, characterized in that, The system performs target recognition on image frames based on a target detection model, and extracts point cloud data from the target detection data of each LiDAR device. After feature extraction, the data is input into the target detection model for classification and recognition. This includes: the camera device acquires the target detection video stream, extracts image frames from the video stream in real time, inputs each image frame into the target detection model for target recognition, and outputs the recognition and classification information; the point cloud data from the target detection data of each LiDAR device is preprocessed, the point cloud data is converted into voxel form using PointPillars, and the PointNet method is used for feature extraction. Differential information is extracted from the point cloud data, including point, voxel, and object-level features. The features are input into the target detection model, and the recognition and classification information for each detection is output.

4. The UAV target detection method for complex areas based on multi-device fusion as described in claim 1, characterized in that, Coordinate system joint calibration is performed based on the recognition and classification information output by the camera target detection and the recognition and classification information output by multiple LiDAR devices. This includes: the pixel coordinates, detection time, and GPS coordinates of the target obtained after target recognition by the camera device; the relative position coordinates, detection time, and GPS coordinates of the target obtained after target recognition by the first LiDAR device; and the relative position coordinates, detection time, and GPS coordinates of the target obtained after target recognition by the second LiDAR device. Based on the above three types of data, coordinate system joint calibration is performed. First, the intrinsic parameters of the camera device are calibrated, and then the extrinsic parameters of the camera device are calibrated based on the first LiDAR device to obtain the relative position. The camera GPS coordinates and the relative position are then transformed to obtain the GPS coordinates. Finally, the detection time and GPS coordinates of the target UAV detected by each device are obtained.

5. The UAV target detection method for complex areas based on multi-device fusion as described in claim 1, characterized in that, The real-time database for drones includes target number, target type, pixel coordinates, relative position, detection time, camera / LiDAR GPS coordinates, and target GPS coordinates.

6. A UAV target detection system for complex areas based on multi-device fusion, specifically implementing the UAV target detection method for complex areas based on multi-device fusion as described in any one of claims 1-5, characterized in that, include: The data acquisition module is used to acquire target detection data from cameras and multiple lidar devices, and to build a real-time database of target UAVs. The target recognition module is used to extract image frames from the camera target detection data, perform target recognition on the image frames based on the target detection model, and output the recognition and classification information. Extract point cloud data from the target detection data of each LiDAR device, extract its features, input it into the target detection model for classification and recognition, and output the recognition and classification information respectively. The joint calibration module is used to perform joint coordinate system calibration based on the position information output by the camera target detection output and the position information output by multiple lidar devices, to build the relationship between coordinates and obtain the target phase position coordinates, detection time and GPS coordinates. The fusion judgment module is used to set a target similarity judgment threshold based on various coordinate information. According to the target UAV fusion judgment method, it determines whether the detection data of each device comes from the same target UAV. If the detection data is determined to come from the same target UAV, the detection data of different devices are fused and the target UAV real-time database is updated to realize the fusion and recognition of target detection data.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV target detection method for complex areas based on multi-device fusion as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the UAV target detection method for complex areas based on multi-device fusion as described in any one of claims 1-5.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to perform the UAV target detection method for complex areas based on multi-device fusion as described in any one of claims 1-5.

Images