Unmanned aerial vehicle target detection positioning method and system based on multi-modal collaborative perception

Abstract

The application discloses a multi-modal cooperative sensing based unmanned aerial vehicle target detection and positioning method and system, relates to the technical field of unmanned aerial vehicle detection, and comprises the following steps: acquiring and aligning multi-modal sensing data, calculating grid cell signal-to-interference-and-noise ratio and occlusion rate, and generating a spatial quality mask graph; distributing noise time steps according to the spatial quality mask graph, and constructing a spatial-modal asynchronous time step matrix; performing joint denoising reconstruction based on the matrix, extracting differential features, and generating a differential guide feature graph; enhancing a target boundary by using the differential guide feature graph, acquiring unmanned aerial vehicle bounding box coordinates and time sequence trajectory points; searching for electromagnetic signals and filtering multipath interference based on unmanned aerial vehicle spatial information constraints, and calculating operator ground three-dimensional coordinates; and combining unmanned aerial vehicle and operator positioning results, matching and outputting prevention and control strategy instructions. The application solves the technical problem of low unmanned aerial vehicle target detection and positioning accuracy in the prior art, and achieves the technical effect of improving unmanned aerial vehicle target and operator positioning accuracy.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) detection technology, and more specifically to a UAV target detection and localization method and system based on multimodal cooperative perception. Background Technology

[0002] With the widespread application of low-altitude drones in inspection, aerial photography, and logistics, accurate detection and positioning of drone targets has become a crucial requirement for low-altitude security and control. In densely built-up urban environments with severe obstruction and complex electromagnetic conditions, different modal sensing data, such as optoelectronic images, radar point clouds, and spectrum monitoring, are susceptible to noise, obstruction, and environmental interference. The sensing quality of each modal data varies across different spatial regions, making it difficult to adapt a uniform data processing approach to the data quality of different areas and modalities. Furthermore, building reflections can easily generate multipath spurious signals in spectrum monitoring data, causing real target signals to mix with interference signals, thus hindering drone target feature extraction, trajectory acquisition, and electromagnetic signal analysis. Summary of the Invention

[0003] This application provides a method and system for UAV target detection and localization based on multimodal cooperative perception, which is used to address the technical problem of low accuracy of UAV target detection and localization in complex environments in the prior art.

[0004] In view of the above problems, this application provides a method and system for UAV target detection and localization based on multimodal cooperative perception.

[0005] The first aspect of this application provides a method for UAV target detection and localization based on multimodal cooperative perception, the method comprising: Acquire multimodal sensing data, align the multimodal sensing data to a preset three-dimensional grid coordinate system, calculate the signal-to-interference-plus-noise ratio (SIR) and occlusion rate of each grid cell under different modes, and generate a spatial quality mask map; based on the SIR and occlusion rate in the spatial quality mask map, allocate independent noise time steps to each grid cell, and construct a spatial-modal asynchronous time step matrix; according to the scheduling constraints of the spatial-modal asynchronous time step matrix, perform joint denoising and reconstruction on the feature maps of each mode, and extract the differential feature tensor between adjacent time step feature maps to generate a differential guided feature map; use the differential guided feature map to enhance the target boundary features, and extract the bounding box coordinates and time-series trajectory points of the UAV target; use the bounding box coordinates and time-series trajectory points of the UAV target as the spatial constraint interval for electromagnetic signal search, use the differential guided feature map to filter multipath interference in ground electromagnetic signals, and calculate the ground three-dimensional coordinates of the UAV operator; combine the bounding box coordinates, time-series trajectory points, and ground three-dimensional coordinates of the UAV target, match and output the control strategy command.

[0006] A second aspect of this application provides a UAV target detection and localization system based on multimodal cooperative perception, the system comprising: The data acquisition module acquires multimodal sensing data, aligns the multimodal sensing data to a preset 3D grid coordinate system, calculates the signal-to-interference-plus-noise ratio (SIR) and occlusion rate of each grid cell under different modalities, and generates a spatial quality mask. The matrix construction module allocates independent noise time steps to each grid cell based on the SIR and occlusion rate in the spatial quality mask, constructing a spatial-modal asynchronous time step matrix. The denoising and reconstruction module performs joint denoising and reconstruction of the feature maps of each modality according to the scheduling constraints of the spatial-modal asynchronous time step matrix, and extracts the difference between feature maps of adjacent time steps. The system uses a feature tensor to generate a differential guided feature map; an extraction module enhances the target boundary features using the differential guided feature map to extract the bounding box coordinates and temporal trajectory points of the UAV target; a calculation module uses the bounding box coordinates and temporal trajectory points of the UAV target as the spatial constraint interval for electromagnetic signal search, filters multipath interference in ground electromagnetic signals using the differential guided feature map, and calculates the ground three-dimensional coordinates of the UAV operator; and a command matching module combines the bounding box coordinates, temporal trajectory points, and ground three-dimensional coordinates of the UAV target to match and output control strategy commands.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application acquires multimodal sensing data, aligns the multimodal sensing data to a preset three-dimensional grid coordinate system, calculates the signal-to-interference-plus-noise ratio (SIR) and occlusion rate of each grid cell under different modes, and generates a spatial quality mask map. Based on the SIR and occlusion rate in the spatial quality mask map, an independent noise time step is allocated to each grid cell, constructing a spatial-modal asynchronous time step matrix. According to the scheduling constraints of the spatial-modal asynchronous time step matrix, the feature maps of each mode are jointly denoised and reconstructed, and the differential feature tensor between adjacent time step feature maps is extracted to generate a differential guided feature map. The differential guided feature map is used to enhance the target boundary features, and the bounding box coordinates and time-series trajectory points of the UAV target are extracted. The bounding box coordinates and time-series trajectory points of the UAV target are used as the spatial constraint interval for electromagnetic signal search. The differential guided feature map is used to filter multipath interference in the ground electromagnetic signal, and the ground three-dimensional coordinates of the UAV operator are calculated. Combining the bounding box coordinates, time-series trajectory points, and ground three-dimensional coordinates of the UAV operator, a control strategy command is matched and output. This invention addresses the technical problem of low accuracy in target detection and positioning of UAVs in complex environments in existing technologies. By performing joint denoising reconstruction and differential feature enhancement based on multimodal perception data, combined with target trajectory constraint electromagnetic signal search and multipath interference filtering, the technical effect of improving the positioning accuracy of UAV targets and their operators is achieved. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A schematic diagram of the UAV target detection and localization method based on multimodal cooperative perception provided in this application embodiment; Figure 2 Spatial quality mask image in the UAV target detection and localization method based on multimodal cooperative perception provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of a UAV target detection and positioning system based on multimodal cooperative perception, provided in an embodiment of this application.

[0010] Figure labeling: Data acquisition module 11, matrix construction module 12, denoising and reconstruction module 13, extraction module 14, calculation module 15, instruction matching module 16. Detailed Implementation

[0011] This application provides a method and system for UAV target detection and localization based on multimodal cooperative perception. It addresses the technical problem of low accuracy in UAV target detection and localization in complex environments in existing technologies. By performing joint denoising reconstruction and differential feature enhancement based on multimodal perception data, combined with target trajectory constraint electromagnetic signal search and multipath interference filtering, it achieves the technical effect of improving the positioning accuracy of UAV targets and their operators.

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] It should be noted that any variations of the terms "include" and "have" are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such processes, methods, products, or devices.

[0014] Example 1, as Figure 1 As shown, this application provides a method for UAV target detection and localization based on multimodal cooperative perception, the method comprising: Step S100: Acquire multimodal sensing data, align the multimodal sensing data to a preset three-dimensional grid coordinate system, calculate the signal-to-interference-plus-noise ratio and occlusion rate of each grid cell under different modes, and generate a spatial quality mask map.

[0015] Furthermore, the method provided in the application embodiments also includes: The multimodal sensing data includes integrated sensing data, photoelectric images, radar point clouds, and spectrum monitoring data.

[0016] In this embodiment, when acquiring multimodal perception data, synchronous perception is performed within the UAV target monitoring area using a sensor-integrated device, an optoelectronic device, a radar device, and a spectrum monitoring device. Specifically, the sensor-integrated device collects communication perception information corresponding to the target and its surrounding space, forming sensor-integrated data; the optoelectronic device continuously collects image sequences of the monitoring area, forming an optoelectronic image containing the UAV target's appearance and spatial location features; the radar device scans the monitoring area to obtain the distance, azimuth, and spatial scattering information of the target and surrounding objects, forming a radar point cloud; simultaneously, the spectrum monitoring device scans and receives radio signals within the monitoring area, acquiring corresponding frequency, power, and signal-to-interference-plus-noise ratio (SINNR) spectrum monitoring data. The data collected by each device are time-synchronized according to the acquisition time, and spatial correlation is performed based on the spatial location and coordinate parameters of each sensing device, thereby obtaining multimodal perception data for subsequent 3D mesh mapping.

[0017] Next, the multimodal sensing data is mapped to a 3D grid coordinate system. For each grid cell, the pixel occlusion rate of the photoelectric image, the point cloud density of the radar point cloud, and the signal-to-interference-plus-noise ratio (SIR / NNR) of the spectrum monitoring data are extracted. When a grid cell contains photoelectric image, radar point cloud, and spectrum monitoring data simultaneously, the pixel occlusion rate, point cloud density, and SIR / NNR of that grid cell are calculated using the corresponding methods. When a grid cell lacks some modal data, calculations are performed based on the actual modal data present in that grid cell. Specifically, when a grid cell does not have a photoelectric image projection area, the pixel occlusion rate of that grid cell is set to 1; when a grid cell does not have radar point cloud data, the point cloud density of that grid cell is set to 0; when a grid cell does not have spectrum monitoring data, the SIR / NNR of that grid cell is set to a preset minimum value. After the above processing, the pixel occlusion rate, point cloud density, and SIR / NNR of each grid cell are normalized and fused according to corresponding weighting coefficients to generate a spatial quality mask map representing the data quality assessment value of each grid cell.

[0018] Furthermore, the method provided in the application embodiment, which involves acquiring multimodal sensing data, aligning the multimodal sensing data to a preset three-dimensional mesh coordinate system, calculating the signal-to-interference-plus-noise ratio and occlusion rate of each mesh cell under different modes, and generating a spatial quality mask map, further includes: The multimodal sensing data is mapped to the three-dimensional grid coordinate system; for each grid cell, the pixel occlusion rate of the photoelectric image, the point cloud density of the radar point cloud, and the signal-to-interference-plus-noise ratio of the spectrum monitoring data are extracted respectively; the pixel occlusion rate, point cloud density, and signal-to-interference-plus-noise ratio are normalized and weighted and fused to obtain a spatial quality mask map that expresses the data quality assessment value of the grid cell.

[0019] In this embodiment, when mapping multimodal sensing data to a three-dimensional grid coordinate system, the origin of the three-dimensional grid coordinate system is first determined within the monitoring area, and the X-axis, Y-axis, and Z-axis directions are set. The grid length in the X-axis direction is set to Lx, the grid width in the Y-axis direction is set to Ly, and the grid height in the Z-axis direction is set to Lz. The monitoring area is continuously divided according to the preset grid size to obtain multiple grid units. For any sampling point with three-dimensional coordinates, the grid number to which the sampling point belongs is calculated. The X-axis grid number is equal to the difference between the X-coordinate of the sampling point and the X-coordinate of the origin, divided by Lx, rounded down, and then added by 1. The Y-axis grid number is equal to the difference between the Y-coordinate of the sampling point and the Y-coordinate of the origin, divided by Ly, rounded down, and then added by 1. The Z-axis grid number is equal to the difference between the Z-coordinate of the sampling point and the Z-coordinate of the origin, divided by Lz, rounded down, and then added by 1. The corresponding grid unit is determined based on the three obtained grid numbers. For radar point clouds, the range, azimuth, and elevation angles of each radar point are read. The distance to the radar point relative to the radar equipment in the X direction is obtained by multiplying the distance by the cosine of the elevation angle, then by the cosine of the azimuth angle; the distance in the Y direction is obtained by multiplying the distance by the cosine of the elevation angle, then by the sine of the azimuth angle; and the distance in the Z direction is obtained by multiplying the distance by the sine of the elevation angle. These are then added to the X, Y, and Z coordinates of the radar equipment in the three-dimensional grid coordinate system to obtain the three-dimensional coordinates of the radar point, and written into the corresponding grid cell according to the above grid number calculation method. For integrated sensor data, the target range, azimuth, and elevation angles obtained from the integrated sensor device are read, and the corresponding three-dimensional coordinates are obtained using the same trigonometric function calculation method, and then written into the corresponding grid cell. For photoelectric images, the pre-calibrated focal length, principal point coordinates, installation coordinates, and attitude parameters of the photoelectric device are read. The three-dimensional coordinates of the eight vertices of each grid cell are read sequentially. Based on the installation coordinates and attitude parameters of the photoelectric device, the lateral, longitudinal, and depth coordinates of each vertex relative to the device are calculated. The focal length is multiplied by the lateral coordinate, divided by the depth coordinate, and then the lateral coordinate of the principal point is added to obtain the lateral pixel coordinates. The focal length is multiplied by the longitudinal coordinate, divided by the depth coordinate, and then the longitudinal coordinate of the principal point is added to obtain the longitudinal pixel coordinates. The photoelectric image region corresponding to the current grid cell is determined based on the pixel coordinates of the eight vertices. For spectrum monitoring data, the installation X, Y, and Z coordinates of the corresponding spectrum monitoring device are read according to the device number. The grid cell containing the spectrum monitoring device is then determined using the grid number calculation method described above. The spectrum monitoring data obtained by that device at the current acquisition time is recorded in the corresponding grid cell. After the above processing is completed, each grid cell records the integrated sensor data, photoelectric image, radar point cloud, and spectrum monitoring data corresponding to its spatial location.

[0020] Next, for each grid cell, the pixel occlusion rate of the photoelectric image, the point cloud density of the radar point cloud, and the signal-to-interference-plus-noise ratio of the spectrum monitoring data are extracted. For the photoelectric image region of the current grid cell, the actual depth of each pixel position is first determined based on the radar points in the radar point cloud corresponding to the image region. Then, the distances from the eight vertices of the current grid cell to the photoelectric device are calculated. The three-dimensional coordinates of the eight vertices of the current grid cell and the three-dimensional coordinates of the photoelectric device are read respectively. For each vertex, the coordinate difference between the vertex and the photoelectric device in the X-axis, Y-axis, and Z-axis directions are calculated. The three coordinate differences are squared and summed, and the square root of the sum is taken to obtain the distance from the vertex to the photoelectric device. The distances corresponding to the eight vertices are obtained in the same way, and the magnitudes of the eight distances are compared. The distance with the smallest value is determined as the boundary depth of the current grid cell on the side closer to the photoelectric device. Pixels with actual depth in the current photoelectric image region are read one by one. The actual depth of the pixel is compared with the boundary depth. When the actual depth of the pixel is less than the boundary depth and the difference between the two is greater than a preset depth threshold, the pixel is recorded as an occluded pixel. First, count the number of occluded pixels, then count the total number of pixels participating in depth comparison within the current photoelectric image area. Calculate the pixel occlusion rate of the current grid cell using the formula: Pixel Occlusion Rate = Number of Occluded Pixels ÷ Total Number of Counting Pixels. For example, if the total number of counting pixels within the current photoelectric image area is 1000, and 300 are occluded, then the pixel occlusion rate is 300 ÷ 1000 = 0.3. For radar point clouds, count the number of valid radar points located within the boundaries of the current grid cell's X, Y, and Z axes. Calculate the spatial volume of the current grid cell using the formula: Grid Cell Volume = Lx × Ly × Lz. Then calculate the point cloud density using the formula: Point Cloud Density = Number of Valid Radar Points ÷ Grid Cell Volume. For example, if the grid cell volume is 8 cubic meters and contains 40 valid radar points, then the point cloud density is 40 ÷ 8 = 5 points per cubic meter. For spectrum monitoring data, the power of the target signal at the current monitoring frequency band is read as the target signal power; the power of other frequency points within the monitoring band that are higher than the preset noise threshold is read, and the power of these frequency points is added together to obtain the interference signal power; the power of frequency points without effective signal peaks is read, and the power of these frequency points is added together and divided by the corresponding number of frequency points to obtain the noise power; then the signal-to-interference-plus-noise ratio (SIR) is calculated according to the formula: SIR = Target signal power ÷ Sum of interference signal power and noise power. For example, if the target signal power is 60, the interference signal power is 15, and the noise power is 5, then the SIR is 60 ÷ (15 + 5) = 3. All grid cells are processed sequentially according to the above process to obtain the pixel occlusion rate, point cloud density, and SIR corresponding to each grid cell.

[0021] Finally, when normalizing and weighting the pixel occlusion rate, point cloud density, and signal-to-interference-plus-noise ratio (SNR), the normalized values ​​of the three data points are first calculated separately. Since a higher pixel occlusion rate indicates a greater degree of occlusion, the normalized pixel occlusion rate is calculated as 1 - pixel occlusion rate. For example, if the pixel occlusion rate is 0.3, the normalized pixel occlusion rate is 1 - 0.3 = 0.7. For point cloud density, the point cloud densities of all grid cells are statistically analyzed to obtain the maximum and minimum point cloud densities. The normalized point cloud density is then calculated as: point cloud density normalized = current grid cell point cloud density minus minimum point cloud density, divided by the difference between the maximum and minimum point cloud densities. For example, if the current grid cell point cloud density is 5, the maximum point cloud density among all grid cells is 8, and the minimum point cloud density is 2, then the normalized point cloud density is (5 - 2) ÷ (8 - 2) = 0.5. For the signal-to-interference-plus-noise ratio (SIR / NDR), the SIR / NDR of all grid cells is statistically analyzed to obtain the maximum and minimum SIR / NDR. Then, the normalized SIR / NDR value is calculated as follows: SIR / NDR of the current grid cell minus the minimum SIR / NDR, then divided by the difference between the maximum and minimum SIR / NDR. For example, if the current grid cell SIR / NDR is 3, the maximum SIR / NDR is 5, and the minimum SIR / NDR is 1, then the normalized SIR / NDR value is (3-1) ÷ (5-1) = 0.5. When the maximum and minimum values ​​are the same, the corresponding normalized value is set to 1. After normalization, corresponding weighting coefficients are set according to the influence of different modal perception data on the UAV target detection results. The weighting coefficient for pixel occlusion rate is set to 0.3, the weighting coefficient for point cloud density is set to 0.3, and the weighting coefficient for SIR / NDR is set to 0.4. The sum of the three weighting coefficients is then equal to 1. The normalized pixel occlusion rate is multiplied by the corresponding weighting coefficient, the normalized point cloud density is multiplied by the corresponding weighting coefficient, and the normalized signal-to-interference-plus-noise ratio is multiplied by the corresponding weighting coefficient. These three products are then summed to obtain the data quality assessment value for the current grid cell. The same method is used to calculate the data quality assessment values ​​for all grid cells sequentially. Then, based on the X-axis, Y-axis, and Z-axis grid indices of each grid cell, the corresponding data quality assessment values ​​are written to the appropriate positions to obtain the spatial quality mask. The spatial quality mask is shown below. Figure 2 As shown, the data is organized according to the spatial arrangement of each grid cell in the 3D grid coordinate system. Each position corresponds to a specific grid cell, and the data quality evaluation value of that grid cell is recorded after normalization and weighted fusion of pixel occlusion rate, point cloud density, and signal-to-interference-plus-noise ratio. Since the pixel occlusion rate is normalized by subtracting the pixel occlusion rate from 1, and the point cloud density and signal-to-interference-plus-noise ratio are both processed by forward normalization, the larger the data quality evaluation value in the spatial quality mask, the higher the overall data quality of the corresponding grid cell, and the smaller the data quality evaluation value, the lower the overall data quality of the corresponding grid cell.

[0022] Step S200: Based on the signal-to-interference-plus-noise ratio and occlusion rate in the spatial quality mask, assign an independent noise time step to each grid cell and construct a spatial-modal asynchronous time step matrix.

[0023] In this embodiment, based on the signal-to-interference-plus-noise ratio and occlusion rate in the spatial quality mask image, and combined with preset high noise time step threshold, low noise time step threshold and quality threshold, the data quality and occlusion of each grid cell are determined respectively, and independent noise time steps are assigned to each modal feature channel corresponding to different grid cells; then, the noise time steps of each feature channel are combined according to the grid cell position and modal dimension to construct a spatial-modal asynchronous time step matrix.

[0024] Furthermore, in the method provided in the application embodiments, based on the signal-to-interference-plus-noise ratio and occlusion rate in the spatial quality mask image, an independent noise time step is assigned to each grid cell to construct a spatial-modal asynchronous time step matrix, which further includes: Set high-noise time step thresholds and low-noise time step thresholds; when the spatial quality mask indicates that the data quality of a certain grid cell is lower than the quality threshold or there is occlusion, allocate a high-noise time step close to 1 to the feature channel of that grid cell; when the spatial quality mask indicates that the data quality of a certain grid cell is higher than the quality threshold, allocate a low-noise time step close to 0 to the feature channel of that grid cell; combine the time steps of each channel according to the grid cell position to form a spatial-modal asynchronous time step matrix that includes spatial position and modal dimension.

[0025] In this embodiment, high-noise time step thresholds and low-noise time step thresholds are first set. The noise time step value range is set to 0 to 1, the low-noise time step threshold is set to 0.3, and the high-noise time step threshold is set to 0.7. Simultaneously, quality thresholds and occlusion rate thresholds are set; for example, the quality threshold is set to 0.5, and the occlusion rate threshold is set to 0.3. The data quality evaluation value, signal-to-interference-plus-noise ratio (SIR), and occlusion rate corresponding to each grid cell in the spatial quality mask image are read, and each grid cell is processed sequentially according to the X-axis grid number, Y-axis grid number, and Z-axis grid number in the three-dimensional grid coordinate system. For the signal-to-interference-plus-noise ratio (SNR), first, the maximum and minimum SNR of the corresponding modal feature channels in all grid cells are calculated. The minimum SNR of the current grid cell is subtracted from the current SNR, and then divided by the difference between the maximum and minimum SNR to obtain a normalized SNR value between 0 and 1. When the maximum and minimum SNRs are equal, the corresponding normalized SNR value is set to 1. For the occlusion rate, the pixel occlusion rate between 0 and 1 obtained during the aforementioned spatial quality mask generation process is directly read, and this pixel occlusion rate is used as the occlusion rate of the photoelectric image feature channel corresponding to the current grid cell. For the current grid cell, the data quality assessment value is compared with a quality threshold, and the occlusion rate is also compared with an occlusion rate threshold. When the data quality assessment value is less than 0.5, or the occlusion rate is greater than or equal to 0.3, the high-noise time step allocation process begins; when the data quality assessment value is greater than or equal to 0.5 and the occlusion rate is less than 0.3, the low-noise time step allocation process begins. Thus, high-noise time step allocation results or low-noise time step allocation results are obtained for each grid cell.

[0026] When the spatial quality mask indicates that the data quality of a certain grid cell is below the quality threshold or that there is occlusion, a high-noise time step close to 1 is assigned to the feature channel of that grid cell. Specifically, the normalized signal-to-interference-plus-noise ratio (SNR) of the current feature channel is first subtracted from 1 to obtain the noise ratio corresponding to the SNR. Then, the noise ratio is added to the occlusion rate of the current grid cell and divided by 2 to obtain the noise ratio value of the current feature channel. Subsequently, the high-noise time step threshold of 0.7 is subtracted from 1 to obtain the adjustable range of 0.3 for the high-noise time step. The noise ratio value is multiplied by 0.3 and then added to 0.7 to obtain the high-noise time step corresponding to the current feature channel. For example, if the normalized SNR of the current feature channel is 0.4 and the occlusion rate of the current grid cell is 0.6, then the noise ratio is [(1-0.4)+0.6]÷2=0.6, and the high-noise time step is 0.7+0.6×0.3=0.88. Following the same calculation process, the normalized signal-to-interference-plus-noise ratio (SNR) values ​​and occlusion conditions of the corresponding feature channels in the current grid cell for photoelectric images, radar point clouds, and spectrum monitoring data are read, and the corresponding high-noise time steps are calculated for each feature channel, ensuring that each feature channel has an independent noise time step. When the spatial quality mask indicates that the data quality of a certain grid cell is higher than the quality threshold, a low-noise time step close to 0 is assigned to the feature channel of that grid cell. Specifically, 1 is subtracted from the normalized SNR value of the current feature channel to obtain the noise ratio, and then the noise ratio is multiplied by the low-noise time step threshold of 0.3 to obtain the low-noise time step of the current feature channel. For example, if the normalized SNR value of the current feature channel is 0.8, then the low-noise time step is (1-0.8)×0.3=0.06. Each modal feature channel within the same grid cell is processed sequentially, and then all grid cells are processed in the same way to obtain the independent noise time step corresponding to each feature channel in each grid cell; among them, the high-noise time steps are all between 0.7 and 1, and the low-noise time steps are all between 0 and 0.3.

[0027] After allocating noise time steps for all grid cells, the time steps of each channel are combined according to the grid cell position to form a spatial-modal asynchronous time step matrix containing both spatial location and modal dimension. Specifically, in the 3D grid coordinate system, each grid cell is read sequentially according to the X-axis grid number (from smallest to largest), Y-axis grid number (from smallest to largest), and Z-axis grid number (from smallest to largest). Then, the noise time steps of the corresponding feature channels of the photoelectric image, radar point cloud, and spectrum monitoring data in that grid cell are read according to a pre-determined modal arrangement. A time step data group is established for each grid cell. The noise time steps of the photoelectric image feature channels are written to the first position, the noise time steps of the radar point cloud feature channels are written to the second position, and the noise time steps of the spectrum monitoring data feature channels are written to the third position. If multiple feature channels exist, they are written sequentially according to fixed channel numbers. Subsequently, the time step data groups of each grid cell are written to their corresponding spatial positions according to their X-axis, Y-axis, and Z-axis grid numbers. For example, if the noise time steps corresponding to the photoelectric image feature channel, radar point cloud feature channel, and spectrum monitoring data feature channel of a certain grid cell are 0.88, 0.12, and 0.76 respectively, then 0.88, 0.12, and 0.76 are recorded sequentially at the corresponding position in that grid cell; the next grid cell records the noise time steps corresponding to its modal feature channels in the same order. After writing the data for all grid cells in the above order, a spatial-modal asynchronous time step matrix is ​​obtained. Each matrix position is determined by the grid cell's X-axis grid number, Y-axis grid number, Z-axis grid number, and modal feature channel number. Each matrix element stores the independent noise time step for the corresponding grid cell and corresponding modal feature channel.

[0028] Step S300: Based on the scheduling constraints of the spatial-modal asynchronous time step matrix, perform joint denoising and reconstruction on the feature maps of each modality, and extract the differential feature tensors between adjacent time step feature maps to generate differential guided feature maps.

[0029] In this embodiment, based on the scheduling constraints of the spatial-modal asynchronous time step matrix, the spatial-modal asynchronous time step matrix is ​​input into the denoising network. The sampling step size in the denoising iteration process is controlled according to the noise time step corresponding to each grid cell, and the feature maps of each modality are jointly denoised and reconstructed. In the denoising iteration process, the denoised feature map of step t and the denoised feature map of step (t-1) are extracted, and the two are subtracted element-wise to obtain the differential feature tensor. Then, the differential feature tensor is mapped with an activation function to generate a differential guided feature map that expresses high-frequency edges and texture constraints.

[0030] Furthermore, in the method provided in the application embodiments, according to the scheduling constraints of the spatial-modal asynchronous time step matrix, joint denoising and reconstruction of the feature maps of each modality are performed, and the differential feature tensors between adjacent time step feature maps are extracted to generate differential guided feature maps, which further includes: The spatial-modal asynchronous time step matrix is ​​input into the denoising network to control the sampling step size of different grid cells during the denoising iteration process. During the denoising iteration process, the denoising feature map of step t and the denoising feature map of step (t-1) are extracted. The denoising feature map of step t and the denoising feature map of step (t-1) are subtracted element-wise to obtain the differential feature tensor. After being mapped by the activation function, a differential guided feature map expressing high-frequency edges and texture constraints is generated.

[0031] In this embodiment, when inputting the spatial-modal asynchronous time step matrix into the denoising network, the feature maps corresponding to the photoelectric image, radar point cloud, and spectrum monitoring data are first arranged to the same spatial size according to the grid cell positions in the three-dimensional grid coordinate system, and the grid positions in each modal feature map correspond one-to-one with the grid positions in the spatial-modal asynchronous time step matrix. Specifically, the photoelectric image, radar point cloud, and spectrum monitoring data are first converted into feature maps corresponding to the three-dimensional grid coordinates, so that each feature map position corresponds to a three-dimensional grid cell; then, according to the grid cell positions and modal channel numbers recorded in the spatial-modal asynchronous time step matrix, the noise time step corresponding to each modal feature map position is read, so that each modal feature value has a corresponding noise time step. The modal feature maps are then stitched together according to the channel direction, and the noise time step at the corresponding position is used as the network input parameter, and input together with the stitched modal feature map into the denoising network. The denoising network employs a convolutional denoising network, taking each modality feature map and its corresponding noise time step as input, sequentially passing them through convolutional layers, ReLU activation layers, and another convolutional layer, outputting a noise feature map of the same size as the input feature map. During training, known noise-free feature maps are supplemented with Gaussian noise of a preset intensity to form the training input, using the added Gaussian noise as supervision data. The mean square error between the noise feature map output by the denoising network and the Gaussian noise is calculated and backpropagated until the mean square error stabilizes, at which point the network parameters are saved, resulting in the denoising network. During denoising iterations, the maximum discrete time step is set to 1000. For each grid cell and its corresponding modality feature channel, the noise time step at the corresponding position in the spatial-modal asynchronous time step matrix is ​​read, multiplied by the maximum discrete time step of 1000, and rounded to the nearest integer to obtain the corresponding discrete time step. Specifically, the current discrete time step is equal to the product of the noise time step and the maximum discrete time step, rounded to the nearest integer, thus converting the continuous noise time step (within the range of 0 to 1) into a discrete time step in the denoising iteration process. The sampling step size corresponding to the high-noise time step is set to 1, and the sampling step size corresponding to the low-noise time step is set to 2. For each grid cell, the modal feature value and its corresponding discrete time step are read. The modal feature value and the noise time step are concatenated according to the channel direction and then input into the denoising network to maintain the spatial and modal channel correspondence between the input features and the noise scheduling parameters. The denoising network outputs the noise feature value corresponding to the grid cell. The noise feature value is then subtracted from the current feature value to obtain the denoised feature value. Subsequently, the corresponding sampling step size is subtracted from the current discrete time step to obtain the next discrete time step, and the above denoising calculation is repeated until the discrete time step is reduced to 0.The above calculations are performed on photoelectric images, radar point clouds, and spectrum monitoring data according to their respective noise time steps in the spatial-modal asynchronous time step matrix. After each denoising iteration, the modal features in the same grid cell are spliced ​​together according to the channel direction, and then 1×1 convolution is used to calculate to obtain the joint denoising and reconstruction feature map corresponding to the current time step.

[0032] During the denoising iteration process, the joint denoising and reconstructed feature maps obtained from two adjacent denoising iterations are continuously saved according to the same grid cell position and feature channel order. The joint denoising and reconstructed feature map obtained at the current time step is denoised as the t-th step denoising feature map, and the joint denoising and reconstructed feature map obtained at the next adjacent denoising time step is denoised as the (t-1)-th step denoising feature map, ensuring that the t-th step denoising feature map and the (t-1)-th step denoising feature map have the same spatial size and number of channels. Subsequently, the feature values ​​of the two denoising feature maps are read one by one according to the grid cell position and feature channel. The feature value at the current position in the t-th step denoising feature map is subtracted from the feature value at the same position in the (t-1)-th step denoising feature map, and the difference is written to the corresponding position. After all positions are calculated, a differential feature tensor is obtained. The differential feature tensor is then input into the ReLU activation function for element-wise calculation. Differential feature values ​​greater than 0 remain unchanged, while differential feature values ​​less than or equal to 0 are set to 0. All activated feature values ​​are arranged according to the original grid cell position and feature channel order to obtain a differential guided feature map expressing high-frequency edges and texture constraints.

[0033] Step S400: Enhance target boundary features using differential guided feature maps, and extract the bounding box coordinates and temporal trajectory points of the UAV target.

[0034] In this embodiment, the target boundary features are enhanced using differential guided feature maps. The differential guided feature maps are concatenated with the denoised feature maps of step t-1. High signal-to-interference-plus-noise ratio (SINR) modal features are used to impose structural constraints on low SINR or occluded region features. Target prediction is performed on the concatenated feature maps, and candidate 3D regions with prediction probabilities higher than the confidence threshold are selected. Based on the candidate 3D regions, 3D bounding box regression and temporal point clustering are performed to obtain and output the bounding box coordinates and temporal trajectory points of the UAV target.

[0035] Furthermore, the method provided in the application embodiments, which utilizes differential guided feature maps to enhance target boundary features and extracts the bounding box coordinates and temporal trajectory points of the UAV target, also includes: The differential guided feature map is concatenated with the denoised feature map of step t-1 to strengthen the structural constraint of high signal-to-interference-plus-noise ratio (SINR) modal features on low SINR or occluded region features; candidate 3D regions with prediction probabilities higher than the confidence threshold are extracted from the concatenated feature map; 3D bounding box regression and temporal point clustering are performed based on the candidate 3D regions to output the bounding box coordinates and temporal trajectory points of the UAV target.

[0036] In this embodiment, the differential guided feature map and the denoised feature map of step t-1 are first concatenated. Specifically, the differential guided feature map and the denoised feature map of step t-1 are kept to have the same number of grids on the X-axis, Y-axis, and Z-axis, and features are read in the same grid cell order. For any grid cell, the feature values ​​corresponding to the photoelectric image, radar point cloud, and spectrum monitoring data in the denoised feature map of step t-1 are read respectively, and then the differential feature values ​​corresponding to the same grid cell in the differential guided feature map are read. The above feature values ​​are arranged sequentially along the feature channel direction to form the concatenated feature of that grid cell. Channel concatenation means keeping the spatial position of the grid cell unchanged and only increasing the number of feature channels corresponding to the same grid cell. Subsequently, the signal-to-interference-plus-noise ratio (SNR) and occlusion rate corresponding to the grid cell in the spatial quality mask image are read. Modes with an SNR higher than a preset SNR threshold and an occlusion rate lower than a preset occlusion rate threshold are identified as high SNR modes. Modes with an SNR not higher than the preset SNR threshold or an occlusion rate reaching the preset occlusion rate threshold are identified as low SNR modes or modes corresponding to occluded regions. The preset SNR threshold and preset occlusion rate threshold are determined based on UAV target detection experimental data. Specifically, UAV target detection data under different environmental conditions are collected within a preset monitoring area. The UAV target detection results under different SNR conditions are statistically analyzed, and the lowest SNR corresponding to meeting the preset detection accuracy requirement is determined as the preset SNR threshold. Similarly, the maximum occlusion rate corresponding to meeting the preset detection accuracy requirement is statistically analyzed, and the maximum occlusion rate is determined as the preset occlusion rate threshold. For each mode, the signal-to-interference-plus-noise ratio (SNR) is normalized using the maximum and minimum SNR values ​​across all grid cells, resulting in a normalized SNR value between 0 and 1. The occlusion rate is subtracted from 1 to obtain the unoccluded ratio. The normalized SNR value is then added to the unoccluded ratio and divided by 2 to obtain the feature weight of that mode in the current grid cell. The feature values ​​of each mode in the denoised feature map of step t-1 are multiplied by their corresponding feature weights, and then concatenated with the differential feature values ​​at the same position in the differential guided feature map. Modes with high SNR receive larger feature weights, while modes with low SNR or occluded regions receive smaller feature weights. After calculating all grid cells, the concatenated features of each grid cell are rearranged according to their original X-axis, Y-axis, and Z-axis grid indices to obtain the concatenated feature map, thus completing the structural constraint of high SNR mode features on low SNR or occluded region features.

[0037] Candidate 3D regions with predicted probabilities higher than a confidence threshold are extracted from the stitched feature map. Specifically, a target prediction layer is set after the stitched feature map. The target prediction layer uses a 1×1×1 convolutional kernel to perform a weighted sum of all channel features of each grid cell and adds a bias value to obtain the target prediction value of that grid cell. The target prediction value is then input into the Sigmoid function, which calculates the prediction probability between 0 and 1 by dividing by 1 and the negative exponent of the target prediction value. The prediction probability represents the target prediction result corresponding to the current grid cell, and the confidence threshold is a pre-set judgment value between 0 and 1. The predicted probability of each grid cell is compared with the confidence threshold one by one according to the grid number on the X-axis, Y-axis, and Z-axis. If the predicted probability is higher than the confidence threshold, the grid cell is retained; if the predicted probability is not higher than the confidence threshold, the grid cell is deleted. Then, starting with the first retained grid cell, examine its six adjacent grid cells in the positive and negative X-axis, Y-axis, and Z-axis directions. Group these six adjacent grid cells together, and continue examining the six adjacent grid cells surrounding each newly grouped grid cell until no new adjacent retained grid cells remain. This group of spatially continuous grid cells is defined as a candidate 3D region. The remaining retained grid cells that have not yet been grouped are processed in the same way to obtain all candidate 3D regions. For each candidate 3D region, the minimum X-coordinate, maximum X-coordinate, minimum Y-coordinate, maximum Y-coordinate, minimum Z-coordinate, and maximum Z-coordinate of all grid cells within it are read. These six coordinates define the initial spatial boundary of the candidate 3D region. A candidate 3D region is a 3D spatial region composed of grid cells with a predicted probability higher than the confidence threshold that are interconnected in the 3D grid coordinate system.

[0038] Finally, 3D bounding box regression and temporal point clustering are performed based on the candidate 3D regions. First, an initial 3D bounding box is calculated for each candidate 3D region. The initial center point X-coordinate is equal to the sum of the maximum and minimum X-coordinates divided by 2; the initial center point Y-coordinate is equal to the sum of the maximum and minimum Y-coordinates divided by 2; the initial center point Z-coordinate is equal to the sum of the maximum and minimum Z-coordinates divided by 2; the initial length is equal to the maximum X-coordinate minus the minimum X-coordinate; the initial width is equal to the maximum Y-coordinate minus the minimum Y-coordinate; and the initial height is equal to the maximum Z-coordinate minus the minimum Z-coordinate. 3D bounding box regression refers to calculating the initial center point and corrections for the initial length, width, and height based on the characteristics of the candidate 3D region. Specifically, the stitched features corresponding to all grid cells within the candidate 3D region are averaged to obtain the region feature vector of that candidate 3D region. This region feature vector is input into the bounding box regression layer. The bounding box regression layer is a fully connected layer that outputs six values, which are used as corrections for the X-coordinate, Y-coordinate, Z-coordinate, length, width, and height, respectively. During the training phase, the center point coordinates and dimensions of the labeled UAV 3D bounding box are subtracted from the center point coordinates and dimensions of the initial 3D bounding box to obtain six corresponding annotation corrections. The sum of the absolute errors between the six corrections output by the bounding box regression layer and the six annotation corrections is used as the training loss, and the bounding box regression layer parameters are updated through backpropagation. After training, the bounding box regression layer parameters are saved. During target detection, the initial three center point coordinates are added with their corresponding coordinate corrections, and the initial length, initial width, and initial height are added with their corresponding size corrections to obtain the bounding box coordinates of the UAV target. Subsequently, the center points of each bounding box obtained at consecutive acquisition times are saved in chronological order. The X, Y, and Z coordinates of each center point and the acquisition time are combined to form a time-series point. Time-series point clustering refers to grouping time-series points belonging to the same UAV target into the same group based on spatial distance and acquisition time. Specifically, starting from the earliest acquired time point, the 3D Euclidean distance between each time point at the next acquisition time and the current time point is calculated. The 3D Euclidean distance is equal to the square root of the sum of the squares of the differences in the X, Y, and Z coordinates of the two time points. When the 3D Euclidean distance is not greater than a preset distance threshold and the acquisition time difference between the two time points is not greater than a preset time threshold, the two time points are grouped together. When multiple time points meet the conditions at the same acquisition time, the time point with the smallest 3D Euclidean distance is selected and added to the current group. The newly added time point is then used as the current time point to continue processing the next acquisition time until there are no subsequent time points that meet the distance and time thresholds. The same process is applied to ungrouped time points, and the time points in the same group are arranged from earliest to latest acquisition time to obtain the time trajectory points of the UAV target. Finally, the bounding box coordinates and time trajectory points of the UAV target are output.

[0039] Step S500: Use the bounding box coordinates of the UAV target and the time-series trajectory points as the spatial constraint interval for electromagnetic signal search, use the differential guided feature map to filter multipath interference in the ground electromagnetic signal, and calculate the ground three-dimensional coordinates of the UAV operator.

[0040] In this embodiment, the bounding box coordinates and time-series trajectory points of the UAV target are first used as the spatial constraint interval for electromagnetic signal search, and the search angle range of the electromagnetic signal's angle of arrival is limited based on the bounding box coordinates and time-series trajectory points. Next, the structural constraint weights in the differential guidance feature map are applied to the ground spectrum monitoring data to attenuate the multipath pseudo-signals generated by building reflections, resulting in a denoised electromagnetic signal. Then, based on the denoised electromagnetic signal, the position is solved by combining the time difference of arrival algorithm and the trajectory back-inference path, and the ground three-dimensional coordinates of the UAV operator are obtained and output.

[0041] Furthermore, in the method provided in the application embodiment, the bounding box coordinates and time-series trajectory points of the UAV target are used as the spatial constraint interval for electromagnetic signal search. Differential guided feature maps are used to filter multipath interference in the ground electromagnetic signal, and the ground three-dimensional coordinates of the UAV operator are calculated. The method also includes: The search angle range of the electromagnetic signal's angle of arrival is limited by the bounding box coordinates of the UAV target and the time-series trajectory points; the structural constraint weights in the differential guidance feature map are applied to the ground spectrum monitoring data to attenuate the multipath pseudo-signals generated by building reflections; the ground three-dimensional coordinates of the UAV operator are solved by combining the denoised electromagnetic signal with the time difference of arrival algorithm and the trajectory back-inference path.

[0042] In this embodiment, the bounding box coordinates of the UAV target at the current acquisition time and the time-series trajectory points formed by multiple consecutive acquisition times are first read, and the device coordinates of each ground spectrum monitoring device in the three-dimensional grid coordinate system are read. For any ground spectrum monitoring device, the eight vertices of the UAV target bounding box and the time-series trajectory points adjacent to the current acquisition time are sequentially used as angle calculation points. The X-coordinate of each angle calculation point is subtracted from the X-coordinate of the ground spectrum monitoring device to obtain the X-direction coordinate difference, the Y-coordinate is subtracted from the Y-coordinate of the ground spectrum monitoring device to obtain the Y-direction coordinate difference, and the Z-coordinate is subtracted from the Z-coordinate of the ground spectrum monitoring device to obtain the Z-direction coordinate difference. The corresponding azimuth angle is calculated based on the arctangent value of the Y-direction coordinate difference and the X-direction coordinate difference, and the corresponding pitch angle is calculated based on the arctangent value between the square root of the sum of the squares of the Z-direction coordinate difference and the X-direction coordinate difference and the Y-direction coordinate difference. After calculating all angle points, the minimum and maximum azimuth angles are selected from the obtained azimuth angles, and the minimum and maximum pitch angles are selected from the obtained pitch angles. The range between the minimum and maximum azimuth angles and the range between the minimum and maximum pitch angles are jointly determined as the angle of arrival search angle range corresponding to the ground spectrum monitoring equipment. Subsequently, the electromagnetic signals received by the ground spectrum monitoring equipment are direction-finding analyzed to obtain the azimuth angle, pitch angle, frequency, received power, and acquisition time of each electromagnetic signal component. Electromagnetic signal components whose azimuth and pitch angles are both within the angle of arrival search angle range are retained, while electromagnetic signal components that exceed the angle of arrival search angle range are deleted. Thus, the angle of arrival search angle range of the electromagnetic signals is limited by the bounding box coordinates of the UAV target and the time-series trajectory points.

[0043] Next, for each retained electromagnetic signal component, its azimuth and elevation angles are read, and the signal propagation direction is determined according to the azimuth and elevation angles, using the equipment coordinates of the corresponding ground spectrum monitoring device as the starting coordinates. Along this propagation direction, the passed grid cells are sequentially determined according to the grid size of the three-dimensional grid coordinate system, and the differential characteristic values ​​corresponding to each passed grid cell in the differential guidance feature map are read. For the case where the same grid cell has multiple characteristic channels, the differential characteristic values ​​of each characteristic channel of the grid cell are added together and divided by the number of characteristic channels to obtain the structural characteristic value of the grid cell. Then, the maximum and minimum structural characteristic values ​​are determined from the structural characteristic values ​​of all grid cells within the current search range. The minimum structural characteristic value is subtracted from the current grid cell's structural characteristic value, and then divided by the difference between the maximum and minimum structural characteristic values ​​to obtain a normalized structural characteristic value between 0 and 1. Along the propagation direction corresponding to the current electromagnetic signal component, the normalized structural characteristic values ​​of all passed grid cells are added together and divided by the number of passed grid cells to obtain the average structural characteristic value corresponding to the electromagnetic signal component. Finally, the average structural characteristic value is subtracted from 1 to obtain the structural constraint weight in the differential guidance feature map. The received power of the current electromagnetic signal component is multiplied by the structural constraint weight to obtain the weighted received power. When the propagation direction passes through the edge region of a building, the corresponding differential eigenvalue increases, the average structural eigenvalue increases accordingly, the structural constraint weight decreases, and the received power of the corresponding electromagnetic signal component is reduced. All electromagnetic signal components within the angle of arrival search angle range are processed using the same calculation process. Then, electromagnetic signal components with weighted received power lower than a preset power threshold are deleted, while those with weighted received power reaching the preset power threshold are retained, resulting in the denoised electromagnetic signal. This completes the attenuation of multipath pseudo-signals generated by building reflections in the ground spectrum monitoring data.

[0044] Finally, the denoised electromagnetic signals obtained from multiple ground-based spectrum monitoring devices were matched according to frequency and acquisition time. Signal components with a frequency difference not exceeding a preset frequency threshold and acquisition times within the same acquisition period were identified as the same electromagnetic signal, and the reception time of each electromagnetic signal arriving at each ground-based spectrum monitoring device was recorded. One ground-based spectrum monitoring device was selected as a reference device, and the reception time of the reference device was subtracted from the reception time of the other ground-based spectrum monitoring devices to obtain the corresponding arrival time difference. Each arrival time difference was multiplied by the electromagnetic wave propagation speed to obtain the distance difference between the UAV operator and the corresponding ground-based spectrum monitoring device and the reference device. Simultaneously, the time-series trajectory points of the UAV target are read from morning to night according to the acquisition time. The X-coordinate, Y-coordinate, and Z-coordinate of two adjacent time-series trajectory points are subtracted respectively to obtain the three-directional displacements at adjacent times. The average value of multiple consecutive three-directional displacements is then calculated to obtain the average motion direction of the UAV target in the X, Y, and Z axes. The average motion direction is reversed, and the spatial position is continuously calculated in the opposite direction from the earliest time-series trajectory point to form a trajectory reverse path. When the trajectory reverse path reaches the preset ground height, the corresponding X, Y, and Z coordinates are recorded as the initial coordinates for position calculation. Starting from the initial coordinates for position calculation, the three-dimensional Euclidean distance from the coordinates to each ground spectrum monitoring device is calculated. Then, the three-dimensional Euclidean distance to the reference device is subtracted from each three-dimensional Euclidean distance to obtain the calculated distance difference. The distance difference obtained from the arrival time difference is subtracted from each calculated distance difference, and the squares of the resulting differences are summed to obtain the total error. The X, Y, and Z coordinates of the initial position calculation are adjusted sequentially. After each adjustment, the total error is recalculated, and the coordinate adjustment results that reduce the total error are retained. The coordinate adjustment is continued until the difference between the total error obtained from two consecutive calculations is less than the preset error threshold. The X, Y, and Z coordinates at this time are then determined as the ground three-dimensional coordinates of the UAV operator.

[0045] Step S600: Combine the bounding box coordinates of the UAV target, the time-series trajectory points, and the ground three-dimensional coordinates of the UAV operator to match and output the prevention and control strategy command.

[0046] In this embodiment, the bounding box coordinates, time-series trajectory points, and ground three-dimensional coordinates of the UAV target are combined to determine whether the UAV target has intruded into the preset no-fly zone. When an intrusion is determined to have occurred, the ground three-dimensional coordinates of the UAV operator are sent to the ground patrol terminal, and the bounding box coordinates of the UAV target are sent to the ground electromagnetic suppression equipment or photoelectric tracking and disposal equipment. Finally, the corresponding prevention and control strategy instruction is matched from the preset disposal library according to the intrusion depth of the UAV target and output.

[0047] Furthermore, the method provided in the application embodiment, which combines the bounding box coordinates of the UAV target, the time-series trajectory points, and the ground three-dimensional coordinates of the UAV operator to match and output control strategy instructions, also includes: Determine whether the bounding box coordinates and time-series trajectory points of the UAV target have intruded into the preset anti-fly zone; if an intrusion occurs, send the ground three-dimensional coordinates of the UAV operator to the ground patrol terminal, and send the bounding box coordinates of the UAV target to the ground electromagnetic suppression equipment or photoelectric tracking and disposal equipment; match the prevention and control strategy instructions from the preset disposal database according to the intrusion depth.

[0048] Furthermore, the method provided in the application embodiments also includes: The control strategy instructions include at least one of the following: audible and visual warnings, electronic fence interception, directional spectrum interference, and forced landing and return.

[0049] In this embodiment, the pre-defined no-fly zone is pre-divided according to the aforementioned three-dimensional grid coordinate system and recorded as a set of no-fly zone grid cells. Each no-fly zone grid cell has a corresponding X-axis grid number, Y-axis grid number, and Z-axis grid number. The bounding box coordinates of the UAV target at the current moment are read, and the minimum X-coordinate, maximum X-coordinate, minimum Y-coordinate, maximum Y-coordinate, minimum Z-coordinate, and maximum Z-coordinate of the bounding box are obtained respectively. According to the aforementioned grid number calculation method, the minimum X-coordinate and maximum X-coordinate are subtracted from the X-axis origin coordinate of the three-dimensional grid coordinate system, then divided by the X-axis grid length and rounded down to determine the starting and ending grid numbers of the bounding box in the X-axis direction. The same calculation method is used to determine the starting and ending grid numbers of the Y-axis and Z-axis directions. All grid cells within the above three direction grid number ranges are determined as the grid cells covered by the UAV target bounding box. Subsequently, the X, Y, and Z coordinates of the time-series trajectory points are read sequentially, and the corresponding grid numbers are calculated according to the grid sizes of the X, Y, and Z axes to determine the grid cell containing each time-series trajectory point. The grid cells covered by the UAV target bounding box and the grid cells containing each time-series trajectory point are compared one by one with the no-fly zone grid cell set. When the X, Y, and Z axis grid numbers of at least one of the grid cells match the grid numbers in the no-fly zone grid cell set, it is determined that the UAV target has intruded. Further, boundary grid cells are determined in the set of no-fly zone grid cells. Among them, the no-fly zone grid cells that are directly adjacent to the grid cells outside the no-fly zone in the X-axis, Y-axis or Z-axis direction are called boundary grid cells. The X coordinate of the center point of the current UAV target bounding box is calculated as the sum of the minimum X coordinate and the maximum X coordinate divided by 2, the Y coordinate is calculated as the sum of the minimum Y coordinate and the maximum Y coordinate divided by 2, and the Z coordinate is calculated as the sum of the minimum Z coordinate and the maximum Z coordinate divided by 2. Then, the three-dimensional distance between the center point and the center point of each boundary grid cell is calculated. The three-dimensional distance is equal to the square root of the sum of the squares of the differences in the X coordinates, the Y coordinates and the Z coordinates of the two points. The minimum value among all three-dimensional distances is taken as the intrusion depth of the current UAV target.

[0050] Upon determining that an intrusion has occurred, the aforementioned ground-based 3D coordinates of the drone operator are read. The X, Y, and Z coordinates, along with the current drone target number and coordinate generation time, are sequentially written into the positioning data field, and the positioning data is then sent to the ground patrol terminal. The bounding box coordinates of the current drone target are read, and the minimum X, maximum X, minimum Y, maximum Y, minimum Z, and maximum Z coordinates, along with the drone target number and bounding box generation time, are sequentially written into the target data field. The target data is then sent to ground-based electromagnetic suppression equipment or photoelectric tracking and handling equipment. When determining the current spatial center position of the UAV target, the center X coordinate is obtained by dividing the sum of the minimum and maximum X coordinates by 2, the center Y coordinate is obtained by dividing the sum of the minimum and maximum Y coordinates by 2, and the center Z coordinate is obtained by dividing the sum of the minimum and maximum Z coordinates by 2. When the coordinates of the ground electromagnetic suppression equipment or photoelectric tracking and disposal equipment are known, the X-direction coordinate difference is obtained by subtracting the equipment X coordinate from the center X coordinate of the UAV target, the Y-direction coordinate difference is obtained by subtracting the equipment Y coordinate from the center Y coordinate of the UAV target, and the Z-direction coordinate difference is obtained by subtracting the equipment Z coordinate from the center Z coordinate of the UAV target. The arctangent value is calculated using the Y-direction coordinate difference and the X-direction coordinate difference to obtain the target azimuth angle, and the arctangent value is calculated using the square root of the sum of the squares of the Z-direction coordinate difference, the X-direction coordinate difference, and the Y-direction coordinate difference to obtain the target pitch angle. The calculated azimuth angle, pitch angle, and bounding box coordinates are then written into the target data of the corresponding disposal equipment.

[0051] Finally, prevention and control strategy instructions are matched from a pre-set response library based on the intrusion depth. Multiple consecutive intrusion depth intervals are pre-defined in the pre-set response library according to intrusion depth. Each intrusion depth interval records the lower limit, upper limit, and corresponding prevention and control strategy instructions, arranged in ascending order of the lower limit. The currently calculated intrusion depth is read and compared sequentially with the lower and upper limits of each intrusion depth interval. When the intrusion depth is greater than or equal to the lower limit of an intrusion depth interval but less than its upper limit, the corresponding prevention and control strategy instruction for that interval is read. When the intrusion depth is greater than or equal to the lower limit of the highest intrusion depth interval, the prevention and control strategy instruction corresponding to that highest intrusion depth interval is read. Prevention and control strategy instructions include at least one of audible and visual warnings, electronic fence interception, directional spectrum interference, and forced landing / return. Corresponding instruction numbers are assigned to each of these in the pre-set response library. When one intrusion depth interval corresponds to one prevention and control strategy instruction, one instruction number is read; when one intrusion depth interval corresponds to multiple prevention and control strategy instructions, multiple instruction numbers are read sequentially. The matched command number, UAV target number, UAV target bounding box coordinates, and current intrusion depth are combined to form the prevention and control strategy command data, and the corresponding audible and visual warnings, electronic fence interception, directional spectrum interference, and forced landing and return commands are issued according to the matched command number.

[0052] In summary, the embodiments of this application have at least the following technical effects: This application acquires multimodal sensing data, aligns the multimodal sensing data to a preset three-dimensional grid coordinate system, calculates the signal-to-interference-plus-noise ratio (SIR) and occlusion rate of each grid cell under different modes, and generates a spatial quality mask map. Based on the SIR and occlusion rate in the spatial quality mask map, an independent noise time step is allocated to each grid cell, constructing a spatial-modal asynchronous time step matrix. According to the scheduling constraints of the spatial-modal asynchronous time step matrix, the feature maps of each mode are jointly denoised and reconstructed, and the differential feature tensor between adjacent time step feature maps is extracted to generate a differential guided feature map. The differential guided feature map is used to enhance the target boundary features, and the bounding box coordinates and time-series trajectory points of the UAV target are extracted. The bounding box coordinates and time-series trajectory points of the UAV target are used as the spatial constraint interval for electromagnetic signal search. The differential guided feature map is used to filter multipath interference in the ground electromagnetic signal, and the ground three-dimensional coordinates of the UAV operator are calculated. Combining the bounding box coordinates, time-series trajectory points, and ground three-dimensional coordinates of the UAV operator, a control strategy command is matched and output. This invention addresses the technical problem of low accuracy in target detection and positioning of UAVs in complex environments in existing technologies. By performing joint denoising reconstruction and differential feature enhancement based on multimodal perception data, combined with target trajectory constraint electromagnetic signal search and multipath interference filtering, the technical effect of improving the positioning accuracy of UAV targets and their operators is achieved.

[0053] Example 2, based on the same inventive concept as the UAV target detection and localization method based on multimodal cooperative perception in the previous examples, such as... Figure 3 As shown, this application provides a UAV target detection and localization system based on multimodal cooperative perception. The system and method embodiments in this application are based on the same inventive concept. The system includes: The data acquisition module 11 is used to acquire multimodal sensing data, align the multimodal sensing data to a preset three-dimensional grid coordinate system, calculate the signal-to-interference-plus-noise ratio (SIR) and occlusion rate of each grid cell under different modes, and generate a spatial quality mask map. The matrix construction module 12 is used to allocate independent noise time steps to each grid cell according to the SIR and occlusion rate in the spatial quality mask map, and construct a spatial-modal asynchronous time step matrix. The denoising and reconstruction module 13 is used to perform joint denoising and reconstruction of the feature maps of each modality according to the scheduling constraints of the spatial-modal asynchronous time step matrix, and extract the difference between the feature maps of adjacent time steps. The module generates a differential guided feature map by dividing the feature tensor; the extraction module 14 is used to enhance the target boundary features using the differential guided feature map and extract the bounding box coordinates and time-series trajectory points of the UAV target; the calculation module 15 is used to use the bounding box coordinates and time-series trajectory points of the UAV target as the spatial constraint interval for electromagnetic signal search, and uses the differential guided feature map to filter multipath interference in the ground electromagnetic signal to calculate the ground three-dimensional coordinates of the UAV operator; the command matching module 16 is used to combine the bounding box coordinates, time-series trajectory points and ground three-dimensional coordinates of the UAV target to match and output the prevention and control strategy command.

[0054] Furthermore, the system is also used to implement the following functions: The multimodal sensing data includes integrated sensing data, photoelectric images, radar point clouds, and spectrum monitoring data.

[0055] Furthermore, the system is also used to implement the following functions: The multimodal sensing data is mapped to the three-dimensional grid coordinate system; for each grid cell, the pixel occlusion rate of the photoelectric image, the point cloud density of the radar point cloud, and the signal-to-interference-plus-noise ratio of the spectrum monitoring data are extracted respectively; the pixel occlusion rate, point cloud density, and signal-to-interference-plus-noise ratio are normalized and weighted and fused to obtain a spatial quality mask map that expresses the data quality assessment value of the grid cell.

[0056] Furthermore, the system is also used to implement the following functions: Set high-noise time step thresholds and low-noise time step thresholds; when the spatial quality mask indicates that the data quality of a certain grid cell is lower than the quality threshold or there is occlusion, allocate a high-noise time step close to 1 to the feature channel of that grid cell; when the spatial quality mask indicates that the data quality of a certain grid cell is higher than the quality threshold, allocate a low-noise time step close to 0 to the feature channel of that grid cell; combine the time steps of each channel according to the grid cell position to form a spatial-modal asynchronous time step matrix that includes spatial position and modal dimension.

[0057] Furthermore, the system is also used to implement the following functions: The spatial-modal asynchronous time step matrix is ​​input into the denoising network to control the sampling step size of different grid cells during the denoising iteration process. During the denoising iteration process, the denoising feature map of step t and the denoising feature map of step (t-1) are extracted. The denoising feature map of step t and the denoising feature map of step (t-1) are subtracted element-wise to obtain the differential feature tensor. After being mapped by the activation function, a differential guided feature map expressing high-frequency edges and texture constraints is generated.

[0058] Furthermore, the system is also used to implement the following functions: The differential guided feature map is concatenated with the denoised feature map of step t-1 to strengthen the structural constraint of high signal-to-interference-plus-noise ratio (SINR) modal features on low SINR or occluded region features; candidate 3D regions with prediction probabilities higher than the confidence threshold are extracted from the concatenated feature map; 3D bounding box regression and temporal point clustering are performed based on the candidate 3D regions to output the bounding box coordinates and temporal trajectory points of the UAV target.

[0059] Furthermore, the system is also used to implement the following functions: The search angle range of the electromagnetic signal's angle of arrival is limited by the bounding box coordinates of the UAV target and the time-series trajectory points; the structural constraint weights in the differential guidance feature map are applied to the ground spectrum monitoring data to attenuate the multipath pseudo-signals generated by building reflections; the ground three-dimensional coordinates of the UAV operator are solved by combining the denoised electromagnetic signal with the time difference of arrival algorithm and the trajectory back-inference path.

[0060] Furthermore, the system is also used to implement the following functions: Determine whether the bounding box coordinates and time-series trajectory points of the UAV target have intruded into the preset anti-fly zone; if an intrusion occurs, send the ground three-dimensional coordinates of the UAV operator to the ground patrol terminal, and send the bounding box coordinates of the UAV target to the ground electromagnetic suppression equipment or photoelectric tracking and disposal equipment; match the prevention and control strategy instructions from the preset disposal database according to the intrusion depth.

[0061] Furthermore, the system is also used to implement the following functions: The control strategy instructions include at least one of the following: audible and visual warnings, electronic fence interception, directional spectrum interference, and forced landing and return.

[0062] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for UAV target detection and localization based on multimodal cooperative perception, characterized in that, The method includes: Acquire multimodal sensing data, align the multimodal sensing data to a preset three-dimensional grid coordinate system, calculate the signal-to-interference-plus-noise ratio and occlusion rate of each grid cell under different modes, and generate a spatial quality mask map; Based on the signal-to-interference-plus-noise ratio and occlusion rate in the spatial quality mask image, an independent noise time step is assigned to each grid cell to construct a spatial-modal asynchronous time step matrix. Based on the scheduling constraints of the spatial-modal asynchronous time step matrix, the feature maps of each modality are jointly denoised and reconstructed, and the differential feature tensors between adjacent time step feature maps are extracted to generate differential guided feature maps. The target boundary features are enhanced by using differential guided feature maps to extract the bounding box coordinates and temporal trajectory points of the UAV target; The bounding box coordinates and time-series trajectory points of the UAV target are used as the spatial constraint interval for electromagnetic signal search. The differential guided feature map is used to filter multipath interference in the ground electromagnetic signal and calculate the ground three-dimensional coordinates of the UAV operator. By combining the bounding box coordinates of the UAV target, the time-series trajectory points, and the ground three-dimensional coordinates of the UAV operator, a control strategy command is matched and output.

2. The UAV target detection and localization method based on multimodal cooperative perception as described in claim 1, characterized in that, The multimodal sensing data includes integrated sensing data, photoelectric images, radar point clouds, and spectrum monitoring data.

3. The UAV target detection and localization method based on multimodal cooperative perception as described in claim 2, characterized in that, Acquire multimodal sensing data, align the multimodal sensing data to a preset three-dimensional grid coordinate system, calculate the signal-to-interference-plus-noise ratio and occlusion rate of each grid cell under different modes, and generate a spatial quality mask map, including: Map the multimodal sensing data to the three-dimensional mesh coordinate system; For each grid cell, the pixel occlusion rate of the photoelectric image, the point cloud density of the radar point cloud, and the signal-to-interference-plus-noise ratio of the spectrum monitoring data are extracted respectively. The pixel occlusion rate, point cloud density, and signal-to-interference-plus-noise ratio are normalized and weighted to obtain a spatial quality mask that expresses the data quality evaluation value of the grid cell.

4. The UAV target detection and localization method based on multimodal cooperative perception as described in claim 1, characterized in that, Based on the signal-to-interference-plus-noise ratio and occlusion rate in the spatial quality mask image, an independent noise time step is assigned to each grid cell, and a spatial-modal asynchronous time step matrix is ​​constructed, including: Set the high noise time step threshold and the low noise time step threshold; When the spatial quality mask indicates that the data quality of a certain grid cell is lower than the quality threshold or that there is occlusion, a high-noise time step close to 1 is assigned to the feature channel of that grid cell. When the spatial quality mask indicates that the data quality of a certain grid cell is higher than the quality threshold, a low-noise time step close to 0 is allocated to the feature channel of that grid cell; The time steps of each channel are combined according to the grid cell position to form a spatial-modal asynchronous time step matrix that includes spatial position and modal dimension.

5. The UAV target detection and localization method based on multimodal cooperative perception as described in claim 1, characterized in that, Based on the scheduling constraints of the spatial-modal asynchronous time-step matrix, joint denoising and reconstruction are performed on the feature maps of each modality, and the differential feature tensors between adjacent time-step feature maps are extracted to generate differential guided feature maps, including: The spatial-modal asynchronous time step matrix is ​​input into the denoising network to control the sampling step size of different grid cells during the denoising iteration process; During the denoising iteration process, the denoised feature map at step t and the denoised feature map at step (t-1) are extracted. The denoised feature map of step t is subtracted element-wise from the denoised feature map of step t-1 to obtain the differential feature tensor. After being mapped by the activation function, a differential guided feature map expressing high-frequency edges and texture constraints is generated.

6. The UAV target detection and localization method based on multimodal cooperative perception as described in claim 5, characterized in that, The target boundary features are enhanced using differential guided feature maps, and the bounding box coordinates and temporal trajectory points of the UAV target are extracted, including: The differential guided feature map is channel-stitched with the denoised feature map of step t-1 to strengthen the structural constraint of high signal-to-interference-plus-noise ratio modal features on low signal-to-interference-plus-noise ratio or occluded region features. Extract candidate 3D regions with predicted probabilities higher than the confidence threshold from the stitched feature map; Based on the candidate 3D region, 3D bounding box regression and temporal point clustering are performed to output the bounding box coordinates and temporal trajectory points of the UAV target.

7. The UAV target detection and localization method based on multimodal cooperative perception as described in claim 1, characterized in that, Using the bounding box coordinates and time-series trajectory points of the UAV target as the spatial constraint interval for electromagnetic signal search, and employing differential guided feature maps to filter multipath interference in ground electromagnetic signals, the ground three-dimensional coordinates of the UAV operator are calculated, including: The search angle range of the electromagnetic signal's angle of arrival is limited by the bounding box coordinates of the UAV target and the time-series trajectory points; The structural constraint weights in the differential guided feature map are applied to the ground spectrum monitoring data to attenuate the multipath pseudo-signals generated by building reflections. By combining the denoised electromagnetic signal with the time difference of arrival algorithm and the trajectory back-deduction path, the ground three-dimensional coordinates of the UAV operator are solved.

8. The UAV target detection and localization method based on multimodal cooperative perception as described in claim 1, characterized in that, By combining the bounding box coordinates of the UAV target, the time-series trajectory points, and the ground 3D coordinates of the UAV operator, a control strategy command is matched and output, including: Determine whether the bounding box coordinates and time-series trajectory points of the drone target have intruded into the preset no-fly zone; If an intrusion occurs, the ground three-dimensional coordinates of the drone operator will be sent to the ground patrol terminal, and the bounding box coordinates of the drone target will be sent to the ground electromagnetic suppression equipment or photoelectric tracking and disposal equipment. Match prevention and control strategy instructions from the preset handling library based on the intrusion depth.

9. The UAV target detection and localization method based on multimodal cooperative perception as described in claim 8, characterized in that, The control strategy instructions include at least one of the following: audible and visual warnings, electronic fence interception, directional spectrum interference, and forced landing and return.

10. A UAV target detection and localization system based on multimodal cooperative perception, characterized in that, The system is used to execute the UAV target detection and localization method based on multimodal cooperative perception as described in any one of claims 1-9, and the system includes: The data acquisition module is used to acquire multimodal sensing data, align the multimodal sensing data to a preset three-dimensional grid coordinate system, calculate the signal-to-interference-plus-noise ratio and occlusion rate of each grid cell under different modes, and generate a spatial quality mask map. The matrix construction module is used to allocate independent noise time steps to each grid cell based on the signal-to-interference-plus-noise ratio and occlusion rate in the spatial quality mask image, and construct a spatial-modal asynchronous time step matrix. The denoising and reconstruction module is used to perform joint denoising and reconstruction of the feature maps of each mode according to the scheduling constraints of the spatial-modal asynchronous time step matrix, and extract the differential feature tensor between adjacent time step feature maps to generate differential guided feature maps. The extraction module is used to enhance the target boundary features using differential guided feature maps and extract the bounding box coordinates and temporal trajectory points of the UAV target. The calculation module is used to use the bounding box coordinates and time-series trajectory points of the UAV target as the spatial constraint interval for electromagnetic signal search, and to use differential guided feature maps to filter multipath interference in ground electromagnetic signals and calculate the ground three-dimensional coordinates of the UAV operator. The instruction matching module is used to combine the bounding box coordinates of the UAV target, the time-series trajectory points, and the ground three-dimensional coordinates of the UAV operator to match and output the prevention and control strategy instructions.

Images