A method for detecting unmanned aerial vehicles under visible light and infrared images

Abstract

The application discloses a kind of unmanned aerial vehicle detection methods under visible light and infrared image, comprising: obtaining the weight of csp-block based on M-DyConv by multimodal unmanned aerial vehicle detection and training method;Image acquisition is carried out on target area, and visible light and infrared image are obtained;The visible light and infrared image are input into the csp-block weight, and the corresponding feature map of different scales is generated;The feature map of different scales is input into multi-scale feature dynamic fusion encoder MFDF and is fused, and fusion feature map is generated;The fusion feature map is input into prediction head and is processed, and prediction information is obtained, and the prediction information includes the position and size of unmanned aerial vehicle.The application improves the detection accuracy of unmanned aerial vehicle under different modalities.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a method for detecting drones under visible light and infrared images. Background Technology

[0002] Unmanned aerial vehicles (UAVs) can be remotely operated or equipped with mission programs without a human pilot. In recent years, UAV technology has become extremely popular due to its diverse applications in information and communication technologies. Furthermore, UAVs have become a key entity in scientific research. UAVs have existed since the early 20th century, primarily deployed in hazardous environments due to their small size, ease of operation, and ability to replace human presence. More recently, small UAVs produced by commercial companies are ubiquitous in both commercial and civilian environments. UAVs are currently used in various applications such as detection, search and rescue operations, traffic and weather monitoring, aerial photography, precision agriculture, and more. The emergence of small UAVs has reduced production costs, providing more affordable equipment for the general public. With low-cost development kits, small UAVs weighing less than 20 kg can be easily purchased or assembled online, leading to their rapid proliferation. However, the widespread availability of UAVs brings new opportunities and challenges to the market and scientific development. While UAVs offer advantages such as aerial photography or mapping, they also raise security and privacy concerns. Moreover, the lack of unified industry standards and regulations has resulted in many UAVs operating without oversight, raising significant concerns for air traffic control and posing a potential threat to critical infrastructure. Effective detection of drone targets can help prevent potential hazards and protect society and people's privacy. Therefore, drone detection is crucial. Despite its essential nature, various challenges exist in effectively detecting drones. Currently, there are two main methods for detecting small drones: radar detection and visual detection. Due to differences in drone materials, size, shape, and maximum flight speed, they exhibit different radar scattering and Doppler characteristics, making radar detection inefficient. On the other hand, visual detection is a more reliable and economical option than radar detection. However, visual detection based on a single visible light modality may be insufficient in some extreme environments, such as darkness or heavy fog. Therefore, switching between visible light and infrared modes has become the mainstream visual detection solution for drones. Infrared cameras detect the heat emitted by objects, allowing them to "see" in complete darkness, something visible light cameras cannot do. Furthermore, the images they create are unaffected by lighting conditions (such as shadows, glare, or backlight), making them suitable for monitoring challenging environments. However, due to the significant differences between infrared and visible light modes, most single-modal drone detection algorithms are not effectively adapted to current surveillance equipment. Image-based target detection is a key task in computer vision. Mainstream visual object detection algorithms can be divided into two categories. The first is single-stage object detection algorithms. These algorithms do not rely on candidate frames, directly predicting category and location data to produce the final result in a single detection. The second category is two-stage object detection algorithms. These algorithms generate candidate frames in the first stage, and then classify and regress the target in the second stage. While these algorithms have higher accuracy, their detection speed is faster and cannot meet the real-time requirements of UAV detection. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes a method for detecting drones under visible light and infrared images. The method dynamically adjusts the convolution kernel parameters based on the substantial differences between RGB and TIR modes, maximizing the utilization of effective information in the image and improving the detection accuracy of drones under different modes.

[0004] To achieve the above objectives, the present invention provides a method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images, comprising: Obtain an RGB-T target detection dataset of visible light and infrared images, pre-train the target detection dataset, and obtain a TIR image dataset; Based on the TIR image dataset, csp-block weights based on M-DyConv are obtained through a multimodal UAV detection and training method. Image acquisition is performed on the target area to obtain visible light and infrared images; The visible light and infrared images are input into the csp-block weights to generate corresponding feature maps at different scales. The feature maps at different scales are input into the Multi-Scale Feature Dynamic Fusion Encoder (MFDF) for fusion to generate a fused feature map. The fused feature map is input into the prediction head for processing to obtain prediction information, which includes the position and size of the UAV.

[0005] Optionally, methods for obtaining M-DyConv-based csp-block weights through multimodal UAV detection pre-training include: matching the generated infrared dataset with labels, each image should have a corresponding annotation file, the annotation file should contain the target category and bounding box coordinates; inputting the dataset into the model to detect targets, and adjusting the algorithm parameters according to the labels using gradient descent.

[0006] Optionally, methods for acquiring visible light and infrared images of the target area include: determining the acquisition range and selecting visible light and infrared sensor devices for installation; starting the devices and then acquiring visible light and infrared images.

[0007] Optionally, the method of inputting the visible light and infrared images into the csp-block weights to generate corresponding feature maps of different scales includes: loading pre-trained weights, using the visible light or infrared image as input, and propagating it forward through the model; during this process, the image will pass through the csp-block weights, and feature maps of different scales will be obtained by extracting the output of the csp-block.

[0008] Optionally, the method of inputting the feature maps of different scales into the multi-scale feature dynamic fusion encoder (MFDF) for fusion to generate the fused feature map includes: the operation of the multi-scale feature dynamic fusion encoder can be expressed by the following formula: In the formula, Concat represents the concatenation of two feature tensors in the channel dimension. and Let X represent the shallow and deep features of the backbone network, respectively. Then, the input features X are processed by a repetitive encoder, where each encoder consists of two residual sub-blocks. The first sub-block is a key component containing a multi-size dynamic convolutional module, which can be represented as: Where Norm(·) represents layer normalization; The first sub-block represents a multi-scale dynamic convolution module; the second sub-block mainly consists of a two-layer MLP with non-linear activation. Where W1∈RC×rC, W2∈RrC×C are the learnable parameters of the MLP established based on the expansion ratio r, and σ(·) represents the GELU activation function; the GELU activation function can be expressed as: In the formula, Φ(x) is the cumulative function of the Gaussian normal distribution of x, and µ and σ are the mean and standard deviation of the normal distribution. Since the above formula cannot be calculated directly, the GELU function is expressed as: In the formula, σ(·) is the sigmoid function.

[0009] Optionally, the method for pre-training the object detection dataset includes: converting an RGB image to a thermal image involves assigning different grayscale values ​​to objects of different categories in a given image; utilizing existing RGB datasets to provide supplementary annotation information, including location and category details, to enhance the efficiency of image conversion; and combining unique thermal imaging features with annotated RGB datasets to convert RGB images to TIR images.

[0010] Optionally, the working method of the multi-scale feature dynamic fusion encoder is as follows: In the formula, Concat represents the concatenation of two feature tensors in the channel dimension. and These represent the shallow and deep features of the backbone network, respectively.

[0011] Optionally, the process of converting the RGB image into a thermal image specifically includes the following steps: The values ​​of the R, G, and B channels at the same pixel location are averaged and then mapped to grayscale values, thus converting the RGB image into a grayscale image. Use smaller hyperparameters to multiply the background image to distinguish the target from the background in the TIR image.

[0012] Optionally, the RGB image is converted to a TIR image, calculated as follows: Where I(x,y) represents the pixel value of the generated pre-trained image, and x and y represent the positions, I R I G I B This represents the red, green, and blue channels in the original image.

[0013] Technical effects of this invention: This invention discloses a method for detecting drones under visible light and infrared images. It dynamically adjusts the convolution kernel parameters according to the substantial differences between RGB and TIR modes, maximizing the utilization of effective information in the image and improving the detection accuracy of drones under different modes. It enhances the focus on target features by using multi-scale dynamic convolution, thereby improving the utilization rate of backbone network feature extraction. It improves the accuracy of the model by compensating for the lack of infrared image datasets and the overfitting problem related to low data volume. Attached Figure Description

[0014] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a diagram illustrating the overall architecture of the drone detection technology according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the multimodal dynamic convolutional layer structure according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the adaptive backbone network module structure according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the multi-scale feature fusion module in an embodiment of the present invention; Figure 5 This is a heatmap showing the degree of attention paid to UAV targets by the multi-scale feature fusion module in an embodiment of the present invention. Figure 6 This is a diagram illustrating the effect of the pre-training method in an embodiment of the present invention. Figure 7 These are effect diagrams from an embodiment of the present invention; Figure 8 This is a challenge graph showing experimental data from an embodiment of the present invention; Figure 9This is a visualization of the mathematical analysis of experimental data from an embodiment of the present invention. Detailed Implementation

[0015] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0017] This invention provides a novel real-time UAV detection model (RTM-UAVDet). For the macro-architecture of RTM-UAVDet, a typical one-stage object detector is employed, such as... Figure 1 As shown, a backbone network is constructed using a csp-block with multimodal dynamic convolutional layers. Multi-level features C3 and C4 are extracted from the backbone and then fused in an MFDF encoder. A detection head with shared convolutional weights and separate batch normalization (BN) layers is then used to predict the classification and regression results for bounding box detection. This invention integrates a multimodal dynamic convolutional module (M-DyConv module) into the backbone network to discover more effective feature information. Furthermore, to effectively fuse high-resolution detail texture information from the actual underlying layers and abstract features from different modes, a global attention mechanism is proposed to deeply fuse features at different levels. The fused feature information is then input into the detection layer to improve the prediction capability of RTM-UAVDet for UAVs. Through experimental comparison, the backbone network was pruned, reducing the overall network depth and improving the real-time detection efficiency of RTM-UAVDet. Based on the above system, this invention also provides a method for UAV detection under visible light and infrared images, specifically including the following steps: S0. The backbone network can learn representations at different levels, commonly referred to as shallow and deep features. Specifically, shallow features are low-level visual elements extracted from the early layers of the backbone, such as edges and corners. In contrast, deep features are high-level semantic features extracted from deeper layers of the network, such as shape and texture. This concept is fundamental to computer vision because it allows for the identification of complex patterns in images through hierarchical feature extraction. By combining deep and shallow features using an FPN network to construct a multi-scale feature pyramid, deep and shallow features can be fused together. Traditional methods only use simple concatenation followed by single-scale convolutional kernels for feature fusion, ignoring the different contributions of shallow and deep features to detecting targets in different modalities of images and failing to consider the relationships between different features. However, the practical information contained in the features can make a significant contribution to UAV detection. Therefore, it is necessary to dynamically adjust the contribution of features in the image, focusing more on valuable feature information. For UAV detection, both shallow and deep features extracted by the backbone network are important. By combining deep and shallow features using an FPN network to construct a multi-scale feature pyramid, deep and shallow features can be fused together.

[0018] S1. To meet the needs of UAV detection under different modes, this invention proposes a multi-mode UAV detection algorithm based on dynamic convolution mechanism, namely the RTM-UAVDet algorithm. The network structure of this algorithm is as follows: Figure 2 As shown, the model first inputs the feature maps into an average pooling layer, and then outputs two tensors with shapes of 1×1×(Kw×Kh) and 1×1×Kc respectively through two MLP modules. These two tensors then adjust the convolution kernels, a process that can be represented as: In the above formula, H and W represent the height and width of the input feature map, and i and j represent the coordinates of each feature within each channel. Represents a fully connected neuron. The final attention parameters are defined by these parameters. Global average pooling aims to reduce computation, improve real-time efficiency, and maximize the receptive field of the resulting tensor. The obtained global information is then increased in dimensionality according to the proportion specified by the hyperparameters, resulting in a feature vector with global information spanning multiple spatial dimensions. These learned parameters are then transformed into channel attention for the convolutional kernel, as shown in the following equation.

[0019] W1 and W2 are two learnable parameters that transform the original detailed texture features into high-level abstract features, with sizes of c × (c × r) and (c × r) × c, respectively. W1 adds global information in terms of dimension, while W2 converts high-dimensional information into channel concerns of the convolutional kernel. The parameter c is used to expand the kernel-focused features from a 1×1×c convolutional kernel to a KH×KW×KC format for simple dot product operations with the convolutional kernel, as shown in the following equation.

[0020] W1 represents the size of the learnable parameter responsible for increasing the dimensionality of the global information, with a dimension of c × (c × r). Similarly, W2 represents the size of the parameter responsible for transforming high-dimensional information into a focus on the spatial dimension of the convolution kernel, with a dimension of (c × r) × (kw × kh). Finally, the kernel-focused features obtained from the 1 × 1 × c convolution kernel are reconstructed into a KH × KW format using the parameter Uc, facilitating adaptive adjustment of the spatial dimension parameters of the convolution kernel. The process is shown in the following equation.

[0021] S2. A multi-scale feature dynamic fusion encoder for UAV detection is proposed, solving the problem of deep and shallow feature fusion. The encoder is as follows: Figure 3 As shown, it is based on dynamic convolution and consists of three dynamic convolution kernels of different sizes: 1 × 1, 3 × 3, and 5 × 5. By implementing a 1 × 1 convolution kernel, a linear combination of input channels can be created, where each output channel is computed as a weighted sum of its input values. The 1 × 1 convolution kernel, learned during network training, helps balance the increase in exponential parameters caused by using larger kernels. The main function of the 3 × 3 convolution kernel is to detect small-scale features, such as edges, textures, shapes, or other patterns. The 5 × 5 convolution kernel can detect more important drone features in the input image. These features can include more complex drone objects or regions. The multi-scale feature dynamic fusion encoder optimizes the weight distribution of feature maps at different scales, enabling the encoder to directly filter spatial features from other layers, proportionally weighting different feature layers and retaining only relevant information. The operation of the multi-scale feature dynamic fusion encoder can be represented by the following formula: In the formula, Concat represents the concatenation of two feature tensors in the channel dimension. and Let X represent the shallow and deep features of the backbone network, respectively. Then, the input features X are processed by a repetitive encoder, where each encoder consists of two residual sub-blocks. The first sub-block is a key component containing a multi-size dynamic convolutional module, which can be represented as: Norm(·) represents layer normalization. This represents a multi-scale dynamic convolution module. The second sub-block mainly consists of a two-layer MLP with non-linear activation.

[0022] Where W1∈RC×rC, W2∈RrC×C are the learnable parameters of the MLP established based on the expansion ratio r, and σ(·) represents the GELU activation function. The GELU activation function can be expressed as: In the formula, Φ(x) is the cumulative function of the Gaussian normal distribution of x, and µ and σ are the mean and standard deviation of the normal distribution. Since the above formula cannot be calculated directly, the GELU function can be approximated as: In the formula, σ(·) is the sigmoid function. When At first, the GELU output is essentially linear, similar to ReLU. When x approaches 0, GELU exhibits a non-linear output and has a gradient during backpropagation to prevent neuron death. For example... Figure 5 As shown in the heatmap, the multi-scale feature dynamic fusion encoder effectively enhances the overall algorithm's focus on the UAV.

[0023] S3. A pre-training method for multi-modal drone detection is proposed. Pre-trained models are crucial for drone detection algorithms because they provide pre-learned features and patterns that can be used as a starting point for drone identification. These pre-trained models have been trained on large, diverse datasets to identify a wide variety of objects, thus improving the performance of drone detection algorithms without having to start from scratch. Pre-trained models also save significant time and computational resources that would otherwise be required to design and train models from scratch. By leveraging the knowledge and insights gained from previous training, drone detection algorithms can improve performance and learn more specific features of drones based on the features of the pre-trained models. This invention proposes a pre-training method for generating RGB-T object detection datasets. Thermal imaging is a technique that records the amount of radiation emitted by an object and generates different grayscale values ​​for its thermal properties. Therefore, converting RGB images to thermal images involves assigning different grayscale values ​​to different categories of objects in a given image. Furthermore, utilizing existing RGB datasets to provide supplementary annotation information, including location and category details, can potentially enhance the efficiency of image conversion. An innovative semantic segmentation-based method is proposed that combines unique thermal imaging features with annotated RGB datasets to convert RGB images into TIR images. The process of generating a thermal image from an RGB image can be divided into the following steps: S3.1. Take the average value of the R, G, and B channels at the same pixel location, and then map it to grayscale value to convert the RGB image to a grayscale image, as shown in the following formula: To obtain the mask label for each object in the image, two corresponding masks of the same size as the original image were used: one to isolate the background and the other to isolate the foreground.

[0024] S3.2. Utilize larger hyperparameters to augment the foreground mask image, as objects in the foreground typically radiate more heat than the background. In contrast, use smaller hyperparameters to multiply the background image, since the thermal intensity of the background is generally lower and appears black in thermal infrared (TIR) ​​images. This strategy makes it easier to distinguish between targets and background in TIR images during model training. Figure 6 The created simulated infrared image is displayed.

[0025] S4. Using the anti-drone dataset as the experimental dataset, verify the detection performance of RTMUAVDet. The anti-drone dataset comes from the 3rd Anti-Drone Workshop Challenge, which provides many challenging multimodal drone detection video sequences, including various backgrounds such as clouds, buildings, mountains, and oceans, as well as challenging scenarios such as fast movement and scale changes. Figure 7This displays some example images from the dataset. Dataset analysis visualizations include... Figure 8 As shown. This experiment used 582,345 labeled images. Many scenes in this dataset are quite challenging for UAV detection.

[0026] S5. To improve the robustness of RTM-UAVDet, various data augmentation methods were used on the dataset images, such as rotation, mirroring, mosaic, RandAffine, and Mixup. The input image size was set to 640×640. A stochastic gradient descent (SGD) optimizer was used during training, and a warm-up learning rate method was implemented. By gradually increasing the learning rate from a low value, the aim was to avoid gradient divergence problems that might occur when the initial learning rate is too high. This method helps prevent sudden updates to model parameters, which could compromise the stability of the optimization process.

[0027] Example 1 RTM-UAVDet achieves a state-of-the-art accuracy of 66.1% AP0.5:0.95 on the Anti-UAV dataset. Furthermore, RTM-UAVDet outperforms other methods and flop in terms of parameters, making it well-suited for deployment in mobile applications. RTM-UAVDet meets real-time requirements, achieving a maximum inference speed of 72.4 FPS. These results demonstrate that RTM-UAVDet outperforms classic algorithms in multimodal UAV detection, achieving high accuracy while meeting real-time requirements.

[0028] Example 2 A series of ablation experiments were designed on an anti-drone dataset to verify the effectiveness of the proposed method in RTM-UAVDet. As shown in Table 1, YOLO v5 was used as the baseline model. Replacing the MDyConvNet-based backbone network of the baseline model improved the network's AP0.5:0.95. Replacing the FPN with the MFDF encoder significantly improved the detection accuracy with AP0.5:0.95, indicating that feature fusion using attention blocks can improve detection accuracy. Based on this, applying the multimodal pre-training method to the baseline model increased the AP0.5:0.95 from 55.9% to 59.2%. Combining these three methods to implement the RTM-UAVDet algorithm yielded excellent drone detection performance.

[0029] Table 1

[0030] Example 3 The first research question is whether the algorithm proposed in this invention can be validated. Evaluation on an anti-drone dataset demonstrates its effectiveness in validating the applicability of drone detection algorithms. Figure 8 The document describes the challenging properties of the dataset, including a wide range of drone images and extreme cases. Notably, the proposed algorithm demonstrates robust detection performance across these challenging properties. Furthermore, in... Figure 9 In this study, a statistical analysis was performed on the Anti-UAV dataset. This involved categorizing the actual number of pixels occupied by drone targets into three classes: small (less than 1024 pixels), medium (greater than 1024 but less than 9216 pixels), and large (greater than 9216 pixels). Visualization revealed moderate differences in the distribution of small, medium, and large samples. Therefore, the size of drone targets exhibits a balanced distribution in the dataset. Furthermore, visualizing the aspect ratio of drone targets against the aspect ratio of the images revealed a uniform distribution of small and important targets in the dataset. Thus, the balance of drone target salience is evident in the dataset. Additionally, the locational distribution of the drone dataset was described, finding a balanced distribution of drone targets at different locations in the images. This comprehensive analysis highlights the close resemblance between anti-drone data and real-world scenarios, validates the applicability of drone detection algorithms, and underscores the practicality of RTM-UAVDet in real-world environments (RQ1 solution).

[0031] Example 4 This invention relates to a dynamic convolution module and a multi-scale feature encoder. Ablation studies have shown that traditional convolution methods produce suboptimal results. In contrast, multimodal dynamic convolution has proven to be better at adapting to differences between images of different modalities. This confirms the fundamental principles behind the chosen method. Furthermore, the enhancement algorithm for the multi-scale encoder has been validated. Figure 4 Visualize UAV targets within the AP (RQ2 addressing).

[0032] This invention discloses a method for detecting unmanned aerial vehicles (UAVs) in visible light and infrared images. It dynamically adjusts the convolution kernel parameters based on the substantial differences between RGB and TIR modes to maximize the utilization of effective information in the images and improve the detection accuracy of UAVs in different modes. It enhances the focus on target features by using multi-scale dynamic convolution, thereby improving the utilization rate of backbone network feature extraction. It also improves the accuracy of the model by compensating for the lack of infrared image datasets and the overfitting problem associated with low data volume.

[0033] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images, characterized in that, Includes the following steps: Obtain an RGB-T target detection dataset of visible light and infrared images, pre-train the target detection dataset, and obtain a TIR image dataset; Based on the TIR image dataset, a csp-block model based on the multi-scale dynamic convolution module M-DyConv is obtained through a multimodal UAV detection and training method. Image acquisition is performed on the target area to obtain visible light and infrared images; The visible light and infrared images are input into the csp-block model to generate corresponding feature maps at different scales. The feature maps at different scales are input into the Multi-Scale Feature Dynamic Fusion Encoder (MFDF) for fusion to generate a fused feature map. The method of inputting the feature maps of different scales into the multi-scale feature dynamic fusion encoder (MFDF) for fusion to generate the fused feature map includes: The operation of the multi-scale feature dynamic fusion encoder can be expressed by the following formula: In the formula, Concat represents the concatenation of two feature tensors in the channel dimension. and Let X represent the shallow and deep features of the backbone network, respectively. Then, the input features X are processed by a repetitive encoder, where each encoder consists of two residual sub-blocks. The first sub-block is a key component containing a multi-size dynamic convolutional module, which can be represented as: Where Norm(·) represents layer normalization; The first sub-block represents a multi-scale dynamic convolution module; the second sub-block mainly consists of a two-layer MLP with non-linear activation. Where W1∈RC×rC, W2∈RrC×C are the learnable parameters of the MLP established based on the expansion ratio r, and σ(·) represents the GELU activation function; the GELU activation function can be expressed as: In the formula, Φ(x) is the cumulative function of the Gaussian normal distribution of x, and µ and σ are the mean and standard deviation of the normal distribution. Since the above formula cannot be calculated directly, the GELU function is expressed as: In the formula, σ(·) is the sigmoid function; The fused feature map is input into the prediction head for processing to obtain prediction information, which includes the position and size of the UAV.

2. The method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images as described in claim 1, characterized in that, The method for obtaining a csp-block model based on the multi-scale dynamic convolutional module M-DyConv through multimodal UAV detection pre-training includes: matching the generated infrared dataset with labels, each image should have a corresponding annotation file, the annotation file should contain the target category and bounding box coordinates; inputting the dataset into the model to detect targets, and adjusting the algorithm parameters according to the labels using gradient descent.

3. The method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images as described in claim 1, characterized in that, The method for acquiring visible light and infrared images of a target area includes: determining the acquisition range and selecting visible light and infrared sensor devices for installation; starting the devices and acquiring visible light and infrared images.

4. The method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images as described in claim 1, characterized in that, The method for inputting the visible light and infrared images into the csp-block model to generate corresponding feature maps of different scales includes: loading pre-trained weights, using the visible light or infrared image as input, and propagating it forward through the model; during this process, the image passes through the csp-block model, and feature maps of different scales are obtained by extracting the output of the csp-block model.

5. The method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images as described in claim 1, characterized in that, The method for pre-training the object detection dataset includes: converting RGB images to thermal images involves assigning different grayscale values ​​to objects of different categories in a given image; utilizing existing RGB datasets to provide supplementary annotation information, including location and category details, to enhance the efficiency of image conversion; and combining unique thermal imaging features with annotated RGB datasets to convert RGB images to TIR images.

6. The method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images as described in claim 1, characterized in that, The working method of the multi-scale feature dynamic fusion encoder is as follows: In the formula, Concat represents the concatenation of two feature tensors in the channel dimension. and These represent the shallow and deep features of the backbone network, respectively.

7. The method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images as described in claim 5, characterized in that, The process of converting the RGB image into a thermal image specifically includes the following steps: The values ​​of the R, G, and B channels at the same pixel location are averaged and then mapped to grayscale values, thus converting the RGB image into a grayscale image. Use smaller hyperparameters to multiply the background image to distinguish the target from the background in the TIR image.

8. The method for detecting unmanned aerial vehicles (UAVs) under visible light and infrared images as described in claim 5, characterized in that, The RGB image is converted to a TIR image by the following calculation: Where I(x,y) represents the pixel value of the generated pre-trained image, and x and y represent the positions, I R I G I B This represents the red, green, and blue channels in the original image.

Images