Aerial photography scene infrared small target detection method, electronic equipment and storage medium

Abstract

The invention discloses an aerial photography scene infrared small target detection method, electronic equipment and a storage medium, and belongs to the technical field of computer vision and image processing. According to the method, a dynamic multi-scale shared target detection network is trained to achieve infrared target detection, the detection network is composed of a backbone network, a neck network and a head network, and the backbone network comprises a dynamic shared feature decomposition layer and an expanded multi-scale pyramid layer. The neck network is formed by cascading a plurality of up-sampling layers, a splicing layer, a dynamic sharing feature decomposition layer and a convolution layer, and the head network comprises a regression positioning branch and a classification prediction branch. According to the target detection method, a dynamic sharing mechanism and multi-scale feature processing are combined, efficient extraction and fusion of image features are achieved, multi-scale features and context information are fused, high-level semantic information is reserved, low-level detail features are reserved, boundary positioning of a detected target is more accurate, and classification is more reliable.

Description

Technical Field

[0001] The invention relates to an infrared small target detection method, electronic equipment and storage medium in an aerial photography scene, and belongs to the technical field of computer vision and image processing. Background Art

[0002] In modern society, the popularity of aerial photography technology has led to the widespread use of drones in urban planning, disaster monitoring, traffic management, and national defense security. In recent years, infrared detection technology has developed rapidly. Its non-contact and passive detection characteristics have made it a research hotspot in military reconnaissance and civilian fields, and it has attracted widespread attention from all over the world. Infrared detection technology has been widely used in early warning, air defense, and coastal defense systems in the military field, and has also gradually expanded to civilian fields such as medical imaging, traffic management, and autonomous driving. As an important technology in the fields of infrared precision guidance, early warning, video surveillance, and search and tracking, the research on infrared detection is of great significance; however, the poor infrared imaging conditions and the complex background and high viewing angle of aerial photography scenes have brought great challenges to traditional target detection methods.

[0003] Infrared targets are usually small, occupying only a limited number of pixels in the image and with unclear features, and are easily submerged by complex background noise. Although these targets are usually brighter than the background, the contrast between the target and the background in the original infrared image is often very low. In addition, due to the influence of sensor noise and natural factors, many pixel noises similar to high-brightness targets often appear in the image. The low signal-to-noise ratio, few texture features and low resolution of infrared thermal images further increase the difficulty of detection. Infrared small target detection has not yet fully met the requirements of different application scenarios in terms of robustness, robustness and real-time performance. Therefore, it is still the focus and difficulty of current research. There is an urgent need for a small target detection method specifically for infrared aerial photography scenes to improve the recognition accuracy and detection speed of small targets and meet the needs of practical applications.

[0004] In the field of infrared target detection, existing lightweight models usually achieve lightweight by reducing the network structure and the number of network layers, such as using FasterNet or other shallow feature extraction networks, or reducing the number of channels in the model to reduce the amount of calculation. However, these methods will significantly reduce the detection accuracy of the model while lightweighting, and the targets in aerial images are usually small and require high detection accuracy. Existing methods are difficult to meet the needs of human and vehicle target detection in aerial infrared images. Summary of the invention

[0005] Technical problem to be solved by the invention

[0006] Aiming at the limitations of traditional aerial photography infrared scene small target detection methods, this paper proposes an aerial photography scene infrared small target detection method. The method fully considers the characteristics of aerial images, realizes the efficient extraction and fusion of multi-scale features, and improves the detection accuracy of small targets.

[0007] Technical Solution

[0008] In order to achieve the above object, the technical solution provided by the present invention is:

[0009] A method for detecting infrared small targets in an aerial photography scene comprises the following steps:

[0010] Step 1, construct a dynamic multi-scale shared target detection network, the network consists of a backbone network, a neck network, and a head network, wherein the backbone network consists of multiple cascaded convolutional layers, dynamic shared feature decomposition layers, and expanded multi-scale pyramid layers; the neck network is composed of multiple upsampling layers, splicing layers, dynamic shared feature decomposition layers, and convolutional layers; the head network includes multiple two-stream hybrid detection heads, and the two-stream hybrid detection heads include regression positioning branches and classification prediction branches;

[0011] Step 2: preprocess the infrared images of the dataset, convert them into a unified input format, and divide the dataset into a training set and a validation set according to the target category;

[0012] Step 3, input the training set images into the dynamic multi-scale shared object detection network constructed in step 1 for training to determine the parameters of the network, and use the validation set to verify the network detection performance during the training process;

[0013] Step 4: Input the image to be detected into the dynamic multi-scale shared object detection network obtained in step 3, and output the object bounding box as well as the category label and confidence of each object.

[0014] Furthermore, the processing steps of the dynamic multi-scale shared target detection network on the infrared image in steps 3 and 4 are as follows:

[0015] In step a, the image is passed through the backbone network convolution layer to extract the underlying basic features of the image, and then it is passed through several cascaded dynamic shared feature decomposition layers and convolution layers to dynamically extract multi-scale feature information. Finally, the feature map containing global semantic information and local details at different scales is output through the expanded multi-scale pyramid layer.

[0016] Step b: The backbone network outputs the feature map and inputs it into the neck network. The upsampling layer improves the image resolution, the concatenation layer realizes the fusion of information of different scales, and the dynamic shared feature decomposition layer performs feature aggregation and compression.

[0017] Step c: The feature map output by the neck network is input into the head network, and the detection result is output by the dual-stream hybrid detection head.

[0018] Furthermore, in step a, the dynamic shared feature decomposition layer is composed of a cascaded convolutional layer, a separation layer, a dynamic shared bottleneck layer, and a splicing layer. The convolutional layer reduces the number of channels, the separation layer separates the feature maps, the dynamic shared bottleneck layer refines the specific features, and the splicing layer splices the feature maps.

[0019] Furthermore, the dynamic shared bottleneck layer is composed of cascaded convolutional layers and multi-scale shared convolutional layers, wherein the convolutional layers realize local feature extraction of feature maps, and the multi-scale shared convolutional layers realize multi-scale feature extraction.

[0020] Furthermore, the multi-scale shared convolutional layer is composed of a cascaded separation layer, a convolutional layer, and a splicing layer, wherein the separation layer realizes feature map splitting, the convolutional layer realizes the extraction of different target features, and the splicing layer realizes feature map splicing.

[0021] Furthermore, the expanded multi-scale pyramid layer in step a is composed of a cascaded convolutional layer, a hole convolutional layer, and a splicing layer, wherein the convolutional layer reduces the number of channels, the hole convolution layer extracts global context features, and the splicing layer splices feature maps of different scales.

[0022] Furthermore, in step c, the two-stream hybrid detection head is composed of a cascaded convolutional layer, a group normalization layer, an activation function, a re-parameterized convolutional layer, a regression output convolutional layer, and a classification output convolutional layer, wherein the convolutional layer and the group normalization layer standardize the neck network output feature map, the activation function performs a nonlinear transformation on the elements, the re-parameterized convolutional layer performs feature extraction, the regression output convolutional layer calculates the bounding box regression value of each prediction box on each feature map, and the classification output convolutional layer calculates the category score of each prediction box on each feature map.

[0023] An electronic device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; the processor is used to execute the program stored in the memory, and the steps of the above method are implemented when the program is executed.

[0024] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0025] Beneficial Effects

[0026] The present invention designs a dynamic multi-scale shared target lightweight detection network. The method combines a dynamic sharing mechanism with multi-scale feature processing to achieve efficient extraction and fusion of image features. A lightweight detection head is designed based on a heavy parameter convolution method. The method achieves a balance between performance and efficiency.

[0027] The method of the present invention reduces redundant calculations by designing a dynamic shared feature decomposition layer in the backbone network and the neck network, and integrates multi-scale features and contextual information in conjunction with the modular design of a dual-stream hybrid detection head, thereby retaining both high-level semantic information and low-level detail features, making the boundary positioning of infrared detection targets more accurate and the classification more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a structural diagram of a dynamic multi-scale shared target detection network for aerial infrared images of the present invention;

[0029] Figure 2 This is a structural diagram of the dynamic shared feature decomposition module of the present invention;

[0030] Figure 3 This is a structural diagram of the dynamic shared bottleneck module of the present invention;

[0031] Figure 4 This is a multi-scale shared convolution structure diagram of the present invention;

[0032] Figure 5 Expand the multi-scale pyramid module structure diagram of the present invention;

[0033] Figure 6 It is a structural diagram of the dual-flow hybrid detection head of the present invention;

[0034] Figure 7 It is a structural diagram of the re-parameterized convolution of the present invention;

[0035] Figure 8 This is a visual quality comparison chart of typical images of the IResearch test set using the method of the present invention. DETAILED DESCRIPTION

[0036] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and specific implementation methods.

[0037] like Figure 1 The aerial infrared image shown is preprocessed as a standard input format to ensure that it meets the input requirements of the dynamic target detection model and then input into the dynamic multi-scale shared target detection network. The target detection network consists of a backbone network, a neck network, and a head network.

[0038] The backbone network consists of multiple cascaded convolutional layers, dynamic shared feature decomposition layers, and expanded multi-scale pyramid layers. The infrared image is subjected to feature extraction through the backbone network, and the underlying basic features such as edges and textures are extracted through the convolutional layer. After that, multi-scale feature information is dynamically extracted through several cascaded dynamic shared feature decomposition layers and convolutional layers, and feature dimensionality reduction is performed. Finally, the fusion pyramid feature representation of global semantic information and local details at different scales is constructed through the expansion of the multi-scale pyramid layer.

[0039] The neck network is composed of multiple upsampling layers, concatenation layers, dynamic shared feature decomposition layers, and convolution layers. The multi-scale feature maps extracted by the backbone network are input into the neck network, and the multi-scale information is fused step by step through the dynamic shared feature decomposition layer and concatenation layer. The upsampling layer is used to improve the image resolution and ensure the consistency of the feature map at different scales. During the fusion process, the dynamic feature sharing mechanism can adaptively adjust the feature allocation ratio and optimize the feature map information flow.

[0040] The head network contains multiple two-stream hybrid detection heads. After further aggregation of features through convolutional layers and concatenation layers, three scale feature maps are finally output to three two-stream hybrid detection heads. Each two-stream hybrid detection head contains two branches, one of which is responsible for target localization (i.e., bounding box prediction) of infrared images; the other branch is responsible for classification prediction and determining the specific category of the detected target. Finally, the target bounding box on the multi-scale feature map and the category label and confidence of each target are output.

[0041] Example 1

[0042] like Figure 1 As shown, in this implementation, a dynamic multi-scale shared target detection network is designed, specifically:

[0043] In the backbone network, the processed image is input into two convolutional layers at a scale of 640×640×3 to reduce the image dimension. After two convolutional layers, the image size is reduced to 160×160, and the number of channels is increased to 128, that is, the image becomes 160×160×128. The image is then input into four cascaded dynamic shared feature decomposition layers and convolutional layers.

[0044] Example 2

[0045] This embodiment further limits the dynamic multi-scale shared target detection method for aerial infrared images in Embodiment 1. Figure 2 As shown in the figure, the dynamic shared feature decomposition layer module is as follows: in each dynamic shared feature decomposition layer, the input 160×160×128 feature map first passes through a 1×1 convolution layer to reduce the number of channels, thereby reducing the amount of calculation; the separation layer divides the input feature map into two parts Y1 and Y2, providing independent branches for subsequent feature concatenation operations with a scale of 160×160×64; Y2 is recursively processed, and feature interaction is performed through n cascaded dynamic shared bottlenecks, which is responsible for refining specific features and gradually building a multi-scale semantic representation of the target.

[0046] Example 3

[0047] This embodiment further defines the dynamic shared bottleneck layer in Embodiment 2. Figure 3As shown in the figure, the dimension of the input feature Y2 in the dynamic shared bottleneck layer is 160×160×64. First, local features are extracted through a 1×1 convolution layer to keep the spatial dimension of the input feature map unchanged. Then, it is input into the multi-scale shared convolution layer for multi-scale feature extraction. Finally, the input feature Y2 in the dynamic shared bottleneck layer is concatenated with the output feature of the multi-scale shared convolution layer.

[0048] like Figure 4 As shown in the figure, the multi-scale shared convolution layer first splits the input feature map into two parts x1 and x2 through a separation layer. x1 directly transmits cheap features, retains part of the original information, reduces information loss, and has 16 channels; x2 is used for the features of multi-scale convolution processing, and reserves exclusive channels for the multi-scale processing module, with 16 channels. Then x2 is decomposed into two groups, each of which is processed by different convolution kernels 3×3 and 5×5. The 3×3 convolution layer extracts the detailed features x3 of small targets (such as pedestrians and vehicles), and the 5×5 convolution layer extracts the global features x4 of large targets (such as buildings and regional hotspots). The dimensions of x3 and x4 are 160×160×16. The design of the above multi-convolution kernel enables the network to adapt to targets of different scales. In aerial infrared images, the features of small targets (pedestrians and vehicles) and large targets (buildings) can be effectively extracted.

[0049] The feature map x 1， x3 and x4 are concatenated according to the channel to form a multi-scale feature of 160×160×32. Through 1×1 convolution fusion, the original number of channels is restored to 64, further highlighting the target semantic information, and outputting the output feature x5 of the multi-scale shared convolution. Feature x5 is residually connected with feature Y2 to obtain the output feature of the dynamic shared bottleneck layer, whose dimension size is 160×160×64. Finally, the initial grouping feature Y1 is concatenated with all the output high-level features of the recursive dynamic shared bottleneck layer to form the final multi-scale feature map, and then the concatenated features are integrated through 1×1 convolution to output feature x6, with a dimension size of 160×160×128. Then the feature map of the initial scale is residually connected with x6 to obtain the feature map of the fused feature F1, with a dimension size of 160×160×128.

[0050] like Figure 1As shown in the figure, the feature F1 feature map extracted by the first dynamic shared feature decomposition layer is input into the convolution layer for size compression and channel number expansion respectively, and the output feature is a feature map of 80×80×256, which is then input into the next dynamic shared feature decomposition layer for feature extraction, and the output feature map F2 has a dimension size of 80×80×256. Then the feature map F2 is input into the convolution layer and the dynamic shared feature decomposition layer, and the output feature map F3 has a dimension size of 40×40×512. Similarly, the feature map F3 is input into the next convolution layer and the dynamic shared feature decomposition layer, and the output feature map F4 has a dimension size of 20×20×1024. Then the feature map F4 is input into the dilated multi-scale pyramid layer for efficient multi-scale feature extraction.

[0051] Example 4

[0052] This embodiment further limits the expansion multi-scale pyramid module in Embodiment 1. The expansion multi-scale pyramid module is based on the feature sharing mechanism and the receptive field expansion capability brought by the dilated convolution. The target size in the aerial infrared image varies greatly, and the dilated convolution can extract the fine features of small targets and the global features of large targets through different expansion rates.

[0053] like Figure 5 As shown in the figure, the feature map F4 first reduces the number of channels through a 1×1 convolution layer to reduce the amount of calculation, while retaining most of the important information. The output feature map f1 has a dimension of 20×20×512. The feature map f1 is used as the initial feature, and after multiple dilated convolutions, features under different receptive fields are gradually extracted. The dilation rate of the dilated convolution is [1, 3, 5]. Different dilation rates correspond to different receptive fields, which are suitable for detecting small heat sources and large regional hotspots in different aerial infrared images. When the dilation rate is 1, a 3×3 convolution kernel is used to process the input features, and the size of the output feature map f2 is still 20×20×512; when the dilation rate is 3, the dilated convolution expands the receptive field and can capture a wider range of feature patterns; the size of the output feature map f3 remains 20×20×512; when the dilation rate is 5, the dilated convolution further expands the receptive field to extract global context features, and the size of the output feature map f4 is still 20×20×512. All dilated convolutions share a convolution kernel, which can reduce the number of parameters while retaining the consistency of the convolution operation. Then, the feature maps f1, f2, f3, and f4 are concatenated in the channel dimension to form a multi-scale feature map that integrates local and global information. The dimension of the concatenated feature map is 20×20×2048. Finally, a 1×1 convolution layer is used to integrate the number of channels to 1024, and the final feature map F5 is output, whose dimension size is 20×20×1024.

[0054] like Figure 1As shown in the figure, the high-order feature map F5 output by the expanded multi-scale pyramid layer is input into the neck network, first through the upsampling layer, the output size is up to 40 × 40 × 1024 feature map F6, and then the feature map F6 and the feature map F3 are input into the splicing layer to obtain the splicing feature F c1 , with a dimension size of 40×40×1536. Then, this feature map is sent to the dynamic shared feature decomposition layer for feature aggregation and compression, and the output is a 40×40×512 feature map F7. The feature map is again passed through the upsampling layer to output feature F8, with a dimension size of 80×80×512. Then the obtained feature map F8 is concatenated with the feature map F2 to obtain the concatenated feature map F c2 , the dimension size is 80×80×768, then the dynamic shared feature decomposition layer further compresses it to a feature map F9 of 80×80×256, and then the 80×80×256 feature map enters the convolution layer to output a feature map F with reduced size but richer feature information and higher computational efficiency. 10 , whose dimension size is 40×40×256. Then the feature maps F7 and F 10 The feature maps of the feature maps are concatenated to obtain a 40×40×768 feature map F c3 Then this 40×40×768 feature map is passed through the dynamic shared feature decomposition layer again to obtain the feature map F 11 , the dimension size is 40×40×512. Then the feature map F 11 Input to the convolutional layer for further feature compression to obtain the feature map F 12 , the dimension size is 20×20×512. Then the feature map F 12 Concatenate with feature map F5 to obtain concatenated feature F c4 , the dimension size is 20×20×1536. Finally, the concatenated feature F c4 Input to the dynamic shared feature decomposition layer to obtain a 20×20×1024 feature map F 13 .

[0055] The feature maps F9, F 11 , F 13 Input into the dual-stream hybrid detection head for detection and classification.

[0056] Example 5

[0057] This embodiment is a further limitation of the dual-flow lightweight hybrid detection head in Embodiment 1. Figure 6 As shown, first, after a 3×3 convolution layer, the feature map F9, F 11 , F 13The number of channels is reduced to 256, 128, and 64. Then their channels are divided into 16 groups for group normalization, and the channels in each group are normalized independently, using the mean and variance of each group for standardization, and the output is 20×20×256, 40×40×128, and 80×80×64.

[0058] Then apply the SiLU activation function to perform a nonlinear transformation on each element x as follows:

[0059]

[0060] The values ​​of each channel are standardized to change the value range and distribution, improving the stability of the network. This combination of operations is very suitable for object detection tasks in aerial infrared scenes because they can extract and process multi-scale features from the input low-resolution feature map, which helps to detect objects of different sizes and complexities.

[0061] Then, the features are refined through the re-parameterized convolutional layer, that is, two layers of continuous re-parameterized modules, such as Figure 7 As shown in the figure, each re-parameterized convolution layer consists of the following submodules: a 3×3 convolution, a 1×1 convolution, and an optional normalization layer, where the 3×3 convolution extracts local features; the 1×1 convolution enhances channel interaction and combines with the residual features of the input tensor. The second re-parameterized convolution block repeats the above steps to further refine the features. The number of channels of the feature map remains unchanged, still 20×20×256, 40×40×128, 80×80×64, and the features are more discriminative.

[0062] Then, the regression and classification outputs required for target detection are generated through the regression output convolution layer and the classification output convolution layer. The regression output convolution layer is used to calculate the bounding box regression value of each predicted box on each feature map, and the classification output convolution layer is used to calculate the category score of each predicted box on each feature map.

[0063] The results of the regression branch are rescaled, and the regression and classification results are concatenated in the channel dimension: the outputs of the multi-resolution feature maps are concatenated and processed. Finally, the inference stage decodes the regression output into bounding box coordinates and combines it with the classification score to generate the final detection result.

[0064] In order to enhance the detection effect of aerial infrared images, a loss function is needed to constrain the positioning of the target and the smoothing of the label. The loss function of this embodiment consists of bounding box loss, classification loss and confidence loss. The formula is:

[0065]

[0066] in, represents the loss function, and They respectively represent bounding box regression loss (positioning error), classification loss (category prediction error) and confidence loss (judgment of target existence). λ, γ and μ respectively represent weight factors. Accurate positioning is required in aerial photography tasks, especially small targets have high requirements for bounding boxes. The low contrast of infrared images makes classification more challenging, and classification losses need to be appropriately emphasized. Aerial infrared images have sparse targets, and confidence is important for the ability to distinguish between positive and negative samples, but it should be avoided that it is too high, resulting in the weakening of other loss terms. This embodiment sets λ, γ and μ to 0.6, 0.3 and 0.1 respectively through multiple experimental comparisons.

[0067] The bounding box regression loss goal is to minimize the difference between the predicted box and the true box, which is calculated as:

[0068]

[0069] where b pred represents the prediction box, b true Represents the true box. CIoU is an improved bounding box regression loss function in the target detection framework. It combines the center point distance and aspect ratio consistency on the basis of IoU to comprehensively optimize the prediction effect of the bounding box. The form is:

[0070]

[0071] Where IoU represents the intersection over union ratio between the prediction box and the target box, ρ 2 (b,b g ) represents the Euclidean distance between the center points of the prediction box and the target box, c is the length of the diagonal of the minimum closure box containing the prediction box and the target box, v is a parameter to measure the difference in aspect ratio between the prediction box and the target box, α is a weight factor used to balance the consistency of IoU and aspect ratio, The influence of the aspect ratio similarity term is dynamically adjusted according to the IoU value.

[0072] This embodiment uses the BCE (Binary Cross-Entropy) loss function as the classification loss for multi-category classification tasks. The goal is to minimize the difference between the predicted category probability and the true category. The classification loss is calculated as follows:

[0073]

[0074] Where N represents the total number of prediction boxes, c represents the number of categories, yic represents the true label (0 or 1) of the i-th sample belonging to category c. pic represents the probability that the i-th sample is predicted to belong to category c.

[0075] Confidence loss is used to measure whether the predicted box contains the target. This embodiment uses the confidence loss L of the target box to calculate the confidence of the target box. objThe calculation method is:

[0076]

[0077] Among them, N represents the total number of prediction boxes, y i Indicates the target presence mark of the i-th sample (1 indicates a target, 0 indicates no target), p i Represents the confidence of the prediction of the i-th sample.

[0078] Network model training

[0079] The experimental data set is the Pytorch open source machine learning library framework, and the training data uses the open source database Arrow Optoelectronics Infrared Dataset, which contains targets at intersections in multiple different scenarios and at different times. The target categories are divided into 6 categories, namely person, car, bus, cyclist, bike, and truck. The data set is then divided into training set, validation set, and test set in a ratio of 8:1:1. The number of training rounds is 200, Batch_Size is 32, the initial learning rate is 0.001, the cyclic learning rate is 0.1, and the SGD optimizer is used to update the training model.

[0080] Test results

[0081] Model Parameter quantity Calculation Amount Number of images detected per second Accuracy Recall Mean average precision YOLOv5-N 7.1M 4.5G 102 78.3％ 78.9％ 82.6％ YOLOv6-N 4.7M 4.7G 106 79.6％ 77.5％ 81.9％ YOLOv7-tiny 6.2M 13.7G 99 81.5％ 79.7％ 84.1％ YOLOv8-N 3.2M 8.7G 121 81.3％ 78.9％ 83.9％ Gold-YOLO-N 5.6M 12.1G 112 82.1％ 81.0％ 85.4％ Y1.4OLOv9-S 7.1M 26.4G 107 82.9％ 81.4％ 86.1％ YOLOv10-N 2.7M 6.7G 124 81.5％ 80.4％ 84.8％ MambaYOLO-T 6.1M 14.3G 115 82.5％ 80.6％ 85.2％ RT-DETR-R18 20.4M 60G 66 81.7％ 80.2％ 84.6％ Ours 2.2M 3.6G 145 83.5％ 82.7％ 87.5％

[0082] Table 1 Quantitative comparison results of the proposed method and the existing method on the IResearch Optoelectronics test set

[0083] Table 1 shows the comparison results between the method of the present invention and nine existing target detection methods. The comparison indicators include average accuracy, precision, recall, number of parameters, amount of calculation, and number of images detected per second, so as to comprehensively measure the effectiveness and real-time performance of the detection method.

[0084] In terms of the two core indicators of precision and recall, the method of the present invention has a precision of 83.5% and a recall of 82.7%, surpassing YOLOv5-N (precision 78.3%, recall 78.9%) and YOLOv6-N (precision 79.6%, recall 77.5%). In terms of average precision mean, the method of the present invention reached 87.5%, leading most of the comparison models, such as 82.6% of YOLOv5-N and 81.9% of YOLOv6-N, and even surpassing some complex models, such as 84.6% of RT-DETR-R18. In terms of computing resources and speed, the method of the present invention has reached 145 images per second in the number of images detected per second, which is significantly better than other models, such as 106 images per second of YOLOv6-N and 112 images per second of Gold-YOLO-N. The above results show that the method of the present invention not only maintains high precision, but also has a fast reasoning speed. In addition, the method of the present invention has the lowest parameter amount (2.2M) and calculation amount (3.6G), so that the method can run efficiently even on devices with limited hardware resources.

[0085] In summary, the method of the present invention has shown obvious advantages in terms of parameter quantity, computational complexity, number of images detected per second, recall rate, and average precision. These advantages enable the method to not only provide high-efficiency processing capabilities in target detection tasks in aerial infrared scenes, but also ensure high accuracy and high recall rate, which is extremely valuable for practical applications. At the same time, the smaller number of parameters and computational complexity also means that the method of the present invention has a wider range of applicability in resource-constrained environments.

[0086] Figure 8 Shown is a visual quality comparison of the method of the present invention and the baseline model YOLOv8n in four typical images in the Arrow Optoelectronics test set. In these images, the detection results of the method of the present invention show higher target detection accuracy, more accurate positioning of the detection frame, and the ability to detect some smaller or occluded targets, demonstrating excellent robustness and fine feature extraction capabilities. The number next to the detection box indicates the model's confidence in the detection. If the confidence score of this embodiment is generally higher than YOLOv8n, this may indicate that it is more accurate in identifying targets.

[0087] (a) Group of images selects a mid-range perspective from a top-down angle, and there is occlusion of bushes in the scene. In this case, it can be clearly seen that the proposed method successfully detects the human target in the lower right corner without missing any detections, and has a significant advantage in detection accuracy; while the baseline model YOLOv8n has a missed detection problem in this case and fails to detect the human target, indicating that the proposed method is more robust in handling complex occlusion conditions and can better cope with challenges in practical applications.

[0088] Group (b) selects a close-up aerial infrared image scene, in which the target vehicle is partially occluded. In this case, the proposed method can detect large trucks with more pixels and successfully identify the occluded target; while the baseline model YOLOv8n fails to detect the truck and has missed detections. This result shows that the proposed method not only performs well in detecting small targets, but also performs well in detecting medium and large targets, demonstrating its multi-scale target detection capabilities in complex scenes.

[0089] Group (c) tests the model's ability to distinguish between pedestrians and cyclists. For this task, the proposed method can accurately detect cyclists and correctly classify them, while the baseline model YOLOv8n misdetects cyclists as pedestrians in this scene, showing poor recognition ability. This further verifies the accuracy of the proposed method in identifying different target types in complex scenes, and can better cope with fine-grained target detection problems.

[0090] Group (d) of images selected a crowded beach scene, and the detection object is an occluded and deformed crowd. In this scene, the proposed method can successfully detect the occluded and deformed crowd targets, with high detection accuracy and robustness; while the baseline model YOLOv8n will miss some pedestrian targets in this case, indicating that the detection performance of YOLOv8n is reduced in the case of complex background and target overlap. The results show that the proposed method performs better in multi-target, dense scenes and occlusion conditions, and can detect and distinguish different targets more accurately.

[0091] In summary, the proposed method has demonstrated superior detection performance over YOLOv8n in different scenarios and detection tasks, especially in complex occlusion, target overlap, different target scales and category recognition, showing its high robustness and multi-scale feature extraction capabilities. This makes the method have significant application prospects and advantages in aerial infrared image target detection tasks. According to the results of quantitative and qualitative comparisons, the proposed method outperforms the most advanced methods in fusion performance, and the results are consistent with the previous comparison of visual quality.

[0092] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. A method for detecting infrared small targets in aerial photography scenes, characterized in that: The steps include: Step S1, constructing a dynamic multi-scale shared object detection network, the network consists of a backbone network, a neck network, and a head network, wherein the backbone network consists of multiple cascaded convolutional layers, a dynamic shared feature decomposition layer, and an expanded multi-scale pyramid layer; The neck network is composed of multiple upsampling layers, concatenation layers, dynamic shared feature decomposition layers, and convolutional layers cascaded; The head network includes multiple two-stream hybrid detection heads, each of which includes a regression positioning branch and a classification prediction branch; Step S2, preprocessing the infrared images of the data set, converting them into a unified input format, and dividing the data set into a training set and a verification set according to the target category; Step S3, inputting the training set images into the dynamic multi-scale shared object detection network constructed in step S1 for training to determine various parameters of the network, and using the validation set to verify the network detection performance during the training process; Step S4: Input the image to be detected obtained in step S3 into a dynamic multi-scale shared object detection network, and output the object bounding box and the category label and confidence of each object.

2. The method for detecting small infrared targets in aerial photography scenes according to claim 1, characterized in that: The processing steps of the dynamic multi-scale shared target detection network on the infrared image in step S3 and step S4 are as follows: Step a: The image is processed through the backbone network convolution layer to extract the underlying basic features of the image, and then it is processed through several cascaded dynamic shared feature decomposition layers and convolution layers to dynamically extract multi-scale feature information. Finally, the feature map containing global semantic information and local details at different scales is output through the expansion of the multi-scale pyramid layer. Step b: The feature map output by the backbone network is input into the neck network, the resolution of the map is improved by the upsampling layer, the information of different scales is fused by the splicing layer, and the feature is aggregated and compressed by the dynamic shared feature decomposition layer; Step c: The feature map output by the neck network is input into the head network, and the detection result is output by the dual-stream hybrid detection head.

3. The method for detecting small infrared targets in aerial photography scenes as claimed in claim 2, characterized in that: In the step a, the dynamic shared feature decomposition layer is composed of a cascaded convolutional layer, a separation layer, a dynamic shared bottleneck layer, and a splicing layer. The convolutional layer reduces the number of channels, the separation layer separates the feature maps, the dynamic shared bottleneck layer refines specific features, and the splicing layer splices the feature maps.

4. The method for detecting small infrared targets in aerial photography scenes as claimed in claim 3, characterized in that: The dynamic shared bottleneck layer is composed of a cascaded convolutional layer and a multi-scale shared convolutional layer, wherein the convolutional layer realizes local feature extraction of feature maps, and the multi-scale shared convolutional layer realizes multi-scale feature extraction.

5. The method for detecting small infrared targets in aerial photography scenes as claimed in claim 4, characterized in that: The multi-scale shared convolution layer is composed of a cascaded separation layer, a convolution layer, and a splicing layer, wherein the separation layer realizes feature map splitting, the convolution layer realizes the extraction of different target features, and the splicing layer realizes feature map splicing.

6. The method for detecting small infrared targets in aerial photography scenes as claimed in claim 2, characterized in that: The expanded multi-scale pyramid layer in step a is composed of a cascaded convolutional layer, a hole convolutional layer, and a splicing layer, wherein the convolutional layer reduces the number of channels, the hole convolutional layer extracts global context features, and the splicing layer splices feature maps of different scales.

7. The method for detecting small infrared targets in aerial photography scenes as claimed in claim 2, characterized in that: In step c, the dual-stream hybrid detection head is composed of a cascaded convolutional layer, a group normalization layer, an activation function, a re-parameterized convolutional layer, a regression output convolutional layer, and a classification output convolutional layer, wherein the convolutional layer and the group normalization layer standardize the neck network output feature map, the activation function performs a nonlinear transformation on the elements, the re-parameterized convolutional layer performs feature extraction, the regression output convolutional layer calculates the bounding box regression value of each prediction box on each feature map, and the classification output convolutional layer calculates the category score of each prediction box on each feature map.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus. The processor, the communication interface and the memory communicate with each other through the communication bus. The memory is used to store computer programs. The processor is used to execute the program stored in the memory. When the program is executed, the steps of the infrared small target detection method for aerial photography scenes as described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the infrared small target detection method for aerial photography scenes as described in any one of claims 1 to 7 are implemented.

Images