Multi-scale adaptive detection method and system for infrared image of unmanned aerial vehicle, and storage medium

Abstract

The invention relates to a multi-scale adaptive detection method and system for an infrared image of an unmanned aerial vehicle, computer equipment, a storage medium and a computer program product. The method comprises the following steps: preparing and inputting infrared image data; performing reasoning and dynamic adaptive slicing processing on the infrared image data; building a detection framework model; performing model training and optimization based on the sliced infrared image data; and generating an image detection result. The problems of large target scale difference, low contrast, high noise, complex image, blurred target and the like in unmanned aerial vehicle infrared image target detection in the prior art are solved, and the detection performance is remarkably improved. Besides, a YOLOv8 + SAHI + ULSAM integrated model is adopted to carry out target detection on the infrared image of the unmanned aerial vehicle, a target under a complex background can be identified more accurately through the improved YOLOv8 model, the ULSAM module is light in weight, and the calculation burden cannot be remarkably increased.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a multi-scale adaptive detection method, system, storage medium, and computer program product for unmanned aerial vehicle infrared images. Background Art

[0002] With the rapid development of drone technology, drones have been widely used in military reconnaissance, environmental monitoring, disaster relief, and other fields. As an important information source, drone infrared imagery can provide information about the thermal radiation of targets, which is crucial for target detection and identification. However, target detection in drone infrared imagery faces many challenges.

[0003] On the one hand, the objects in infrared images collected by drones vary greatly in size, making it difficult for traditional object detection methods to simultaneously detect objects of different scales. For example, small objects occupy a relatively small proportion of pixels in an image and are easily overlooked or misdetected. Large objects, on the other hand, may have complex features that lead to low detection efficiency.

[0004] On the other hand, infrared images inherently suffer from low contrast and high noise, making target feature extraction difficult and resulting in low target detection accuracy. Traditional target detection methods, such as those based on sliding windows, are inefficient and prone to missed detections. While some deep learning methods have improved detection performance to some extent, they still suffer from inaccurate detection of small targets and insufficient detection of blurred targets when processing UAV infrared images. Therefore, a multi-scale adaptive detection method that effectively addresses the characteristics of UAV infrared images is urgently needed. Summary of the Invention

[0005] Based on this, it is necessary to provide a multi-scale adaptive detection method, system, computer equipment, storage medium and computer program product for UAV infrared images to address the above technical problems.

[0006] In a first aspect, the present application provides a multi-scale adaptive detection method for UAV infrared images, the method comprising:

[0007] Infrared image data preparation and input;

[0008] performing reasoning and dynamic adaptive slicing processing on the infrared image data;

[0009] Build a detection framework model;

[0010] Model training and optimization based on sliced ​​infrared image data;

[0011] Generate image detection results.

[0012] In one embodiment, the infrared image data preparation and input includes:

[0013] Collect drone infrared image datasets in different environments and annotate categories and locations;

[0014] The labeled data set is divided into training set, validation set and test set as the original data.

[0015] In one embodiment, the inference and dynamic adaptive slicing of the infrared image data includes:

[0016] The SAHI inference architecture is used to perform inference and dynamic adaptive slicing processing on the infrared image data.

[0017] In one embodiment, the inference and dynamic adaptive slicing processing of the infrared image data using the SAHI inference architecture includes:

[0018] The SAHI inference architecture is used to perform downsampling coarse detection on the infrared image data, and then restore the resolution for fine detection;

[0019] The target center point and pixel density are counted to construct a heat map, and the density levels are divided after Gaussian filtering and smoothing.

[0020] Based on the heat map density, slices of different sizes are used for high, medium, and low density areas.

[0021] In one embodiment, the model training and optimization based on the sliced ​​infrared image data includes:

[0022] The image processed by dynamic adaptive slicing is input into the improved model, trained with the training set, and the weights are updated through back propagation;

[0023] Use the validation set to evaluate and tune hyperparameters;

[0024] Evaluate the trained model using the test set.

[0025] In one embodiment, generating an image detection result includes:

[0026] The detection results of different slices are fused, the overlapping frames are removed, and the image detection results are generated;

[0027] Output target detection box to mark the target location in the image.

[0028] In a second aspect, the present application also provides a multi-scale adaptive detection system for UAV infrared images, the system comprising:

[0029] Input module, used for infrared image data preparation and input;

[0030] A processing module, configured to perform inference and dynamic adaptive slicing processing on the infrared image data;

[0031] Building modules for building detection framework models;

[0032] Training module, used for model training and optimization based on sliced ​​infrared image data;

[0033] Output module, used to generate image detection results.

[0034] The statistics module is used to count the results and generate a matching result array.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0036] Infrared image data preparation and input;

[0037] performing reasoning and dynamic adaptive slicing processing on the infrared image data;

[0038] Build a detection framework model;

[0039] Model training and optimization based on sliced ​​infrared image data;

[0040] Generate image detection results.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps: starting a kernel function to obtain and filter candidate points of a portion of a subgraph;

[0042] Infrared image data preparation and input;

[0043] performing reasoning and dynamic adaptive slicing processing on the infrared image data;

[0044] Build a detection framework model;

[0045] Model training and optimization based on sliced ​​infrared image data;

[0046] Generate image detection results.

[0047] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0048] Infrared image data preparation and input;

[0049] performing reasoning and dynamic adaptive slicing processing on the infrared image data;

[0050] Build a detection framework model;

[0051] Model training and optimization based on sliced ​​infrared image data;

[0052] Generate image detection results.

[0053] The multi-scale adaptive detection method, system, computer device, storage medium, and computer program product for drone infrared imagery involve preparing and inputting infrared image data; performing inference and dynamic adaptive slicing on the infrared image data; building a detection framework model; training and optimizing the model based on the sliced ​​infrared image data; and generating image detection results. This method addresses existing issues in drone infrared image target detection, such as large target scale differences, low contrast, high noise, complex images, and blurred targets, significantly improving detection performance. Furthermore, the present invention utilizes an integrated YOLOv8, SAHI, and ULSAM model for target detection in drone infrared images. The improved YOLOv8 model enables more accurate identification of targets in complex backgrounds, while the lightweight ULSAM module does not significantly increase computational burden. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of a multi-scale adaptive detection method for UAV infrared images provided by one embodiment of the present invention;

[0055] Figure 2 A schematic diagram of the ULSAM module structure in one embodiment of the present invention;

[0056] Figure 3 This is a comparison chart of the detection effect of the model of the present invention;

[0057] Figure 4 An architecture diagram of a multi-scale adaptive detection system for UAV infrared images provided by one embodiment of the present invention;

[0058] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0060] The present invention adopts the YOLOv8+SAHI+ULSAM integrated model to perform target detection in UAV infrared images, aiming to solve the problems of large target scale difference, low contrast, high noise, complex image and blurred targets in the existing technology in UAV infrared image target detection, and significantly improve the detection performance.

[0061] To achieve the above purpose, please refer to Figure 1 , the present application provides a multi-scale adaptive detection method for UAV infrared images, the method comprising:

[0062] S100, infrared image data preparation and input.

[0063] In an embodiment of the present invention, a dataset of infrared images of drones in different environments is collected and annotated with categories and locations; the annotated dataset is divided into a training set, a validation set, and a test set as raw data.

[0064] Specifically, during data collection and annotation, we collected a dataset of drone infrared images captured in various environments, covering targets of varying sizes, complex backgrounds, and blurred objects. We annotated the dataset, accurately labeling the target categories and locations within the images, creating annotation files containing target category labels and bounding box coordinates. This ensures data diversity and improves model generalization.

[0065] After labeling the data, the dataset needs to be partitioned. The labeled dataset is divided into training, validation, and test sets in a certain ratio (e.g., 8:1:1) to ensure that each dataset fully represents the data distribution for model training, validation, and testing.

[0066] S200: performing inference and dynamic adaptive slicing processing on the infrared image data.

[0067] In this application, the SAHI inference architecture is used to perform inference and dynamic adaptive slicing on the infrared image data. Specifically, step S200 includes the following sub-steps:

[0068] (1) This application uses the SAHI inference architecture to perform downsampling coarse detection on the infrared image data, and then restores the resolution for fine detection.

[0069] Specifically, the present invention adopts a two-stage SAHI inference architecture. First, the input infrared image is downsampled (resolution reduced to 1 / 8), and a fast coarse detection with a low confidence threshold (0.1) is performed using micro-slices (64×64 pixels) to generate preliminary target distribution information. Based on this distribution data, a dynamically adjusted fine-slice detection is then performed on the original resolution image.

[0070] (2) The target center point and pixel density are counted to construct a heat map, and the density levels are divided after Gaussian filtering and smoothing.

[0071] Specifically, during the SAHI inference process, the initial heat map is constructed by statistically analyzing the spatial distribution density of the target center points in each region during the coarse detection phase, and the pixel density information of the target region in the image is used to further enrich the details of the heat map. Specifically, the number of target pixels within a certain range around each pixel is calculated, and this density information is mapped to the color depth of the heat map. The darker the color, the greater the target density. In order to eliminate the influence of isolated noise points on the heat map, Gaussian filtering (σ = 2 pixels) is used to smooth the heat map. Afterwards, the heat map is normalized and quantized so that its value range is between 0 and 1 for subsequent processing and analysis.

[0072] (3) Based on the density of the heat map, slices of different sizes are used for high, medium, and low density areas.

[0073] Specifically, the image area is divided into different density levels based on the normalized heat map density value. High-density areas have a density value greater than 0.5, and the distance between objects in this area is usually less than 1.5 times the average object size, indicating that the objects are relatively dense in this area. Medium-density areas have a density value between 0.2 and 0.5, and objects are loosely clustered in this area. Low-density areas have a density value less than 0.2, and are usually single objects or areas with scattered objects.

[0074] Furthermore, dynamic slicing has significant advantages over fixed slicing. In terms of small target detection accuracy, dynamic slicing can use small-sized slices and high overlap rates in high-density areas to ensure complete coverage of small targets and avoid information loss. Fixed slicing is prone to small target segmentation errors or feature loss due to its large size. In terms of computational efficiency and resource utilization, dynamic slicing can adjust the slice size and overlap rate based on the sparsity of the target. In low-density areas, large slices and low overlap rates are used to reduce redundant calculations and avoid over-segmentation of target-free areas. The unified parameter approach of fixed slicing wastes computing resources in sparse target areas and makes it difficult to meet detection accuracy requirements in dense areas.

[0075] Therefore, based on the generation results of the above-mentioned density heat map, this application implements a dynamic adaptive slicing strategy. For high-density areas, in view of the density of the target, small slices of 256×256 are used for segmentation, and an overlap rate of 30% is set. This ensures that small targets (less than 20 pixels in size) can be fully covered, avoiding the loss of target information due to excessively large slices, thereby improving the detection accuracy of small targets. In medium-density areas, considering the relative looseness of target distribution, medium-sized slices of 384×384 are used, and the overlap rate is set to 25%. This strategy aims to balance detection accuracy and computational efficiency, which can better include the target without causing excessive computational complexity due to too many slices. For low-density areas, since the targets are relatively sparse, large slices of 512×512 are used for segmentation, and the overlap rate is reduced to 20%. This can effectively reduce redundant calculations and improve the overall efficiency of detection while ensuring the detection effect.

[0076] S300, build a detection framework model.

[0077] In this application, YOLOv8 is used as the basic framework, and the ULSAM ultra-lightweight subspace attention module is inserted into the CSPDarknet feature fusion layer to enhance the feature perception of small and blurred targets through the subspace attention mechanism and gating mechanism.

[0078] Specifically, this application innovatively improves the YOLOv8 model structure. The YOLOv8 model is built as the foundational object detection framework, and the ULSAM (Ultra-Lightweight Subspace Attention Module) is precisely inserted into the CSPDarknet feature fusion layers (layers 4, 7, and 10) of the YOLOv8 model. These layers correspond to feature map scales of 80×80, 40×40, and 20×20, respectively. The dynamic feature map size perception mechanism automatically adapts the input feature dimensions at each layer.

[0079] Combined with reference Figure 2 The ULSAM module adopts an innovative subspace attention mechanism, which first decomposes the input features into 8 low-dimensional subspaces (the dimensionality reduction rate is 40%), and then realizes the cross-subspace feature interaction through the gating mechanism. Its core calculation formula is Where σ represents the sigmoid activation function and Wg is the learnable gating weight matrix.

[0080] This design enables the model to dynamically adjust the contribution of features from different subspaces, significantly enhancing the ability to perceive small and blurred objects while maintaining computational efficiency (increasing the number of parameters by only 2.3%). The improved YOLOv8 model can more accurately identify objects in complex backgrounds. Furthermore, the ULSAM module is lightweight, without significantly increasing the computational burden.

[0081] S400, model training and optimization based on the infrared image data after slicing.

[0082] In this application, S400 includes the following sub-steps:

[0083] (1) The image processed by dynamic adaptive slicing is input into the improved model, trained with the training set, and the weights are updated through back propagation.

[0084] First, the image slices after dynamic adaptive slicing are transferred to the improved YOLOv8 target detection model as training data.

[0085] The model is trained using sliced ​​images from the training set. Appropriate loss functions (such as cross-entropy loss for object classification and mean squared error loss for bounding box regression) and optimization algorithms (such as the Adam optimization algorithm) are used to adjust the model's parameters so that it gradually learns the characteristics of the targets in the drone's infrared images. During training, the model continuously updates its weights using a backpropagation algorithm to minimize the loss function.

[0086] (2) Use the validation set to evaluate and adjust hyperparameters.

[0087] During training, the model is validated using the validation set. Based on the validation results, the model's performance is evaluated, such as the percentage of images with detected objects and accuracy. If the validation results are unsatisfactory, the model's hyperparameters, such as the learning rate and batch size, are adjusted to improve performance. For example, if the validation accuracy is low, the learning rate can be appropriately reduced to make model training more stable.

[0088] (3) Use the test set to evaluate the trained model.

[0089] During the testing phase, the trained model is evaluated on the test set. The test set images are processed using the same SAHI inference and slicing pipeline as the training set, and then fed into the model for object detection. The model outputs the detected anchor boxes for each object in each slice, along with their corresponding accuracy.

[0090] S500: Generate image detection results.

[0091] In this application, the detection results of different slices are fused, the overlapping frames are removed, and the image detection results are generated; the target detection frame is output to mark the target position in the image.

[0092] Specifically, the drone detection anchor frame and accuracy output by the model are screened according to the set threshold, the non-maximum suppression algorithm is used to remove overlapping detection frames, all slice detection results are merged to generate a complete image detection frame, and the final detection result containing the target category, bounding box coordinates and confidence is output. The entire process achieves efficient and accurate target detection in drone infrared images through dynamic adaptive slicing technology and the improved YOLOv8-ULSAM model architecture. Combined with reference Figure 3 , Figure 3 Among them, (a) is the test result of the existing technology, and (b) is the test result of the present application. The test result of the present application is obviously better than the existing technology and is clearer and more accurate.

[0093] In summary, this application involves preparing and inputting infrared image data; performing inference and dynamic adaptive slicing on the infrared image data; building a detection framework model; training and optimizing the model based on the sliced ​​infrared image data; and generating image detection results. This approach addresses existing issues in drone infrared image target detection, such as large target scale variations, low contrast, high noise, complex images, and blurred targets, significantly improving detection performance.

[0094] At least part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0095] Based on the same inventive concept, the present application also provides a multi-scale adaptive detection system for drone infrared images. The solution provided by this system is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following multi-scale adaptive detection system embodiments for drone infrared images can be found in the aforementioned multi-scale adaptive detection method for drone infrared images, and will not be further elaborated here.

[0096] In one embodiment, Figure 4 As shown in FIG, a multi-scale adaptive detection system for UAV infrared images is provided, comprising:

[0097] The input module 100 is used for preparing and inputting infrared image data.

[0098] The processing module 200 is used to perform inference and dynamic adaptive slicing processing on the infrared image data.

[0099] The building module 300 is used to build a detection framework model.

[0100] The training module 400 is used to perform model training and optimization based on the sliced ​​infrared image data.

[0101] The output module 500 is used to generate image detection results.

[0102] In one embodiment, the input module 100 is further used to: collect drone infrared image datasets in different environments and mark the categories and locations; divide the marked datasets into a training set, a validation set, and a test set as raw data.

[0103] In one embodiment, the processing module 200 is further configured to perform inference and dynamic adaptive slicing processing on the infrared image data using a SAHI inference architecture.

[0104] In one embodiment, the processing module 200 is further configured to:

[0105] The SAHI inference architecture is used to perform downsampling coarse detection on the infrared image data, and then restore the resolution for fine detection;

[0106] The target center point and pixel density are counted to construct a heat map, and the density levels are divided after Gaussian filtering and smoothing.

[0107] Based on the heat map density, slices of different sizes are used for high, medium, and low density areas.

[0108] In one embodiment, the training module 400 is further configured to:

[0109] The image processed by dynamic adaptive slicing is input into the improved model, trained with the training set, and the weights are updated through back propagation;

[0110] Use the validation set to evaluate and tune hyperparameters;

[0111] Evaluate the trained model using the test set.

[0112] In one embodiment, the output module 500 is further configured to:

[0113] The detection results of different slices are fused, the overlapping frames are removed, and the image detection results are generated;

[0114] Output target detection box to mark the target location in the image.

[0115] Each module in the aforementioned multi-scale adaptive detection system for drone infrared imagery can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor within a computer device in hardware form, or stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0116] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store preset data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a multi-scale adaptive detection method for drone infrared images is implemented.

[0117] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0118] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned multi-scale adaptive detection method for drone infrared images when executing the computer program.

[0119] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the multi-scale adaptive detection method for drone infrared images is implemented.

[0120] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the above-mentioned multi-scale adaptive detection method for drone infrared images.

[0121] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to these.

[0122] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A multi-scale adaptive detection method for UAV infrared images, characterized by: The method comprises: Infrared image data preparation and input; performing reasoning and dynamic adaptive slicing processing on the infrared image data; Build a detection framework model; Model training and optimization based on sliced ​​infrared image data; Generate image detection results.

2. The method according to claim 1, characterized in that The infrared image data preparation and input includes: Collect drone infrared image datasets in different environments and annotate categories and locations; The labeled data set is divided into training set, validation set and test set as the original data.

3. The method according to claim 1, characterized in that The reasoning and dynamic adaptive slicing processing of the infrared image data includes: The SAHI inference architecture is used to perform inference and dynamic adaptive slicing processing on the infrared image data.

4. The method according to claim 3, characterized in that The SAHI inference architecture is used to perform inference and dynamic adaptive slicing processing on the infrared image data, including: The SAHI inference architecture is used to perform downsampling coarse detection on the infrared image data, and then restore the resolution for fine detection; The target center point and pixel density are counted to construct a heat map, and the density levels are divided after Gaussian filtering and smoothing. Based on the heat map density, slices of different sizes are used for high, medium, and low density areas.

5. The method according to claim 2, characterized in that The model training and optimization based on the sliced ​​infrared image data includes: The image processed by dynamic adaptive slicing is input into the improved model, trained with the training set, and the weights are updated through back propagation; Use the validation set to evaluate and tune hyperparameters; Evaluate the trained model using the test set.

6. The method according to claim 1, characterized in that Generating the image detection result includes: The detection results of different slices are fused, the overlapping frames are removed, and the image detection results are generated; Output target detection box to mark the target location in the image.

7. A multi-scale adaptive detection system for UAV infrared images, characterized by: The system comprises: Input module, used for infrared image data preparation and input; A processing module, configured to perform inference and dynamic adaptive slicing processing on the infrared image data; Building modules for building detection framework models; Training module, used for model training and optimization based on sliced ​​infrared image data; Output module, used to generate image detection results.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images