Unmanned aerial vehicle image target detection method, device and equipment and storage medium

Abstract

This application provides a method, apparatus, device, and storage medium for target detection in UAV images, relating to the field of computer technology. It aims to improve the performance of target detection technology in addressing the shortcomings of small target detection in aerial images. The method includes: acquiring UAV aerial images; performing target detection based on the UAV aerial images and a preset target detection model to obtain target detection results, thus completing the detection and recognition of small targets in the UAV images; wherein the target detection model employs an improved RT-DETR algorithm, introducing a first feature extraction subnetwork and a second feature extraction subnetwork into the RT-DETR feature extraction network. The first feature extraction subnetwork is used to extract multi-level features from the sample image and fuse these multi-level features to obtain a first fused image feature; the second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for detecting targets in UAV images. Background Technology

[0002] In recent years, the application scope of drones has been continuously expanding, and their image recognition technology in scenarios such as aerial monitoring and disaster relief has become a research hotspot. Existing image target detection algorithms generally use manually labeled training images to iteratively train the neural network used for detection. Training images with similar styles and their labels constitute a complete dataset.

[0003] However, existing target detection technologies still face multiple technical bottlenecks in aerial image processing. For example, from a high-altitude overhead view, small targets are prone to losing effective information during the multi-level downsampling process of neural networks due to a lack of pixel information and texture features. In addition, the mutual occlusion effect between dense targets further increases the difficulty of recognition. Summary of the Invention

[0004] Based on the above-mentioned technical problems, this application provides a method, apparatus, device and storage medium for target detection in UAV images, which can improve the performance of target detection technology in the case of small target detection in aerial images.

[0005] Firstly, this application provides a method for target detection in UAV images. The method includes: acquiring aerial images taken by a UAV; performing target detection based on the aerial images and a preset target detection model to obtain target detection results, thereby completing the detection and recognition of small targets in the UAV images; wherein the target detection model adopts an improved RT-DETR algorithm, and a first feature extraction subnetwork and a second feature extraction subnetwork are introduced into the feature extraction network of RT-DETR. The first feature extraction subnetwork is used to extract multi-level features of the sample image and fuse the multi-level features to obtain a first fused image feature; the second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature; the second fused image feature enhances the model's mastery of features at each level, and the size of the second fused image feature is smaller than that of the first fused image feature.

[0006] Secondly, this application provides a training method for an object detection model. The method includes: acquiring a training set; the training set contains multiple sample images and object annotation results for the multiple sample images, wherein the multiple sample images are images taken from an aerial perspective, and the annotated objects in the object annotation results are small objects in the images; constructing a neural network model including an input network, a feature extraction network, and an output network; the input network is used to input the sample images into the feature extraction network, which includes a first feature extraction subnetwork and a second feature extraction subnetwork; the first feature extraction subnetwork is used to extract multi-level features from the sample images and fuse the multi-level features to obtain a first fused image feature; the second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature; the output network is used to obtain a target prediction result for the sample images based on the second fused image feature output by the second feature extraction subnetwork; and adjusting the parameters of each network in the neural network model based on the target prediction result and the object annotation result to obtain the object detection model.

[0007] In one possible implementation, the first feature extraction subnetwork includes a hierarchical feature extraction module, multiple convolutional channels, a channel reconstruction module, and an attention fusion module. The hierarchical feature extraction module performs multi-level convolutional processing on the sample image to obtain multi-level features of the sample image. Each convolutional channel receives the corresponding hierarchical features and transmits the corresponding hierarchical features to the channel reconstruction module. The channel reconstruction module performs spatial resolution normalization processing on the received hierarchical features to obtain multi-level features with consistent spatial resolution. The attention fusion module performs interactive fusion on the multi-level features with consistent spatial resolution to obtain the first fused image features.

[0008] In one possible implementation, the second feature extraction subnetwork includes a first downsampling module, a second downsampling module, a third downsampling module, and a downsampling fusion module, which are connected in parallel. The first downsampling module is used to collect key semantic features from the first fused image features to obtain a first downsampling feature map. The second downsampling module is used to perform max pooling on the first fused image features to obtain a second downsampling feature map. The third downsampling module is used to separate high-frequency and low-frequency information from the first fused image features to obtain a third downsampling feature map. The downsampling fusion module is used to fuse the first downsampling feature map, the second downsampling feature map, and the third downsampling feature map to obtain the second fused image features.

[0009] In one possible implementation, the parameters of each network in the neural network model are adjusted based on the target prediction results and target annotation results to obtain the target detection model. This includes: inputting multiple sample images into the neural network model respectively, obtaining the target prediction results output by the neural network model for each sample image; calculating the loss function value based on the target annotation results and target prediction results of the multiple sample images; and adjusting the parameters of each network in the neural network model based on the loss function value until the loss function value converges to obtain the target detection model.

[0010] In one possible implementation, the loss function value is calculated based on the target annotation results and target prediction results of multiple sample images, including: calculating the loss function value of the target annotation results and target prediction results of multiple sample images based on the Inner-CIOU loss function; the formula for the Inner-CIOU loss function is as follows: in, This represents the final Inner-CIOU loss. This represents the CIoU loss, where IoU represents the intersection-union ratio between the predicted bounding box and the target bounding box. The IOU value represents the auxiliary box, where, in, , , , , , , , Where inter represents the area of ​​the intersection between the auxiliary boxes corresponding to the ground truth bounding box and the auxiliary boxes corresponding to the predicted bounding box, and union represents the area of ​​the union of the auxiliary boxes corresponding to the ground truth bounding box and the auxiliary boxes corresponding to the predicted bounding box. , Represents the width and length of the actual bounding box, ( , ) represents the center coordinates of the true bounding box, ( , The coordinates () represent the center coordinates of the anchor frame, w and h represent the width and height of the anchor frame, respectively, and ratio is the corresponding scale factor. , These represent the x-coordinates of the auxiliary boxes corresponding to the real boxes. , These represent the y-coordinates of the four vertices of the auxiliary box corresponding to the real box. , These represent the x-coordinates of the auxiliary boxes corresponding to the predicted boxes. , These represent the ordinates of the auxiliary boxes corresponding to the predicted boxes.

[0011] Thirdly, this application provides a training device for an object detection model, comprising an acquisition unit, a processing unit, and a training unit. The acquisition unit is used to acquire a training set, which includes multiple sample images and target annotation results for the sample images. The sample images are taken from an aerial perspective, and the labeled targets in the target annotation results are small targets in the images. The processing unit is used to construct a neural network model comprising an input network, a feature extraction network, and an output network. The input network is used to input the sample images into the feature extraction network, which includes a first feature extraction subnetwork and a second feature extraction subnetwork. The first feature extraction subnetwork is used to extract multi-level features from the sample images and fuse the multi-level features to obtain a first fused image feature. The second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature. The output network is used to obtain a target prediction result for the sample images based on the second fused image feature output by the second feature extraction subnetwork. The training unit is used to adjust the parameters of each network in the neural network model based on the target prediction result and the target annotation result to obtain an object detection model.

[0012] In one possible implementation, the first feature extraction subnetwork includes a hierarchical feature extraction module, multiple convolutional channels, a channel reconstruction module, and an attention fusion module. The hierarchical feature extraction module performs multi-level convolutional processing on the sample image to obtain multi-level features of the sample image. Each convolutional channel receives the corresponding hierarchical features and transmits the corresponding hierarchical features to the channel reconstruction module. The channel reconstruction module performs spatial resolution normalization processing on the received hierarchical features to obtain multi-level features with consistent spatial resolution. The attention fusion module performs interactive fusion on the multi-level features with consistent spatial resolution to obtain the first fused image features.

[0013] In one possible implementation, the second feature extraction subnetwork includes a first downsampling module, a second downsampling module, a third downsampling module, and a downsampling fusion module, which are connected in parallel. The first downsampling module is used to collect key semantic features from the first fused image features to obtain a first downsampling feature map. The second downsampling module is used to perform max pooling on the first fused image features to obtain a second downsampling feature map. The third downsampling module is used to separate high-frequency and low-frequency information from the first fused image features to obtain a third downsampling feature map. The downsampling fusion module is used to fuse the first downsampling feature map, the second downsampling feature map, and the third downsampling feature map to obtain the second fused image features.

[0014] Fourthly, this application provides an electronic device, including: a processor and a memory; the memory stores processor-executable instructions; when the processor is configured to execute the instructions, the electronic device implements the method described in the first aspect above.

[0015] Fifthly, this application provides a computer program product that, when run in an electronic device, causes the electronic device to execute the methods related to the first aspect described above, thereby implementing the methods of the first aspect.

[0016] In a sixth aspect, this application provides a computer-readable storage medium comprising: software instructions; which, when executed in an electronic device, cause the electronic device to implement the method described in the first aspect.

[0017] The technical solution provided in this application can include the following beneficial effects: This application extracts multi-level features of sample images through a first feature extraction sub-network and fuses these multi-level features to achieve cross-scale feature stitching. Compared with traditional feature extraction operations, this application can fully utilize features at each level, enhance the understanding of global context information, and compensate for the limitations of small target resolution and background interference in UAV imagery. Furthermore, this application downsamples the first fused image features through a second feature extraction sub-network to obtain second fused image features. This application abandons traditional upsampling operations and uses cross-scale feature stitching to replace interpolation operations, reducing the number of model parameters and computational redundancy, while ensuring the accuracy of model recognition and accelerating model training speed.

[0018] The beneficial effects of aspects two through six mentioned above can be referred to in aspect one, and will not be repeated here. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the structure of a model training system provided in an embodiment of this application.

[0021] Figure 2 This is a schematic diagram illustrating the composition of an electronic device provided in an embodiment of this application.

[0022] Figure 3 This is a flowchart illustrating the training method for the target detection model provided in this application embodiment.

[0023] Figure 4 This is a schematic diagram of the overall architecture of the target detection model provided in the embodiments of this application.

[0024] Figure 5 A schematic diagram of the FAD module provided in an embodiment of this application.

[0025] Figure 6 This is a flowchart illustrating the UAV image target detection method provided in an embodiment of this application.

[0026] Figure 7 This is a schematic diagram of the composition of the training device for the target detection model provided in the embodiments of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0028] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] Furthermore, in the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, in the description of the embodiments of this application, "multiple" refers to two or more.

[0030] Before providing a detailed explanation of the embodiments of this application, some related terms and technologies involved in the embodiments of this application will be introduced first.

[0031] The applications of drones are constantly expanding, and their image recognition technology in scenarios such as aerial monitoring and disaster relief has become a research hotspot. Equipped with high-definition image acquisition equipment, drones can provide multi-dimensional data support for urban spatial planning and geographic information modeling, while also possessing the ability to conduct dynamic patrols of key areas, effectively maintaining social security and order. In special scenarios such as disaster relief, this technology can overcome terrain limitations to conduct reconnaissance of high-risk areas, significantly improving emergency response efficiency.

[0032] However, aerial image processing still faces multiple technical bottlenecks: the targets being detected are generally small in size, spatially clustered, subject to significant background interference, and subject to variable lighting conditions, making it difficult for conventional detection methods to meet practical needs in terms of recognition accuracy and computational efficiency. From a high-altitude perspective, small targets are prone to losing effective information during the multi-level downsampling process of neural networks due to a lack of pixel information and texture features. In addition, the mutual occlusion effect between dense targets further increases the difficulty of recognition.

[0033] While current mainstream deep learning models (such as YOLO and DETR) perform well in conventional scene detection, they still have limitations when faced with the lightweight deployment requirements and small target detection requirements unique to aerial photography scenarios, such as high model complexity, weak multi-scale feature integration capabilities, and insufficient sensitivity in small target recognition.

[0034] To address the performance shortcomings of the RT-DETR model in aerial small target detection, this application proposes an improved deep learning architecture. The research focuses on lightweight network design, efficient feature fusion mechanism, and targeted loss function optimization, aiming to achieve a synergistic improvement in detection accuracy and computational efficiency.

[0035] The training method of the target detection model provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0036] The training method for the target detection model provided in this application embodiment can be applied to a model training system. Figure 1 A schematic diagram of one structure of the model training system is shown. For example... Figure 1 As shown, the model training system 10 includes a training device 11 for the target detection model and an aerial image database 12.

[0037] Among them, the aerial image database 12 contains a large number of images taken by drones, as well as labeled images.

[0038] The training device 11 can acquire sample images from the aerial image database 12 and use these sample images to train the model. The specific training process can refer to the training method of the target detection model described in the following method embodiment, which will not be repeated here.

[0039] The training device 11 for the object detection model can be any electronic device with data processing capabilities. For example, the training device 11 for the object detection model can be a server, a computer, or a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. Optionally, the server can be a central server, and the server can also be implemented on a cloud platform. For example, the cloud platform can include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud, and multi-cloud, or any combination thereof. This application embodiment does not limit this.

[0040] The execution entity of the target detection model training method provided in this application embodiment can be the target detection model training device 11 described above. As mentioned above, the target detection model training device 11 can be an electronic device with data processing capabilities, such as a computer or server. Optionally, the target detection model training device 11 can also be a processor (e.g., a central processing unit, CPU) in the aforementioned electronic device; or, the target detection model training device 11 can also be an application (APP) with model training capabilities installed in the aforementioned electronic device; or, the target detection model training device 11 can also be a functional module with model training capabilities in the aforementioned electronic device, etc. This application embodiment does not impose any limitations on this.

[0041] For simplicity, the following description will use the training device 11 of the object detection model as an electronic device.

[0042] Figure 2 This is a schematic diagram illustrating the composition of an electronic device provided in an embodiment of this application. For example... Figure 2 As shown, the electronic device may include: a processor 20, a memory 21, a communication line 22, a communication interface 23, and an input / output interface 24.

[0043] The processor 20, memory 21, communication interface 23 and input / output interface 24 can be connected via communication line 22.

[0044] Processor 20 is used to execute instructions stored in memory 21 to implement the fault analysis method provided in the following embodiments of this application. Processor 20 may be a CPU, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller (MCU), a programmable logic device (PLD), or any combination thereof. Processor 20 may also be any other device with processing capabilities, such as a circuit, device, or software module; this application embodiment does not limit this. In one example, processor 20 may include one or more CPUs, for example... Figure 2 CPU0 and CPU1 in the example. As an optional implementation, the electronic device may include multiple processors; for example, in addition to processor 20, it may also include processor 25. Figure 2 (The example shown is a dashed line).

[0045] The memory 21 is used to store instructions. For example, the instructions may be computer programs. Optionally, the memory 21 may be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions; it may also be a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions; it may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, etc. The embodiments of this application do not limit this.

[0046] It should be noted that the memory 21 can exist independently of the processor 20, or it can be integrated with the processor 20. The memory 21 can be located inside or outside the electronic device, and this embodiment does not impose any restrictions on this.

[0047] Communication line 22 is used to transmit information between the components included in the electronic device.

[0048] The communication interface 23 is used to communicate with other devices (such as the image acquisition device 100 described above) or other communication networks. These other communication networks can be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 23 can be a module, circuit, transceiver, or any device capable of enabling communication.

[0049] Input / output interface 24 is used to enable human-computer interaction between the user and the electronic device. For example, it enables action interaction or information exchange between the user and the electronic device.

[0050] For example, the input / output interface 24 can be a mouse, keyboard, display screen, or touch screen. Action interaction or information exchange between the user and the electronic device can be achieved through a mouse, keyboard, display screen, or touch screen.

[0051] It should be noted that, Figure 2 The structures shown do not constitute a limitation on electronic devices, except... Figure 2 In addition to the components shown, electronic devices may include more or fewer components than illustrated, or combinations of certain components, or different component arrangements.

[0052] The training method of the target detection model provided in the embodiments of this application is described below.

[0053] Figure 3 This is a flowchart illustrating the training method for the target detection model provided in this application embodiment. Optionally, this method can be implemented by someone with the above-described... Figure 2 The electronic device with the hardware structure shown performs, such as Figure 3 As shown, the method includes S301 to S303.

[0054] S301. Obtain the training set.

[0055] The training set contains multiple sample images and their target annotation results. The sample images are taken from an aerial perspective, and the labeled targets in the target annotation results are small targets in the images.

[0056] For example, the sample image can be an image taken by a drone from a high altitude, and the labeled targets in the target annotation results can be vehicles, pedestrians, etc. in the sample image. These targets are relatively small in the aerial image, and traditional target recognition algorithms usually have difficulty recognizing them.

[0057] As one possible approach, aerial images captured by drones are typically stored in an aerial image database. Image annotators can label the images in this database and store the labeled images there. Electronic devices can then retrieve the original aerial images and their corresponding labeled images from the database and use these images as a training set to train a target detection model capable of accurately identifying minute targets.

[0058] S302. Construct a neural network model that includes an input network, a feature extraction network, and an output network.

[0059] The input network is used to input the sample image into the feature extraction network, which includes a first feature extraction subnetwork and a second feature extraction subnetwork. The first feature extraction subnetwork is used to extract multi-level features of the sample image and fuse the multi-level features to obtain a first fused image feature. The second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature. The output network is used to obtain the target prediction result of the sample image based on the second fused image feature output by the second feature extraction subnetwork.

[0060] Understandably, this application embodiment extracts multi-level features from sample images through a first feature extraction sub-network and fuses these multi-level features to achieve cross-scale feature stitching. Compared to traditional feature extraction operations, this application embodiment can fully utilize features at each level, enhance the understanding of global contextual information, and compensate for the limitations of small target resolution and background interference in UAV imagery. Furthermore, this application downsamples the first fused image features through a second feature extraction sub-network to obtain second fused image features. This application embodiment abandons traditional upsampling operations and uses cross-scale feature stitching to replace interpolation operations, reducing the number of model parameters and computational redundancy, ensuring model recognition accuracy while accelerating model training speed.

[0061] In some embodiments, the first feature extraction subnetwork includes a hierarchical feature extraction module, multiple convolutional channels, a channel reconstruction module, and an attention fusion module. The hierarchical feature extraction module is used to perform multi-level convolutional processing on the sample image to obtain multi-level features of the sample image. Each convolutional channel is used to receive the corresponding hierarchical features and transmit the corresponding hierarchical features to the channel reconstruction module. The channel reconstruction module is used to perform spatial resolution normalization processing on the received hierarchical features to obtain multi-level features with consistent spatial resolution. The attention fusion module is used to interactively fuse the multi-level features with consistent spatial resolution to obtain the first fused image features.

[0062] In practical applications, the first feature extraction subnetwork is also known as the Hierarchical Interactive Attention (HIA) module. For example... Figure 4 As shown, after the sample image arrives at the HIA, the hierarchical feature extraction module performs multi-level convolution processing on the sample image to obtain multi-level features (C1-C4) of the sample image. Each level feature has a corresponding convolution channel, and these convolution channels match the size of the corresponding level feature. The convolution channels can be the same or different, and this embodiment does not limit this. For example, Figure 4 The convolutional channels corresponding to C1-C3 can be denoted as M1, and the convolutional channel corresponding to C4 can be denoted as M3. Further, each convolutional channel transmits its corresponding hierarchical features to the channel reconstruction module (CR). The channel reconstruction module performs spatial resolution normalization on the received hierarchical features to obtain multi-level features with consistent spatial resolution. This allows the attention fusion module to interactively fuse these multi-level features with consistent spatial resolution to obtain the first fused image feature.

[0063] Specifically, the CR operation uses a tensor projection method from spatial location to channel dimension. While maintaining computational efficiency, it not only completes the spatial resolution standardization of multi-scale features, but also preserves the topological structure information of the original feature matrix. Figure 4 The squares corresponding to C3-C4 can represent hierarchical channel attention, and the squares corresponding to C3-C4 can represent hierarchical spatial attention. By interacting with local regions along the channel dimension and local regions along the spatial dimension, the contextual association of the channel dimension is enhanced, the global dependency of the spatial dimension is captured, the spatial distribution difference of multi-scale features is resolved, and the purpose of enhancing the information of shallow feature small targets is achieved.

[0064] Assuming the input feature mapping set of HIA is X, the above process can be described as follows: Taking the feature map of the i-th layer as an example, after obtaining... After that, hierarchical consistent multi-head attention is used to capture the global dependencies between local regions along the channel dimension and local regions along the spatial dimension. The above process can be described as follows: ; ; ; in, x ch and x sp These represent operations in the channel and spatial dimensions, respectively. Feature vectors representing channel-dimensional grouping, Feature vectors representing spatial dimension partitioning. The channel grouping matrix represents the layered splicing. This represents a spatial grouping matrix that is layered and spliced. P represents a consistent relative positional encoding across layers.

[0065] Understandably, the HIA module designed in this application abandons the complex multi-level feature upsampling process and its accompanying inter-layer data transmission architecture in traditional models. Such architectures are prone to feature dissipation when information is transferred across layers, while also increasing computational complexity and memory access overhead. This application adopts a multi-scale collaborative mechanism, firstly performing local spatial grouping processing on feature maps of heterogeneous scales, and then achieving efficient fusion of cross-resolution features through a single-step cross-layer neighborhood interaction mechanism. This design enables each scale layer to integrate the semantic information of adjacent layers with adaptive weight coefficients, significantly optimizing the utilization efficiency of computing resources while maintaining the integrity of feature representation.

[0066] In practical applications, the second feature extraction subnetwork is also known as the Frequency Adaptive Down-sampling (FAD) module. In UAV imagery scenarios, small targets (such as pedestrians, vehicles, and boats) typically occupy only a limited pixel area, and their discriminative features (such as edge contours and texture structures) are mainly concentrated in detail components such as edges and textures. However, traditional downsampling methods can lead to blurring of detailed information when reducing resolution, resulting in the loss of information from shallow features. At the same time, due to the dynamic changes in the aerial altitude of UAVs, the imaging scale of the same target exhibits significant differences. Traditional downsampling methods, due to their fixed operating modes, struggle to adaptively retain feature representations of targets at different structural scales and also suffer from computational redundancy. Blindly adopting aggressive downsampling strategies may reduce computational complexity, but at the cost of sacrificing the semantic integrity of small targets. In view of this, the embodiments of this application design a frequency domain adaptive downsampling module, which adopts a frequency domain separation and selective enhancement strategy to retain core edge and texture features during the downsampling stage, dynamically adjust the sampling strategy to adapt to targets of different sizes, solve the problem of multi-scale distribution differences, thereby improving the detection accuracy of small targets in UAV images, optimizing computational efficiency and simplifying the model structure.

[0067] like Figure 4 As shown, the first fused image feature is used as the input to the FAD module. The FAD module downsamples the first fused image feature to obtain the second fused image feature. Further, the output network decodes the second fused image feature through a decoder to obtain the target prediction result from the input sample image.

[0068] In some embodiments, the second feature extraction subnetwork includes a first downsampling module, a second downsampling module, a third downsampling module, and a downsampling fusion module, which are connected in parallel. The first downsampling module is used to collect key semantic features in the first fused image features to obtain a first downsampling feature map. The second downsampling module is used to perform max pooling on the first fused image features to obtain a second downsampling feature map. The third downsampling module is used to separate high-frequency information and low-frequency information in the first fused image features to obtain a third downsampling feature map. The downsampling fusion module is used to fuse the first downsampling feature map, the second downsampling feature map, and the third downsampling feature map to obtain the second fused image features.

[0069] like Figure 5 As shown, in the frequency-centric adaptive downsampling module FAD employing a dual-path parallel processing mechanism, the input feature Z (i.e., the first fused image feature) is first subjected to preliminary dimensionality reduction using a global average pooling layer with a kernel size of 2 and a stride of 1, resulting in feature maps Z1 and Z2. Furthermore, Z1 passes through a 3×3 convolutional layer with a stride of 2, compressing the spatial dimension while preserving key semantic features, outputting Z1. ， One branch of Z2 serves as the input to the frequency focusing module (FF), separating high-frequency information (such as edge textures in an image) from low-frequency information (such as the background in an image), denoted as Z. F Here, FFT and IFFT represent the Fast Fourier Transform and its inverse operation, respectively. Another path to Z2 uses max pooling to obtain Z2. ， And Z2 is convolved using 1×1 convolution. ， With Z F The number of channels is reduced to the required size, and finally compared with Z1. ， The fusion yields the second fused image features.

[0070] S303. Based on the target prediction results and target annotation results, adjust the parameters of each network in the neural network model to obtain the target detection model.

[0071] As one possible implementation, the electronic device can first input multiple sample images into a neural network model, and obtain the target prediction result output by the neural network model for each sample image. Further, the electronic device calculates a loss function value based on the target annotation results and target prediction results of the multiple sample images, and adjusts the parameters of each network in the neural network model according to the loss function value until the loss function value converges, thus obtaining the target detection model.

[0072] In some embodiments, the electronic device can calculate the loss function values ​​of the target annotation results and target prediction results for multiple sample images based on the Inner-CIOU loss function. The Inner-CIOU loss function scales the size of the original bounding box using a preset scaling factor to generate an auxiliary bounding box. The center point of the auxiliary bounding box is at the same position as the original bounding box, and the size of the auxiliary bounding box is smaller than that of the original bounding box. The loss of the Inner-CIOU loss function includes the CIOU loss between the original bounding box and the ground truth bounding box, as well as the CIOU loss between the auxiliary bounding box and the ground truth bounding box.

[0073] Specifically, the formula for the Inner-CIOU loss function can be: in, This represents the final Inner-CIOU loss. This represents the CIoU loss, where IoU represents the intersection-union ratio between the predicted bounding box and the target bounding box. The IOU value represents the auxiliary box, where, in, , , , , , , , Where inter represents the area of ​​the intersection between the auxiliary boxes corresponding to the ground truth bounding box and the auxiliary boxes corresponding to the predicted bounding box, and union represents the area of ​​the union of the auxiliary boxes corresponding to the ground truth bounding box and the auxiliary boxes corresponding to the predicted bounding box. , Represents the width and length of the actual bounding box, ( , ) represents the center coordinates of the true bounding box, ( , The coordinates () represent the center coordinates of the anchor frame, w and h represent the width and height of the anchor frame, respectively, and ratio is the corresponding scale factor. , These represent the x-coordinates of the auxiliary boxes corresponding to the real boxes. , These represent the y-coordinates of the four vertices of the auxiliary box corresponding to the real box. , These represent the x-coordinates of the auxiliary boxes corresponding to the predicted boxes. , These represent the ordinates of the auxiliary boxes corresponding to the predicted boxes.

[0074] It should be noted that in UAV image target detection, accurately and quickly determining the target bounding box position is crucial. Given the limitations of traditional loss functions in practical applications, this application introduces the Inner-CIOU loss function to improve the performance of the detection algorithm. Traditional CIoU loss functions struggle to provide stable and effective feedback when facing such a large scale, potentially leading to slow convergence and accuracy bottlenecks during model training. Furthermore, UAV image backgrounds are complex and varied; dense buildings in urban environments and occlusion by vegetation in natural scenes make the model susceptible to background interference when relying solely on traditional CIoU loss functions, hindering accurate target focusing and resulting in bounding box localization errors. The Inner-CIoU loss function's unique auxiliary bounding box mechanism dynamically adjusts the size of the auxiliary box according to different target scales, providing the model with richer and more accurate localization information. In complex backgrounds, by optimizing the intersection-union-ratio (IoU) calculation logic, the differences between the target and background are highlighted, helping the model more accurately distinguish target outlines. This significantly improves the detection accuracy of various targets in UAV imagery, accelerates algorithm convergence, shortens the model training cycle, and allows the model to reach ideal accuracy more quickly.

[0075] like Figure 6 As shown, after training the target detection model described above, this application embodiment also provides a method for detecting targets in UAV images. This method can be implemented by an individual possessing the aforementioned... Figure 2 The hardware structure shown can be executed by electronic devices, or by drones equipped with target detection models. For example... Figure 6 As shown, the method includes: S401. Acquire aerial images taken by the drone.

[0076] As one possible approach, aerial images captured by drones are typically stored in an aerial image database, from which electronic devices can retrieve drone aerial images.

[0077] As another possible implementation, electronic devices can also receive aerial images transmitted by the drone in real time while the drone is taking aerial photos, and perform small target detection on these aerial images.

[0078] S402. Target detection is performed based on the drone aerial images and the preset target detection model to obtain the target detection results, thus completing the detection and recognition of small targets in the drone images.

[0079] As one possible implementation, electronic devices can input drone aerial images into a trained target detection model to perform target detection, obtain target detection results, and complete the detection and recognition of small targets in drone images.

[0080] The target detection model employs an improved RT-DETR algorithm, introducing a first feature extraction subnetwork and a second feature extraction subnetwork into the RT-DETR feature extraction network. The first feature extraction subnetwork is used to extract multi-level features from the sample image and fuse these multi-level features to obtain a first fused image feature. The second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature. The second fused image feature enhances the model's understanding of features at each level, and the size of the second fused image feature is smaller than that of the first fused image feature.

[0081] RT-DETR (Real-Time Detection Transformer) is a real-time object detection model based on Transformer. Its core design concept addresses the slow convergence and high inference latency issues of traditional DETR models through efficient hybrid encoders, sparse query mechanisms, and dynamic decoding optimization. For a detailed introduction to object detection models, please refer to sections S301-S303 above; further details will not be provided here.

[0082] The foregoing primarily describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0083] In an exemplary embodiment, this application also provides a training apparatus for an object detection model. Figure 7 This is a schematic diagram illustrating the composition of a training apparatus for the target detection model provided in an embodiment of this application. Figure 7 As shown, the training device for the target detection model includes: an acquisition unit 401, a processing unit 402, and a training unit 403.

[0084] The acquisition unit 401 is used to acquire a training set; the training set contains multiple sample images and target annotation results of multiple sample images, the multiple sample images are images taken from an aerial perspective, and the labeled targets in the target annotation results are small targets in the images; the processing unit 402 is used to construct a neural network model including an input network, a feature extraction network, and an output network; the input network is used to input the sample images into the feature extraction network, the feature extraction network includes a first feature extraction subnetwork and a second feature extraction subnetwork, the first feature extraction subnetwork is used to extract multi-level features of the sample images and fuse the multi-level features to obtain a first fused image feature; the second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature; the output network is used to obtain the target prediction result of the sample image based on the second fused image feature output by the second feature extraction subnetwork; the training unit 403 is used to adjust the parameters of each network in the neural network model based on the target prediction result and the target annotation result to obtain a target detection model.

[0085] In one possible implementation, the first feature extraction subnetwork includes a hierarchical feature extraction module, multiple convolutional channels, a channel reconstruction module, and an attention fusion module. The hierarchical feature extraction module performs multi-level convolutional processing on the sample image to obtain multi-level features of the sample image. Each convolutional channel receives the corresponding hierarchical features and transmits the corresponding hierarchical features to the channel reconstruction module. The channel reconstruction module performs spatial resolution normalization processing on the received hierarchical features to obtain multi-level features with consistent spatial resolution. The attention fusion module performs interactive fusion on the multi-level features with consistent spatial resolution to obtain the first fused image features.

[0086] In one possible implementation, the second feature extraction subnetwork includes a first downsampling module, a second downsampling module, a third downsampling module, and a downsampling fusion module, which are connected in parallel. The first downsampling module is used to collect key semantic features from the first fused image features to obtain a first downsampling feature map. The second downsampling module is used to perform max pooling on the first fused image features to obtain a second downsampling feature map. The third downsampling module is used to separate high-frequency and low-frequency information from the first fused image features to obtain a third downsampling feature map. The downsampling fusion module is used to fuse the first downsampling feature map, the second downsampling feature map, and the third downsampling feature map to obtain the second fused image features.

[0087] In one possible implementation, the training unit 403 is specifically used for: inputting multiple sample images into the neural network model respectively, obtaining the target prediction result output by the neural network model for each sample image; calculating the loss function value based on the target annotation result and target prediction result of the multiple sample images; adjusting the parameters of each network in the neural network model based on the loss function value until the loss function value converges, thereby obtaining the target detection model.

[0088] In one possible implementation, training unit 403 is specifically used to: calculate the loss function value between the target annotation results and the target prediction results of multiple sample images based on the Inner-CIOU loss function; the formula for the Inner-CIOU loss function is as follows: in, This represents the final Inner-CIOU loss. This represents the CIoU loss, where IoU represents the intersection-union ratio between the predicted bounding box and the target bounding box. The IOU value represents the auxiliary box, where, in, , , , , , , , Where inter represents the area of ​​the intersection between the auxiliary boxes corresponding to the ground truth bounding box and the auxiliary boxes corresponding to the predicted bounding box, and union represents the area of ​​the union of the auxiliary boxes corresponding to the ground truth bounding box and the auxiliary boxes corresponding to the predicted bounding box. , Represents the width and length of the actual bounding box, ( , ) represents the center coordinates of the true bounding box, ( , The coordinates () represent the center coordinates of the anchor frame, w and h represent the width and height of the anchor frame, respectively, and ratio is the corresponding scale factor. , These represent the x-coordinates of the auxiliary boxes corresponding to the real boxes. , These represent the y-coordinates of the four vertices of the auxiliary box corresponding to the real box. , These represent the x-coordinates of the auxiliary boxes corresponding to the predicted boxes. , These represent the ordinates of the auxiliary boxes corresponding to the predicted boxes.

[0089] It should be noted that, Figure 7 The module division shown is illustrative and represents only one logical functional division; in actual implementation, other division methods are possible. For example, two or more functions can be integrated into a single processing module. These integrated modules can be implemented in hardware or as software functional units.

[0090] In an exemplary embodiment, this application also provides a computer-readable storage medium including software instructions that, when run on an electronic device, cause the electronic device to perform any of the methods provided in the above embodiments.

[0091] In an exemplary embodiment, this application also provides a computer program product containing computer execution instructions, which, when run on an electronic device, causes the electronic device to perform any of the methods provided in the above embodiments.

[0092] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer-executable instructions. When these computer-executable instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer-executable instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a solid-state disk (SSD), etc.

[0093] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0094] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0095] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for target detection in UAV images, characterized in that, The method includes: Acquire drone aerial images; Target detection is performed based on the drone aerial images and the preset target detection model to obtain target detection results, thus completing the detection and recognition of small targets in the drone images; The target detection model employs an improved RT-DETR algorithm, introducing a first feature extraction subnetwork and a second feature extraction subnetwork into the RT-DETR feature extraction network. The first feature extraction subnetwork is used to extract multi-level features from the sample image and fuse these multi-level features to obtain a first fused image feature. The second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature. The second fused image feature enhances the model's understanding of features at each level, and the size of the second fused image feature is smaller than that of the first fused image feature.

2. A training method for an object detection model, characterized in that, The method includes: Obtain a training set; the training set contains multiple sample images and target annotation results of multiple sample images, the multiple sample images are images taken from an aerial perspective, and the labeled targets in the target annotation results are small targets in the images; A neural network model is constructed, comprising an input network, a feature extraction network, and an output network. The input network is used to input sample images into the feature extraction network, which includes a first feature extraction subnetwork and a second feature extraction subnetwork. The first feature extraction subnetwork is used to extract multi-level features from the sample image and fuse these multi-level features to obtain a first fused image feature. The second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature. The output network is used to obtain the target prediction result of the sample image based on the second fused image feature output by the second feature extraction subnetwork. Based on the target prediction results and the target annotation results, the parameters of each network in the neural network model are adjusted to obtain the target detection model.

3. The method according to claim 2, characterized in that, The first feature extraction subnetwork includes a hierarchical feature extraction module, multiple convolutional channels, a channel reconstruction module, and an attention fusion module; the hierarchical feature extraction module is used to perform multi-level convolutional processing on the sample image to obtain multi-level features of the sample image; Each convolutional channel is used to receive the corresponding hierarchical features and transmit the corresponding hierarchical features to the channel reconstruction module; the channel reconstruction module is used to perform spatial resolution normalization processing on the received hierarchical features to obtain multi-level features with consistent spatial resolution; the attention fusion module is used to perform interactive fusion on the multi-level features with consistent spatial resolution to obtain the first fused image features.

4. The method according to claim 2, characterized in that, The second feature extraction subnetwork includes a first downsampling module, a second downsampling module, a third downsampling module, and a downsampling fusion module, which are connected in parallel. The first downsampling module is used to collect key semantic features in the first fused image features to obtain a first downsampling feature map. The second downsampling module is used to perform max pooling on the first fused image features to obtain a second downsampling feature map. The third downsampling module is used to separate high-frequency information and low-frequency information in the first fused image features to obtain a third downsampling feature map. The downsampling fusion module is used to fuse the first downsampling feature map, the second downsampling feature map, and the third downsampling feature map to obtain the second fused image features.

5. The method according to claim 2, characterized in that, The step of adjusting the parameters of each network in the neural network model based on the target prediction result and the target annotation result to obtain the target detection model includes: The multiple sample images are input into the neural network model respectively to obtain the target prediction result output by the neural network model for each sample image; The loss function value is calculated based on the target annotation results and target prediction results of the multiple sample images; The parameters of each network in the neural network model are adjusted according to the loss function value until the loss function value converges, thus obtaining the target detection model.

6. The method according to claim 5, characterized in that, The step of calculating the loss function value based on the target annotation results and target prediction results of the multiple sample images includes: Based on the Inner-CIOU loss function, the loss function values ​​of the target annotation results and target prediction results of the multiple sample images are calculated; the Inner-CIOU loss function scales the size of the original bounding box by a preset scaling factor to generate an auxiliary bounding box; the center point of the auxiliary bounding box is at the same position as the original bounding box, and the size of the auxiliary bounding box is smaller than the size of the original bounding box; the loss of the Inner-CIOU loss function includes the CIOU loss between the original bounding box and the ground truth box, and the CIOU loss between the auxiliary bounding box and the ground truth box.

7. The method according to claim 6, characterized in that, The formula for the Inner-CIOU loss function is as follows: in, This represents the final Inner-CIOU loss. This represents the CIoU loss, where IoU represents the intersection-union ratio between the predicted bounding box and the target bounding box. The IOU value represents the auxiliary box, where, in, , , , , , , , Where inter represents the area of ​​the intersection between the auxiliary boxes corresponding to the ground truth bounding box and the auxiliary boxes corresponding to the predicted bounding box, and union represents the area of ​​the union of the auxiliary boxes corresponding to the ground truth bounding box and the auxiliary boxes corresponding to the predicted bounding box. , Represents the width and length of the actual bounding box, ( , ) represents the center coordinates of the true bounding box, ( , The coordinates () represent the center coordinates of the anchor frame, w and h represent the width and height of the anchor frame, respectively, and ratio is the corresponding scale factor. , These represent the x-coordinates of the auxiliary boxes corresponding to the real boxes. , These represent the y-coordinates of the four vertices of the auxiliary box corresponding to the real box. , These represent the x-coordinates of the auxiliary boxes corresponding to the predicted boxes. , These represent the ordinates of the auxiliary boxes corresponding to the predicted boxes.

8. A training device for an object detection model, characterized in that, The device includes an acquisition unit, a processing unit, and a training unit; The acquisition unit is used to acquire a training set; the training set includes multiple sample images and target annotation results of multiple sample images, the multiple sample images are images taken from an aerial perspective, and the labeled targets in the target annotation results are small targets in the images; The processing unit is used to construct a neural network model including an input network, a feature extraction network, and an output network. The input network is used to input sample images into the feature extraction network, which includes a first feature extraction subnetwork and a second feature extraction subnetwork. The first feature extraction subnetwork is used to extract multi-level features from the sample image and fuse the multi-level features to obtain a first fused image feature. The second feature extraction subnetwork is used to downsample the first fused image feature to obtain a second fused image feature. The output network is used to obtain a target prediction result for the sample image based on the second fused image feature output by the second feature extraction subnetwork. The training unit is used to adjust the parameters of each network in the neural network model based on the target prediction result and the target annotation result to obtain the target detection model.

9. An electronic device, characterized in that, include: Processor and memory; The memory stores instructions that the processor can execute; When the processor is configured to execute the instructions, the electronic device performs the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The readable storage medium includes: software instructions; When the software instructions are executed in an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-7.

Images