Unmanned aerial vehicle night target detection method and device based on improved YOLOv11 and Retinex

Abstract

The application discloses a kind of based on improved YOLOv11 and Retinex's unmanned plane night target detection method and device, it is related to image processing field, including: constructing unmanned plane night target detection model and training, obtain the trained unmanned plane night target detection model;Unmanned plane night target detection model is on the basis of YOLOv11 model neck network's bottom-up path in the splicing layer is replaced with spatial gate control fusion module, replace the detection head in head network with spatial channel decoupling detection head, the regression branch and classification branch in spatial channel decoupling detection head all include two spatial channel blocks and a standard convolution layer connected in turn, and introduce Retinex guided auxiliary supervision branch in training process;Get the night image to be detected that unmanned plane is photographed and input to the trained unmanned plane night target detection model, obtain the detection result of each target in the night image to be detected.The application can effectively improve the detection precision of night target.

Description

Technical Field

[0001] This invention relates to the field of image processing, and specifically to a method and apparatus for nighttime target detection of unmanned aerial vehicles based on improved YOLOv11 and Retinex. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are increasingly undertaking perception tasks in scenarios such as traffic monitoring, emergency rescue, and security patrols. Because UAVs typically take images from high altitudes, targets in the images are often very small. Combined with insufficient lighting and high noise levels at night, targets can easily blend into the background, making detection very difficult. Currently, mainstream target detection methods fall into two categories: two-stage detectors (such as Faster R-CNN) and single-stage detectors (such as the YOLO series and SSD). The YOLO series is widely used on UAV platforms due to its speed and ease of deployment. YOLOv11 is a relatively new version of this series. Its backbone network uses stacked C3k2 modules, combined with SPPF spatial pyramid pooling and C2PSA attention mechanisms to extract multi-scale features. The detection model typically consists of three parts: the backbone responsible for feature extraction, the neck responsible for fusing multi-scale information (generally using an FPN+PAN structure), and the detection head responsible for outputting bounding boxes. In the Neck section, FPN transmits semantic information from higher levels to lower levels through top-down upsampling, while PAN transmits detailed information from lower levels to higher levels from bottom to top. When the two paths intersect at each scale, they are usually fused using the Concatenate operation (splicing along the channel dimension).

[0003] Existing work on nighttime target detection for drones can be broadly categorized into three directions. The first is low-light detection based on image enhancement. The idea is to brighten the dark image before feeding it to the detector. Examples include Retinexformer, which uses a Transformer for Retinex decomposition and enhancement, but the network is relatively heavy; EnlightenGAN, which uses a GAN for unsupervised enhancement, but the structure is unstable; and Zero-DCE, which uses curve estimation for lightweight enhancement, but only adjusts brightness. The second direction is joint enhancement and detection, such as DENet, PE-YOLO, 2PCNet, and PIA, which train enhancement and detection together. However, most of these methods do not utilize the physical prior of Retinex reflectance to guide the feature learning of the detection network. The third direction is multi-scale feature fusion and detection head improvement, but existing methods are structurally complex, computationally expensive, and lack specific handling of nighttime noise.

[0004] In summary, existing technologies have three main shortcomings:

[0005] (1) There is a disconnect between low-light enhancement and detection. There is no gradient transfer between the enhancement module and the detection network. The enhancement result cannot be optimized according to the detection requirements. Moreover, the enhancement module itself takes up additional inference time.

[0006] (2) Multi-scale feature fusion has no spatial discrimination capability. The Concatenate operation treats all spatial locations equally, while the noise distribution in nighttime images is uneven, and the low-quality features in the noise region will be directly transmitted to the downstream.

[0007] (3) The classification and localization tasks in the detection head conflict with each other, and the standard convolution has a large computational cost on small targets, which leads to the low detection accuracy of small targets at night. Summary of the Invention

[0008] The purpose of this application is to propose a method and device for nighttime target detection of unmanned aerial vehicles based on improved YOLOv11 and Retinex, which addresses the aforementioned technical problems.

[0009] In a first aspect, the present invention provides a method for nighttime target detection of unmanned aerial vehicles based on improved YOLOv11 and Retinex, comprising the following steps:

[0010] A UAV night target detection model was constructed and trained to obtain a trained UAV night target detection model. Based on the YOLOv11 model, the UAV night target detection model replaced the splicing layer in the bottom-up path of the neck network with a spatial gating fusion module, and replaced the detection head in the head network with a spatial channel decoupled detection head. The regression branch and classification branch in the spatial channel decoupled detection head both include two spatial channel blocks connected in sequence and a standard convolutional layer. A Retinex-guided auxiliary supervision branch was introduced during the training process.

[0011] The system acquires nighttime images of the target to be detected taken by a drone and inputs them into a trained drone nighttime target detection model. Multi-scale image features are extracted through the backbone network. The multi-scale image features are then input into the neck network for feature fusion to obtain multi-scale fused features. These multi-scale fused features are then input into the head network, and the detection head outputs the detection results of each target in the nighttime image after spatial channel decoupling.

[0012] Preferably, the spatial gating fusion module includes a first convolutional layer and a second convolutional layer arranged in parallel, and a spatial gating unit and a third convolutional layer connected in sequence. The input features of the spatial gating fusion module include high-level semantic features output from the upsampling layer and low-level detail features output from the backbone network. The high-level semantic features and low-level detail features are respectively input into the first convolutional layer and the second convolutional layer arranged in parallel to extract the first intermediate feature and the second intermediate feature. The first intermediate feature and the second intermediate feature are added element-wise and then input into the spatial gating unit to obtain the gating weight. The first intermediate feature and the second intermediate feature are weighted and fused according to the gating weight to obtain the first fused feature. The first fused feature is passed through the third convolutional layer to obtain the second fused feature. The feature obtained by concatenating the high-level semantic features and the low-level detail features is added element-wise to the second fused feature to obtain the output feature of the spatial gating fusion module.

[0013] As a preferred embodiment, the spatial gating unit includes a fourth convolutional layer, a fifth convolutional layer, and a first sigmoid function layer connected in sequence.

[0014] The calculation process for weighted fusion is as follows:

[0015] ;

[0016] in, and These represent the first intermediate feature and the second intermediate feature, respectively. Indicates the gating weight, This represents the first fusion feature.

[0017] As a preferred embodiment, the spatial channel block includes a channel segmentation unit and parallel processing branches and identity branches. The processing branches include a depth-separable convolutional layer and a seventh convolutional layer connected in sequence.

[0018] The input features of the spatial channel block are divided in half along the channel dimension by the channel segmentation unit to obtain the first sub-feature and the second sub-feature. The first sub-feature is input to the processing branch and passes through the depthwise separable convolutional layer and the seventh convolutional layer in sequence to obtain the processed features. The second sub-feature remains unchanged in the identity branch or is adjusted by the unbiased eighth convolutional layer to obtain the identity features. The processed features and the identity features are concatenated along the channel dimension to obtain the output features of the spatial channel block.

[0019] Preferably, the auxiliary supervision branch is used only during training and not during inference. The auxiliary supervision branch includes the teacher network and the student network. The student network includes the initial convolutional layer and the first feature extraction stage in the backbone network of the UAV night target detection model, as well as the reflectivity decoder.

[0020] The nighttime image of each sample in the training data is input into the initial convolutional layer and the first feature extraction stage of the backbone network of the UAV night target detection model to obtain the first-scale image features. The first-scale image features are input into the reflectance decoder to obtain the predicted reflectance.

[0021] The nighttime image of each sample in the training data is input into the teacher network to obtain the true reflectance;

[0022] The total loss function used during the training of the UAV night target detection model for:

[0023] ;

[0024] in, This represents the original detection loss of the YOLOv11 model. Indicates the predicted reflectance. Indicates true reflectance. Represents the balance coefficient. This represents the L1 norm.

[0025] Preferably, the teacher network includes a pre-trained DecomNet model, and the reflectivity component of its output is selected as the true reflectivity.

[0026] The reflectivity decoder includes a first convolutional module, a second convolutional module, a bilinear upsampling layer, a ninth convolutional layer, and a second sigmoid function layer connected in sequence. Both the first and second convolutional modules use a tenth convolutional layer and a ReLU activation function layer connected in sequence.

[0027] Secondly, the present invention provides a nighttime target detection device for unmanned aerial vehicles based on improved YOLOv11 and Retinex, comprising:

[0028] The model building module is configured to build and train a UAV night target detection model to obtain a trained UAV night target detection model. The UAV night target detection model is based on the YOLOv11 model. The splicing layer in the bottom-up path of the neck network is replaced with a spatial gating fusion module, and the detection head in the head network is replaced with a spatial channel decoupled detection head. The regression branch and classification branch in the spatial channel decoupled detection head both include two spatial channel blocks connected in sequence and a standard convolutional layer. A Retinex-guided auxiliary supervision branch is introduced during the training process.

[0029] The detection module is configured to acquire nighttime images of the target to be detected taken by the UAV and input them into a trained UAV nighttime target detection model. Multi-scale image features are extracted through the backbone network. The multi-scale image features are input into the neck network for feature fusion to obtain multi-scale fused features. The multi-scale fused features are input into the head network, and the detection head outputs the detection results of each target in the nighttime image to be detected after spatial channel decoupling.

[0030] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0031] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0032] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) The UAV night target detection method based on improved YOLOv11 and Retinex mentioned in this invention is improved on the basis of YOLOv11 model. In the neck network, a spatial gated fusion module is used to replace the original splicing layer in the bottom-up path of the neck network. This spatial gated fusion module improves the spatial perception capability in the fusion process and can automatically assign different complementary weights to different positions. Noise-heavy areas transmit fewer features, while areas with targets retain more semantic information. The design of residual connection ensures that the fusion will not destroy the original features.

[0035] (2) The UAV nighttime target detection method based on improved YOLOv11 and Retinex mentioned in this invention replaces the original detection head with a spatial channel decoupled detection head. This spatial channel decoupled detection head divides classification and localization into two independent branches, avoiding the contradiction of the two competing for features. Both branches use spatial channel blocks, and each spatial channel block only performs real convolution calculation on half of the channels. The identity branch hardly consumes any computation, so the computation is about half that of standard convolution, and the parameter efficiency is higher. At the same time, the identity branch retains the original information as is and does not lose features.

[0036] (3) The UAV nighttime target detection method based on improved YOLOv11 and Retinex mentioned in this invention introduces a Retinex-guided auxiliary supervision branch during training, enabling the backbone network to learn features independent of illumination. Since the reflectivity component output by the teacher network is itself illumination-invariant as a supervision signal, the backbone network can better focus on the features of the object itself under its guidance, unaffected by changes in nighttime illumination. The auxiliary supervision branch is completely removed during inference, without increasing the computational load of inference. The Retinex-guided auxiliary supervision branch can improve the quality of the input features of the backbone network, the spatial gating fusion module can improve the intermediate fusion quality of the neck network, and the spatial channel decoupling detection can improve the final detection quality of the head network. These three aspects do not conflict with each other and work together, improving the UAV's nighttime target detection capability without significantly increasing the model complexity. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating an embodiment of the UAV nighttime target detection method based on improved YOLOv11 and Retinex, which is an example of this application.

[0039] Figure 2 This is a schematic diagram of the overall architecture of a UAV night target detection model based on the improved YOLOv11 and Retinex UAV night target detection method, which is an embodiment of this application.

[0040] Figure 3 This is a schematic diagram of the spatial gating fusion module of the UAV nighttime target detection method based on the improved YOLOv11 and Retinex, which is an embodiment of this application.

[0041] Figure 4 This is a schematic diagram of the spatial channel decoupled detection head of the UAV nighttime target detection method based on the improved YOLOv11 and Retinex, which is an embodiment of this application.

[0042] Figure 5 This is a schematic diagram of the spatial channel block structure of the UAV nighttime target detection method based on the improved YOLOv11 and Retinex, which is an embodiment of this application.

[0043] Figure 6This is a schematic diagram of the Retinex-guided auxiliary supervision branch in an embodiment of the UAV night target detection method based on the improved YOLOv11 and Retinex, which is an embodiment of this application.

[0044] Figure 7 A schematic diagram of a UAV nighttime target detection device based on improved YOLOv11 and Retinex, which is an embodiment of this application;

[0045] Figure 8 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0047] Figure 1 This application illustrates an embodiment of a nighttime target detection method for drones based on improved YOLOv11 and Retinex, comprising the following steps:

[0048] S1. Construct and train a UAV night target detection model to obtain a trained UAV night target detection model. Based on the YOLOv11 model, the UAV night target detection model replaces the splicing layer in the bottom-up path of the neck network with a spatial gating fusion module, and replaces the detection head in the head network with a spatial channel decoupled detection head. The regression branch and classification branch in the spatial channel decoupled detection head both include two spatial channel blocks connected in sequence and a standard convolutional layer. A Retinex-guided auxiliary supervision branch is introduced during training.

[0049] Specifically, such as Figure 2As shown, the UAV night target detection model proposed in the embodiments of this application is an improvement on the existing YOLOv11 model. The existing YOLOv11 model includes a backbone network, a neck network, and a head network connected in sequence. The neck network adopts a bidirectional feature fusion structure composed of bottom-up and top-down paths, and the head network includes four detection heads of different scales. The UAV night target detection model proposed in this application keeps the backbone network of the existing YOLOv11 model unchanged, replaces the concatenation layer in the bottom-up path of the neck network of the existing YOLOv11 model with a spatial gated fusion module (SGF), while keeping the concatenation layer in the top-down path unchanged, and replaces the detection head in the head network with a spatial channel decoupled detection head (SCDetect Head). The first two standard convolutional layers in the regression branch and the first two depthwise separable convolutional layers in the classification branch of the detection head of the existing YOLOv11 model are replaced with spatial channel blocks (SCB) to obtain the spatial channel decoupled detection head. Furthermore, a Retinex-guided auxiliary supervision branch is introduced during the training of the UAV night target detection model, but this auxiliary supervision branch is not used during inference.

[0050] The following details the structure of the spatial gating fusion module, the spatial channel decoupling detection head, and the Retinex-guided auxiliary supervision branch.

[0051] In a specific embodiment, the spatial gating fusion module includes a first convolutional layer and a second convolutional layer arranged in parallel, as well as a spatial gating unit and a third convolutional layer connected in sequence. The input features of the spatial gating fusion module include high-level semantic features output from the upsampling layer and low-level detail features output from the backbone network. The high-level semantic features and low-level detail features are respectively input into the first convolutional layer and the second convolutional layer arranged in parallel to extract a first intermediate feature and a second intermediate feature. The first intermediate feature and the second intermediate feature are added element-wise and then input into the spatial gating unit to obtain the gating weight. The first intermediate feature and the second intermediate feature are weighted and fused according to the gating weight to obtain a first fused feature. The first fused feature is passed through the third convolutional layer to obtain a second fused feature. The feature obtained by concatenating the high-level semantic features and the low-level detail features is added element-wise to the second fused feature to obtain the output feature of the spatial gating fusion module.

[0052] In a specific embodiment, the spatial gating unit includes a fourth convolutional layer, a fifth convolutional layer, and a first Sigmoid function layer connected in sequence;

[0053] The calculation process for weighted fusion is as follows:

[0054] ;

[0055] in, and These represent the first intermediate feature and the second intermediate feature, respectively. Indicates the gating weight, This represents the first fusion feature.

[0056] Specifically, in the embodiments of this application, the spatially gated fusion module is deployed in three fusion nodes in the bottom-up path (corresponding to the FPN structure) of the neck network, replacing the original concatenate operation, i.e., the splicing layer. The top-down path (corresponding to the PAN structure) retains the original concatenate operation. The spatially gated fusion module receives two features from adjacent layers, i.e., high-level semantic features. and low-level detail features High-level semantic features The output features of the upsampled layer located in front of the spatial gated fusion module are low-level detailed features. For the first three scales of image features output by the backbone network, low-level detail features are used in the first spatially gated fusion module of the top-down path. The third-scale image features output by the third C3k2 module in the backbone network; low-level detail features for the second spatially gated fusion module of the top-down path. The second-scale image features are output by the second C3k2 module in the backbone network; low-level detail features are output by the third spatially gated fusion module for the top-down path. The first-scale image features output by the first C3k2 module in the backbone network are used as inputs; while the fourth-scale image features output by the C2PSA module in the backbone network are used as inputs to the first upsampling layer of the top-down path.

[0057] Furthermore, such as Figure 3 As shown, for the spatial gating fusion module, two sets of parallel convolutional layers with a kernel size of 1×1 are first used to encode the high-level semantic features. and low-level detail features The projection is onto a uniform hidden dimension, where the number of channels is 0.5 times the smaller of the two input channels. The first and second intermediate features obtained after projection are element-wise summed and then input into a spatial gating unit. This spatial gating unit consists of a fourth convolutional layer with a 3×3 kernel (reducing the number of channels to half that of the hidden channels), followed by a fifth convolutional layer with a 1×1 kernel (reducing to 1 channel), and then a first sigmoid function layer. Each spatial location generates a weight value between 0 and 1, resulting in the gating weights. Then, the first and second intermediate features are weighted and fused using a complementary method based on the gating weight: the first intermediate feature is multiplied by the gating weight, and the second intermediate feature is multiplied by 1 minus the result of the gating weight. Finally, the two are added together. This method preserves more high-level semantic information in the target region and more low-level details in noisy regions. The resulting first fused feature is then projected back to the original channel dimension through a third convolutional layer with a kernel size of 1×1, and combined with the high-level semantic features. and low-level detail features The directly spliced ​​features are then residually connected and added together to obtain the output features of the spatial gating fusion module, ensuring that the original features are not destroyed even when the gating weights are not ideal.

[0058] Specifically, such as Figure 4 As shown, the Spatial Channel Decoupled Detection Head (SCDetect Head) follows the decoupling approach, separating the classification and regression tasks into two independent branches: a classification branch and a regression branch, to avoid the classification and localization tasks competing for the same features. The neck network outputs fused features at four scales: P2, P3, P4, and P5. After these fused features are input into the detector head, they are fed into the regression branch (Box Branch) for generating bounding boxes and the classification branch (Classification Branch) for generating categories, respectively. Each branch consists of two concatenated Spatial Channel Blocks (SCBs) followed by a standard convolutional layer with a 1×1 kernel size as the output layer.

[0059] Let ch be the number of input feature channels at the current scale, nc be the number of dataset classes, and reg_max be the number of discrete intervals for the Distribution Focal Loss (DFL) used for the bounding box (default value is 16; each of the four coordinates of the bounding box is represented by reg_max discrete probability values, so the regression branch outputs a total of 4×reg_max channels). The number of intermediate channels in the regression branch is c_reg = max(16, ch / 4, 4×reg_max), where max represents the maximum value, and 4×reg_max channels are output for bounding box coordinate prediction; the number of intermediate channels in the classification branch is c_cls = max(ch, min(nc,100)), and nc channels are output for class confidence prediction.

[0060] Both the classification and regression branches use spatial channel blocks to replace the original two structures. Therefore, the number of parameters and computation of the entire detection head is significantly reduced compared to the detection head of the existing YOLOv11 model. Furthermore, classification and localization follow independent feature paths and do not interfere with each other, which is beneficial to improving the detection accuracy of small targets.

[0061] In a specific embodiment, the spatial channel block includes a channel segmentation unit and parallel processing branches and identity branches. The processing branches include a depth-separable convolutional layer and a seventh convolutional layer connected in sequence.

[0062] The input features of the spatial channel block are divided in half along the channel dimension by the channel segmentation unit to obtain the first sub-feature and the second sub-feature. The first sub-feature is input to the processing branch and passes through the depthwise separable convolutional layer and the seventh convolutional layer in sequence to obtain the processed features. The second sub-feature remains unchanged in the identity branch or is adjusted by the unbiased eighth convolutional layer to obtain the identity features. The processed features and the identity features are concatenated along the channel dimension to obtain the output features of the spatial channel block.

[0063] Specifically, such as Figure 5 As shown, the number of input channels in the spatial channel block proposed in this application is denoted as c_in and the number of output channels is denoted as c_out. In the spatial channel block, the input features of the spatial channel block are first split in half at the middle position along the channel dimension by a channel segmentation unit using a Split operation: the first half of the channels are used as the processing branch, and the second half of the channels are used as the identity branch. The processing branch adopts a serial structure: first, a depthwise separable convolutional layer with a kernel size of 3×3 (DWConv, i.e., grouped convolution with the number of groups equal to the number of channels, each channel is equipped with an independent convolutional kernel with a kernel size of 3×3) is used to extract the spatial geometric information (such as boundaries and shapes) of the target. The number of input channels and the number of output channels of this depthwise separable convolutional layer remain unchanged at c_in / 2; then, a seventh convolutional layer with a kernel size of 1×1 is used to perform cross-channel fusion, and the number of channels is adjusted from c_in / 2 to c_out / 2. The identity branch performs virtually no processing: when the number of input and output channels matches, it directly uses the Identity identity mapping to preserve the features as is (in this case, the output c_in / 2 = c_out / 2). When they do not match, it uses a lightweight adjustment with an unbiased 1×1 kernel in the eighth convolutional layer (mapping the number of channels to c_out / 2). The outputs of the two branches are concatenated along the channel dimension using Concat, resulting in c_out / 2 channels output by both the processing branch and the identity branch, which together equal c_out. Since the actual convolution computation only occurs on half of the channels in the processing branch, the identity branch incurs almost no computational cost. Therefore, the computational cost of a spatial channel block is approximately half that of an equivalent standard convolutional layer; simultaneously, the identity branch completely preserves the original information, avoiding information loss.

[0064] In a specific embodiment, the auxiliary supervision branch is used only during the training process and not during the inference process. The auxiliary supervision branch includes the teacher network and the student network. The student network includes the initial convolutional layer and the first feature extraction stage and the reflectivity decoder in the backbone network of the UAV night target detection model.

[0065] The nighttime image of each sample in the training data is input into the initial convolutional layer and the first feature extraction stage of the backbone network of the UAV nighttime target detection model to obtain the first-scale image features. The first-scale image features are then input into the reflectance decoder to obtain the predicted reflectance.

[0066] The nighttime image of each sample in the training data is input into the teacher network to obtain the true reflectance;

[0067] The total loss function used during the training of the UAV night target detection model for:

[0068] ;

[0069] in, This represents the original detection loss of the YOLOv11 model. Indicates the predicted reflectance. Indicates true reflectance. Represents the balance coefficient. This represents the L1 norm.

[0070] In a specific embodiment, the teacher network includes a pre-trained DecomNet model, and selects the reflectance component of its output as the true reflectance.

[0071] The reflectivity decoder includes a first convolutional module, a second convolutional module, a bilinear upsampling layer, a ninth convolutional layer, and a second sigmoid function layer connected in sequence. Both the first and second convolutional modules use a tenth convolutional layer and a ReLU activation function layer connected in sequence.

[0072] Specifically, the Retinex-guided auxiliary supervision branch proposed in the embodiments of this application only participates during the training phase and is removed during inference. Retinex theory decomposes an image S into the product of illumination L and reflectivity R, where reflectivity R is independent of illumination and only reflects the object's intrinsic properties. The embodiments of this application utilize this principle to design the auxiliary supervision structure of the teacher-student network. For example... Figure 6 As shown, in one example, the teacher network uses a pre-trained DecomNet model. First, it concatenates the maximum channel value from the RGB channels of the input image with the original three channels to form a 4-channel input. Then, it passes through a 9×9 convolutional layer and uses the ReLU activation function (channel count increases from 4 to 64) to extract shallow features. Next, it passes through five 3×3 convolutional layers (channel count increases from 64 to 64) for deep modeling. Finally, it uses a 3×3 convolutional layer (channel count increases from 64 to 4) for output. The results of the first three channels and the last channel are then processed through a Sigmoid activation function to obtain the reflectance component. And the illumination component L. The teacher network is pre-trained on low-light-normal-light image pairs, and its parameters are frozen during subsequent detection training, with the reflectance component... The actual reflectance is used in the training process of the UAV night target detection model. The student network consists of a lightweight reflectance decoder and the initial convolutional layer and the first feature extraction stage of the backbone network of the UAV night target detection model. The first-scale image features extracted from the initial convolutional layer and the first feature extraction stage of the backbone network are input to the reflectance decoder. It first passes through two sets of tenth convolutional layers with a kernel size of 3×3 and uses the ReLU activation function, then passes through a 4x bilinear upsampling layer to restore the input resolution, and finally outputs the predicted reflectance through a ninth convolutional layer with a kernel size of 3×3 and a second sigmoid function layer. Further utilize this predicted reflectance With reflectivity component An auxiliary loss is constructed by combining a balance coefficient, in one example where the balance coefficient λ is 0.2.

[0073] The total loss function used during the training of the UAV night target detection model is the sum of the original detection loss and the auxiliary loss of the YOLOv11 model. During inference, this auxiliary branch is completely removed without adding any extra overhead. After training the UAV night target detection model using this total loss function, the trained UAV night target detection model is obtained.

[0074] S2: Acquire the nighttime image to be detected taken by the UAV and input it into the trained UAV nighttime target detection model. Multi-scale image features are extracted through the backbone network. The multi-scale image features are input into the neck network for feature fusion to obtain multi-scale fused features. The multi-scale fused features are input into the head network. After spatial channel decoupling, the detection head outputs the detection results of each target in the nighttime image to be detected.

[0075] Specifically, in the embodiments of this application, after the trained UAV night target detection model is deployed, the night images to be detected taken by the UAV can be input into the trained UAV night target detection model for inference to obtain the detection results of each target. The detection results include the bounding box coordinates, category and confidence of the target. The categories of the detected targets include vehicles or people.

[0076] In one example, the present invention is deployed as a complete UAV-borne target detection system for nighttime road traffic monitoring. The entire detection system consists of five parts: a UAV flight platform, an image acquisition module, an onboard edge computing unit, a detection model, and a result output and alarm module. The UAV flight platform is responsible for performing hovering or cruising flight missions over nighttime roads, carrying the image acquisition module and the onboard edge computing unit. The image acquisition module uses a visible light camera (an infrared camera can also be used if needed) to acquire images or video streams of the road scene below in real time during flight. Due to insufficient light at night, the acquired images generally have characteristics such as low illumination, high noise, and small target size. The onboard edge computing unit uses embedded edge computing devices such as the NVIDIA Jetson series. Its computing power and power consumption are limited, therefore the deployed detection model must be lightweight. Therefore, the trained UAV nighttime target detection model is deployed on an airborne edge computing unit. The UAV, equipped with this unit, cruises on roads at night, capturing real-time nighttime images using its image acquisition module. These images are then input into the trained UAV nighttime target detection model on the airborne edge computing unit for inference. The model outputs the detection results for each target in the nighttime image and transmits them in two paths: one triggers an alarm locally, and the other is transmitted back to the ground monitoring station via a wireless communication link. This supports applications such as nighttime traffic violation detection, accident warnings, road condition monitoring, and traffic flow statistics. Since the Retinex-guided auxiliary supervision branch has been completely removed during the inference stage, retaining only the main detection path, nighttime detection capabilities are maintained without increasing inference overhead.

[0077] This detection system has clear engineering value for nighttime traffic monitoring tasks: it can detect and locate vehicles and pedestrians on roads in real time, and effectively identify small targets at a distance (such as distant vehicles and pedestrians) even under low-light conditions. This supports applications such as nighttime traffic violation detection (e.g., illegal parking, driving against traffic), traffic accident early warning (e.g., abnormal parking, pedestrians accidentally entering the motor vehicle lane), nighttime road condition monitoring, and traffic flow statistics. Because the trained UAV nighttime target detection model is lightweight, has a small number of parameters and computational requirements, it can run in real time on the onboard edge computing unit, meeting the real-time requirements of UAV nighttime inspections.

[0078] The technical solution of the present invention will be described below through specific embodiments.

[0079] Example 1

[0080] Embodiment 1 of this application constructs an AeroNight target detection dataset for UAV nighttime scenes. The construction process is as follows:

[0081] First, original images were selected from two publicly available drone datasets, UAVDT and VisDrone, and the grayscale mean of each image was calculated. Images with a grayscale mean of less than 80 were identified as nighttime images and retained, thus filtering out samples that met the characteristics of low light at night.

[0082] The original annotation categories were then uniformly mapped to four categories: pedestrians, small vehicles, medium vehicles, and large vehicles. Each of the selected images was manually checked using a self-built annotation review tool to ensure annotation quality.

[0083] Finally, all the selected images were shuffled using a fixed random seed and divided into training, validation, and test sets in a ratio of approximately 6880:766:1932. The resulting dataset contains 9578 nighttime drone images, of which approximately 63% are small targets with an area smaller than 32×32 pixels, exhibiting typical small target-dominated characteristics, making it suitable for training and evaluation of nighttime small target detection. The embodiments of this application use YOLOv11s as the baseline model for improvement. The backbone adopts the standard configuration of YOLOv11s, sequentially performing 3×3 convolutional downsampling with a stride of 2 (64→128→256→512→1024 channels). After each downsampling, a C3k2 module is used to extract features. Finally, an SPPF (pooling kernel size of 5×5) and a C2PSA attention module are used to output image features at four scales: first scale (128 channels, stride 4), second scale (256 channels, stride 8), third scale (512 channels, stride 16), and fourth scale (1024 channels, stride 32). The Neck's FPN path deploys spatial gating fusion modules at three nodes: P5→P4, P4→P3, and P3→P2. Taking P5→P4 as an example, the P5 feature is first upsampled by 2 times and then fed into the spatial gating fusion module together with the P4 feature from the backbone. The hidden channels of the spatial gating fusion module are min(512,512)×0.5=256. The same applies to P4→P3 and P3→P2. The features after FPN fusion are then processed by the C3k2 module, and the PAN path maintains the standard concatenate operation. Finally, the Neck outputs fused features at four scales: P2 (128 channels), P3 (256 channels), P4 (512 channels), and P5 (1024 channels). In the Retinex auxiliary supervision branch, the parameters of the teacher network are frozen after pre-training, with a balance coefficient λ=0.2. The SCDetect head receives the fused features at four scales: P2, P3, P4, and P5, corresponding to the number of channels ch [128, 256, 512, 1024]. The regression branch (cv2) has a maximum of 16, 128 / 4, and reg_max×4 intermediate channels (c2). It consists of two concatenated SCB layers followed by a 1×1 standard convolutional output, resulting in 4×reg_max channels for bounding box regression. The classification branch (cv3) also has a maximum of 128, min(nc, 100) intermediate channels (c3). It similarly consists of two concatenated SCB layers followed by a 1×1 standard convolutional output, resulting in nc channels for classification. Each SCB in both branches halves the input features along the channels. The processing branch uses a 3×3 depthwise separable convolutional layer (number of groups equal to number of channels) concatenated with a 1×1 convolutional layer. The identity branch uses either an Identity identity mapping or an unbiased 1×1 convolutional layer. The outputs of the two branches are concatenated along the channel dimension. All four scales share the same detection head structure.

[0084] The training process used the PyTorch 2.0 framework, Ultralytics 8.3.83 platform, and an RTX 3090 GPU. The input image resolution was 640×640. The AdamW optimizer was used with a learning rate of 0.005 and a batch size of 16. After a 2-epoch warm-up, the training proceeded for 200 epochs. During inference, the Retinex auxiliary branch was removed, and only the main detection path was used. The input was a nighttime image captured by a drone, and the output included bounding box coordinates, class, and confidence score.

[0085] Example 2

[0086] In Embodiment 2 of this application, the baseline model is replaced from YOLOv11s to YOLOv11m (medium scale). The number of channels in the Backbone and Neck is increased accordingly. The hidden dimension of SGF is automatically adjusted with the input channels (still taking min(c_h,c_l)×0.5). The number of channels in SCDetect is automatically adapted to the baseline channels. The module structure and deployment location are exactly the same as in Embodiment 1 of this application, and the training parameters and inference process are consistent. This shows that the present invention does not depend on a baseline model of a specific size, can be adapted to different versions of the YOLOv11 series (n / s / m / l / x), and can also be transferred to other detectors with FPN+PAN structures.

[0087] The data mentioned in the embodiments of this application were all obtained in a legal and compliant manner, and their application scenarios are also legal and compliant.

[0088] Further reference Figure 7 As an implementation of the methods shown in the above figures, this application provides an embodiment of a UAV nighttime target detection device based on improved YOLOv11 and Retinex. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0089] This application provides a drone nighttime target detection device based on improved YOLOv11 and Retinex, including:

[0090] Model building module 1 is configured to build and train a UAV night target detection model to obtain a trained UAV night target detection model. The UAV night target detection model is based on the YOLOv11 model. The splicing layer in the bottom-up path of the neck network is replaced with a spatial gating fusion module, and the detection head in the head network is replaced with a spatial channel decoupled detection head. The regression branch and classification branch in the spatial channel decoupled detection head both include two spatial channel blocks connected in sequence and a standard convolutional layer. A Retinex-guided auxiliary supervision branch is introduced during training.

[0091] Detection module 2 is configured to acquire nighttime images of the target to be detected taken by the UAV and input them into the trained UAV nighttime target detection model. Multi-scale image features are extracted through the backbone network. The multi-scale image features are input into the neck network for feature fusion to obtain multi-scale fused features. The multi-scale fused features are input into the head network, and the detection head outputs the detection results of each target in the nighttime image to be detected after spatial channel decoupling.

[0092] Figure 8 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 8 As shown, the electronic device of this embodiment includes a processor 801 and a memory 802; wherein the memory 802 is used to store computer execution instructions; and the processor 801 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0093] Alternatively, the memory 802 can be either standalone or integrated with the processor 801.

[0094] When the memory 802 is set up independently, the electronic device also includes a bus 803 for connecting the memory 802 and the processor 801.

[0095] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 801, implement the above method.

[0096] This invention also provides a computer program product, including a computer program that, when executed by a processor 801, implements the above-described method.

[0097] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0098] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0099] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0100] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 801 to execute some steps of the methods of the various embodiments of this application.

[0101] It should be understood that the processor 801 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 801 can be any conventional processor 801. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 801 executing the steps, or as a combination of hardware and software modules within the processor 801 executing the steps.

[0102] The memory 802 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0103] Bus 803 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 803 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 803 in the accompanying drawings of this application is not limited to only one bus 803 or one type of bus 803.

[0104] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0105] An exemplary storage medium is coupled to a processor 801, enabling the processor 801 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor 801. The processor 801 and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor 801 and the storage medium can exist as discrete components in an electronic device or a host device.

[0106] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for nighttime target detection of unmanned aerial vehicles (UAVs) based on improved YOLOv11 and Retinex, characterized in that, Includes the following steps: A drone nighttime target detection model is constructed and trained to obtain a trained drone nighttime target detection model. The drone nighttime target detection model is based on the YOLOv11 model. The splicing layer in the bottom-up path of the neck network is replaced with a spatial gating fusion module, and the detection head in the head network is replaced with a spatial channel decoupled detection head. The regression branch and classification branch in the spatial channel decoupled detection head both include two spatial channel blocks connected in sequence and a standard convolutional layer. A Retinex-guided auxiliary supervision branch is introduced during the training process. The nighttime image to be detected is acquired by the UAV and input into the trained UAV nighttime target detection model. Multi-scale image features are extracted through the backbone network. The multi-scale image features are input into the neck network for feature fusion to obtain multi-scale fused features. The multi-scale fused features are input into the head network, and the detection results of each target in the nighttime image to be detected are output through the spatial channel decoupling detection head.

2. The UAV nighttime target detection method based on improved YOLOv11 and Retinex according to claim 1, characterized in that, The spatial gating fusion module includes a first convolutional layer and a second convolutional layer arranged in parallel, as well as a spatial gating unit and a third convolutional layer connected in sequence; the input features of the spatial gating fusion module include high-level semantic features output from the upsampling layer and low-level detail features output from the backbone network. The high-level semantic features and low-level detail features are respectively input into a first convolutional layer and a second convolutional layer set in parallel to extract a first intermediate feature and a second intermediate feature. The first intermediate feature and the second intermediate feature are added element by element and then input into the spatial gating unit to obtain gating weights. The first intermediate feature and the second intermediate feature are weighted and fused according to the gating weights to obtain a first fused feature. The first fused feature is passed through the third convolutional layer to obtain a second fused feature. The feature obtained by concatenating the high-level semantic features and the low-level detail features is added element by element to the second fused feature to obtain the output feature of the spatial gating fusion module.

3. The UAV nighttime target detection method based on improved YOLOv11 and Retinex according to claim 2, characterized in that, The spatial gating unit includes a fourth convolutional layer, a fifth convolutional layer, and a first Sigmoid function layer connected in sequence; The calculation process for the weighted fusion is as follows: ； in, and These represent the first intermediate feature and the second intermediate feature, respectively. Indicates the gating weight, This represents the first fusion feature.

4. The UAV nighttime target detection method based on improved YOLOv11 and Retinex according to claim 1, characterized in that, The spatial channel block includes a channel segmentation unit and parallel processing branches and identity branches. The processing branches include a depth-separable convolutional layer and a seventh convolutional layer connected in sequence. The input features of the spatial channel block are divided in half along the channel dimension by the channel segmentation unit to obtain the first sub-feature and the second sub-feature. The first sub-feature is input to the processing branch and passes through the depthwise separable convolutional layer and the seventh convolutional layer in sequence to obtain the processed features. The second sub-feature remains unchanged in the identity branch or is adjusted by the unbiased eighth convolutional layer to obtain the identity features. The processed features and the identity features are concatenated along the channel dimension to obtain the output features of the spatial channel block.

5. The UAV nighttime target detection method based on improved YOLOv11 and Retinex according to claim 1, characterized in that, The auxiliary supervision branch is used only during training and not during inference. The auxiliary supervision branch includes a teacher network and a student network. The student network includes the initial convolutional layer and the first feature extraction stage and the reflectivity decoder in the backbone network of the UAV night target detection model. The nighttime image of each sample in the training data is input into the initial convolutional layer and the first feature extraction stage of the backbone network of the UAV nighttime target detection model to obtain image features at the first scale. The image features at the first scale are input into the reflectance decoder to obtain the predicted reflectance. The nighttime image of each sample in the training data is input into the teacher network to obtain the true reflectance; The total loss function used by the UAV night target detection model during training. for: ； in, This represents the original detection loss of the YOLOv11 model. Indicates the predicted reflectance. Indicates true reflectance. Represents the balance coefficient. This represents the L1 norm.

6. The UAV nighttime target detection method based on improved YOLOv11 and Retinex according to claim 5, characterized in that, The teacher network includes a pre-trained DecomNet model, and selects the reflectance component of its output as the true reflectance. The reflectivity decoder includes a first convolutional module, a second convolutional module, a bilinear upsampling layer, a ninth convolutional layer, and a second sigmoid function layer connected in sequence. Both the first and second convolutional modules employ a tenth convolutional layer and a ReLU activation function layer connected in sequence.

7. A nighttime target detection device for unmanned aerial vehicles based on improved YOLOv11 and Retinex, characterized in that, include: The model building module is configured to build and train a UAV night target detection model to obtain a trained UAV night target detection model. The UAV night target detection model is based on the YOLOv11 model. The splicing layer in the bottom-up path of the neck network is replaced with a spatial gating fusion module, and the detection head in the head network is replaced with a spatial channel decoupled detection head. The regression branch and classification branch in the spatial channel decoupled detection head both include two spatial channel blocks connected in sequence and a standard convolutional layer. A Retinex-guided auxiliary supervision branch is introduced during the training process. The detection module is configured to acquire nighttime images of the target to be detected taken by the UAV and input them into the trained UAV nighttime target detection model. Multi-scale image features are extracted through the backbone network. The multi-scale image features are input into the neck network for feature fusion to obtain multi-scale fused features. The multi-scale fused features are input into the head network, and the detection results of each target in the nighttime image to be detected are output through the spatial channel decoupling detection head.

8. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, characterized in that, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images