A Deep Learning-Based End-to-End Real-Time Object Detection Method and System for Low-Light Enhancement

Abstract

This invention relates to the fields of computer vision and deep learning technology, and discloses an end-to-end real-time target detection method and system based on low-light enhancement. The method includes: constructing an end-to-end neural network model integrating an image enhancement module and a target detection decoding module; the image enhancement module using brightness enhancement features as guidance and illumination-guided attention blocks to perform multi-scale feature extraction and fusion; the target detection decoding module reusing the decoding features generated by the image enhancement module and connecting them to detection heads that do not share weights; and training the neural network model using a two-stage training strategy: using the neural network model after the second stage of training to perform end-to-end real-time target detection on the input low-light image. This invention can achieve high-precision target detection in low-light environments by rapidly training on low-light image enhancement datasets and ordinary lighting condition business datasets without requiring a low-light target detection dataset.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, specifically to a deep learning-based end-to-end real-time target detection method and system based on low-light enhancement. Background Technology

[0002] Existing visible light target detection methods for low-light scenes can be broadly classified into two categories.

[0003] The first category is the cascaded "enhancement-then-detection" approach. This type of approach typically first uses traditional image enhancement algorithms such as histogram equalization and gamma correction, or a deep learning low-light enhancement network based on Retinex theory, to enhance the brightness and contrast of the original low-light image. Then, the enhanced image is used as input to a general object detection network (such as Faster R-CNN, YOLO series, etc.) to perform the detection task. The enhancement and detection networks are structurally independent and optimized separately during training, forming a cascaded processing framework.

[0004] This type of scheme has the following main shortcomings: (1) Since the enhancement module and the detection module are two independent networks, there is a lack of deep coupling between them for the detection task. The optimization target of the enhancement network is mainly oriented towards subjective visual quality rather than downstream detection accuracy. It may destroy high-frequency details related to the target during denoising, smoothing and other processes, which is not conducive to subsequent target representation; (2) The cascaded structure leads to redundant algorithm parameters. The enhancement network and the detection network each build feature pyramids or multi-scale feature extraction modules separately. There is a situation where multi-scale features are repeatedly calculated for the same input image, which increases the overall computational load and memory overhead, resulting in a significant decrease in inference speed. It has high requirements for hardware resources (GPU memory, CPU, etc.) and is difficult to meet the real-time constraints of embedded terminals or vehicle platforms; (3) In terms of training process, it is usually necessary to build paired datasets suitable for low light enhancement tasks and labeled datasets suitable for target detection tasks. The training of enhancement and detection is independent of each other, making it difficult to achieve knowledge sharing and joint optimization between tasks. It also increases the cost of data collection and labeling.

[0005] The second category is the scheme that directly uses low-light image datasets to train the detection network. This type of method omits explicit image enhancement steps and directly inputs low-light images into a single-stage or multi-stage detection network, improving detection performance in low-light scenes by adjusting the network structure or loss function. This type of method avoids redundant computation in cascaded schemes to some extent, but it also has obvious limitations: (1) There is currently a lack of large-scale, comprehensive open-source target detection datasets for low-light environments. Existing public low-light detection datasets are relatively limited in terms of sample quantity, scene diversity, and annotation precision, and cannot be directly applied to a large number of real-world tasks; (2) There are significant differences between the target category settings, acquisition scenes, and shooting angles of existing low-light detection datasets and actual engineering applications. For example, they are mostly concentrated on road monitoring with fixed perspectives or a few traffic participant categories, while in scenarios such as maritime supervision and unmanned robot inspection, the target category, perspective pitch angle, and imaging scale distribution are inconsistent with these. This difference in data distribution makes it difficult for detection models trained directly on existing low-light datasets to achieve good generalization performance in real-world application scenarios. (3) The cost of data collection and labeling in low light environments is higher than that in normal lighting environments. Furthermore, the frequency of target appearance in low light environments is significantly lower than that in normal lighting environments, resulting in a longer time and higher cost for data collection and filtering.

[0006] In summary, current technologies lack an end-to-end low-light target detection method that can fully utilize low-light enhancement features within a unified network architecture, avoid redundant feature calculations, and balance detection accuracy and real-time performance under limited data resources. Summary of the Invention

[0007] To address the aforementioned technical issues, this invention provides a deep learning-based end-to-end real-time target detection method and system based on low-light enhancement. Without requiring a low-light target detection dataset, it can be rapidly trained using a low-light image enhancement dataset and a business dataset based on normal lighting conditions, achieving high-precision target detection in low-light environments.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] In a first aspect, the present invention provides a deep learning-based end-to-end real-time target detection method based on low-light enhancement, comprising:

[0010] An end-to-end neural network model integrating an image enhancement module and an object detection decoding module is constructed. The image enhancement module uses brightness enhancement features as a guide and uses illumination-guided attention blocks to perform multi-scale feature extraction and fusion to obtain decoded features and output an illumination-enhanced image. The object detection decoding module reuses the decoded features generated by the image enhancement module and connects to detection heads that do not share weights to predict the bounding box and category of the target.

[0011] The neural network model is trained using a two-stage training strategy: In the first stage, an augmentation dataset containing pairs of low-light images and corresponding normal-light images is used to train the image augmentation module parameters of the neural network model in combination with the generated illumination augmentation images, and the object detection decoding module parameters are frozen; In the second stage, an object detection dataset containing only normal-light images is used, and the optimal image augmentation module parameters obtained in the first stage are loaded to train the object detection decoding module parameters of the neural network model, while the image augmentation module parameters are frozen.

[0012] The neural network model, after completing the second stage of training, is used to perform end-to-end real-time target detection on the input low-light image.

[0013] In one embodiment, the image enhancement module uses brightness enhancement features as a guide and performs multi-scale feature extraction and fusion through illumination-guided attention blocks to obtain decoded features and output an illumination-enhanced image, specifically including:

[0014] The mean value of each pixel in the input low-light image in the channel dimension is calculated to obtain the illumination prior information. The low-light image and the illumination prior information are concatenated in the channel dimension and then processed by convolution to generate a brightness enhancement map and brightness enhancement features. Tensor dot multiplication is performed on the brightness enhancement map and the input low-light image to generate a brightened image.

[0015] Multi-scale features are generated based on the brightened image: the multi-scale features include first-scale features. Second-scale features and third-scale features ;in, It is obtained by convolving the brightened image. Through the The result is obtained by downsampling and convolving the brightness enhancement feature with the result of the illumination-guided attention block processing. Through the The result is obtained by further downsampling and convolving the result of downsampling the brightness enhancement features and processing them with the illumination-guided attention block;

[0016] Based on the multi-scale features and the brightness enhancement features, a first decoding feature for detection is generated through illumination-guided attention blocks and upsampling fusion operations. Second decoding feature and third decoding features ;

[0017] right After performing the convolution operation, the image is stitched together with the brightened image to obtain the illumination-enhanced image.

[0018] In one embodiment, the process of concatenating the low-light image with prior illumination information along the channel dimension, generating a brightness enhancement map and brightness enhancement features through convolution, and then performing a tensor dot product between the brightness enhancement map and the input low-light image to generate a brightened image specifically includes:

[0019] Low-light images The illumination prior information is concatenated along the channel dimension and fed into two convolutional modules to generate brightness enhancement features. :

[0020] ;

[0021] This indicates that the mean value along the channel dimension is calculated for each pixel location. This indicates the concatenation of channel-dimensional tensors. This represents a convolution with a stride of 1 and a kernel size of . The first convolutional layer, This indicates a stride of 1 and a kernel size of 1. Grouped convolutional layers with padding of 2;

[0022] Calculate the brightness enhancement map :

[0023] ;

[0024] This indicates a stride of 1 and a kernel size of 1. The second convolutional layer;

[0025] Calculate the brightened image :

[0026] ;

[0027] This represents the tensor dot product.

[0028] In one embodiment, the It is obtained by convolving the brightened image. Through the The result is obtained by downsampling and convolving the brightness enhancement feature with the result of the illumination-guided attention block processing. Through the The result is obtained by downsampling and convolving the brightness enhancement features with the result of illumination-guided attention block processing, specifically including:

[0029] ;

[0030] ;

[0031] ;

[0032] in, This indicates a stride of 1 and a kernel size of 1. The fourth convolutional layer with padding of 1; This indicates a stride of 2 and a kernel size of . The fifth, sixth, and seventh convolutional layers are padded with 1. Use lighting to guide attention to blocks.

[0033] In one embodiment, based on the multi-scale features and the brightness enhancement features, a first decoding feature for detection is generated through illumination-guided attention blocks and upsampling fusion operations. Second decoding feature and third decoding features ;

[0034] ;

[0035] ;

[0036] ;

[0037] in, This indicates a stride of 2 and a kernel size of . The sixth convolutional layer with padding of 1. This indicates a stride of 2 and a kernel size of . The eighth convolutional layer with padding of 1; This indicates a stride of 2 and a kernel size of . A deconvolutional layer filled with zeros; This indicates a stride of 1 and a kernel size of 1. The tenth convolutional layer with padding of 1; This indicates a stride of 2 and a kernel size of . A deconvolutional layer filled with zeros; This indicates a stride of 1 and a kernel size of 1. The twelfth convolutional layer with padding of 1. Use lighting to guide attention to blocks.

[0038] In one embodiment, the pair After performing the convolution operation, the image is concatenated with the brightened image to obtain the illumination-enhanced image, which specifically includes:

[0039] ;

[0040] To enhance the image with illumination, This indicates a stride of 1 and a kernel size of 1. The thirteenth convolutional layer with padding set to 1.

[0041] In one embodiment, the target detection decoding module reuses the decoding features generated by the image enhancement module and connects to detection heads that do not share weights to predict the bounding box and category of the target, specifically including:

[0042] The three detection heads respectively use the first decoding feature Second decoding feature and third decoding features As input, to predict the bounding box of the target. The three detectors use a combination of class probability scores and weight parameters that are not shared between the three detectors. The weights of the bounding box calculation branches and classification branches within each detector are also not shared.

[0043] Will , , Unified as :

[0044] ;

[0045] ;

[0046] in, Indicates that the convolution kernel is Convolutional layers; Indicates that the convolution kernel is The convolutional layer.

[0047] In one embodiment, the method further includes: if the bounding boxes obtained by decoding features at different scales overlap, deduplication of the bounding boxes is performed using nonmaximum suppression techniques.

[0048] In one embodiment, during the first stage of training, the loss function used is L1 loss; the optimal neural network model snapshot is selected and retained based on the peak signal-to-noise ratio of the validation set, and used to initialize the neural network model parameters for the second stage.

[0049] In the second stage of training, the loss function used is the sum of CIoU loss, binary cross-entropy loss, and distributed focus loss; the best neural network model snapshot is selected and retained based on the mean of the average precision of the validation set for final inference.

[0050] In a second aspect, the present invention provides a computer system including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any embodiment of the first aspect.

[0051] Compared with the prior art, the beneficial technical effects of the present invention are:

[0052] Low-light target detection enhancement: This invention achieves high-precision end-to-end low-light target detection by sharing most of the network parameters and jointly training low-light image enhancement and target detection structures.

[0053] Flexible deployment: This invention achieves a first-level real-time target detection network by designing a simple and efficient network structure, avoiding a large number of redundant parameters. The small number of parameters allows the network to be quickly deployed on edge devices or server devices.

[0054] Dataset production costs are greatly reduced: the first stage of low-light enhancement uses an open-source dataset, and the second stage of target detection can use an existing business dataset. There is no need to collect additional production data to create a low-light target detection dataset, which greatly reduces the cost of data collection and dataset annotation. Attached Figure Description

[0055] Figure 1 This is a flowchart of the method of the present invention;

[0056] Figure 2 This is a structural diagram of the neural network model of the present invention. Detailed Implementation

[0057] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0058] like Figure 1 As shown, this invention provides a deep learning-based end-to-end real-time target detection method based on low-light enhancement, comprising:

[0059] S1. Construct an end-to-end neural network model that integrates an image enhancement module and an object detection decoding module. The image enhancement module uses brightness enhancement features as a guide and performs multi-scale feature extraction and fusion through illumination-guided attention blocks (IGBA) to obtain decoded features and output an illumination-enhanced image. The object detection decoding module reuses the decoded features generated by the image enhancement module and connects to detection heads that do not share weights to predict the bounding box and category of the target.

[0060] S2, the neural network model is trained using a two-stage training strategy: In the first stage, the image enhancement module parameters of the neural network model are trained using an augmentation dataset containing pairs of low-light images and corresponding normal-light images, combined with the generated illumination-enhanced images, and the object detection decoding module parameters are frozen; In the second stage, the object detection decoding module parameters of the neural network model are trained using an object detection dataset containing only normal-light images, and the optimal image enhancement module parameters obtained in the first stage are loaded, while the image enhancement module parameters are frozen.

[0061] S3 uses the neural network model after the second stage of training to perform end-to-end real-time target detection on the input low-light image.

[0062] The light-guided attention block used in this invention is existing technology, as detailed in the literature "Retinexformer: one stage retinex-based transformer for low-light image enhancement".

[0063] The core objective of this invention is to provide an end-to-end real-time target detection scheme that integrates low-light enhancement, breaking through the dependence of existing technologies on low-light datasets and solving the problem of insufficient workflow performance of cascaded schemes. By constructing an end-to-end integrated neural network model for low-light image enhancement and target detection, and proposing a corresponding two-stage neural network model training strategy, this invention achieves end-to-end low-light image enhancement and high-precision target detection without relying on low-light target detection datasets. It requires no manual intervention and significantly improves the detection accuracy and speed of the detector on RGB images in low-light environments, meeting the high-precision target detection requirements in scenarios such as unmanned driving and inspection robots that are sensitive to real-time performance and illumination.

[0064] The following sections will provide a detailed introduction to the construction process of the neural network model and the two-stage training strategy.

[0065] 1. Neural network model construction.

[0066] The neural network model structure proposed in this invention is as follows: Figure 2 As shown. Suppose the input is a low-light image. The size is , Low-light image Width and height.

[0067] In the neural network model, the mean values ​​of the three channels (R channel, G channel, and B channel) of each pixel in the input low-light image are first calculated to obtain illumination prior information, the size of which is [missing information]. The input low-light image and prior lighting information are concatenated along the channel dimension and fed into three convolutional modules to generate a brightness enhancement map. and brightness enhancement features The process can be represented by formulas (1) and (2):

[0068] (1)

[0069] (2)

[0070] in, This indicates that the mean value along the channel dimension is calculated for each pixel location. This indicates the concatenation of channel-dimensional tensors. This represents a convolution with a stride of 1 and a kernel size of . The first convolutional layer; This indicates a stride of 1 and a kernel size of 1. The second convolutional layer; This indicates a stride of 1 and a kernel size of 1. Grouped convolutional layers with padding of 2.

[0071] based on and input low-light image Use formula (3) to generate a brightened image. :

[0072] (3)

[0073] in This represents the tensor dot product.

[0074] generated This will be used as input to the denoiser (the part of the image enhancement module that generates an illumination-enhanced image based on the brightened image) to generate multi-scale features during the encoder stage. Its calculation method can be expressed by the following formula:

[0075] (4)

[0076] (5)

[0077] (6)

[0078] in, This indicates a stride of 1 and a kernel size of 1. The fourth convolutional layer with padding of 1; , and Both indicate a stride of 2 and a kernel size of . A convolutional layer with padding of 1. The dimensions are respectively , and . Indicates batch size, Indicates the number of channels.

[0079] based on Feature maps are generated during the decoding stage of the denoiser. Its calculation method can be expressed by the following formula:

[0080] (7)

[0081] (8)

[0082] (9)

[0083] in, This indicates a stride of 2 and a kernel size of . The eighth convolutional layer with padding of 1; This indicates a stride of 2 and a kernel size of . A deconvolutional layer filled with zeros; This indicates a stride of 1 and a kernel size of 1. The tenth convolutional layer with padding of 1; This indicates a stride of 2 and a kernel size of . A deconvolutional layer filled with zeros; This indicates a stride of 1 and a kernel size of 1. The twelfth convolutional layer with padding of 1. Use lighting to guide attention to blocks.

[0084] Therefore, the enhanced image generated during the first stage of training can be represented by formula (10) as follows:

[0085] (10)

[0086] This indicates a stride of 1 and a kernel size of 1. The thirteenth convolutional layer with padding set to 1.

[0087] The illumination-enhanced image is used for the first stage of training. That is, the input for the first stage of training is the low-light image, and the output is the illumination-enhanced image. Combined with the corresponding normal illumination image and loss function, the first stage of training is achieved.

[0088] In the second stage of training and the final model inference, the three decoupled detection heads respectively... The bounding box of the target is used as input. The three decoupled detector heads, along with the category probability score, can be represented by formulas (11) and (12). The weight parameters are not shared among the three detector heads, and the weights are not shared between the bounding box calculation branch and the classification branch within each detector head. , , Unified as :

[0089] (11)

[0090] (12)

[0091] in, Indicates that the convolution kernel is Convolutional layers; Indicates that the convolution kernel is The convolutional layer.

[0092] Because three scales are used to calculate the bounding box (bbox) and the class probability score, some pixel locations will have a certain amount of overlapping bounding boxes. This invention uses non-maximum suppression techniques to remove these duplicate bounding boxes.

[0093] 2. Two-stage training strategy.

[0094] Phase 1: Training the low-light enhancement model:

[0095] After completing the construction of the neural network model, the first step is to... Figure 2 The parameters within the image enhancement module are trained, while the gradient of the parameters in the target detection decoding module is frozen, thus enabling gradient backpropagation and parameter updates to be performed only on the image enhancement module.

[0096] The data augmentation methods used in this stage include random cropping, random flipping, and random rotation.

[0097] The dataset used in this stage is a comprehensive dataset resulting from the merging of multiple open-source datasets. The input images for this dataset are low-light images, and the ground truth images are normal-light images with no object detection sample annotations.

[0098] The loss function used in this stage is L1 loss.

[0099] The second stage is the training of the object detection model.

[0100] The neural network model parameters are initialized using the snapshot of the neural network model with the optimal peak signal-to-noise ratio (PSNR) obtained from the first stage of training. Based on this, [the following steps are taken]. Figure 2 The parameters within the target detection decoding module are trained, while the gradient of the parameters in the image enhancement module is frozen, thus enabling gradient backpropagation and parameter updates to be performed only on the target detection decoding module.

[0101] The data augmentation methods used in this stage include random flipping, data mixing augmentation (Mixup), mosaic data augmentation (Mosaic), and copy-paste data augmentation (Copy Psate).

[0102] The dataset used in this stage comes from a target detection dataset tailored to a specific business scenario, including normally lit images and target bounding boxes and category names on normally lit images.

[0103] The loss function used in this stage is the sum of CIoU loss, Binary Cross Entropy loss, and Distribution Focal Loss (DFL).

[0104] During inference, the optimal neural network model snapshot obtained from the second stage training is used to achieve end-to-end real-time target detection in low-light images.

[0105] Example:

[0106] The following section uses "Nighttime Target Detection of Robotic Vacuum Cleaners" as an example to explain the implementation process in detail.

[0107] (1) Preparation of open source low light dataset.

[0108] The open-source datasets LOL-v1, LOL-v2, SID, SMID, SDSD-indoor, SDSD-outdoor, MIT-AdobeFiveK, and NTIRE 2024 were merged into the MixtureScale dataset. Duplicate images were removed, and all images were uniformly sampled to a width and height of 640×640 pixels. The dataset was then divided into training, validation, and test sets in a 6:2:2 ratio.

[0109] (2) Preparation of business dataset.

[0110] Video data captured by the robot's camera was acquired, and images were extracted at 10-second intervals. Images were quickly filtered to remove those with severe motion blur and those without targets. 95% of the resulting business dataset consisted of well-lit images. This data was labeled with COCO-formatted bounding boxes, identifying 30 obstacle categories. This dataset was named YW30 and divided into training, validation, and test sets in a 6:2:2 ratio.

[0111] (3) The structure of the neural network model is as follows Figure 2 As shown.

[0112] (4) Two-stage training strategy (S4).

[0113] Phase 1: Will Figure 2 The gradient attribute of the object detection decoding module is set to `requires_grad = False`, which freezes the gradient of the object detection decoding module's parameters, allowing gradient backpropagation and parameter updates to be performed only on the image augmentation module. Data augmentation methods used in this stage include random cropping, random flipping, and random rotation. The dataset used in this stage is the MixtureScale dataset mentioned above. The loss function used in this stage is L1 loss.

[0114] The second stage: The network parameters are initialized using the model snapshot with the best PSNR value obtained from the first stage of training. Based on this, the parameters in the object detection decoding module are trained, while the gradients of the parameters in the image enhancement module are frozen (by setting the attribute require_grad = False), meaning that gradient backpropagation and parameter updates are only performed on the object detection decoding module.

[0115] The data augmentation techniques used in this phase include random flipping, Mixup, Mosaic, and Copy Paste. The dataset used in this phase is the YW30 dataset. The loss functions used in this phase are CIoU loss, Binary Cross Entropy loss, and Distributed Focal Loss (DFL).

[0116] (5) Optimal Model Snapshot Selection: During the first-stage training process, the optimal PSNR (Peak Signal-to-Noise Ratio) of the model snapshot generated in each epoch is calculated. The model snapshot with the optimal PSNR is selected for initializing the model parameters during the second-stage training. During the second-stage training process, the mean average precision (mAP) of the model snapshots for each epoch is calculated, and the model snapshot with the optimal mAP is retained for use during inference. Accuracy evaluation is performed on the validation set.

[0117] (6) Model Inference and Final Accuracy Evaluation: The highest-accuracy model snapshot saved in the second stage is used to initialize the entire network parameters and perform inference. The final accuracy evaluation will be conducted on the test set. The inference results... These represent the pixel coordinates of the top left corner of each target. bottom right pixel coordinates The target's category (cls) and the probability (score) of the target belonging to that category.

[0118] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0119] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0120] In one embodiment, the present invention provides a computer system, which may be a server. The computer system includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data used in the methods described above. The network interface communicates with external terminals via a network connection. The computer program is executed by the processor to implement the methods described above.

[0121] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0123] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A deep learning-based end-to-end real-time target detection method based on low-light enhancement, characterized in that, include: Construct an end-to-end neural network model that integrates an image enhancement module and an object detection decoding module; The image enhancement module, guided by brightness enhancement features, extracts and fuses multi-scale features through illumination-guided attention blocks to obtain decoded features and output an illumination-enhanced image. Specifically, this includes: calculating the mean of each pixel in the input low-light image along the channel dimension to obtain illumination prior information; concatenating the low-light image and illumination prior information along the channel dimension, followed by convolution processing to generate a brightness enhancement map and brightness enhancement features; performing tensor dot multiplication on the brightness enhancement map and the input low-light image to generate a brightened image; and generating multi-scale features based on the brightened image: these multi-scale features include first-scale features. Second-scale features and third-scale features ; It is obtained by convolving the brightened image. Through the The result is obtained by downsampling and convolving the brightness enhancement feature with the result of the illumination-guided attention block processing. Through the The result of downsampling the brightness enhancement features and processing them with an illumination-guided attention block is then downsampled and convolved to obtain the first decoding feature for detection. Based on the multi-scale features and the brightness enhancement features, the first decoding feature for detection is generated through illumination-guided attention blocks and upsampling fusion operations. Second decoding feature and third decoding features ;right After performing the convolution operation, the image is stitched together with the brightened image to obtain the illumination-enhanced image; The object detection decoding module reuses the decoded features generated by the image enhancement module and connects them to detection heads that do not share weights to predict the bounding box and category of the object. Specifically, this includes three detection heads that respectively... , and As input, to predict the bounding box of the target. And the category probability score, and the weight parameters are not shared among the three detectors, and the weights are not shared between the bounding box calculation branches and the classification branches within each detector; , , Unified as : ; ;in, Indicates that the convolution kernel is Convolutional layers; Indicates that the convolution kernel is Convolutional layers; The neural network model is trained using a two-stage training strategy: In the first stage, an augmentation dataset containing pairs of low-light images and corresponding normal-light images is used to train the image augmentation module parameters of the neural network model in combination with the generated illumination augmentation images, and the object detection decoding module parameters are frozen; In the second stage, an object detection dataset containing only normal-light images is used, and the optimal image augmentation module parameters obtained in the first stage are loaded to train the object detection decoding module parameters of the neural network model, while the image augmentation module parameters are frozen. The neural network model, after completing the second stage of training, is used to perform end-to-end real-time target detection on the input low-light image.

2. The deep learning-based end-to-end real-time target detection method based on low-light enhancement according to claim 1, characterized in that, The process of concatenating the low-light image with prior illumination information along the channel dimension, generating a brightness enhancement map and brightness enhancement features through convolution, and then performing a tensor dot product between the brightness enhancement map and the input low-light image to generate a brightened image specifically includes: Low-light images The illumination prior information is concatenated along the channel dimension and fed into two convolutional modules to generate brightness enhancement features. : ； This indicates that the mean value along the channel dimension is calculated for each pixel location. This indicates the concatenation of channel-dimensional tensors. This represents a convolution with a stride of 1 and a kernel size of . The first convolutional layer, This indicates a stride of 1 and a kernel size of 1. Grouped convolutional layers with padding of 2; Calculate the brightness enhancement map : ； This indicates a stride of 1 and a kernel size of 1. The second convolutional layer; Calculate the brightened image : ； This represents the tensor dot product.

3. The deep learning-based end-to-end real-time target detection method based on low-light enhancement according to claim 1, characterized in that, The It is obtained by convolving the brightened image. Through the The result is obtained by downsampling and convolving the brightness enhancement feature with the result of the illumination-guided attention block processing. Through the The result is obtained by downsampling and convolving the brightness enhancement features with the result of illumination-guided attention block processing, specifically including: ； ； ； in, This indicates a stride of 1 and a kernel size of 1. The fourth convolutional layer with padding of 1; This indicates a stride of 2 and a kernel size of . The fifth, sixth, and seventh convolutional layers are padded with 1. Use lighting to guide attention to blocks.

4. The deep learning-based end-to-end real-time target detection method based on low-light enhancement according to claim 1, characterized in that, Based on the multi-scale features and the brightness enhancement features, a first decoding feature for detection is generated through illumination-guided attention blocks and upsampling fusion operations. Second decoding feature and third decoding features ; ； ； ； in, This indicates a stride of 2 and a kernel size of . The sixth convolutional layer with padding of 1. This indicates a stride of 2 and a kernel size of . The eighth convolutional layer with padding of 1; This indicates a stride of 2 and a kernel size of . A deconvolutional layer filled with zeros; This indicates a stride of 1 and a kernel size of 1. The tenth convolutional layer with padding of 1; This indicates a stride of 2 and a kernel size of . A deconvolutional layer filled with zeros; This indicates a stride of 1 and a kernel size of 1. The twelfth convolutional layer with padding of 1. Use lighting to guide attention to blocks.

5. The deep learning-based end-to-end real-time target detection method based on low-light enhancement according to claim 1, characterized in that, The pair After performing the convolution operation, the image is concatenated with the brightened image to obtain the illumination-enhanced image, which specifically includes: ； To enhance the image with illumination, This indicates a stride of 1 and a kernel size of 1. The thirteenth convolutional layer with padding set to 1.

6. The deep learning-based end-to-end real-time target detection method based on low-light enhancement according to claim 1, characterized in that, Also includes: The bounding boxes obtained by decoding features at different scales overlap, and non-maximum suppression techniques are used to remove duplicates from the bounding boxes.

7. The deep learning-based end-to-end real-time target detection method based on low-light enhancement according to claim 1, characterized in that, During the first stage of training, the loss function used is L1 loss; the optimal neural network model snapshot is selected and retained based on the peak signal-to-noise ratio of the validation set, and used to initialize the neural network model parameters for the second stage. In the second stage of training, the loss function used is the sum of CIoU loss, binary cross-entropy loss, and distributed focus loss; the best neural network model snapshot is selected and retained based on the mean of the average precision of the validation set for final inference.

8. A computer system comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images