Low-illumination target detection method based on DGA-ISP

Abstract

The application discloses a low-illumination target detection method based on DGA-ISP, comprising the following steps: 1) constructing a normal-illumination discrete prior feature dictionary: processing and dividing a low-illumination Bayer RAW image dataset and a corresponding normal-illumination dataset, and obtaining the discrete prior feature dictionary by using a discrete feature quantization adversarial network pre-training; 2) designing a dynamic linear mapping module, extracting local and global features of the RAW image, generating an adaptive sensor calibration matrix to complete preliminary color correction; 3) designing a feature alignment and retrieval module based on reversible normalization flow, aligning the feature distribution after calibration with the normal-illumination feature, and retrieving the lossless high-quality prior feature in the feature dictionary; 4) designing a polynomial nonlinear module and a target detection network, using the retrieved prior feature to guide polynomial coefficient prediction, completing adaptive nonlinear stretching of the image and target detection, and outputting semantic category labels and spatial position bounding boxes of the target. The method combines the physical interpretability of RAW domain processing with the prior advantage of the discrete feature dictionary, can effectively suppress noise and supplement lost texture details in a low-illumination environment, and greatly improves the accuracy and robustness of target detection.

Description

Technical Field

[0001] This invention relates to the fields of image processing and target detection technology, specifically a low-light target detection method based on prior feature dictionary-guided adaptive image signal processing (DGA-ISP). Background Technology

[0002] Object detection, as one of the core tasks in computer vision, has broad application prospects in scenarios such as autonomous driving, security monitoring, and intelligent robots. However, most mainstream object detection methods are currently designed for normal illumination environments with good lighting conditions and clear imaging. In complex environments such as nighttime or extremely low illumination, the accuracy and robustness of object detection often face severe challenges.

[0003] Imaging in low-light environments is typically limited by the extremely small number of photons and complex sensor noise. Images captured under these conditions not only have extremely low brightness and poor contrast, but also generally suffer from extremely low signal-to-noise ratio (SNR), severe color shift, and significant loss of object edge texture features. Accurate target localization and category recognition in such images with abundant degraded information presents an extremely challenging problem.

[0004] Existing low-light object detection solutions generally suffer from the following shortcomings: First, traditional image signal processing (ISP) pipelines or existing low-light image enhancement algorithms typically operate in the sRGB color space. While forcibly increasing image brightness, these methods often inevitably amplify hidden noise and even produce structural artifacts, destroying the original texture features. Second, existing object detection networks (such as Faster R-CNN and YOLO) are prone to losing already weak low-level details when performing multi-layer convolutional downsampling, resulting in extremely poor detection performance for small objects and overlapping targets. Finally, current conventional solutions usually treat "low-light image enhancement" and "object detection" as two completely independent and separate modules connected in series. This approach not only ignores the extremely rich linear physical information in RAW domain images, but also, the features extracted from the enhanced sRGB images, designed to cater to human visual habits, are often not the optimal solution for machine vision detection tasks, thus causing a bottleneck in the accuracy of low-light object detection.

[0005] To overcome the performance bottleneck caused by feature loss and noise amplification under low illumination, and to fully utilize the underlying physical information of the RAW domain, there is an urgent need for a low-light target detection method that can effectively supplement clear texture prior features and jointly optimize adaptive ISP processing and target detection tasks end-to-end. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a low-light RAW image target detection method based on prior feature dictionary guidance and adaptive ISP. This method can overcome the bottlenecks of extreme feature loss and noise amplification in low-light environments, directly performing physical correction and clear texture reconstruction in the RAW domain, achieving end-to-end high-precision target detection.

[0007] The technical solution to achieve the objective of this invention is:

[0008] like Figure 1 As shown, a low-light target detection method based on DGA-ISP includes the following steps:

[0009] 1) Constructing a discrete prior feature dictionary for normal illumination: The low-illumination Bayer RAW image dataset and the corresponding normal illumination dataset (using a self-built dataset containing 2000 pairs of high-quality RAW images) are processed and partitioned, and a discrete prior feature dictionary is obtained by pre-training a Discrete Feature Quantization Adversarial Network (DFQ-GAN). The process is as follows:

[0010] 1-1) Collect and integrate low-light RAW images and their corresponding detection labels, and construct the DGA-ISP-Data dataset together with normal-light images (this self-built dataset contains 2000 pairs of high-quality RAW images). Divide this dataset into training and testing sets in an 8:2 ratio, as shown in Table 1:

[0011] Table 1:

[0012] Data content Training set (80%) Test set (20%) Total dataset (100%) Remark Normal illumination image dataset Nor-images-Train Nor-images-Test Nor-images-Data Used for pre-training to extract high-quality prior features Low-light image dataset Dark-images-Train Dark-images-Test Dark-images-Data Includes detection labels for end-to-end training.

[0013] 1-2) such as Figure 2 As shown, a Discrete Feature Quantization Adversarial Network (DFQ-GAN) is pre-trained using a normal illumination image dataset. By aggregating structural texture patches of sharp images in the latent feature space, a discrete prior feature dictionary is constructed. During the training process of the Discrete Feature Quantization Adversarial Network (DFQ-GAN), the quantization loss function formula (1) used to optimize the prior feature dictionary is as follows:

[0014] (1)

[0015] This step uses vector quantization to encode common texture features in high-resolution images into 1024 discrete codewords and store them in a dictionary, providing strong prior support for subsequent processing to repair degraded features contaminated by noise.

[0016] 1-3) Preprocess the low-light image dataset to extract the semantic category labels and spatial location bounding boxes (center point coordinates, width, and height) of the targets, which serve as a supervised reference for subsequent target detection training. The bounding box contains the center point coordinates, border length, and height of the target. By establishing refined annotation information, accurate ground truth is provided for the joint optimization of the detection heads in the subsequent step 4).

[0017] 2) such as Figure 3 As shown, a dynamic linear mapping module is designed to extract local and global features of the RAW image and generate an adaptive sensor calibration matrix to complete the initial color correction. The process is as follows:

[0018] 2-1) The input four-channel Bayer RAW image is fed into a two-stream network architecture composed of residual blocks. On the one hand, local feature vectors describing pixel-level details, texture edges, and noise distribution are extracted through convolutional layers. On the other hand, the global average pooling layer aggregates the contextual lighting information of the entire image to extract a global feature vector describing the overall color shift distribution. This dual-stream design enables the network to simultaneously address microscopic pixel-level degradation and macroscopic changes in ambient light when processing low-light images, providing a comprehensive feature base for accurate correction.

[0019] 2-2) Introduce a cross-attention mechanism, using the extracted feature vectors respectively and For query items, using preset... Dimensional basic camera processing matrix For key and value terms, a local calibration matrix for local texture compensation is dynamically generated based on physical priors by calculating relevance weights. And a global calibration matrix for global color cast correction. This step utilizes a cross-attention mechanism to give the model "adaptive" perception capabilities, enabling it to no longer rely on fixed processing parameters but to dynamically calculate the optimal color compensation weights based on the specific lighting conditions of the current input image.

[0020] 2-3) Using the fusion formula The fundamental matrix is ​​linearly added to the dynamically generated calibration offset to calculate the final adaptive linear transformation matrix. It is then applied directly to the original RAW image to perform channel mapping and color correction, eliminating severe color shifts while preserving physical linearity, and finally outputting a pre-corrected feature map. Traditional image signal processing tends to amplify noise in complex nonlinear spaces. This step strictly restricts linear mapping operations to the RAW domain, successfully correcting color cast while preserving the sensor's original physical response characteristics to the greatest extent, thus laying a clean data foundation for subsequent feature alignment.

[0021] 3) such as Figure 4 , Figure 5 As shown, a feature alignment and retrieval module based on reversible normalized flow (I-LNF) is designed to align the calibrated feature distribution with the normal illumination features and retrieve lossless, high-quality prior features from the feature dictionary. The process is as follows:

[0022] 3-1) Construct a conditional coding network to extract degradation guidance features: The preliminary corrected feature map output from step 2) The input to the conditional encoder, which consists of multiple residual convolutional layers, involves layer-by-layer downsampling in the spatial dimension and depth extraction of channel information to obtain conditional features that characterize the global illumination distribution, noise intensity, and structural degradation in low-light scenes. This provides precise spatial location guidance and distribution mapping constraints for the subsequent reversible transformation process. The degradation degree in low-light images is often uneven across different regions. This step, by extracting conditional features, essentially provides a "navigation map" for subsequent feature transformation, informing the network of the appropriate level of restoration effort at different spatial locations.

[0023] 3-2) Performing feature distribution alignment using reversible normalized flow: Constructing a reversible transformation layer group based on a squeeze-free layer structure Given the features Under constraints, feature maps in low-light degradation distribution spaces are transformed through nonlinear latent mapping. Transforming to a linear latent space that fits the prior space of normal illumination, its mathematical expression is: In this process, the latent features of the output are minimized by reducing the negative log-likelihood loss function during the training phase. The probability distribution approximates the normal illumination feature distribution, thereby eliminating nonlinear noise interference in the input image and correcting feature deviations. During the feature distribution alignment process of the reversible normalized flow, the negative log-likelihood loss function is used to constrain the distribution transformation. Formula (2) is as follows:

[0024] (2).

[0025] Traditional feature transformations are prone to irreversible information loss. This invention introduces Reversible Normalized Flow (I-LNF), which utilizes its strict mathematical bijective properties and logarithmic Jacobian determinant constraints to perfectly achieve cross-alignment from low-light degenerate distributions to normal prior distributions without losing any physical dimensions.

[0026] 3-3) Retrieving a high-quality prior feature dictionary based on Euclidean distance: This involves retrieving a high-quality prior feature dictionary from a pre-trained discrete prior feature dictionary. The process performs feature codeword retrieval based on the nearest neighbor algorithm, and calculates the aligned latent features. With each discrete codeword in the dictionary Minimum Euclidean distance between The unstable features in the latent space are replaced with lossless, high-fidelity codewords stored in the dictionary, thereby obtaining high-quality prior features containing clear texture and structural priors. This serves as a guiding signal for subsequent nonlinear enhancement modules. Even after feature alignment, textures completely lost under low light conditions cannot be generated out of thin air. This step uses a "retrieval-replacement" mechanism to directly retrieve matching high-frequency structural features from a high-quality knowledge base (dictionary), fundamentally avoiding the image blurring problem common in generative methods and providing highly discriminative and clear edge information for subsequent object detection.

[0027] 4) such as Figure 6 , Figure 7 As shown, a polynomial nonlinear module and object detection network are designed. The retrieved prior features guide the prediction of polynomial coefficients, enabling adaptive nonlinear stretching of the image and object detection. The output includes the semantic category label and spatial bounding box of the target. The process is as follows:

[0028] 4-1) Preset nonlinear mapping basis function set: such as Figure 6 As shown, a set of nonconvex polynomial basis functions from first to n orders is predefined. The basis functions simulate the nonlinear tone mapping characteristics in low-light imaging through predefined monotonicity and coordinate constraints. This aims to specifically stretch dark details in the image during subsequent enhancement processes and effectively suppress highlight clipping and color distortion that may occur in high dynamic range scenes. Traditional global gamma curves often cannot simultaneously cover both extremely dark and extremely bright regions. This step introduces high-order nonconvex polynomial basis functions, providing the network with a highly flexible brightness adjustment space, ensuring that while brightening dark targets, it does not cause overexposure of background light sources.

[0029] 4-2) Using prior features to guide polynomial coefficient prediction: The high-quality prior features retrieved in step 3) Compared with the preliminary correction feature map After feature fusion, the input is given to a convolutional neural network, which then utilizes... The provided clear texture direction and illumination distribution patterns serve as "mentor signals," guiding the network to predict pixel-level coefficient matrices that correspond one-to-one with each basis function. This allows for accurate modeling of the enhancement levels in different spatial regions of an image. This is one of the core innovations of this invention. Due to the feature map... In low-light conditions, relying solely on the enhancement coefficient to predict its own performance is highly prone to bias. Therefore, it is necessary to introduce high-quality prior features containing clear structure. Participating in prediction is like putting "night vision goggles" on the network, enabling it to accurately determine the edges of objects and apply appropriate stretching weights at those edges.

[0030] 4-3) Perform adaptive nonlinear stretching to generate enhanced images: Apply the predicted coefficient matrix to the preliminary correction feature map. Perform nonlinear mapping operations using the formula By dynamically adjusting the brightness and color of each pixel, the final output image exhibits significantly improved visual quality, effectively enhanced texture details, and physical consistency. This mapping process is calculated independently at the pixel level, meaning that each pixel in the image has its own unique stretching curve. This extreme adaptability greatly restores the shape of the target hidden in low light.

[0031] 4-4) Joint optimization of target detection output and end-to-end: such as Figure 7 As shown, the generated enhanced image The input is fed into an object detection network containing a classification head and a regression head. Through deep feature extraction and bounding box regression, the final output is the semantic category label and spatial location bounding box of the target. During the training phase, the entire DGA-ISP system is jointly optimized end-to-end through object detection loss and self-enhancement regularization loss, enabling the preceding ISP processing module to spontaneously learn the enhancement strategy most beneficial to the object recognition task. When the object detection network outputs the prediction result, its object detection loss function... Classification loss With bounding box regression loss The composition, formula (3) is as follows:

[0032] (3).

[0033] The optimization goal of conventional ISP systems is to make images "look pleasing to the eye," but this is often not the optimal solution for machine vision. Introducing object detection loss to directly supervise the front-end ISP network overturns the traditional fragmented paradigm.

[0034] 4-5) During the end-to-end joint training process, the overall DGA-ISP model enables the preceding ISP feature alignment module to spontaneously serve the final target recognition task, and its final training optimization of the total loss function... The total loss function, which is the sum of the negative log-likelihood loss and the target detection loss, is given by formula (4) as follows:

[0035] (4).

[0036] Formula (4) mathematically restores the "image's underlying physical structure (PPE)". ")" and "high-level semantic recognition" The model is fully bound to the target features. Under this end-to-end constraint, the model can not only recover lost target features, but also maintain extremely high detection accuracy and robustness in extremely noisy environments.

[0037] The dataset construction and prior dictionary initialization process described in step 1) is as follows:

[0038] 1) Dataset partitioning process: In step 1-1), the DGA-ISP-Data dataset is constructed by integrating image pairs from the same source but with different lighting conditions. Nor-images-Data serves as the normal illumination reference set and is used to execute the Discrete Feature Quantization Adversarial Network (DFQ-GAN) (see...). Figure 2 The feature space is constructed as follows: Dark-images-Data is used as the training set for low-light object detection, and the ratio of the training set to the test set is strictly limited to 8:2. This partitioning method ensures that the model can learn a deterministic mapping relationship from low light to normal light in the same physical scene, providing a reliable data foundation for the extraction of prior features.

[0039] 2) Feature Dictionary Construction Process: In steps 1-2), the discrete feature quantization adversarial network is trained using a subset of Nor-images-Train. By performing vector quantization on the latent features of high-resolution images, a prior feature dictionary containing 1024 discrete codewords is constructed, with each codeword having a feature dimension of 64. This parameter configuration ensures that the dictionary can cover sufficiently rich high-resolution texture priors while effectively controlling the space and computational complexity in the dictionary retrieval process, achieving the best balance between feature representation capability and computational efficiency.

[0040] 3) Label Preprocessing: In steps 1-3), the annotation information of each object to be detected is extracted from Dark-images-Data. The annotation information includes the target's category label, center point coordinates, and border width and height. Simultaneously, a detection difficulty assessment mechanism is set up for any target in the image. Its detection difficulty level The determination formula (5) is as follows:

[0041]

[0042] In the formula, For the goal pixel area The preset threshold for small target pixel size; For the goal With any other target The intersection and union ratio, The preset overlap threshold is used; targets judged as 1 are treated as difficult samples and given weighted attention in subsequent training. By introducing this rigorous difficulty judgment mechanism, the network is forced to assign higher optimization weights to small targets and overlapping targets that are easily missed in low-light environments during end-to-end training, thereby significantly improving the recall and robustness of the detection model in extreme environments.

[0043] The specific construction and calibration process of the dynamic linear mapping module mentioned in step 2) is as follows:

[0044] 1) Dual-stream feature extraction process: In step 2-1), using a three-layer continuous... The local flow network, with convolutional layers and residual connections, extracts a 64-dimensional local feature vector by performing local spatial convolution operations on the input Bayer RAW image. Simultaneously, a global flow network containing a global average pooling layer and two fully connected layers (with 128 hidden dimensions) is used to extract a 64-dimensional global feature vector by performing mean statistics and non-linear dimensionality compression on all pixels in the image. This dual-stream feature extraction mechanism can simultaneously capture microscopic pixel-level degradation details and macroscopic global scene illumination information, thus laying a multi-scale feature foundation for subsequent calculation of more accurate calibration weights.

[0045] 2) Attention weight calculation process: In step 2-2), three independent linear network layers are set up to respectively process the local feature vectors. With global feature vectors Mapped to a 64-dimensional query item matrix and And flattened Dimensional Basic Camera Processing Matrix The unified mapping is a 64-dimensional key-value matrix. With numerical term matrix The correlation weights between features and the fundamental matrix are calculated using a contracted dot product attention mechanism, and then the output is computed in parallel. Dimensional local calibration matrix With global calibration matrix The core attention calculation formulas (6) and (7) are as follows:

[0046]

[0047]

[0048] In the formula, This is the transpose of the key-value matrix. The channel dimension of the feature vector. To prevent excessively large inner product values ​​from causing gradient vanishing, a scaling factor is used. Traditional Image Processing (ISP) systems typically employ a fixed calibration matrix, which cannot adapt to the extremely complex dynamic color shifts under low light conditions. This step innovatively introduces a cross-attention mechanism, allowing the image's own degradation features to actively "query" and "modify" the underlying physical matrix, endowing the model with a high degree of adaptive color compensation capability.

[0049] 3) Linear mapping correction process: In steps 2-3), the matrix fusion formula is used. The final adaptive linear transformation matrix is ​​calculated. and take it as The convolutional kernel is applied to the original Bayer RAW image, mapping the number of channels from 4 to 3, and outputting a pre-corrected feature map. This step transforms the complex adaptive calibration process into the fusion of weight matrices and an extremely lightweight convolution operation. This not only greatly optimizes computational efficiency but, more importantly, strictly maintains the linear physical response within the RAW data domain, avoiding the cascading amplification of noise caused by prematurely introducing nonlinear operations.

[0050] The feature alignment and retrieval process based on reversible normalized flow described in step 3) is as follows:

[0051] 1) Multi-scale conditional encoding process: In step 3-1), a conditional encoder containing 4 layers of cascaded residual convolutional blocks and a downsampling layer with a stride of 2 is used to pre-correct the feature map. Hierarchical feature extraction was performed to obtain 64-dimensional conditional features describing the degradation level of the current scene and global semantic information. This conditional feature can serve as a strong guiding signal for subsequent reversible mapping, enabling the network to perceive differences in illumination and degradation levels in different regions of the image, thereby achieving extremely precise spatial location guidance and alignment with non-uniform feature distributions.

[0052] 2) Spatial Dimension Compression Transformation Process: Before entering the reversible transformation network, the input feature map is compressed by a Squeeze layer. By slicing and stacking the feature maps in the spatial dimension to the channel dimension, the spatial resolution of the feature maps is reduced to half of the original, while the number of channels is increased from 64 dimensions to 256 dimensions. This enhances the network's feature representation capabilities without losing original information. This checkerboard-like spatial compression operation not only significantly reduces the memory footprint and computational complexity of the subsequent reversible normalization flow, but also perfectly preserves all high-frequency response information in the original feature maps while expanding the receptive field.

[0053] 3) Alignment transformation process based on affine coupling layers: In step 3-2), a reversible transformation network consisting of 8 cascaded affine coupling layers is constructed. In terms of conditional features Under the guidance constraints, a deterministic distribution distortion is performed to project the feature map in the degenerate space onto the linear latent space, making its probability distribution approximate the normal illumination feature distribution, and outputting the aligned latent features. Unlike traditional feedforward convolutional neural networks, which are prone to irreversible information loss, this step utilizes the rigorous mathematical bijective property of reversible flow to transform the extremely complex nonlinear noise stripping problem into a deterministic mapping problem in the latent space, thus fundamentally eliminating the loss of details during feature recovery.

[0054] 4) High-quality prior feature retrieval process: In step 3-3), latent features are calculated using the L2 norm (Euclidean distance). With a prior feature dictionary containing 1024 discrete codewords The similarity distance between them; based on the distance minimization criterion The corresponding discrete codeword index is retrieved, and the 64-dimensional quantized vector under that index is extracted as the retrieval result. High-quality prior features with high-resolution texture supplementation capability are then reconstructed. Even if the features are aligned with the probability distribution, textures that have completely disappeared due to extreme physical degradation under low light cannot be generated out of thin air. An innovative "distance retrieval and physical replacement" strategy is adopted to directly retrieve lossless, clear texture priors from the knowledge base, fundamentally overcoming the problem of blurred edges and lack of realism in low-light image restoration.

[0055] The specific network structure and end-to-end optimization process of the polynomial nonlinear module and target detection network mentioned in step 4) are as follows:

[0056] 1) Construction process of nonlinear mapping basis functions: In step 4-1), a set of monotone nonconvex polynomial basis functions consisting of orders 1 to 8 is preset. The basis functions are forced to pass strictly through the origin (0,0) and the normalized endpoint (1,1) in the spatial coordinate system through coordinate constraints, and are used to simulate the nonlinear large stretching of the dark area of ​​RAW data and the suppression of the overflow of the highlight area.

[0057] 2) Specific structure and process of the polynomial coefficient prediction network: In step 4-2), high-quality prior features are... Compared with the preliminary correction feature map After concatenation along the channel dimension, the data is input into a multinomial coefficient prediction network. This coefficient prediction network consists of three layers. The system consists of alternating convolutional layers and LeakyReLU activation functions, and finally passes through a single layer. The convolutional layer precisely maps the number of output channels to 8 channels. Utilizing... Guided by the extracted clear texture direction and lighting patterns, the network dynamically predicts the corresponding 8 mapping coefficients for each pixel position in the feature map. To construct a pixel-level adaptive mapping matrix.

[0058] 3) Image adaptive enhancement process: In step 4-3), the predicted mapping coefficient matrix is ​​used to preliminarily correct the feature map. Perform mapping operations to enhance the image. Each pixel value is calculated using the following formula (8), achieving dynamic brightness enhancement while maintaining physical structure:

[0059]

[0060] 4) Target detection network structure and loss constraint process: In step 4-4), the enhanced image... The input object detection network sequentially comprises: a backbone network for extracting multi-scale features, a neck network for fusing cross-scale semantic information, and a feature pyramid network containing... A depthwise separable convolutional detection head. It outputs semantic category labels and spatial bounding boxes for the target through classification and regression branches; its target detection loss function is... It consists of the classification cross-entropy loss and the generalized intersection-union ratio (GIoU) ​​regression loss, and the calculation formula (3) is as follows:

[0061]

[0062] In the formula, For real category labels, To predict class probabilities, This represents the total number of categories; For the predicted spatial location bounding box, The bounding box represents the actual spatial location. These are the regression weight coefficients. By introducing a detailed network layer structure and precise loss constraints, we ensure that the deep semantic features of the enhanced image can be effectively parsed.

[0063] 5) End-to-end joint optimization process: During the training phase, the overall DGA-ISP system jointly optimizes the preceding reversible normalized flow feature alignment module with the subsequent detection task. The end-to-end total training loss function... negative log-likelihood loss With target detection loss The weighted summation form, formula (4), is as follows:

[0064]

[0065] In the formula, To balance the hyperparameter weights of the feature distribution alignment task and the object detection task gradients, a joint loss is used. Guided by this loss, the backpropagated gradient can directly penetrate the detection network and update the parameters of the preceding ISP and multinomial modules, completely breaking down the barrier between traditional image enhancement and machine vision detection.

[0066] This technical solution addresses the problem of extremely low-light environments where the very limited number of photons and complex sensor noise lead to extremely dark target images, blurred edges, and complete loss of texture structure. It also addresses the issue that existing isolated image enhancement algorithms easily amplify noise, resulting in poor target detection accuracy and robustness. This solution first constructs a dynamic linear mapping module in the RAW domain, which contains the original physical information, for lossless adaptive color and lighting calibration. Then, it uses strictly lossless invertible normalized flow (I-LNF) to align the degraded feature distribution and supplements it with lossless high-resolution texture features retrieved from a pre-trained discrete prior feature dictionary. Finally, it uses these high-quality prior features to guide pixel-level prediction of polynomial nonlinear coefficients, achieving adaptive image brightening. The front-end ISP image processing and the back-end target detection network are jointly optimized end-to-end using a unified loss function.

[0067] This method completely breaks down the barrier between traditional image signal processing (ISP) and machine vision detection tasks, avoiding the noise cascading amplification effect caused by forcibly brightening in the conventional sRGB color space. Through a discrete dictionary retrieval and replacement mechanism, it effectively supplements high-frequency details that are difficult to capture in low-light environments. Simultaneously, it uses detection loss to guide the direction of image enhancement, making the enhanced image's feature representation capabilities more aligned with the recognition needs of machine vision. This significantly reduces the model's false negative and false positive rates for small targets and severely occluded objects in low light, thereby significantly improving the final accuracy and generalization ability of target detection in extremely complex low-light scenes. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of the DGA-ISP network model in the embodiment;

[0069] Figure 2 This is a schematic diagram of the DFQ-GAN structure and dictionary construction in the embodiment;

[0070] Figure 3 This is a schematic diagram of the dynamic linear mapping module in the embodiment;

[0071] Figure 4 This is a schematic diagram of the I-LNF feature alignment and retrieval module in the embodiment;

[0072] Figure 5 This is a schematic diagram of the affine coupling layer structure in the embodiment;

[0073] Figure 6 This is a schematic diagram of the polynomial nonlinear enhancement module in the embodiment;

[0074] Figure 7 This is a schematic diagram of the end-to-end joint optimization process in the embodiment. Detailed Implementation

[0075] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this is not intended to limit the scope of the invention.

[0076] Example:

[0077] Reference Figure 1 As shown, a low-light target detection method based on DGA-ISP includes the following steps:

[0078] 1) The low-light Bayer RAW image dataset and the corresponding normal-light dataset (using a self-built dataset containing 2000 pairs of high-quality RAW images) are processed and partitioned, and a discrete prior feature dictionary is obtained by pre-training a Discrete Feature Quantization Adversarial Network (DFQ-GAN). The process is as follows:

[0079] 1-1) Collect and integrate low-light RAW images and their corresponding detection labels, and construct the DGA-ISP-Data dataset together with normal-light images (this self-built dataset contains 2000 pairs of high-quality RAW images). Divide this dataset into training and testing sets in an 8:2 ratio, as shown in Table 1:

[0080] Table 1:

[0081] Data content Training set (80%) Test set (20%) Total dataset (100%) Remark Normal illumination image dataset Nor-images-Train Nor-images-Test Nor-images-Data Used for pre-training to extract high-quality prior features Low-light image dataset Dark-images-Train Dark-images-Test Dark-images-Data Includes detection labels for end-to-end training.

[0082] in,

[0083] 1-1-1) Nor-images-Train: This subset contains a large number of well-lit, textured RAW images. In implementation, it is specifically used for the independent pre-training stage of the Discrete Feature Quantization Adversarial Network (DFQ-GAN). The network learns and extracts high-quality high-frequency structures and true color distributions by encoding and vectorizing sharp image patches from this subset, thereby constructing a clear and lossless prior feature dictionary.

[0084] 1-1-2) Nor-images-Test: This is mainly used to evaluate the reconstruction quality and feature quantization error of DFQ-GAN in real time during the pre-training stage, ensuring that the generated prior feature dictionary also has good representation and generalization ability for unseen normal illumination scenes, and preventing the dictionary from overfitting.

[0085] 1-1-3) Dark-images-Train: This subset is the core data-driven source for end-to-end joint optimization in this invention. It not only includes degraded RAW images with extremely low illumination and high noise, but also strictly binds the ground truth bounding boxes and class labels of the targets. During the training phase, it is input into the complete DGA-ISP network, and through backpropagation of the total loss function, it synchronously guides the parameter updates of the front-end dynamic calibration matrix, I-LNF alignment network, polynomial stretching coefficients, and back-end target detection network.

[0086] 1-1-4) Dark-images-Test: This serves as a blind evaluation benchmark for the final model performance and is used only during the model's forward inference testing phase. By comparing the detection results output by the model on this subset with the true labels, the accuracy and robustness of the invention in target detection under unseen complex low-light and high-noise environments are verified.

[0087] 1-2) Pre-training of prior feature dictionary: Refer to Figure 2 As shown, a Discrete Feature Quantization Adversarial Network (DFQ-GAN) is pre-trained using a normal illumination image dataset. In the specific implementation, the Adam optimizer is used, with an initial learning rate set to... The batch size was set to 16, and the training lasted for 200 epochs. A network structure including an encoder, quantizer, and decoder was constructed to encode the input sharp image into a latent space for discretization and aggregation. The decoder then outputs the reconstructed high-resolution image. During this image reconstruction and adversarial training process, backpropagation continuously updated the network weights and constructed a discrete prior feature dictionary. In the training process of the Discrete Feature Quantization Adversarial Network (DFQ-GAN), in addition to calculating the image reconstruction loss and adversarial loss, the quantization loss function formula (1) used to directly optimize the prior feature dictionary is as follows:

[0088]

[0089] This step uses a vector quantization mechanism to encode common texture features in high-definition images into 1024 discrete codewords (in this embodiment, the feature dimension of each codeword is set to 64 dimensions) and store them in a dictionary, providing strong prior support for repairing degraded features contaminated by noise in subsequent processing.

[0090] 1-3) Data Label Preprocessing: The low-light image dataset is preprocessed to extract the semantic category labels and spatial location bounding boxes (center point coordinates, width, and height) of the targets, which serve as a supervised reference for subsequent target detection training. The bounding box contains the target's center point coordinates, border length, and height. By establishing refined annotation information, accurate ground truth is provided for the joint optimization of the detection heads in the subsequent step 4).

[0091] 2) Reference Figure 3 A dynamic linear mapping module is constructed to extract local and global features of the RAW image and generate an adaptive sensor calibration matrix to complete the initial color correction. The specific implementation process is as follows:

[0092] 2-1) Two-stream feature extraction: The input four-channel Bayer RAW image is fed into a two-stream network architecture composed of residual blocks. In specific implementation, on the one hand, the local stream network adopts three consecutive layers. Convolutional layers are connected to residuals in each layer. After the convolutional kernel performs spatial sliding to capture the differences between adjacent pixels, it is immediately followed by a ReLU non-linear activation function. The convolutional operation extracts a 64-dimensional local feature vector describing pixel-level details, texture edges, and noise distribution. On the other hand, the Global Flow Network contains a global average pooling layer and two fully connected layers with a hidden dimension of 128. By aggregating the contextual lighting information of the entire image, it extracts a 64-dimensional global feature vector describing the overall color shift distribution. This dual-stream design enables the network to simultaneously address microscopic pixel-level degradation and macroscopic changes in ambient light when processing low-light images, providing a comprehensive feature base for accurate correction.

[0093] 2-2) Attention Weight Calculation: Introducing a cross-attention mechanism, setting up independent linear network layers to calculate the local feature vectors extracted above. With global feature vectors Each is mapped to a 64-dimensional query matrix. and At the same time, with the preset Dimensional basic camera processing matrix (After being flattened and mapped to 64 dimensions) is a key matrix. Sum of Value Items Matrix By calculating relevance weights using a scaled dot product attention formula, local texture compensation is dynamically and in parallel generated based on physical priors. Dimensional Local Calibration Matrix And for global color cast correction Dimensional global calibration matrix The core calculation formulas (6) and (7) are as follows:

[0094]

[0095]

[0096] This step utilizes a cross-attention mechanism to give the model "adaptive" perception capabilities, enabling it to no longer rely on fixed processing parameters but to dynamically calculate the optimal color compensation weights based on the specific lighting conditions of the current input image.

[0097] 2-3) Linear mapping correction: using the fusion formula The fundamental matrix With dynamically generated calibration offset and By performing linear addition, the final adaptive linear transformation matrix is ​​obtained. In implementation, it will be treated as The convolutional kernel is applied directly to the original Bayer RAW image to perform channel mapping and color correction, reducing the number of channels from 4 to 3, and finally outputting a pre-corrected feature map. Traditional image signal processing tends to amplify noise in complex nonlinear spaces. This step strictly limits the linear mapping operation to the RAW domain, which not only successfully corrects color cast but also preserves the original physical response characteristics of the sensor to the greatest extent, laying a clean data foundation for subsequent feature alignment.

[0098] 3) Reference Figure 4 and reference Figure 5 A feature alignment and retrieval module based on reversible normalized flow (I-LNF) is constructed to align the calibrated feature distribution with the normal illumination features and retrieve lossless, high-quality prior features from the feature dictionary. The specific implementation process is as follows:

[0099] 3-1) Construct a conditional coding network to extract degradation guidance features: The preliminary corrected feature map output from step 2) The input is fed into a conditional encoder consisting of multiple residual convolutional layers. In this embodiment, the conditional encoder contains four cascaded residual convolutional blocks, each followed by a downsampling layer with a stride of 2. Through layer-by-layer downsampling in the spatial dimension and depth extraction of channel information, 64-dimensional conditional features capable of characterizing the global illumination distribution, noise intensity, and structural degradation degree of low-light scenes are obtained. The degradation degree of low-light images is often uneven across different regions. By extracting this conditional feature, we can provide accurate spatial location guidance and distribution mapping constraints for subsequent feature transformation, which is equivalent to providing the network with a "navigation map" that tells us the recovery strength at different locations.

[0100] 3-2) Perform feature distribution alignment using reversible normalized flows: e.g. Figure 4 As shown, a reversible transformation layer group is constructed. Before entering the reversible transformation layer group, it is preferable to pass the input feature map through a Squeeze layer. By performing checkerboard-style slicing in the spatial dimension and stacking it to the channel dimension, the spatial resolution of the feature map is reduced to half of the original, while the number of channels is increased from 64 dimensions to 256 dimensions, thereby enhancing the feature representation capability of the network without losing the original information.

[0101] Subsequently, given conditional features Under the constraints, an invertible transformation network consisting of 8 cascaded affine coupling layers is used to transform the feature map in the low-illuminance degradation distribution space to a linear latent space that fits the prior space of normal illuminance through nonlinear latent mapping. Its mathematical expression is as follows: In this process, the latent features of the output are minimized by reducing the negative log-likelihood loss function during the training phase. It approximates the normal illumination characteristic distribution in terms of probability distribution. The negative log-likelihood loss function is used to constrain the distribution transformation. Formula (2) is as follows:

[0102]

[0103] This step utilizes the rigorous mathematical bijective property of reversible normalized flow (I-LNF) to achieve cross-alignment from low-light degenerate distribution to normal prior distribution without losing any physical dimensions, thus eliminating nonlinear noise interference.

[0104] 3-3) Retrieving a high-quality prior feature dictionary based on Euclidean distance: This involves retrieving a high-quality prior feature dictionary from a pre-trained discrete prior feature dictionary. Feature codeword retrieval based on the nearest neighbor algorithm is performed on a dataset containing 1024 discrete codewords of 64 dimensions. Specifically, the aligned latent features are calculated using the L2 norm. With each discrete codeword in the dictionary The minimum Euclidean distance between them is determined by the following formula:

[0105]

[0106] Based on the distance minimization criterion, the corresponding discrete codeword index is retrieved, and the unstable features in the latent space are replaced with lossless, high-fidelity codewords stored in the dictionary, thereby obtaining high-quality prior features containing clear texture and structural priors. This serves as a guiding signal for subsequent nonlinear enhancement modules. This step, through a "retrieval-replacement" mechanism, directly retrieves matching high-frequency structural features from a high-quality knowledge base, effectively solving the common image blurring problem in generative methods.

[0107] 4) Reference Figure 6 , Figure 7 A polynomial nonlinear module and an object detection network are constructed. Prior features are retrieved to guide polynomial coefficient prediction, enabling adaptive nonlinear stretching of the image and object detection. The semantic category label and spatial bounding box of the object are output. The specific implementation process is as follows:

[0108] 4-1) Preset nonlinear mapping basis function set: such as Figure 6 As shown, a set of nonconvex polynomial basis functions consisting of orders 1 to 8 is predefined. The basis functions, through predefined monotonicity and coordinate constraints, strictly pass through the origin (0,0) and the normalized endpoint (1,1) to simulate the nonlinear tone mapping characteristics in low-light imaging. Traditional global gamma curves often cannot simultaneously cover extremely dark and extremely bright regions. This step introduces high-order nonconvex polynomial basis functions, providing the network with a highly flexible brightness adjustment space, ensuring that while brightening targets in dark areas, it does not cause overexposure of background light sources.

[0109] 4-2) Specific structure and process of the multinomial coefficient prediction network: The high-quality prior features obtained in step 3) are used to predict the multinomial coefficients. Compared with the preliminary correction feature map After concatenation along the channel dimension, the data is input into a multinomial coefficient prediction network. In a specific implementation, the coefficient prediction network consists of three consecutive layers. The system consists of alternating convolutional layers and LeakyReLU activation functions, and finally passes through a single layer. The convolutional layer precisely maps the number of output channels to 8 channels. Utilizing... The provided clear texture direction and illumination distribution patterns serve as "mentor signals," guiding the network to predict eight pixel-level mapping coefficients that correspond one-to-one with each basis function. Due to feature maps In low-light conditions, relying solely on its own predictions is prone to bias. Introducing high-quality prior features into the prediction is like giving the network "night vision goggles," enabling it to accurately determine the edges of objects and apply appropriate stretching weights at those edges.

[0110] 4-3) Perform adaptive nonlinear stretching to generate enhanced images: Apply the predicted coefficient matrix to the preliminary correction feature map. Perform nonlinear mapping operations to enhance the image. Each pixel value is calculated using the following formula (8):

[0111]

[0112] This mapping process is calculated independently at the pixel level, meaning that each pixel in the image has its own stretching curve. This extreme adaptability effectively supplements texture details while maintaining physical consistency, greatly restoring the shape of the target hidden in low light.

[0113] 4-4) Target detection output and network structure: such as Figure 7 As shown, the generated enhanced image The input is fed into the object detection network. In a preferred embodiment, the object detection network adopts the YOLO architecture, which includes a backbone network (using a CSP structure) for extracting multi-scale features, a neck network (using a PANet structure) for fusing cross-scale semantic information, and a feature pyramid neck network (using a PANet structure) for fusing cross-scale semantic information. A decoupled detection head using depthwise separable convolutions. The target detection network outputs its target detection loss function when providing prediction results. It consists of the classification cross-entropy loss and the generalized intersection-union ratio (GIoU) ​​bounding box regression loss, as shown in formula (3):

[0114]

[0115] The optimization goal of a conventional ISP system is to make images “look good.” This step introduces object detection loss to directly supervise the front-end ISP network, overturning the traditional fragmented paradigm.

[0116] 4-5) End-to-end joint optimization process: During the training phase, the overall DGA-ISP model enables the preceding ISP feature alignment module to spontaneously serve the final target recognition task. Its final training optimization results in a total loss function... negative log-likelihood loss With target detection loss The sum of these, and the formula (4) for the total loss function, are as follows:

[0117]

[0118] In this embodiment, the balancing weight hyperparameter Set to 0.1. Formula (4) mathematically represents "image underlying physical recovery ( ")" and "high-level semantic recognition" The end-to-end constraint allows backpropagation gradients to directly penetrate the detection network and update preceding modules, enabling the model to not only recover lost target features but also maintain extremely high detection accuracy and robustness even in extremely noisy environments.

Claims

1. As shown in Figure 1, a low-light target detection method based on DGA-ISP is characterized in that, Includes the following steps: 1) Constructing a discrete prior feature dictionary for normal illumination: The low-illumination Bayer RAW image dataset and the corresponding normal illumination dataset (using a self-built dataset containing 2000 pairs of high-quality RAW images) are processed and partitioned, and a discrete prior feature dictionary is obtained by pre-training a Discrete Feature Quantization Adversarial Network (DFQ-GAN). The process is as follows: 1-1) Collect and integrate low-light RAW images and their corresponding detection labels, and construct the DGA-ISP-Data dataset together with normal-light images (this self-built dataset contains 2000 pairs of high-quality RAW images). Divide this dataset into training and testing sets in an 8:2 ratio, as shown in Table 1. Table 1: 1-2) As shown in Figure 2, a Discrete Feature Quantization Adversarial Network (DFQ-GAN) is pre-trained using a normal illumination image dataset. By aggregating structural texture patches of clear images in the latent feature space, a discrete prior feature dictionary is constructed. During the training process of the Discrete Feature Quantization Adversarial Network (DFQ-GAN), it is used to optimize the prior feature dictionary. Quantization loss function Formula (1) is as follows: （1）； 1-3) Preprocess the low-light image dataset to extract the semantic category label and spatial location bounding box of the target in each image, which will serve as a supervised reference for subsequent target detection. 2) As shown in Figure 3, a dynamic linear mapping module is designed to extract local and global features of the RAW image and generate an adaptive sensor calibration matrix to complete the initial color correction. The process is as follows: 2-1) The input four-channel Bayer RAW image is fed into a two-stream network architecture composed of residual blocks. On the one hand, local feature vectors describing pixel-level details, texture edges, and noise distribution are extracted through convolutional layers. On the other hand, the global average pooling layer aggregates the contextual lighting information of the entire image to extract a global feature vector describing the overall color shift distribution. . 2-2) Introduce a cross-attention mechanism, using the extracted feature vectors respectively and For query items, using preset... Dimensional basic camera processing matrix For key and value terms, a local calibration matrix for local texture compensation is dynamically generated based on physical priors by calculating correlation weights. And a global calibration matrix for global color cast correction. . 2-3) Using the fusion formula The fundamental matrix is ​​linearly added to the dynamically generated calibration offset to calculate the final adaptive linear transformation matrix. It is then applied directly to the original RAW image to perform channel mapping and color correction, eliminating severe color shifts while preserving physical linearity, and finally outputting a pre-corrected feature map. . 3) As shown in Figures 4 and 5, a feature alignment and retrieval module based on reversible normalized flow (I-LNF) is designed to align the calibrated feature distribution with the normal illumination features and retrieve lossless, high-quality prior features from the feature dictionary. The process is as follows: 3-1) Construct a conditional coding network to extract degradation guidance features: The preliminary corrected feature map output from step 2) The input to the conditional encoder, which consists of multiple residual convolutional layers, involves layer-by-layer downsampling in the spatial dimension and depth extraction of channel information to obtain conditional features that characterize the global illumination distribution, noise intensity, and structural degradation in low-light scenes. This provides precise spatial location guidance and distribution mapping constraints for the subsequent reversible transformation process. 3-2) Performing feature distribution alignment using reversible normalized flow: Constructing a reversible transformation layer group based on a squeeze-free layer structure Given the features Under constraints, feature maps in low-light degradation distribution spaces are transformed through nonlinear latent mapping. Transforming to a linear latent space that fits the prior space of normal illumination, its mathematical expression is: In this process, the latent features of the output are minimized by reducing the negative log-likelihood loss function during the training phase. The probability distribution approximates the normal illumination feature distribution, thereby eliminating nonlinear noise interference in the input image and correcting feature deviations. During the feature distribution alignment process of the reversible normalized flow, the negative log-likelihood loss function is used to constrain the distribution transformation. Formula (2) is as follows: （2）。 3-3) Retrieving a high-quality prior feature dictionary based on Euclidean distance: This involves retrieving a high-quality prior feature dictionary from a pre-trained discrete prior feature dictionary. The process performs feature codeword retrieval based on the nearest neighbor algorithm, and calculates the aligned latent features. Minimum Euclidean distance between the dictionary and each discrete codeword The unstable features in the latent space are replaced with lossless, high-fidelity codewords stored in the dictionary, thereby obtaining high-quality prior features containing clear texture and structural priors. This serves as a guiding signal for subsequent nonlinear enhancement modules. 4) As shown in Figures 6 and 7, a polynomial nonlinear module and a target detection network are designed. The retrieved prior features guide the prediction of polynomial coefficients, completing adaptive nonlinear stretching of the image and target detection. The semantic category label and spatial bounding box of the target are output. The process is as follows: 4-1) Preset nonlinear mapping basis function set: As shown in Figure 6, a set of nonconvex polynomial basis functions consisting of first to n orders is pre-defined. The basis functions simulate the nonlinear tone mapping characteristics in low-light imaging through predefined monotonicity and coordinate constraints, aiming to specifically stretch the dark details of the image during subsequent enhancement and effectively suppress the bright part overflow and color distortion that may occur in high dynamic range scenes. 4-2) Using prior features to guide polynomial coefficient prediction: The high-quality prior features retrieved in step 3) Compared with the preliminary correction feature map After feature fusion, the input is given to a convolutional neural network, which then utilizes... The provided clear texture direction and illumination distribution patterns serve as "mentor signals," guiding the network to predict pixel-level coefficient matrices that correspond one-to-one with each basis function. This enables accurate modeling of the enhancement levels in different spatial regions of an image. 4-3) Perform adaptive nonlinear stretching to generate enhanced images: Apply the predicted coefficient matrix to the preliminary correction feature map. Perform nonlinear mapping operations using the formula Dynamic brightness and color adjustments are made to each pixel, resulting in an enhanced image with significantly improved visual quality, effectively enhanced texture details, and physical consistency. . 4-4) Object detection output and end-to-end joint optimization: As shown in Figure 7, the generated enhanced image is... The input is fed into an object detection network containing a classification head and a regression head. Through deep feature extraction and bounding box regression, the final output is the semantic category label and spatial location bounding box of the object. During the training phase, the entire DGA-ISP system is jointly optimized end-to-end using object detection loss and self-enhancement regularization loss. This enables the preceding ISP processing modules to spontaneously learn the enhancement strategies most beneficial to the object recognition task. When the object detection network outputs prediction results, its object detection loss function... Classification loss With bounding box regression loss The composition, formula (3) is as follows: （3）。 4-5) During the end-to-end joint training process, the overall DGA-ISP model enables the preceding ISP feature alignment module to spontaneously serve the final target recognition task, and its final training optimization of the total loss function... The total loss function, which is the sum of the negative log-likelihood loss and the target detection loss, is given by formula (4) as follows: （4）。 2. The low-light target detection method based on DGA-ISP according to claim 1, characterized in that, The dataset construction and prior dictionary initialization process described in step 1) is as follows: 1) Dataset partitioning process: In step 1-1), the DGA-ISP-Data dataset is constructed by integrating image pairs from the same source but with different lighting conditions. Nor-images-Data serves as a normal illumination reference set and is used to construct the feature space of the Discrete Feature Quantization Adversarial Network (DFQ-GAN) (see Figure 2). Dark-images-Data serves as a training set for low-light target detection, and the ratio of the training set to the test set is strictly limited to 8:

2. 2) Feature dictionary construction process: In steps 1-2), the discrete feature quantization adversarial network is trained using a subset of Nor-images-Train. By performing vector quantization on the latent features of high-definition images, a prior feature dictionary containing 1024 discrete codewords is constructed, and the feature dimension of each codeword is defined as 64 dimensions. 3) Label Preprocessing: In steps 1-3), the annotation information of each object to be detected is extracted from Dark-images-Data. The annotation information includes the target's category label, center point coordinates, and border width and height. Simultaneously, a detection difficulty assessment mechanism is set up for any target in the image. Its detection difficulty level The determination formula (5) is as follows: （5） In the formula, For the goal pixel area The preset threshold for small target pixel size; For the goal With any other target The intersection and union ratio, This is a preset overlap threshold; Targets classified as 1 are treated as difficult samples and given weighted attention in subsequent training.

3. The low-light target detection method based on DGA-ISP according to claim 1, characterized in that, The specific construction and calibration process of the dynamic linear mapping module mentioned in step 2) is as follows: 1) Two-stream feature extraction process: utilizing a three-layer continuous... The local flow network, with convolutional layers and residual connections, extracts a 64-dimensional local feature vector by performing local spatial convolution operations on the input Bayer RAW image. Simultaneously, a global flow network containing a global average pooling layer and two fully connected layers (with 128 hidden dimensions) is used to extract a 64-dimensional global feature vector by performing mean statistics and non-linear dimensionality compression on all pixels in the image. ; 2) Attention weight calculation process: Three independent linear network layers are set up to calculate the local feature vectors respectively. global feature vector Mapped to a 64-dimensional query term matrix and And flattened Dimensional Basic Camera Processing Matrix The unified mapping is a 64-dimensional key-value matrix. With numerical term matrix ; The scaling dot product attention mechanism is used to calculate the correlation weights between features and the fundamental matrix, and then the output is computed in parallel. Dimensional local calibration matrix With global calibration matrix The core attention calculation formulas (6) and (7) are as follows: （6） （7） 3) Linear mapping correction process: using matrix fusion formula The final adaptive linear transformation matrix is ​​calculated. and take it as The convolutional kernel is applied to the original Bayer RAW image, mapping the number of channels from 4 to 3, and outputting a pre-corrected feature map. .

4. The low-light target detection method based on DGA-ISP according to claim 1, characterized in that, The feature alignment and retrieval process based on reversible normalized flow described in step 3) is as follows: 1) Multi-scale conditional encoding process: In step 3-1), a conditional encoder containing 4 layers of cascaded residual convolutional blocks and a downsampling layer with a stride of 2 is used to pre-correct the feature map. Hierarchical feature extraction was performed to obtain 64-dimensional conditional features describing the degradation level of the current scene and global semantic information. ; 2) Spatial dimension squeezing transformation process: Before entering the reversible transformation network, the input feature map is sliced ​​in the spatial dimension and stacked to the channel dimension through the Squeeze layer, which reduces the spatial resolution of the feature map to 1 / 2 of the original, while increasing the number of channels from 64 dimensions to 256 dimensions, so as to enhance the feature expression ability of the network without losing the original information. 3) Alignment transformation process based on affine coupling layers: In step 3-2), a reversible transformation network consisting of 8 cascaded affine coupling layers is constructed. In terms of conditional features Under the guidance of constraints, the feature map in the degenerate space is transformed by performing deterministic distribution distortion. Project the data onto a linear latent space, making its probability distribution approximate the normal illumination feature distribution, and output the aligned latent features. ; 4) High-quality prior feature retrieval process: In step 3-3), latent features are calculated using the L2 norm (Euclidean distance). With a prior feature dictionary containing 1024 discrete codewords The similarity distance between them; based on the distance minimization criterion The corresponding discrete codeword index is retrieved, and the 64-dimensional quantized vector under that index is extracted as the retrieval result. High-quality prior features with high-resolution texture supplementation capability are then reconstructed. .

5. The low-light target detection method based on DGA-IS according to claim 1, characterized in that, The specific process and end-to-end optimization of the polynomial nonlinear module and target detection network described in step 4) are as follows: 1) Construction process of nonlinear mapping basis functions: In step 4-1), a set of monotone nonconvex polynomial basis functions consisting of orders 1 to 8 is preset. The basis functions are forced to pass strictly through the origin (0,0) and the normalized endpoint (1,1) in the spatial coordinate system through coordinate constraints, and are used to simulate the nonlinear large stretching of the dark area of ​​RAW data and the suppression of the overflow of the highlight area. 2) Polynomial coefficient prediction process: In step 4-2), high-quality prior features are... Compared with the preliminary correction feature map After feature fusion, the input coefficient prediction network is used. The extracted clear texture direction and lighting patterns serve as a guide for the feature map. The eight mapping coefficients corresponding to each pixel position are dynamically predicted. To construct a pixel-level adaptive mapping matrix; 3) Image adaptive enhancement process: In step 4-3), the predicted mapping coefficient matrix is ​​used to preliminarily correct the feature map. Perform mapping operations to enhance the image. Each pixel value is calculated using the following formula (8), achieving dynamic brightness enhancement while maintaining physical structure: （8） 4) Target detection and loss constraint process: In step 4-4), the enhanced image... Input the object detection network, and output the semantic category label and spatial location bounding box of the object through classification and regression branches; Its target detection loss function It consists of the classification cross-entropy loss and the generalized intersection-union ratio (GIoU) ​​regression loss, and the calculation formula is shown in the above formula (3): In the formula, For real category labels, To predict class probabilities, This represents the total number of categories; For the predicted spatial location bounding box, The bounding box represents the actual spatial location. These are the regression weight coefficients; 5) End-to-end joint optimization process: During the training phase, the overall DGA-ISP system jointly optimizes the preceding reversible normalized flow feature alignment module with the subsequent detection task. The end-to-end total training loss function... negative log-likelihood loss With target detection loss The weighted summation form is shown in formula (4): In the formula, Hyperparameter weights are used to balance the gradients of the feature distribution alignment task and the object detection task.

Images