Zero-shot frequency domain and spatial domain dual-branch low-light target detection method and system thereof

Abstract

The application discloses a zero-sample frequency domain and spatial domain double-branch low-light target detection method and system, and the method comprises the following steps: inputting a normal-light image into a dark processing module to obtain a low-light image; inputting the low-light image into a spatial domain non-light feature extraction module to obtain spatial domain non-light features and inputting the low-light image into a frequency domain non-light feature extraction module to obtain frequency domain non-light features; obtaining a light component of the low-light image by using a light component estimation module based on the low-light image, and adaptively fusing the light component of the low-light image with a multi-scale feature map to obtain an enhanced image. The detection system comprises the dark processing module, the spatial domain non-light feature extraction module, the frequency domain non-light feature extraction module, an adaptive fusion module, a channel attention module and the light component estimation module. The method can effectively improve the target detection performance in a low-light environment without the participation of real low-light data, and has strong modular compatibility and industrial adaptability.

Description

Technical Field

[0001] This invention relates to the field of computer vision, specifically to a zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method and system. Background Technology

[0002] Low-light target detection is an important technology in computer vision, designed to handle target recognition and detection tasks in low-light environments. With the continuous development of applications such as intelligent security, autonomous driving, and drone inspection, the demand for low-light target detection technology is increasing. In low-light environments, due to insufficient ambient light, target images often suffer from low contrast, high noise, and blurred details, posing significant challenges to the accuracy and robustness of target detection algorithms.

[0003] Since the early 1990s, researchers in object detection have been exploring ways to improve performance in low-light environments. From traditional methods based on handcrafted features to the introduction of deep learning, low-light object detection technology has undergone several stages of development. Early research primarily relied on image enhancement techniques, such as histogram equalization, contrast enhancement, and noise suppression, to improve the quality of low-light images. However, these traditional methods often failed to fully extract deep features from images and had limited effectiveness in complex environments. With the rapid development of deep learning technology, especially the application of convolutional neural networks (CNNs) and generative adversarial networks (GANs), low-light object detection methods have gradually shifted towards automation and end-to-end learning approaches. In recent years, many studies have begun to utilize deep learning models for low-light image processing and object detection, improving the model's adaptability to images under low-light conditions. For example, object detection algorithms based on deep convolutional neural networks can automatically learn features from low-light images and effectively identify targets, no longer relying on manually designed features.

[0004] However, target detection in low-light environments still faces many challenges, especially regarding the loss of target information and image quality degradation in low-light images. First, low-light environments introduce noise into captured images or videos, resulting in unclear target edges and thus affecting detection performance. Second, due to poor lighting conditions, details in the image are often difficult to capture, placing higher demands on deep learning-based target detection algorithms. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method and system, aiming to solve the problems of low coupling between enhancement and detection tasks, strong dependence on real low-light data, and insufficient feature representation ability in existing low-light target detection methods.

[0006] The technical solution adopted in this invention is: a zero-sample frequency domain and spatial domain dual-branch low-light target detection method and system, the specific steps of which include:

[0007] Step S1: Input a normal lighting image into the dark processing module to obtain a low-light image;

[0008] Step S2: Input the low-light image obtained in step S1 into the spatial domain non-light feature extraction module to obtain spatial domain non-light features and input it into the frequency domain non-light feature extraction module to obtain frequency domain non-light features; at the same time, input the normal lighting image into the spatial domain non-light feature extraction module to obtain spatial domain non-light features and input it into the frequency domain non-light feature extraction module to obtain frequency domain non-light features.

[0009] Step S3: Based on the spatial domain non-illuminated features and frequency domain non-illuminated features obtained in step S2, input them into the adaptive fusion module for fusion to obtain a preliminary multi-scale feature map;

[0010] Step S4: Based on the preliminary multi-scale feature map obtained in step S3, the channel attention module is used to obtain the final multi-scale feature map;

[0011] Step S5: Input the low-light image from step S1 into the illumination component estimation module to obtain the illumination component of the low-light image. Input the illumination component of the low-light image and the final multi-scale feature map from step S4 into the adaptive fusion module for adaptive fusion. After obtaining the enhanced image, input it into the target detection model.

[0012] Furthermore, in step S1, the normal lighting image is input into the dark processing module to obtain the low-light image; the specific steps are as follows:

[0013] Step S11: Apply an inverse gamma transform to each pixel of the input normal illumination image;

[0014] Step S12: Using a randomly selected camera color correction matrix, each pixel processed in step S11 is converted from the standard red-green-blue color space to the camera's native color space to obtain each pixel after color space conversion.

[0015] Step S13: For each pixel after color space conversion in step S12, a brightness scaling factor is randomly sampled from the light attenuation coefficient distribution. The brightness scaling factor of each pixel after color space conversion in step S12 is then used for overall intensity scaling to obtain a preliminary low-light image.

[0016] Step S14: Random noise is added to the low-light image initially generated in step S13 to obtain the low-light image.

[0017] Further, in step S2, the low-light image obtained in step S1 is input into the spatial domain non-lighting feature extraction module to obtain spatial domain non-lighting features; the specific steps are as follows:

[0018] Step S201: The spatial domain non-illumination feature extraction module applies a logarithmic transformation to each of the three channels of the input low-illumination image; low-illumination image Where R, G, and B represent the red channel, green channel, and blue channel, respectively;

[0019] Step S202: Use shallow convolutional kernels to perform preliminary feature extraction on the red channel R, green channel G, and blue channel B to obtain three sets of basic mapping features, as shown in the formula:

[0020] ;

[0021] in, This represents the basic mapping feature of the first channel c in pixel coordinates. The first channel c belongs to one of the three channels: red channel R, green channel G, and blue channel B. `log` represents the logarithmic transformation. This represents the shallow convolution kernel acting on the first channel c. This represents the pixel value at pixel coordinate c in the first channel of the low-light image;

[0022] Step S203: Using three sets of basic mapping features, the three channels (red channel R, green channel G, and blue channel B) are paired in the spatial domain, and the non-illumination features between channels are obtained by calculating the channel element-level difference using cross-color ratio, as shown in the formula:

[0023] ;

[0024] Where c1 is the second channel and c2 is the third channel, the second channel c1 and the third channel c2 are also one of the three channels: red channel R, green channel G, and blue channel B. ; This indicates that the non-illumination characteristics between the second channel c1 and the third channel c2 are obtained by cross-color ratio calculation; This represents the basic mapping feature of the second channel c1 in pixel coordinates; This represents the basic mapping feature of the third channel c2 in pixel coordinates;

[0025] Step S204: The non-illuminated features of the three channels (red channel R, green channel G, and blue channel B) are concatenated along the channel direction to obtain the spatial domain non-illuminated features, as shown in the formula:

[0026] ;

[0027] in, Represents the non-illuminated features of the spatial domain. This indicates a splicing operation. This represents the non-illuminated features between the red and green channels. This represents the non-illumination characteristics between the red and blue channels. Indicates the non-illuminating characteristics between the green channel and the blue channel;

[0028] Similarly, the non-illuminated features in the spatial domain are obtained by inputting the normally illuminated image into the spatial domain non-illuminated feature extraction module based on steps S201-S204.

[0029] Furthermore, the low-light image obtained in step S1 is input into the frequency domain non-lighting feature extraction module to obtain frequency domain non-lighting features; the specific steps are as follows:

[0030] Step S211: The frequency domain non-illumination feature extraction module applies a two-dimensional Fourier transform to each of the three channels of the low-illumination image to obtain the complex spectral features of the corresponding channels. ,in, The real part represents the characteristic of the spectrum of a complex number. Represents the imaginary unit. The imaginary part represents the spectral characteristics of a complex number;

[0031] Step S212: Design a complex convolution kernel, using a pair of weights applied to the real and imaginary parts of the input features, as shown in the formula:

[0032] ;

[0033] in, This represents the frequency domain features extracted from the first channel c through complex convolution. The weights representing the complex spectral characteristics of the first channel c are... The weights representing the real part of the complex spectral characteristics of the first channel c are... The weights represent the imaginary part of the complex spectral characteristics of the first channel c;

[0034] Step S213 involves extracting the frequency domain features from the first channel c in the frequency domain through complex convolution. Perform inverse Fourier transform back to the spatial domain spatial domain include , , In the spatial domain The cross-color ratio feature is obtained by performing cross-color ratio analysis, as shown in the formula:

[0035] ;

[0036] in, This represents the cross-color ratio feature between the red and green color channels extracted in the frequency domain. This represents the features of the red channel extracted by complex convolution, which are then transformed back to the spatial domain by inverse Fourier transform. This represents the features of the green channel extracted through complex convolution, which are then transformed back to the spatial domain by inverse Fourier transform.

[0037] Similarly, we can obtain:

[0038] ;

[0039] ;

[0040] in, This represents the features of the blue channel extracted through complex convolution, which are then transformed back to the spatial domain using inverse Fourier transform. This represents the cross-color ratio feature between the red and blue channels extracted in the frequency domain. This represents the cross-color ratio feature between the green and blue channels extracted in the frequency domain;

[0041] Step S214: Concatenate the cross-color ratio features to obtain the frequency domain non-illumination features, as shown in the formula:

[0042] ;

[0043] in, Represents non-illuminated features in the frequency domain;

[0044] Similarly, the normal illumination image is input into the frequency domain non-illumination feature extraction module to obtain the frequency domain non-illumination features based on steps S211-S214.

[0045] Furthermore, in step S3, the spatial domain non-illuminated features and frequency domain non-illuminated features obtained in step S2 are input into the adaptive fusion module for fusion to obtain a preliminary multi-scale feature map; the specific steps are as follows:

[0046] Step S31, spatial domain non-illuminated features Frequency domain non-illuminated features The concatenated feature map is obtained by concatenating along the channel dimension. ;

[0047] Step S32: A 3×3 convolutional layer is used to concatenate the feature maps. Compression is performed, and two dynamic weight graphs are generated by activating a normalized exponential function, as shown in the formula:

[0048] ;

[0049] in, A dynamic weighted graph representing the metric space domain. Represents a dynamic weighted graph in the frequency domain of the metric. Represents the normalized exponential function, () indicates a convolution operation to perform a 3×3 convolutional layer;

[0050] Step S33, based on spatial domain non-illuminated features Frequency domain non-illuminated features And two dynamic weight maps yield preliminary multi-scale feature maps; as shown in the formula:

[0051] ;

[0052] in, This represents a preliminary multi-scale feature map.

[0053] Furthermore, in step S4, the preliminary multi-scale feature map obtained in step S3 is used to obtain the final multi-scale feature map using a channel attention module; the specific steps are as follows:

[0054] Step S41, process the preliminary multi-scale feature map Apply global average pooling, as shown in the formula:

[0055] ;

[0056] in, The result represents the global average pooling result, where H represents the height of the multi-scale feature map, W represents the width of the multi-scale feature map, m represents the index of the multi-scale feature map in height, and n represents the index of the multi-scale feature map in width.

[0057] Step S42: Construct a nonlinear channel relationship model using two fully connected layers, as shown in the formula:

[0058] ;

[0059] Where s represents the attention vector, This represents the Sigmoid function. Represents the ReLU function. , These represent the weight matrices used by the channel attention module for dimensionality reduction and dimensionality increase, respectively.

[0060] Step S43: Re-weight and fuse the initial multi-scale feature maps using the attention vector s. Each channel is as shown in the formula:

[0061] ;

[0062] in, This represents the final multi-scale feature map;

[0063] Step S44: Set a consistency loss for the final multi-scale feature map. The final multi-scale feature map is divided into the final multi-scale feature map of the low-light image and the final multi-scale feature map of the normal-light image.

[0064] Furthermore, in step S5, the low-light image from step S1 is input into the illumination component estimation module to obtain the illumination component of the low-light image. The illumination component of the low-light image is then input into the adaptive fusion module along with the final multi-scale feature map from step S4 for adaptive fusion. The resulting enhanced image is then input into the target detection model. The specific steps are as follows:

[0065] Step S51: The input low-light image is smoothed by Gaussian filtering, and the illumination component of the low-light image is obtained by multi-channel gradient calculation.

[0066] Step S52: Adaptively fuse the illumination component of the low-light image with the multi-scale feature map of the low-light image in the adaptive fusion module to obtain the enhanced image. Then, put the enhanced image into the target detection model for target detection.

[0067] Furthermore, a zero-sample frequency-domain and spatial-domain dual-branch low-light target detection system is provided, applied to the zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method. The detection system includes:

[0068] The dark processing module is used to obtain a low-light image from the input normal lighting image;

[0069] The spatial domain non-illuminated feature extraction module is used to extract spatial domain non-illuminated features from illuminated images;

[0070] The frequency domain non-illuminated feature extraction module is used to extract the frequency domain non-illuminated features of illuminated images;

[0071] The adaptive fusion module is used to fuse the non-illuminated features extracted by the spatial domain non-illuminated feature extraction module and the frequency domain non-illuminated feature extraction module to obtain a preliminary multi-scale feature map.

[0072] The channel attention module is used to obtain the final multi-scale feature map from the initial multi-scale feature map through the channel attention mechanism.

[0073] The illumination component estimation module is used to extract the illumination components of low-light images.

[0074] The beneficial effects of this invention are:

[0075] This invention proposes a method that can seamlessly integrate with any target detection model, achieving excellent results in low-light target detection tasks. Compared to images without low-light enhancement, the method proposed in this invention improves the average accuracy mAP50 of the benchmark model Yola by 4.6%.

[0076] The frequency domain non-illumination feature extraction module proposed in this invention can effectively remove the influence of illumination components. At the same time, since feature extraction is performed in the frequency domain, it maintains a lightweight design while effectively extracting global features of the illuminated image. Attached Figure Description

[0077] Figure 1 This is a flowchart illustrating the present invention.

[0078] Figure 2 This is an overall framework diagram of the present invention.

[0079] Figure 3 This is a diagram of the frequency domain non-illuminated feature extraction module of the present invention.

[0080] Figure 4 This is a diagram of the spatial domain non-illuminated feature extraction module of the present invention. Detailed Implementation

[0081] This embodiment provides a zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method and system, such as... Figure 1 As shown, it includes the following steps:

[0082] Step S1: Input a normal lighting image into the dark processing module to obtain a low-light image;

[0083] Step S2: Input the low-light image obtained in step S1 into the spatial domain non-light feature extraction module to obtain spatial domain non-light features and input it into the frequency domain non-light feature extraction module to obtain frequency domain non-light features; at the same time, input the normal lighting image into the spatial domain non-light feature extraction module to obtain spatial domain non-light features and input it into the frequency domain non-light feature extraction module to obtain frequency domain non-light features.

[0084] Step S3: Based on the spatial domain non-illuminated features and frequency domain non-illuminated features obtained in step S2, input them into the adaptive fusion module for fusion to obtain a preliminary multi-scale feature map;

[0085] Step S4: Based on the preliminary multi-scale feature map obtained in step S3, the channel attention module is used to obtain the final multi-scale feature map;

[0086] Step S5: Input the low-light image from step S1 into the illumination component estimation module to obtain the illumination component of the low-light image. Input the illumination component of the low-light image and the final multi-scale feature map from step S4 into the adaptive fusion module for adaptive fusion. After obtaining the enhanced image, input it into the target detection model.

[0087] Furthermore, in step S1, the normal lighting image is input into the dark processing module to obtain the low-light image; the specific steps are as follows:

[0088] Step S11: Apply an inverse gamma transform to each pixel of the input normal illumination image;

[0089] Step S12: Using a randomly selected camera color correction matrix, each pixel processed in step S11 is converted from the standard red-green-blue color space to the camera's native color space to obtain each pixel after color space conversion.

[0090] Step S13: For each pixel after color space conversion in step S12, a brightness scaling factor is randomly sampled from the light attenuation coefficient distribution. The brightness scaling factor of each pixel after color space conversion in step S12 is then used for overall intensity scaling to obtain a preliminary low-light image.

[0091] Step S14: Random noise is added to the low-light image initially generated in step S13 to obtain the low-light image.

[0092] Further, in step S2, the low-light image obtained in step S1 is input into the spatial domain non-lighting feature extraction module to obtain spatial domain non-lighting features; the specific steps are as follows:

[0093] Step S201: The spatial domain non-illumination feature extraction module applies a logarithmic transformation to each of the three channels of the input low-illumination image; low-illumination image Where R, G, and B represent the red channel, green channel, and blue channel, respectively;

[0094] Step S202: Use shallow convolutional kernels to perform preliminary feature extraction on the red channel R, green channel G, and blue channel B to obtain three sets of basic mapping features, as shown in the formula:

[0095] ;

[0096] in, This represents the basic mapping feature of the first channel c in pixel coordinates. The first channel c belongs to one of the three channels: red channel R, green channel G, and blue channel B. `log` represents the logarithmic transformation. This represents the shallow convolution kernel acting on the first channel c. This represents the pixel value at pixel coordinate c in the first channel of the low-light image;

[0097] Step S203: Using three sets of basic mapping features, the three channels (red channel R, green channel G, and blue channel B) are paired in the spatial domain, and the non-illumination features between channels are obtained by calculating the channel element-level difference using cross-color ratio, as shown in the formula:

[0098] ;

[0099] Where c1 is the second channel and c2 is the third channel, the second channel c1 and the third channel c2 are also one of the three channels: red channel R, green channel G, and blue channel B. ; This indicates that the non-illumination characteristics between the second channel c1 and the third channel c2 are obtained by cross-color ratio calculation; This represents the basic mapping feature of the second channel c1 in pixel coordinates; This represents the basic mapping feature of the third channel c2 in pixel coordinates;

[0100] Step S204: The non-illuminated features of the three channels (red channel R, green channel G, and blue channel B) are concatenated along the channel direction to obtain the spatial domain non-illuminated features, as shown in the formula:

[0101] ;

[0102] in, Represents the non-illuminated features of the spatial domain. This indicates a splicing operation. This represents the non-illuminated features between the red and green channels. This represents the non-illumination characteristics between the red and blue channels. Indicates the non-illuminating characteristics between the green channel and the blue channel;

[0103] Similarly, the non-illuminated features in the spatial domain are obtained by inputting the normally illuminated image into the spatial domain non-illuminated feature extraction module based on steps S201-S204.

[0104] Furthermore, the low-light image obtained in step S1 is input into the frequency domain non-lighting feature extraction module to obtain frequency domain non-lighting features; the specific steps are as follows:

[0105] Step S211: The frequency domain non-illumination feature extraction module applies a two-dimensional Fourier transform to each of the three channels of the low-illumination image to obtain the complex spectral features of the corresponding channels. ,in, The real part represents the characteristic of the spectrum of a complex number. Represents the imaginary unit. The imaginary part represents the spectral characteristics of a complex number;

[0106] Step S212: Design a complex convolution kernel, using a pair of weights applied to the real and imaginary parts of the input features, as shown in the formula:

[0107] ;

[0108] in, This represents the frequency domain features extracted from the first channel c through complex convolution. The weights representing the complex spectral characteristics of the first channel c are... The weights representing the real part of the complex spectral characteristics of the first channel c are... The weights represent the imaginary part of the complex spectral characteristics of the first channel c;

[0109] Step S213 involves extracting the frequency domain features from the first channel c in the frequency domain through complex convolution. Perform inverse Fourier transform back to the spatial domain spatial domain include , , In the spatial domain The cross-color ratio feature is obtained by performing cross-color ratio analysis, as shown in the formula:

[0110] ;

[0111] in, This represents the cross-color ratio feature between the red and green color channels extracted in the frequency domain. This represents the features of the red channel extracted by complex convolution, which are then transformed back to the spatial domain by inverse Fourier transform. This represents the features of the green channel extracted through complex convolution, which are then transformed back to the spatial domain by inverse Fourier transform.

[0112] Similarly, we can obtain:

[0113] ;

[0114] ;

[0115] in, This represents the features of the blue channel extracted through complex convolution, which are then transformed back to the spatial domain using inverse Fourier transform. This represents the cross-color ratio feature between the red and blue channels extracted in the frequency domain. This represents the cross-color ratio feature between the green and blue channels extracted in the frequency domain;

[0116] Step S214: Concatenate the cross-color ratio features to obtain the frequency domain non-illumination features, as shown in the formula:

[0117] ;

[0118] in, Represents non-illuminated features in the frequency domain;

[0119] Similarly, the normal illumination image is input into the frequency domain non-illumination feature extraction module to obtain the frequency domain non-illumination features based on steps S211-S214.

[0120] Based on the classic Lambertian hypothesis, the observed value of any pixel can be expressed as the product of the illumination component and the non-illumination feature. The non-illumination feature is only related to the object's surface material, inherent color, texture, and other attributes, and does not change with illumination. This invention utilizes this physical law, combined with cross-color ratios, and achieves accurate extraction of spatial domain non-illumination features through multi-channel logarithmic transformation and difference calculation. The difference operation effectively counteracts local illumination components, ensuring that the output feature only reflects the relative changes in non-illumination features between different materials on the object's surface, achieving illumination-invariant feature representation.

[0121] To enrich non-illumination features and enhance the ability to extract non-illumination features from objects of the same color family, this invention introduces a frequency domain modeling strategy and proposes a frequency-domain non-illumination module (FNIM) based on complex convolution.

[0122] Furthermore, in step S3, the spatial domain non-illuminated features and frequency domain non-illuminated features obtained in step S2 are input into the adaptive fusion module for fusion to obtain a preliminary multi-scale feature map; the specific steps are as follows:

[0123] Step S31, spatial domain non-illuminated features Frequency domain non-illuminated features The concatenated feature map is obtained by concatenating along the channel dimension. ;

[0124] Step S32: A 3×3 convolutional layer is used to concatenate the feature maps. Compression is performed, and two dynamic weight graphs are generated by activating a normalized exponential function, as shown in the formula:

[0125] ;

[0126] in, A dynamic weighted graph representing the metric space domain. Represents a dynamic weighted graph in the frequency domain of the metric. Represents the normalized exponential function, () indicates a convolution operation to perform a 3×3 convolutional layer;

[0127] Step S33, based on spatial domain non-illuminated features Frequency domain non-illuminated features And two dynamic weight maps yield preliminary multi-scale feature maps; as shown in the formula:

[0128] ;

[0129] in, This represents a preliminary multi-scale feature map.

[0130] Furthermore, in step S4, the preliminary multi-scale feature map obtained in step S3 is used to obtain the final multi-scale feature map using a channel attention module; the specific steps are as follows:

[0131] Step S41, process the preliminary multi-scale feature map Apply global average pooling, as shown in the formula:

[0132] ;

[0133] in, The result represents the global average pooling result, where H represents the height of the multi-scale feature map, W represents the width of the multi-scale feature map, m represents the index of the multi-scale feature map in height, and n represents the index of the multi-scale feature map in width.

[0134] Step S42: Construct a nonlinear channel relationship model using two fully connected layers, as shown in the formula:

[0135] ;

[0136] Where s represents the attention vector, This represents the Sigmoid function. Represents the ReLU function. , These represent the weight matrices used by the channel attention module for dimensionality reduction and dimensionality increase, respectively.

[0137] Step S43: Re-weight and fuse the initial multi-scale feature maps using the attention vector s. Each channel is as shown in the formula:

[0138] ;

[0139] in, This represents the final multi-scale feature map;

[0140] Step S44: Set a consistency loss for the final multi-scale feature map. The final multi-scale feature map is divided into the final multi-scale feature map of the low-light image and the final multi-scale feature map of the normal-light image.

[0141] Among them, consistency loss , This represents the final multi-scale feature map of a normally illuminated image. This represents the final multi-scale feature map of a low-light image.

[0142] Furthermore, in step S5, the low-light image from step S1 is input into the illumination component estimation module to obtain the illumination component of the low-light image. The illumination component of the low-light image is then input into the adaptive fusion module along with the final multi-scale feature map from step S4 for adaptive fusion. The resulting enhanced image is then input into the target detection model. The specific steps are as follows:

[0143] Step S51: The input low-light image is smoothed by Gaussian filtering, and the illumination component of the low-light image is obtained by multi-channel gradient calculation.

[0144] Step S52: The illumination component of the low-light image and the final multi-scale feature map of the low-light image are adaptively fused in the adaptive fusion module to obtain the enhanced image. The enhanced image is then put into the target detection model for target detection.

[0145] Furthermore, a zero-sample frequency-domain and spatial-domain dual-branch low-light target detection system is provided, applied to the zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method. The detection system includes:

[0146] The dark processing module is used to obtain a low-light image from the input normal lighting image;

[0147] The spatial domain non-illuminated feature extraction module is used to extract spatial domain non-illuminated features from illuminated images;

[0148] The frequency domain non-illuminated feature extraction module is used to extract the frequency domain non-illuminated features of illuminated images;

[0149] The adaptive fusion module is used to fuse the non-illuminated features extracted by the spatial domain non-illuminated feature extraction module and the frequency domain non-illuminated feature extraction module to obtain a preliminary multi-scale feature map.

[0150] The channel attention module is used to obtain the final multi-scale feature map from the initial multi-scale feature map through the channel attention mechanism.

[0151] The illumination component estimation module is used to extract the illumination components of low-light images.

[0152] The illumination component estimation module recovers illumination information from the image that is helpful for downstream detection. This module is based on the gray-level guide principle and uses the Minkowski norm and higher-order statistical information of the image to estimate the local dominant illumination direction and intensity.

[0153] Among them, the pair of weights in step 212 is , . The weights representing the real part of the spectral characteristics of a complex number are... The weights represent the imaginary part of the spectral characteristics of complex numbers.

[0154] Table 1 shows a comparison of the average accuracy mAP50 of Yolov8n, FeatEnHancer, PE-YOLO, GLARE, LLIEDiff, HVI, Yola with the present invention (a zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method).

[0155] Table 1. Comparison of average accuracy between the present invention and other methods

[0156]

[0157] like Figure 2 As shown, a normal illumination image is input into a dark processing module to obtain a low illumination image. The normal illumination image is then input into a spatial domain non-illumination feature extraction module to obtain spatial domain non-illumination features, and into a frequency domain non-illumination feature extraction module to obtain frequency domain non-illumination features. These are then combined into an adaptive fusion module to obtain a preliminary multi-scale feature map. The low illumination image is also input into a spatial domain non-illumination feature extraction module to obtain spatial domain non-illumination features, and into a frequency domain non-illumination feature extraction module to obtain frequency domain non-illumination features. These are then combined into an adaptive fusion module to obtain a preliminary multi-scale feature map. The spatial domain non-illumination feature extraction module and the frequency domain non-illumination feature extraction module are parallel structures, forming a dual-branch structure. The preliminary multi-scale feature map is then processed by a channel attention module to obtain a final multi-scale feature map. A consistency loss is applied between the final multi-scale feature maps. The low illumination image is processed by an illumination estimation module to obtain illumination components, which are then combined with the final multi-scale feature map and input into an adaptive fusion module to obtain an enhanced image. This enhanced image is then fed into a target detection model for target detection.

[0158] like Figure 3 As shown, the input illumination image is separated into three color channels: red (R), green (G), and blue (B). Then, Fourier transform is performed on each channel to convert it to the frequency domain. Complex convolution is used to extract the features of each channel. Finally, inverse Fourier transform is performed to return it to the spatial domain. Cross-color ratio is used to extract the non-illumination features in the frequency domain.

[0159] like Figure 4 As shown, the input illumination image is separated into three color channels: red (R), green (G), and blue (B). Complex convolution is used to extract features from each channel. In the figure, w represents the weight of the shallow convolution kernel. Then, the spatial domain non-illumination features are extracted using a cross-color ratio method.

Claims

1. A zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method, characterized in that, The specific steps of the target detection method are as follows: Step S1: Input a normal lighting image into the dark processing module to obtain a low-light image; Step S2: Input the low-light image obtained in step S1 into the spatial domain non-light feature extraction module to obtain spatial domain non-light features and input it into the frequency domain non-light feature extraction module to obtain frequency domain non-light features; at the same time, input the normal lighting image into the spatial domain non-light feature extraction module to obtain spatial domain non-light features and input it into the frequency domain non-light feature extraction module to obtain frequency domain non-light features. The specific steps are as follows: Step S201: The spatial domain non-illuminated feature extraction module applies a logarithmic transformation to each of the three channels of the input low-illuminated image. Step S202: Use shallow convolution kernels to perform preliminary feature extraction on the red channel R, green channel G, and blue channel B to obtain three sets of basic mapping features; Step S203: Using three sets of basic mapping features, the three channels of red channel R, green channel G, and blue channel B are combined in pairs in the spatial domain, and the non-illumination features between channels are obtained by calculating the channel element-level difference using cross-color ratio. Step S204: The non-illuminated features of the three channels (red channel R, green channel G, and blue channel B) are spliced ​​together according to the channel direction to obtain the spatial domain non-illuminated features. Similarly, the spatial domain non-illumination features of the normally illuminated image are obtained based on steps S201-S204. Step S211: The frequency domain non-illumination feature extraction module applies a two-dimensional Fourier transform to the three channels of the low-illumination image to obtain the complex spectral features of the corresponding channels. Step S212: Design a complex convolution kernel and apply a pair of weights to the real and imaginary parts of the input features respectively; Step S213 involves extracting the frequency domain features from the first channel c in the frequency domain through complex convolution. Perform inverse Fourier transform back to the spatial domain In the spatial domain Cross-color ratio features are obtained by performing cross-color ratio analysis. Step S214: Concatenate the cross-color ratio features to obtain the frequency domain non-illuminated features; Similarly, based on steps S211-S214, the frequency domain non-illumination features of the normally illuminated image are obtained; Step S3: Based on the spatial domain non-illuminated features and frequency domain non-illuminated features obtained in step S2, input them into the adaptive fusion module for fusion to obtain a preliminary multi-scale feature map; Step S4: Based on the preliminary multi-scale feature map obtained in step S3, the channel attention module is used to obtain the final multi-scale feature map; Step S5: Input the low-light image from step S1 into the illumination component estimation module to obtain the illumination component of the low-light image. Input the illumination component of the low-light image and the final multi-scale feature map from step S4 into the adaptive fusion module for adaptive fusion. After obtaining the enhanced image, input it into the target detection model.

2. The zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method according to claim 1, characterized in that, In step S1, a normal lighting image is input into the dark processing module to obtain a low-light image; the specific steps are as follows: Step S11: Apply an inverse gamma transform to each pixel of the input normal illumination image; Step S12: Using a randomly selected camera color correction matrix, each pixel processed in step S11 is converted from the standard red-green-blue color space to the camera's native color space to obtain each pixel after color space conversion. Step S13: For each pixel after color space conversion in step S12, a brightness scaling factor is randomly sampled from the light attenuation coefficient distribution. The brightness scaling factor of each pixel after color space conversion in step S12 is then used for overall intensity scaling to obtain a preliminary low-light image. Step S14: Random noise is added to the low-light image initially generated in step S13 to obtain the low-light image.

3. The zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method according to claim 2, characterized in that, Step S2: Input the low-light image obtained in step S1 into the spatial domain non-light feature extraction module to obtain spatial domain non-light features; The specific steps are as follows: Step S201, the low-light image Where R, G, and B represent the red channel, green channel, and blue channel, respectively; Step S202, the three sets of basic mapping features are as shown in the formula: ； in, This represents the basic mapping feature of the first channel c in pixel coordinates. The first channel c belongs to one of the three channels: red channel R, green channel G, and blue channel B. `log` represents the logarithmic transformation. This represents the shallow convolution kernel acting on the first channel c. This represents the pixel value at pixel coordinate c in the first channel of the low-light image; Step S203, the non-illuminated features between the channels are as shown in the formula: ； Where c1 is the second channel and c2 is the third channel, the second channel c1 and the third channel c2 are also one of the three channels: red channel R, green channel G, and blue channel B. ; This indicates that the non-illumination characteristics between the second channel c1 and the third channel c2 are obtained by cross-color ratio calculation; This represents the basic mapping feature of the second channel c1 in pixel coordinates; This represents the basic mapping feature of the third channel c2 in pixel coordinates; Step S204, the non-illuminated features of the spatial domain are as shown in the formula: ； in, Represents the non-illuminated features of the spatial domain. This indicates a splicing operation. This represents the non-illuminated features between the red and green channels. This represents the non-illumination characteristics between the red and blue channels. Indicates the non-illuminating characteristics between the green channel and the blue channel; Similarly, the spatial domain non-illuminated features of the normally illuminated image are obtained based on steps S201-S204.

4. The zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method according to claim 3, characterized in that, The low-light image obtained in step S1 is input into the frequency domain non-light feature extraction module to obtain the frequency domain non-light features; The specific steps are as follows: Step S211, the complex spectral features ,in, The real part represents the characteristic of the spectrum of a complex number. Represents the imaginary unit. The imaginary part represents the spectral characteristics of a complex number; Step S212: Design a complex convolution kernel, using a pair of weights applied to the real and imaginary parts of the input features, as shown in the formula: ； in, This represents the frequency domain features extracted from the first channel c through complex convolution. The weights representing the complex spectral characteristics of the first channel c are... The weights representing the real part of the complex spectral characteristics of the first channel c are... The weights represent the imaginary part of the complex spectral characteristics of the first channel c; Step S213, the spatial domain include , , The cross-color ratio feature is shown in the formula: ； in, This represents the cross-color ratio feature between the red and green color channels extracted in the frequency domain. This represents the features of the red channel extracted by complex convolution, which are then transformed back to the spatial domain by inverse Fourier transform. This represents the features of the green channel extracted through complex convolution, which are then transformed back to the spatial domain by inverse Fourier transform. Similarly, we can obtain: ； ； in, This represents the features of the blue channel extracted through complex convolution, which are then transformed back to the spatial domain using inverse Fourier transform. This represents the cross-color ratio feature between the red and blue channels extracted in the frequency domain. This represents the cross-color ratio feature between the green and blue channels extracted in the frequency domain; Step S214: Concatenate the cross-color ratio features to obtain the frequency domain non-illumination features, as shown in the formula: ； in, Represents non-illuminated features in the frequency domain; Similarly, based on steps S211-S214, the frequency domain non-illumination features of the normally illuminated image are obtained.

5. The zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method according to claim 4, characterized in that, In step S3, the spatial domain non-illuminated features and frequency domain non-illuminated features obtained in step S2 are input into the adaptive fusion module for fusion to obtain a preliminary multi-scale feature map; the specific steps are as follows: Step S31, spatial domain non-illuminated features Frequency domain non-illuminated features The concatenated feature map is obtained by concatenating along the channel dimension. ; Step S32: A 3×3 convolutional layer is used to concatenate the feature maps. Compression is performed, and two dynamic weight graphs are generated by activating a normalized exponential function, as shown in the formula: ； in, A dynamic weighted graph representing the metric space domain. Represents a dynamic weighted graph in the frequency domain of the metric. Represents the normalized exponential function, () indicates a convolution operation to perform a 3×3 convolutional layer; Step S33, based on spatial domain non-illuminated features Frequency domain non-illuminated features And two dynamic weight maps yield preliminary multi-scale feature maps; as shown in the formula: ； in, This represents a preliminary multi-scale feature map.

6. The zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method according to claim 5, characterized in that, In step S4, the preliminary multi-scale feature map obtained in step S3 is used to obtain the final multi-scale feature map using a channel attention module; the specific steps are as follows: Step S41, process the preliminary multi-scale feature map Apply global average pooling, as shown in the formula: ； in, The result represents the global average pooling result, where H represents the height of the multi-scale feature map, W represents the width of the multi-scale feature map, m represents the index of the multi-scale feature map in height, and n represents the index of the multi-scale feature map in width. Step S42: Construct a nonlinear channel relationship model using two fully connected layers, as shown in the formula: ； Where s represents the attention vector, This represents the Sigmoid function. Represents the ReLU function. , These represent the weight matrices used by the channel attention module for dimensionality reduction and dimensionality increase, respectively. Step S43: Re-weight and fuse the initial multi-scale feature maps using the attention vector s. Each channel is as shown in the formula: ； in, This represents the final multi-scale feature map; Step S44: Set a consistency loss for the final multi-scale feature map. The final multi-scale feature map is divided into the final multi-scale feature map of the low-light image and the final multi-scale feature map of the normal-light image.

7. The zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method according to claim 6, characterized in that, In step S5, the low-light image from step S1 is input into the illumination component estimation module to obtain the illumination components of the low-light image. The illumination components of the low-light image are then input into the adaptive fusion module along with the final multi-scale feature map from step S4 for adaptive fusion. The resulting enhanced image is then input into the target detection model. The specific steps are as follows: Step S51: The input low-light image is smoothed by Gaussian filtering, and the illumination component of the low-light image is obtained by multi-channel gradient calculation. Step S52: Adaptively fuse the illumination component of the low-light image with the multi-scale feature map of the low-light image in the adaptive fusion module to obtain the enhanced image. Then, put the enhanced image into the target detection model for target detection.

8. A zero-sample frequency-domain and spatial-domain dual-branch low-light target detection system, applied to the zero-sample frequency-domain and spatial-domain dual-branch low-light target detection method of claim 7, characterized in that, The detection system includes: The dark processing module is used to obtain a low-light image from the input normal lighting image; The spatial domain non-illuminated feature extraction module is used to extract spatial domain non-illuminated features from illuminated images; The frequency domain non-illuminated feature extraction module is used to extract the frequency domain non-illuminated features of illuminated images; The adaptive fusion module is used to fuse the non-illuminated features extracted by the spatial domain non-illuminated feature extraction module and the frequency domain non-illuminated feature extraction module to obtain a preliminary multi-scale feature map. The channel attention module is used to obtain the final multi-scale feature map from the initial multi-scale feature map through the channel attention mechanism. The illumination component estimation module is used to extract the illumination components of low-light images.

Images