Adaptive enhanced target detection method in weak light environment based on zero-shot learning

Abstract

The invention discloses a method for adaptively enhancing target detection in a weak-light environment based on zero-sample learning, and relates to the field of target detection. First, a data set including an original image and its corresponding dark-light image is obtained. Then, a DAI-Net model including a first channel, a second channel and a synthesis channel is constructed, wherein the first channel extracts and decodes low-level features, the second channel uses a pre-trained Retinex decomposition network to obtain pseudo-true values ​​of reflectivity and illumination information, and the synthesis channel uses the information to reconstruct a new image and decode it again. Then, a variety of loss functions are used to train the model to obtain a dark-light adaptive target enhancement detection model. Finally, the model is used to detect the target to be detected in the input image. The method significantly improves the target detection performance in a weak-light environment.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and in particular to an adaptive enhanced target detection method in a weak light environment based on zero-sample learning. Background Art

[0002] As a core research direction in the field of computer vision, the main challenge of object detection is to accurately identify and locate various objects in images. With the development of technology, the emergence of large-scale datasets has greatly promoted the development of this field. For example, general datasets such as COCO and Open Images, as well as the WIDER FACE dataset specifically for face recognition, provide rich training samples for algorithms. These datasets contain a large amount of labeled information, enabling advanced detectors to learn multiple features of targets in different environments, thereby achieving significant improvements in precision and positioning accuracy.

[0003] At the same time, Retinex theory plays an indispensable role in low-light vision tasks. The theory provides a unique image decomposition model that divides the image into two parts: reflectance and illumination, and synthesizes the original image through a multiplication relationship. The reflectance part reflects the essential properties of the object and does not change with changes in lighting conditions; while the illumination describes the lighting conditions of the scene, and its changes will directly affect the brightness and color of the image. This decomposition method not only reveals the basic principles of the human visual system's perception of color and brightness, but also makes it possible to maintain the consistency of the visual features of objects under different lighting conditions. Therefore, when performing image enhancement in low-light environments, the method based on Retinex theory can effectively improve image visibility and reduce noise and color distortion caused by low light by processing the illumination component to highlight the reflectance information.

[0004] However, even with these advances, object detection still faces many challenges in low-light scenes. Factors such as dark environments, insufficient lighting, or improper exposure time can lead to a decline in image quality, manifested as low visibility, color distortion, and high noise levels. These problems seriously affect the performance of object detectors, making it difficult to work accurately in low-light environments. To solve these problems, researchers have tried a variety of methods, such as using image enhancement techniques based on Retinex theory, or adjusting existing detectors to adapt to low-light conditions. However, these methods have their own limitations. For example, image enhancement technology requires a large amount of low-light image data support, and data collection and annotation in this area are relatively difficult, limited in quantity, and cover narrow scenes. Therefore, finding a more effective solution is still a current research hotspot. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an adaptive enhanced target detection method in a weak light environment based on zero-sample learning, comprising: Acquire a data set, wherein the data set includes a plurality of training samples, each of which includes: an original image and a corresponding low-light image; the low-light image is a synthetic image generated from the original image through a physically inspired low-light synthesis pipeline; each original image and low-light image is provided with a training label; the training label includes position information and category information of the target to be detected; A DAI-Net model including a first channel, a second channel and a synthesis channel is constructed based on the target detector; wherein: the first channel is used to extract low-level features corresponding to the original image and the dark-light image, and the low-level features are decoded to obtain the reflectivity corresponding to each image as the first decoded reflectivity; The second channel includes a pre-trained Retinex decomposition network, which is used to decompose the original image and the dark-light image to obtain pseudo-true values ​​of reflectivity and illumination information corresponding to the original image and the dark-light image respectively; The synthesis channel is used to reconstruct the first image and the second image using the pseudo-true value of the illumination information decomposed by the second channel and the first decoded reflectivity decoded by the first channel; the first channel is also used to decode the first image and the second image to obtain the second decoded reflectivity; Based on each channel of the DAI-Net model, multiple loss functions are constructed, and the DAI-Net model is trained by multiple loss functions and data sets to obtain a dark-light adaptive target enhancement detection model; the loss function includes a decomposition consistency loss function and a mutual feature alignment loss function; the mutual feature alignment loss function is used to align the original image extracted by the first channel with the low-level features of the dark-light image; the decomposition consistency loss function is used to constrain the consistency of the decoded reflectivity obtained by twice decoding the first channel; The input image is processed through the dark-light adaptive target enhancement detection model to detect and output the location information and category information of the target to be detected.

[0006] Furthermore, the first channel includes a first segmentation channel and a second segmentation channel, and the construction method thereof is: segmenting the backbone network of the target detector based on the Retinex theory to obtain the first segmentation channel and the second segmentation channel; The first segmentation channel is used to extract low-level features corresponding to the original image and the dark-light image, and decode the low-level features to obtain the reflectivity corresponding to each image as the first decoding reflectivity; and is also used to decode the first image and the second image to obtain the second decoding reflectivity; The second segmentation channel is used to detect the position information and category information of the target to be detected through the low-level features extracted by the first segmentation channel and the secondary decoding reflectivity obtained by decoding.

[0007] Furthermore, the backbone network of the target detector is segmented based on the Retinex theory to obtain the first segmentation channel and the second segmentation channel, specifically: Segmentation is performed at the second convolutional layer of the backbone network: the part before the second convolutional layer is set as the front-end part, and a reflectivity decoder is set at the output end of the front-end part, thereby obtaining a first segmentation channel formed by the front-end part and the reflectivity decoder; the entire backbone network is set as the second segmentation channel; the reflectivity decoder is used to decode low-level features.

[0008] Furthermore, the synthesis channel is used to reconstruct the first image and the second image using the pseudo-true value of the illumination information obtained by decomposing the second channel and the first decoded reflectivity obtained by decoding the first channel, specifically: The true value of the illumination information of the original image obtained by decomposing the second channel The true value of illumination information of dark light image , and the first decoded reflectivity of the original image obtained by decoding the first channel First decoded reflectance with dark light image Forming the first information pair With the second information ;in, represents a dark light image, represents the original image; Exchange the first information pair With the second information The first decoded reflectivity in the image is obtained to obtain the first reconstructed information pair With the second reconstruction information ; By first reconstructing the information Reconstruct the original image to obtain the first image. The reconstruction formula is: ; In the formula, represents the decoding process of the first channel for the original image and the dark light image, 2 represents the reconstruction process, represents the first reconstructed image; Through the second reconstruction information Reconstruct the dark light image to obtain the second image. The reconstruction formula is: ; In the formula, represents the reconstructed second image.

[0009] Furthermore, the loss function also includes: Detection loss function, which is used to evaluate the performance of the object detector in detecting the object to be detected; Reflectivity learning loss function, used to optimize the output quality of the reflectivity decoder; Image decomposition loss function, used to measure the accuracy of the pseudo-true value of the reflectivity and illumination information decomposed by the Retinex decomposition network; The total loss function of the DAI-Net model is used to combine the constructed loss functions to achieve comprehensive optimization of the DAI-Net model in target detection, reflectivity decoding and image decomposition.

[0010] Furthermore, the formula expression of the decomposition consistency loss function is: ; In the formula, represents the secondary decoding reflectivity obtained by decoding the first image by the first channel, represents the secondary decoding reflectivity obtained by decoding the second image by the first channel, express and The L1 norm between express and The L1 norm between Represents the loss value of the decomposition consistency loss function; The formula expression of the mutual feature alignment loss function is: ; In the formula, express Divergence, and They respectively represent the low-level features of the original image and the dark-light image after being preprocessed by the front-end part, wherein the preprocessing includes flattening and spatial averaging. Represents the loss value of the mutual feature alignment loss function.

[0011] Furthermore, the formula expression of the image decomposition loss function is: ; In the formula, represents the reconstruction loss, which is used to evaluate the difference between the reconstructed first image and the original image, and the difference between the reconstructed second image and the dark light image; represents the illumination smoothness loss, which is used to make the reconstructed first image and the second image have a smooth transition; Represents the lighting smoothness weight; ,in, represents the reflectivity consistency loss, and They respectively represent the first decoded reflectivity corresponding to the original image and the dark-light image output by the reflectivity decoder, or the second decoded reflectivity corresponding to the first image and the second image output by the reflectivity decoder; is equal to value 1 or value 2; Represents the reflectivity consistency weight.

[0012] Furthermore, the formula expression of the reflectivity learning loss function is: ; In the formula, Represents the pseudo-true value of the reflectivity decomposed by the Retinex decomposition network, Indicates the first decoding reflectivity or the second decoding reflectivity output by the reflectivity decoder. represents the structural similarity index, express and The mean absolute error between Represents the loss value of the reflectivity learning loss function; The total loss function of the DAI-Net model is expressed as: ; In the formula, Represents the loss value of the detection loss function, represents the weight of the mutual feature alignment loss function in the total loss function, represents the weight of the decomposition consistency loss function in the total loss function, Represents the total loss.

[0013] Furthermore, before training the DAI-Net model, the process also includes: freezing the pre-trained weights in the pre-trained Retinex decomposition network.

[0014] Furthermore, the target detector is a YOLO model; the reflectivity decoder includes: a first convolutional layer, a first ReLU activation function, a second convolutional layer and a second ReLU activation function connected in sequence.

[0015] The beneficial effects of the embodiments of the present invention include: (1) The present invention first obtains a data set including an original image and its corresponding low-light image (generated by a low-light synthesis pipeline); then, a DAI-Net model including a first channel, a second channel and a synthesis channel is constructed, wherein the first channel extracts and decodes low-level features, the second channel uses a pre-trained Retinex decomposition network to obtain reflectivity and illumination information pseudo-true values, and the synthesis channel uses this information to reconstruct a new image and decode it again; then, the model is trained using multiple loss functions to obtain a low-light adaptive target enhancement detection model; finally, the model is used to detect the target to be detected in the input image, and the method significantly improves the target detection performance in a low-light environment; by combining Retinex theory and multi-channel design, the model can extract illumination invariant features (reflectivity) from the original image and the low-light image, and maintain stable detection accuracy under different illumination conditions; at the same time, a method of joint training with multiple loss functions is adopted to further enhance the robustness and generalization ability of the model, so that it can still work effectively when there is a lack of labeled data under specific low-light conditions; in addition, this method reduces the dependence on large-scale labeled data, improves image quality, and reduces noise interference, thereby showing higher reliability and wide applicability in practical applications; (2) The method proposed in the present invention has the ability of zero-sample learning, so that the model can still be trained with a dataset of images with good lighting even when there is a lack of labeled data under specific low-light conditions; through a physically inspired low-light synthesis pipeline, the corresponding low-light image is synthesized from the original image (i.e., the image with good lighting), thereby eliminating the process of acquiring the real low-light image; this method not only greatly reduces the dependence on a large amount of labeled data, but also significantly improves the versatility and flexibility of the model, especially in low-light environments where it is difficult to obtain high-quality labeled data. This synthesis method reduces the need for large-scale labeled data, allowing the model to be deployed in more diverse application scenarios without the need to collect and label a large amount of data specifically for each lighting condition. Therefore, the present invention improves the robustness and generalization ability of the model through innovative data synthesis and processing techniques, so that it can work effectively under different lighting conditions; (3) In the present invention, the first channel extracts and decodes the reflectivity (including the first decoded reflectivity and the second decoded reflectivity) that is not affected by illumination changes based on the Retinex theory, ensuring that the essential features of the object remain stable; the pre-trained Retinex decomposition network in the second channel decomposes the input image into reflectivity and illumination information pseudo-true values, and the synthesis channel uses this information to reconstruct a new image; since the reflectivity is illumination invariant, the model can maintain consistent target features even under different illumination conditions, thereby improving the accuracy and robustness of target detection; this separation and recombination mechanism reduces the dependence on large-scale annotated data and significantly improves the robustness and generalization ability of the model; therefore, based on the stability of the reflectivity, the model achieves effective adaptation to different illumination conditions; (4) The present invention designs a variety of loss functions (such as decomposition consistency loss, mutual feature alignment loss, reflectivity learning loss, image decomposition loss, etc.), and jointly trains the DAI-Net model through these loss functions; this approach not only improves the performance of the model in target detection tasks, but also enhances its robustness to illumination changes, ensuring consistency and stability under different illumination conditions; (5) The reflectance pseudo-true value generated by the Retinex decomposition network in the present invention can effectively remove the impact of illumination changes, improve the overall image quality and reduce noise, which is crucial for subsequent target detection tasks because high-quality input images help improve the accuracy and reliability of detection results.

[0016] The details of one or more embodiments of the invention are set forth in the following drawings and description so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0018] Figure 1 This is a flow chart of a method for adaptively enhancing target detection in a weak light environment based on zero-sample learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] Embodiments of the present embodiment will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present embodiment are shown in the accompanying drawings, it should be understood that the present embodiment can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein, which are instead provided for a more thorough and complete understanding of the present embodiment. It should be understood that the drawings and embodiments of the present embodiment are only for exemplary purposes and are not intended to limit the scope of protection of the present embodiment.

[0020] In order to enhance target detection in low-light environments, such as Figure 1 As shown, the embodiment of the present invention proposes an adaptive enhanced target detection method in a weak light environment based on zero-shot learning. It should be explained that zero-shot learning means that the model can recognize new categories or adapt to new environmental conditions without direct training data.

[0021] The adaptive enhanced target detection method comprises: Acquire a data set, wherein the data set includes a plurality of training samples, each of which includes: an original image (i.e., an image with good lighting) and a corresponding dark-light image; the dark-light image is a synthetic image generated from the original image through a physically inspired low-light synthesis pipeline; each original image and the dark-light image is provided with a training label; the training label includes position information and category information of the target to be detected; This embodiment generates low-light images from original images through a physically inspired low-light synthesis pipeline. This method can generate a large amount of synthetic data and enrich the dataset. Although these synthetic data are not low-light images in real scenes, they provide diverse lighting conditions and can help the model generalize better.

[0022] Specifically, the method proposed in the present invention has the ability of zero-sample learning, so that the model can still be trained with a well-lit image data set even when there is a lack of labeled data under specific low-light conditions; through a physically inspired low-light synthesis pipeline, the corresponding low-light image is synthesized from the original image (i.e., the image with good lighting), thereby eliminating the process of acquiring the real low-light image; this method not only greatly reduces the dependence on a large amount of labeled data, but also significantly improves the versatility and flexibility of the model, especially in the case of difficulty in obtaining high-quality labeled data in low-light environments. This synthesis method reduces the demand for large-scale labeled data, allowing the model to be deployed in more diverse application scenarios without the need to collect and label a large amount of data specifically for each lighting condition. Therefore, the present invention improves the robustness and generalization ability of the model through innovative data synthesis and processing technology, so that it can work effectively under different lighting conditions.

[0023] Based on the target detector, a DAI-Net model including the first channel, the second channel and the synthesis channel is constructed;

[0024] In this embodiment, the target detector is a YOLO model, but is not limited to the YOLO model.

[0025] The first channel is used to extract low-level features corresponding to the original image and the dark-light image, and decode the low-level features to obtain the reflectivity corresponding to each image as the first decoded reflectivity;

[0026] The first channel includes a first segmentation channel and a second segmentation channel, and the construction method thereof is: based on the Retinex theory, the backbone network of the target detector is segmented to obtain the first segmentation channel and the second segmentation channel;

[0027] The backbone network of the target detector is segmented based on the Retinex theory to obtain the first segmentation channel and the second segmentation channel, specifically: The segmentation is performed at the second convolution layer of the backbone network: the part before the second convolution layer is set as the front end part, and a reflectivity decoder is set at the output end of the front end part, thereby obtaining a first segmentation channel formed by the front end part and the reflectivity decoder; the entire backbone network is set as the second segmentation channel; the reflectivity decoder is used to decode the low-level features. The reflectivity decoder includes: a first convolution layer, a first ReLU activation function, a second convolution layer and a second ReLU activation function connected in sequence.

[0028] The first segmentation channel is used to extract low-level features corresponding to the original image and the dark-light image, and decode the low-level features to obtain the reflectivity corresponding to each image as the first decoding reflectivity; and is also used to decode the first image and the second image to obtain the second decoding reflectivity;

[0029] The second segmentation channel is used to detect the position information and category information of the target to be inspected through the low-level features extracted by the first segmentation channel and the secondary decoded reflectivity obtained by decoding (the detection head of the second segmentation channel detects the position information and category information of the target to be inspected based on the low-level features and the secondary decoded reflectivity corresponding to the second image).

[0030] It should be explained that reflectivity reflects the essential properties of an object and is not affected by changes in illumination. The present invention segments the backbone network of the target detector based on the Retinex theory and decodes the low-level features through the reflectivity decoder. The model can focus on extracting stable reflectivity without requiring a large amount of labeled data under different illumination conditions.

[0031] The second channel includes a pre-trained Retinex decomposition network, which is used to decompose the original image and the dark-light image to obtain pseudo-true values ​​of reflectivity and illumination information corresponding to the original image and the dark-light image respectively;

[0032] The reflectance pseudo-true value generated by the Retinex decomposition network in the present invention can effectively remove the impact of illumination changes, improve the overall image quality and reduce noise, which is crucial for subsequent target detection tasks because high-quality input images help improve the accuracy and reliability of detection results.

[0033] The synthesis channel is used to reconstruct the first image and the second image by using the pseudo-true value of the illumination information obtained by decomposing the second channel and the first decoding reflectivity obtained by decoding the first channel;

[0034] In this embodiment, a new image is reconstructed by synthesizing the channel using the pseudo-true value of the illumination information obtained by decomposing the second channel and the first decoded reflectivity obtained by decoding the first channel. This method can generate more training samples during the training process and further reduce the dependence on the actual dark light image annotation data.

[0035] The synthesis channel is used to reconstruct the first image and the second image using the pseudo-true value of the illumination information decomposed by the second channel and the first decoded reflectivity decoded by the first channel, specifically: The true value of the illumination information of the original image obtained by decomposing the second channel The true value of illumination information of dark light image , and the first decoded reflectivity of the original image obtained by decoding the first channel First decoded reflectance with dark light image Forming the first information pair With the second information ;in, represents a dark light image, represents the original image; Exchange the first information pair With the second information The first decoded reflectivity in the image is obtained to obtain the first reconstructed information pair With the second reconstruction information ; By first reconstructing the information Reconstruct the original image to obtain the first image. The reconstruction formula is: ; In the formula, represents the decoding process of the first channel for the original image and the dark light image, 2 represents the reconstruction process, represents the first reconstructed image; Through the second reconstruction information Reconstruct the dark light image to obtain the second image. The reconstruction formula is: ; In the formula, represents the reconstructed second image.

[0036] In the present invention, the first channel extracts and decodes the reflectivity (including the first decoded reflectivity and the second decoded reflectivity) that is not affected by illumination changes based on the Retinex theory, ensuring that the essential characteristics of the object remain stable; the pre-trained Retinex decomposition network in the second channel decomposes the input image into reflectivity and illumination information pseudo-true values, and the synthesis channel reconstructs a new image using this information; since the reflectivity is illumination invariant, the model can maintain consistent target features even under different illumination conditions, thereby improving the accuracy and robustness of target detection; this separation and recombination mechanism reduces the dependence on large-scale labeled data and significantly improves the robustness and generalization ability of the model; therefore, through the stability of the reflectivity, the model achieves effective adaptation to different illumination conditions.

[0037] Before training the DAI-Net model, it also includes: freezing the pre-trained weights in the pre-trained Retinex decomposition network.

[0038] It should be noted that before starting to train the DAI-Net model, a strategy of freezing the weights of the pre-trained Retinex decomposition network was adopted. This is to retain the important feature representations learned from a large amount of data and prevent the loss of these valuable prior knowledge during fine-tuning for specific tasks. By freezing these pre-trained weights, it is ensured that the Retinex decomposition network performs well when dealing with changes in illumination, while allowing other parts of the model (such as the backbone network and the detection head) to focus on learning features related to the target task. This approach not only improves the efficiency of model training, but also enhances the adaptability and robustness of the model to different lighting conditions, especially the ability to perform accurate target detection in low-light environments. In addition, this strategy allows newly added layers to quickly learn and adapt to specific tasks on the existing basis without being disturbed by the update of the underlying network parameters.

[0039] The present invention uses the pseudo-true value of reflectivity decomposed by a pre-trained Retinex decomposition network to train the reflectivity decoder. The purpose of this is to let the model learn how to extract stable features (such as secondary decoded reflectivity) from the input image that are not affected by illumination changes.

[0040] Based on each channel of the DAI-Net model, multiple loss functions are constructed, and the DAI-Net model is trained by multiple loss functions and data sets to obtain a dark-light adaptive target enhancement detection model; the loss function includes a decomposition consistency loss function and a mutual feature alignment loss function; the mutual feature alignment loss function is used to align the original image extracted by the first channel with the low-level features of the dark-light image; the decomposition consistency loss function is used to constrain the consistency of the decoded reflectivity obtained by twice decoding the first channel; The loss function also includes: Detection loss function, which is used to evaluate the performance of the object detector in detecting the object to be detected; Reflectivity learning loss function, used to optimize the output quality of the reflectivity decoder; Image decomposition loss function, used to measure the accuracy of the pseudo-true value of the reflectivity and illumination information decomposed by the Retinex decomposition network; The total loss function of the DAI-Net model is used to combine the constructed loss functions to achieve comprehensive optimization of the DAI-Net model in target detection, reflectivity decoding and image decomposition.

[0041] Specifically: The detection loss function is used to evaluate the performance of the target detector in identifying a specific target, ensuring that the model can accurately locate and classify the target to be detected in the image. Next, the reflectivity learning loss function focuses on optimizing the output quality of the reflectivity decoder, ensuring that the reflectivity information extracted from complex lighting conditions is as close to the actual situation as possible, thereby improving the realism and usability of the image.

[0042] Furthermore, the image decomposition loss function is used to ensure that the Retinex decomposition network can work stably under various lighting conditions and generate high-quality decomposition results. Finally, in order to achieve comprehensive optimization of the DAI-Net model in target detection, reflectivity decoding, and image decomposition, the total loss function combines the various loss functions constructed above. In this way, effective learning and performance improvement of each channel are promoted, and an efficient and robust task processing model is realized. This multi-level loss function design strategy provides a solid theoretical foundation and technical support for solving complex image processing problems.

[0043] At the same time, the present invention introduces the Retinex decomposition network to extract the reflectance component and designs specific loss functions (such as mutual feature alignment loss and decomposition consistency loss) to enable the model to adapt to various lighting conditions. In this way, even if some extreme lighting conditions are not encountered during training, the model can generalize well and accurately perform target detection tasks.

[0044] The formula expression of the decomposition consistency loss function is: ; In the formula, represents the secondary decoding reflectivity obtained by decoding the first image by the first channel, represents the secondary decoding reflectivity obtained by decoding the second image by the first channel, express and The L1 norm between express and The L1 norm between Represents the loss value of the decomposition consistency loss function;

[0045] The formula expression of the mutual feature alignment loss function is: ; In the formula, express Divergence, and They respectively represent the low-level features of the original image and the dark-light image after being preprocessed by the front-end part, wherein the preprocessing includes flattening and spatial averaging. Represents the loss value of the mutual feature alignment loss function.

[0046] The formula expression of the image decomposition loss function is: ; In the formula, represents the reconstruction loss, which is used to evaluate the difference between the reconstructed first image and the original image, and the difference between the reconstructed second image and the dark light image; represents the illumination smoothness loss, which is used to make the reconstructed first image and the second image have a smooth transition; Represents the lighting smoothness weight; ,in, represents the reflectivity consistency loss, and They respectively represent the first decoded reflectivity corresponding to the original image and the dark-light image output by the reflectivity decoder, or the second decoded reflectivity corresponding to the first image and the second image output by the reflectivity decoder; is equal to value 1 or value 2; Represents the reflectivity consistency weight.

[0047] The formula expression of the reflectivity learning loss function is: ; In the formula, Represents the pseudo-true value of the reflectivity decomposed by the Retinex decomposition network, Indicates the first decoding reflectivity or the second decoding reflectivity output by the reflectivity decoder. represents the structural similarity index, express and The mean absolute error between Represents the loss value of the reflectivity learning loss function;

[0048] Specifically, It is the pseudo-true value of the reflectivity decomposed by the Retinex decomposition network for the original image or dark light image; It represents the first decoded reflectivity corresponding to the original image or the dark-light image output by the reflectivity decoder, or the second decoded reflectivity corresponding to the first image and the second image output by the reflectivity decoder.

[0049] The total loss function of the DAI-Net model is expressed as: ; In the formula, Represents the loss value of the detection loss function, represents the weight of the mutual feature alignment loss function in the total loss function, represents the weight of the decomposition consistency loss function in the total loss function, Represents the total loss.

[0050] The present invention designs a variety of loss functions (such as decomposition consistency loss, mutual feature alignment loss, reflectivity learning loss, image decomposition loss, etc.), and jointly trains the DAI-Net model through these loss functions; this approach not only improves the performance of the model in target detection tasks, but also enhances its robustness to illumination changes, ensuring consistency and stability under different illumination conditions.

[0051] The input image is processed through the dark-light adaptive target enhancement detection model to detect and output the location information and category information of the target to be detected.

[0052] In order to verify the effectiveness of the present invention, this embodiment has carried out comparative experiments in tasks in low-light environments such as face detection, object detection and image classification, and has compared them with more advanced algorithms in the current field.

[0053] 1. In the face detection task, the public WIDER FACE and COCO were used as training datasets, and DARK FACE was selected as the test set. The basic face detector, i.e., the target detector, uses DSFD or YOLOv3. During the training process, the loss weight parameters are set as follows: =0.5, =0.01, =0.001, = 0.1, and fine-tuned on the WIDERFACE validation set. and The test results are shown in Table 1 below.

[0054] Table 1:

[0055] In Table 1, mAP(%) represents the average precision of the face detection dark light test set (DARK FACE). In the table:

[0056] First test:

[0057] 1. Fully supervised training means that the model is trained with a large amount of labeled data and can achieve excellent performance on the test set. For example, the model is first fully trained on the WIDER FACE dataset and then applied to the DARKFACE test set.

[0058] 2. Zero-shot adaptive training means that the model can be trained with source domain data without direct contact with the target domain data, and achieve good performance on the target domain data. For example, in the zero-shot adaptation setting, the DAI-Net model is trained only with the WIDER FACE dataset (i.e., the method of synthesizing dark light images from original images through a physically inspired low-light synthesis pipeline in this invention), and then applied to the DARK FACE test set.

[0059] Under this training method, the mAP of the DAI-Net model reaches 28.0%, which is higher than 18.4% of CIConv and 25.7% of Sim-MinMax.

[0060] 3. Fine-tuning training, for example, the model is first fully trained on the WIDER FACE dataset, then fine-tuned using WIDER FACE as a validation set, and finally applied to the DARK FACE test set. Under this training method, the mAP of the DAI-Net model is 52.9%.

[0061] Second test:

[0062] 1. Fully supervised training means that the model is trained with a large amount of labeled data and can achieve excellent performance on the test set. For example, the model is first fully trained on the COCO dataset and then applied to the DARK FACE test set. Under this training method, the mAP of the DAI-Net model is 57.0%, which exceeds the 55.8% of MAET and other related methods. It can be understood here that DAI-Net also performs better than other methods under the fully supervised setting.

[0063] 2. In the object detection task, YOLOv3 is used as the object detector, COCO is used as the training set, and ExDark is used as the test set. The hyperparameter settings are consistent with the face detection experiment. The test results are shown in Table 2 below.

[0064] Table 2:

[0065] Table 2 shows the performance indicators of several different methods when detecting different categories of objects. The values ​​in the table represent the detection accuracy of each method in each category (usually expressed as a percentage). The specific explanation is as follows:

[0066] 1. Detection methods: Several different detection models are listed, including YOLON, KinD, Zero-DCE, YOLOL, MAET, and DAI-Net.

[0067] 2. Categories: Each column represents a specific detection category, including bicycle, boat, bottle, bus, car, cat, chair, cup, dog, motorcycle, person and table.

[0068] These values ​​represent the detection accuracy of each method on different categories. As can be seen from the table, the DAI-Net trained by the present invention performs best on most categories, while YOLON performs poorly on some categories. Overall, the total average accuracy of the DAI-Net trained by the present invention is the highest, at 78.3%.

[0069] 3. In the image classification task, ResNet-18 was used as the target detector and experiments were conducted on the night image classification dataset CODaN. The hyperparameter settings were also consistent with the face detection experiment. The test results are shown in Table 3 below.

[0070] Table 3:

[0071] Table 3 shows the Top-1 accuracy of different methods for image classification tasks on the CODaN dataset.

[0072] 1. Top-1 accuracy: Top-1 accuracy refers to the probability that the most likely category predicted by the model matches the true category. For example, if a model correctly predicts the most likely category for each sample in the test set, then its Top-1 accuracy is 100%.

[0073] 2. Performance comparison of different methods:

[0074] MAET: The Top-1 accuracy is 56.48%, which means that among all test samples, 56.48% of the samples are correctly predicted as the most likely category.

[0075] ClConv: Top-1 accuracy is 60.32%, slightly higher than MAET, but still lower.

[0076] Sim-MinMax: The Top-1 accuracy is 65.87%, which further improves the accuracy.

[0077] DAI-Net: The Top-1 accuracy is 68.44%, the highest among all models.

[0078] 3. Conclusion: The Top-1 accuracy of the DAI-Net trained by the present invention on the CODaN dataset significantly exceeds that of other methods, reaching 68.44%. This shows that the DAI-Net trained by the present invention has a strong generalization ability in image classification tasks, can be effectively applied to dark-light image classification tasks, and has obvious advantages over other comparison models.

[0079] The present invention first obtains a data set including an original image and its corresponding dark-light image (generated by a low-light synthesis pipeline); then, a DAI-Net model including a first channel, a second channel and a synthesis channel is constructed, wherein the first channel extracts and decodes low-level features, the second channel uses a pre-trained Retinex decomposition network to obtain reflectivity and illumination information pseudo-true values, and the synthesis channel uses this information to reconstruct a new image and decode it again; then, a plurality of loss functions are used to train the model to obtain a dark-light adaptive target enhancement detection model; finally, the model is used to detect the target to be detected in the input image, and the method significantly improves the target detection performance in a low-light environment; by combining Retinex theory and multi-channel design, the model can extract illumination invariant features (reflectivity) from the original image and the dark-light image, and maintain stable detection accuracy under different illumination conditions; at the same time, a plurality of loss functions are used for joint training to further enhance the robustness and generalization ability of the model, so that it can still work effectively when there is a lack of labeled data under specific low-light conditions; in addition, this method reduces the dependence on large-scale labeled data, improves image quality, and reduces noise interference, thereby showing higher reliability and wide applicability in practical applications.

[0080] It should be noted that the term "including" and its variations used in the embodiments of the present invention are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0081] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0082] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A method for adaptively enhancing target detection in a weak light environment based on zero-sample learning, characterized in that: include: Acquire a data set, wherein the data set includes a plurality of training samples, each of which includes: an original image and a corresponding low-light image; the low-light image is a synthetic image generated from the original image through a physically inspired low-light synthesis pipeline; each original image and low-light image is provided with a training label; the training label includes position information and category information of the target to be detected; A DAI-Net model including a first channel, a second channel and a synthesis channel is constructed based on the target detector; wherein: the first channel is used to extract low-level features corresponding to the original image and the dark-light image, and the low-level features are decoded to obtain the reflectivity corresponding to each image as the first decoded reflectivity; The second channel includes a pre-trained Retinex decomposition network, which is used to decompose the original image and the dark-light image to obtain pseudo-true values ​​of reflectivity and illumination information corresponding to the original image and the dark-light image respectively; The synthesis channel is used to reconstruct the first image and the second image using the pseudo-true value of the illumination information decomposed by the second channel and the first decoded reflectivity decoded by the first channel; the first channel is also used to decode the first image and the second image to obtain the second decoded reflectivity; Based on each channel of the DAI-Net model, multiple loss functions are constructed, and the DAI-Net model is trained by multiple loss functions and data sets to obtain a dark-light adaptive target enhancement detection model; the loss function includes a decomposition consistency loss function and a mutual feature alignment loss function; the mutual feature alignment loss function is used to align the original image extracted by the first channel with the low-level features of the dark-light image; the decomposition consistency loss function is used to constrain the consistency of the decoded reflectivity obtained by twice decoding the first channel; The input image is processed through the dark-light adaptive target enhancement detection model to detect and output the location information and category information of the target to be detected.

2. According to claim 1, a method for adaptively enhancing target detection in a weak light environment based on zero-sample learning is characterized in that: The first channel includes a first segmentation channel and a second segmentation channel, and the construction method thereof is: based on the Retinex theory, the backbone network of the target detector is segmented to obtain the first segmentation channel and the second segmentation channel; The first segmentation channel is used to extract low-level features corresponding to the original image and the dark-light image, and decode the low-level features to obtain the reflectivity corresponding to each image as the first decoding reflectivity; and is also used to decode the first image and the second image to obtain the second decoding reflectivity; The second segmentation channel is used to detect the position information and category information of the target to be detected through the low-level features extracted by the first segmentation channel and the secondary decoding reflectivity obtained by decoding.

3. According to claim 2, the method for adaptively enhancing target detection in a weak light environment based on zero-sample learning is characterized in that: The backbone network of the target detector is segmented based on the Retinex theory to obtain the first segmentation channel and the second segmentation channel, specifically: Segmentation is performed at the second convolutional layer of the backbone network: the part before the second convolutional layer is set as the front end part, and a reflectivity decoder is set at the output end of the front end part, thereby obtaining a first segmentation channel formed by the front end part and the reflectivity decoder; Set the entire backbone network as the second split channel; The reflectivity decoder is used to decode low-level features.

4. According to claim 3, the method for adaptively enhancing target detection in a weak light environment based on zero-sample learning is characterized in that: The synthesis channel is used to reconstruct the first image and the second image using the pseudo-true value of the illumination information decomposed by the second channel and the first decoded reflectivity decoded by the first channel, specifically: The true value of the illumination information of the original image obtained by decomposing the second channel The true value of illumination information of dark light image , and the first decoded reflectivity of the original image obtained by decoding the first channel First decoded reflectance with dark light image Forming the first information pair With the second information ;in, represents a dark light image, represents the original image; Exchange the first information pair With the second information The first decoded reflectivity in the image is obtained to obtain the first reconstructed information pair With the second reconstruction information ; By first reconstructing the information Reconstruct the original image to obtain the first image. The reconstruction formula is: ; In the formula, represents the decoding process of the first channel for the original image and the dark light image, 2 represents the reconstruction process, represents the first reconstructed image; Through the second reconstruction information Reconstruct the dark light image to obtain the second image. The reconstruction formula is: ; In the formula, represents the reconstructed second image.

5. According to claim 4, the method for adaptively enhancing target detection in a weak light environment based on zero-sample learning is characterized in that: The loss function also includes: Detection loss function, which is used to evaluate the performance of the object detector in detecting the object to be detected; Reflectivity learning loss function, used to optimize the output quality of the reflectivity decoder; Image decomposition loss function, used to measure the accuracy of the pseudo-true value of the reflectivity and illumination information decomposed by the Retinex decomposition network; The total loss function of the DAI-Net model is used to combine the constructed loss functions to achieve comprehensive optimization of the DAI-Net model in target detection, reflectivity decoding and image decomposition.

6. The method for adaptively enhancing target detection in a weak light environment based on zero-sample learning according to claim 5, characterized in that: The formula expression of the decomposition consistency loss function is: ； In the formula, represents the secondary decoding reflectivity obtained by decoding the first image by the first channel, represents the secondary decoding reflectivity obtained by decoding the second image by the first channel, express and The L1 norm between express and The L1 norm between Represents the loss value of the decomposition consistency loss function; The formula expression of the mutual feature alignment loss function is: ； In the formula, express Divergence, and They respectively represent the low-level features of the original image and the dark-light image after being preprocessed by the front-end part, wherein the preprocessing includes flattening and spatial averaging. Represents the loss value of the mutual feature alignment loss function.

7. The method for adaptively enhancing target detection in a weak light environment based on zero-sample learning according to claim 6, characterized in that: The formula expression of the image decomposition loss function is: ； In the formula, represents the reconstruction loss, which is used to evaluate the difference between the reconstructed first image and the original image, and the difference between the reconstructed second image and the dark light image; represents the illumination smoothness loss, which is used to make the reconstructed first image and the second image have a smooth transition; Represents the lighting smoothness weight; ,in, represents the reflectivity consistency loss, and They respectively represent the first decoded reflectivity corresponding to the original image and the dark-light image output by the reflectivity decoder, or the second decoded reflectivity corresponding to the first image and the second image output by the reflectivity decoder; is equal to value 1 or value 2; Represents the reflectivity consistency weight.

8. The method for adaptively enhancing target detection in a weak light environment based on zero-sample learning according to claim 7, characterized in that: The formula expression of the reflectivity learning loss function is: ； In the formula, Represents the pseudo-true value of the reflectivity decomposed by the Retinex decomposition network, It indicates the first decoding reflectivity or the second decoding reflectivity output by the reflectivity decoder. represents the structural similarity index, express and The mean absolute error between Represents the loss value of the reflectivity learning loss function; The total loss function of the DAI-Net model is expressed as: ; In the formula, Represents the loss value of the detection loss function, represents the weight of the mutual feature alignment loss function in the total loss function, represents the weight of the decomposition consistency loss function in the total loss function, Represents the total loss.

9. The method for adaptively enhancing target detection in a weak light environment based on zero-sample learning according to any one of claims 1 to 8, characterized in that: Before training the DAI-Net model, it also includes: freezing the pre-trained weights in the pre-trained Retinex decomposition network.

10. The method for adaptively enhancing target detection in a weak light environment based on zero-sample learning according to any one of claims 3 to 8, characterized in that: The target detector is a YOLO model; the reflectivity decoder includes: a first convolutional layer, a first ReLU activation function, a second convolutional layer and a second ReLU activation function connected in sequence.

Images