A glare self-adaptive discrimination method for low-light scenes

Abstract

The application discloses a glare self-adaptive discrimination method for a low-light scene, and comprises the following steps: acquiring a low-light image containing glare and performing pretreatment, inputting the pretreated image into a trained glare representation network to obtain a predicted glare component image, generating a soft guide mask and a glare region of interest mask based on the predicted glare component image, performing mask-guided feature aggregation on network features by using the glare region of interest mask to obtain a glare type representation, and inputting the glare type representation into a type classifier to output a glare type result. The application can realize accurate identification of a glare region in a low-light scene and adaptive discrimination of a glare type, and provides reliable prior information for subsequent glare removal and visual tasks.

Description

Technical Field

[0001] This invention belongs to the field of computer image technology, specifically relating to an adaptive glare discrimination method for low-light scenes. Background Technology

[0002] In images captured at night or under strong light sources, multiple reflections or scattering within the lens can cause glare phenomena such as localized highlights, artifacts, and color distortion. Glare typically manifests as polygonal light spots, halos, streaks, and diffused halos, characterized by strong spatial locality, drastic brightness variations, and complex boundary transitions. It not only obscures the texture and edge information of objects in the image but also significantly interferes with subsequent visual tasks such as object detection and semantic segmentation.

[0003] Existing image enhancement and restoration methods mostly focus on overall brightness adjustment, noise removal, or global contrast enhancement, lacking specific identification of glare regions. Some methods treat glare as ordinary noise and process it globally, easily leading to loss of background details or glare residue. Although deep learning-based glare removal methods have emerged in recent years, their effectiveness is highly dependent on the accurate identification of glare regions and types; if reliable information on glare location and type cannot be obtained in advance, subsequent models will struggle to suppress glare artifacts while preserving background texture.

[0004] Therefore, how to accurately and efficiently identify glare areas and further determine the type of glare in low-light scenes, so as to provide reliable prior information for subsequent glare removal or other visual processing tasks, has become an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive glare discrimination method for low-light scenes, so as to solve the problems of difficulty in glare recognition, difficulty in accurately distinguishing glare types, and lack of discrimination results that have guiding significance for subsequent processing in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A glare adaptive discrimination method for low-light scenes includes:

[0008] A low-light image containing glare is acquired and preprocessed; the preprocessed image is input into a trained glare representation network to obtain a predicted glare component image; a soft-guided mask and a glare region of interest mask are generated based on the predicted glare component image; the glare region of interest mask is used to perform mask-guided feature aggregation on the network features to obtain a glare type representation; the glare type representation is input into a type classifier to output the glare type result.

[0009] Furthermore, the preprocessing includes performing size unification, bit depth unification, pixel normalization, and data enhancement on the low-light image with glare; the data enhancement includes one or more of random rotation, flipping, translation, or color dithering.

[0010] Furthermore, the generation of the soft guide mask and the glare region of interest mask includes: taking the maximum value of the predicted glare component image in the channel dimension to obtain a brightness map; thresholding the brightness map using a hard thresholding Sigmoid function to obtain a hard protection mask; and multiplying the soft protection mask output by the soft thresholding Sigmoid function element by element with the hard protection mask to obtain the soft guide mask.

[0011] The glare region of interest mask is obtained by performing a morphological dilation operation on the hard protection mask to cover the glare boundary halo region.

[0012] Furthermore, the mask-guided feature aggregation includes: weighting the network feature map according to the interest region mask, and performing weighted pooling in the spatial dimension to obtain a region-level glare type representation, thereby suppressing the interference of the background region on type discrimination.

[0013] Furthermore, the type classifier outputs probability vectors for four types of glare, and determines the glare type result based on the highest probability.

[0014] Furthermore, the glare representation network is an encoder-decoder structure, and multi-scale features are fused through skip connections to improve the glare region representation capability.

[0015] Furthermore, the trained glare representation network and the type classifier are obtained through joint training, and the joint loss includes pixel reconstruction loss and perceptual loss for glare component prediction, and cross-entropy loss for type discrimination; wherein the cross-entropy loss is weighted and calculated under the guidance of the region of interest mask.

[0016] Furthermore, the joint training also includes: jointly training the glare representation network and the type classifier using the Adam optimizer, setting the initial learning rate of the Adam optimizer to 10. -4 .

[0017] Furthermore, the glare type includes one of the following four categories: dot glare, ring glare, strip glare, and diffuse halo; wherein the dot glare includes polygonal spot glare.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] (1) The present invention obtains the predicted glare component image through the glare characterization network, and further generates a soft guide mask and a glare region of interest mask, which can accurately locate the glare region in low light scene and provide reliable prior information for subsequent type discrimination and downstream recovery tasks.

[0020] (2) The present invention suppresses the interference of the background region on type discrimination by using the feature aggregation method guided by the mask, so that the type classifier pays more attention to the significant features of the glare body and its boundary region, thereby improving the accuracy and robustness of glare type discrimination.

[0021] (3) By jointly training the glare component prediction task and the glare type classification task, the present invention enables the representation network to learn the pixel-level glare distribution while taking into account the type semantic information, thereby improving the model's ability to distinguish various glare forms.

[0022] (4) The soft guide mask, glare region of interest mask and glare type results output by the present invention have good versatility and can be directly used as guide information for subsequent visual tasks such as image deglare, target detection and semantic segmentation, which is conducive to improving the overall processing effect in low light glare interference scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating an adaptive glare detection method for low-light scenes according to an embodiment of the present invention.

[0025] Figure 2 This is a logical schematic diagram of a glare adaptive discrimination method for low-light scenes in an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram illustrating the relationship between the soft bootstrap mask, the ROI mask, and the output results in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention more apparent and understandable, the technical solutions of this invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Please see Figures 1 to 3 As shown, this invention provides a glare adaptive discrimination method for low-light scenes.

[0029] First, acquire low-light images with glare and perform preprocessing. Preprocessing includes size unification, bit depth unification, pixel normalization, and data augmentation; among which, data augmentation includes one or more of random rotation, flipping, translation, or color dithering.

[0030] In practical implementation, low-light glare image samples can be constructed based on public datasets and self-collected data. The samples include low-light glare images of various glare types under different background scenes. During preprocessing, the input images can be uniformly adjusted to 640×640×3 RGB images, the bit depth of the RGB images can be uniformly adjusted to 8 bits, and the pixel values ​​can be normalized to the [0,1] range. On this basis, data augmentation operations such as random rotation, flipping, translation, and color jitter can be further implemented to improve the model's adaptability to different scenes and different glare forms.

[0031] The above preprocessing can reduce the distribution differences caused by different devices, different acquisition conditions and different low-light backgrounds, and improve the stability and generalization ability of the glare discrimination model in the training and inference stages.

[0032] After obtaining the preprocessed image, the preprocessed image is input into the trained glare representation network to obtain the predicted glare component image.

[0033] The glare representation network has an encoder-decoder structure and integrates multi-scale features through skip connections to improve the representation capability of glare regions.

[0034] Specifically, the glare representation network adopts an encoder-decoder architecture, with basic units using convolutional modules that incorporate simplified gating mechanisms and channel attention. The input layer receives a preprocessed image of size 640×640×3; the encoder includes multiple downsampling stages, each consisting of several basic units and convolutional layers with a stride of 2, used to progressively extract deep features and expand the receptive field; intermediate blocks are used to further extract high-level semantic information related to glare; the decoder is symmetrical to the encoder, progressively restoring spatial resolution through upsampling modules and skip connections, and fusing shallow and deep features, finally generating a predicted glare component image F of the same size as the input through the output layer. p .

[0035] In one embodiment, the basic unit may sequentially include layer normalization, 1×1 convolutional channel expansion, 3×3 depthwise convolution, simplified gating, simplified channel attention, and residual connections; a dual attention module may also be introduced in the decoding stage to enhance the ability to express key features related to glare regions.

[0036] After obtaining the predicted glare component image, a soft guide mask and a glare region of interest mask are generated based on the predicted glare component image.

[0037] Specifically, the maximum value of the predicted glare component image in the channel dimension can be taken first to obtain a single-channel brightness map.

[0038] A single-channel brightness map can be represented as:

[0039]

[0040] Where L represents the single-channel luminance map, F p This represents the predicted glare component image, where x represents the pixel position in the image. This represents the operation of finding the maximum value in the channel dimension c.

[0041] After obtaining the brightness map, the brightness map is thresholded using a hard thresholding Sigmoid function with a high slope to obtain a hard protection mask.

[0042] A hard-protection mask can be represented as:

[0043]

[0044] Among them, M g This represents the hard protection mask, σ(k) h ) indicates that the slope is k h The Sigmoid function, This represents the hard threshold. Let k be a value. h =10 6 , =10 -6 This causes pixels with brightness slightly above the threshold to quickly approach 1, while background areas approach 0.

[0045] Subsequently, a soft-threshold Sigmoid function is introduced on the basis of the hard protection mask, and its output is multiplied element by element with the hard protection mask to obtain the soft boot mask.

[0046] In practice, a soft threshold Sigmoid function is introduced on the basis of the hard protection mask to generate a soft protection mask with continuous transition values ​​at the glare boundary. The soft protection mask is then multiplied element by element with the hard protection mask to obtain a soft guidance mask, so as to ensure that the soft protection mask value in the non-glare area is forced to 0.

[0047] The soft boot mask can be represented as:

[0048]

[0049] Among them, M s Represents the soft boot mask, σ(k) s) indicates that the slope is k s The Sigmoid function, represents the soft threshold, and ⊙ represents element-wise multiplication. Set k... s =50, =0.02, which makes the glare boundary region have a smooth transition value and reduces the ringing effect in subsequent processing.

[0050] Furthermore, a morphological dilation operation is performed on the hard protection mask to obtain a region of interest (ROI) mask, which covers the main glare area and its edge halo area. The dilation operation can be implemented using 3×3 max pooling with padding of 1, causing the mask to expand outward by one or more pixels.

[0051] It can be represented as:

[0052]

[0053] Among them, M r This represents the ROI mask corresponding to the region of interest in the glare, and D() represents the morphological dilation operation.

[0054] By generating the brightness map, hard protection mask, soft guide mask, and ROI mask in the above order, a soft guide mask with smooth boundary transition characteristics and a glare interest region mask that can cover the glare edge halo area can be obtained simultaneously, providing multi-level region prior information for subsequent type discrimination.

[0055] After generating the soft-guided mask and the glare region of interest mask, the network features are aggregated using mask-guided features based on the glare region of interest mask to obtain the glare type representation.

[0056] Mask-guided feature aggregation includes weighting the network feature map by ROI mask and performing weighted pooling in the spatial dimension to obtain a region-level glare type representation.

[0057] Specifically, network features F for type discrimination can be selected from the mid-to-high-level feature maps of the glare representation network. c The network features are then element-wise weighted with the ROI mask to enhance the response in glare regions and suppress the response in background regions. Based on this, weighted pooling is performed on the weighted features in the spatial dimension to obtain a region-level glare type representation z. f .

[0058] Mask-guided feature aggregation can be represented as:

[0059]

[0060] Among them, z fThe `Pool()` method represents the type of glare, and `F` represents the weighted pooling operation performed along the spatial dimension. c This represents the network features used for type identification.

[0061] In one embodiment, the weighted pooling operation can be implemented using global average pooling or global weighted average pooling to suppress background response while preserving the main discriminative features of the glare region.

[0062] The feature aggregation method guided by the above mask can significantly reduce the interference of the background region on the glare type discrimination, and make the aggregated representation more focused on the glare body and its boundary region features, thereby improving the stability and accuracy of type discrimination.

[0063] After obtaining the glare type representation, the glare type representation is input into the type classifier, which outputs the glare type result. The type classifier outputs probability vectors for four types of glare and determines the glare type result based on the highest probability. Glare types include point glare, ring glare, strip glare, and diffuse halo, where point glare includes polygonal spot glare.

[0064] Specifically, the glare type can be characterized by z f Input a fully connected layer or other classification head, output a four-dimensional type probability vector p, and then determine the glare type result based on the category corresponding to the highest probability.

[0065] The type classification process can be represented as:

[0066]

[0067] Where p represents the glare type probability vector, W represents the classifier weight matrix, b represents the classifier bias term, and z f This represents the type of glare, and Softmax() represents the soft maximum normalization function. p represents the final output glare type result. i represents the predicted probability of the i-th type of glare, and argmax represents the operation of taking the index of the category corresponding to the maximum probability.

[0068] The four types of glare can be defined as point glare, ring glare, strip glare, and diffuse halo, respectively. Among them, point glare can further cover typical glare forms such as polygonal light spots.

[0069] Furthermore, to obtain the aforementioned trained glare representation network and type classifier, the glare representation network and type classifier can be jointly trained.

[0070] The limited joint loss includes pixel reconstruction loss and perceptual loss for glare component prediction, and cross-entropy loss for type discrimination. To alleviate the imbalance in the number of glare samples of various types, class weights can be introduced into the cross-entropy loss and / or focal loss can be used to replace the cross-entropy loss.

[0071] In the specific training process, a glare recognition dataset can be constructed based on publicly available datasets and self-collected data. For each training sample, a preprocessed low-light image containing glare can be used as input, and the corresponding ground truth image of glare, F, can be used as input. gt As a monitoring signal for glare component prediction, and to label the glare type. As a type discrimination supervision signal, the dataset can be divided into training, validation, and test sets in an 8:1:1 ratio.

[0072] For the glare component prediction task, pixel reconstruction loss and perceptual loss can be used together to supervise the difference between the predicted glare component image and the ground truth glare image.

[0073] The pixel reconstruction loss can be expressed as:

[0074]

[0075] Among them, L r F represents the pixel reconstruction loss, where N represents the total number of pixels in the image, and F represents the pixel reconstruction loss. p F represents the predicted glare component image. gt Represents the true value image of glare. This represents the L1 norm.

[0076] Perceptual loss can be expressed as:

[0077]

[0078] Among them, L p N represents the perceived loss. f φ represents the number of feature layers involved in the perceptual loss calculation. j () represents the feature map output by the pre-trained VGG16 feature extraction network at layer j.

[0079] For type discrimination tasks, the cross-entropy loss can be weighted under the guidance of ROI mask, so that type classification pays more attention to the effective area of ​​glare.

[0080] The cross-entropy loss guided by the ROI mask can be expressed as:

[0081]

[0082] Among them, L c Represents the type classification loss, α rw represents the region weight coefficients derived from the ROI mask. i y represents the category weight of the i-th type of glare. i p represents the true label of the i-th type of glare. i Let represent the predicted probability of the i-th type of glare.

[0083] In one embodiment, α r It can be determined based on the average response value of the ROI mask, the effective pixel ratio, or other equivalent region statistics, so as to enhance the constraint effect of the effective glare region on the type discrimination loss.

[0084] In scenarios with unbalanced sample distribution, Focal Loss can be used instead of the aforementioned Cross-Entropy Loss.

[0085] The focal loss can be expressed as:

[0086]

[0087] Among them, L f γ represents the focus loss, and γ represents the focus loss modulation factor.

[0088] In implementations that use Focal Loss instead of Cross-Entropy Loss, the type classification loss term L in the joint loss function... c It can be replaced with L accordingly. f .

[0089] Furthermore, the pixel reconstruction loss, perception loss, and type classification loss can be weighted and summed to obtain a joint training loss function.

[0090] The joint loss function can be expressed as:

[0091]

[0092] Where Loss represents the joint loss function, λ r λ represents the weighting coefficients of the pixel reconstruction loss. p λ represents the weighting coefficient of the perceptual loss. c L represents the weight coefficients of the type classification loss. r L represents the pixel reconstruction loss. p L represents perceived loss. c This represents the type classification loss. In one embodiment, λ can be taken as... r =0.2、λ p =0.5、λ c =0.3.

[0093] In one embodiment, the dazzling representation network and the type classifier can be jointly trained using an adaptive moment estimation (Adam) optimizer, with an initial learning rate set to 10.-4 The learning rate is adjusted using a cosine annealing strategy. If the validation set loss does not decrease for several consecutive training rounds, an early stopping mechanism can be used to terminate training in order to avoid model overfitting.

[0094] Based on the above embodiments, the technical effects of the present invention at each technical stage can be further summarized as follows.

[0095] By performing size unification, bit depth unification, pixel normalization, and data augmentation on low-light images with glare, the differences in data distribution caused by different acquisition devices, different low-light backgrounds, and different glare patterns can be reduced, and the consistency and diversity of training samples can be improved. This solves the problems of insufficient stability and limited generalization ability of glare recognition in complex low-light scenes in background technology.

[0096] A glare representation network with an encoder-decoder structure is constructed, and a multi-scale feature fusion and attention enhancement mechanism are combined to generate a predicted glare component image. This enables fine-grained representation of the main glare region and its boundary structure, thereby solving the problems of existing technologies that lack targeted identification of glare regions and tend to treat glare as ordinary noise.

[0097] The predicted glare component image is further used to generate a brightness map, a hard protection mask, a soft guide mask, and a region of interest mask. This not only enables precise localization of the main glare area but also preserves the smooth transition information at the glare boundary. This solves the problems in background techniques, such as the difficulty in accurately depicting glare boundaries and the easy occurrence of glare residue or accidental damage to background details.

[0098] By performing mask-guided feature aggregation on network features based on region of interest masks and combining it with a type classifier to output glare type results, the interference of background regions on the classification process can be effectively suppressed, making the model more focused on the glare body and its boundary features. This solves the problems of glare type being difficult to distinguish accurately and type discrimination results lacking stability in background technology.

[0099] Joint training is performed on the glare component prediction task and the glare type discrimination task. By introducing mechanisms such as interest region mask weighting, class weighting and focus loss into the type discrimination loss, the ability to represent glare regions can be improved while enhancing the adaptability to difficult-to-classify samples and class imbalanced scenes. This solves the problems of insufficient accuracy in glare type discrimination and weak robustness in complex scenes in background technology.

[0100] The final output of the soft-guided mask, region of interest mask, and glare type results can serve as prior information for subsequent visual tasks such as image deglare, object detection, and semantic segmentation, providing explicit guidance for downstream processing. This solves the problem of the lack of reliable glare discrimination results in the background technology that can directly serve subsequent visual tasks.

[0101] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A glare adaptive discrimination method for low-light scenes, characterized in that, include: Acquire low-light images containing glare and perform preprocessing; The preprocessed image is input into the trained glare representation network to obtain the predicted glare component image; The glare representation network adopts an encoder-decoder structure. The encoder includes a downsampling stage, which consists of basic units and convolutional layers with a stride of 2, used to progressively extract deep features and expand the receptive field. The basic units adopt convolutional modules that include simplified gating mechanisms and simplified channel attention. The decoder is symmetrical to the encoder and progressively restores spatial resolution through upsampling modules and skip connections, and fuses shallow and deep features to improve the glare region representation capability. Generating a soft-guide mask and a glare region of interest mask based on the predicted glare component image includes: The luminance map is obtained by taking the maximum value in the channel dimension of the predicted glare component image; the luminance map is thresholded using the hard thresholding Sigmoid function to obtain a hard protection mask; the soft protection mask output by the soft thresholding Sigmoid function is multiplied element-wise with the hard protection mask to obtain a soft guidance mask; the glare region of interest mask is the region of interest mask obtained by performing a morphological dilation operation on the hard protection mask to cover the glare boundary halo region; The glare type characterization is obtained by using the glare interest region mask to perform mask-guided feature aggregation on network features. The mask-guided feature aggregation includes: weighting the network feature map according to the glare interest region mask, and performing weighted pooling in the spatial dimension to obtain a region-level glare type representation, thereby suppressing the interference of the background region on type discrimination; The glare type characterization is input into the type classifier, which outputs the glare type result.

2. The method according to claim 1, characterized in that, The preprocessing includes unifying the size, bit depth, and pixels of the low-light image with glare, as well as data enhancement; the data enhancement includes one or more of random rotation, flipping, translation, or color dithering.

3. The method according to claim 1, characterized in that, The type classifier outputs probability vectors for four types of glare, and determines the glare type result based on the highest probability.

4. The method according to claim 1, characterized in that, The trained glare representation network and the type classifier are obtained through joint training. The joint loss includes pixel reconstruction loss and perceptual loss for glare component prediction, and cross-entropy loss for type discrimination; wherein the cross-entropy loss is weighted and calculated under the guidance of the region of interest mask.

5. The method according to claim 4, characterized in that, To alleviate the imbalance in the number of glare samples of various types, class weights are introduced into the cross-entropy loss and / or focus loss is used to replace the cross-entropy loss.

6. The method according to claim 4, characterized in that, The joint training further includes: jointly training the glare representation network and the type classifier using the Adam optimizer, with the initial learning rate of the Adam optimizer set to 10. -4 .

7. The method according to claim 1, characterized in that, The glare type includes one of the following four categories: dot glare, ring glare, strip glare, and diffuse halo; wherein the dot glare includes polygonal spot glare.

Images