Semi-supervised low-light image enhancement method based on potential space mean value teacher network

Abstract

The invention provides a semi-supervised low-light image enhancement method and device based on a potential spatial mean teacher network, and relates to the technical field of computer vision. The method comprises the following steps: inputting a low-light image data set into a teacher network to generate a pseudo label set and a potential space vector set; inputting the low-light image data set into a student network to generate a prediction image set and a potential space vector set; carrying out representative vector selection through a pseudo-tag adaptation module and a principal component analysis method to obtain a base vector sequence; on the basis of a Gaussian regression method, potential spatial vector distribution of the unmarked data is calculated, posterior distribution is obtained, and total loss is calculated; optimizing the network according to the total loss; obtaining a to-be-enhanced low-light image; and based on the optimized student network and the optimized teacher network, performing image enhancement on the to-be-enhanced low-light image to obtain an enhanced image. The low-light image enhancement method based on semi-supervision is good in enhancement effect and strong in generalization ability.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a semi-supervised low-light image enhancement method and device based on a latent space mean teacher network. Background Art

[0002] Image processing plays a key role in numerous technological fields, but low-light images remain a significant challenge. Low-light images are often acquired under suboptimal lighting conditions, such as backlighting, underexposure, and low-light conditions. These images exhibit undesirable characteristics such as low visibility, low contrast, and excessive noise. These image quality issues not only result in a poor visual experience for users but also pose a significant obstacle to machine vision analysis, potentially leading to inaccurate results in subsequent image analysis tasks such as object detection and segmentation.

[0003] Traditional low-light image enhancement (LLIE) methods rely on optimization rules and manually defined prior knowledge. The drawback of this approach is that it can easily produce poor image quality under varying lighting conditions, leading to loss of detail and color distortion. With the rapid development of deep learning technology, supervised deep learning methods have been widely used in the field of LLIE, using labeled data to improve image quality. However, these methods face numerous challenges, the most prominent of which is the difficulty in obtaining labeled data. Furthermore, the models often suffer from overfitting and limited generalization, making them difficult to achieve good results in unseen scenes. Another class of image enhancement methods that do not use labeled data include unsupervised learning and zero-shot learning. Unsupervised methods use unpaired images to construct networks, which improves generalization performance to a certain extent but fails to effectively capture the intrinsic structure of image signals. Zero-shot methods achieve LLIE by carefully designing self-learning functions, but the learning process is often unstable due to the need for carefully designed loss functions. Semi-supervised learning methods have been widely used in tasks such as image deraining, dehazing, and image restoration, but research in the field of low-light image enhancement is still in its early stages. Existing semi-supervised LLIE methods also have certain problems. For example, some methods may introduce artifacts or noise in complex lighting scenes.

[0004] In the existing technology, there is a lack of a semi-supervised low-light image enhancement method with good enhancement effect and strong generalization ability. Summary of the Invention

[0005] To address the technical issues of low visual quality and weak model generalization in complex scenes in existing technologies, the present invention provides a semi-supervised low-light image enhancement method and device based on a latent space mean teacher network. The technical solution is as follows:

[0006] In one aspect, a semi-supervised low-light image enhancement method based on a latent space mean teacher network is provided. The method is implemented by a semi-supervised low-light image enhancement device, and the method includes:

[0007] Obtain a low-light image dataset;

[0008] The low-light image dataset is input into the teacher network for pseudo label generation to obtain a first pseudo label set, a first pseudo latent space vector set, a second pseudo label set, and a second pseudo latent space vector set; the low-light image dataset is input into the student network for image prediction to obtain a first predicted image set, a first latent space vector set, a second predicted image set, and a second latent space vector set;

[0009] Based on the first pseudo latent space vector set and the first latent space vector set, according to the first pseudo label and the first predicted image set, a vector sequence is constructed by a pseudo label adaptation module to obtain an optimized vector sequence; based on the optimized vector sequence, representative vectors are selected by principal component analysis to obtain a basis vector sequence;

[0010] Based on the Gaussian regression method, the latent space vector distribution of the unlabeled data is calculated according to the basis vector sequence, the second pseudo latent space vector set and the second latent space vector set to obtain the posterior distribution;

[0011] A loss function is calculated based on the labeled data set, the second pseudo label set, the first predicted image set, the second predicted image set, the second pseudo latent space vector set, and the posterior distribution to obtain an overall loss;

[0012] According to the overall loss, the Adam optimizer is used to update the parameters of the student network to obtain the optimized student network; based on the optimized student network, the exponential moving average method is used to update the parameters of the teacher network to obtain the optimized teacher network;

[0013] Obtain a low-light image to be enhanced; perform image enhancement on the low-light image to be enhanced based on the optimized student network and the optimized teacher network to obtain an enhanced image.

[0014] On the other hand, a semi-supervised low-light image enhancement device based on a latent space mean teacher network is provided. The device is applied to a semi-supervised low-light image enhancement method based on a latent space mean teacher network. The device includes:

[0015] A data acquisition module, used to acquire a low-light image dataset;

[0016] A semi-supervised image prediction module is configured to input a low-light image dataset into a teacher network for pseudo-label generation to obtain a first pseudo-label set, a first pseudo latent space vector set, a second pseudo-label set, and a second pseudo latent space vector set; and input the low-light image dataset into a student network for image prediction to obtain a first predicted image set, a first latent space vector set, a second predicted image set, and a second latent space vector set;

[0017] a basis vector sequence acquisition module, configured to construct a vector sequence based on the first pseudo latent space vector set and the first latent space vector set, according to the first pseudo label and the first predicted image set, by using a pseudo label adaptation module to obtain an optimized vector sequence; and select representative vectors based on the optimized vector sequence by using a principal component analysis method to obtain a basis vector sequence;

[0018] a posterior distribution calculation module, configured to calculate the latent space vector distribution of unlabeled data based on the basis vector sequence, the second pseudo latent space vector set, and the second latent space vector set based on a Gaussian regression method to obtain a posterior distribution;

[0019] an overall loss calculation module, configured to calculate a loss function based on the labeled data set, the second pseudo label set, the first predicted image set, the second predicted image set, the second pseudo latent space vector set, and the posterior distribution to obtain an overall loss;

[0020] The model optimization module is used to update the parameters of the student network using the Adam optimizer based on the overall loss to obtain an optimized student network; based on the optimized student network, the teacher network is updated using the exponential moving average method to obtain an optimized teacher network;

[0021] The low-light image enhancement module is used to obtain the low-light image to be enhanced; based on the optimized student network and the optimized teacher network, the low-light image to be enhanced is enhanced to obtain an enhanced image.

[0022] On the other hand, a semi-supervised low-light image enhancement device is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned semi-supervised low-light image enhancement methods based on the latent space mean teacher network is implemented.

[0023] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned semi-supervised low-light image enhancement methods based on the latent space mean teacher network.

[0024] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0025] This paper proposes a semi-supervised low-light image enhancement method based on a latent space mean teacher network. This method uses a semi-supervised learning approach and utilizes both labeled and unlabeled data for training, effectively alleviating the difficulty of obtaining labeled data and improving data utilization efficiency. By combining the latent space mean teacher with a Gaussian process, it can better mine the data's latent space features, reduce domain shift, and enhance the model's generalization ability, enabling it to perform well in different lighting scenarios. Furthermore, by combining multiple loss functions, it measures and optimizes differences across multiple dimensions, such as reconstruction, structural similarity, and perceptual features, resulting in significant low-light image enhancement, higher output image quality, and better visual effects. This paper proposes a low-light image enhancement method based on semi-supervision, with excellent enhancement effects and strong generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 This is a flow chart of a semi-supervised low-light image enhancement method based on a latent space mean teacher network provided by an embodiment of the present invention;

[0028] Figure 2 This is a block diagram of a semi-supervised low-light image enhancement device based on a latent space mean teacher network provided by an embodiment of the present invention;

[0029] Figure 3 4 is a schematic structural diagram of a semi-supervised low-light image enhancement device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0031] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0032] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0033] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0034] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0035] The embodiment of the present invention provides a semi-supervised low-light image enhancement method based on a latent space mean teacher network. The method can be implemented by a semi-supervised low-light image enhancement device, which can be a terminal or a server. Figure 1 The flowchart of the semi-supervised low-light image enhancement method based on the latent space mean teacher network is shown. The processing flow of the method may include the following steps:

[0036] S1. Obtain a low-light image dataset.

[0037] Among them, the low-light image dataset includes a labeled dataset and an unlabeled dataset; the labeled dataset includes a labeled low-light image set and a labeled normal-light image set; the unlabeled dataset is an unlabeled low-light image set.

[0038] In a feasible embodiment, in the present invention, the low-light image dataset includes a labeled dataset and an unlabeled dataset; the labeled dataset and unlabeled datasets As shown in the following formulas (1) and (2):

[0039] (1);

[0040] (2);

[0041] in, A collection of low-light images representing markers; represents a set of normal light images; represents a collection of unlabeled low-light images; Indicates the first low-light images; Represents The corresponding normal light image; Indicates the first A low-light image.

[0042] S2. Input the low-light image dataset into the teacher network for pseudo-label generation to obtain a first pseudo-label set, a first pseudo latent space vector set, a second pseudo-label set, and a second pseudo latent space vector set; input the low-light image dataset into the student network for image prediction to obtain a first predicted image set, a first latent space vector set, a second predicted image set, and a second latent space vector set.

[0043] Optionally, inputting the low-light image dataset into the teacher network to generate pseudo labels to obtain a first pseudo label set, a first pseudo latent space vector set, a second pseudo label set, and a second pseudo latent space vector set, including:

[0044] Input the labeled data set into the encoder of the teacher network to extract the intermediate vectors to obtain a first intermediate vector set; perform latent space mapping on the first intermediate vector set to obtain a first pseudo latent space vector set;

[0045] Inputting the first intermediate vector set and the first pseudo latent space vector set into the decoder of the teacher network to generate pseudo labels, thereby obtaining a first pseudo label set;

[0046] Input the unlabeled data set into the encoder of the teacher network to extract the intermediate vectors to obtain a second intermediate vector set; perform latent space mapping on the second intermediate vector set to obtain a second pseudo latent space vector set;

[0047] The second intermediate vector set and the second pseudo latent space vector set are input into the decoder of the teacher network for pseudo label generation to obtain a second pseudo labeler.

[0048] In one possible implementation, the labeled image , and input it into the teacher network. The encoder of the teacher network processes the image, generates intermediate features, and then obtains the pseudo latent space vector The decoder uses these intermediate features and pseudo latent space vectors to generate Scale pseudo labels . After entering the teacher network, the encoder of the teacher network generates intermediate features, thereby obtaining a pseudo latent space vector , the decoder then uses these to generate pseudo labels .

[0049] Optionally, inputting the low-light image dataset into the student network for image prediction to obtain a first predicted image set, a first latent space vector set, a second predicted image set, and a second latent space vector set, including:

[0050] Input the labeled data set into the encoder of the student network to extract the intermediate vectors to obtain a third intermediate vector set; perform latent space mapping on the third intermediate vector set to obtain a first latent space vector set;

[0051] Inputting the third intermediate vector set and the first latent space vector set into the decoder of the student network for image prediction to obtain a first predicted image set;

[0052] Input the unlabeled data set into the encoder of the student network to extract the intermediate vectors to obtain a fourth intermediate vector set; perform latent space mapping on the fourth intermediate vector set to obtain a second latent space vector set;

[0053] The fourth intermediate vector set and the second latent space vector set are input into the decoder of the student network for image prediction to obtain a second predicted image set.

[0054] In one possible implementation, the image is marked is also input into the student network. The encoder of the student network generates intermediate features and latent space vectors , the decoder generates the corresponding Scaled output image ,in, Unlabeled images When input into the student network, the encoder of the student network generates intermediate features, thereby producing a latent space vector , the decoder generates the output image based on these .

[0055] S3. Based on the first pseudo latent space vector set and the first latent space vector set, according to the first pseudo label and the first predicted image set, a vector sequence is constructed through the pseudo label adaptation module to obtain an optimized vector sequence; based on the optimized vector sequence, representative vectors are selected through the principal component analysis method to obtain a basis vector sequence.

[0056] Optionally, based on the first pseudo latent space vector set and the first latent space vector set, according to the first pseudo label and the first predicted image set, constructing a vector sequence by a pseudo label adaptation module to obtain an optimized vector sequence includes:

[0057] Based on the labeled data set, performing peak signal-to-noise ratio calculation according to the first pseudo label set and the first predicted image set to obtain a first metric score set and a second metric score set;

[0058] Comparing the metric scores according to the first metric score set and the second metric score set to obtain a comparison result set;

[0059] Based on the first pseudo latent space vector set and the first latent space vector set, a vector sequence is constructed according to the comparison result set to obtain an optimized vector sequence.

[0060] In one feasible implementation, the present invention introduces a pseudo label adaptation module (PAM) to select a latent space vector that can help guide the learning of the student network. According to the Peak Signal-to-Noise Ratio (PSNR) metric score, the output of the student network and the teacher network on the labeled data is and Select the latent space vector in . Optimize the vector sequence As shown in formula (3):

[0061] (3);

[0062] in, Is a selection function that determines the selection based on the comparison of PSNR metric scores still ; Represents a vector sequence The i-th latent space vector in .

[0063] S4. Based on the Gaussian regression method, the latent space vector distribution of the unlabeled data is calculated according to the basis vector sequence, the second pseudo latent space vector set and the second latent space vector set to obtain the posterior distribution.

[0064] Optionally, based on a Gaussian regression method, calculating the latent space vector distribution of the unlabeled data according to the basis vector sequence, the second pseudo latent space vector set, and the second latent space vector set to obtain a posterior distribution includes:

[0065] Performing vector mean calculation based on the basis vector sequence to obtain a first mean; performing vector mean calculation based on the second pseudo latent space vector set and the second latent space vector set to obtain a second mean;

[0066] constructing a joint model of the latent space vector distribution using a Gaussian regression method according to the first mean, the second mean, the second pseudo latent space vector set, and the second latent space vector set to obtain a vector distribution model;

[0067] The posterior distribution of the unlabeled latent space vectors is obtained by calculating the vector distribution model according to the second pseudo latent space vector set, the second latent space vector set and the basis vector sequence.

[0068] In a feasible embodiment, the present invention uses principal component analysis (PCA) to optimize the vector sequence Select the most representative Basis vectors , these vectors form , is determined as a basis vector sequence. Assume that the latent space vector of the unlabeled data is yes The linear combination of the midpoint vector plus noise interference , Represents Gaussian distribution modeling, latent space vector It can be expressed as the following formula (4):

[0069] (4);

[0070] in, represents the coefficient; Indicates that the first The latent space vector obtained by inputting the image into the student network; Indicates the representative vectors; N is the upper limit of the number of vectors used to represent the vector combination.

[0071] The distribution of follows Gaussian Process (GP), The distribution of also follows GP. Gaussian Process Regression (GPR) is used to jointly model the latent space vector distribution of labeled and unlabeled data. The distribution is expressed as follows (5):

[0072] (5);

[0073] in, is the kernel function, defined as ; is a sequence of vectors The vector set consisting of the most representative n vectors selected by PCA The vector in ; is the set of latent space vectors of unlabeled data; express The mean of express The mean of represents the noise variance; Represents the identity matrix.

[0074] According to the mathematical model of the vector distribution of the joint distribution, the posterior distribution of the unlabeled latent space vector is calculated, wherein the posterior distribution of the unlabeled latent space vector is calculated according to the joint distribution. ,in, and As shown in the following formulas (6) and (7):

[0075] (6);

[0076] (7);

[0077] in, Represents the latent space vector of unlabeled data The mean predicted value under the current model; Represents the latent space vector of unlabeled data The calculation results related to the covariance matrix under the current model.

[0078] S5. Calculate the loss function based on the labeled data set, the second pseudo label set, the first predicted image set, the second predicted image set, the second pseudo latent space vector set, and the posterior distribution to obtain an overall loss.

[0079] Optionally, a loss function is calculated based on the labeled dataset, the second pseudo label set, the first predicted image set, the second predicted image set, the second pseudo latent space vector set, and the posterior distribution to obtain an overall loss, including:

[0080] The loss function is calculated based on the labeled dataset and the first predicted image to obtain the labeling loss; the labeling loss includes reconstruction loss, negative SSIM loss and perceptual loss;

[0081] Calculate the loss function based on the second pseudo label set and the second predicted image set to obtain an unlabeled loss;

[0082] Calculate the loss function based on the posterior distribution and the second pseudo latent space vector set to obtain the auxiliary learning loss function;

[0083] The overall loss is obtained by weighted calculation based on auxiliary learning loss, labeled loss and unlabeled loss.

[0084] In a feasible implementation, in order to constrain the learning process, the present invention defines an auxiliary learning loss function of the auxiliary GPR: As shown in formula (8):

[0085] (8);

[0086] For the convenience of calculation, the auxiliary GPR loss Variants are As shown in formula (9):

[0087] (9);

[0088] in, is the pseudo latent space vector generated by the teacher network for the mth image in the unlabeled data; is the unlabeled data latent space vector An estimate of some mean under the current model.

[0089] Overall loss By marking loss , unlabeled loss and auxiliary GPR loss Variant composition, overall loss It can be expressed as the following formula (10):

[0090] (10);

[0091] in, and are parameters updated using an exponential annealing function during training, with final values of 0.2 and 0.01 respectively.

[0092] Labeling loss Losses from reconstruction , Negative SSIM loss (Negative Structural Similarity Index Loss, Negative SSIM Loss) and perceptual loss The losses can be expressed as follows (11), (12), (13), (14):

[0093] (11);

[0094] (12);

[0095] (13);

[0096] (14);

[0097] in, and is the weighting parameter; is the weighting parameter for the scale; It is the j-th scale image of the same size as the output obtained by bilinear interpolation of the original true value (GroundTruth, GT); is the j-th scale output image of the labeled data generated by the student network, which is the result of the model prediction; SSIM is an indicator used to measure the structural similarity of two images; Represents a pre-trained VGG-16 network.

[0098] Unlabeled loss It can be expressed as the following formula (15):

[0099] (15);

[0100] in, is the j-th scale pseudo label of the unlabeled data generated by the teacher network, which is used to guide the learning of the student network.

[0101] S6. Based on the overall loss, use the Adam optimizer to update the parameters of the student network to obtain the optimized student network; based on the optimized student network, use the exponential moving average method to update the parameters of the teacher network to obtain the optimized teacher network.

[0102] In a feasible implementation, the total loss calculated is , use Adam optimizer to update the weights of the student network The parameters of the Adam optimizer are set to , , The initial learning rate is After each training cycle, the weights of the teacher network are updated using exponential moving average , the update formula is ,momentum .

[0103] S7. Obtain a low-light image to be enhanced; based on the optimized student network and the optimized teacher network, perform image enhancement on the low-light image to be enhanced to obtain an enhanced image.

[0104] In a feasible implementation, the effectiveness of the present invention is verified by comparing the method of the present invention with 15 state-of-the-art LLIE methods, including traditional methods (Local Interpretable Model-agnostic Explanations, LIME), supervised learning methods, unsupervised learning methods, zero-shot methods, and semi-supervised learning methods.

[0105] The proposed method was trained using randomly cropped image patches of size 256×256. Unlabeled data was synthesized using Adobe Photoshop Lightroom software. From the labeled normal light images, 5,000 low-light images were synthesized by setting the exposure parameters to [-5, 0], the contrast parameters to [-100, 100], and the vividness parameters to [-100, 0]. The Adam optimizer was used for training, with the parameters set to , , , the initial learning rate is , train for 600 epochs. The weighting parameters are set based on experience, such as n=16, , , , , , , and updated using the exponential annealing function during training epoch, the final state is , During testing, the input image was symmetrically padded to a multiple of 32×32 and then cropped back to its original size after inference. All of the above tests were performed on a PC with a single RTX 4090 GPU. The comparison results are shown in Table 1 (Image Enhancement Comparison Table).

[0106] Table 1

[0107]

[0108] The present invention points out the effectiveness of the method from 6 indicators and compares it with the semi-supervised method, the zero-sample method, and the fully supervised method. The method of the present invention obtains the best indicator results.

[0109] This paper proposes a semi-supervised low-light image enhancement method based on a latent space mean teacher network. This method uses a semi-supervised learning approach and utilizes both labeled and unlabeled data for training, effectively alleviating the difficulty of obtaining labeled data and improving data utilization efficiency. By combining the latent space mean teacher with a Gaussian process, it can better mine the data's latent space features, reduce domain shift, and enhance the model's generalization ability, enabling it to perform well in different lighting scenarios. Furthermore, by combining multiple loss functions, it measures and optimizes differences across multiple dimensions, such as reconstruction, structural similarity, and perceptual features, resulting in significant low-light image enhancement, higher output image quality, and better visual effects. This paper proposes a low-light image enhancement method based on semi-supervision, with excellent enhancement effects and strong generalization capabilities.

[0110] Figure 2This is a block diagram of a semi-supervised low-light image enhancement device based on a latent space mean teacher network according to an exemplary embodiment. The device is used in a semi-supervised low-light image enhancement method based on a latent space mean teacher network. Figure 2 The device includes a data acquisition module 210, a semi-supervised image prediction module 220, a basis vector sequence acquisition module 230, a posterior distribution calculation module 240, an overall loss calculation module 250, a model optimization module 260, and a low-light image enhancement module 270. Among them:

[0111] A data acquisition module 210 is configured to acquire a low-light image dataset;

[0112] The semi-supervised image prediction module 220 is configured to input the low-light image dataset into the teacher network for pseudo-label generation to obtain a first pseudo-label set, a first pseudo latent space vector set, a second pseudo-label set, and a second pseudo latent space vector set; and input the low-light image dataset into the student network for image prediction to obtain a first predicted image set, a first latent space vector set, a second predicted image set, and a second latent space vector set.

[0113] A basis vector sequence acquisition module 230 is configured to construct a vector sequence based on the first pseudo latent space vector set and the first latent space vector set, according to the first pseudo label and the first predicted image set, using a pseudo label adaptation module to obtain an optimized vector sequence; and select representative vectors based on the optimized vector sequence using a principal component analysis method to obtain a basis vector sequence;

[0114] a posterior distribution calculation module 240 for calculating the latent space vector distribution of the unlabeled data based on the basis vector sequence, the second pseudo latent space vector set, and the second latent space vector set based on a Gaussian regression method to obtain a posterior distribution;

[0115] an overall loss calculation module 250 for calculating a loss function based on the labeled dataset, the second pseudo label set, the first predicted image set, the second predicted image set, the second pseudo latent space vector set, and the posterior distribution to obtain an overall loss;

[0116] The model optimization module 260 is used to update the parameters of the student network using the Adam optimizer according to the overall loss to obtain an optimized student network; based on the optimized student network, the teacher network is updated using the exponential moving average method to obtain an optimized teacher network;

[0117] The low-light image enhancement module 270 is used to obtain a low-light image to be enhanced; based on the optimized student network and the optimized teacher network, the low-light image to be enhanced is enhanced to obtain an enhanced image.

[0118] The feature of the invention is that the low-light image dataset includes a labeled dataset and an unlabeled dataset; the labeled dataset includes a labeled low-light image set and a labeled normal-light image set; the unlabeled dataset is an unlabeled low-light image set.

[0119] Optionally, the semi-supervised image prediction module 220 is further configured to:

[0120] Input the labeled data set into the encoder of the teacher network to extract the intermediate vectors to obtain a first intermediate vector set; perform latent space mapping on the first intermediate vector set to obtain a first pseudo latent space vector set;

[0121] Inputting the first intermediate vector set and the first pseudo latent space vector set into the decoder of the teacher network to generate pseudo labels, thereby obtaining a first pseudo label set;

[0122] The unlabeled data set is input into the encoder of the teacher network to extract the intermediate vectors to obtain a second intermediate vector set; the second intermediate vector set is subjected to latent space mapping to obtain a second pseudo latent space vector set;

[0123] The second intermediate vector set and the second pseudo latent space vector set are input into the decoder of the teacher network for pseudo label generation to obtain a second pseudo labeler.

[0124] Optionally, the semi-supervised image prediction module 220 is further configured to:

[0125] Input the labeled data set into the encoder of the student network to extract the intermediate vectors to obtain a third intermediate vector set; perform latent space mapping on the third intermediate vector set to obtain a first latent space vector set;

[0126] Inputting the third intermediate vector set and the first latent space vector set into the decoder of the student network for image prediction to obtain a first predicted image set;

[0127] Input the unlabeled data set into the encoder of the student network to extract the intermediate vectors to obtain a fourth intermediate vector set; perform latent space mapping on the fourth intermediate vector set to obtain a second latent space vector set;

[0128] The fourth intermediate vector set and the second latent space vector set are input into the decoder of the student network for image prediction to obtain a second predicted image set.

[0129] Optionally, the basis vector sequence acquisition module 230 is further configured to:

[0130] Based on the labeled data set, performing peak signal-to-noise ratio calculation according to the first pseudo label set and the first predicted image set to obtain a first metric score set and a second metric score set;

[0131] Comparing the metric scores according to the first metric score set and the second metric score set to obtain a comparison result set;

[0132] Based on the first pseudo latent space vector set and the first latent space vector set, a vector sequence is constructed according to the comparison result set to obtain an optimized vector sequence.

[0133] Optionally, the posterior distribution calculation module 240 is further configured to:

[0134] Performing vector mean calculation based on the basis vector sequence to obtain a first mean; performing vector mean calculation based on the second pseudo latent space vector set and the second latent space vector set to obtain a second mean;

[0135] constructing a joint model of the latent space vector distribution using a Gaussian regression method according to the first mean, the second mean, the second pseudo latent space vector set, and the second latent space vector set to obtain a vector distribution model;

[0136] The posterior distribution of the unlabeled latent space vectors is obtained by calculating the vector distribution model according to the second pseudo latent space vector set, the second latent space vector set and the basis vector sequence.

[0137] Optionally, the overall loss calculation module 250 is further configured to:

[0138] The loss function is calculated based on the labeled dataset and the first predicted image to obtain the labeling loss; the labeling loss includes reconstruction loss, negative SSIM loss and perceptual loss;

[0139] Calculate the loss function based on the second pseudo label set and the second predicted image set to obtain an unlabeled loss;

[0140] Calculate the loss function based on the posterior distribution and the second pseudo latent space vector set to obtain the auxiliary learning loss function;

[0141] The overall loss is obtained by weighted calculation based on auxiliary learning loss, labeled loss and unlabeled loss.

[0142] This paper proposes a semi-supervised low-light image enhancement method based on a latent space mean teacher network. This method uses a semi-supervised learning approach and utilizes both labeled and unlabeled data for training, effectively alleviating the difficulty of obtaining labeled data and improving data utilization efficiency. By combining the latent space mean teacher with a Gaussian process, it can better mine the data's latent space features, reduce domain shift, and enhance the model's generalization ability, enabling it to perform well in different lighting scenarios. Furthermore, by combining multiple loss functions, it measures and optimizes differences across multiple dimensions, such as reconstruction, structural similarity, and perceptual features, resulting in significant low-light image enhancement, higher output image quality, and better visual effects. This paper proposes a low-light image enhancement method based on semi-supervision, with excellent enhancement effects and strong generalization capabilities.

[0143] Figure 3 FIG. 1 is a structural diagram of a semi-supervised low-light image enhancement device provided by an embodiment of the present invention. Figure 3 As shown, the semi-supervised low-light image enhancement device may include the above Figure 2 The semi-supervised low-light image enhancement apparatus based on the latent space mean teacher network is shown. Optionally, the semi-supervised low-light image enhancement apparatus 310 may include a first processor 2001.

[0144] Optionally, the semi-supervised low-light image enhancement device 310 may further include a memory 2002 and a transceiver 2003 .

[0145] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0146] The following combination Figure 3 The components of the semi-supervised low-light image enhancement device 310 are described in detail:

[0147] The first processor 2001 is the control center of the semi-supervised low-light image enhancement device 310 and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0148] Optionally, the first processor 2001 may perform various functions of the semi-supervised low-light image enhancement device 310 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0149] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 3 CPU0 and CPU1 are shown in FIG.

[0150] In a specific implementation, as an embodiment, the semi-supervised low-light image enhancement device 310 may also include multiple processors, such as Figure 3 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0151] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0152] Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and accessed through the interface circuit ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0153] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0154] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 3 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0155] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and communicate with the first processor 2001 through the interface circuit of the semi-supervised low-light image enhancement device 310 ( Figure 3 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0156] It should be noted that Figure 3 The structure of the semi-supervised low-light image enhancement device 310 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0157] In addition, the technical effects of the semi-supervised low-light image enhancement device 310 can refer to the technical effects of the semi-supervised low-light image enhancement method based on the latent space mean teacher network described in the above method embodiment, and will not be repeated here.

[0158] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0159] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0160] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0161] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0162] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0163] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0164] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0165] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0166] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0167] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0168] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0169] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0170] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A semi-supervised low-light image enhancement method based on latent space mean teacher network, characterized in that The method comprises: Obtain a low-light image dataset; The low-light image dataset is input into the teacher network for pseudo label generation to obtain a first pseudo label set, a first pseudo latent space vector set, a second pseudo label set, and a second pseudo latent space vector set; the low-light image dataset is input into the student network for image prediction to obtain a first predicted image set, a first latent space vector set, a second predicted image set, and a second latent space vector set; Based on the first pseudo latent space vector set and the first latent space vector set, according to the first pseudo label and the first predicted image set, a vector sequence is constructed by a pseudo label adaptation module to obtain an optimized vector sequence; based on the optimized vector sequence, representative vectors are selected by principal component analysis to obtain a basis vector sequence; Based on the Gaussian regression method, the latent space vector distribution of the unlabeled data is calculated according to the basis vector sequence, the second pseudo latent space vector set and the second latent space vector set to obtain the posterior distribution; A loss function is calculated based on the labeled data set, the second pseudo label set, the first predicted image set, the second predicted image set, the second pseudo latent space vector set, and the posterior distribution to obtain an overall loss; According to the overall loss, the Adam optimizer is used to update the parameters of the student network to obtain the optimized student network; based on the optimized student network, the exponential moving average method is used to update the parameters of the teacher network to obtain the optimized teacher network; Obtain a low-light image to be enhanced; perform image enhancement on the low-light image to be enhanced based on the optimized student network and the optimized teacher network to obtain an enhanced image.

2. The semi-supervised low-light image enhancement method based on latent space mean teacher network according to claim 1, characterized in that The low-light image dataset includes a labeled dataset and an unlabeled dataset; the labeled dataset includes a labeled low-light image set and a labeled normal-light image set; and the unlabeled dataset is an unlabeled low-light image set.

3. The semi-supervised low-light image enhancement method based on latent space mean teacher network according to claim 2, characterized in that The low-light image dataset is input into the teacher network to generate pseudo labels to obtain a first pseudo label set, a first pseudo latent space vector set, a second pseudo label set, and a second pseudo latent space vector set, including: Input the labeled data set into the encoder of the teacher network to extract the intermediate vectors to obtain a first intermediate vector set; perform latent space mapping on the first intermediate vector set to obtain a first pseudo latent space vector set; Inputting the first intermediate vector set and the first pseudo latent space vector set into the decoder of the teacher network to generate pseudo labels, thereby obtaining a first pseudo label set; Input the unlabeled data set into the encoder of the teacher network to extract the intermediate vectors to obtain a second intermediate vector set; perform latent space mapping on the second intermediate vector set to obtain a second pseudo latent space vector set; The second intermediate vector set and the second pseudo latent space vector set are input into the decoder of the teacher network for pseudo label generation to obtain a second pseudo labeler.

4. The semi-supervised low-light image enhancement method based on latent space mean teacher network according to claim 2, characterized in that The low-light image dataset is input into the student network for image prediction to obtain a first predicted image set, a first latent space vector set, a second predicted image set, and a second latent space vector set, including: Input the labeled data set into the encoder of the student network to extract the intermediate vectors to obtain a third intermediate vector set; perform latent space mapping on the third intermediate vector set to obtain a first latent space vector set; Inputting the third intermediate vector set and the first latent space vector set into the decoder of the student network for image prediction to obtain a first predicted image set; Input the unlabeled data set into the encoder of the student network to extract the intermediate vectors to obtain a fourth intermediate vector set; perform latent space mapping on the fourth intermediate vector set to obtain a second latent space vector set; The fourth intermediate vector set and the second latent space vector set are input into the decoder of the student network for image prediction to obtain a second predicted image set.

5. The semi-supervised low-light image enhancement method based on latent space mean teacher network according to claim 1, characterized in that The method of constructing a vector sequence based on the first pseudo latent space vector set and the first latent space vector set, according to the first pseudo label and the first predicted image set, by a pseudo label adaptation module to obtain an optimized vector sequence includes: Based on the labeled data set, performing peak signal-to-noise ratio calculation according to the first pseudo label set and the first predicted image set to obtain a first metric score set and a second metric score set; Comparing the metric scores according to the first metric score set and the second metric score set to obtain a comparison result set; Based on the first pseudo latent space vector set and the first latent space vector set, a vector sequence is constructed according to the comparison result set to obtain an optimized vector sequence.

6. The semi-supervised low-light image enhancement method based on latent space mean teacher network according to claim 1, characterized in that The Gaussian regression method is based on calculating the latent space vector distribution of the unlabeled data according to the basis vector sequence, the second pseudo latent space vector set and the second latent space vector set to obtain the posterior distribution, including: Performing vector mean calculation based on the basis vector sequence to obtain a first mean; performing vector mean calculation based on the second pseudo latent space vector set and the second latent space vector set to obtain a second mean; constructing a joint model of the latent space vector distribution using a Gaussian regression method according to the first mean, the second mean, the second pseudo latent space vector set, and the second latent space vector set to obtain a vector distribution model; The posterior distribution of the unlabeled latent space vectors is obtained by calculating the vector distribution model according to the second pseudo latent space vector set, the second latent space vector set and the basis vector sequence.

7. The semi-supervised low-light image enhancement method based on latent space mean teacher network according to claim 1, characterized in that The loss function is calculated based on the labeled data set, the second pseudo label set, the first predicted image set, the second predicted image set, the second pseudo latent space vector set, and the posterior distribution to obtain the overall loss, including: Calculating a loss function based on the labeled data set and the first predicted image to obtain a labeled loss; the labeled loss includes a reconstruction loss, a negative SSIM loss, and a perceptual loss; Calculate the loss function based on the second pseudo label set and the second predicted image set to obtain an unlabeled loss; Calculate the loss function based on the posterior distribution and the second pseudo latent space vector set to obtain the auxiliary learning loss function; The overall loss is obtained by weighted calculation based on auxiliary learning loss, labeled loss and unlabeled loss.

8. A semi-supervised low-light image enhancement device based on a latent space mean teacher network, wherein the semi-supervised low-light image enhancement device based on a latent space mean teacher network is used to implement the semi-supervised low-light image enhancement method based on a latent space mean teacher network according to any one of claims 1 to 7, characterized in that: The device comprises: A data acquisition module, used to acquire a low-light image dataset; A semi-supervised image prediction module is configured to input a low-light image dataset into a teacher network for pseudo-label generation to obtain a first pseudo-label set, a first pseudo latent space vector set, a second pseudo-label set, and a second pseudo latent space vector set; and input the low-light image dataset into a student network for image prediction to obtain a first predicted image set, a first latent space vector set, a second predicted image set, and a second latent space vector set; a basis vector sequence acquisition module, configured to construct a vector sequence based on the first pseudo latent space vector set and the first latent space vector set, according to the first pseudo label and the first predicted image set, by using a pseudo label adaptation module to obtain an optimized vector sequence; and select representative vectors based on the optimized vector sequence by using a principal component analysis method to obtain a basis vector sequence; a posterior distribution calculation module, configured to calculate the latent space vector distribution of unlabeled data based on the basis vector sequence, the second pseudo latent space vector set, and the second latent space vector set based on a Gaussian regression method to obtain a posterior distribution; an overall loss calculation module, configured to calculate a loss function based on the labeled data set, the second pseudo label set, the first predicted image set, the second predicted image set, the second pseudo latent space vector set, and the posterior distribution to obtain an overall loss; The model optimization module is used to update the parameters of the student network using the Adam optimizer based on the overall loss to obtain an optimized student network; based on the optimized student network, the teacher network is updated using the exponential moving average method to obtain an optimized teacher network; The low-light image enhancement module is used to obtain the low-light image to be enhanced; based on the optimized student network and the optimized teacher network, the low-light image to be enhanced is enhanced to obtain an enhanced image.

9. A semi-supervised low-light image enhancement device, characterized in that The semi-supervised low-light image enhancement device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images