Image processing method, electronic device and chip

Abstract

The application discloses an image processing method, an electronic device and a chip. The image processing method comprises the following steps: training an initial enhancement model by using a dark-light image data set and an enhanced image data set, and obtaining an image enhancement model; wherein the initial enhancement model is used for extracting at least one image feature based on decoupling representation, and performing image enhancement processing according to the at least one image feature; and performing enhancement processing on a to-be-processed dark-light image by using the image enhancement model, and obtaining an enhanced image.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to an image processing method, electronic device, and chip. Background Technology

[0002] With the widespread adoption and development of smartphones, the demands on mobile phone photography are also increasing. However, in low-light or night scenes, mobile phone images are often too dark, failing to clearly reveal details and resulting in low image quality. To address the issue of low image quality in night scenes, image enhancement is necessary.

[0003] Common deep learning-based image enhancement methods often require a data-driven approach. These methods train convolutional neural networks on large-scale paired datasets to learn the mapping relationship between low-light night scene images and high-quality enhanced images. These methods have the following main drawbacks: acquiring a paired dataset is very costly and complex; using datasets from fixed scenes results in networks with poor robustness and generalization, often leading to poor enhancement performance in new scenes.

[0004] It is evident that current methods for enhancing low-light images generally suffer from problems such as being time-consuming, labor-intensive, highly complex, and producing poor processing results. Summary of the Invention

[0005] This application provides an image processing method, electronic device, and chip that can meet the application scenarios with high real-time requirements, thereby improving the efficiency and effect of image processing.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0008] An initial enhancement model is trained using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on decoupled representation, and to perform image enhancement processing based on the at least one image feature;

[0009] The image enhancement model is used to enhance the low-light image to be processed, resulting in an enhanced image.

[0010] Secondly, embodiments of this application provide an electronic device, which includes: a training unit and a processing unit.

[0011] The training unit is used to train an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on decoupled representation, and to perform image enhancement processing based on the at least one image feature;

[0012] The processing unit is used to enhance the low-light image to be processed using the image enhancement model to obtain the enhanced image.

[0013] Thirdly, embodiments of this application provide an electronic device, which includes a processor and a memory storing processor-executable instructions. When the instructions are executed by the processor, the image processing method described above is implemented.

[0014] Fourthly, embodiments of this application provide a chip, which includes programmable logic circuits and / or program instructions, and when the chip is running, it implements the image processing method described above.

[0015] This application provides an image processing method, electronic device, and chip. The electronic device trains an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model. The initial enhancement model is used to extract at least one image feature based on decoupled representations and perform image enhancement processing based on these features. The image enhancement model then enhances the low-light image to be processed, obtaining an enhanced image. Therefore, in this application, the image enhancement model used for enhancing low-light images is trained based on decoupled representations including content features and / or low-light features. Thus, the low-light image dataset and the enhanced image dataset used during training do not require a mapping relationship, thereby reducing the cost and complexity of model training. Furthermore, the introduction of decoupled representations improves the robustness and generalization of the network model, thereby enhancing the image processing effect. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a deep learning-based image enhancement method.

[0017] Figure 2 This is a diagram illustrating the implementation structure of a method that combines supervised and unsupervised low-light image enhancement techniques.

[0018] Figure 3 This is a schematic diagram illustrating the implementation process of the image processing method.

[0019] Figure 4 This is a schematic diagram of the initial enhancement model;

[0020] Figure 5 This is a schematic diagram of the loss function;

[0021] Figure 6 This is a schematic diagram of the structure of an image enhancement model;

[0022] Figure 7 This is a schematic diagram of image enhancement processing;

[0023] Figure 8 This is a schematic diagram of model training.

[0024] Figure 9 This is a schematic diagram of a low-light enhancement image dataset.

[0025] Figure 10 This is a schematic diagram of the training process for the initial augmentation model;

[0026] Figure 11 This is a schematic diagram illustrating the calculation process of the loss function;

[0027] Figure 12 This is a schematic diagram illustrating image enhancement processing using an image enhancement model.

[0028] Figure 13 Schematic diagram of the composition structure of an electronic device Figure 1 ;

[0029] Figure 14 Schematic diagram of the composition structure of an electronic device Figure 2 . Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the relevant application and not for limiting the application. Furthermore, it should be noted that, for ease of description, only the parts related to the relevant application are shown in the accompanying drawings.

[0031] With the continuous development of technology, electronic devices such as mobile phones and tablets are becoming increasingly comprehensive in their functions. Among them, the image processing function of electronic devices plays an important role in people's daily work and life. For example, with the popularization and development of smartphones, people are more and more willing to use their phones to take pictures, and their requirements for mobile phone photography are also getting higher and higher. However, in low light or night scene scenes, mobile phone images are too dark, and details in the image cannot be clearly presented, resulting in low image quality.

[0032] To address the issue of low-quality night scene imaging on mobile phones, image enhancement is necessary. With the development of deep learning, a classic image enhancement technique is to use convolutional neural networks for image enhancement. Figure 1 This is a schematic diagram of a deep learning-based image enhancement method, such as... Figure 1As shown, after acquiring RAW image data from the sensor, it is rearranged and then input into a convolutional neural network (ConvNet). Deep learning-based methods require acquiring a training dataset, typically consisting of one-to-one correspondences between low-light sample images and low-light enhanced images. Supervised training is then performed, applying L1 loss to both the low-light enhanced images and the network-reconstructed images to gradually converge the network. Ultimately, a trained convolutional neural network is obtained, which, when given a low-light image, directly outputs a low-light enhanced, low-noise image. For example, a common method uses U-Net as the convolutional neural network to reconstruct low-light images, obtaining enhanced night scene images.

[0033] While supervised training can produce enhanced night scene images, image details are not well preserved, and the perceptual quality still needs improvement. To address this issue, a two-stage enhancement process can be employed. Figure 2 The diagram shows the implementation structure of a method combining supervised and unsupervised low-light image enhancement. Figure 2 As shown, given a night scene image as input, this method first uses a supervised network to obtain a supervised augmented image, and then uses an unsupervised network to further enhance it, resulting in an augmented image with better perceptual quality.

[0034] However, common deep learning-based image enhancement methods often require a data-driven approach. These methods train convolutional neural networks on large-scale paired datasets to learn the mapping relationship between low-light night scene images and high-quality enhanced images. While much more efficient than traditional image enhancement processes, these methods still have some drawbacks. First, existing methods require training with one-to-one paired low-light and enhanced images. Acquiring these paired images requires specialized equipment and scene preparation, and acquiring such datasets is very time-consuming, sometimes even introducing motion blur due to object movement. Therefore, obtaining a paired dataset is very costly and complex. Second, because the acquired datasets are one-to-one paired datasets from fixed scenes, the robustness and generalization of the trained network are weak. This network often performs poorly when faced with new scenes. Existing supervised methods typically address this issue by acquiring additional data and retraining. This further increases the time cost and difficulty of model deployment.

[0035] It is evident that current methods for enhancing low-light images generally suffer from problems such as being time-consuming, labor-intensive, complex, and producing poor processing results.

[0036] To address the aforementioned issues, in embodiments of this application, the electronic device trains an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model. The initial enhancement model is used to extract at least one image feature based on decoupled representations and to perform image enhancement processing based on these features. The image enhancement model then enhances the low-light image to be processed, yielding an enhanced image. Therefore, in embodiments of this application, the image enhancement model used for enhancing low-light images is trained based on decoupled representations including content features and / or low-light features. Thus, the low-light image dataset and the enhanced image dataset used during training do not require a mapping relationship, thereby reducing the cost and complexity of model training. Furthermore, the introduction of decoupled representations improves the robustness and generalization of the network model, thereby enhancing the image processing effect.

[0037] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0038] One embodiment of this application provides an image processing method. Figure 3 This is a schematic diagram illustrating the implementation process of an image processing method, such as... Figure 3 As shown in the embodiments of this application, the method for implementing image processing by an electronic device may include the following steps:

[0039] Step 101: Train the initial enhancement model using the low-light image dataset and the enhanced image dataset to obtain the image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on the decoupled representation and perform image enhancement processing based on at least one image feature.

[0040] In embodiments of this application, the electronic device may first train an initial enhancement model using a low-light image dataset and an enhanced image dataset based on a decoupled representation including content features and / or low-light features, to obtain an image enhancement model. The images in the low-light image dataset and the images in the enhanced image dataset may not have a one-to-one correspondence.

[0041] It is understood that, in the embodiments of this application, the electronic device is any terminal device with communication and storage functions, such as: tablet computer, mobile phone, e-reader, remote control, personal computer (PC), laptop computer, in-vehicle device, smart TV, wearable device, personal digital assistant (PDA), portable media player (PMP), navigation device and other terminal devices.

[0042] It should be noted that, in the embodiments of this application, decoupling means separating certain distinguishing factors or features from the data. For example, decoupling aims to separate vectors from an image that control generation factors such as light source, object shape, and surface material.

[0043] Accordingly, in the embodiments of this application, during the model training process based on decoupled representation, the electronic device can choose to separate content features and / or dark light features from the image, so that the content features and dark light features can be used to complete the unsupervised training of the model.

[0044] It should be noted that, in the embodiments of this application, based on Retinex theory, illuminance can be separated to eliminate the influence of uneven illumination and improve image quality. Therefore, in this application, a dark image can be decomposed into the image content itself and a dark mask superimposed on it. Thus, this decomposition process can be simulated using a decoupled representation method that includes content features and / or dark features to extract the content features and dark features in the image.

[0045] It is understood that, in the embodiments of this application, the training dataset in common technologies can generally include unprocessed images and processed images with a one-to-one correspondence. However, the image processing method proposed in the embodiments of this application can utilize decoupled representations to train the initial enhancement model. Therefore, for the training dataset, it is not required that there be a one-to-one mapping relationship between the images in the low-light image dataset and the images in the enhancement image dataset.

[0046] It should be noted that, in the embodiments of this application, the low-light image dataset may include at least one low-light image, wherein the at least one low-light image can be a low-light image of any angle, any environment, and any subject being photographed, and the embodiments of this application do not limit the acquisition method of the at least one low-light image.

[0047] It should be noted that, in the embodiments of this application, the enhanced image dataset may include at least one frame of low-light enhanced image, wherein the at least one frame of low-light enhanced image can be a low-light image of any angle, any environment, and any subject being photographed, and the embodiments of this application do not limit the acquisition method of the at least one frame of low-light enhanced image.

[0048] It is understood that, in the embodiments of this application, there may be a certain correspondence between the first image in the low-light image dataset (i.e., any one or more frames of low-light images) and the second image in the enhanced image dataset (i.e., any one or more frames of low-light enhanced images), or there may be no correspondence between them. That is to say, in this application, a pixel-level one-to-one correspondence is not required between the first image and the second image.

[0049] Therefore, in the embodiments of this application, since only one dark light image dataset and one enhanced image dataset are needed when training the initial enhancement model, and it is not required that there be a one-to-one correspondence between the images in the two datasets at the pixel level, the cost of image acquisition is greatly reduced and the complexity of the image processing process is reduced.

[0050] Furthermore, in embodiments of this application, the initial enhancement model includes a content encoder, a low-light encoder, a content image generator, and a low-light image generator. That is, the initial enhancement model can be composed of a content encoder, a low-light encoder, a content image generator, and a low-light image generator.

[0051] It should be noted that, in the embodiments of this application, the content encoder can be used to extract content features, the low-light encoder can be used to extract low-light features, the content image generator can be used to reconstruct images based on content features, and the low-light image generator can be used to reconstruct images by combining low-light features and content features.

[0052] For example, in some embodiments, for the initial enhancement model, the content encoder can be used to extract the content features of the input image, and then output the content features to the content image generator and the dark light image generator respectively; the dark light encoder can be used to extract the dark light features of the input image, and then output the dark light features to the dark light image generator; the content image generator can perform image generation processing according to the input content features; and the dark light image generator can perform image reconstruction processing according to the input content features and dark light features.

[0053] In other words, in the embodiments of this application, two types of encoders are designed for the initial enhancement model: a content encoder and a dark light encoder. The content encoder can learn to represent the content (content features) of the image itself. This content has domain consistency and is not affected by the dark light mask; that is, the content features of the same pair of clear and dark images should be consistent. The dark light encoder, on the other hand, can learn to represent the dark light mask (dark light features) in the dark light image. This dark light mask is domain-specific; clear images do not possess this feature, only dark light images do.

[0054] Furthermore, in the embodiments of this application, two generators are designed for the initial enhancement model: a content image generator and a dark-light image generator. The content image generator, by inputting domain-consistent content features, can obtain a clear content image with dark light removed. The dark-light image generator, by inputting domain-consistent content features and domain-specific dark-light features, can obtain a dark-light image with dark light added to the content.

[0055] Furthermore, in the embodiments of this application, the image enhancement model includes a trained content encoder and a trained content image generator. That is, the image enhancement model can be composed of a trained content encoder and a trained content image generator.

[0056] It should be noted that, in the embodiments of this application, the trained content encoder can be used to extract content features, and the trained content image generator can be used to reconstruct images based on content features to obtain the corresponding enhanced images.

[0057] In other words, in the embodiments of this application, for an image enhancement model consisting of a trained content encoder and a trained content image generator, for an input dark light image to be processed, the trained content encoder can learn to represent the content (content features) of the image itself, and then input the content to the trained content image generator. Accordingly, the content image generator can obtain a clear content image with the dark light removed by using the content features consistent with the input domain, that is, obtain the corresponding enhanced image.

[0058] Furthermore, in embodiments of this application, when the electronic device trains an initial enhancement model using a low-light image dataset and an enhanced image dataset based on a decoupled representation including content features and / or dark-light features to obtain an image enhancement model, it can first obtain the first content feature and the first dark-light feature corresponding to the first image in the low-light image dataset, and the second content feature corresponding to the second image in the enhanced image dataset, respectively, through a content encoder and a dark-light encoder; then, based on the first content feature, the second content feature, and the first dark-light feature, it performs generation processing through a content image generator and a dark-light image generator to obtain the first enhanced image, the first reconstructed image, the second reconstructed image, and the... First, a low-light image is obtained; then, the third content feature corresponding to the first enhanced image and the fourth content feature and second low-light feature corresponding to the first low-light image are obtained through the content encoder and low-light encoder, respectively; then, based on the third content feature, the fourth content feature and the second low-light feature, the content image generator and the low-light image generator are used to generate the first loop image and the second loop image, respectively; finally, the first reconstructed image, the second reconstructed image, the first enhanced image, the first low-light image, the first loop image and the second loop image can be used to perform unsupervised training on the content encoder and the content image generator to obtain the trained content encoder and the trained content image generator.

[0059] It is understood that, in the embodiments of this application, a one-to-one correspondence between the first image and the second image is not required, that is, there is no mapping relationship between the first image and the second image.

[0060] Furthermore, in the embodiments of this application, when the electronic device obtains the first content feature and the first dark light feature corresponding to the first image in the dark light image dataset and the second content feature corresponding to the second image in the enhanced image dataset through the content encoder and the dark light encoder, respectively, it can extract the first content feature corresponding to the first image through the content encoder; at the same time, it can extract the first dark light feature corresponding to the first image through the dark light encoder; and it can extract the second content feature corresponding to the second image through the content encoder.

[0061] Furthermore, in the embodiments of this application, when the electronic device generates a first enhanced image, a first reconstructed image, a second reconstructed image, and a first dark light image based on the first content feature, the second content feature, and the first dark light feature through a content image generator and a dark light image generator, respectively, it can input the first content feature into the content image generator to obtain the first enhanced image corresponding to the first image; and input the first content feature and the first dark light feature into the dark light image generator to obtain the first reconstructed image corresponding to the first image; simultaneously, it can input the second content feature into the content image generator to obtain the second reconstructed image corresponding to the second image; and input the second content feature and the first dark light feature into the dark light image generator to obtain the first dark light image corresponding to the second image.

[0062] Furthermore, in the embodiments of this application, when the electronic device obtains the third content feature corresponding to the first enhanced image and the fourth content feature and the second dark light feature corresponding to the first dark light image respectively through the content encoder and the dark light encoder, it can extract the third content feature corresponding to the first enhanced image through the content encoder; at the same time, it can extract the fourth content feature corresponding to the first dark light image through the content encoder; and it can extract the second dark light feature corresponding to the first dark light image through the dark light encoder.

[0063] Furthermore, in the embodiments of this application, when the electronic device generates a first loop image and a second loop image by performing generation processing through a content image generator and a dark light image generator based on the third content feature, the fourth content feature, and the second dark light feature, the third content feature and the second dark light feature can be input into the dark light image generator to obtain the first loop image corresponding to the first image; at the same time, the fourth content feature can be input into the content image generator to obtain the second loop image corresponding to the second image.

[0064] For example, in the embodiments of this application, Figure 4 This is a schematic diagram of the initial enhancement model, such as... Figure 4 As shown, the initial enhancement model may include a content encoder E C Dark Encoder E D Content Image Generator G C G, a low-light image generatorD Among them, E in the initial enhancement model C E D G C G D They all share weights.

[0065] It should be noted that, in the embodiments of this application, the first image can be first input to the content encoder E. C and dark light encoder E D Then by E C The first content feature corresponding to the first image is output, and E is used to... D The system outputs the first dark light feature corresponding to the first image, and can also input the second image into E. C , by E C The corresponding second content feature of the second image is output.

[0066] It should be noted that, in the embodiments of this application, after obtaining the first content feature, the second content feature, and the first dark light feature, the first content feature can be further input into the content image generator G. C , by G C The first enhanced image corresponding to the first output image can also be generated by simultaneously inputting the first content feature and the first dark light feature into the dark light image generator G. D , by G D The first reconstructed image corresponding to the first image can also be output, and the second content features can be input into G. C , by G C The second reconstructed image corresponding to the second image can also be output, and the second content features and the first dark light features can be input into G. D , by G D The corresponding output is the first dark-light image of the second image.

[0067] It should be noted that, in the embodiments of this application, after obtaining the first enhanced image, the first reconstructed image, the second reconstructed image, and the first low-light image respectively, the first enhanced image can be further input into E. C , by E C The third content feature corresponding to the first enhanced image can also be input into E. C , by E C The fourth content feature corresponding to the first dark image can also be output, and the first dark image can be input to E. D , by E D The corresponding output is the second dark light feature of the first dark light image.

[0068] It should be noted that, in the embodiments of this application, after obtaining the third content feature, the fourth content feature, and the second dark light feature respectively, the third content feature and the second dark light feature can be further input into G simultaneously. D , by G D The first loop image corresponding to the first image is output; the fourth content feature can also be input into G. C , by G C The corresponding second loop image is output.

[0069] Therefore, in the embodiments of this application, when training the initial enhancement model based on a decoupled representation including content features and / or dark light features, E can be used. C and E D These two encoders extract and decompose the content features (reflection features) and dark light features from the first image. While extracting features from the first image, they also extract the content features from the second image. Then, G... C By utilizing the content features of the first image, a clearly brightened first enhanced image can be obtained, G D The first reconstructed image can be reconstructed using the content features and dark-light features of the first image. While enhancing and reconstructing the first image, G... C The second reconstructed image is also reconstructed using the content features of the second image. D A first low-light image is generated using the content features of the second image and the low-light features of the first image. Subsequently, the content features and low-light features of the first low-light image are extracted, and the content features of the first enhanced image are also extracted. Finally, a low-light image, i.e., the first loop image, is generated using the content features of the first enhanced image and the low-light features of the first low-light image. A clear image, i.e., the second loop image, is generated using the content features of the first low-light image.

[0070] It should be noted that, in the embodiments of this application, the first reconstructed image, the first dark light image, the first loop image, and the second loop image can all be further used to perform unsupervised training on the initial enhancement model, so as to gradually learn the image content and the distribution of dark light in round after round of training.

[0071] Furthermore, in embodiments of this application, when the electronic device performs unsupervised training on the content encoder and content image generator using the first reconstructed image, the second reconstructed image, the first enhanced image, the first low-light image, the first loop image, and the second loop image to obtain the trained content encoder and the trained content image generator, it can determine the total loss function based on the first reconstructed image, the second reconstructed image, the first enhanced image, the first low-light image, the first loop image, and the second loop image; and use the total loss function to correct the content encoder and the content image generator to obtain the trained content encoder and the trained content image generator.

[0072] It should be noted that, in the embodiments of this application, the total loss function can consist of three parts, specifically a first loss function, a second loss function, and a third loss function. The first loss function can be the cycle-consistent loss L. cycle The second loss function can be the image reconstruction loss L. recon The third loss function can be the domain adversarial loss L. adv .

[0073] It is understood that in the embodiments of this application, the first loss function, namely the cycle consistency loss, is to ensure that the image generated in the final cycle is consistent with the content of the initial image; the second loss function, namely the image reconstruction loss, is to ensure that a clear image containing only content can be generated using content features, and at the same time, to ensure that a dark image under the content can be generated using content features and dark light features; the third loss function, namely the domain adversarial loss, is to ensure that the distribution of the reconstructed image is consistent with the real image.

[0074] Furthermore, in the embodiments of this application, when determining the total loss function based on the first reconstructed image, the second reconstructed image, the first enhanced image, the first low-light image, the first loop image, and the second loop image, a first loss function can be determined based on the first image, the second image, the first loop image, and the second loop image; simultaneously, a second loss function can be determined based on the first image, the second image, the first reconstructed image, and the second reconstructed image; and a third loss function can also be determined based on the first image, the second image, the first enhanced image, and the first low-light image; finally, the total loss function can be determined based on the first loss function, the second loss function, and the third loss function.

[0075] For example, in an embodiment of this application, assume the first image is x, the second image is y, and the first loop image is x. cycle The second loop image is y cycle Then, the first loss function can be determined according to the following formula, namely L. cycle :

[0076] Lcycle =|xx cycle |1+|yy cycle |1 (1)

[0077] For example, in an embodiment of this application, assume the first image is x, the second image is y, and the first reconstructed image is x. recon The second reconstructed image is y recon Then, the second loss function can be determined according to the following formula, namely L. recon :

[0078] L recon =|xx recon |1+|yy recon |1 (2)

[0079] For example, in an embodiment of this application, assume the first image is x, the second image is y, and the first enhanced image is x. enhance The first dark-light image is y dark Then, the third loss function can be determined according to the following formula, namely L. adv :

[0080]

[0081]

[0082]

[0083] Where E() represents the expectation operator; D N This is a low-light image discriminator used to distinguish whether an image is a generated low-light image or a real low-light image; D c This is a content discriminator used to determine whether an image is a real, sharp image or a generated, sharp image. That is, L... adv It can include D c corresponding and D N corresponding

[0084] For example, in the embodiments of this application, Figure 5 A schematic diagram of the loss function, as shown below. Figure 5As shown, the total loss function can be composed of a first loss function, a second loss function, and a third loss function. Specifically, when determining the first loss function, the consistency loss between the first image and the first loop image can be determined to obtain loss function 1. Simultaneously, the consistency loss between the second image and the second loop image can be determined to obtain loss function 2. The first loss function can then be determined using loss functions 1 and 2. When determining the second loss function, the reconstruction loss between the first image and the first reconstructed image can be determined to obtain loss function 3. Simultaneously, the reconstruction loss between the second image and the second reconstructed image can be determined to obtain loss function 4. The second loss function can then be determined using loss functions 3 and 4. When determining the third loss function, the adversarial loss between the first image and the first low-light image can be determined using a discriminator to obtain loss function 5. Simultaneously, the adversarial loss between the second image and the first enhanced image can be determined using a discriminator to obtain loss function 6. The third loss function can then be determined using loss functions 5 and 6.

[0085] Furthermore, in the embodiments of this application, after determining the total loss function through the calculation of the first loss function, the second loss function, and the third loss function, the total loss function can be used to correct the content encoder and the content image generator, and finally the trained content encoder and the trained content image generator are obtained, constituting an image enhancement model.

[0086] Step 102: Enhance the low-light image to be processed using an image enhancement model to obtain the enhanced image.

[0087] In the embodiments of this application, the electronic device trains an initial enhancement model using a low-light image dataset and an enhanced image dataset. After obtaining the image enhancement model, the device can use the image enhancement model to enhance the low-light image to be processed and obtain the corresponding enhanced image.

[0088] Furthermore, in the embodiments of this application, when enhancing the low-light image to be processed using the image enhancement model, the fifth content feature of the low-light image to be processed can be extracted first using the trained content encoder; then the fifth content feature can be input into the trained content image generator to obtain the enhanced image.

[0089] It is understood that, in the embodiments of this application, Figure 6 A schematic diagram of the structure of an image enhancement model, such as Figure 6 As shown, after training the initial augmentation model using the low-light image dataset and the augmented image dataset, only the trained content encoder (E) needs to be retained. C ) and trained content image generator (G CThis can be used as the inference structure. For an image enhancement model based on this inference structure, inputting a low-light image of a night scene will output an enhanced, clear image.

[0090] It should be noted that, in the embodiments of this application, the image enhancement model can be used to enhance the low-light image to be processed, generating the corresponding enhanced image. The image enhancement model can consist of a trained content encoder and a trained content image generator. The trained content encoder can be used to extract content features, and the trained content image generator can be used to reconstruct the image based on the content features to obtain the corresponding enhanced image.

[0091] For example, in the embodiments of this application, Figure 7 This is a schematic diagram of image enhancement processing, such as... Figure 7 As shown, for the input dark light image to be processed, the trained content encoder can learn to represent the content (content features) of the image itself, and then input the content into the trained content image generator. Correspondingly, the content image generator can obtain a clear content image with the dark light removed by using the content features consistent with the input domain, that is, obtain the corresponding enhanced image.

[0092] In summary, the image processing method proposed in steps 101 and 102 above can include a night scene image enhancement method based on decoupled representation including content features and / or dark light features. This involves an unsupervised enhancement method, which does not require training with paired dark light and clear image datasets during model training, thus saving data acquisition costs while achieving excellent image enhancement results.

[0093] Understandably, common night scene image enhancement techniques learn the mapping relationship from low-light images to enhanced images through supervised training. However, this learning method often leads to network overfitting and poor enhancement performance in new scenes, failing to restore details effectively. To address the problems of common techniques, this application introduces decoupled representations for unsupervised training, resulting in a network with stronger robustness and generalization capabilities, as well as better detail restoration and noise suppression in new scenes.

[0094] Therefore, the embodiments of this application propose a night scene image enhancement method based on decoupled representations including content features and / or dark light features. An encoder is designed to decompose an image into content features and dark light features, achieving unsupervised training for night scene enhancement without requiring a one-to-one paired dataset. Specifically, the image enhancement method proposed in this application, during model training, can decompose dark light images into content features and dark light features, extract content from clear images, and then synthesize them. A reasonable loss function is designed to ensure successful decoupled representation learning between the encoder and generator.

[0095] It is understood that, in the embodiments of this application, the robustness and generalization of the network are improved precisely because decoupled representations are introduced into the training process of the model, and the trained image enhancement model has good color and detail recovery capabilities even on new data and new scenes.

[0096] It should be noted that, in the embodiments of this application, in addition to the two types of encoders, namely the content encoder and the dark light encoder, and the two types of image generators, namely the content image generator and the dark light image generator, the initial enhancement model can also be designed with a noise encoder and a noise image generator corresponding to noise processing, thereby further improving the image processing capability of the image enhancement model.

[0097] It is understood that, in the embodiments of this application, the model training process may also involve other types of encoders and image generators, and is not limited to content encoders, dark light encoders, noise encoders, content image generators, dark light image generators, or noise image generators.

[0098] This application provides an image processing method in which an electronic device trains an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model. The initial enhancement model is used to extract at least one image feature based on decoupled representations and to perform image enhancement processing based on these features. The image enhancement model then enhances the low-light image to be processed, obtaining an enhanced image. Therefore, in this application, the image enhancement model used for enhancing low-light images is trained based on decoupled representations including content features and / or low-light features. Thus, the low-light image dataset and the enhanced image dataset used during training do not require a mapping relationship, thereby reducing the cost and complexity of model training. Furthermore, the introduction of decoupled representations improves the robustness and generalization of the network model, thereby enhancing the image processing effect.

[0099] Based on the above embodiments, another embodiment of this application proposes an image processing method, which can be a night scene image enhancement method based on decoupled representation including content features and / or dark light features. An encoder is designed to decompose the image into content features and dark light features, realizing unsupervised training for night scene enhancement, and without the need for one-to-one paired datasets, which greatly reduces the time cost and complexity of training.

[0100] In other words, the embodiments of this application propose a night scene image enhancement method based on decoupled representations including content features and / or dark light features. This method does not require training with paired dark light image and enhanced image datasets, and is an unsupervised night scene image enhancement method. This greatly reduces the cost of collecting paired datasets, and the trained enhancement network has stronger robustness and generalization.

[0101] Furthermore, in the embodiments of this application, Figure 8 A schematic diagram of model training, such as Figure 8 As shown, when training the model, the dataset can be constructed first (step 201). Then, the network (initial augmentation model) can be trained based on the decoupled representation including content features and / or dark and light features (step 202). Finally, after the network converges, an inference structure (image augmentation model) that is much simpler than the training structure can be obtained, and the image augmentation model can be used for subsequent inference (step 204).

[0102] It is understood that the decoupled representation used for model training in the embodiments of this application is an unsupervised training method. Specifically, a low-light image dataset and an enhanced image dataset can be constructed first, and the images in the low-light image dataset and the images in the enhanced image dataset do not necessarily have a one-to-one correspondence.

[0103] It should be noted that, in the embodiments of this application, a low-light enhancement image dataset that does not require one-to-one pairing is first constructed, that is, the constructed dataset includes a low-light image dataset and an enhancement image dataset that do not have a corresponding relationship.

[0104] In some embodiments, Figure 9 This is a schematic diagram of a low-light enhancement image dataset, such as... Figure 9 As shown, the low-light image dataset may include at least one low-light image, wherein the at least one low-light image can be a low-light image from any angle, in any environment, and with any subject. Correspondingly, the enhanced image dataset may include at least one low-light enhanced image, wherein the at least one low-light enhanced image can be a low-light image from any angle, in any environment, and with any subject.

[0105] It is understood that, in the embodiments of this application, the training dataset in common technologies can generally include unprocessed images and processed images with a one-to-one correspondence. However, the image processing method proposed in the embodiments of this application can utilize decoupled representations to train the initial enhancement model. Therefore, for the training dataset, it is not required that there be a one-to-one mapping relationship between the images in the low-light image dataset and the images in the enhancement image dataset.

[0106] Furthermore, in the embodiments of this application, since only one dark light image dataset and one enhanced image dataset are needed when training the initial enhancement model, and it is not required that there be a one-to-one correspondence between the images in the two datasets at the pixel level, the cost of image acquisition is greatly reduced and the complexity of the image processing process is reduced.

[0107] The fundamental tenet of Retinex theory is that the color of an object is determined by its ability to reflect light; the color of an object is unaffected by the non-uniformity of illumination and possesses uniformity. Retinex theory posits that the image S seen by an observer is obtained by the reflection of incident light L from the object's surface, and the reflectivity R is determined by the object itself and is unaffected by changes in the incident light L. In other words, the original image S is the product of the illumination image L and the reflectivity image R, as expressed by the following formula:

[0108] S(x,y)=R(x,y)×L(x,y) (6)

[0109] Image enhancement based on Retinex transforms the image into the logarithmic domain, i.e., s = logS, l = logL, r = logR, thereby converting the above formulas into a sum relationship s = r + l, and separating the illuminance l from it to eliminate the influence of uneven illumination and improve image quality.

[0110] Based on the Retinex theory described above, a low-light image can be decomposed into the image content itself and a low-light mask superimposed on it. Therefore, this decomposition process can be simulated using a decoupled representation that includes content features and / or low-light features to extract the content features and low-light features from the image.

[0111] In the embodiments of this application, two types of encoders are designed: content encoder E C and dark light encoder E D The content encoder can learn to represent the content of the image itself. This content is domain-consistent and unaffected by the dark lighting mask; the content features of the same pair of sharp and dark images should be consistent. The dark lighting encoder, on the other hand, can learn to represent the dark lighting mask in the dark image. This dark lighting mask is domain-specific and does not exist in the sharp image; it is unique to the image itself.

[0112] In the embodiments of this application, two generators are also designed: a content image generator G. C and low-light image generator G D A content image generator that takes consistent content features as input can produce a clear content image with dark areas removed, while a dark image generator that takes consistent content features as input and domain-specific dark features can produce a dark image with dark areas added to the content.

[0113] Furthermore, in the embodiments of this application, during the training process based on decoupled representations, content features and / or dark light features can be selected to separate from the image, so that the content features and dark light features can be used to complete the unsupervised training of the model.

[0114] In some embodiments, the meanings of the symbols involved in training the initial augmentation model are shown in the table below:

[0115] Table 1

[0116]

[0117]

[0118] Based on Table 1 above, in the training process based on decoupled representations including content features and / or dark light features, x and y represent the unpaired dark light image (first image) and the clear image (second image), respectively, E C and E D These represent the content encoder and the low-light encoder, respectively. G C and G D These represent the content image generator and the low-light image generator, respectively. and Let x and y represent the content features respectively. and Let x and y represent the dark light characteristics, respectively. enhance and x recon Let y represent the enhancement result and the reconstruction result of the low-light image x, respectively. dark and y recon These are the results of darkening the sharp image y and the reconstruction result of the sharp image y, respectively. It is a network-enhanced content feature of the low-light image x. and These are the content features and dark features of the dark image y, respectively, and x. cycle Indicates using x enhance Content features and y dark Dark-light image reconstructed from dark-light features, y cycle Indicates the use of y dark A clear image reconstructed from content features.

[0119] In some embodiments, Figure 10 This is a schematic diagram illustrating the training process of the initial augmentation model, as shown below. Figure 10 As shown, the initial enhancement model may include a content encoder E C Dark Encoder E D Content Image Generator G C G, a low-light image generator D Among them, E in the initial enhancement model C E D G C G D All weights are shared. Throughout the training process based on decoupled representations including content features and / or dark light features, two encoders can be used to extract and decompose the content features (reflection features) and dark light features from the night scene image x. While extracting the features of x, the content features of the clear image y are also extracted. Then, using the content features of x, a clear and brightened image x can be obtained. enhance An image x can be reconstructed using the content features and dark-light features of x. recon While enhancing and reconstructing x, y was also reconstructed using the content features of y. recon A dark-light image y was generated using the content features of y and the dark-light features of x. dark Subsequently, y was extracted separately. dark Content features and dark light features, extract x enhance The content characteristics. Finally, using x... enhance Content features and y dark Dark light features generate dark light images x cycle , using y dark Content features generate clear images y cycle .

[0120] It should be noted that, in the embodiments of this application, y is generated cycle x cycle y recon x recon y dark The goal is to train the network in an unsupervised manner, so that the network can gradually learn the image content and the distribution of dark light in rounds of training.

[0121] Furthermore, in the embodiments of this application, when generating y based on the initial enhancement model... cycle x cycle y recon x recon Then, the relevant loss function can be calculated.

[0122] In some embodiments, the total loss function can be divided into three parts: the first loss function can be the cycle-consistent loss L. cycle The second loss function can be the image reconstruction loss L. recon The third loss function can be the domain adversarial loss L. adv Therefore, the total loss function L can be expressed as follows:

[0123]

[0124] For example, in the embodiments of this application, Figure 11 The diagram illustrates the calculation process of the loss function, as shown below. Figure 11 As shown, the role of cycle consistency loss is to ensure that the content of the image generated in the final cycle is consistent with that of the initial image. Its calculation process is as shown in formula (1), that is, using the first image as x, the second image as y, and the first cycle image as x cycle The second loop image is y cycle Determine the first loss function L cycle For image reconstruction loss, it is desired that content features can be used to generate a clear image containing only the content, and that content features and dark light features can be used to generate a dark light image containing that content. To achieve this, the image reconstruction loss is calculated as shown in formula (2), that is, using the first image as x, the second image as y, and the first reconstructed image as x. recon The second reconstructed image is y recon Determine the second loss function L recon For the domain adversarial loss, in order to generate its sharp image x from the dark image x. enhance Generate the corresponding dark image y from the clear image y. dark It is necessary to ensure that the generated x enhance The distribution is consistent with the real, clear image y, and the generated y dark The distribution is consistent with the real dark-light image x. Therefore, two discriminators are needed. One is the dark-light image discriminator D. N One is the content discriminator, which distinguishes whether the image is a generated dark-light image or a real dark-light image; the other is the content discriminator D. c The algorithm determines whether an image is a real, clear image or a generated, clear image. The domain adversarial loss is calculated using formulas (3), (4), and (5), i.e., using the first image as x, the second image as y, and the first enhanced image as x. enhance The first dark-light image is y dark Calculate D respectively c corresponding and D N corresponding Therefore, it can be based on and Further determine the third loss function L adv .

[0125] Furthermore, in the embodiments of this application, after determining the total loss function through the calculation of the first loss function, the second loss function, and the third loss function, the total loss function can be used to correct the content encoder and the content image generator, and finally the trained content encoder and the trained content image generator are obtained, constituting an image enhancement model.

[0126] In other words, after training the initial enhancement model using a low-light image dataset and an enhanced image dataset based on decoupled representations including content features and / or low-light features, only the trained content encoder and the trained content image generator need to be retained as the inference structure. For an image enhancement model based on this inference structure, an input low-light image of a night scene can output a clear enhanced image.

[0127] In some embodiments, Figure 12 This is a schematic diagram illustrating image enhancement processing using an image enhancement model, such as... Figure 12 As shown, for the input low-light image x1 to be processed, the content encoder E C It can learn to represent the content of the image itself. (Content features), and then input the content into the content image generator G. C Correspondingly, the content image generator can obtain a clear content image with dark light removed by utilizing the consistent content features of the input domain, that is, obtain the corresponding enhanced image x2.

[0128] In summary, the image processing method proposed in this application can include a night scene image enhancement method based on decoupled representation including content features and / or dark light features. This involves an unsupervised enhancement method, which does not require training with paired dark light and clear image datasets during model training, thereby saving data acquisition costs while achieving excellent image enhancement results.

[0129] Understandably, common night scene image enhancement techniques learn the mapping relationship from low-light images to enhanced images through supervised training. However, this learning method often leads to network overfitting and poor enhancement performance in new scenes, failing to restore details effectively. To address the problems of common techniques, this application introduces decoupled representations for unsupervised training, resulting in a network with stronger robustness and generalization capabilities, as well as better detail restoration and noise suppression in new scenes.

[0130] Therefore, the embodiments of this application propose a night scene image enhancement method based on decoupled representations including content features and / or dark light features. An encoder is designed to decompose an image into content features and dark light features, achieving unsupervised training for night scene enhancement without requiring a one-to-one paired dataset. Specifically, the image enhancement method proposed in this application, during model training, can decompose dark light images into content features and dark light features, extract content from clear images, and then synthesize them. A reasonable loss function is designed to ensure successful decoupled representation learning between the encoder and generator.

[0131] It is understood that, in the embodiments of this application, the robustness and generalization of the network are improved precisely because decoupled representations are introduced into the training process of the model, and the trained image enhancement model has good color and detail recovery capabilities even on new data and new scenes.

[0132] It should be noted that, in the embodiments of this application, in addition to the two types of encoders, namely the content encoder and the dark light encoder, and the two types of image generators, namely the content image generator and the dark light image generator, the initial enhancement model can also be designed with a noise encoder and a noise image generator corresponding to noise processing, thereby further improving the image processing capability of the image enhancement model.

[0133] It is understood that, in the embodiments of this application, the model training process may also involve other types of encoders and image generators, and is not limited to content encoders, dark light encoders, noise encoders, content image generators, dark light image generators, or noise image generators.

[0134] This application provides an image processing method in which an electronic device trains an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model. The initial enhancement model is used to extract at least one image feature based on decoupled representations and to perform image enhancement processing based on these features. The image enhancement model then enhances the low-light image to be processed, obtaining an enhanced image. Therefore, in this application, the image enhancement model used for enhancing low-light images is trained based on decoupled representations including content features and / or low-light features. Thus, the low-light image dataset and the enhanced image dataset used during training do not require a mapping relationship, thereby reducing the cost and complexity of model training. Furthermore, the introduction of decoupled representations improves the robustness and generalization of the network model, thereby enhancing the image processing effect.

[0135] Based on the above embodiments, in another embodiment of this application... Figure 13 Schematic diagram of the composition structure of an electronic device Figure 1 ,like Figure 13 As shown, the electronic device 10 proposed in this application embodiment may include: a training unit 11 and a processing unit 12.

[0136] The training unit 11 is used to train an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on decoupled representation, and to perform image enhancement processing based on the at least one image feature;

[0137] The processing unit 12 is used to enhance the low-light image to be processed using the image enhancement model to obtain the enhanced image.

[0138] In the embodiments of this application, further, Figure 14 Schematic diagram of the composition structure of an electronic device Figure 2 ,like Figure 14 As shown, the electronic device 10 proposed in this application embodiment may further include a processor 13, a memory 14 storing instructions executable by the processor 13, and further, the electronic device 10 may also include a communication interface 15 and a bus 16 for connecting the processor 13, the memory 14 and the communication interface 15.

[0139] In the embodiments of this application, the processor 13 can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other types, and this application embodiment does not specifically limit this. The electronic device 10 may also include a memory 14, which can be connected to the processor 13. The memory 14 is used to store executable program code, which includes computer operation instructions. The memory 14 may include high-speed RAM memory and may also include non-volatile memory, such as at least two disk drives.

[0140] In embodiments of this application, bus 16 is used to connect communication interface 15, processor 13 and memory 14 and the mutual communication between these devices.

[0141] In embodiments of this application, memory 14 is used to store instructions and data.

[0142] Further, in an embodiment of this application, the processor 13 is used to train an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on decoupled representation, and to perform image enhancement processing according to the at least one image feature; the image enhancement model is used to enhance the low-light image to be processed to obtain an enhanced image.

[0143] In practical applications, the aforementioned memory 14 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 13.

[0144] Furthermore, in this embodiment, the functional modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0145] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] This application provides an electronic device that trains an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model. The initial enhancement model is used to extract at least one image feature based on decoupled representations and perform image enhancement processing based on these features. The image enhancement model then enhances the low-light image to be processed, obtaining an enhanced image. Therefore, in this application, the image enhancement model for enhancing low-light images is trained based on decoupled representations including content features and / or low-light features. Thus, the low-light image dataset and the enhanced image dataset used during training do not require a mapping relationship, thereby reducing the cost and complexity of model training. Furthermore, the introduction of decoupled representations improves the robustness and generalization of the network model, thereby enhancing the image processing effect.

[0147] This application provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the image processing method described above.

[0148] Specifically, the program instructions corresponding to an image processing method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the program instructions corresponding to an image processing method in the storage media are read or executed by an electronic device, the following steps are included:

[0149] An initial enhancement model is trained using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on decoupled representation, and to perform image enhancement processing based on the at least one image feature;

[0150] The image enhancement model is used to enhance the low-light image to be processed, resulting in an enhanced image.

[0151] This application provides a chip, which includes programmable logic circuitry and / or program instructions. When the chip runs, it implements the image processing method described above, specifically including the following steps:

[0152] An initial enhancement model is trained using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on decoupled representation, and to perform image enhancement processing based on the at least one image feature;

[0153] The image enhancement model is used to enhance the low-light image to be processed, resulting in an enhanced image.

[0154] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0155] This application is described with reference to schematic and / or block diagrams of implementations of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the schematic and / or block diagrams can be implemented by computer program instructions, and combinations of blocks in the schematic and / or block diagrams can be implemented. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the schematic and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the implementation flow diagram. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. An image processing method, characterized in that, The method includes: An initial enhancement model is trained using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on decoupled representation, and to perform image enhancement processing based on the at least one image feature; The image enhancement model is used to enhance the low-light image to be processed, and the enhanced image is obtained. in, The decoupling characterization includes content features and / or dark-light features; The initial enhancement model includes a content encoder, a low-light encoder, a content image generator, and a low-light image generator; The image enhancement model includes a trained content encoder and a trained content image generator; wherein... The content encoder is used to extract content features from the input image and output the content features to the content image generator and the low-light image generator, respectively. The low-light encoder is used to extract the low-light features of the input image and output the low-light features to the low-light image generator; The content image generator is used to perform image generation processing based on the input content features; The low-light image generator is used to perform image reconstruction processing based on the input content features and the low-light features.

2. The method according to claim 1, characterized in that, The process of training an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model includes: The content encoder and the dark light encoder respectively obtain the first content feature and the first dark light feature corresponding to the first image in the dark light image dataset, and the second content feature corresponding to the second image in the enhanced image dataset; Based on the first content feature, the second content feature, and the first dark light feature, the content image generator and the dark light image generator are used to generate a first enhanced image, a first reconstructed image, a second reconstructed image, and a first dark light image, respectively. The third content feature corresponding to the first enhanced image and the fourth content feature and the second dark light feature corresponding to the first dark light image are obtained through the content encoder and the dark light encoder, respectively. Based on the third content feature, the fourth content feature, and the second dark light feature, the first loop image and the second loop image are obtained by generating the content image generator and the dark light image generator, respectively. Using the first reconstructed image, the second reconstructed image, the first enhanced image, the first low-light image, the first loop image, and the second loop image, the content encoder and the content image generator are trained unsupervised to obtain the trained content encoder and the trained content image generator.

3. The method according to claim 2, characterized in that, The step of obtaining the first content feature and the first dark light feature corresponding to the first image in the dark light image dataset, and the second content feature corresponding to the second image in the enhanced image dataset, respectively, through the content encoder and the dark light encoder, includes: The first content feature corresponding to the first image is extracted using the content encoder; The first dark light feature corresponding to the first image is extracted using the dark light encoder; The second content feature corresponding to the second image is extracted using the content encoder.

4. The method according to claim 2, characterized in that, The process of generating a first enhanced image, a first reconstructed image, a second reconstructed image, and a first dark light image based on the first content feature, the second content feature, and the first dark light feature, through the content image generator and the dark light image generator, respectively, includes: The first content feature is input into the content image generator to obtain the first enhanced image corresponding to the first image; The first content feature and the first dark light feature are input into the dark light image generator to obtain the first reconstructed image corresponding to the first image; The second content feature is input into the content image generator to obtain the second reconstructed image corresponding to the second image; The second content feature and the first dark light feature are input into the dark light image generator to obtain the first dark light image corresponding to the second image.

5. The method according to claim 2, characterized in that, The step of obtaining the third content feature corresponding to the first enhanced image and the fourth content feature and second dark light feature corresponding to the first dark light image through the content encoder and the dark light encoder respectively includes: The third content feature corresponding to the first enhanced image is extracted using the content encoder; The fourth content feature corresponding to the first low-light image is extracted using the content encoder; The second dark light feature corresponding to the first dark light image is extracted using the dark light encoder.

6. The method according to claim 2, characterized in that, The process of generating a first loop image and a second loop image based on the third content feature, the fourth content feature, and the second dark light feature, using the content image generator and the dark light image generator, includes: The third content feature and the second dark light feature are input into the dark light image generator to obtain the first loop image corresponding to the first image; The fourth content feature is input into the content image generator to obtain the second loop image corresponding to the second image.

7. The method according to claim 2, characterized in that, The step of using the first reconstructed image, the second reconstructed image, the first enhanced image, the first low-light image, the first loop image, and the second loop image to perform unsupervised training on the content encoder and the content image generator to obtain the trained content encoder and the trained content image generator includes: Based on the first reconstructed image, the second reconstructed image, the first enhanced image, the first low-light image, the first loop image, and the second loop image, determine the total loss function; The content encoder and the content image generator are corrected using the total loss function to obtain the trained content encoder and the trained content image generator.

8. The method according to claim 7, characterized in that, The step of determining the total loss function based on the first reconstructed image, the second reconstructed image, the first enhanced image, the first low-light image, the first loop image, and the second loop image includes: A first loss function is determined based on the first image, the second image, the first cyclic image, and the second cyclic image. A second loss function is determined based on the first image, the second image, the first reconstructed image, and the second reconstructed image. A third loss function is determined based on the first image, the second image, the first enhanced image, and the first low-light image. The total loss function is determined based on the first loss function, the second loss function, and the third loss function.

9. The method according to claim 1, characterized in that, The process of enhancing the low-light image to be processed using the image enhancement model to obtain the enhanced image includes: The trained content encoder is used to extract the fifth content feature of the low-light image to be processed; The fifth content feature is input into the trained content image generator to obtain the enhanced image.

10. The method according to claim 2, wherein, There is no mapping relationship between the first image and the second image.

11. An electronic device, characterized in that, The electronic device includes: a training unit and a processing unit. The training unit is used to train an initial enhancement model using a low-light image dataset and an enhanced image dataset to obtain an image enhancement model; wherein, the initial enhancement model is used to extract at least one image feature based on decoupled representation, and to perform image enhancement processing based on the at least one image feature; The processing unit is used to perform enhancement processing on the low-light image to be processed through the image enhancement model to obtain the enhanced image; in, The decoupling characterization includes content features and / or dark-light features; The initial enhancement model includes a content encoder, a low-light encoder, a content image generator, and a low-light image generator; The image enhancement model includes a trained content encoder and a trained content image generator; wherein... The content encoder is used to extract content features from the input image and output the content features to the content image generator and the low-light image generator, respectively. The low-light encoder is used to extract the low-light features of the input image and output the low-light features to the low-light image generator; The content image generator is used to perform image generation processing based on the input content features; The low-light image generator is used to perform image reconstruction processing based on the input content features and the low-light features.

12. An electronic device, characterized in that, The electronic device includes a processor and a memory storing processor-executable instructions, which, when executed by the processor, implement the method as described in any one of claims 1-10.

13. A chip, characterized in that, The chip includes programmable logic circuitry and / or program instructions, which, when the chip is running, implement the method as described in any one of claims 1-10.

Images