Image enhancement method, device and storage medium

Abstract

The present invention provides an image enhancement method, device and storage medium. The image enhancement method comprises: obtaining a Y channel image of an image to be enhanced in a YUV color space and inputting it into a trained CNN network to obtain a preliminary enhanced image of the Y channel; performing superpixel segmentation on the preliminary enhanced image of the Y channel, and copying the segmentation result of the preliminary enhanced image of the Y channel to the Y channel image; inputting the preliminary enhanced image of the Y channel after superpixel segmentation and the Y channel image after superpixel segmentation into an ANP network to obtain a final enhanced image of the Y channel; synthesizing the final enhanced image of the Y channel, the U channel image and the V channel image to obtain an enhanced image. The CNN network is used to enhance the brightness of the Y channel image to restore the hidden main features; the preliminary enhanced image is subjected to superpixel segmentation and ANP neural network processing to realize adaptive enhancement of different areas in the image, and the detailed features of different positions are mined to promote the efficient implementation of subsequent computer vision tasks.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image enhancement method, device and storage medium. Background Art

[0002] Bright images taken under uniformly distributed light have obvious features and contain rich details. When the ambient light is insufficient or there is an abnormal exposure problem, people may take pictures with poor lighting (these pictures are called low-light pictures). These dark pictures not only affect human observation, but also seriously affect subsequent computer vision tasks. With the rapid development of computer technology, more and more people choose electronic devices to store and process digital images, which makes computer vision tasks particularly important. Low-light image enhancement is usually a key part of preprocessing and is of great significance to improving the efficiency and performance of digital image processing.

[0003] In the prior art, for low-light image enhancement, there are usually methods based on histogram equalization (HE), methods based on the Retinex model, and methods based on the defogging model. The disadvantage of the HE model is that the merging and stretching of grayscale may cause color migration and detail loss in the image. The Retinex model will affect the robustness of the image and may even cause very obvious color distortion in certain scenes. The low-light enhancement method based on the defogging model lacks physical model support, and the brightness and contrast effects of its final enhanced image are average, and its enhancement quality still needs to be improved. In addition, most of the methods mentioned so far only focus on the visual enjoyment brought to the observer by the final enhancement result, while ignoring key detail information such as image features required for subsequent computer vision tasks. Features are crucial for images. They can enhance the robustness of images and improve the efficiency of subsequent target segmentation and image recognition operations. Summary of the invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art and provides an image enhancement method, device and storage medium.

[0005] In order to achieve the above-mentioned purpose of the present invention, according to a first aspect of the present invention, the present invention provides an image enhancement method, comprising: obtaining a Y channel image, a U channel image and a V channel image of a Y channel image in a YUV color space; inputting the Y channel image into a trained CNN network to obtain a Y channel preliminary enhanced image; performing superpixel segmentation on the Y channel preliminary enhanced image, and copying the segmentation result of the Y channel preliminary enhanced image to the Y channel image; inputting the Y channel preliminary enhanced image after superpixel segmentation and the Y channel image after superpixel segmentation into an ANP network to obtain a Y channel final enhanced image; and synthesizing the Y channel final enhanced image, the U channel image and the V channel image to obtain an enhanced image.

[0006] The above technical solution: The image to be enhanced is generally a low-light image with low brightness. The Y channel representing brightness in the YUV color space is separated from the U and V channels representing chrominance. Selecting to process the Y channel image can effectively enhance the low-light image. The CNN network is used to enhance the brightness of the Y channel image and restore the main features hidden in the dark; starting from the local part of the image, the Y channel preliminary enhanced image is subjected to superpixel segmentation and ANP neural network processing to achieve adaptive enhancement of different areas in the image, and further explore the detailed features of different positions in the image, while bringing a better visual experience, promoting the efficient implementation of subsequent computer vision tasks. The segmentation result is directly copied to the corresponding Y channel image, so that there is a completely consistent segmentation area between the Y channel image and the Y channel preliminary enhanced image.

[0007] In a preferred embodiment of the present invention, the CNN network training process includes: acquiring multiple initial images, reducing the brightness of the initial images to obtain the images to be enhanced corresponding to the initial images, and using the Y channel images of the images to be enhanced as CNN training samples to construct a CNN training set; constructing a CNN network; and iteratively training the CNN network using the CNN training set.

[0008] The above technical solution: The initial image is generally a bright image, which has uniform illumination distribution and obvious features and contains rich detail information. The bright image is reduced in brightness to be used as a training sample, so that the bright and clear image features can be learned by the CNN network as much as possible.

[0009] In a preferred embodiment of the present invention, during the CNN network training process, the mean square error between the training sample and the Y channel image of the initial image corresponding to the training sample is used as the loss function.

[0010] The above technical solution: feedback updates the network parameters by minimizing the mean square error. The smaller the mean square error, the closer the CNN network output image is to the target image, and the better the image restoration effect.

[0011] In a preferred embodiment of the present invention, the CNN network includes a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, a third convolutional layer, a third activation layer, a fourth convolutional layer and a fourth activation layer connected in sequence.

[0012] The above technical solution: CNN adopts a shallow network structure with a simple structure and short training time, and can complete feature extraction, feature mapping and image restoration of the image to be enhanced, as a priori step in the image enhancement method of the present invention.

[0013] In a preferred embodiment of the present invention, the convolution kernel size of the first convolution layer is 9×9, the activation function of the first activation layer is LRelu, the convolution kernel size of the second convolution layer is 1×1, the activation function of the second activation layer is Relu, the convolution kernel size of the third convolution layer and the fourth convolution layer is 5×5, and the activation function of the third activation layer and the fourth activation layer is Relu.

[0014] In a preferred embodiment of the present invention, the specific process of inputting the Y channel preliminary enhanced image after superpixel segmentation and the Y channel image after superpixel segmentation into the ANP network to obtain the final enhanced image of the Y channel is as follows: step A, assuming that the Y channel preliminary enhanced image and the Y channel image are both superpixel segmented into M image areas, the Y channel preliminary enhanced image and the M image areas of the Y channel image correspond one to one, and M is a positive integer; step B, inputting the first image area of ​​the Y channel image and the first image area of ​​the Y channel preliminary enhanced image into the ANP network to obtain the first iteration result; step C, combining the nth image area of ​​the Y channel image with the n-1th iteration result to obtain a first combined image, combining the first image area of ​​the Y channel preliminary enhanced image to the nth image area to obtain a second combined image, and inputting the first combined image and the second combined image into the ANP network to obtain the nth iteration result; n is a positive integer, and 2≤n≤N; step D, if n is equal to M, stop the iteration and output the nth iteration result; if n is less than M, set n=n+1, and return to step C.

[0015] The above technical solution: after an image area is initially enhanced by the ANP network, it needs to be spliced ​​and combined with the next image area, and then sent to the ANP network together for the next iterative enhancement step, and finally this step is repeated continuously until all image areas in the Y channel image have completed the enhancement process. This interlocking enhancement process can enable the ANP network structure to better focus on various feature detail information in the local area of ​​the image, and also make the splicing and fusion between the image areas more natural, thereby obtaining a better overall enhancement result.

[0016] In a preferred embodiment of the present invention, the ANP network includes a first MLP module, an attention mechanism module, a normal distribution feature acquisition module and a second MLP module; the output end of the first MLP module is respectively connected to the input end of the attention mechanism module and the input end of the normal distribution feature acquisition module, the output end of the attention mechanism module is connected to the input end of the second MLP module, and the output end of the normal distribution feature acquisition module is connected to the input end of the second MLP module.

[0017] The above technical solution: ANP combines an attention mechanism to reason about multiple possible mapping distributions between the same input data, which can better achieve image enhancement.

[0018] In a preferred embodiment of the present invention, the attention mechanism module adopts a multi-head attention mechanism.

[0019] The above technical solution: The multi-head attention mechanism is multi-dimensional, and can jointly pay attention to information from different positions in different spaces in the image. It can focus on more detailed features in the image, so that the enhanced image has more detailed features, which is conducive to performing computer vision tasks.

[0020] In order to achieve the above-mentioned purpose of the present invention, according to the second aspect of the present invention, the present invention provides an image enhancement device, comprising: an image acquisition module, used to acquire a Y channel image, a U channel image and a V channel image of an image to be enhanced in a YUV color space; a first enhancement module, used to input the Y channel image into a trained CNN network to obtain a preliminary Y channel enhanced image; a superpixel segmentation module, which performs superpixel segmentation on the preliminary Y channel enhanced image and copies the segmentation result of the preliminary Y channel enhanced image to the Y channel image; a second enhancement module, which inputs the preliminary Y channel enhanced image after superpixel segmentation and the Y channel image after superpixel segmentation into an ANP network to obtain a final Y channel enhanced image; and a synthesis module, which synthesizes the final Y channel enhanced image, the U channel image and the V channel image to obtain an enhanced image.

[0021] The above technical solution: uses the CNN network to enhance the brightness of the Y channel image and restore the main features hidden in the dark; starting from the local image, the Y channel preliminary enhanced image is subjected to superpixel segmentation and ANP neural network processing to achieve adaptive enhancement of different areas in the image, and further explore the detailed features of different positions in the image, while bringing a better visual experience, promoting the efficient implementation of subsequent computer vision tasks.

[0022] In order to achieve the above-mentioned purpose of the present invention, according to the third aspect of the present invention, the present invention provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the image enhancement method as described in the first aspect of the present invention.

[0023] The above technical solution: uses the CNN network to enhance the brightness of the Y channel image and restore the main features hidden in the dark; starting from the local image, the Y channel preliminary enhanced image is subjected to superpixel segmentation and ANP neural network processing to achieve adaptive enhancement of different areas in the image, and further explore the detailed features of different positions in the image, while bringing a better visual experience, promoting the efficient implementation of subsequent computer vision tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a schematic flow chart of an image enhancement method in a preferred embodiment of the present invention;

[0025] Figure 2 is a schematic diagram of the overall flow of an image enhancement algorithm in another preferred embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the network structure of a CNN in a preferred embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram of the network structure of ANP in a preferred embodiment of the present invention;

[0028] Figure 5 It is a schematic diagram comparing the enhancement effects of various algorithms in an application scenario of the present invention;

[0029] Figure 6 It is a schematic diagram comparing the target detection effects of various algorithms in an application scenario of the present invention. DETAILED DESCRIPTION

[0030] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0031] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0032] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0033] The present invention discloses an image enhancement method. In a preferred embodiment, Figure 1 As shown, including:

[0034] Step S1, obtaining the Y channel image, U channel image and V channel image of the image to be enhanced in the YUV color space. The image to be enhanced is preferably, but not limited to, a low illumination image or an image with uneven illumination, or a normal brightness image in which more details are expected to be restored. If the image to be enhanced is an RGB image, then the RGB format image needs to be converted to a YUV format image first, and each pixel in the image to be enhanced is converted according to the following formula:

[0035]

[0036] Among them, R, G, and B represent the R channel value, G channel value, and B channel value of the pixel point respectively; Y, U, and V represent the Y channel value, U channel value, and V channel value of the pixel point respectively.

[0037] Step S2, input the Y channel image into the trained CNN network to obtain the Y channel preliminary enhanced image.

[0038] Step S3, superpixel segmentation is performed on the Y channel preliminary enhanced image, and the segmentation result of the Y channel preliminary enhanced image is copied to the Y channel image, so that the Y channel preliminary enhanced image and the Y channel image have the same superpixel segmentation result. After segmentation, the Y channel preliminary enhanced image corresponds to the image area of ​​the Y channel image one by one. Existing superpixel segmentation methods can be used for superpixel segmentation, such as Simple Linear Iterative Clustering (SLIC) technology to segment the Y channel preliminary enhanced image obtained by the CNN network.

[0039] Step S4, input the Y channel preliminary enhanced image after superpixel segmentation and the Y channel image after superpixel segmentation into the ANP network to obtain the final enhanced image of the Y channel. ANP, namely Attentive Neural Process, refers to the attention neural process network. ANP is different from CNP and NP in that ANP can infer the existence of multiple possible mapping distributions between the same input data, and can also capture the output synergy effect of a given input. The network structure of ANP is preferably but not limited to the structure disclosed in the website https: / / www.zhihu.com / question / 285664896, which will not be repeated here.

[0040] Step S5, synthesizing the Y channel final enhanced image, the U channel image and the V channel image to obtain an enhanced image of the image to be enhanced and outputting it.

[0041] In a preferred embodiment, the CNN network training process includes: acquiring multiple initial images, reducing the brightness of the initial images to obtain images to be enhanced corresponding to the initial images, and the initial images are preferably, but not limited to, images with moderate brightness, uniform illumination distribution, and obvious details. Specifically, a fixed formula can be used to generate the Y channel image of the corresponding low-light image from the initial image as a training set. The Y channel images of the images to be enhanced are used as CNN training samples to form a CNN training set, and the Y channel images of the initial images corresponding to the CNN training samples are used as labels for the CNN training samples, so that the CNN network can learn the difference between the Y channel images of the initial images and the training samples, and then output a preliminary enhanced image with restored brightness; constructing a CNN network, further preferably, as Figure 3 As shown, the CNN network includes a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, a third convolutional layer, a third activation layer, a fourth convolutional layer and a fourth activation layer connected in sequence; the CNN network is iteratively trained using a CNN training set.

[0042] In this embodiment, preferably, during the CNN network training process, the mean square error (MSE) between the training sample and the Y channel image of the initial image corresponding to the training sample is used as the loss function, and the network parameters are updated through feedback of the loss function. A stable mapping relationship between the Y channel image of the initial image and the output image is established by minimizing the difference in pixel value between them.

[0043] In this embodiment, it is further preferred that the convolution kernel size of the first convolution layer is 9×9, the activation function of the first activation layer is LRelu, the convolution kernel size of the second convolution layer is 1×1, the activation function of the second activation layer is Relu, the convolution kernel size of the third and fourth convolution layers is 5×5, and the activation function of the third and fourth activation layers is Relu. Considering the flexibility of the brightness transformation of the image, that is, the mapping relationship of the network results requires nonlinear transformation, the network selects ReLu and LeakyReReLu (LReLu) functions as activation functions to complete more complex tasks. Their addition increases the sparsity of the network and reduces the mutual dependence between input and output data, thereby avoiding the disappearance of gradients and alleviating the situation of data overfitting. The advantages of nonlinear activation functions allow neural networks to better retain the detailed feature information of the original image in the field of image enhancement, which is conducive to simple and fast feature recovery.

[0044] In a preferred embodiment, Figure 2 As shown, in step S4, the specific process of inputting the Y channel preliminary enhanced image after superpixel segmentation and the Y channel image after superpixel segmentation into the ANP network to obtain the Y channel final enhanced image is:

[0045] Step A, assuming that both the Y channel preliminary enhanced image and the Y channel image are superpixel segmented into M image regions, the Y channel preliminary enhanced image and the M image regions of the Y channel image are in one-to-one correspondence, and M is a positive integer.

[0046] Step B, inputting the first image area of ​​the Y channel image and the first image area of ​​the Y channel preliminary enhanced image into the ANP network to obtain the first iteration result;

[0047] Step C, combining the nth image area of ​​the Y channel image with the n-1th iteration result to obtain a first combined image, combining the first image area of ​​the Y channel preliminary enhanced image to the nth image area to obtain a second combined image, and inputting the first combined image and the second combined image into the ANP network to obtain the nth iteration result; n is a positive integer, and 2≤n≤M;

[0048] Step D: if n is equal to M, stop iterating and output the result of the nth iteration; if n is less than M, set n=n+1 and return to step C.

[0049] In this embodiment, preferably, Figure 4As shown, the ANP network includes a first MLP module, an attention mechanism module, a normal distribution feature acquisition module and a second MLP module; the Y channel image and the Y channel preliminary enhanced image are input into the first MLP module, the attention mechanism module and the second MLP module together; the output end of the first MLP module is connected to the input end of the attention mechanism module and the input end of the normal distribution feature acquisition module respectively, the output end of the attention mechanism module is connected to the input end of the second MLP module, and the output end of the normal distribution feature acquisition module is connected to the input end of the second MLP module. MLP is a multilayer perceptron (Multilayer Perceptron, MLP).

[0050] In this embodiment, the ANP regression process is divided into two stages. First, in the training stage, the network selects the Y channel initial enhanced image region obtained after the CNN prior step and superpixel segmentation operation and its corresponding Y channel image region, randomly selects N pixel pairs from their size range, and uses them as context subsets C, and then further randomly selects N c pixel pairs as the target T. They are passed through the first MLP module and the attention mechanism module to obtain the data expression r * Then, the first MLP module is combined with the normal distribution to obtain the distribution expression z between the data, z includes the mean and variance, and finally r * , z and T work together and complement each other. After the second MLP module, the distribution F(t) of the model is obtained. According to this distribution F(t) mapping, the final enhanced image of the Y channel is obtained. In the test phase, the target input is set to T′, which is determined by the input image area. It is a sequence of intensity values ​​obtained by uniform distribution between the lowest pixel intensity and the highest pixel intensity value in the image area. By combining this sequence with the parameters of the previous training phase, the mapping distribution of pixel intensity from low to high in the area can be obtained. The whole process requires multiple training and testing, and C and T should be randomly re-selected for each training to consolidate the network structure. After multiple trainings, the loss function tends to converge, and the enhanced result of the original low-light image area can be obtained by using this stable distribution.

[0051] In this embodiment, three paths are indispensable in the training phase of ANP.

[0052] The first is the deterministic path. It consists of a first MLP module, a multilayer perceptron, and an attention module, and the parameters of the encoders they need are shared among all context subsets C. Because the present invention is to find the correspondence between the pixel intensities of two-dimensional image regions, this is essentially a one-dimensional regression task. Therefore, a three-layer nonlinear MLP with 128 hidden layer neurons combined with a ReLu activation function is sufficient to simulate the interaction between each context subset C of the input. In this path, the attention mechanism added to the network allows the network structure to focus on the weight distribution of its input data. It can perform weighted average calculations on the hidden states of the encoder at each step to obtain subsequent data, so that the network can more effectively select image information for targeted enhancement.

[0053] Then there is the latent path. It is similar to the deterministic path in that the encoder parameters it contains are also shared by all context subsets C, and the hidden layer sizes in the two paths are the same. This path needs to pass through two fully connected layers to calculate the mean and variance between the input data pairs to obtain the data distribution expression z, so that the marginal distribution of the prediction results generated by the entire network architecture is more relevant. The deterministic path models the precise local structure, and the r generated by it is * This allows specific queries to get closer to what the network thinks is relevant for the final prediction. However, the latent path models the global structure achieved by the stochastic process, with the goal of enforcing dependencies between object detections while maintaining the global structure.

[0054] The last is the decoding path. * , z and the target T are sent to the decoder module, i.e., the second MLP module. The ANP network uses the maximum a posteriori estimation (MAP) to minimize the structural risk and infer the model parameters. MAP is selected as the loss function to learn the parameter relationship between the context subset C and the target T.

[0055] After a low-light superpixel image area is initially enhanced by the ANP network, it needs to be spliced ​​and combined with the next low-light superpixel image area, and then sent to the ANP network for the next enhancement step. Finally, this step is repeated until all superpixel areas in the original low-light image have completed the enhancement process. This interlocking enhancement process allows the ANP network structure to better focus on various feature details in the local area of ​​the image, and also makes the splicing and fusion between the image areas more natural, thereby obtaining a better overall enhancement result.

[0056] In this embodiment, it is further preferred that the attention mechanism module adopts a multi-head attention mechanism.

[0057] In an application scenario, the enhancement effect of the image enhancement method provided by the present invention is verified. The ultimate goal of the low-light image enhancement method proposed by the present invention is to achieve a good balance between visual effects and image features, so as not to overemphasize the aesthetics of the image and ignore the feature information that makes the image have practical application significance.

[0058] 1. Enhanced algorithm visual effect verification

[0059] Three real low-light images are randomly selected from the ExDark dataset. The three images are processed by the existing LIME, CLAHE, RetinexNet, OCTM, BIMEF, DeHaze and GP image enhancement methods, as well as the image enhancement algorithm provided by the present invention. The processing results are shown in Figure 2. Figure 5 As shown. Figure 5 It can be found that the image enhancement method provided by the present invention has the highest acceptance, which means that its restoration effect is more in line with the public's visual perception. At the same time, according to the feedback from various observers, LIME often overexposes the light source position of its own image, while the BIMEF and DeHaze algorithms are relatively natural but the enhancement effect is not obvious. At the same time, the enhancement effect of GP makes the image seem to be covered with a layer of fog, losing the vividness of the color itself. OCTM has obvious distortion at the aperture, which greatly reduces the readability of the image. The color distortion of CLAHE makes the image look dirty, and the color saturation of RetinexNet is unbalanced, converting the photograph into a painting, and losing the meaning of low-light enhancement to restore image information.

[0060] 2. Machine feature recognition and verification

[0061] As a classic in the field of target detection, Yolov3 not only maintains the advantages of high-speed computing based on v1 and v2, but also enhances the detection ability of small targets, thereby further improving its accuracy. While keeping the Yolov3 network parameters and structure unchanged, the enhancement results of LIME, RetinexNet, GP and the image enhancement algorithm provided by the present invention are tested through Yolov3 to test the effect of each method on subsequent computer vision tasks. The comparison results are shown in the figure. Figure 6 As shown. Figure 6 It can be seen that the method proposed in the present invention has the best detection results. Figure 6 In the example on the left, Yolov3 can detect the two most prominent objects from the enhanced image obtained by the algorithm of the present invention: a car and a building. In the example on the right, Yolov3 can accurately detect the existence of a bus from the enhanced image obtained by the image enhancement method of the present invention, instead of misidentifying it as a car, a ship or other confusing objects, and the image enhancement method of the present invention is also the only enhancement algorithm that allows Yolov3 to detect the existence of a street lamp. The ANP network makes the enhancement process pay more attention to local areas, so that the final restored image feature information is more prominent, and the target detection results obtained by Yolov3 are more in line with the actual situation.

[0062] In summary, this experiment effectively demonstrates the feasibility of the low-light image enhancement technology provided by the present invention in practical applications, and the low-light image enhancement technology based on attention neural process ANP and superpixel segmentation proposed by the present invention can be used as a preprocessing method for high-level computer vision tasks such as target detection and image classification.

[0063] The present invention also discloses an image enhancement device, comprising: an image acquisition module, used for acquiring a Y channel image, a U channel image and a V channel image of a Y channel image in a YUV color space; a first enhancement module, used for inputting the Y channel image into a trained CNN network to obtain a preliminary enhanced image of the Y channel; a superpixel segmentation module, which performs superpixel segmentation on the preliminary enhanced image of the Y channel and copies the segmentation result of the preliminary enhanced image of the Y channel to the Y channel image; a second enhancement module, which inputs the preliminary enhanced image of the Y channel after superpixel segmentation and the Y channel image after superpixel segmentation into an ANP network to obtain a final enhanced image of the Y channel; and a synthesis module, which synthesizes the final enhanced image of the Y channel, the U channel image and the V channel image to obtain an enhanced image.

[0064] The present invention also discloses a computer-readable storage medium, in which at least one instruction, at least one program, code set or instruction set is stored, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the above-mentioned image enhancement method.

[0065] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0066] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. An image enhancement method, characterized in that: include: Obtain the Y channel image, U channel image and V channel image of the image to be enhanced in the YUV color space; Input the Y channel image into the trained CNN network to obtain a Y channel preliminary enhanced image; Performing superpixel segmentation on the Y channel preliminary enhanced image, and copying the segmentation result of the Y channel preliminary enhanced image to the Y channel image; The Y channel preliminary enhanced image after superpixel segmentation and the Y channel image after superpixel segmentation are input into the ANP network to obtain the final enhanced image of the Y channel. The specific process is as follows: Step A, assuming that both the Y channel preliminary enhanced image and the Y channel image are superpixel segmented into M image regions, the Y channel preliminary enhanced image and the M image regions of the Y channel image are in one-to-one correspondence, and M is a positive integer; Step B, inputting the first image area of ​​the Y channel image and the first image area of ​​the Y channel preliminary enhanced image into the ANP network to obtain the first iteration result; Step C, combining the nth image area of ​​the Y channel image with the n-1th iteration result to obtain a first combined image, combining the first image area of ​​the Y channel preliminary enhanced image to the nth image area to obtain a second combined image, and inputting the first combined image and the second combined image into the ANP network to obtain the nth iteration result; n is a positive integer, and 2≤n≤M; Step D, if n is equal to M, stop iteration and output the result of the nth iteration; if n is less than M, set n=n+1 and return to step C; The enhanced image is obtained by synthesizing the Y channel final enhanced image, the U channel image and the V channel image.

2. The image enhancement method according to claim 1, characterized in that: The CNN network training process includes: Acquire multiple initial images, reduce the brightness of the initial images to obtain images to be enhanced corresponding to the initial images, and use the Y channel images of the images to be enhanced as CNN training samples to construct a CNN training set; Build a CNN network; The CNN network is iteratively trained using the CNN training set.

3. The image enhancement method according to claim 2, characterized in that: During the CNN network training process, the mean square error between the training sample and the Y channel image of the initial image corresponding to the training sample is used as the loss function.

4. The image enhancement method according to claim 2 or 3, characterized in that: The CNN network includes a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, a third convolutional layer, a third activation layer, a fourth convolutional layer and a fourth activation layer which are connected in sequence.

5. The image enhancement method according to claim 4, characterized in that: The convolution kernel size of the first convolution layer is 9×9, the activation function of the first activation layer is LRelu, the convolution kernel size of the second convolution layer is 1×1, the activation function of the second activation layer is Relu, the convolution kernel size of the third and fourth convolution layers is 5×5, and the activation functions of the third and fourth activation layers are Relu.

6. The image enhancement method according to claim 1, characterized in that: The ANP network includes a first MLP module, an attention mechanism module, a normal distribution feature extraction module and a second MLP module; The output end of the first MLP module is connected to the input end of the attention mechanism module and the input end of the normal distribution feature acquisition module respectively, the output end of the attention mechanism module is connected to the input end of the second MLP module, and the output end of the normal distribution feature acquisition module is connected to the input end of the second MLP module.

7. The image enhancement method according to claim 6, characterized in that: The attention mechanism module adopts a multi-head attention mechanism.

8. An image enhancement device, characterized in that: include: An image acquisition module is used to acquire a Y channel image, a U channel image and a V channel image of a Y channel image in a YUV color space; The first enhancement module is used to input the Y channel image into a trained CNN network to obtain a Y channel preliminary enhanced image; A superpixel segmentation module performs superpixel segmentation on the Y channel preliminary enhanced image, and copies the segmentation result of the Y channel preliminary enhanced image to the Y channel image; The second enhancement module inputs the Y channel preliminary enhanced image after superpixel segmentation and the Y channel image after superpixel segmentation into the ANP network to obtain the final enhanced image of the Y channel. The specific process is as follows: Step A, assuming that both the Y channel preliminary enhanced image and the Y channel image are superpixel segmented into M image regions, the Y channel preliminary enhanced image and the M image regions of the Y channel image are in one-to-one correspondence, and M is a positive integer; Step B, inputting the first image area of ​​the Y channel image and the first image area of ​​the Y channel preliminary enhanced image into the ANP network to obtain the first iteration result; Step C, combining the nth image area of ​​the Y channel image with the n-1th iteration result to obtain a first combined image, combining the first image area of ​​the Y channel preliminary enhanced image to the nth image area to obtain a second combined image, and inputting the first combined image and the second combined image into the ANP network to obtain the nth iteration result; n is a positive integer, and 2≤n≤M; Step D, if n is equal to M, stop iteration and output the result of the nth iteration; if n is less than M, set n=n+1 and return to step C; The synthesis module synthesizes the Y channel final enhanced image, the U channel image and the V channel image to obtain the enhanced image.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the image enhancement method as described in any one of claims 1 to 7.

Images