Image beautification processing method, image beautification processing device and electronic equipment

Abstract

The present disclosure relates to the technical field of image processing, and provides a processing method and device for image beautification, a computer readable storage medium and an electronic device. The method comprises: pre-processing a to-be-beautified image to obtain first image data corresponding to the to-be-beautified image, wherein the to-be-beautified image is one or more of an image lacking dark detail and an image containing noise; inputting the first image data into a trained generative adversarial deep neural network, performing beautification processing on the first image data based on a generator network of the generative adversarial deep neural network to obtain second image data; and performing post-processing operation corresponding to the pre-processing operation on the second image data to obtain a beautified image, wherein the beautified image is one or more of an image with restored dark detail and an image with removed noise. The technical solution can improve the image processing efficiency and accurately restore the dark detail and eliminate the image noise.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method for beautifying an image, a device for beautifying an image, and an electronic device for implementing the method. Background Art

[0002] In low-light conditions (also known as "dark light environments"), if the camera's sensitivity is not adjusted, the resulting image will be blurry and lack detail in dark areas. Specifically, the dark areas of the image will lack detail, resulting in poor image quality. However, in low-light conditions, adjusting the camera's sensitivity will result in images with a significant amount of noise. While this can salvage some dark details, the image quality is also significantly compromised by the presence of excessive noise.

[0003] For the above-mentioned problem of image capture under insufficient light conditions, the relevant technology provides an exposure solution. Specifically, in a film camera, exposure refers to the process of placing a pre-coated photosensitive plate (Presensitized Plate, referred to as: PS plate) on the platemaking machine workbench and obtaining a latent or visible image by exposing the film. In a digital camera, exposure refers to the operation of generating a digital image by passing light through an image sensor. Among them, exposure is subject to the exposure time and light transmittance. When the exposure time is insufficient or the light transmittance is insufficient, the photo will be underexposed, and the details of the dark part of the image will be unclear or lost. When the exposure time is too long or the light transmittance is too large, the photo will be overexposed, making the details of the bright part of the image unclear or lost.

[0004] It can be seen that for images captured in a dark environment, the image beautification processing methods provided by related technologies have low processing efficiency.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of this disclosure. Summary of the Invention

[0006] The purpose of the present disclosure is to provide an image beautification processing method, an image beautification processing device, a computer storage medium and an electronic device, thereby improving the processing efficiency of images captured in a low-light environment at least to a certain extent.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0008] According to one aspect of the present disclosure, a method for image beautification is provided, comprising: preprocessing an image to be beautified to obtain first image data corresponding to the image to be beautified, wherein the image to be beautified is one or more of an image lacking dark details and an image containing noise; inputting the first image data into a trained generative adversarial deep neural network, and performing beautification processing on the first image data based on a generator network of the generative adversarial deep neural network to obtain second image data; and performing a post-processing operation corresponding to the preprocessing operation on the second image data to obtain a beautified image, wherein the beautified image is one or more of an image with restored dark details and an image with removed noise.

[0009] In some embodiments of the present disclosure, based on the aforementioned scheme, preprocessing the image to be beautified to obtain first image data corresponding to the image to be beautified includes: obtaining a pixel matrix of the image to be beautified; performing data type conversion on the pixel matrix to obtain a first matrix after data type normalization; and performing de-averaging and normalization on the first matrix to obtain first image data corresponding to the image to be beautified.

[0010] In some embodiments of the present disclosure, based on the aforementioned scheme, the post-processing operation corresponding to the pre-processing operation is performed on the second image data to obtain a beautified image, including: performing data type conversion on the second image data to obtain a second matrix; and performing averaging and truncation on the second matrix to obtain a pixel matrix corresponding to the beautified image, so as to generate the beautified image based on the pixel matrix.

[0011] In some embodiments of the present disclosure, based on the aforementioned scheme, before inputting the first image data into the trained generative adversarial deep neural network, the method further includes: obtaining multiple groups of training samples, wherein each group of training samples includes an input image and a target image; inputting the input image into the generator network in the generative adversarial deep neural network, and beautifying the input image based on the generator network to obtain a generated image; inputting the generated image, the input image, and the target image into the discriminator network in the generative adversarial deep neural network to obtain a first discrimination result on the pixel value of the generated image and a second discrimination result on the pixel value of the target image; and optimizing the parameters of the generative adversarial deep neural network according to the first discrimination result and the second discrimination result to obtain the image beautification model.

[0012] In some embodiments of the present disclosure, based on the aforementioned scheme, optimizing the parameters of the generative adversarial deep neural network according to the first discrimination result and the second discrimination result includes: optimizing the parameters of the generator network and optimizing the parameters of the discriminator network according to one or more of the pixel difference between the generated image and the target image, the difference between adjacent pixels in the generated image, and the pixel difference between multiple generated images output by the generator network.

[0013] In some embodiments of the present disclosure, based on the aforementioned scheme, optimizing the model parameters of the image beautification model according to the first discrimination result and the second discrimination result includes: keeping the parameters of the generator network unchanged, and optimizing the parameters of the discriminator network according to the pixel difference between the generated image and the target image; and keeping the optimized parameters of the discriminator network unchanged, and optimizing the parameters of the generator network according to one or more of the adjacent pixel differences in the generated image and the pixel differences between the multiple generated images output by the generator network.

[0014] In some embodiments of the present disclosure, based on the aforementioned solution, the first input image in the training sample is an image lacking dark details, and the first target image corresponding to the first input image is an image that restores dark details.

[0015] In some embodiments of the present disclosure, based on the aforementioned solution, the second input image in the training sample is an image containing noise, and the second target image corresponding to the second input image is an image with noise removed.

[0016] In some embodiments of the present disclosure, based on the aforementioned scheme, the generator network includes a convolution layer, a pooling layer, a pixel supplementation layer, a deconvolution layer and a pixel normalization layer.

[0017] In some embodiments of the present disclosure, based on the aforementioned solution, in the generator network, the upsampling parameters and the downsampling parameters are correspondingly equal.

[0018] In some embodiments of the present disclosure, based on the aforementioned solution, the discriminator network includes multiple convolutional layers.

[0019] According to one aspect of the present disclosure, a device for processing image beautification is provided, the device comprising: a preprocessing module, configured to preprocess an image to be beautified, and obtain first image data corresponding to the image to be beautified, wherein the image to be beautified is one or more of an image lacking dark details and an image containing noise; an image beautification processing module, configured to input the image data into a trained generative adversarial deep neural network, and perform beautification processing on the first image data based on a generator network of the generative adversarial deep neural network to obtain second image data; and a post-processing module, configured to perform a post-processing operation corresponding to the preprocessing operation on the second image data to obtain a generated image, wherein the generated image is one or more of an image with restored dark details and an image with removed noise.

[0020] In some embodiments of the present disclosure, based on the above solution, the preprocessing module includes:

[0021] an acquiring unit, configured to acquire a pixel matrix of the image to be beautified;

[0022] a first processing unit configured to perform data type conversion on the pixel matrix to obtain a first matrix with normalized data type;

[0023] The second processing unit is configured to perform de-averaging and normalization processing on the first matrix to obtain first image data corresponding to the image to be beautified.

[0024] In some embodiments of the present disclosure, based on the above solution, the post-processing module includes: a conversion unit and a third processing unit.

[0025] The above-mentioned conversion unit is configured to perform data type conversion on the second image data to obtain a second matrix; and the above-mentioned third processing unit is configured to perform averaging and truncation processing on the second matrix to obtain a pixel matrix corresponding to the beautified image, so as to generate the beautified image based on the pixel matrix.

[0026] In some embodiments of the present disclosure, based on the aforementioned solution, the image beautification processing device further includes a model training module for training the generative adversarial deep neural network, including: a sample acquisition unit, an image generation unit, an image discrimination unit, and a parameter optimization unit.

[0027] The above-mentioned sample acquisition unit is configured to: acquire multiple groups of training samples, wherein each group of training samples includes an input image and a target image; the above-mentioned image generation unit is configured to: input the input image into the generator network in the generative adversarial deep neural network, and beautify the input image based on the generator network to obtain a generated image; the above-mentioned image discrimination unit is configured to: input the generated image, the input image and the target image into the discriminator network in the generative adversarial deep neural network to obtain a first discrimination result on the pixel value of the generated image and a second discrimination result on the pixel value of the target image; and the above-mentioned parameter optimization unit is configured to: optimize the parameters of the generative adversarial deep neural network according to the first discrimination result and the second discrimination result to obtain the image beautification model.

[0028] In some embodiments of the present disclosure, based on the aforementioned scheme, the above-mentioned parameter optimization unit is specifically configured to: optimize the parameters of the generator network and optimize the parameters of the discriminator network according to one or more of the pixel difference between the generated image and the target image, the adjacent pixel difference in the generated image, and the pixel difference between the multiple generated images output by the generator network.

[0029] In some embodiments of the present disclosure, based on the above solution, the above model training module includes: a first training unit and a second training unit.

[0030] The above-mentioned first training unit is configured to: keep the parameters of the generator network unchanged, and optimize the parameters of the discriminator network according to the pixel difference between the generated image and the target image; and the above-mentioned second training unit is configured to: keep the optimized parameters of the discriminator network unchanged, and optimize the parameters of the generator network according to one or more of the adjacent pixel differences in the generated image and the pixel differences between the multiple generated images output by the generator network.

[0031] In some embodiments of the present disclosure, based on the aforementioned solution, the first input image in the training sample is an image lacking dark details, and the first target image corresponding to the first input image is an image that restores dark details.

[0032] In some embodiments of the present disclosure, based on the aforementioned solution, the second input image in the training sample is an image containing noise, and the second target image corresponding to the second input image is an image with noise removed.

[0033] In some embodiments of the present disclosure, based on the aforementioned scheme, the generator network includes a convolution layer, a pooling layer, a pixel supplementation layer, a deconvolution layer and a pixel normalization layer.

[0034] In some embodiments of the present disclosure, based on the aforementioned solution, in the generator network, the upsampling parameters and the downsampling parameters are correspondingly equal.

[0035] In some embodiments of the present disclosure, based on the aforementioned solution, the discriminator network includes multiple convolutional layers.

[0036] According to one aspect of the present disclosure, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image beautification processing method described in the first aspect is implemented.

[0037] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the image beautification processing method described in the first aspect above by executing the executable instructions.

[0038] As can be seen from the above technical solutions, the image beautification processing method, image beautification processing device, computer storage medium, and electronic device in the exemplary embodiments of the present disclosure have at least the following advantages and positive effects:

[0039] In the technical solutions provided by some embodiments of the present disclosure, the image to be beautified is preprocessed to obtain first image data corresponding to the image to be beautified, and the first image data is input into a trained generative adversarial deep neural network. The generator network based on the generative adversarial deep neural network performs beautification processing on the first image data, for example, convolution processing, deconvolution processing, and normalization processing on the first image data. Furthermore, the second image data is subjected to post-processing operations corresponding to the above-mentioned preprocessing operations to obtain a beautified image. Beautifying images by generating adversarial deep neural networks can improve image processing efficiency. At the same time, the technical solutions provided by the present disclosure can be used to correct and beautify images taken in dark environments. Specifically, for underexposed images taken in dark environments, their dark details can be restored, and for overexposed images taken in dark environments, their noise can be eliminated. In addition, it can also provide repair capabilities for unclear images caused by shaking when taking photos in dark environments.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0042] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solutions of the embodiments of the present disclosure can be applied;

[0043] Figure 2 A schematic diagram showing a flow chart of a method for image beautification in an exemplary embodiment of the present disclosure;

[0044] Figure 3 A schematic diagram of a process of an image preprocessing method according to an embodiment of the present disclosure is shown;

[0045] Figure 4 A schematic diagram of a process for training a generative adversarial deep neural network according to an embodiment of the present disclosure is shown;

[0046] Figure 5 A schematic diagram of an input image in a training sample according to an embodiment of the present disclosure is shown;

[0047] Figure 6 Shown with Figure 5 The input samples shown are schematic diagrams of target images in the same set of training samples;

[0048] Figure 7 A schematic diagram of an input image in a training sample according to an embodiment of the present disclosure is shown;

[0049] Figure 8 Shown with Figure 7 The input samples shown are schematic diagrams of target images in the same set of training samples;

[0050] Figure 9 A schematic diagram of the structure of a generator network according to an embodiment of the present disclosure is shown;

[0051] Figure 10 FIG2 shows a schematic diagram of the structure of a discriminator network according to an embodiment of the present disclosure;

[0052] Figure 11 A schematic diagram of a framework for a training method for generating adversarial deep neural networks according to an embodiment of the present disclosure is shown;

[0053] Figure 12 A schematic flow chart of a method for optimizing anti-neural network parameters according to an embodiment of the present disclosure is shown;

[0054] Figure 13 A flowchart of an image post-processing method according to an embodiment of the present disclosure is shown;

[0055] Figure 14 A structural diagram of a processing device for image beautification in an exemplary embodiment of the present disclosure is shown; and

[0056] Figure 15 A structural diagram of an electronic device in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0057] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art.

[0058] The features, structures, or characteristics described in the example implementations can be combined in any suitable manner in one or more implementations. In the following description, numerous specific details are provided to give a thorough understanding of implementations of the present disclosure. One skilled in the relevant art will recognize, however, that the implementations of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, etc. In other instances, well-known structures have not been described in detail so as not to obscure aspects of the present disclosure.

[0059] Furthermore, the block diagram of the embodiments shown in the figures can be a functional description of the embodiments rather than a structural description. That is, the functions described can be implemented in software or other functional means rather than in hardware. In other words, the block diagram of the embodiments is a functional description of the embodiments rather than a structural description of the embodiments.

[0060] The flowcharts shown in the figures are merely exemplary and do not necessarily include all of the content and operations / steps, nor do they necessarily need to be executed in the order described. For example, some operations / steps can be further broken down, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to actual conditions.

[0061] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present disclosure can be applied is shown.

[0062] As Figure 1As shown, system architecture 100 may include a mobile terminal 101, a network 102, and a server 103. The mobile terminal 101 may be a terminal device with a camera and a display screen, such as a mobile phone, a portable computer, or a tablet computer; the network 102 is a medium for providing a communication link between the mobile terminal 101 and the server 103. The network 102 may include various connection types, such as wired communication links, wireless communication links, etc. In the embodiment of the present disclosure, the network 102 between the mobile terminal 101 and the server 103 may be a wireless communication link, specifically a mobile network.

[0063] It should be understood that Figure 1 The number of terminals, networks, and servers shown is merely illustrative. Any number of terminals, networks, and servers may be used depending on implementation needs. For example, server 103 may be a server cluster consisting of multiple servers, or a server cluster consisting of multiple servers, which may be used to store information related to image beautification processing.

[0064] In one embodiment of the present disclosure, after the mobile terminal 101 obtains the image to be beautified, it sends the image to be beautified to the server 103; the server 103 preprocesses the image to be beautified and obtains first image data corresponding to the above-mentioned image to be beautified; the server 103 inputs the first image data into the trained generative adversarial deep neural network, and beautifies the first image data based on the generator network of the generative adversarial deep neural network; and further, the server 103 performs a post-processing operation corresponding to the pre-processing operation on the above-mentioned second image data to obtain a beautified image, and returns the beautified image to the mobile terminal 101.

[0065] In one embodiment of the present disclosure, after the mobile terminal 101 obtains the image to be beautified, it preprocesses the image to be beautified, obtains first image data corresponding to the above-mentioned image to be beautified, and sends the first image data to the server 103; the server 103 inputs the first image data into the trained generative adversarial deep neural network, and beautifies the first image data based on the generator network of the generative adversarial deep neural network; and further, the server 103 returns the second image data to the mobile terminal 101; the mobile terminal 101 performs a post-processing operation corresponding to the pre-processing operation on the above-mentioned second image data to obtain a beautified image.

[0066] It should be noted that the image beautification processing method provided in the embodiments of the present disclosure is generally executed by the server 103, and accordingly, the image beautification processing device is generally provided in the server 103. However, in other embodiments of the present disclosure, the terminal may also have similar functions as the server to perform the image beautification processing solution provided in the embodiments of the present disclosure.

[0067] In order to solve the problem of low efficiency in processing images captured in dark environments in the image beautification processing methods provided by related technologies, the inventors conducted the following attempts:

[0068] 1. A method for processing low-light images based on a convolutional neural network algorithm. Specifically, the method for processing low-light images based on a convolutional neural network algorithm is to design a convolutional network structure, extract image features through the convolutional structure, and iteratively train the corresponding indicators to obtain a method for generating a target image. The algorithm based on traditional image processing is a method for processing images based on basic computer graphics principles. This method can also be used in conjunction with a convolutional neural network algorithm to process images. Finally, the method based on high-performance camera hardware benefits from the improvement of hardware performance or indicators to improve the quality of photos in adverse environments. However, the method for processing low-light images based on a convolutional neural network algorithm is limited by its training method, and the target image it ultimately generates has poor effects in denoising, restoring dark details, and improving clarity.

[0069] 2. Methods for processing exposure and noise in photos taken in dark environments based on traditional image processing algorithms. Traditional image processing algorithms cannot intelligently handle overexposure or underexposure in photos taken in dark environments. The processing often results in underexposure in other areas that were originally properly exposed after overexposed areas have been processed. Furthermore, traditional image processing algorithms have weak denoising capabilities. Generally speaking, denoising with traditional image processing algorithms can lead to image blur and loss of detail.

[0070] 3. Methods based on high-performance camera hardware to improve image quality: Methods based on high-performance camera hardware rely on high-performance camera hardware, but these new hardware are often expensive, resulting in higher image processing costs.

[0071] In the technical solution provided by the present disclosure, an image beautification model is determined based on training a generative adversarial deep neural network, and the image to be processed is beautified by the image beautification model. According to the technical solution provided by the present disclosure, images taken in dark environments can be corrected and beautified. Specifically, for underexposed images taken in dark environments, the dark details can be restored, and for overexposed images taken in dark environments, the noise in the images can be eliminated. In addition, the technical solution provided by the present disclosure can provide repair capabilities for unclear images caused by shaking of photos taken in dark environments. At the same time, this solution has low cost for processing images and a wide range of applications.

[0072] Among them, the generative adversarial deep neural network is a type of neural network. Compared with traditional neural networks, its main feature is that in addition to having a generator network structure, it also has a discriminator network structure. The generator network is used to generate images, while the discriminator network is used to judge whether an image (including target images and generated images) is true or false. In the process of training the generative adversarial deep neural network, iterative training is performed by calculating the difference between the generated image and the target image, as well as the error in the discriminator network's judgment of the image. Furthermore, through the adversarial training process of the generator network and the discriminator network, the network parameters of the generator network are optimized, so that the generated image is close to the target requirements. Therefore, the generative adversarial deep neural network can generate higher-quality images because of the adversarial nature of its generator network and discriminator network.

[0073] In an embodiment of the present disclosure, a method for image beautification is provided, which at least to some extent overcomes the shortcomings of the related art or the inventor's above-mentioned attempts. The execution subject of the image beautification method provided in this embodiment can be a device with computing and processing functions.

[0074] Figure 2 A flowchart of a method for image beautification in an exemplary embodiment of the present disclosure is shown. Figure 2 The image beautification processing method provided in this embodiment includes:

[0075] Step S210: pre-processing the image to be beautified to obtain first image data corresponding to the image to be beautified, wherein the image to be beautified is one or more of an image lacking dark details and an image containing noise;

[0076] Step S220: inputting the first image data into the trained generative adversarial deep neural network, and performing beautification processing on the first image data based on the generator network of the generative adversarial deep neural network to obtain second image data; and

[0077] Step S230 , performing a post-processing operation corresponding to the pre-processing operation on the second image data to obtain a beautified image, wherein the beautified image is one or more of an image with restored dark details and an image with noise removed.

[0078] exist Figure 2In the technical solution provided by the illustrated embodiment, the image to be beautified is preprocessed to obtain first image data corresponding to the image to be beautified, and the first image data is input into a trained generative adversarial deep neural network. The generator network based on the generative adversarial deep neural network performs beautification processing on the first image data, for example, convolution processing, deconvolution processing, and normalization processing on the first image data. Furthermore, the second image data is subjected to post-processing operations corresponding to the above-mentioned preprocessing operations to obtain a beautified image. Beautifying images by generating adversarial deep neural networks can improve image processing efficiency. At the same time, the technical solution provided by the present disclosure can be used to correct and beautify images taken in dark environments. Specifically, for underexposed images taken in dark environments, their dark details can be restored, and for overexposed images taken in dark environments, their noise can be eliminated. In addition, the technical solution provided by the present disclosure can provide repair capabilities for unclear images caused by shaking when taking photos in dark environments.

[0079] The following Figure 2 The specific implementation of each step of the embodiment shown is described in detail:

[0080] In an exemplary embodiment, Figure 3 FIG2 shows a flow chart of an image preprocessing method according to an embodiment of the present disclosure, which can be used as a specific implementation of step S210. Figure 3 The image preprocessing method provided in this embodiment includes steps S310 to S330.

[0081] In step S310, a pixel matrix of the image to be beautified is obtained.

[0082] In an exemplary embodiment, the image to be enhanced is an image lacking dark details (i.e., a blurred image due to underexposure when photographed in a dark environment), and / or the image to be enhanced is an image containing noise (i.e., a noisy image due to overexposure when photographed in a dark environment). Furthermore, the pixel matrix is ​​a matrix composed of pixel values ​​of the image. To perform image correction processing, the pixel values ​​of the image to be enhanced are obtained, and then the pixel matrix corresponding to the image to be enhanced is determined.

[0083] In step S320 , data type conversion is performed on the pixel matrix to obtain a first matrix with normalized data type.

[0084] In an exemplary embodiment, after obtaining the pixel matrix of the image to be beautified, the data type of the pixel values ​​in the pixel matrix is ​​integer, and floating-point pixel data is obtained through data type conversion.

[0085] In step S330, de-averaging and normalization are performed on the first matrix to obtain first image data corresponding to the image to be beautified.

[0086] In a neural network, a large eigenvalue x will result in a large value for W*x+b (where W is the neural network's weight parameter and b is the neural network's bias parameter). This will cause the output of the activation function (such as ReLU) to have a small change in the corresponding value. This will lead to a vanishing gradient problem during backpropagation gradient calculation, resulting in very small parameter changes and difficulty fitting.

[0087] Therefore, in an exemplary embodiment, to reduce the difficulty of subsequent fitting, after obtaining the pixel matrix of the floating-point pixel data, it is necessary to perform a de-meaning process on it. Specifically, each dimension of the pixel matrix of the floating-point pixel data is subtracted from the mean of the corresponding dimension, so that each dimension of the floating-point pixel data is centered.

[0088] In an exemplary embodiment, in order to control the scales of various features within the same range and to obtain an optimal solution, variance normalization is performed on the data after the above-mentioned de-meaning processing.

[0089] pass Figure 3 The image preprocessing method provided in the illustrated embodiment obtains first image data corresponding to the image to be beautified. Furthermore, before inputting the first image data into the trained generative adversarial deep neural network in step S220, a generative adversarial deep neural network needs to be trained.

[0090] In an exemplary embodiment, Figure 4 FIG2 shows a flow chart of a method for training a generative adversarial deep neural network according to an embodiment of the present disclosure. Figure 4 The image preprocessing method provided in this embodiment includes steps S410 to S440.

[0091] In step S410, multiple groups of training samples are obtained, wherein each group of training samples includes an input image and a target image.

[0092] In this exemplary embodiment, each group of training samples includes a paired input image and a target image. Figure 5-Figure 8 Two sets of training samples according to an embodiment of the present disclosure are shown.

[0093] For example, the first input image in the training sample is an image lacking dark details, and the first target image corresponding to the first input image is an image with restored dark details. Figure 5 The image a with missing dark details is shown in the figure, and the target image in the same group is the image b with restored dark details (such as Figure 6 ).

[0094] For example, the second input image in the training sample is an image containing noise, and the second target image corresponding to the second input image is an image with noise removed. Figure 7 The image c containing noise is shown in the figure, and the target image in the same group is the image d with noise removed (as shown in the figure). Figure 8 ).

[0095] It can be seen that the input image in each set of training samples is the image to be beautified, and the target image is the image expected to be beautified.

[0096] By obtaining the above-mentioned images with missing dark details and the corresponding images with restored dark details, as well as obtaining the above-mentioned images with noise and the corresponding images with noise removed, as training data sets, an image beautification model that can restore dark details and remove noise is trained.

[0097] In step S420, the input image is input into a generator network in a generative adversarial deep neural network, and the input image is beautified based on the generator network to obtain a generated image; and in step S430, the generated image, the input image and the target image are input into a discriminator network in a generative adversarial deep neural network to obtain a first discrimination result on the pixel value of the generated image and a second discrimination result on the pixel value of the target image.

[0098] In an exemplary embodiment, before introducing the training process of the generative adversarial deep neural network, the network structure of the generative adversarial deep neural network used in this embodiment is first described. Specifically, Figure 9 and Figure 10 The network structures of the generator network and the discriminator network of the generative adversarial deep neural network according to an embodiment of the present disclosure are respectively shown.

[0099] refer to Figure 9 The generator network provided in this embodiment includes a convolution layer, a pooling layer, a pixel complement layer, a deconvolution layer, and a pixel normalization layer. In the neural network architecture, convolution and pooling operations can be used to extract deep features of an image. However, compared with the input image, multiple convolution and pooling operations cause the obtained feature map to continuously decrease, resulting in information loss. Therefore, in order to reduce information loss, in this embodiment, for each downsampling (such as Figure 9 , from 91 to 910), using the corresponding upsampling (such as Figure 9In the example, 911 to 920) refers to the size of the restored input image. Therefore, in this embodiment, in the generator network, the upsampling parameters are equal to the downsampling parameters, so that the image is abbreviated in the upsampling stage and the image is correspondingly amplified in the downsampling stage. That is, in this embodiment, the generator network adopts the Unet network structure (such as Figure 9 The dotted line in the image is in a "U" shape), which can reduce the loss of original information during network transmission, thereby improving the structural inconsistency or semantic inconsistency in the output process of the second half of the network, and ultimately improving the quality of the beautified image.

[0100] In an exemplary embodiment, Figure 9 The generator network shown also uses the tanh function as the activation function in upsampling 920.

[0101] In an exemplary embodiment, referring to Figure 10 The discriminator network Discriminator of the generative adversarial deep neural network model provided in this embodiment includes multiple convolutional layers, wherein the sigmoid function is used as the activation function in the downsampling 1110.

[0102] In an exemplary embodiment, Figure 9 and Figure 10 The network structures of the generator network and the discriminator network are shown schematically only. In actual operation, the network structure can be adjusted according to actual needs, so the network structure of the generative adversarial deep neural network for image beautification disclosed in this disclosure is scalable.

[0103] In an exemplary embodiment, Figure 11 FIG2 shows a schematic diagram of a framework of a training method for generating an adversarial deep neural network according to an embodiment of the present disclosure. Figure 11 , the generative adversarial deep neural network includes a generator network G1110 and a discriminator network D1120.

[0104] The generator network G is used to beautify the image to be beautified (input image) (for example: convolution processing, deconvolution processing and normalization processing, etc.), and output the image after beautification processing as the generated image. The discriminator network D is used to receive the above-mentioned generated image and the target image in the same group as the above-mentioned image to be beautified (input image), and then judge whether an image (including the target image and the generated image) is true or false. At the same time, the training goal of the discriminator network D is to judge the target image as true and the generated image as false. The training goal of the generator network G is to beautify the image to be beautified (input image) to obtain a generated image that is judged as true by the discriminator network, that is, to make the generated image closer to the target image to achieve a fake-real effect.

[0105] Specifically, refer to Figure 11 , after preprocessing the input image 111 in a set of training samples, it is input to the generator network G1110 (e.g. Figure 9 The generator network G1110 beautifies the image 111 and generates the generated image 112. Furthermore, the input image 111, the target image 113 in the same set of training samples, and the generated image 112 are input to the discriminator network D1120 (e.g. Figure 10 Specifically, the discriminator network D1120 obtains a first discrimination result regarding the pixel value of the generated image 111 based on the comparison between the generated image 112 and the input image 111, and the discriminator network D1120 obtains a second discrimination result regarding the pixel value of the target image 113 based on the comparison between the target image 113 and the input image 111.

[0106] Continue to refer Figure 4 In step S440, the parameters of the generative adversarial deep neural network are optimized according to the first discrimination result and the second discrimination result to obtain the image beautification model.

[0107] In an exemplary embodiment, the parameters of the generator network and the parameters of the discriminator network are optimized based on the pixel difference between the generated image 112 and the target image 113, the difference between adjacent pixels in the generated image 112 and / or the pixel difference between the multiple generated images output by the generator network G1110.

[0108] In an exemplary embodiment, the data corresponding to the input image 111 input to the generator network G1110 is z, and the data corresponding to the generated image 112 output by the generator network G1110 is G(z; θ g ). The data x corresponding to the input target image 113 of the discriminator network D1120 and the data corresponding to the generated image 112 are G(z; θ g ), outputs a binary confidence D(x;θ) indicating that the input is a target image or a generated image d ). Furthermore, in this embodiment, the following loss function can be defined:

[0109]

[0110] Among them, x i 、z i They are the data corresponding to the target image 113 and the data corresponding to the generated image 112. The optimization goal is: min{G[max(D(Loss))]}.

[0111] In an exemplary embodiment, the discriminator network D1120 is first trained, that is, the parameters θ of the discriminator network D1120 are iteratively optimized. d , so that the discrimination accuracy of the discriminator network D1120 reaches the preset threshold. Further, the generator network G1110 is retrained, that is, the parameters θ of the generator network G1110 are iteratively optimized. g , so that the generator network G1110 can generate the beautified images expected by the user.

[0112] For example, Figure 12 FIG2 shows a flow chart of a method for optimizing anti-neural network parameters according to an embodiment of the present disclosure. Figure 12 The method provided in this embodiment includes step S1210 and step S1220.

[0113] In step S1210, the parameters of the generator network are kept unchanged, and the parameters of the discriminator network are optimized according to the pixel difference between the generated image and the target image.

[0114] For example, the data z corresponding to the generated image is i Used to reflect the pixel value of the generated image and the data x corresponding to the above target image i The pixel values ​​of the target image are used to reflect the pixel values ​​of the target image. Further, the parameters θ of the discriminator network are optimized according to the pixel difference between the generated image and the target image. d A specific implementation method may be:

[0115] According to the data x corresponding to the target image 113 in the M group of training samples i =x 1 ,,,x m , and the data z corresponding to the generated image 112 in the above M groups of training samples i =z 1 ,,,z m Calculate the following formula:

[0116]

[0117] Then, the parameters θ of the above discriminator network D are updated by gradient descent method d .

[0118] In step S1220, the optimized parameters of the discriminator network are kept unchanged, and the parameters of the generator network are optimized according to one or more of the adjacent pixel differences in the generated image and the pixel differences between the multiple generated images output by the generator network.

[0119] For example, the adjacent pixel difference in the generated image refers to the difference between two adjacent pixels in the same generated image. The adjacent pixel difference in the same generated image can help the generator network determine the location of noise / the location of dark details, and thus accurately beautify the image. i is a pixel matrix, then the data z corresponding to the image is generated i It can be used to reflect the difference between adjacent pixels in the above generated image.

[0120] Exemplarily, the pixel difference between the multiple generated images output by the generator network refers to: for the same input image, the multiple generated images output by the generator network G (such as z i and z 1+i ). For example, generate image z i The pixel value of point A and the generated image z 1+i The pixel value of point A in the image. The pixel difference at the same position of multiple generated images corresponding to the same input image can help measure the image beautification ability of the generator network, and thus beautify the image with high quality. Among them, the data z corresponding to the multiple generated images 1 ,,,z m It can be used to reflect the pixel differences between multiple generated images output by the above generator network.

[0121] Furthermore, the parameters θ of the generator network are optimized based on the adjacent pixel differences in the generated image and / or the pixel differences between the multiple generated images output by the generator network. g A specific implementation method may be:

[0122] For example, the data z corresponding to the generated image 112 in the above M groups of training samples is i =z 1 ,,,z m Calculate the following formula:

[0123]

[0124] Then, the parameters θ of the generator network G are updated by gradient descent g .

[0125] according to Figure 12 The technical solution provided by the embodiment shown is to first train the discriminator network D to ensure that the discrimination accuracy of the discriminator network reaches a preset threshold. Then, the generator network G is trained based on the trained discriminator network. When training the generator network G, since the target image is not generated by the generator network, logD(x i ) is set to 0, so only [logD(1-G(z i ))] gradient.

[0126] According to the technical solutions provided in the above embodiments, during the training of a generative adversarial deep neural network, iterative training is performed by calculating the difference between the generated image and the target image, as well as the error in the discriminator network's judgment of the image. Furthermore, through the adversarial training process between the generator network and the discriminator network, the network parameters of the generator network are optimized, so that the generated image approaches the target requirement.

[0127] In an exemplary embodiment, the trained generative adversarial deep neural network is used as an image beautification model to beautify the image to be beautified. Figure 2 In step S220, the first image data is input into the trained generative adversarial deep neural network, and the first image data is beautified based on the generator network of the generative adversarial deep neural network to obtain second image data.

[0128] In an exemplary embodiment, after the image to be beautified undergoes the aforementioned preprocessing, the first image data is obtained and input into the image beautification model (i.e., the trained generative adversarial deep neural network). The generator network of the trained generative adversarial deep neural network then beautifies the first image data, thereby outputting beautified second image data. Specifically, the second image data is pixel data corresponding to the beautified image.

[0129] In an exemplary embodiment, referring to Figure 2 In step S230, a post-processing operation corresponding to the pre-processing operation is performed on the second image data to obtain a beautified image, wherein the beautified image is one or more of an image with restored dark details and an image with removed noise.

[0130] In an exemplary embodiment, the beautified image refers to an image in which dark details are restored (i.e., an image after repairing a blurred image caused by underexposure when shooting in a dark environment), and / or, the beautified image refers to an image in which noise is removed (i.e., an image after repairing a noisy image caused by overexposure when shooting in a dark environment).

[0131] In an exemplary embodiment, Figure 13 FIG2 shows a flow chart of an image post-processing method according to an embodiment of the present disclosure, which can be used as a specific implementation of step S230. Figure 13 The image preprocessing method provided in this embodiment includes step S1310 and step S1320.

[0132] In step S1310, the data type of the second image data is converted to obtain a second matrix; and in step S1320, the second matrix is ​​averaged and truncated to obtain a pixel matrix corresponding to the beautified image, so as to generate the beautified image based on the pixel matrix.

[0133] In an exemplary embodiment, the post-processing process corresponds to the pre-processing process described above.

[0134] Exemplarily, because the pixel matrix corresponding to the image to be enhanced undergoes data type conversion during the preprocessing process, the postprocessing process requires the opposite data type conversion. For example, if the data type of the pixel matrix of image s to be enhanced is converted from integer to floating point during the preprocessing process, the data type of the second image after the enhancement process will be floating point, requiring a data type conversion to integer during the postprocessing process. This ensures that the pixel data types before and after the enhancement remain unchanged, while only dark details and noise are accurately restored, thereby improving the enhanced image quality.

[0135] The image beautification method based on a generative adversarial deep neural network provided by this disclosure achieves high clarity, restored dark details, and accurate noise removal. It also mitigates blurring caused by jitter in dark environments. After receiving the output, users can customize the optimization intensity or perform other photo post-processing. This method shortens the user's photo processing time, does not rely on expensive video or photography hardware, and offers high scalability and portability.

[0136] Those skilled in the art will appreciate that all or part of the steps for implementing the above-mentioned embodiments are implemented as computer programs executed by a processor (including a CPU and a GPU). For example, the training of the above-mentioned generative adversarial deep neural network is implemented by a GPU, or based on the trained generative adversarial deep neural network, the beautification processing of the image to be beautified is implemented using a CPU or a GPU. When the computer program is executed by the processor, the above-mentioned functions defined by the above-mentioned method provided in the present disclosure are performed. The program can be stored in a computer-readable storage medium, which can be a read-only memory, a disk or an optical disk, etc.

[0137] Furthermore, it should be noted that the aforementioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the aforementioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0138] The following describes an embodiment of an image beautification processing apparatus of the present disclosure, which can be used to perform the image beautification processing method of the present disclosure.

[0139] Figure 14 A structural schematic diagram of an image beautification processing apparatus in an exemplary embodiment of the present disclosure is shown. As shown in the figure, the image beautification processing apparatus 1400 includes a preprocessing module 1401, an image beautification processing module 1402, and a post-processing module 1403. Among them: Figure 14

[0140] The preprocessing module 1401 is configured to pre-process a to-be-beautified image to obtain first image data corresponding to the to-be-beautified image, wherein the to-be-beautified image is one or more of an image lacking dark detail and an image containing noise; the image beautification processing module 1402 is configured to input the image data into a trained generative adversarial deep neural network, perform beautification processing on the first image data based on a generator network of the generative adversarial deep neural network, and obtain second image data; and the post-processing module 1403 is configured to perform post-processing operation corresponding to the preprocessing operation on the second image data to obtain a generated image, wherein the generated image is one or more of an image restoring dark detail and an image removing noise.

[0141] In an exemplary embodiment, the preprocessing module 1401 described above includes an acquisition unit, a first processing unit, and a second processing unit. Among them: the acquisition unit is configured to acquire a pixel matrix of the to-be-beautified image; the first processing unit is configured to perform data type conversion on the pixel matrix to obtain a first matrix after data type normalization; and the second processing unit is configured to perform de-meaning processing and normalization processing on the first matrix to obtain the first image data corresponding to the to-be-beautified image.

[0142] In an exemplary embodiment, the post-processing module 1403 includes a conversion unit and a third processing unit. Among them: the conversion unit is configured to perform data type conversion on the second image data to obtain a second matrix; and the third processing unit is configured to perform mean-adding processing and truncation processing on the second matrix to obtain a pixel matrix corresponding to the beautified image, so as to generate the beautified image according to the pixel matrix.

[0143] ​In the example embodiment, the image beautification processing apparatus 1400 further comprises a model training module for training the generative adversarial deep neural network, comprising a sample acquisition unit, an image generation unit, an image discrimination unit and a parameter optimization unit. The sample acquisition unit is configured to acquire a plurality of sets of training samples, wherein each set of training samples comprises an input image and a target image. The image generation unit is configured to input the input image into a generator network in the generative adversarial deep neural network, and perform beautification processing on the input image based on the generator network to obtain a generated image. The image discrimination unit is configured to input the generated image, the input image and the target image into a discriminator network in the generative adversarial deep neural network to obtain a first discrimination result about the pixel values of the generated image and a second discrimination result about the pixel values of the target image. The parameter optimization unit is configured to optimize the parameters of the generative adversarial deep neural network according to the first discrimination result and the second discrimination result to obtain the image beautification model.

[0144] In the example embodiment, the parameter optimization unit is configured to optimize the parameters of the generator network and optimize the parameters of the discriminator network according to one or more of the pixel difference between the generated image and the target image, the adjacent pixel difference in the generated image and the pixel difference between a plurality of generated images output by the generator network.

[0145] In the example embodiment, the model training module comprises a first training unit and a second training unit. The first training unit is configured to keep the parameters of the generator network unchanged, and optimize the parameters of the discriminator network according to the pixel difference between the generated image and the target image. The second training unit is configured to keep the optimized parameters of the discriminator network unchanged, and optimize the parameters of the generator network according to one or more of the adjacent pixel difference in the generated image and the pixel difference between a plurality of generated images output by the generator network.

[0146] In the example embodiment, the first input image in the training sample is an image lacking dark detail, and the first target image corresponding to the first input image is an image with restored dark detail.

[0147] In the example embodiment, the second input image in the training sample is an image containing noise, and the second target image corresponding to the second input image is an image with removed noise.

[0148] In the example embodiment, the generator network comprises a convolution layer, a pooling layer, a pixel supplement layer, a deconvolution layer and a pixel normalization layer.

[0149] In an exemplary embodiment, in the generator network, upsampling parameters and downsampling parameters are correspondingly equal.

[0150] In an exemplary embodiment, the discriminator network includes a plurality of convolutional layers.

[0151] The specific details of each unit in the above-mentioned image beautification processing device have been described in detail in the corresponding image beautification processing method, and therefore will not be repeated here.

[0152] Figure 15 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present disclosure is shown.

[0153] It should be noted that Figure 15 The computer system 1500 of the electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0154] like Figure 15 As shown, the computer system 1500 includes a processor 1501, wherein the processor 1501 may include: a graphics processing unit (GPU), a central processing unit (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1502 or the program loaded from the storage part 1508 to the random access memory (RAM) 1503. Various programs and data required for system operation are also stored in the RAM 1503. The processor (GPU / CPU) 1501, ROM 1502 and RAM 1503 are connected to each other via a bus 1504. An input / output (I / O) interface 1505 is also connected to the bus 1504.

[0155] The following components are connected to the I / O interface 1505: an input section 1506 including a keyboard, a mouse, and the like; an output section 1507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 1508 including a hard disk; and a communication section 1509 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 1509 performs communication processing via a network such as the Internet. A drive 1510 is also connected to the I / O interface 1505 as needed. Removable media 1511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 1510 as needed, so that computer programs read from the removable media can be installed in the storage section 1508 as needed.

[0156] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1509, and / or installed from the removable medium 811. When the computer program is executed by the processor (GPU / CPU) 1501, the various functions defined in the system of the present application are performed. In some embodiments, the computer system 1500 may also include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0157] It should be noted that the computer-readable medium shown in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0159] The units described in the embodiments of the present disclosure can be implemented in the form of software, or can be implemented in the form of hardware, or can be implemented in the form of a combination of software and hardware. The units described can also be arranged in a processor. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0160] As another aspect, the present disclosure also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the method described in the above embodiments.

[0161] For example, the electronic device can implement the steps shown in Figure 2 S210, pre-processing a to-be-beautified image to obtain first image data corresponding to the to-be-beautified image, wherein the to-be-beautified image is one or more of an image lacking dark part details and an image containing noise; S220, inputting the first image data into a trained generative adversarial deep neural network, performing beautification processing on the first image data based on a generator network of the generative adversarial deep neural network, to obtain second image data; and S230, performing post-processing operation corresponding to the pre-processing operation on the second image data, to obtain a beautified image, wherein the beautified image is one or more of an image with restored dark part details and an image with removed noise.

[0162] For another example, the electronic device can implement each step as shown in Figures 3 to 13

[0163] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units for embodiment.

[0164] ​Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0165] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0166] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for image beautification, characterized in that: The method comprises: Performing a preprocessing operation on the image to be beautified to obtain first image data corresponding to the image to be beautified, wherein the image to be beautified is one or more of an image lacking dark details and an image containing noise; Inputting the first image data into a trained generative adversarial deep neural network included in an image beautification model, and performing beautification processing on the first image data based on a generator network of the generative adversarial deep neural network to obtain second image data; performing a post-processing operation corresponding to the pre-processing operation on the second image data to obtain a beautified image, wherein the beautified image is one or more of an image with restored dark details and an image with noise removed; the post-processing operation includes adjusting a beautification intensity of the second image data according to a user setting; Before inputting the first image data into the trained generative adversarial deep neural network, the method further includes: Acquire multiple groups of training samples, wherein each group of training samples includes an input image and a target image; Input the input image into a generator network in a generative adversarial deep neural network, and perform beautification processing on the input image based on the generator network to obtain a generated image; the generator network includes n layers of upsampling layers and n layers of downsampling layers connected in sequence, the upsampling parameters of the i-th upsampling layer are equal to the downsampling parameters of the n-i+1-th downsampling layer, and i∈[1,n]; Inputting the generated image, the input image, and the target image into a discriminator network in a generative adversarial deep neural network to obtain a first discrimination result regarding a pixel value of the generated image and a second discrimination result regarding a pixel value of the target image; Optimizing the parameters of the generative adversarial deep neural network according to the first discrimination result and the second discrimination result to obtain the image beautification model; The step of optimizing the parameters of the generative adversarial deep neural network according to the first discrimination result and the second discrimination result to obtain the image beautification model includes: Keeping the parameters of the generator network unchanged, iteratively optimize the parameters of the discriminator network according to the pixel difference between the generated image and the target image until the discrimination accuracy of the discriminator network reaches a preset threshold; Keeping the optimized parameters of the discriminator network unchanged, iteratively optimize the parameters of the generator network according to one or more of the adjacent pixel differences of the generated image and the pixel differences between the multiple generated images output by the generator network; wherein the adjacent pixel differences of the generated image are used to represent: the pixel differences between adjacent pixels in the same generated image; the pixel differences between the multiple generated images are used to represent: the pixel differences between pixels at the same position in the multiple generated images generated for the same input image.

2. The image beautification processing method according to claim 1, characterized in that: The preprocessing operation of the image to be beautified to obtain first image data corresponding to the image to be beautified includes: Obtaining a pixel matrix of the image to be beautified; Performing data type conversion on the pixel matrix to obtain a first matrix after data type normalization; De-averaging and normalization are performed on the first matrix to obtain first image data corresponding to the image to be beautified.

3. The image beautification processing method according to claim 2, characterized in that: The performing a post-processing operation corresponding to the pre-processing operation on the second image data to obtain a beautified image includes: performing data type conversion on the second image data to obtain a second matrix; The second matrix is ​​subjected to averaging and truncation processing to obtain a pixel matrix corresponding to the beautified image, so as to generate the beautified image according to the pixel matrix.

4. The image beautification processing method according to any one of claims 1 to 3, characterized in that: The first input image in the training sample is an image lacking dark details, and the first target image corresponding to the first input image is an image with restored dark details.

5. The image beautification processing method according to claim 4, characterized in that: The second input image in the training sample is an image containing noise, and the second target image corresponding to the second input image is an image with noise removed.

6. The image beautification processing method according to any one of claims 1 to 3, characterized in that: The discriminator network includes multiple convolutional layers.

7. An image beautification processing device, characterized in that: The device comprises: a preprocessing module configured to perform a preprocessing operation on the image to be beautified to obtain first image data corresponding to the image to be beautified, wherein the image to be beautified is one or more of an image lacking dark details and an image containing noise; An image beautification processing module is configured to input the first image data into a trained generative adversarial deep neural network included in an image beautification model, and perform beautification processing on the first image data based on a generator network of the generative adversarial deep neural network to obtain second image data; the generator network includes n layers of upsampling layers and n layers of downsampling layers connected in sequence, the upsampling parameters of the i-th upsampling layer are equal to the downsampling parameters of the n-i+1-th downsampling layer, and i∈[1,n]; a post-processing module configured to perform a post-processing operation corresponding to the pre-processing operation on the second image data to obtain a generated image, wherein the generated image is one or more of an image with restored dark details and an image with noise removed; the post-processing operation includes adjusting a beautification intensity of the second image data according to a user setting; A sample acquisition unit is configured to acquire multiple groups of training samples, wherein each group of training samples includes an input image and a target image; an image generation unit, configured to input the input image into a generator network in a generative adversarial deep neural network, and perform beautification processing on the input image based on the generator network to obtain a generated image; An image discrimination unit is configured to input the generated image, the input image, and the target image into a discriminator network in a generative adversarial deep neural network, and obtain a first discrimination result regarding the pixel value of the generated image and a second discrimination result regarding the pixel value of the target image; a parameter optimization unit, configured to optimize the parameters of the generative adversarial deep neural network according to the first discrimination result and the second discrimination result to obtain the image beautification model; The step of optimizing the parameters of the generative adversarial deep neural network according to the first discrimination result and the second discrimination result to obtain the image beautification model includes: Keeping the parameters of the generator network unchanged, iteratively optimize the parameters of the discriminator network according to the pixel difference between the generated image and the target image until the discrimination accuracy of the discriminator network reaches a preset threshold; Keeping the optimized parameters of the discriminator network unchanged, iteratively optimize the parameters of the generator network according to one or more of the adjacent pixel differences of the generated image and the pixel differences between the multiple generated images output by the generator network; wherein the adjacent pixel differences of the generated image are used to represent: the pixel differences between adjacent pixels in the same generated image; the pixel differences between the multiple generated images are used to represent: the pixel differences between pixels at the same position in the multiple generated images generated for the same input image.

8. The image beautification processing device according to claim 7, characterized in that: The preprocessing module includes: an acquiring unit, configured to acquire a pixel matrix of the image to be beautified; a first processing unit configured to perform data type conversion on the pixel matrix to obtain a first matrix with normalized data type; The second processing unit is configured to perform de-averaging and normalization processing on the first matrix to obtain first image data corresponding to the image to be beautified.

9. The image beautification processing device according to claim 8, characterized in that: The post-processing module comprises: a conversion unit, configured to perform data type conversion on the second image data to obtain a second matrix; The third processing unit is configured to perform averaging and truncation processing on the second matrix to obtain a pixel matrix corresponding to the beautified image, so as to generate the beautified image according to the pixel matrix.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image beautification method according to any one of claims 1 to 6 is implemented.

11. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the image beautification processing method according to any one of claims 1 to 6.

12. A computer program product, characterized in that The invention comprises a computer program carried on a computer-readable medium, wherein the computer program contains program codes for executing the image beautification processing method according to any one of claims 1 to 6.

Images