Image processing method, device, electronic equipment, storage medium and system

Abstract

The present disclosure relates to an image processing method and device, electronic equipment and storage medium, and relates to the technical field of image processing. In the present disclosure, a first image to be beautified is obtained. The quality of the first image is detected to obtain the brightness quality and dynamic range quality of the first image. In the case where the brightness quality and / or the dynamic range quality does not satisfy a preset condition, the brightness and / or the dynamic range of the first image are processed to obtain a second image whose brightness quality and dynamic range quality both satisfy the preset condition. The second image is input into a pre-trained image beautification model to obtain reference feature information of the second image output by the image beautification model, the reference feature information being feature information for improving the aesthetic standard of the second image. The reference feature information is fused into the second image to obtain a third image after beautification. In the image beautification process, problems such as dark light, weak light and backlight in the image can be effectively solved, and the image quality can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, electronic device, storage medium and system. Background Technology

[0002] With the widespread adoption of mobile devices and their performance gradually improving, capturing images and videos using these devices has become a new internet culture. Consequently, users are demanding higher and higher standards for portrait beautification technology and effects.

[0003] During the photography process, due to the performance of the terminal equipment, shooting techniques, and environmental factors, images often suffer from problems such as low light, weak light, and backlighting, resulting in poor overall image quality. Related image processing technologies still have certain limitations, typically only able to beautify imperfections in the portrait, such as smoothing the skin, removing wrinkles, and slimming the face, but unable to further improve the overall image quality. Summary of the Invention

[0004] This disclosure provides an image processing method, apparatus, electronic device, and storage medium that can solve the problem of being unable to further improve the image capture quality.

[0005] The technical solution of this disclosure is as follows:

[0006] According to a first aspect of the present disclosure, an image processing method is provided, comprising: acquiring a first image to be beautified; performing quality detection on the first image to obtain the brightness quality and dynamic range quality of the first image; if the brightness quality and / or dynamic range quality do not meet preset conditions, processing the brightness and / or dynamic range of the first image to obtain a second image in which both brightness quality and dynamic range quality meet preset conditions; inputting the second image into a pre-trained image beautification model to obtain reference feature information of the second image output by the image beautification model, wherein the reference feature information is used to improve the aesthetic standard of the second image; and fusing the reference feature information into the second image to obtain a beautified third image.

[0007] Optionally, the reference feature information is fused into the second image to obtain the beautified third image. This includes: performing N downsampling operations on the second image to obtain N feature information of different sizes to be processed, where N is greater than or equal to 1. The reference feature information and the N feature information of different sizes are then fused N times sequentially to obtain N fusion results, and the Nth fusion result is used as the third image. Specifically, the first fusion result is the result obtained by fusing the reference feature information with the Nth feature information to be processed, and the i-th fusion result is the result obtained by fusing the (i-1)-th fusion result with the (N+1-i)th feature information to be processed.

[0008] Optionally, based on the reference feature information and N feature information of different sizes to be processed, N fusion operations are performed sequentially to obtain N fusion results, including: multiplying the reference feature information by the gamma value of the Nth feature information to obtain the first multiplication result; adding the first multiplication result to the beta value of the Nth feature information to obtain the first addition result; and performing an upsampling operation on the first addition result to determine the first fusion result. Then, multiplying the (i-1)th fusion result by the gamma value of the (N+1-i)th feature information to obtain the i-th multiplication result; adding the i-th multiplication result to the beta value of the (N+1-i)th feature information to obtain the i-th addition result; and performing an upsampling operation on the i-th addition result to obtain the i-th fusion result.

[0009] Optionally, quality detection is performed on the first image to obtain its brightness quality and dynamic range quality. This includes: segmenting the first image into multiple sub-images of the same size; determining the encoding information for each sub-image, which includes one or more of hash encoding information, content encoding information, style encoding information, and location encoding information. The content encoding information is used to characterize image content features, the style encoding information is used to characterize image style features, and the location encoding information is used to characterize the location distribution features of image content. The encoding information of each sub-image is input into a pre-trained quality detection model to obtain the brightness quality and dynamic range quality of the first image output by the quality detection model.

[0010] Optionally, if the brightness quality does not meet preset conditions, the brightness and / or dynamic range of the first image are processed to obtain a second image whose brightness and dynamic range both meet preset conditions. This includes: performing a first convolution operation on the first image to obtain a first guide map; downsampling the first image to obtain a first sampled image; inputting the first sampled image into a first neural network to obtain first mapping data output by the first neural network, the first mapping data including the brightness features of each pixel in the first image; and obtaining the second image based on the first guide map and the first mapping data.

[0011] Optionally, if the dynamic range quality does not meet the preset conditions, the brightness and / or dynamic range of the first image are processed to obtain a second image whose brightness and dynamic range both meet the preset conditions. This includes: performing a second convolution operation on the image to be processed to obtain a second guide image, and performing a third convolution operation on the image to be processed to obtain a third guide image. The convolution kernels corresponding to the second and third convolution operations are different. The image to be processed is either the first image or an image that has undergone brightness processing. The grayscale information and Gaussian blur information of each pixel in the image to be processed are determined. The second image is obtained based on the second guide image, the third guide image, and the grayscale information and Gaussian blur information of each pixel in the image to be processed.

[0012] Optionally, determining the grayscale information and Gaussian blur information of each pixel in the image to be processed includes: downsampling the image to be processed to obtain a second sampled image; performing grayscale processing and Gaussian blur processing on the second sampled image to obtain a grayscale image and a Gaussian blur image corresponding to the second sampled image; and inputting the grayscale image and the Gaussian blur image into a second neural network to obtain second mapping information and third mapping information output by the first neural network. The second mapping information includes the grayscale information of each pixel in the image to be processed, and the third mapping information includes the Gaussian blur information of each pixel in the image to be processed.

[0013] Optionally, the first image is an image that includes the facial region of the user object.

[0014] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising: an image acquisition unit configured to acquire a first image to be beautified; a quality detection unit configured to perform quality detection on the first image to obtain brightness quality and dynamic range quality of the first image; an image determination unit configured to process the brightness and / or dynamic range of the first image if the brightness quality and / or dynamic range quality do not meet preset conditions, to obtain a second image in which both brightness quality and dynamic range quality meet preset conditions; a feature determination unit configured to input the second image into a pre-trained image beautification model to obtain reference feature information of the second image output by the image beautification model, wherein the reference feature information is used to improve the aesthetic standard of the second image; and a feature fusion unit configured to fuse the reference feature information into the second image to obtain a beautified third image.

[0015] Optionally, the feature fusion unit is specifically configured to perform: N downsampling operations on the second image to obtain N feature information to be processed at different sizes, where N is greater than or equal to 1. Then, it performs N fusion operations sequentially on the reference feature information and the N feature information to be processed at different sizes to obtain N fusion results, and uses the Nth fusion result as the third image. Specifically, the first fusion result is the result obtained by fusing the reference feature information with the Nth feature information to be processed, and the i-th fusion result is the result obtained by fusing the (i-1)-th fusion result with the (N+1-i)th feature information to be processed.

[0016] Optionally, the feature fusion unit is specifically configured to perform the following operations: multiplying the reference feature information by the gamma value of the Nth feature information to obtain the first multiplication result; adding the first multiplication result to the beta value of the Nth feature information to obtain the first addition result; performing an upsampling operation on the first addition result to determine the first fusion result. Then, multiplying the (i-1)th fusion result by the gamma value of the (N+1-i)th feature information to obtain the i-th multiplication result; adding the i-th multiplication result to the beta value of the (N+1-i)th feature information to obtain the i-th addition result; and performing an upsampling operation on the i-th addition result to obtain the i-th fusion result.

[0017] Optionally, the quality detection unit is specifically configured to perform the following: Segment the first image into multiple sub-images of the same size. Determine the encoding information for each sub-image, which includes one or more of hash encoding information, content encoding information, style encoding information, and location encoding information. The content encoding information is used to characterize the image content features, the style encoding information is used to characterize the image style features, and the location encoding information is used to characterize the image content location distribution features. Input the encoding information of each sub-image into a pre-trained quality detection model to obtain the brightness quality and dynamic range quality of the first image output by the quality detection model.

[0018] Optionally, if the brightness quality does not meet preset conditions, the image determination unit is specifically configured to perform: a first convolution operation on the first image to obtain a first guide map; downsampling the first image to obtain a first sampled image; inputting the first sampled image into a first neural network to obtain first mapping data output by the first neural network, the first mapping data including the brightness features of each pixel in the first image; and obtaining a second image based on the first guide map and the first mapping data.

[0019] Optionally, if the dynamic range quality does not meet the preset conditions, the image determination unit is specifically configured to perform: a second convolution operation on the image to be processed to obtain a second guide image, and a third convolution operation on the image to be processed to obtain a third guide image, wherein the convolution kernels corresponding to the second and third convolution operations are different. The image to be processed is either the first image or an image that has undergone brightness processing. The grayscale information and Gaussian blur information of each pixel in the image to be processed are determined. The second image is obtained based on the second guide image, the third guide image, and the grayscale information and Gaussian blur information of each pixel in the image to be processed.

[0020] Optionally, the image determination unit is specifically configured to perform the following: downsampling the image to be processed to obtain a second sampled image; performing grayscale processing and Gaussian blur processing on the second sampled image to obtain a grayscale image and a Gaussian blurred image corresponding to the second sampled image; inputting the grayscale image and the Gaussian blurred image into a second neural network to obtain second mapping information and third mapping information output by the first neural network. The second mapping information includes the grayscale information of each pixel in the image to be processed, and the third mapping information includes the Gaussian blur information of each pixel in the image to be processed.

[0021] Optionally, the first image is an image that includes the facial region of the user object.

[0022] According to a third aspect of the present disclosure, an electronic device is provided, which may include: a processor and a memory for storing processor-executable instructions. The processor is configured to execute instructions to implement any of the optional image processing methods described in the first aspect above.

[0023] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which instructions are stored, such that when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform any of the optional image processing methods of the first aspect described above.

[0024] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including computer instructions that, when executed on an electronic device, cause the electronic device to perform an image processing method as optionally implemented in the first aspect.

[0025] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0026] Based on any of the above aspects of this disclosure, a first image to be beautified is acquired. The first image undergoes quality detection to obtain its brightness quality and dynamic range quality. If the brightness quality and / or dynamic range quality do not meet preset conditions, the brightness and / or dynamic range of the first image are processed to obtain a second image where both brightness quality and dynamic range quality meet the preset conditions. The second image is input into a pre-trained image beautification model to obtain reference feature information of the second image output by the image beautification model. This reference feature information is used to improve the aesthetic standards of the second image. The reference feature information is fused into the second image to obtain a beautified third image. Therefore, this disclosure can achieve adaptive adjustment of the brightness and / or dynamic range of the first image, effectively solving problems such as low light, weak light, and backlight in the image beautification process, and improving image quality.

[0027] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0028] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0029] Figure 1 A schematic diagram of an image processing system provided in an embodiment of this disclosure is shown;

[0030] Figure 2 A flowchart illustrating a model architecture provided in an embodiment of this disclosure is shown;

[0031] Figure 3 A schematic flowchart of an image processing method provided in an embodiment of this disclosure is shown;

[0032] Figure 4 A schematic diagram illustrating the implementation process of an image processing method provided in an embodiment of this disclosure is shown.

[0033] Figure 5 A schematic flowchart of another image processing method provided in an embodiment of this disclosure is shown;

[0034] Figure 6 A schematic diagram illustrating the implementation process of another image processing method provided in this disclosure embodiment is shown;

[0035] Figure 7 A schematic flowchart of another image processing method provided in an embodiment of this disclosure is shown;

[0036] Figure 8A schematic diagram illustrating the implementation process of another image processing method provided in this disclosure embodiment is shown;

[0037] Figure 9 A schematic flowchart of another image processing method provided in an embodiment of this disclosure is shown;

[0038] Figure 10 A schematic diagram illustrating the implementation process of another image processing method provided in this disclosure embodiment is shown;

[0039] Figure 11 A schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure is shown;

[0040] Figure 12 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0041] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0042] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0043] Furthermore, in the description of the embodiments of this disclosure, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, in the description of the embodiments of this disclosure, "multiple" refers to two or more.

[0044] It should also be understood that the term "comprising" indicates the presence of the described feature, whole, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements and / or components.

[0045] The data disclosed herein may be data authorized by the user or fully authorized by all parties.

[0046] With the widespread adoption of mobile devices and their performance gradually improving, capturing images and videos using these devices has become a new internet culture. Consequently, users are demanding higher and higher standards for portrait beautification technology and effects.

[0047] During the photography process, due to the performance of the terminal device, shooting techniques, and environmental factors, images often suffer from problems such as low light, weak light, and backlighting, resulting in poor overall image quality. Related image processing technologies still have certain limitations; they can typically only beautify imperfections in portraits, such as smoothing the skin, removing wrinkles, and slimming the face, but cannot solve problems like low light, weak light, and backlighting that affect image quality.

[0048] Based on this, embodiments of this application provide an image processing method, including: acquiring a first image to be beautified; performing quality detection on the first image to obtain its brightness quality and dynamic range quality; if the brightness quality and / or dynamic range quality do not meet preset conditions, processing the brightness and / or dynamic range of the first image to obtain a second image where both brightness quality and dynamic range quality meet preset conditions; inputting the second image into a pre-trained image beautification model to obtain reference feature information of the second image output by the image beautification model, the reference feature information being used to improve the aesthetic standards of the second image; and fusing the reference feature information into the second image to obtain a beautified third image. It is evident that this disclosure can achieve adaptive adjustment of the brightness and / or dynamic range of the first image, effectively solving problems such as low light, weak light, and backlight in the image beautification process, thereby improving image quality.

[0049] The following provides illustrative examples of application scenarios for the image processing method provided in the embodiments of this disclosure:

[0050] refer to Figure 1 This illustrates a schematic diagram of the structure of an image processing system provided in an embodiment of this disclosure. Figure 1 As shown, the image processing system may include a server 101 and a terminal device 102. Specifically, the terminal device 102 can be connected to the server 101 via a wired network or a wireless network.

[0051] For example, the terminal device in this disclosure embodiment may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc., which can install and use image processing applications. This disclosure embodiment does not impose any special restrictions on the specific form of the electronic device.

[0052] The server 101 can be used to receive images sent by a user through a terminal device. These images can be images taken by the user or stored in the terminal device. The server 101 can use the technical solution provided in this disclosure to perform quality detection on the image, process the image based on the quality detection results, and simultaneously enhance the image using a pre-trained image enhancement model to obtain an enhanced image. Then, the enhanced image is returned to the user's terminal device via wired or wireless means.

[0053] In some embodiments, the image processing system includes, for example, Figure 2 The model architecture 210 is shown. (As shown) Figure 2 As shown, the model architecture 210 includes a quality detection module 201, a brightness adjustment module 202, a dynamic range adjustment module 203, and an aesthetic adjustment module 204.

[0054] The input to the aforementioned model architecture 210 is a first image. This first image can be an image captured by the user in real time or an image stored in the terminal device 102. After the user selects the first image, the terminal device responds to the image processing command input by the user and sends the first image to the server 101.

[0055] The quality detection module 201 is used to determine the corresponding quality detection result based on the input first image. The quality detection module 201 can be a pre-trained quality detection model; inputting the first image into this model yields the corresponding quality detection result. The quality detection result consists of multiple parts, each characterizing the brightness quality and dynamic range quality of the first image. This quality detection result is used to determine whether adjustments to the brightness and / or dynamic range of the first image are necessary.

[0056] It should be noted that when both the brightness and dynamic range of the first image need to be adjusted, you can first adjust the brightness of the first image to obtain the image with adjusted brightness, and then adjust the dynamic range of the image with adjusted brightness. Alternatively, you can first adjust the dynamic range of the first image to obtain the image with adjusted dynamic range, and then adjust the brightness of the image with adjusted dynamic range.

[0057] The following example, which involves adjusting the brightness first and then the dynamic range, illustrates the functions of the brightness adjustment module 202 and the dynamic range adjustment module 203.

[0058] When only the brightness of the first image needs to be adjusted, or when both the brightness and dynamic range of the first image need to be adjusted, the first image is input to the brightness adjustment module 202. When only the dynamic range of the first image needs to be adjusted, the first image is input to the dynamic range adjustment module 203.

[0059] When only the brightness of the first image needs adjustment, the brightness adjustment module 202 adjusts the brightness of the first image to obtain the second image. When both the brightness and dynamic range of the first image need adjustment, the brightness adjustment module 202 adjusts the brightness of the first image to obtain the adjusted image, and then inputs the adjusted image to the dynamic range adjustment module 203.

[0060] When it is necessary to adjust the dynamic range of the first image, the dynamic range adjustment module 203 is used to adjust the dynamic range of the first image or the brightness adjustment module 202 input image after brightness adjustment to obtain the second image.

[0061] The aesthetic adjustment module 204 is used to beautify the second image to obtain the third image.

[0062] Based on the above model architecture 210, the specific implementation of the image processing method provided in the embodiments of this disclosure will be introduced.

[0063] The application scenario of the image processing method described above is merely an exemplary one. For example, in other application scenarios, the image processing method provided in this disclosure can also be executed independently by the terminal device to complete the image processing process.

[0064] In a more specific implementation, the image processing method provided in this disclosure can be executed by an image processing apparatus. The image processing apparatus can be a server or a terminal device, or be contained within a server or a terminal device; no limitation is made here. For example, the image processing apparatus can be contained within the aforementioned server 101.

[0065] In this disclosure, the server or terminal device can specifically be an electronic device. Furthermore, the entity executing the image processing method provided in this disclosure can be an electronic device.

[0066] The following will describe the specific implementation of the image processing method provided in the embodiments of this disclosure, in conjunction with the application scenarios described in the above image processing system.

[0067] Figure 3 This is a flowchart illustrating an image processing method provided in an embodiment of this disclosure. Figure 3 As shown, the image processing method may include the following steps S301-S305:

[0068] S301, Obtain the first image to be beautified.

[0069] The first image is an image to be beautified sent by the user through a terminal device. This first image can be an image taken by the user through the terminal device or stored in the terminal device. In response to an image processing command input by the user, the terminal device sends the first image to the server.

[0070] Generally, the image processing instructions input by the user instruct the first image to undergo beautification processing. Therefore, the first image can be an image including the user's facial region. Of course, the first image can also include the user's limb regions. The user's facial region, limb regions, and the scene information in which the user is situated can all be beautification objects. It is easy to understand that the first image may also not include the user's facial region and / or limb regions. For example, in some scenarios, the user can utilize the image processing functions of an image processing application on the terminal device to process images containing arbitrary content.

[0071] S302. Perform quality detection on the first image to obtain the brightness quality and dynamic range quality of the first image.

[0072] When a user captures an image, the brightness of each pixel in the image often varies due to the performance of the terminal device, the shooting technique, and environmental factors. Some pixels are too bright, while others are too dim, and the brightness difference between the excessively bright and dim pixels can be significant, leading to problems such as low light, weak light, and backlighting in the image. Dynamic range is used to characterize the degree of difference in brightness between the corresponding pixels in an image.

[0073] Figure 4 This is a schematic diagram illustrating an implementation process of S302 as exemplarily shown in this disclosure. Figure 5 This is a detailed flowchart of S302 as exemplarily shown in this disclosure. Figure 4 and Figure 5As shown, in some embodiments, in step S302 above, quality detection is performed on the first image to obtain a quality detection result, which may specifically include the following steps:

[0074] S501, Cut the first image into multiple sub-images of the same size.

[0075] Specifically, the sub-image has the same size; for example, the sub-image can be a sub-image with a size of 64*64 pixels.

[0076] S502. Determine the encoding information for each sub-image. The encoding information includes one or more of the following: hash encoding information, content encoding information, style encoding information, and location encoding information.

[0077] Specifically, the hash encoding information is determined by transforming each sub-image to obtain the feature vector corresponding to each sub-image, and then hash encoding the feature vector. The hash encoding algorithm is preset and is not restricted here.

[0078] As those skilled in the art will know, the perceptron model is a binary linear classification model, belonging to the category of discriminative and non-probabilistic models. In short, this model involves finding a hyperplane that separates the linearly separable dataset into two parts.

[0079] In this embodiment, by performing positional encoding on sub-images to obtain positional encoding information, content encoding information and style encoding information of each sub-image can be extracted using a multilayer perceptron (such as an 8-layer perceptron). The content encoding information is used to characterize image content features, the style encoding information is used to characterize image style features, and the positional encoding information is used to characterize the positional distribution features of image content.

[0080] S503. Input the encoding information of each sub-image into the pre-trained quality detection model to obtain the quality detection result output by the quality detection model.

[0081] In the above implementation, the quality detection result includes brightness quality and dynamic range quality.

[0082] like Figure 4 As shown, the quality detection module 201 includes an encoder and a multilayer perceptron. The encoder can be a transformer encoder, consisting of N encoders. Each encoder has two sub-layer connections: one is a multi-head self-attention layer with a normalization layer and residual units; the other is a feedforward layer with normalization layer and residual units. These layers are used to concatenate the input positional encoding information, content encoding information, style encoding information, and hash encoding information, and then input them into the multilayer perceptron to obtain the quality detection result of the first image.

[0083] like Figure 5As shown, one or more of the hash encoding information, content encoding information, style encoding information and position encoding information mentioned above are input into the transformer encoder and concatenated to obtain the feature vector to be detected. The feature vector to be detected is then input into the multilayer perceptron, and the quality detection result corresponding to the first image is output.

[0084] The aforementioned quality detection model is a pre-trained neural network model. The training samples can be image samples with known brightness quality and dynamic range quality. By inputting the training samples into the quality detection model, the output results are obtained, and the loss of the quality detection model is determined based on the sample labels corresponding to the training samples. Then, the model parameters are continuously optimized based on the model loss until a quality detection model that meets the conditions is obtained. Here, the sample labels are used to mark the brightness quality and dynamic range quality of the training samples.

[0085] As can be seen from S501-S503 above, this implementation provides a specific way to obtain the quality detection result corresponding to the first image. By cutting the first image into multiple sub-images, the quality detection result is determined according to the encoding information corresponding to each sub-image. The quality detection result can be used to determine the image quality of the first image and further determine the optimization target of the first image.

[0086] S303. If the brightness quality and / or dynamic range quality do not meet the preset conditions, process the brightness and / or dynamic range of the first image to obtain a second image in which both the brightness quality and dynamic range quality meet the preset conditions.

[0087] In some embodiments, before processing the brightness and / or dynamic range of the first image if the brightness quality and / or dynamic range quality do not meet preset conditions, the method further includes: determining an optimization target for the first image based on the brightness quality and dynamic range quality characterized by the quality detection results.

[0088] If the brightness quality represented by the quality detection results does not meet the preset conditions, the optimization target of the first image is brightness.

[0089] Specifically, the brightness quality and dynamic range quality represented by the quality detection results can be expressed by different scores. For example, the brightness quality represented by the quality detection results can be scored as 4 points, and the dynamic range quality represented by the quality detection results can be scored as 7 points. The preset condition is that the scores of the brightness quality and dynamic range quality represented by the quality detection results are greater than 6 points. Since the dynamic range quality represented by the quality detection results does not meet the preset condition, but the brightness quality represented by the quality detection results does meet the preset condition, the optimization target of the first image is brightness.

[0090] If the dynamic range quality represented by the quality detection results does not meet the preset conditions, the optimization target of the first image is the dynamic range.

[0091] For example, the score corresponding to the brightness quality represented by the quality detection result can be 7 points, and the score corresponding to the dynamic range quality represented by the quality detection result can be 4 points. The preset condition is that the scores corresponding to the brightness quality and dynamic range quality represented by the quality detection result are greater than 6 points. Since the dynamic range quality represented by the quality detection result does not meet the preset condition, but the brightness quality represented by the quality detection result meets the preset condition, the optimization target of the first image is dynamic range.

[0092] If the brightness and dynamic range quality represented by the quality detection results do not meet the preset conditions, the optimization target of the first image is brightness and dynamic range.

[0093] For example, the score corresponding to the brightness quality represented by the quality detection result can be 8 points, and the score corresponding to the dynamic range quality represented by the quality detection result can be 5 points. The preset condition is that the scores corresponding to the brightness quality and dynamic range quality represented by the quality detection result are greater than 8 points. Since the dynamic range quality represented by the quality detection result does not meet the preset condition, but the brightness quality represented by the quality detection result does meet the preset condition, the optimization target of the first image is brightness and dynamic range.

[0094] As can be seen from the above, by setting corresponding preset conditions for the brightness quality and dynamic range quality represented by the quality detection results, it is easier to determine the optimization target of the first image and avoid using brightness or dynamic range as the optimization target of the first image when the brightness quality or dynamic range quality meets the preset conditions.

[0095] It is readily understood that, under conditions where the brightness quality does not meet the preset requirements, the above-mentioned S303 includes the step of adjusting the brightness of the first image, for example, as follows: Figure 7 S701-S704 are shown in the diagram. When the dynamic range quality does not meet a preset condition, S303 includes a step of adjusting the dynamic range of the first image, for example, as described below. Figure 9 S901-S905 are shown in the figure.

[0096] Figure 6 This is a schematic diagram illustrating an implementation process of S303 as exemplarily shown in this disclosure. Figure 7 This is a detailed flowchart of S303 as exemplarily shown in this disclosure. Figure 6 and Figure 7 As shown, in some embodiments, if the brightness quality does not meet the preset conditions, the brightness and / or dynamic range of the first image are processed to obtain a second image whose brightness quality and dynamic range quality both meet the preset conditions. This includes the following steps:

[0097] S701. Perform a first convolution operation on the first image to obtain a first guide image.

[0098] Specifically, if the first image is in RGB format, it is converted to YUV format. The luminance channel (Y channel) of the first image is extracted and subjected to multiple convolution operations using a convolution kernel, for example, two convolution operations, to obtain the first guiding image.

[0099] S702. Downsample the first image to obtain the first sampled image.

[0100] Specifically, a downsampling operation is performed on the first image to obtain a first sampled image. The resolution of the first sampled image can be preset; in this embodiment, the resolution of the first sampled image is 256x256.

[0101] S703. Input the first sampled image into the first neural network to obtain the first mapping data output by the first neural network. The first mapping data includes the brightness features of each pixel on the first image.

[0102] The first neural network described above is a pre-trained neural network model. The training samples can be image samples with known first mapping information. By inputting the training samples into the first neural network, the output result is obtained, and the loss of the first neural network is determined according to the sample labels corresponding to the training samples. Then, the model parameters are continuously optimized according to the model loss until a first neural network that meets the conditions is obtained. Here, the sample labels are the first mapping information used to mark the training samples.

[0103] Specifically, by inputting the first sampled image into the VGG19 model, the perceptual features of the first sampled image are obtained. The perceptual features of the first sampled image are then input into the first neural network to obtain the first mapping data.

[0104] S704. Based on the first guide image and the first mapping data, obtain the second image.

[0105] Based on the first mapping data and the first guiding image, the second image is determined through preset operations, which involve calling the slice function and the apply function to generate the second image.

[0106] In some embodiments, before obtaining the second image based on the first guide map and the first mapping data, the first mapping data is further optimized. The optimization method may be to decompose and reconstruct the matrix corresponding to the first mapping data, which can reduce noise and make the third image smoother.

[0107] As can be seen from S701-S704 above, this implementation provides a specific way to adjust the brightness quality of an image. By adjusting the brightness of the image, it avoids areas that are too bright or too dark in the image and improves the image quality.

[0108] Figure 8 This is a schematic diagram illustrating one implementation process of the above-described S303 as exemplarily shown in this disclosure. Figure 9 This is a detailed flowchart of S303 as exemplarily illustrated in this disclosure. Figure 8 and Figure 9 As shown, in some embodiments, if the brightness quality does not meet a preset condition, the brightness and / or dynamic range of the first image are processed to obtain a second image whose brightness quality and dynamic range quality both meet the preset conditions, including:

[0109] S901. Perform a second convolution operation and a third convolution operation on the first image to obtain a second guide image and a third guide image. The second and third convolution operations use different convolution kernels.

[0110] S902. Downsample the first image to obtain the second sampled image.

[0111] Specifically, the first image is downsampled to obtain a second sampled image, the resolution of which can be preset. In this embodiment, the resolution of the second sampled image is 256x256.

[0112] S903. Perform grayscale processing and Gaussian blur processing on the second sampled feature image to obtain the grayscale image and Gaussian blurred image corresponding to the second sampled image.

[0113] S904. Input the grayscale image and the Gaussian blurred image into the second neural network to obtain the second mapping information and the third mapping information output by the first neural network. The second mapping information includes the grayscale information of each pixel in the first image, and the third mapping information includes the Gaussian blurred information of each pixel in the first image.

[0114] The aforementioned second neural network is a pre-trained neural network model. The training samples can be grayscale images and Gaussian blurred images of image samples with known second and third mapping information. By inputting the training samples into the second neural network, the output result is obtained, and the loss of the second neural network is determined according to the sample labels corresponding to the training samples. Then, the model parameters are continuously optimized based on the model loss until a second neural network that meets the conditions is obtained. Here, the sample labels are used to label the second and third mapping information of the training samples.

[0115] Specifically, the grayscale image and Gaussian blurred image of the second sampled image are input into the second neural network to obtain the second mapping data and the third mapping data.

[0116] S905. Obtain the second image based on the second guide image, the third guide image, the second mapping information, and the third mapping information.

[0117] Specifically, the second guide map, the third guide map, the second mapping information, and the third mapping information are fed into the encoder for encoding to obtain the encoding corresponding to the second image. This encoding is then input into the AdaIN layer for integration to obtain the second image. In some embodiments, before obtaining the second image from the second guide map, the third guide map, the second mapping information, and the third mapping information, the second and third mapping data are further optimized. The optimization method can be to decompose and reconstruct the matrices corresponding to the second and third mapping data, which can reduce noise and make the third image smoother.

[0118] As can be seen from S901-S905 above, this implementation provides a specific way to adjust the dynamic range quality of an image. It processes the dynamic range of the image from two dimensions: grayscale and Gaussian blur, to avoid problems such as dark light, weak light, and backlight, thereby improving image quality.

[0119] S304. Input the second image into the pre-trained image beautification model to obtain the reference feature information of the second image output by the image beautification model. The reference feature information is used to improve the aesthetic standard of the second image.

[0120] The image beautification model described above is a pre-trained neural network model. Training samples can be training images and their corresponding labels, which represent aesthetic feature evaluations of the training images. Aesthetic feature evaluation is the assessment of the beauty or ugliness of different parts of the training image. For example, if a training image includes a facial region, the aesthetic feature evaluation could be that the area corresponding to the nose is beautiful, and the area corresponding to the eyes is ugly. The aesthetic model trained using these training samples can extract the aesthetic features of the input image.

[0121] S305. The reference feature information is fused into the second image to obtain the beautified third image.

[0122] In some embodiments, S305 specifically includes:

[0123] First, the second image is input into a pre-trained image enhancement model to obtain reference feature information of the second image output by the image enhancement model. The reference feature information is used to improve the aesthetic standards of the second image.

[0124] The aforementioned reference aesthetic features can characterize the features of the areas in the second image that need to beautify. For example, when a user inputs image A into the image beautification model, the facial area corresponding to image A has acne scars. Then, the reference feature information of the output second image is used to characterize the feature information for removing the acne scars from image A.

[0125] Then, the second image is downsampled N times to obtain N sampled images of different sizes, where N ≥ 1. Each downsampling operation on the second image yields a sampled image of one size.

[0126] Figure 10 This is a schematic diagram illustrating an implementation process of S305 as exemplarily shown in this disclosure. Specifically, Figure 10 The implementation process of S305 when N=5 is shown. For example... Figure 10 As shown, performing a downsampling operation on the second image yields a sampled image of size S2. Performing another downsampling operation on the sampled image of size S2 yields a sampled image of size S4. By performing five downsampling operations on the second image, five sampled images of different sizes—S2, S4, S8, S16, and S32—can be obtained.

[0127] Finally, the reference feature information is fused with N sampled images of different sizes to obtain the third image. Specifically, with the N sampled images of different sizes arranged in descending order of size, N fusion operations are performed sequentially based on the reference feature information and the N sampled images of different sizes, resulting in N fusion results. The Nth fusion result is then used as the third image. The first fusion result is obtained by fusing the reference feature information with the Nth sampled image, the i-th fusion result is obtained by fusing the (i-1)-th fusion result with the (N+1-i)-th sampled image, and 1 ≤ i ≤ N.

[0128] by Figure 10 Taking the process shown as an example, firstly, the sampled image of size S32 is fused with the reference feature information to obtain the first fusion result. After upsampling the first fusion result, it is fused with the sampled image of size S16 to obtain the second fusion result. After upsampling the second fusion result, it is fused with the sampled image of size S8 to obtain the third fusion result. After upsampling the third fusion result, it is fused with the sampled image of size S4 to obtain the fourth fusion result. After upsampling the fourth fusion result, it is fused with the sampled image of size S2 to obtain the third image.

[0129] Depend on Figure 10As can be seen, five downsampling operations were performed on the second image, resulting in five sampled images of different sizes: S2, S4, S8, S16, and S32. The reference feature information of the second image output by the image beautification model was fused with the S32-sized sampled image to obtain the first fusion result. This first fusion result was then upsampled to obtain the S16-sized first fusion result, which was then fused with the S16-sized sampled image to obtain the second fusion result. These fusion results were then sequentially upsampled and fused with the corresponding S8, S4, and S2-sized sampled images to finally obtain the third image, which is the beautified version of the second image. In cases where the facial area corresponding to the second image has numerous acne scars, the third image represents the second image with the acne scars removed.

[0130] As can be seen from the above, this implementation provides a specific way to process a second image using a pre-trained image beautification model to obtain a third image. By inputting the second image into the pre-trained image beautification model to obtain the corresponding reference aesthetic features, the reference aesthetic features and N sampled images of different sizes of the second image are fused to obtain the third image, thereby beautifying the second image and improving the user experience.

[0131] In conjunction with the present disclosure, as shown in S301-S305, by using a pre-trained image beautification model, the aesthetic standards of the image are improved, while the optimization target for the first image is determined based on the brightness quality and dynamic range quality of the first image. The optimization target includes brightness and / or dynamic range, thereby achieving adaptive adjustment of the brightness and / or dynamic range of the first image to be beautified.

[0132] It is understood that, in actual implementation, the terminal / server of this disclosure embodiment may include one or more hardware structures and / or software modules for implementing the aforementioned corresponding resource scheduling method, and these hardware structures and / or software modules may constitute an electronic device. Those skilled in the art should readily recognize that, based on the algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0133] Based on this understanding, the present disclosure also provides an image processing apparatus that can be applied to electronic devices. Figure 11 A schematic diagram of the structure of an image processing apparatus provided in an embodiment of this disclosure is shown. For example... Figure 11As shown, the image processing apparatus may include: an image acquisition unit 1101, a quality detection unit 1102, an image determination unit 1103, a feature determination unit 1104, and a feature fusion unit 1105. The image acquisition unit 1101 is configured to acquire a first image to be beautified. For example, it executes... Figure 3 The step S301 shown is described. The quality detection unit 1102 is configured to perform quality detection on the first image to obtain the brightness quality and dynamic range quality of the first image. For example, it performs... Figure 3 The step S302 shown is described. The image determination unit 1103 is configured to process the brightness and / or dynamic range of the first image when the brightness quality and / or dynamic range quality do not meet preset conditions, to obtain a second image where both the brightness quality and dynamic range quality meet the preset conditions. For example, executing... Figure 3 The step S303 shown is described. The feature determination unit 1104 is configured to input the second image into a pre-trained image enhancement model to obtain reference feature information of the second image output by the image enhancement model. The reference feature information is used to improve the aesthetic standards of the second image. For example, performing... Figure 3 The step S304 shown. The feature fusion unit 1105 is configured to fuse reference feature information into the second image to obtain the beautified third image. For example, it executes... Figure 3 The step S305 is shown.

[0134] Optionally, the feature fusion unit 1105 is specifically configured to perform: N downsampling operations on the second image to obtain N feature information to be processed at different sizes, where N is greater than or equal to 1. Then, it performs N fusion operations sequentially on the reference feature information and the N feature information to be processed at different sizes to obtain N fusion results, and uses the Nth fusion result as the third image. Specifically, the first fusion result is the fusion result obtained by fusing the reference feature information with the Nth feature information to be processed, and the i-th fusion result is the fusion result obtained by fusing the (i-1)-th fusion result with the (N+1-i)th feature information to be processed.

[0135] Optionally, the feature fusion unit 1105 is specifically configured to perform the following operations: multiplying the reference feature information by the gamma value of the Nth feature information to obtain the first multiplication result; adding the first multiplication result to the beta value of the Nth feature information to obtain the first addition result; performing an upsampling operation on the first addition result to determine the first fusion result. Then, multiplying the (i-1)th fusion result by the gamma value of the (N+1-i)th feature information to obtain the i-th multiplication result; adding the i-th multiplication result to the beta value of the (N+1-i)th feature information to obtain the i-th addition result; and performing an upsampling operation on the i-th addition result to obtain the i-th fusion result.

[0136] Optionally, the quality detection unit 1102 is specifically configured to perform the following: Segment the first image into multiple sub-images of the same size. Determine the encoding information for each sub-image, which includes one or more of hash encoding information, content encoding information, style encoding information, and location encoding information. The content encoding information is used to characterize image content features, the style encoding information is used to characterize image style features, and the location encoding information is used to characterize the location distribution features of image content. Input the encoding information of each sub-image into a pre-trained quality detection model to obtain the brightness quality and dynamic range quality of the first image output by the quality detection model.

[0137] Optionally, if the brightness quality does not meet the preset conditions, the image determination unit 1103 is specifically configured to perform: a first convolution operation on the first image to obtain a first guide map; downsampling the first image to obtain a first sampled image; inputting the first sampled image into a first neural network to obtain first mapping data output by the first neural network, the first mapping data including the brightness features of each pixel in the first image; and obtaining a second image based on the first guide map and the first mapping data.

[0138] Optionally, if the dynamic range quality does not meet the preset conditions, the image determination unit 1103 is specifically configured to perform: a second convolution operation on the image to be processed to obtain a second guide image, and a third convolution operation on the image to be processed to obtain a third guide image, wherein the convolution kernels corresponding to the second and third convolution operations are different. The image to be processed is either the first image or an image that has undergone brightness processing. The grayscale information and Gaussian blur information of each pixel in the image to be processed are determined. The second image is obtained based on the second guide image, the third guide image, and the grayscale information and Gaussian blur information of each pixel in the image to be processed.

[0139] Optionally, the image determination unit 1103 is specifically configured to perform the following: downsampling the image to be processed to obtain a second sampled image; performing grayscale processing and Gaussian blur processing on the second sampled image to obtain a grayscale image and a Gaussian blurred image corresponding to the second sampled image; inputting the grayscale image and the Gaussian blurred image into a second neural network to obtain second mapping information and third mapping information output by the first neural network. The second mapping information includes the grayscale information of each pixel in the image to be processed, and the third mapping information includes the Gaussian blur information of each pixel in the image to be processed.

[0140] Optionally, the first image is an image that includes the facial region of the user object.

[0141] As described above, the embodiments of this disclosure can divide the electronic device into functional modules according to the above method examples. The integrated modules can be implemented in hardware or as software functional modules. Furthermore, it should be noted that the module division in these embodiments is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into a single processing module.

[0142] The specific methods by which each module performs its operations and the beneficial effects of the image processing apparatus in the above embodiments have been described in detail in the foregoing method embodiments, and will not be repeated here.

[0143] This disclosure also provides an electronic device. Figure 12 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown. The electronic device may be an image processing apparatus and may include at least one processor 1201, a communication bus 1202, a memory 1203, and at least one communication interface 1204.

[0144] Processor 1201 can be a processor (central processing unit, CPU), microprocessor unit, ASIC, or one or more integrated circuits for controlling the execution of programs according to the present disclosure. As an example, combined with... Figure 11 The functions implemented by the image acquisition unit 1101, quality detection unit 1102, image determination unit 1103, feature determination unit 1104, and feature fusion unit 1105 in the electronic device are the same as those of the other components. Figure 12 The processor 1201 in it performs the same function.

[0145] The communication bus 1202 may include a path for transmitting information between the aforementioned components.

[0146] Communication interface 1204 uses any transceiver-like device for communicating with other devices or communication networks, such as servers, Ethernet, radio access networks (RAN), wireless local area networks (WLAN), etc.

[0147] As an example, memory 1203 may be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory may exist independently and be connected to the processing unit via a bus. Memory may also be integrated with the processing unit.

[0148] The memory 1203 stores the application code that executes the present invention, and its execution is controlled by the processor 1201. The processor 1201 executes the application code stored in the memory 1203 to implement the functions of the method of the present invention.

[0149] In a specific implementation, as one embodiment, the processor 1201 may include one or more CPUs, for example... Figure 12 CPU0 and CPU1 in the CPU.

[0150] In a specific implementation, as one example, an electronic device may include multiple processors, for example... Figure 12 Processors 1201 and 1205 are mentioned. Each of these processors can be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0151] In a specific implementation, as one embodiment, the electronic device may further include an input device 1206 and an output device 1207. The input device 1206 and output device 1207 communicate and can accept user input in various ways. For example, the input device 1206 may be a mouse, keyboard, touchscreen device, or sensing device. The output device 1207 communicates with the processor 1201 and can display information in various ways. For example, the output device 1201 may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, etc.

[0152] Those skilled in the art will understand that Figure 12 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0153] This disclosure also provides an electronic device. This electronic device can be an image processing apparatus. The electronic device can vary considerably depending on its configuration or performance, and may include one or more processors and one or more memories. The memory stores at least one instruction, which is loaded and executed by the processor to implement the image processing methods provided in the various method embodiments described above. Of course, the electronic device may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The electronic device may also include other components for implementing device functions, which will not be elaborated upon here.

[0154] This disclosure also provides a computer-readable storage medium including instructions stored thereon. When the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer is able to perform the image processing method provided in the embodiments shown above. For example, the computer-readable storage medium can be a memory 1203 including instructions, which can be executed by a processor 1201 of a terminal to complete the above method. As another example, the computer-readable storage medium can be a memory including instructions, which can be executed by a processor of an electronic device to complete the above method. Optionally, the computer-readable storage medium can be a non-transitory computer-readable storage medium, such as a ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0155] This disclosure also provides a computer program product including computer instructions that, when executed on an electronic device, cause the electronic device to perform the above-described actions. Figure 1 - Figure 10 Image processing method shown in any of the accompanying figures.

[0156] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0157] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, include: Obtain the first image to be beautified; The first image is subjected to quality detection to obtain the brightness quality and dynamic range quality of the first image; If the brightness quality and / or dynamic range quality do not meet the preset conditions, the brightness and / or dynamic range of the first image are processed to obtain a second image in which both the brightness quality and the dynamic range quality meet the preset conditions. The second image is input into a pre-trained image enhancement model to obtain reference feature information of the second image output by the image enhancement model. The reference feature information is used to improve the aesthetic standard of the second image. The second image is downsampled N times to obtain N feature information of different sizes to be processed, where N is greater than or equal to 1; The reference feature information and the N feature information of different sizes are fused together N times to obtain N fusion results, and the Nth fusion result is used as the beautified third image. The first fusion result is the fusion result obtained by fusing the reference feature information with the Nth feature information to be processed, and the ith fusion result is the fusion result obtained by fusing the (i-1)th fusion result with the (N+1-i)th feature information to be processed.

2. The image processing method according to claim 1, characterized in that, The process involves sequentially performing N fusion operations based on the reference feature information and the N feature information of different sizes to be processed, resulting in N fusion results, including: The reference feature information is multiplied by the gamma value of the Nth feature information to be processed to obtain the first multiplication result. The first multiplication result is added to the beta value of the Nth feature information to obtain the first addition result. The first addition result is upsampled to determine the first fusion result. Multiply the (i-1)th fusion result by the gamma value of the (N+1-i)th feature information to obtain the i-th multiplication result. Add the i-th multiplication result by the beta value of the (N+1-i)th feature information to obtain the i-th addition result. Perform an upsampling operation on the i-th addition result to obtain the i-th fusion result.

3. The image processing method according to claim 1, characterized in that, The step of performing quality detection on the first image to obtain the brightness quality and dynamic range quality of the first image includes: The first image is cut into multiple sub-images of the same size; The encoding information of each sub-image is determined. The encoding information includes one or more of hash encoding information, content encoding information, style encoding information, and location encoding information. The content encoding information is used to characterize the image content features, the style encoding information is used to characterize the image style features, and the location encoding information is used to characterize the image content location distribution features. The encoded information of each of the sub-images is input into a pre-trained quality detection model to obtain the brightness quality and dynamic range quality of the first image output by the quality detection model.

4. The image processing method according to claim 1, characterized in that, If the brightness quality does not meet the preset conditions, the step of processing the brightness and / or dynamic range of the first image to obtain a second image in which both brightness and dynamic range meet the preset conditions includes: Perform a first convolution operation on the first image to obtain a first guide image; The first image is downsampled to obtain a first sampled image; The first sampled image is input into the first neural network to obtain the first mapping data output by the first neural network. The first mapping data includes the brightness features of each pixel in the first image. The second image is obtained based on the first guide map and the first mapping data.

5. The image processing method according to claim 1, characterized in that, When the dynamic range quality does not meet the preset conditions, the step of processing the brightness and / or dynamic range of the first image to obtain a second image in which both brightness and dynamic range meet the preset conditions includes: A second convolution operation is performed on the image to be processed to obtain a second guide image, and a third convolution operation is performed on the image to be processed to obtain a third guide image. The convolution kernels corresponding to the second convolution operation and the third convolution operation are different. The image to be processed is the first image or an image that has undergone brightness processing. Determine the grayscale information and Gaussian blur information of each pixel in the image to be processed; The second image is obtained based on the second guide image, the third guide image, and the grayscale information and Gaussian blur information of each pixel in the image to be processed.

6. The image processing method according to claim 5, characterized in that, Determining the grayscale information and Gaussian blur information of each pixel in the image to be processed includes: The image to be processed is downsampled to obtain a second sampled image; The second sampled image is subjected to grayscale processing and Gaussian blur processing respectively to obtain the grayscale image and Gaussian blurred image corresponding to the second sampled image; The grayscale image and the Gaussian blur image are input into a second neural network to obtain second mapping information and third mapping information output by the second neural network. The second mapping information includes the grayscale information of each pixel in the image to be processed, and the third mapping information includes the Gaussian blur information of each pixel in the image to be processed.

7. The image processing method according to claim 1, characterized in that, The first image is an image that includes the facial region of the user object.

8. An image processing apparatus, characterized in that, include: The image acquisition unit is configured to acquire the first image to be beautified. The quality detection unit is configured to perform quality detection on the first image to obtain the brightness quality and dynamic range quality of the first image; The image determination unit is configured to process the brightness and / or dynamic range of the first image when the brightness quality and / or dynamic range quality do not meet the preset conditions, so as to obtain a second image in which both the brightness quality and the dynamic range quality meet the preset conditions. The feature determination unit is configured to input the second image into a pre-trained image enhancement model to obtain reference feature information of the second image output by the image enhancement model, wherein the reference feature information is used to improve the aesthetic standard of the second image. The feature fusion unit is configured to perform N downsampling operations on the second image to obtain N feature information to be processed of different sizes, where N is greater than or equal to 1; The reference feature information and the N feature information of different sizes are fused together N times to obtain N fusion results, and the Nth fusion result is used as the beautified third image. The first fusion result is the fusion result obtained by fusing the reference feature information with the Nth feature information to be processed, and the ith fusion result is the fusion result obtained by fusing the (i-1)th fusion result with the (N+1-i)th feature information to be processed.

9. The image processing apparatus according to claim 8, characterized in that, The feature fusion unit is specifically configured to perform: The reference feature information is multiplied by the gamma value of the Nth feature information to be processed to obtain the first multiplication result. The first multiplication result is added to the beta value of the Nth feature information to obtain the first addition result. The first addition result is upsampled to determine the first fusion result. Multiply the (i-1)th fusion result by the gamma value of the (N+1-i)th feature information to obtain the i-th multiplication result. Add the i-th multiplication result by the beta value of the (N+1-i)th feature information to obtain the i-th addition result. Perform an upsampling operation on the i-th addition result to obtain the i-th fusion result.

10. The image processing apparatus according to claim 8, characterized in that, The quality inspection unit is specifically configured to perform: The first image is cut into multiple sub-images of the same size; The encoding information of each sub-image is determined. The encoding information includes one or more of hash encoding information, content encoding information, style encoding information, and location encoding information. The content encoding information is used to characterize the image content features, the style encoding information is used to characterize the image style features, and the location encoding information is used to characterize the image content location distribution features. The encoded information of each of the sub-images is input into a pre-trained quality detection model to obtain the brightness quality and dynamic range quality of the first image output by the quality detection model.

11. The image processing apparatus according to claim 8, characterized in that, When the brightness quality does not meet the preset conditions, the image determination unit is specifically configured to perform: Perform a first convolution operation on the first image to obtain a first guide image; The first image is downsampled to obtain a first sampled image; The first sampled image is input into the first neural network to obtain the first mapping data output by the first neural network. The first mapping data includes the brightness features of each pixel in the first image. The second image is obtained based on the first guide map and the first mapping data.

12. The image processing apparatus according to claim 8, characterized in that, When the dynamic range quality does not meet the preset condition, the image determination unit is specifically configured to perform: A second convolution operation is performed on the image to be processed to obtain a second guide image, and a third convolution operation is performed on the image to be processed to obtain a third guide image. The convolution kernels corresponding to the second convolution operation and the third convolution operation are different. The image to be processed is the first image or an image that has undergone brightness processing. Determine the grayscale information and Gaussian blur information of each pixel in the image to be processed; The second image is obtained based on the second guide image, the third guide image, and the grayscale information and Gaussian blur information of each pixel in the image to be processed.

13. The image processing apparatus according to claim 12, characterized in that, The image determination unit is specifically configured to perform: The image to be processed is downsampled to obtain a second sampled image; The second sampled image is subjected to grayscale processing and Gaussian blur processing respectively to obtain the grayscale image and Gaussian blurred image corresponding to the second sampled image; The grayscale image and the Gaussian blur image are input into a second neural network to obtain second mapping information and third mapping information output by the second neural network. The second mapping information includes the grayscale information of each pixel in the image to be processed, and the third mapping information includes the Gaussian blur information of each pixel in the image to be processed.

14. The image processing apparatus according to claim 8, characterized in that, The first image is an image that includes the facial region of the user object.

15. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the image processing method as described in any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the image processing method as described in any one of claims 1-7.

Images