Image intelligent beautification method, system, electronic device and storage medium

Abstract

The present invention discloses an intelligent image beautification method, system, electronic device and storage medium, the method comprising: obtaining a sample data set, the sample data set comprising a plurality of sample image pairs, the sample image pairs comprising a training image and a corresponding beautified image; performing feature extraction processing on the training image; using the training image after feature extraction processing and the corresponding beautified image to train a preset beautification coefficient estimation model to obtain a target beautification coefficient estimation model; obtaining a target image, and performing feature extraction processing on the target image; using the target beautification coefficient estimation model to process the target image after feature extraction processing to obtain a beautification estimation coefficient of the target image; performing beautification processing on the target image according to the beautification estimation coefficient to obtain a target beautification image corresponding to the target image. The present invention can solve the problems of cumbersome image beautification operation and low efficiency in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image intelligent beautification method, system, electronic equipment and storage medium. Background Art

[0002] Images ensure intuitive content presentation and are a crucial medium for information transmission, making them widely used by online travel agencies (OTAs). Effective and accurate image presentation can significantly enhance the user experience, thereby increasing user conversion rates. Due to the complex nature of OTA image sources, most images showcasing key content such as hotels and destinations are not captured by professional photographers. Image quality is significantly affected by the photographer's skill, the lighting conditions, and the equipment used. Therefore, improving the aesthetics of low-quality images by enhancing lighting, clarity, and color tone, without altering the content, can significantly enhance the user experience and possesses significant application value.

[0003] Currently, image beautification is mainly achieved through manual operation using image processing software, which is relatively cumbersome and has low efficiency for processing large quantities of images. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide an intelligent image beautification method, system, electronic device and storage medium to solve the problems of cumbersome image beautification operations and low efficiency in the prior art.

[0005] In order to achieve the above object, the present invention provides an image intelligent beautification method, comprising:

[0006] Obtain a sample data set, wherein the sample data set includes a plurality of sample image pairs, wherein the sample image pairs include a training image and a corresponding beautified image;

[0007] performing feature extraction processing on the training image;

[0008] Using the training images processed by feature extraction and the corresponding beautified images, a preset beautification coefficient estimation model is trained to obtain a target beautification coefficient estimation model;

[0009] Acquire a target image and perform feature extraction processing on the target image;

[0010] Processing the target image after feature extraction using the target beautification coefficient estimation model to obtain a beautification estimation coefficient of the target image;

[0011] The target image is beautified according to the beautification estimation coefficient to obtain a target beautified image corresponding to the target image.

[0012] In a preferred embodiment of the present invention, after obtaining the target beautified image corresponding to the target image, the method further includes:

[0013] The target image and the target beautified image corresponding to the target image are processed using a preset beautification quality assessment model to obtain a probability that the target image is effectively beautified, and whether the target image is effectively beautified is determined based on the probability.

[0014] In a preferred embodiment of the present invention, the method further comprises:

[0015] When it is determined that the target image has been effectively beautified, displaying a target beautified image corresponding to the target image;

[0016] When it is determined that the target image has not been effectively beautified, the target image is displayed.

[0017] In a preferred embodiment of the present invention, before performing feature extraction processing on the training image, the method further comprises: performing downsampling processing on the training image;

[0018] Before performing feature extraction processing on the target image, the method further includes: performing downsampling processing on the target image.

[0019] In a preferred embodiment of the present invention, the beautification coefficient estimation model is a bilateral grid network model as shown in the following formula (3):

[0020]

[0021] Among them, S represents the beautification estimation coefficient, W p represents the network weight of point p on the input image, and F represents the preset domain range of point p. represents the Gaussian kernel function in the spatial domain, represents the pixel value domain Gaussian kernel function, ||pq|| represents the pixel distance between point p and point q, |I p -I q | represents the pixel difference between point p and point q, I q The pixel representing the q point.

[0022] In a preferred embodiment of the present invention, the step of training a preset beautification coefficient estimation model using the training image after feature extraction processing and the corresponding beautified image to obtain a target beautification coefficient estimation model includes:

[0023] Processing the training image after feature extraction using the beautification coefficient estimation model to obtain a beautification estimation coefficient corresponding to the training image;

[0024] beautify the training image according to the beautification estimation coefficient to obtain a target beautified image corresponding to the training image;

[0025] Constructing a loss function based on the target beautified image and the beautified image corresponding to the training image;

[0026] Adjust the parameters of the beautification coefficient estimation model based on the loss function, and return to the step of processing the training image after feature extraction using the beautification coefficient estimation model until the loss function meets a predetermined condition, thereby obtaining a target beautification coefficient estimation model.

[0027] In a preferred embodiment of the present invention, the step of constructing a loss function based on the target beautified image and the beautified image corresponding to the training image includes:

[0028] Based on the target beautified image and the beautified image corresponding to the training image, the reconstruction loss L is calculated according to formula (5): r :

[0029]

[0030] Wherein, I′ represents the beautified image corresponding to the training image, represents the target beautified image corresponding to the training image, and S represents the corresponding beautification estimation coefficient;

[0031] Based on the target beautified image and the beautified image corresponding to the training image, the color loss L is calculated according to formula (6): c :

[0032]

[0033] Among them, F(I′) P represents a three-dimensional vector composed of the RGB pixel values ​​at point P in the beautified image corresponding to the training image, represents a three-dimensional vector composed of the RGB pixel values ​​at the point corresponding to the position of point P in the target beautified image corresponding to the training image, represents F(I′) P and The angle difference;

[0034] According to the reconstruction loss L r and the color loss L c Construct the loss function.

[0035] In order to achieve the above object, the present invention provides an image intelligent beautification system, comprising:

[0036] A sample acquisition module is used to acquire a sample data set, wherein the sample data set includes a plurality of sample image pairs, and the sample image pairs include a training image and a corresponding beautified image;

[0037] A first feature extraction module, configured to perform feature extraction processing on the training image;

[0038] A model training module is used to train a preset beautification coefficient estimation model using the training image after feature extraction processing and the corresponding beautified image to obtain a target beautification coefficient estimation model;

[0039] A target image acquisition module is used to acquire a target image;

[0040] A second feature extraction module is used to perform feature extraction processing on the target image;

[0041] a model processing module, configured to process the target image after feature extraction using the target beautification coefficient estimation model to obtain a beautification estimation coefficient of the target image;

[0042] A beautification processing module is used to perform beautification processing on the target image according to the beautification estimation coefficient to obtain a target beautified image corresponding to the target image.

[0043] In a preferred embodiment of the present invention, the system further comprises:

[0044] The beautification evaluation module is used to process the target image and the target beautified image corresponding to the target image using a preset beautification quality evaluation model to obtain the probability that the target image is effectively beautified, and to determine whether the target image has been effectively beautified based on the probability.

[0045] In a preferred embodiment of the present invention, the system further comprises:

[0046] The display module is used to display the target beautified image corresponding to the target image when the target image has been effectively beautified, and to display the target image when the target image has not been effectively beautified.

[0047] In a preferred embodiment of the present invention, the system further comprises:

[0048] a first downsampling module, configured to perform downsampling processing on the training image before the first feature extraction module performs feature extraction processing on the training image;

[0049] The second downsampling module is configured to perform downsampling processing on the target image before the second feature extraction module performs feature extraction processing on the target image.

[0050] In a preferred embodiment of the present invention, the beautification coefficient estimation model is a bilateral grid network model as shown in the following formula (3):

[0051]

[0052] Among them, S represents the beautification estimation coefficient, W p represents the network weight of point p on the input image, and F represents the preset domain range of point p. represents the Gaussian kernel function in the spatial domain, represents the pixel value domain Gaussian kernel function, ||pq|| represents the pixel distance between point p and point q, |I p -I q | represents the pixel difference between point p and point q, I q The pixel representing the q point.

[0053] In a preferred embodiment of the present invention, the model training module includes:

[0054] a model processing unit, configured to process the training image after feature extraction using the beautification coefficient estimation model to obtain a beautification estimation coefficient corresponding to the training image;

[0055] a beautification unit, configured to perform beautification processing on the training image according to the beautification estimation coefficient to obtain a target beautified image corresponding to the training image;

[0056] A loss function construction unit, configured to construct a loss function based on the target beautified image and the beautified image corresponding to the training image;

[0057] An iterative training unit is used to adjust the parameters of the beautification coefficient estimation model based on the loss function and re-call the model processing unit until the loss function meets a predetermined condition to obtain a target beautification coefficient estimation model.

[0058] In a preferred embodiment of the present invention, the loss function construction unit includes:

[0059] The reconstruction loss calculation subunit is used to calculate the reconstruction loss L according to formula (5) based on the target beautified image and the beautified image corresponding to the training image. r :

[0060]

[0061] Wherein, I′ represents the beautified image corresponding to the training image, represents the target beautified image corresponding to the training image, and S represents the corresponding beautification estimation coefficient;

[0062] The color loss calculation subunit is used to calculate the color loss L according to formula (6) based on the target beautified image and the beautified image corresponding to the training image. c :

[0063]

[0064] Among them, F(I′) P represents a three-dimensional vector composed of the RGB pixel values ​​at point P in the beautified image corresponding to the training image, represents a three-dimensional vector composed of the RGB pixel values ​​at the point corresponding to the position of point P in the target beautified image corresponding to the training image, represents F(I′) P and The angle difference;

[0065] Loss combination subunit, for reconstructing the loss L r and the color loss L c Construct the loss function.

[0066] In order to achieve the above objectives, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0067] In order to achieve the above object, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0068] By adopting the above technical solution, the present invention has the following beneficial effects:

[0069] The present invention uses deep learning methods to learn training images and corresponding beautified images, and can quickly and effectively perform intelligent beautification processing on images, efficiently improve the quality of images without changing the image content, significantly save operating and maintenance costs, ensure the beauty and accuracy of image display, and effectively improve the user service experience in scenarios such as OTA. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 This is a flowchart of the intelligent image beautification method according to embodiment 1 of the present invention;

[0071] Figure 2 This is a flow chart of the intelligent image beautification method according to embodiment 2 of the present invention;

[0072] Figure 3 This is a structural block diagram of an intelligent image beautification system according to embodiment 3 of the present invention;

[0073] Figure 4 This is a structural block diagram of the model training module in Example 3 of the present invention;

[0074] Figure 5 This is a structural block diagram of a loss function construction unit in Example 3 of the present invention;

[0075] Figure 6 This is a structural block diagram of an intelligent image beautification system according to a fourth embodiment of the present invention;

[0076] Figure 7 This is a hardware architecture diagram of an electronic device according to embodiment 5 of the present invention. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0078] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0079] Example 1

[0080] This embodiment provides an image intelligent beautification method, such as Figure 1 As shown, the method includes the following steps:

[0081] S1. Obtain a sample dataset, comprising several sample image pairs, each comprising a training image and a corresponding beautified image. In this embodiment, a large number of images (e.g., 20,000) can be randomly sampled from an image library in advance. Multiple visual experts score the images based on factors such as lighting, hue, and clarity, with the mean of the scores serving as the aesthetic evaluation score for the corresponding image. The 1,000 images with the highest scores are selected as a portion of training images, and these training images themselves serve as their corresponding beautified images. Simultaneously, 1,000 images with lower scores are selected as another portion of training images. An artist uses an image beautification tool such as Photoshop to adjust these images in terms of color, hue, brightness, and contrast, resulting in 1,000 beautified images serving as the beautified images for the corresponding training images. Thus, a dataset containing 2,000 sample image pairs is constructed. It should be understood that the aforementioned numbers, such as 1,000 and 2,000, are merely examples, and the present invention does not impose any limitation on the number of training images.

[0082] S2: Downsample the training image to obtain a low-resolution image corresponding to the training image. For example, the training image is uniformly downsampled to a fixed resolution of 256×256. The purpose of downsampling in this embodiment is to reduce the computational complexity of subsequent processing. It should be noted that the present invention does not impose any restrictions on the downsampled resolution.

[0083] S3. Perform feature extraction on the downsampled training image (i.e., the low-resolution image obtained in step S2) to obtain a low-resolution feature image corresponding to the training image. Specifically, first, feature encoding can be performed on the low-resolution image using a VGG16 model pre-trained on ImageNet to obtain a feature encoding result. Feature extraction is then performed on the feature encoding result using a preset feature extraction network comprising three convolutional layers to obtain a corresponding low-resolution feature image.

[0084] S4, using the training image after feature extraction processing (ie the aforementioned low-resolution feature image) and the corresponding beautified image, a preset beautification coefficient estimation model is trained to obtain a target beautification coefficient estimation model.

[0085] For example, suppose the input image I is transformed into a beautified image through a series of transformations F. Then we have:

[0086]

[0087]

[0088] Where S = F -1 , S represents the beautification estimation coefficient.

[0089] In this embodiment, the beautification coefficient estimation model adopts the bilateral grid network model shown in the following formula (3):

[0090]

[0091] Among them, S represents the beautification estimation coefficient, W p represents the network weight of point p on the input image, and F represents the preset domain range of point p. represents the Gaussian kernel function in the spatial domain, represents the pixel value domain Gaussian kernel function, N represents the dimension, ||pq|| represents the pixel distance between point p and point q, |I p -I q | represents the pixel difference between point p and point q, I q The pixel representing the q point.

[0092] Specifically, this embodiment trains the beautification coefficient estimation model through the following steps:

[0093] S41 , using a beautification coefficient estimation model to process a training image after feature extraction, to obtain a beautification estimation coefficient S corresponding to the training image.

[0094] S42, based on the beautification estimation coefficient S, the training image I is beautified according to the following formula (4) to obtain the target beautified image corresponding to the training image

[0095]

[0096] It should be noted that the training image I in this step refers to the full-resolution training image before downsampling.

[0097] S43, constructing a loss function based on the target beautified image and the beautified image corresponding to the training image. In this embodiment, the process of constructing the loss function is as follows:

[0098] S431, based on the target beautified image and the beautified image corresponding to the training image, calculate the reconstruction loss L according to formula (5) r :

[0099]

[0100] Where I′ represents the beautified image corresponding to the training image, represents the target beautified image corresponding to the training image, and S represents the corresponding beautification estimation coefficient.

[0101] S432, based on the target beautified image and the beautified image corresponding to the training image, calculate the color loss L according to formula (6) c :

[0102]

[0103] Among them, F(I′) P Represents the three-dimensional vector composed of the RGB pixel values ​​at point P in the beautified image corresponding to the training image, Represents the three-dimensional vector composed of the RGB pixel values ​​at the point corresponding to the position of point P in the target beautified image corresponding to the training image, represents F(I′) P and angular difference.

[0104] S433, based on the aforementioned reconstruction loss L r and color loss L c Construct the loss function. In this embodiment, the reconstruction loss L r With color loss L c The loss function is obtained by adding them together.

[0105] By setting the reconstruction loss, this embodiment can constrain the grayscale of the target beautified image to be close to the grayscale of the beautified image; by setting the color loss, the color of the target beautified image can be constrained to be close to the color of the beautified image, thereby achieving a better beautification effect.

[0106] At step S44, the parameters of the beautification coefficient estimation model are adjusted based on the aforementioned loss function, and the process returns to step S31 to iteratively train the beautification coefficient estimation model until the loss function satisfies a predetermined condition (e.g., converges to a stable state and is less than a predetermined value). The beautification coefficient estimation model obtained at this point is then designated as the target beautification coefficient estimation model. In this embodiment, the parameters of the beautification coefficient estimation model are preferably updated using backpropagation of the loss.

[0107] S5, when the target image needs to be beautified, the target image is acquired and down-sampled and feature extracted. The down-sampling and feature extraction process of this step can refer to the above steps S2 and S3.

[0108] S6, using the aforementioned target beautification coefficient estimation model to process the target image after feature extraction to obtain a beautification estimation coefficient of the target image.

[0109] S7, processing the target image according to the beautification estimation coefficient of the target image to obtain the target beautified image corresponding to the target image. Specifically, the target beautified image corresponding to the target image can be obtained by the following formula (7):

[0110]

[0111] Among them, S0 represents the beautification estimation coefficient of the target image, and I0 represents the full-resolution target image.

[0112] In summary, since the beautified images corresponding to the training images have improved defects in dimensions such as lighting, picture clarity, and color tone, and have a higher aesthetic appeal, this embodiment uses deep learning methods to learn the training images and their corresponding beautified images, and can quickly and effectively perform intelligent beautification processing on the images, and efficiently improve the quality of the images without changing the image content, which can significantly save operating and maintenance costs, ensure the beauty and accuracy of the image display, and effectively improve the user service experience in scenarios such as OTA.

[0113] Example 2

[0114] like Figure 2 As shown, the image intelligent beautification method of this embodiment adds the following steps after step S7 of embodiment 1:

[0115] S8, using a preset beautification quality assessment model to process the target image and the target beautified image corresponding to the target image, obtain the probability that the target image is effectively beautified, and determine whether the target image has been effectively beautified based on the probability. For example, a probability threshold τ is set, and when the probability obtained is When , the target image is considered to have been effectively beautified, otherwise, the target image is considered to have not been effectively beautified.

[0116] Preferably, the beautification quality assessment model adopts a network model composed of multiple layers (e.g., 16 layers) of convolutional layers. In this embodiment, the beautification quality assessment model is iteratively trained using a number of images and labels indicating whether each image is effectively beautified. During training, the cross-entropy loss function L shown in the following formula (8) is used to learn the weight coefficients of the beautification quality assessment model until L meets a predetermined condition (e.g., converges to a stable state and is less than a predetermined value):

[0117]

[0118] in, It represents the probability of whether the image output by the beautification quality assessment model has been effectively beautified, and y represents the label of whether the corresponding image is effectively beautified.

[0119] S9: When it is predicted that the target image has been effectively beautified, display a target beautified image corresponding to the target image.

[0120] S10: When it is predicted that the target image has not been effectively beautified, display the target image.

[0121] Through the above steps, this embodiment can ensure the output quality of the beautified image.

[0122] It should be noted that for the sake of simplicity, Examples 1 and 2 are described as a series of actions. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0123] Example 3

[0124] This embodiment provides an intelligent image beautification system. Figure 3 As shown, the system 10 includes:

[0125] The sample acquisition module 11 is used to obtain a sample dataset, which includes several sample image pairs, each of which includes training images and corresponding beautified images. In this embodiment, a large number of images (e.g., 20,000) can be randomly extracted from an image library in advance. Multiple visual experts score the images based on factors such as lighting, hue, and clarity, and the average of the scores serves as the aesthetic evaluation score of the corresponding image. The 1,000 images with the highest scores are selected as a portion of training images, and these training images themselves serve as the corresponding beautified images. Simultaneously, 1,000 images with lower scores are selected as another portion of training images. An artist uses image beautification tools such as Photoshop to adjust these images in terms of color, hue, brightness, and contrast, obtaining 1,000 beautified images as the beautified images of the corresponding training images. Thus, a dataset containing 2,000 sample image pairs is constructed. It should be understood that the aforementioned numbers, such as 1,000 and 2,000, are merely examples, and the present invention does not impose any limitation on the number of training images.

[0126] The first downsampling module 12 is configured to downsample the training image to obtain a low-resolution image corresponding to the training image. For example, the training image is uniformly downsampled to a fixed resolution of 256×256. The purpose of downsampling in this embodiment is to reduce the computational complexity of subsequent processing. It should be noted that the present invention does not impose any limitations on the downsampling resolution.

[0127] The first feature extraction module 13 is configured to perform feature extraction on the downsampled training image (i.e., the low-resolution image obtained in step S2) to obtain a low-resolution feature image corresponding to the training image. Specifically, the low-resolution image may be first feature-encoded using a VGG16 model pre-trained on ImageNet to obtain a feature encoding result; then, a feature extraction network comprising three convolutional layers is used to perform feature extraction on the feature encoding result to obtain a corresponding low-resolution feature image.

[0128] The model training module 14 is used to train a preset beautification coefficient estimation model using the training image after feature extraction (ie the aforementioned low-resolution feature image) and the corresponding beautified image to obtain a target beautification coefficient estimation model.

[0129] For example, suppose the input image I is transformed into a beautified image through a series of transformations F. Then we have:

[0130]

[0131]

[0132] Where S = F -1 , S represents the beautification estimation coefficient.

[0133] In this embodiment, the beautification coefficient estimation model adopts the bilateral grid network model shown in the following formula (3):

[0134]

[0135] Among them, S represents the beautification estimation coefficient, W p represents the network weight of point p on the input image, and F represents the preset domain range of point p. represents the spatial domain Gaussian kernel function and represents the pixel value domain Gaussian kernel function and N represents the dimension, ||pq|| represents the pixel distance between point p and point q, |I p -I q | represents the pixel difference between point p and point q, I q The pixel representing the q point.

[0136] Specifically, if Figure 4 As shown, the model training module 14 of this embodiment includes:

[0137] The model processing unit 141 is configured to process a training image after feature extraction using a beautification coefficient estimation model to obtain a beautification estimation coefficient S corresponding to the training image.

[0138] The beautification unit 142 is used to beautify the training image I according to the beautification estimation coefficient S according to the following formula (4) to obtain the target beautified image corresponding to the training image

[0139]

[0140] It should be noted that the training image I in this step refers to the full-resolution training image before downsampling.

[0141] The loss function construction unit 143 is used to construct a loss function based on the target beautified image and the beautified image corresponding to the training image. Figure 5 As shown, the loss function construction unit 143 includes:

[0142] The reconstruction loss calculation subunit 1431 is used to calculate the reconstruction loss L according to formula (5) based on the target beautified image and the beautified image corresponding to the training image. r :

[0143]

[0144] Where I′ represents the beautified image corresponding to the training image, represents the target beautified image corresponding to the training image, and S represents the corresponding beautification estimation coefficient.

[0145] The color loss calculation subunit 1432 is used to calculate the color loss L according to formula (6) based on the target beautified image and the beautified image corresponding to the training image. c :

[0146]

[0147] Among them, F(I′) P Represents the three-dimensional vector composed of the RGB pixel values ​​at point P in the beautified image corresponding to the training image, Represents the three-dimensional vector composed of the RGB pixel values ​​at the point corresponding to the position of point P in the target beautified image corresponding to the training image, represents F(I′) P and angular difference.

[0148] The loss combination subunit 1433 is used to reconstruct the loss L r and color loss L cConstruct the loss function. In this embodiment, the reconstruction loss L r With color loss L c The loss function is obtained by adding them together.

[0149] By setting the reconstruction loss, this embodiment can constrain the grayscale of the target beautified image to be close to the grayscale of the beautified image; by setting the color loss, the color of the target beautified image can be constrained to be close to the color of the beautified image, thereby achieving a better beautification effect.

[0150] Iterative training unit 144 is configured to adjust the parameters of the beautification coefficient estimation model based on the aforementioned loss function and re-invoke model processing unit 141 for iterative training until the loss function satisfies a predetermined condition (e.g., converges to a stable state and is less than a predetermined value). The beautification coefficient estimation model obtained at this point is designated as the target beautification coefficient estimation model. In this embodiment, backpropagation of the loss is preferably used to update the parameters of the beautification coefficient estimation model.

[0151] The target image acquisition module 15 is used to acquire the target image.

[0152] The second downsampling module 16 is configured to downsample the target image.

[0153] The second feature extraction module 17 is used to perform feature extraction processing on the target image.

[0154] The model processing module 18 is used to process the target image after feature extraction using the target beautification coefficient estimation model to obtain the beautification estimation coefficient of the target image.

[0155] The beautification processing module 19 is used to process the target image according to the beautification estimation coefficient of the target image to obtain the target beautified image corresponding to the target image. Specifically, the target beautified image corresponding to the target image can be obtained by the following formula (7):

[0156]

[0157] Among them, S0 represents the beautification estimation coefficient of the target image, and I0 represents the full-resolution target image.

[0158] In summary, since the beautified images corresponding to the training images have improved defects in dimensions such as lighting, picture clarity, and color tone, and have a higher aesthetic appeal, this embodiment uses deep learning methods to learn the training images and their corresponding beautified images, and can quickly and effectively perform intelligent beautification processing on the images, and efficiently improve the quality of the images without changing the image content, which can significantly save operating and maintenance costs, ensure the beauty and accuracy of the image display, and effectively improve the user service experience in scenarios such as OTA.

[0159] Example 4

[0160] like Figure 6 As shown, the image intelligent beautification system 10 of this embodiment adds the following modules on the basis of embodiment 3:

[0161] The beautification evaluation module 21 is used to process the target image and the target beautified image corresponding to the target image using a preset beautification quality evaluation model to obtain the probability that the target image is effectively beautified, and to determine whether the target image is effectively beautified based on the probability. For example, a probability threshold τ is set, and when the probability obtained is When , the target image is considered to have been effectively beautified, otherwise, the target image is considered to have not been effectively beautified.

[0162] Preferably, the beautification quality assessment model adopts a network model composed of multiple layers (e.g., 16 layers) of convolutional layers. In this embodiment, the beautification quality assessment model is iteratively trained using a number of images and labels indicating whether each image is effectively beautified. During training, the cross-entropy loss function L shown in the following formula (8) is used to learn the weight coefficients of the beautification quality assessment model until L meets a predetermined condition (e.g., converges to a stable state and is less than a predetermined value):

[0163]

[0164] in, It represents the probability of whether the image output by the beautification quality assessment model has been effectively beautified, and y represents the label of whether the corresponding image is effectively beautified.

[0165] The display module 22 is used to display the target beautified image corresponding to the target image when it is predicted that the target image has been effectively beautified; and is also used to display the target image itself when it is predicted that the target image has not been effectively beautified.

[0166] Through the above modules, this embodiment can ensure the output quality of image beautification.

[0167] Example 5

[0168] This embodiment provides an electronic device, which can be expressed in the form of a computing device (for example, a server device), including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image intelligent beautification method provided in Embodiment 1 or 2 can be implemented.

[0169] Figure 7 The hardware structure diagram of this embodiment is shown in FIG. Figure 7 As shown, the electronic device 9 specifically includes:

[0170] At least one processor 91, at least one memory 92, and a bus 93 for connecting different system components (including the processor 91 and the memory 92), wherein:

[0171] The bus 93 includes a data bus, an address bus, and a control bus.

[0172] The memory 92 includes a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read-only memory (ROM) 923 .

[0173] Memory 92 also includes a program / utility 925 having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0174] The processor 91 executes various functional applications and data processing by running computer programs stored in the memory 92, such as the image intelligent beautification method provided in embodiment 1 or 2 of the present invention.

[0175] The electronic device 9 can further communicate with one or more external devices 94 (e.g., a keyboard, pointing device, etc.). Such communication can be performed via an input / output (I / O) interface 95. Furthermore, the electronic device 9 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 96. The network adapter 96 communicates with other modules of the electronic device 9 via a bus 93. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device 9, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0176] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0177] Example 6

[0178] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the image intelligent beautification method provided in embodiment 1 or 2 are implemented.

[0179] The readable storage medium may include, but is not limited to, a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0180] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to enable the terminal device to execute the steps of the image intelligent beautification method described in Example 1 or 2.

[0181] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or entirely on the remote device.

[0182] Although specific embodiments of the present invention have been described above, those skilled in the art will appreciate that these are merely illustrative and that the scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, and such changes and modifications are intended to fall within the scope of the present invention.

Claims

1. An intelligent image beautification method, characterized in that: The method includes: Obtain a sample data set, wherein the sample data set includes a plurality of sample image pairs, wherein the sample image pairs include a training image and a corresponding beautified image; Performing feature extraction processing on the training image; Using the training images processed by feature extraction and the corresponding beautified images, a preset beautification coefficient estimation model is trained to obtain a target beautification coefficient estimation model; Acquire a target image, and perform feature extraction processing on the target image; Processing the target image after feature extraction using the target beautification coefficient estimation model to obtain a beautification estimation coefficient of the target image; Performing beautification processing on the target image according to the beautification estimation coefficient to obtain a target beautified image corresponding to the target image; The step of training a preset beautification coefficient estimation model using the training image after feature extraction processing and the corresponding beautified image to obtain a target beautification coefficient estimation model includes: Using the beautification coefficient estimation model to process the training image after feature extraction, to obtain the beautification estimation coefficient corresponding to the training image; Beautify the training image according to the beautification estimation coefficient to obtain a target beautified image corresponding to the training image; Constructing a loss function based on the target beautified image and the beautified image corresponding to the training image; The parameters of the beautification coefficient estimation model are adjusted based on the loss function, and the step of processing the training image after feature extraction using the beautification coefficient estimation model is returned to until the loss function meets a predetermined condition, thereby obtaining a target beautification coefficient estimation model.

2. The method for intelligent image beautification according to claim 1, characterized in that: After obtaining the target beautified image corresponding to the target image, the method further includes: The target image and the target beautified image corresponding to the target image are processed using a preset beautification quality assessment model to obtain a probability that the target image is effectively beautified, and whether the target image is effectively beautified is determined based on the probability.

3. The method for intelligent image beautification according to claim 2, characterized in that: The method further comprises: When it is determined that the target image has been effectively beautified, displaying a target beautified image corresponding to the target image; When it is determined that the target image has not been effectively beautified, the target image is displayed.

4. The method for intelligent image beautification according to claim 1, characterized in that: Before performing feature extraction processing on the training image, the method further includes: performing downsampling processing on the training image; Before performing feature extraction processing on the target image, the method further includes: performing downsampling processing on the target image.

5. The method for intelligent image beautification according to claim 1, characterized in that: The beautification coefficient estimation model is a bilateral grid network model as shown in the following formula (3): Among them, S represents the beautification estimation coefficient, W p represents the network weight of point p on the input image, F represents the preset domain range of point p, represents the Gaussian kernel function in the spatial domain, represents the pixel value range Gaussian kernel function, ||pq|| represents the pixel distance between point p and point q, |I p -I q | represents the pixel difference between point p and point q, I q Represents the pixel at point q.

6. The method for intelligent image beautification according to claim 1, characterized in that: The step of constructing a loss function based on the target beautified image and the beautified image corresponding to the training image comprises: Based on the target beautified image and the beautified image corresponding to the training image, the reconstruction loss L is calculated according to formula (5): r : Wherein, I′ represents the beautified image corresponding to the training image, represents the target beautified image corresponding to the training image, and S represents the corresponding beautification estimation coefficient; Based on the target beautified image and the beautified image corresponding to the training image, the color loss L is calculated according to formula (6): c : Among them, F(I′) P represents a three-dimensional vector composed of RGB pixel values ​​at point P in the beautified image corresponding to the training image, represents a three-dimensional vector composed of RGB pixel values ​​at the point corresponding to the position of point P in the target beautified image corresponding to the training image, Denote F(I′) P and The angle difference According to the reconstruction loss L r and the color loss L c Construct the loss function.

7. An intelligent image beautification system, characterized in that: include: A sample acquisition module, used to acquire a sample data set, wherein the sample data set includes a plurality of sample image pairs, wherein the sample image pairs include a training image and a corresponding beautified image; A first feature extraction module, used for performing feature extraction processing on the training image; A model training module is used to train a preset beautification coefficient estimation model using the training image after feature extraction processing and the corresponding beautified image to obtain a target beautification coefficient estimation model; A target image acquisition module, used for acquiring a target image; A second feature extraction module, used for performing feature extraction processing on the target image; A model processing module, used to process the target image after feature extraction using the target beautification coefficient estimation model to obtain a beautification estimation coefficient of the target image; A beautification processing module, used for performing beautification processing on the target image according to the beautification estimation coefficient to obtain a target beautified image corresponding to the target image; The model training module includes: A model processing unit, used to process the training image after feature extraction using the beautification coefficient estimation model to obtain a beautification estimation coefficient corresponding to the training image; A beautification unit, configured to perform beautification processing on the training image according to the beautification estimation coefficient to obtain a target beautified image corresponding to the training image; A loss function construction unit, used to construct a loss function based on the target beautified image and the beautified image corresponding to the training image; The iterative training unit is used to adjust the parameters of the beautification coefficient estimation model based on the loss function and re-call the model processing unit until the loss function meets a predetermined condition to obtain a target beautification coefficient estimation model.

8. The intelligent image beautification system according to claim 7, characterized in that: The system further comprises: The beautification evaluation module is used to process the target image and the target beautification image corresponding to the target image using a preset beautification quality evaluation model to obtain the probability that the target image is effectively beautified, and determine whether the target image is effectively beautified based on the probability.

9. The intelligent image beautification system according to claim 8, characterized in that: The system further comprises: The display module is used to display the target beautified image corresponding to the target image when the target image has been effectively beautified, and to display the target image when the target image has not been effectively beautified.

10. The intelligent image beautification system according to claim 7, characterized in that: The system further comprises: A first downsampling module, configured to perform downsampling processing on the training image before the first feature extraction module performs feature extraction processing on the training image; The second downsampling module is used to perform downsampling processing on the target image before the second feature extraction module performs feature extraction processing on the target image.

11. The intelligent image beautification system according to claim 7, characterized in that: The beautification coefficient estimation model is a bilateral grid network model as shown in the following formula (3): Among them, S represents the beautification estimation coefficient, W p represents the network weight of point p on the input image, F represents the preset domain range of point p, represents the Gaussian kernel function in the spatial domain, represents the pixel value range Gaussian kernel function, ||pq|| represents the pixel distance between point p and point q, |I p -I q | represents the pixel difference between point p and point q, I q Represents the pixel at point q.

12. The intelligent image beautification system according to claim 7, characterized in that: The loss function construction unit includes: The reconstruction loss calculation subunit is used to calculate the reconstruction loss L according to formula (5) based on the target beautified image and the beautified image corresponding to the training image. r : Wherein, I′ represents the beautified image corresponding to the training image, represents the target beautified image corresponding to the training image, and S represents the corresponding beautification estimation coefficient; The color loss calculation subunit is used to calculate the color loss L according to formula (6) based on the target beautified image and the beautified image corresponding to the training image. c : Among them, F(I′) P represents a three-dimensional vector composed of RGB pixel values ​​at point P in the beautified image corresponding to the training image, represents a three-dimensional vector composed of RGB pixel values ​​at the point corresponding to the position of point P in the target beautified image corresponding to the training image, Denote F(I′) P and The angle difference The loss combination subunit is used to reconstruct the loss L r and the color loss L c Construct the loss function.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image intelligent beautification method according to any one of claims 1 to 6 is implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image intelligent beautification method according to any one of claims 1 to 6 are implemented.

Images