Training method, device, medium and equipment for aesthetic image generation model
By introducing aesthetic sense and clarity discriminant models in the training process of aesthetic sense map generation model, the problem that aesthetic sense maps in the existing technology may ignore clarity is solved, and the generated aesthetic sense maps are beautiful and clear in the light and dark dimensions.
Patent Information
- Application Number
- CN202210001642.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-04
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2042-01-04
AI Technical Summary
The prior art is difficult to effectively generate aesthetic images that conform to the aesthetics of ordinary people, and the generated aesthetic images may ignore the clarity of the pictures, resulting in the problem of local images being not fine or too high brightness.
By introducing aesthetic discriminant model and clarity discriminant model, the training process of aesthetic graph generation model is supervised, and the gap in aesthetic discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discriminant discrimin
The generated aesthetic pictures are not only bright but also clear, but more in line with the aesthetics of ordinary people, solving the problem that aesthetic pictures may ignore clarity in the existing technology.
Smart Images
Figure CN114463199B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the field of information technology. More specifically, the embodiments of the present disclosure relate to a training method, device, medium and equipment for an aesthetic image generation model. Background Art
[0002] Aesthetic images are obtained by beautifying and adjusting the original images. Compared with the original images, aesthetic images are more in line with the aesthetic tastes of ordinary people.
[0003] Based on this, a technical solution is needed that can effectively generate aesthetic images. Summary of the invention
[0004] In this context, the embodiments of the present disclosure are intended to provide a method, apparatus, medium, and device for training an aesthetic image generation model so as to generate relatively clear aesthetic images.
[0005] In a first aspect of the embodiments of the present disclosure, a method for training an aesthetic image generation model is provided, comprising:
[0006] Input the original image into the aesthetic image generation model to be trained, and output the first image;
[0007] Inputting the first image and the second image obtained by aesthetically enhancing the original image into the aesthetic discrimination model respectively, and determining the aesthetic discrimination difference between the first image and the second image;
[0008] Inputting the first image set into the aesthetic image generation model and outputting the second image set; the first image set includes the blurred original image and / or the blurred second image;
[0009] Inputting the first image set and the second image set into a clarity discrimination model respectively, and determining a clarity discrimination gap between the first image set and the second image set;
[0010] The aesthetic image generation model is trained with the training goal of reducing the gap between beauty judgment and clarity judgment.
[0011] In some embodiments, it also includes:
[0012] Inputting the original image and the first image into the image content classification model respectively, and obtaining content features output by a feature extraction network in the image content classification model;
[0013] The training goal also includes: reducing the gap between the content features corresponding to the original image and the content features corresponding to the first image.
[0014] In some embodiments, it also includes:
[0015] Input the second image into the aesthetic image generation model, and output a third image;
[0016] Determine a pixel value matrix of the second picture and a pixel value matrix of the third picture;
[0017] The training goal also includes: reducing the difference between the absolute value of the pixel value matrix of the second picture and the absolute value of the pixel value matrix of the third picture.
[0018] In some embodiments, it also includes:
[0019] Before training the aesthetic image generation model, the first image is marked as not aesthetically pleasing, the second image is marked as aesthetically pleasing, and the aesthetic discrimination model is trained.
[0020] In some embodiments, it also includes:
[0021] The training of the aesthetic image generation model is suspended, and the training of the aesthetic discrimination model is continued. Then, based on the trained aesthetic discrimination model, the training of the aesthetic image generation model is continued.
[0022] In some embodiments, it also includes:
[0023] Before training the aesthetic image generation model, the first image set is marked as clear, the blurred first image set is marked as unclear, and a clarity discrimination model is trained.
[0024] In some embodiments, it also includes:
[0025] The training of the aesthetic image generation model is suspended, and the training of the clarity discrimination model is continued. Then, based on the trained clarity discrimination model, the training of the aesthetic image generation model is continued.
[0026] In some embodiments, the image is input into the definition discrimination model in the following manner:
[0027] Determine the high-frequency information area of the image, where the high-frequency information area is the area in the image where the gray value changes by a greater than a specified amount;
[0028] The image corresponding to the high-frequency information area of the picture is input into the clarity discrimination model.
[0029] In a second aspect of the embodiments of the present disclosure, a method for generating an aesthetic image is provided, comprising:
[0030] Get the target image to be beautified;
[0031] The target image is input into the aesthetic image generation model, and the aesthetic image corresponding to the target image is output.
[0032] In a third aspect of the embodiments of the present disclosure, a training device for an aesthetic image generation model is provided, comprising:
[0033] A first output module, inputting the original image into the aesthetic image generation model to be trained, and outputting a first image;
[0034] A first discrimination module inputs the first image and the second image obtained by aesthetically enhancing the original image into an aesthetic discrimination model, and determines the aesthetic discrimination difference between the first image and the second image;
[0035] A second output module, inputting the first image set into the aesthetic image generation model, and outputting a second image set; the first image set includes the blurred original image and / or the blurred second image;
[0036] A second discrimination module inputs the first image set and the second image set into a definition discrimination model respectively, and determines a definition discrimination gap between the first image set and the second image set;
[0037] The training module trains the aesthetic image generation model with the goal of reducing the aesthetic judgment gap and the clarity judgment gap.
[0038] In a fourth aspect of the embodiments of the present disclosure, there is provided an aesthetic image generating device, comprising:
[0039] Acquisition module, to obtain the target image to be beautified;
[0040] The generation module inputs the target image into the aesthetic image generation model and outputs the aesthetic image corresponding to the target image.
[0041] In a fifth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect or the second aspect is implemented.
[0042] In a sixth aspect of an embodiment of the present disclosure, a computing device is provided, comprising a memory and a processor; the memory is used to store computer instructions executable on the processor, and the processor is used to implement the method described in the first aspect or the second aspect when executing the computer instructions.
[0043] In the above-mentioned technical solution provided in the present disclosure, an aesthetic discrimination model and a clarity discrimination model are used to supervise the training process of the aesthetic image generation model, wherein the aesthetic discrimination model is used to discriminate the aesthetic differences in the light and dark dimensions of the image, and the clarity discrimination model is used to discriminate the clarity differences of the image.
[0044] Through the above technical solution, the aesthetic images generated by the trained aesthetic image generation model are not only gorgeous but also clear, and such aesthetic images are more in line with the aesthetic tastes of ordinary people. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:
[0046] Figure 1 An exemplary method for training an aesthetic image generation model is provided;
[0047] Figure 2 Provide original images and manually adjusted images as examples;
[0048] Figure 3 The relationship between the aesthetic image generation model and the discriminant model under the GAN architecture is provided as an example;
[0049] Figure 4 An exemplary method of providing a training device for an aesthetic image generation model is provided;
[0050] Figure 5 An aesthetic image generation device is exemplarily provided;
[0051] Figure 6 A computer-readable storage medium is provided.
[0052] Figure 7 A schematic diagram of the structure of a computing device is provided as an example.
[0053] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. The number of any element in the drawings is for example rather than limitation, and any naming is only for distinction and does not have any limiting meaning. DETAILED DESCRIPTION
[0054] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.
[0055] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.
[0056] According to an embodiment of the present disclosure, a training method, device, medium and equipment for an aesthetic image generation model are proposed.
[0057] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.
[0058] In some business scenarios, it is necessary to beautify pictures so that the beautified pictures can better meet the aesthetic taste of ordinary people and attract the attention of users. For example, in the business scenario of information recommendation, the browsing entrances corresponding to several pieces of information can be displayed in the information recommendation interface. In order to attract users to click on the browsing entrance of the information, the browsing entrance can display a cover picture, which is generally obtained by beautifying the pictures contained in the information. The beautified pictures can be called aesthetic pictures.
[0059] Although manual beautification of the original image can produce an aesthetic image with good aesthetic effect, it is labor-intensive and inefficient when faced with the need to generate a large number of aesthetic images.
[0060] Therefore, we can adopt the method of training an aesthetic image generation model, and use the trained aesthetic image generation model to replace manual work to generate aesthetic images based on the original image. However, how to make the aesthetic image generated by the aesthetic image generation model have a higher aesthetic effect (as close to the artificial beautification effect as possible) is a technical problem that needs to be solved urgently.
[0061] An optional method for training an aesthetic image generation model is to introduce an aesthetic discrimination model to supervise the training process of the aesthetic image generation model. However, due to the algorithmic limitations of the existing aesthetic image discrimination model, its aesthetic discrimination ability is not as comprehensive as that considered when manually performing aesthetic discrimination. It only mechanically focuses on the aesthetic degree of the image in the brightness dimension (i.e., gorgeousness). Therefore, the aesthetic image generation model trained based on this method can usually only adjust the original image in the brightness dimension, such as increasing the brightness of part of the image, or enhancing the contrast of part of the image. This can certainly improve the gorgeousness of the image, but ignores the clarity of the image. The resulting aesthetic image may have negative effects such as local images that are not fine, local images that are too bright and white, etc. Such aesthetic images are actually not in line with the aesthetic taste of ordinary people.
[0062] To this end, another method for training the aesthetic image generation model, which will be described in detail later, is to introduce not only an aesthetic discrimination model to supervise the training process of the aesthetic image generation model, but also a clarity discrimination model to supervise the training process of the aesthetic image generation model.
[0063] Specifically, during the training process, on the one hand, the aesthetic discrimination model is used to respectively discriminate the aesthetics of the first image (the image generated after the original image is passed through the aesthetic image generation model) and the second image (the image obtained after aesthetically enhancing the original image), and determine the aesthetic discrimination difference between the two. The smaller the aesthetic discrimination difference is, the better the aesthetic effect (i.e., the degree of gorgeousness) of the aesthetic image generated by the aesthetic image generation model in the light and dark dimension is; on the other hand, the clarity discrimination model is used to respectively discriminate the clarity of the first image set (the original image and / or the second image) and the second image set (the image set generated after the blurred first image set is passed through the aesthetic image generation model), and determine the clarity discrimination difference between the two. The smaller the clarity discrimination difference is, the smaller the difference in clarity between the aesthetic image generated by the aesthetic image generation model based on the blurred image and the non-blurred image is, which means that the aesthetic image generation model has a stronger clarity improvement ability. The training goal of the aesthetic image generation model is to simultaneously reduce the aesthetic perception judgment gap and the clarity judgment gap, so as to train the aesthetic image generation model.
[0064] Through the above technical solution, in the process of training the aesthetic image generation model, the clarity discrimination model can effectively restrict the algorithmic limitations of the aesthetic discrimination model, so that the aesthetic image generation model can learn how to improve the gorgeousness of the image while ensuring that the image is clear enough. The aesthetic image generated by the aesthetic image generation model trained in this way is not only gorgeous but also clear, and such aesthetic images are more in line with the aesthetic taste of ordinary people.
[0065] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.
[0066] Figure 1 An exemplary method for training an aesthetic image generation model is provided, comprising the following steps:
[0067] S100: Input the original image into the aesthetic image generation model to be trained, and output a first image.
[0068] S102: Inputting the first image and the second image obtained by aesthetically enhancing the original image into an aesthetic discrimination model respectively, and determining the aesthetic discrimination gap between the first image and the second image.
[0069] S104: Input the fuzzy first image set into the aesthetic image generation model, and output a second image set.
[0070] S106: Input the first image set and the second image set into the clarity discrimination model respectively, and determine the clarity discrimination gap between the first image set and the second image set.
[0071] S108: Training an aesthetic image generation model with the goal of reducing the aesthetic judgment gap and the clarity judgment gap.
[0072] Figure 1 The figure shows the process of training the aesthetic image generation model based on a single sample data (i.e., the original image). In the field of artificial intelligence, it is usually necessary to train a model based on a collection of multiple sample data to achieve a better model training effect. Therefore, it is easy to understand that the process of training the aesthetic image generation model can be performed based on each original image in the original image collection. Figure 1 For the convenience of description, the following is based on a single original image. Figure 1 The method shown is described as an example.
[0073] The original image can be understood as the image to be beautified, and the original image can be regarded as an image with low aesthetics. After the original image is input into the aesthetic image generation model to be trained, the first output image can be understood as the aesthetic image obtained by beautifying the original image by the aesthetic image generation model in an untrained state.
[0074] The second image is a picture obtained by aesthetically enhancing the original image. The aesthetic enhancement here can be understood as a relatively comprehensive aesthetic enhancement, which is a picture that is both gorgeous and clear. Therefore, the second image can be regarded as a picture with high aesthetics.
[0075] It should be noted that the above-mentioned aesthetic enhancement method can be to beautify the original image manually, or to beautify the original image by training an artificial intelligence model with other technical concepts different from the technical concepts provided by the present disclosure. Figure 2 The original image and the manually adjusted image (the image after artificial aesthetic enhancement) are provided as examples.
[0076] The aesthetic discrimination model usually only focuses on the aesthetics (brightness) in the light and dark dimension, while the clarity discrimination model focuses on the clarity of the image. Usually, the input of the aesthetic discrimination model is the image, and the output is the aesthetics of the image. Alternatively, the input of the aesthetic discrimination model is two images, and the output is the judgment result of which image is more beautiful.
[0077] The aesthetic judgment gap between the first picture and the second picture can be understood as the gap between the aesthetic judgment result of the first picture and the aesthetic judgment result of the second picture. The aesthetic judgment result can usually be a probability value representing the possibility of having beauty. Alternatively, the aesthetic judgment gap between the first picture and the second picture can be understood as a binary result of which of the first picture and the second picture is more aesthetically pleasing, based on the second picture.
[0078] In addition, in some embodiments, the picture itself may be used as an input of the clarity discrimination model, so that the clarity discrimination model discriminates the clarity of the picture based on the picture as a whole.
[0079] In other embodiments, it is considered that the main factor that determines whether the image is clear is the high-frequency information area in the image. The high-frequency information area is the area in the image where the grayscale value changes by a greater than a specified amplitude. The high-frequency information area is usually the edge where the main image and the background image in the image are connected. The faster the grayscale value changes in the high-frequency information area, the higher the corresponding frequency, the more obvious the distinction between the main image and the background image in the image, and the clearer the image. Therefore, the high-frequency information area of the image can be used as the input of the clarity discrimination model, so that the clarity discrimination model focuses on the high-frequency information area to judge the clarity of the image.
[0080] In step S204, the first image set may include the original image and / or the second image. The so-called blurred first image set may refer to an image set after blurring the images in the first image set. The blurring process here may be understood as adding noise to the image so that some image areas in the image become blurred.
[0081] It is easy to understand that if the first image set only includes the original image or the second image, the second image set will only include the aesthetic image corresponding to the corresponding blurred original image, or the aesthetic image corresponding to the corresponding blurred second image.
[0082] It is also easy to understand that the first image set and the second image set are respectively input into the clarity discrimination model, which can actually be the clarity discrimination model performing clarity discrimination on each image in the first image set and performing clarity discrimination on each image in the second image set. Alternatively, the input of the clarity discrimination model can be two image sets, and the output is the discrimination result of which image set is clearer.
[0083] The set of clarity judgment results of each image in the first image set is the clarity judgment result of the first image set; the set of clarity judgment results of each image in the second image set is the clarity judgment result of the second image set. The clarity judgment gap between the first image set and the second image set can be the gap between the clarity judgment result of the first image set and the clarity judgment result of each image in the second image set. The clarity judgment result can usually be a probability value that characterizes the possibility of clarity. Alternatively, the clarity judgment gap between the first image set and the second image set can also be understood as a binary result of which one of the first image set and the second image set is clearer, based on the first image set.
[0084] In some embodiments, a trained, ready-made aesthetic discrimination model and / or clarity discrimination model may be selected.
[0085] In other embodiments, a sample data set (ie, a set of original images) used to train an aesthetic image generation model may be used in advance to train an aesthetic discrimination model and / or a clarity discrimination model.
[0086] Specifically, before training the aesthetic image generation model, the first image can be marked as not aesthetically pleasing and the second image can be marked as aesthetically pleasing, and the aesthetic discrimination model can be trained. Alternatively, before training the aesthetic image generation model, the first image set can be marked as clear and the blurred first image set can be marked as unclear, and the clarity discrimination model can be trained.
[0087] In addition, during the training of the aesthetic image generation model, after several iterations of the aesthetic image generation model, the training of the aesthetic image generation model can be suspended, and the training of the aesthetic discrimination model can be continued, so as to further improve the effect of the aesthetic discrimination model by using the aesthetic image generation model that has already had a certain effect. Then, based on the aesthetic discrimination model that has continued to be trained, the training of the aesthetic image generation model can be continued.
[0088] Similarly, the training of the aesthetic image generation model can be paused, and the training of the clarity discrimination model can be continued. Then, based on the trained clarity discrimination model, the training of the aesthetic image generation model can be continued.
[0089] In other words, model training can be performed in the order of training the discriminant model (aesthetic discrimination model and / or clarity discriminant model) - training the aesthetic image generation model - continuing to train the discriminant model - continuing to train the aesthetic image generation model - ... - continuing to train the discriminant model - continuing to train the aesthetic image generation model. This can be understood as adopting the training idea of adversarial learning. For example, the GAN architecture in adversarial learning can be used to train the aesthetic image generation model and the discriminant model. Figure 3 The relationship between the aesthetic image generation model and the discriminant model under the GAN architecture is provided as an example.
[0090] In addition, in some embodiments, in order to ensure that the aesthetic sense generated by the aesthetic image generation model does not lose the content of the original image as much as possible, the original image and the first image can be respectively input into the image content classification model to obtain the content features output by the feature extraction network in the image content classification model. Correspondingly, the training goal can also include: reducing the gap between the content features corresponding to the original image and the content features corresponding to the first image. Of course, other technical means can also be used to obtain the content features of the original image and the first image, which will not be given as examples here.
[0091] In some embodiments, considering that the original image input into the aesthetic image generation model may have a relatively high aesthetic value, in order to let the aesthetic image generation model learn not to make too many modifications to the original image with high aesthetic value, the second image can also be input into the aesthetic image generation model, and the third image is output, and then the pixel value matrix of the second image and the pixel value matrix of the third image are determined. Correspondingly, the training goal can also include: reducing the difference between the absolute value of the pixel value matrix of the second image and the absolute value of the pixel value matrix of the third image. Of course, in addition to the pixel value matrix, other data that can characterize the visual features of the second image and the third image can also be obtained, and examples are not given here.
[0092] After the aesthetic image generation model is trained, the target image to be beautified can be obtained, and then the target image is input into the aesthetic image generation model to output the aesthetic image corresponding to the target image.
[0093] Figure 4 An exemplary training device for an aesthetic image generation model is provided, comprising:
[0094] The first output module 401 inputs the original image into the aesthetic image generation model to be trained and outputs a first image;
[0095] The first discrimination module 402 inputs the first image and the second image obtained by aesthetically enhancing the original image into an aesthetic discrimination model, and determines the aesthetic discrimination difference between the first image and the second image;
[0096] The second output module 403 inputs the blurred first image set into the aesthetic image generation model and outputs a second image set; the first image set includes the original image and / or the second image;
[0097] The second discrimination module 404 inputs the first image set and the second image set into a definition discrimination model respectively, and determines a definition discrimination difference between the first image set and the second image set;
[0098] The training module 405 trains the aesthetic image generation model with the goal of reducing the aesthetic judgment gap and the clarity judgment gap.
[0099] In some embodiments, it also includes:
[0100] The content feature acquisition module 406 inputs the original image and the first image into the image content classification model respectively, and acquires the content features output by the feature extraction network in the image content classification model;
[0101] The training goal also includes: reducing the gap between the content features corresponding to the original image and the content features corresponding to the first image.
[0102] In some embodiments, it also includes:
[0103] The pixel value matrix determination module 407 inputs the second picture into the aesthetic image generation model and outputs a third picture;
[0104] Determine a pixel value matrix of the second picture and a pixel value matrix of the third picture;
[0105] The training goal also includes: reducing the difference between the absolute value of the pixel value matrix of the second picture and the absolute value of the pixel value matrix of the third picture.
[0106] In some embodiments, the training module 405, before training the aesthetic image generation model, marks the first image as not aesthetically pleasing and marks the second image as aesthetically pleasing, and trains the aesthetic discrimination model.
[0107] In some embodiments, the training module 405 suspends the training of the aesthetic image generation model, continues to train the aesthetic discrimination model, and then continues to train the aesthetic image generation model based on the trained aesthetic discrimination model.
[0108] In some embodiments, the training module 405, before training the aesthetic image generation model, marks the first image set as clear and the blurred first image set as unclear, and trains a clarity discrimination model.
[0109] In some embodiments, the training module 405 suspends the training of the aesthetic image generation model, continues to train the clarity discrimination model, and then continues to train the aesthetic image generation model based on the clarity discrimination model after continued training.
[0110] In some embodiments, the second discrimination module 404 inputs the image into the definition discrimination model in the following manner:
[0111] Determine the high-frequency information area of the image, where the high-frequency information area is the area in the image where the gray value changes by a greater than a specified amount;
[0112] The image corresponding to the high-frequency information area of the picture is input into the clarity discrimination model.
[0113] Figure 5 An aesthetic image generation device is exemplarily provided, comprising:
[0114] An acquisition module 501 is used to acquire a target image to be beautified;
[0115] The generation module 502 inputs the target image into the aesthetic image generation model and outputs the aesthetic image corresponding to the target image.
[0116] Figure 61 is a schematic diagram of a computer-readable storage medium provided by the present disclosure. A computer program is stored on the medium 140. When the program is executed by a processor, the method of any embodiment of the present disclosure is implemented.
[0117] The present disclosure also provides a computing device, including a memory and a processor; the memory is used to store computer instructions that can be executed on the processor, and the processor is used to implement the method of any embodiment of the present disclosure when executing the computer instructions.
[0118] Figure 7 is a schematic diagram of a computing device provided by the present disclosure, such as Figure 7 As shown, the computing device 15 may include, but is not limited to: a processor 151 , a memory 152 , and a bus 153 connecting different system components (including the memory 152 and the processor 151 ).
[0119] The memory 152 stores computer instructions, which can be executed by the processor 151, so that the processor 151 can execute the training method of the aesthetic image generation model of any embodiment of the present disclosure. The memory 152 may include a random access memory unit RAM 1521, a cache memory unit 1522 and / or a read-only memory unit ROM 1523. The memory 152 may also include: a program tool 1525 having a set of program modules 1524, and the program modules 1524 include but are not limited to: an operating system, one or more application programs, other program modules and program data, and one or more combinations of these program modules may include the implementation of a network environment.
[0120] The bus 153 may include, for example, a data bus, an address bus, and a control bus. The computing device 150 may also communicate with an external device 155 through an I / O interface 154, and the external device 155 may be, for example, a keyboard, a Bluetooth device, etc. The computing device 150 may also communicate with one or more networks through a network adapter 156, for example, the network may be a local area network, a wide area network, a public network, etc. As shown in the figure, the network adapter 156 may also communicate with other modules of the computing device 15 through the bus 153.
[0121] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.
[0122] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.
Claims
1. A training method for an aesthetic image generation model, comprising: Input the original image into the aesthetic image generation model to be trained, and output the first image; Inputting the first image and the second image obtained by aesthetically enhancing the original image into the aesthetic discrimination model respectively, and determining the aesthetic discrimination difference between the first image and the second image; Inputting the blurred first image set into the aesthetic image generation model, and outputting a second image set; the first image set includes the original image and / or the second image; Inputting the first image set and the second image set into a clarity discrimination model respectively, and determining a clarity discrimination gap between the first image set and the second image set; The aesthetic image generation model is trained with the goal of reducing the aesthetic judgment gap and the clarity judgment gap. The method further comprises: Input the second image into the aesthetic image generation model, and output a third image; Determine a pixel value matrix of the second picture and a pixel value matrix of the third picture; The training goal also includes: reducing the difference between the absolute value of the pixel value matrix of the second picture and the absolute value of the pixel value matrix of the third picture.
2. The method of claim 1, further comprising: Inputting the original image and the first image into the image content classification model respectively, and obtaining content features output by a feature extraction network in the image content classification model; The training goal also includes: reducing the gap between the content features corresponding to the original image and the content features corresponding to the first image.
3. The method of claim 1, further comprising: Before training the aesthetic image generation model, the first image is marked as not aesthetically pleasing, the second image is marked as aesthetically pleasing, and the aesthetic discrimination model is trained.
4. The method of claim 3, further comprising: The training of the aesthetic image generation model is suspended, and the training of the aesthetic discrimination model is continued. Then, based on the trained aesthetic discrimination model, the training of the aesthetic image generation model is continued.
5. The method of claim 1, further comprising: Before training the aesthetic image generation model, the first image set is marked as clear, the blurred first image set is marked as unclear, and a clarity discrimination model is trained.
6. The method of claim 5, further comprising: The training of the aesthetic image generation model is suspended, and the training of the clarity discrimination model is continued. Then, based on the trained clarity discrimination model, the training of the aesthetic image generation model is continued.
7. The method according to any one of claims 1 to 6, wherein the image is input into the definition discrimination model in the following manner: Determine the high-frequency information area of the image, where the high-frequency information area is the area in the image where the gray value changes by a greater than a specified amount; The image corresponding to the high-frequency information area of the picture is input into the clarity discrimination model.
8. A method for generating an aesthetic image, comprising: Get the target image to be beautified; The target image is input into the aesthetic image generation model trained based on the method according to any one of claims 1 to 7, and the aesthetic image corresponding to the target image is output.
9. A training device for an aesthetic image generation model, comprising: A first output module, inputting the original image into the aesthetic image generation model to be trained, and outputting a first image; A first discrimination module inputs the first image and the second image obtained by aesthetically enhancing the original image into an aesthetic discrimination model, and determines the aesthetic discrimination difference between the first image and the second image; A second output module, inputting the blurred first image set into the aesthetic image generation model, and outputting a second image set; the first image set includes the original image and / or the second image; A second discrimination module inputs the first image set and the second image set into a definition discrimination model respectively, and determines a definition discrimination gap between the first image set and the second image set; The training module trains the aesthetic image generation model with the goal of reducing the aesthetic judgment gap and the clarity judgment gap; A pixel value matrix determination module inputs the second image into the aesthetic image generation model and outputs a third image; Determine a pixel value matrix of the second picture and a pixel value matrix of the third picture; The training goal also includes: reducing the difference between the absolute value of the pixel value matrix of the second picture and the absolute value of the pixel value matrix of the third picture.
10. The apparatus according to claim 9, further comprising: A content feature acquisition module, inputting the original image and the first image into the image content classification model respectively, and acquiring content features output by a feature extraction network in the image content classification model; The training goal also includes: reducing the gap between the content features corresponding to the original image and the content features corresponding to the first image.
11. The apparatus as claimed in claim 9, wherein the training module, before training the aesthetic image generation model, marks the first image as not aesthetically pleasing and marks the second image as aesthetically pleasing, and trains the aesthetic discrimination model.
12. The device as claimed in claim 11, wherein the training module suspends the training of the aesthetic image generation model, continues to train the aesthetic discrimination model, and then continues to train the aesthetic image generation model based on the trained aesthetic discrimination model.
13. The apparatus as claimed in claim 9, wherein the training module, before training the aesthetic image generation model, marks the first image set as clear and marks the blurred first image set as unclear, and trains a clarity discrimination model.
14. The device as claimed in claim 13, wherein the training module suspends the training of the aesthetic image generation model, continues to train the clarity discrimination model, and then continues to train the aesthetic image generation model based on the clarity discrimination model after continued training.
15. The device according to any one of claims 9 to 14, wherein the second discrimination module inputs the image into the clarity discrimination model in the following manner: Determine the high-frequency information area of the image, where the high-frequency information area is the area in the image where the gray value changes by a greater than a specified amount; The image corresponding to the high-frequency information area of the picture is input into the clarity discrimination model.
16. An aesthetic image generating device, comprising: Acquisition module, to obtain the target image to be beautified; A generation module inputs a target image into an aesthetic image generation model trained based on the method described in any one of claims 1 to 7, and outputs an aesthetic image corresponding to the target image.
17. A computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method according to any one of claims 1 to 8.
18. A computing device, comprising a memory and a processor; the memory is used to store computer instructions executable on the processor, and the processor is used to implement the method according to any one of claims 1 to 8 when executing the computer instructions.