Model parameter updating method and device

Abstract

The invention discloses a model parameter updating method and device, and belongs to the technical field of image processing. The server inputs the M groups of first images into a first scoring model, scores the M groups of first images in one-to-one correspondence with the M beautification models, and outputs M groups of first scoring scores, model parameters of the first scoring model are used for representing image aesthetic tendency of the first user, and M is an integer greater than 1; based on the M groups of first scores, selecting a first beautification model in the M beautification models and N second images in the M groups of first images, N being a positive integer; based on the beautifying parameters of the N second images and the beautifying parameters of the S third images, a first loss value is calculated, and S is a positive integer smaller than or equal to N; and on the basis of the first loss value, updating model parameters of the first beautification model to obtain a second beautification model. Wherein each third image is an image of which the image content is matched with the second image in the beautified image corresponding to the first beautification model.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and specifically relates to a model parameter updating method and device thereof. Background Art

[0002] With the continuous development of electronic technology, the functions of electronic devices are becoming more and more powerful. For example, electronic devices can process captured images through built-in image processing algorithms to obtain images with higher image quality.

[0003] However, according to the above method, since the electronic device uses a built-in image processing algorithm to process the image, the image style of the processed image may be relatively simple. Therefore, how to make the processed image conform to the user's image aesthetic preference is an urgent problem to be solved. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a model parameter updating method and device thereof, which can make the processed image conform to the user's image aesthetic preference.

[0005] In a first aspect, an embodiment of the present application provides a model parameter updating method, the method comprising: inputting M groups of first images into a first scoring model, scoring the M groups of first images, and outputting M groups of first scoring values, wherein the M groups of first images correspond one-to-one to M beautification models, and the model parameters of the first scoring model are used to characterize the image aesthetic tendency of the first user, where M is an integer greater than 1; based on the M groups of first scoring values, selecting a first beautification model from the M beautification models and N second images from the M groups of first images, where N is a positive integer; calculating a first loss value based on the beautification parameters of the N second images and the beautification parameters of the S third images, where S is a positive integer less than or equal to N; and updating the model parameters of the first beautification model based on the first loss value to obtain a second beautification model. Wherein, each of the third images is an image whose image content matches one of the second images in the beautification image corresponding to the first beautification model.

[0006] In the second aspect, the embodiment of the present application provides a model parameter updating device, which includes: a processing module, a selection module and an updating module. The processing module is used to input M groups of first images into a first scoring model, score the M groups of first images, and output M groups of first scoring values, wherein the M groups of first images correspond one-to-one to M beautification models, and the model parameters of the first scoring model are used to characterize the image aesthetic tendency of the first user, where M is an integer greater than 1; the selection module is used to select the first beautification model among the M beautification models and the N second images among the M groups of first images based on the M groups of first scoring values ​​output by the processing module, where N is a positive integer less than or equal to M; the processing module is used to calculate a first loss value based on the beautification parameters of the N second images and the beautification parameters of the N third images selected by the selection module, where S is a positive integer less than or equal to N; the updating module is used to update the model parameters of the first beautification model based on the first loss value calculated by the processing module to obtain a second beautification model. Each of the third images is an image whose image content in the beautified image corresponding to the first beautification model matches one of the second images.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0011] In an embodiment of the present application, the server may input M groups of first images into a first scoring model, score the M groups of first images, output M groups of first scoring values, and may select the first beautification model from the M beautification models and N second images from the M groups of first images based on the M groups of first scoring values, calculate the first loss value based on the beautification parameters of the N second images and the beautification parameters of the S third images, and update the model parameters of the first beautification model based on the first loss value to obtain the second beautification model. Among them, the M groups of first images correspond one-to-one to the M beautification models, the model parameters of the first scoring model are used to characterize the image aesthetic tendency of the first user, M is an integer greater than 1, N is a positive integer, S is a positive integer less than or equal to N, and each third image is an image whose image content matches a second image in the beautification image corresponding to the first beautification model. Through this scheme, since the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user, the first scoring model can score the M groups of first images based on the image aesthetic tendency of the first user, that is, the M groups of first scoring scores output by the first scoring model can characterize the user's aesthetic evaluation or aesthetic preference for the M groups of first images; thereby ensuring that the first beautification model and the N second images selected based on the M groups of first scoring scores can be in line with the image aesthetic tendency of the first user. Therefore, based on the beautification parameters of the N second images and the beautification parameters of the S third images (the third images are beautified images corresponding to the first beautification model, and the image content matches the second images), the first loss value is calculated, and based on the first loss value, the model parameters of the first beautification model are updated, so that the second beautification model whose image aesthetic tendency conforms to the user's aesthetics can be obtained, that is, the image aesthetic tendency corresponding to the image beautified by the second beautification model conforms to the image aesthetic tendency of the first user. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flow chart of a model parameter updating method provided in an embodiment of the present application;

[0013] Figure 2 is an example schematic diagram of a beautification model structure provided in an embodiment of the present application;

[0014] Figure 3 is an example schematic diagram of obtaining U model parameters provided in an embodiment of the present application;

[0015] Figure 4 is a structural diagram of a model parameter updating device provided in an embodiment of the present application;

[0016] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0017] Figure 6 It is a hardware schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0019] The nouns or terms in the embodiments of the present application are explained below.

[0020] Mean Squared Error (MSE) loss function: It is a commonly used loss function in machine learning and deep learning, mainly used for regression problems. Its function is to evaluate the difference between the model's predicted value and the actual value, and optimize the model parameters by minimizing the difference.

[0021] Batch size: used to specify the number of training samples used in each model update.

[0022] Iteration number (epoch): It is used to indicate the process of traversing the entire training data set once in model training. When an epoch is completed, the model has performed a forward propagation and a backward propagation on all samples in the training set, thereby updating the model parameters.

[0023] Aesthetic Visual Analysis (AVA): is a large-scale aesthetics dataset containing more than 255,000 images, each with multiple ratings ranging from 1 to 10. This dataset is used for research on aesthetic assessment and image quality assessment.

[0024] Flickr Aesthetic Scores (Flicker-AES): is an aesthetic dataset based on the Flickr platform, containing a large number of user-uploaded images and their aesthetic scores.

[0025] Theme-based Aesthetic Dataset with 66000 Images (TAD66K): is a theme-oriented aesthetic dataset containing 66,000 images covering 47 different themes. The images of the 47 themes in the theme-based aesthetic dataset can include portrait images, landscape images, animal images, etc.

[0026] Parameter Adjustment for Image Quality (PARA): is a dataset focused on image parameter adjustment, which contains a large number of images with different parameter adjustments and their quality scores. For example, in the dataset of image parameter adjustment, image quality parameters may include brightness, contrast, saturation, sharpness, etc.

[0027] Konstanz Image Quality 10K (KonIQ-10K): is a large-scale image quality assessment dataset containing more than 10,000 images and their quality scores.

[0028] Spatial Pooling for Image Quality Assessment (SPAQ): is an image quality assessment dataset based on spatial pooling, which contains a large number of images with different distortions and their quality scores.

[0029] Contrastive Language-Image Pre-training (CLIP): It is used to pre-train with a large amount of paired image and text data to learn the alignment relationship between images and text.

[0030] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0031] The model parameter updating method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0032] The model parameter updating method and device provided in the embodiments of the present application can be applied to scenes where images are processed, such as beautified, to obtain images with higher image quality.

[0033] The model parameter updating method provided in the embodiment of the present application can be executed by a model parameter updating device, which can be an electronic device, or a functional module and an entity module in the electronic device that can implement the model parameter updating method. The specific method can be determined according to actual use requirements, and the embodiment of the present application does not limit it. The following takes the electronic device as a server as an example, and the execution of the model parameter updating method by the server as an example to illustrate the model parameter updating method provided in the embodiment of the present application.

[0034] The present application embodiment provides a model parameter updating method, Figure 1 FIG. 1 is a flow chart showing a method for updating model parameters provided in an embodiment of the present application. Figure 1 As shown, the model parameter updating method provided in the embodiment of the present application may include steps 101 to 104.

[0035] Step 101: The server inputs M groups of first images into a first scoring model, scores the M groups of first images, and outputs the M groups of first scoring scores.

[0036] Among them, the M groups of first images correspond one-to-one to the M beautification models, the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user, and M is an integer greater than 1.

[0037] It can be understood that the above-mentioned M groups of first images correspond one-to-one to the M groups of first scoring values, and the images in each group of first images correspond one-to-one to the scores in a group of first scoring values.

[0038] Specifically, each first image may correspond to a score in a set of first scoring scores, and the score may represent the first user's image aesthetic inclination for the first image. It can be understood that the larger the score, the more the first image conforms to the first user's image aesthetic inclination; the smaller the score, the further the first image deviates from the first user's image aesthetic inclination.

[0039] It can be understood that the first scoring values ​​of the M groups correspond one-to-one to the M beautification models.

[0040] In some embodiments of the present application, M groups of first images may serve as input data of the first scoring model.

[0041] In some embodiments of the present application, the first user may be the owner user of the terminal.

[0042] In some embodiments of the present application, the model parameters of the first scoring model are adjusted according to the score of the first user for beautifying the image, and therefore, the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user. It can be understood that the scoring standard of the first scoring model matches the scoring standard of the first user for scoring the image.

[0043] In some embodiments of the present application, for the relevant description of the first scoring model, reference can be made to the relevant description of the first scoring model in the following embodiments, which will not be described here to avoid repetition.

[0044] In some embodiments of the present application, the first user's image aesthetic tendency may represent the first user's aesthetic preference for image style, wherein the image style may include but is not limited to at least one of the following: clarity, color, tone, composition, line, brushstroke, material and texture, etc.

[0045] In some embodiments of the present application, "M groups of first images correspond one-to-one to M beautification models" can be understood as: the M groups of first images are respectively M groups of images beautified and output by M beautification models, that is, each group of first images is a group of beautified images beautified and output by one of the M beautification models.

[0046] In some embodiments of the present application, the image contents of any two beautified images in each group of first images are different.

[0047] In some embodiments of the present application, the above-mentioned M groups of first images correspond to the same group of input images.

[0048] In some embodiments of the present application, each group of first images includes: a beautification model beautifies the images in the first data set and outputs a beautified image.

[0049] The first data set includes G types of images, each type of image includes Q images, and the subjects (such as photographed objects) or image contents of the Q images are different, and G and Q are both positive integers.

[0050] Each type of image may include any of the following: portrait images, landscape images such as scenery images, animal images, plant images, and any other possible image categories.

[0051] For example, the server can input three types of images in the first data set, namely portrait images, landscape images and animal images, each type of image includes 1000 images, that is, a total of 3000 images in the first data set. The images in the first data set are respectively input into 10 (i.e., M=10) beautification models for beautification processing, and 10 groups of first images are obtained and output, and the 10 groups of first images correspond to the 10 beautification models one by one. Among them, each group of first images can include 3000 beautified images.

[0052] Then, the server may input the above 10 groups of first images into the first scoring model, score the 10 groups of first images respectively, and output 10 groups of first scoring values, wherein the 10 groups of first images correspond one to one with the 10 groups of first scoring values.

[0053] In some embodiments of the present application, it is assumed that each group of first images includes K images, and the image contents of the K images are different, and the image contents of the K images in different groups match, such as the image contents of the K images in the i-th group of first images are respectively the same as the image contents of the K images in the j-th group of first images, then: the server can first regroup the M groups of first images according to the image content to obtain K groups of images, each group of images includes M first images corresponding to M beautification models one by one (for example, when the M beautification models are 10 beautification models, the server can take the 10 first images with the same face image in the M groups of first beautification models as 1 group). It can be understood that the image styles of the M first images are different. Then, the server can input the K groups of images into the first scoring model in sequence, so that the first scoring model outputs K groups of scoring values ​​corresponding to the K groups of images one by one.

[0054] It can be understood that, for each group of first scoring values ​​in the above M groups of first scoring values, a group of first scoring values ​​includes scoring values ​​corresponding to images with the same image style in the above K groups of images.

[0055] In some embodiments of the present application, the server may also input the M groups of first images into the first scoring model in sequence for scoring, so that the first scoring model outputs the above-mentioned M groups of first scoring values.

[0056] In some embodiments of the present application, when the first data set includes portrait images, the server may select the portrait images in the first data set based on the cultural circle of the first user, skin color characteristics, facial features, etc. In other words, the portrait images in the first data set may be portrait images selected by the server based on the consumer markets and cultural circles of different terminals.

[0057] In some embodiments of the present application, the server can timely supplement the data in the first data set and update the first data set based on: public data sets on the Internet, user-generated content (such as feedback content of the first user on the model or the first data set), historical data (such as images historically taken by the first user), and multimodal data (such as image data, text data, and audio data of different modalities), etc., so as to use the updated first data set to supervise or update the M beautification models to improve the beautification capabilities of the M beautification models.

[0058] In some embodiments of the present application, for each group of first images in M ​​groups of first images, the server may input the i-th first image in a group of first images into the first scoring model, score the i-th first image, and output the score corresponding to the i-th first image; then, the server may input the i+1-th first image in a group of first images into the first scoring model, score the i+1-th first image, and output the score corresponding to the i+1-th first image, and so on, until the first scoring model outputs the score corresponding to the last image in the group of first images, to obtain a group of first scoring scores corresponding to the group of first images, where i is a positive integer.

[0059] In some embodiments of the present application, the model parameters of the above-mentioned M beautification models can characterize the image aesthetic tendency corresponding to the beautification model. Specifically, the beautification model can beautify the input image into a beautified image that conforms to its image aesthetic tendency. For example, if the image aesthetic tendency corresponding to a beautification model is: retro style, then the beautification model can beautify the input image into an image with a retro style.

[0060] In some embodiments of the present application, the M beautification models correspond to different image aesthetic tendencies.

[0061] For example, M beautification models correspond to M image aesthetic tendencies, and the M image aesthetic tendencies are different.

[0062] For example, when M=10 and the image style is tone, the image aesthetic tendencies corresponding to the 10 beautification models can be: cold color temperature, warm color temperature, bright brightness, dark brightness, high contrast, high saturation, color balance, brightness balance, soft tone and hard tone.

[0063] In some embodiments of the present application, when each beautification model corresponds to at least two image aesthetic tendencies, the image aesthetic tendencies corresponding to any two beautification models among the M beautification models are at least partially different.

[0064] In some embodiments of the present application, when each beautification model corresponds to an image aesthetic tendency, after an image is input into the beautification model, the beautification model can beautify the image and output a beautified image corresponding to the image, so that the image style of the beautification image matches the image aesthetic tendency corresponding to the beautification model.

[0065] In some embodiments of the present application, the model structures of the M beautification models may be the same.

[0066] In some embodiments of the present application, the model structure of the beautification model may be a multi-layer pyramid structure, and each layer of the pyramid structure includes a U-shaped network (unet).

[0067] Combine the following Figure 2, the model structure of the beautification model provided in the embodiment of the present application, hereinafter referred to as the beautification model structure, is exemplarily described.

[0068] For example, assuming that the beautification model structure is a 3-layer pyramid structure, each layer of the pyramid structure includes 1 unet, each layer of the 3-layer pyramid structure includes unet1, unet2 and unet3 respectively, and the beautification model structure includes 1 supplementary unet4, and the input image size is 1024*1024*3, then the processing process of the image in the beautification model is:

[0069] like Figure 2 As shown in the figure, an image of size 1024*1024*3 (i.e. input 1024) is input into the beautification model, and the beautification model can downsample layer by layer. For example, the input 1024 is downsampled by 3 layers (i.e. Figure 2 After processing, it can be converted into an image of size 128*128*3 (that is, input 128).

[0070] Then, on the one hand, the input 128 can be input into unet1 from path 10, and then upsampled in unet1 (i.e. Figure 2 On the other hand, the input 128 is upsampled by path 11 and converted into a blurred image of the input 256. The detail information in the detail layer of the input 256 can be obtained by subtracting the blurred image corresponding to the input 256 from the input 256.

[0071] Next, the upsampled image output by unet1 (i.e., unet1 output 256) and the detail information corresponding to input 256 can be added and sent to unet2, and unet2 upsamples the output 256 so that the output 256 can be restored to output 512, while restoring the detail content of input 512.

[0072] And so on, until the detail information in the detail layer of unet3 output 1024 and input 1024 is added and sent to unet4, and the final result is output by unet4.

[0073] It can be understood that the upsampled image output by each unet is added to the corresponding detail information and sent to the next unet, so as to ensure that the detail information in the detail layer of the image in the unet is not lost, so that after the upsampling processing of the next unet, the image can be restored to the resolution of the corresponding image and the detail content of the image can be restored (such as unet1 output 256 is sent to unet2, and after upsampling processing, unet2 can restore the output 256 to output 512 and restore the detail content of input 512). This process helps unet generate accurate segmentation results of the image, and thus can improve the accuracy of the beautification model in beautifying the image.

[0074] It can be seen that the size of each unet input image is different. The beautification model can enlarge the output image layer by layer according to the above upsampling process, and finally obtain an output image of size 1024*1024*3 (beautified image after beautification processing). The output image has the same content as the image in the input beautification model but a different style.

[0075] Among them, an image with a size of 1024*1024*3 indicates that the resolution of the image is 1024*1024, and the number of color channels of the image is 3, that is, three layers of 3 channels of red, green, and blue (Red, Green, Blue, RGB).

[0076] In some embodiments of the present application, after the image is upsampled in UNET, some details of the image will be lost. Therefore, the image output after the upsampling process of UNET is added with the detail information of the corresponding detail layer to obtain a result image without loss of details.

[0077] In some embodiments of the present application, the M beautification models are trained using the same training method.

[0078] In some embodiments of the present application, the training data sets of the above-mentioned M beautification models are the same, and the label data sets are different. Specifically, the label data sets of different beautification models correspond to different image aesthetic tendencies.

[0079] In some embodiments of the present application, the training data may be referred to as input data.

[0080] In some embodiments of the present application, label data may be referred to as true value data or reference data.

[0081] In some embodiments of the present application, the training data sets of the M beautification models may all be second data sets.

[0082] In some embodiments of the present application, the images in the second data set and the images in the first data set have the following characteristics: the images in the two data sets have the same image category, the number of images in each category is the same, and the image subjects (such as photographed objects) or image contents of the images in the two data sets are different. In this way, the data of the first data set and the second data set can be aligned, improving the accuracy of the beautification model in beautifying the image.

[0083] In some embodiments of the present application, the second data set may include G types of images, each type of image includes Q images, and the image subjects (such as photographed objects) or image contents of the Q images are different, and G and Q are both positive integers.

[0084] For example, when G=3 and Q=1000, the second data set may include three types of images, namely portrait images, landscape images and animal images. Each type of image may include 1000 images with different image subjects or image contents. That is to say, the second data set may include 1000 portrait images, 1000 landscape images and 1000 animal images.

[0085] In some embodiments of the present application, a professional photographer may be asked to take M+1 groups of images with different image aesthetic tendencies but the same image content, one group of images in the M+1 groups of images is used as the input data set of M models, and the remaining M groups of images are used as the label data sets of M models. In this way, M beautification models that meet the aesthetic tendencies of professional photographers can be customized.

[0086] In some embodiments of the present application, the label data sets of M beautification models may be: professional photo retouchers process the images in the second data set into M groups of retouched images according to the image aesthetic tendencies corresponding to the M beautification models, thereby obtaining M retouched data sets.

[0087] Among them, the image aesthetic tendencies corresponding to the M photo retouching data sets match the image aesthetic tendencies corresponding to the M beautification models, and a group of retouched images in the M groups of retouched images constitutes one retouched data set in the M photo retouching data sets.

[0088] That is to say, the labeled data sets of the M beautification models are: M retouching data sets obtained by manually retouching the images in the second data set by professional retouchers, and each retouching data set corresponds to an image aesthetic tendency, and different retouching data sets correspond to different image aesthetic tendencies.

[0089] Specifically, a professional photo retoucher can manually retouch the G*Q images in the second data set according to the image aesthetic tendency corresponding to the beautification model to obtain M retouched data sets.

[0090] In some embodiments of the present application, each beautification model corresponds to a label data set, that is, each beautification model corresponds to one retouching data set among M retouching data sets, and the one retouching data set matches the image aesthetic tendency corresponding to the beautification model.

[0091] For example, the image aesthetic tendency corresponding to the beautification model is that the color temperature is relatively cold, so the label data set corresponding to the beautification model is a retouching data set with a relatively cold color temperature among the M retouching data sets.

[0092] In some embodiments of the present application, the server may perform model training (such as Figure 2 The model shown in the figure is used for model training) to obtain a beautified model.

[0093] The beautification model training process provided in the embodiment of the present application is exemplarily described below with reference to specific examples.

[0094] For example, assuming that the i-th image is any image in the second data set, and the j-th beautification model is any beautification model in the M beautification models, then the i-th image can be asked to be retouched by a professional photo retoucher according to the image aesthetic tendency corresponding to the j-th beautification model to obtain a retouched image. Then, the retouched image can be used as label data in the beautification model training process, and the i-th image can be used as input data in the beautification model training process to train the j-th model until the model convergence condition is reached, thereby obtaining the j-th beautification model.

[0095] Among them, the model convergence conditions may include at least one of the following: the error between the predicted value and the true value is less than a preset threshold, the weight change between two iterations is less than a preset threshold, the model reaches the maximum number of iterations, the loss function no longer changes significantly, and the performance of the training data set and the true value data set no longer improves significantly.

[0096] In some embodiments of the present application, the server may use a loss function, calculate a loss value, and adjust the parameters of the model to obtain a trained beautification model. Specifically, for each beautification model, the server may train the model based on the loss value using an AdamW optimizer by presetting a learning rate, batch size, and epoch to obtain a trained beautification model.

[0097] For example, assuming that the second data set includes 3000 images, and the batch size of the model is preset to 4, that is, the images input into the model each time include 4 different images in the second data set, that is, every 4 images in the second data set are grouped together, and the epoch preset value is 20, then: if the image aesthetic tendency corresponding to one of the M beautification models is that the color temperature is cold, the training process of the beautification model may include:

[0098] Step 1. The server inputs the i-th group of images (each group of images includes 4 images) in the second data set into a model, beautifies the 4 images in the group, and outputs the beautified images corresponding to the 4 images (i.e., the prediction results of the model). Then, the server can use the MSE loss function to calculate the i-th loss value based on the beautified images corresponding to the 4 images and the images in the corresponding label data set with a cooler color temperature whose image content matches the image content of the 4 images, and based on the i-th loss value and the preset learning rate (such as 0.0001), use the AdamW optimizer to adjust the model parameters of the model for the i-th time.

[0099] Step 2, the server inputs the i+1th group of images in the second data set into the model, beautifies the four images in the group, and outputs the beautified images of the four images. Then, the server can use the MSE loss function to calculate the i+1th loss value based on the beautified images of the four images, the image aesthetic tendency of the image in the label data set is cold color temperature, and the image content matches the image content of the four images, and based on the i+1th loss value and the preset learning rate value (0.0001), the AdamW optimizer is used to adjust the model parameters of the above model for the i+1th time. And so on, until the model parameters of the model are adjusted for the 3000 / 4th time based on the 3000 / 4th loss value and the preset learning rate (such as 0.0001), the AdamW optimizer is used to complete an epoch of the model. It can be understood that after the model completes 20 epochs, a beautification model is obtained, and the image aesthetic tendency corresponding to the beautification model is cold color temperature.

[0100] In some embodiments of the present application, after the server obtains the i-th loss value, the server may also adjust the preset learning rate based on the i-th loss value using the AdamW optimizer to obtain the adjusted preset learning rate. Thus, after the server calculates and obtains the i+1-th loss value, the model parameters of the model may be adjusted for the i+1th time based on the i+1-th loss value and the adjusted preset learning rate.

[0101] In some embodiments of the present application, the learning rate, batch size, and number of epochs for model training can be adjusted manually or by an algorithm strategy.

[0102] In some embodiments of the present application, the MSE loss function can be expressed as the following formula (1).

[0103]

[0104] Among them, MSE represents the loss value, n represents the number of images in the batch size, and y represents the true value corresponding to the input image (the image after the input image is processed). Represents the prediction value of the beautification model for the input image (i.e. the image output by the model).

[0105] Specifically, is the beautification parameter or beautification feature of the output image corresponding to the input image, and y is the beautification parameter or beautification feature of the true value image corresponding to the input image.

[0106] In this way, since M models can be trained based on the same training data set (such as the first data set) and M label data sets corresponding to different image aesthetic tendencies, M beautification models can be obtained, so that each beautification model can obtain the processing ability to beautify the image into the corresponding image aesthetic tendency. In other words, after the image is beautified by the beautification model, it can have the image aesthetic tendency corresponding to the beautification model.

[0107] Step 102: The server selects a first beautification model from among the M beautification models and N second images from among the M groups of first images based on the M groups of first scoring scores.

[0108] Wherein, N can be a positive integer.

[0109] In some embodiments of the present application, the server may select N second images from the M groups of first images based on the M groups of first scoring values, the first scoring threshold, and the second scoring threshold.

[0110] In some embodiments of the present application, the N second images may include: T images with the highest score among the first images whose image contents match, and whose score is greater than or equal to the first score threshold; NT images with the lowest score among the first images whose image contents match, and whose score is less than or equal to the second score threshold. The first score threshold is greater than the second score threshold, T is an integer greater than or equal to 0, and T is less than or equal to N.

[0111] In some embodiments of the present application, the “first image with matching image content” may include: the first image in the M groups of first images, the first image with a matching degree between image contents greater than a matching degree threshold.

[0112] In some embodiments of the present application, the server may select, based on the above K groups of images, an image with the highest score in each group of images and a score greater than or equal to a first score threshold as one of the T images.

[0113] In some embodiments of the present application, the images with the highest score among the first images with matching image contents and whose scores are greater than or equal to the first score threshold may constitute a highest-score image set.

[0114] In some embodiments of the present application, the server may select, based on the above K groups of images, an image with the lowest score in each group of images and a score less than or equal to a second score threshold as one of the NT images.

[0115] In some embodiments of the present application, the images with the lowest score among the first images with matching image contents and whose score is less than or equal to the second score threshold may constitute the lowest score image set.

[0116] In some embodiments of the present application, the image aesthetic tendency corresponding to the highest-scoring image set has the highest matching degree with the image aesthetic tendency of the first user, or the image style of the highest-scoring image set has the highest matching degree with the image style preferred by the first user; the image aesthetic tendency corresponding to the lowest-scoring image set has the lowest matching degree with the image aesthetic tendency of the first user, or the image style of the lowest-scoring image set has the lowest matching degree with the image style preferred by the first user. In this way, the server can determine the image aesthetic tendency of the first user based on the highest-scoring image set and the lowest-scoring image set. In other words, the server can determine the image aesthetic tendency of the first user based on the score threshold.

[0117] In some embodiments of the present application, image contents of at least some of the T images and the NT images may match.

[0118] In this way, the server can distinguish N second images based on image content, the first score threshold and the second score threshold, and can judge the image aesthetic tendency of the first user according to the score threshold. Therefore, the server uses the N second images to update the model parameters of the first beautification model, and can obtain a second beautification model whose aesthetics are more inclined to the personal style of the first user, thereby meeting the user's personalized needs for image beautification and improving the accuracy and flexibility of image beautification.

[0119] It can be understood that the first beautification model can be the one among the M beautification models whose corresponding image aesthetic tendency is most matched with the image aesthetic tendency preferred by the first user, such as the image aesthetic tendency corresponding to the image processed by the first beautification model is closest to the image aesthetic tendency preferred by the first user.

[0120] In some embodiments of the present application, the server may calculate the scoring values ​​corresponding to the M beautification models based on the M groups of first scoring values, and then select the first beautification model among the M beautification models according to the scoring values ​​corresponding to the M beautification models.

[0121] In some embodiments of the present application, the scoring value corresponding to the beautification model is determined according to a set of first scoring values ​​corresponding to the beautification model.

[0122] In some embodiments of the present application, the scoring value corresponding to each beautification model may be at least one of a total score (method 1), an average score (method 2), and a root mean square score (method 3) of a set of first scoring values.

[0123] In some embodiments of the present application, in the above step 102, "the server selects the first beautification model from the M beautification models based on the M groups of first scoring scores" can be specifically implemented through the following steps 102A and 102B.

[0124] Step 102A: The server determines the scoring values ​​corresponding to the M beautification models according to the M groups of first scoring values.

[0125] Step 102B: The server determines the beautification model with the highest score among the M beautification models as the first beautification model.

[0126] In some embodiments of the present application, in the above-mentioned method 1, for any group of first scoring scores in the M groups of first scoring scores, the server can calculate the total score of the scoring scores in a group of first scoring scores, and determine the total score as the scoring score corresponding to the beautification model corresponding to the group of first scoring scores.

[0127] For example, when M=10, and each group of first images includes 3000 first images, the server can input the 3000 first images in a group of first images into the first scoring model for scoring, and output 3000 scoring values, which can constitute a group of first scoring values ​​corresponding to the group of first images. Then the server can sum the 3000 scoring values ​​to obtain a scoring value corresponding to a beautification model. After obtaining the scoring values ​​corresponding to 10 beautification models, the server can determine the beautification model with the highest scoring value among the 10 beautification models as the base model, that is, the first beautification model mentioned above.

[0128] In some embodiments of the present application, in the above-mentioned method 2, for each group of first scoring scores in the M groups of first scoring scores, the server can calculate the average score of the scoring scores in a group of first scoring scores, and determine the average score as the scoring score corresponding to the beautification model corresponding to the group of first scoring scores.

[0129] In some embodiments of the present application, in the above-mentioned method 3, for any group of first scoring scores in the M groups of first scoring scores, the server can calculate the root mean square value of the scoring scores in a group of first scoring scores, and determine the root mean square value as the scoring score corresponding to the beautification model corresponding to the group of first scoring scores.

[0130] In some embodiments of the present application, the server may also determine a beautification model corresponding to the maximum number of scores among the M beautification models as the first beautification model based on the number of scores in the M groups of first scoring scores that is greater than or equal to the third preset score.

[0131] The third preset score may represent an image aesthetic tendency or image style acceptable to the user. For example, the third preset score may be a passing score, such as 60 points.

[0132] For example, assuming that 10 (i.e., M=10) groups of first scoring values ​​are respectively represented as a1, a2, ..., a10, and the number of scores in these 10 groups of first scoring values ​​that are greater than or equal to the third preset threshold increases successively, then: the server can determine the beautification model corresponding to a10 among the 10 beautification models as the first beautification model.

[0133] In this way, since the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user, the M groups of first scoring scores can characterize the first user's image aesthetic tendency for the M groups of first images. Therefore, the higher the scoring value corresponding to the beautification model, the more consistent the image aesthetic tendency corresponding to the beautification model is with the image aesthetic tendency of the first user, that is, ensuring that the image aesthetic tendency corresponding to the beautification model with the highest scoring value among the M beautification models is most consistent with the image aesthetic tendency of the first user, thereby improving the accuracy of selecting the beautification model.

[0134] Step 103: The server calculates a first loss value based on the beautification parameters of the N second images and the beautification parameters of the S third images.

[0135] Wherein, S is a positive integer less than or equal to N, and each third image is an image whose image content in the beautified image corresponding to the first beautification model matches a second image.

[0136] In some embodiments of the present application, the third image may be an image obtained by beautification processing by the first beautification model, and the image content of the third image matches the image content of the second image.

[0137] Here, “the image content of the third image matches the image content of the second image” can be understood as: the matching degree between the image content of the third image and the image content of the second image is greater than a matching degree threshold.

[0138] For example, for a beautified image beautified by the first beautification model, if the matching degree between the image content of the beautified image and the image content of a second image is greater than or equal to 90% (ie, the matching degree threshold), the beautified image can be confirmed as a third image.

[0139] For example, for a beautified image beautified by the first beautification model, if the matching degree between the main content of the beautified image and the main content of a second image is greater than or equal to 90% (ie, the matching degree threshold), the beautified image can be confirmed as a third image.

[0140] For example, for a beautified image beautified by the first beautification model, if the matching degree between the foreground content of the beautified image and the foreground content of a second image is greater than or equal to 90% (ie, the matching degree threshold), the beautified image can be confirmed as a third image.

[0141] It can be understood that the image contents of the above S third images are different.

[0142] In some embodiments of the present application, the server may select a third image from a group of first images corresponding to the first beautification model based on the image content of the second image.

[0143] In some embodiments of the present application, the server may first find the original image corresponding to the second image, and then input the original image into the first beautification model for beautification processing to obtain the third image.

[0144] Specifically, the server may select S images with different image contents from the N second images, and input the original images corresponding to the S images into the first beautification model for beautification processing, and output S third images.

[0145] In some embodiments of the present application, the server may use T images and NT images out of N second images as label data sets of a first beautification model, use original images corresponding to the T images and original images corresponding to the NT images as input data sets, and calculate a first loss value based on beautification parameters of the N second images and beautification parameters of the N third images.

[0146] In some embodiments of the present application, the first loss value may include at least one of the following: a second loss value and a third loss value.

[0147] In some embodiments of the present application, the above step 103 can be specifically implemented by at least one of the following steps 103A and 103B.

[0148] Step 103A: The server calculates a second loss value based on beautification parameters of T images among the N second images and beautification parameters of T third images.

[0149] In some embodiments of the present application, the image contents of the T images and the T third images correspond one-to-one, that is, the image contents match each other.

[0150] In some embodiments of the present application, the server may calculate the second loss value using the following formula (2) based on the beautification parameters of the T images and the beautification parameters of the T third images.

[0151]

[0152] Wherein, MSE2 represents the second loss value, n1 represents the number of images, n1=T, y represents the true value (the image in T images) corresponding to the input image (i.e., the original image corresponding to T images), It represents the prediction value of the first beautification model for the input image (that is, the beautified image obtained and output by the first beautification model through beautification processing on the input image, that is, the third image).

[0153] Step 103B: The server calculates a third loss value based on the beautification parameters of the NT images and the beautification parameters of the NT third images.

[0154] In some embodiments of the present application, the server may input the original images corresponding to the NT images into the first beautification model for beautification processing, obtain and output NT third images, and then the server may calculate the third loss value based on the beautification parameters of the above NT images and the beautification parameters of the NT third images using the contrastive learning loss function, i.e., formula (3).

[0155]

[0156] Among them, L1 represents the third loss value, n2 represents the number of images of the original image corresponding to the NT images, and Negtive represents the image with the lowest score evaluated by the first scoring model (that is, the image among the NT images). It represents the prediction value of the first beautification model for the input image (the original image corresponding to the NT images) (that is, the beautified image obtained and output by the first beautification model after beautifying the input image).

[0157] In some embodiments of the present application, the purpose of using the contrast loss function to calculate the third loss value is to enable the first beautification model to be away from the data with the lowest score given by the first user. In other words, it is to avoid that the image output by the first beautification model is an image that is far away from the image aesthetic preference of the first user.

[0158] In some embodiments of the present application, the server may update the model parameters of the first beautification model based on any of the following items:

[0159] The second loss value;

[0160] The third loss value;

[0161] The second loss value and the third loss value.

[0162] In some embodiments of the present application, the model parameter updating method provided in the embodiments of the present application does not limit the order of step 103A and step 103B, and the order of use of the two steps can be determined based on actual usage.

[0163] In this way, since the server can calculate the second loss value and the third loss value based on the beautification parameters of T images and the beautification parameters of T third images, and the beautification parameters of NT images and the beautification parameters of NT third images, respectively, the server can update the model parameters of the first beautification model according to different loss values, so that the first beautification model can be closer to the image aesthetic tendency of the first user, or further away from the image aesthetic tendency of the first user, thereby improving the accuracy of updating the model parameters of the first beautification model.

[0164] Step 104: The server updates the model parameters of the first beautification model based on the first loss value to obtain a second beautification model.

[0165] In some embodiments of the present application, the server can use a loss function to calculate a first loss value by presetting a learning rate and a batch size, and based on the first loss value, use an AdamW optimizer to update the model parameters of the first beautification model to obtain a second beautification model.

[0166] For example, the server can preset the learning rate value to 0.0001 and the batch size value to 1, use a loss function (such as an MSE loss function) to calculate a first loss value, and based on the first loss value, use an AdamW optimizer to update the model parameters of the first beautification model to obtain a second beautification model.

[0167] In the model parameter updating method provided in some embodiments of the present application, since the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user, the first scoring model can score the M groups of first images based on the image aesthetic tendency of the first user, that is, the M groups of first scoring scores output by the first scoring model can characterize the user's aesthetic evaluation or aesthetic preference for the M groups of first images; thereby ensuring that the first beautification model and the N second images selected based on the M groups of first scoring scores can be consistent with the image aesthetic tendency of the first user. Therefore, based on the beautification parameters of the N second images and the beautification parameters of the S third images (the third image is the beautified image corresponding to the first beautification model, and the image content matches the second image), the first loss value is calculated, and based on the first loss value, the model parameters of the first beautification model are updated, so that the second beautification model whose image aesthetic tendency conforms to the user's aesthetics can be obtained, that is, the image aesthetic tendency corresponding to the image beautified by the second beautification model conforms to the image aesthetic tendency of the first user.

[0168] In some embodiments of the present application, if the first user is not satisfied with the image aesthetic tendency of the beautified image obtained and output after the image is beautified by the second beautification model, the server can add the second beautification model to the preset beautification model (that is, add the second beautification model to the above-mentioned M beautification models, and the number of beautification models in the server becomes M+1 beautification models), and repeat the above steps provided in the embodiments of the present application, further fine-tune the second beautification model (that is, update the model parameters of the second beautification model), and obtain a user-customized beautification model (that is, the new second beautification model), so that the image aesthetic tendency corresponding to the second beautification model can be more inclined to the user's personal style, meet the user's personalized customized image processing needs, and the model parameters of the second beautification model can be continuously updated, so that the second beautification model can meet the user's growing aesthetic needs for images, so that the server can create a beautification model exclusive to the user's personal image aesthetic tendency.

[0169] In this way, the server can tailor an exclusive beautification model for the user in the cloud based on the user's personal choice. When the first user selects the photography style he wants on the terminal, the server can provide the first user with a beautification plan generated in the cloud that conforms to the first user's image aesthetic preferences and enables the first user to fully demonstrate his or her aesthetic personality.

[0170] In some embodiments of the present application, before the above step 101, the model parameter updating method provided by the embodiment of the present application may include the following steps 105 to 108.

[0171] Step 105: The server obtains M groups of second scoring scores given by the first user to the M groups of fourth images.

[0172] The M groups of fourth images correspond one-to-one to the M beautification models.

[0173] In some embodiments of the present application, "M groups of fourth images correspond one-to-one to M beautification models" can be understood as: the M groups of fourth images are respectively M groups of images beautified and output by M beautification models, that is, each group of fourth images is a group of beautified images beautified and output by one of the M beautification models.

[0174] In some embodiments of the present application, the M groups of fourth images are obtained by beautifying a group of images using M beautification models respectively; that is, the M groups of fourth images correspond to the same group of input images.

[0175] The above group of input images includes G types of images, and each type of image includes R images with different image contents.

[0176] For example, M=10, G=3, R=1, the server can input 3 images, namely 1 portrait image, 1 landscape image and 1 animal image, into 10 beautification models for beautification processing, and output 10 groups of fourth images with different image aesthetic tendencies, and each group of fourth images includes 3 beautified images of different categories. In this way, the richness of the image content and image aesthetic tendencies of the M groups of fourth images can be ensured.

[0177] In some embodiments of the present application, the above-mentioned R images may be any R images in each type of image in the first data set, where R is a positive integer and R is less than or equal to Q.

[0178] For example, when R=1 and the image category is animal image, the R images may be a kitten image in the animal images of the first data set; when R=2, the R images may be a kitten image and a puppy image in the animal images of the first data set.

[0179] In some embodiments of the present application, the M groups of second scoring values ​​correspond one-to-one to the M groups of fourth images, and each group of second scoring values ​​corresponds one-to-one to an image in a group of fourth images.

[0180] In some embodiments of the present application, the second scoring values ​​are obtained by the first user manually scoring the M groups of fourth images, and each second scoring value may correspond to one of the fourth images.

[0181] In some embodiments of the present application, the first user can manually score the M groups of fourth images according to his or her own image aesthetic tendency for the M groups of fourth images. Thus, the score of each second scoring score can represent: the first user's preference for the image aesthetic tendency corresponding to a fourth image. It can be understood that the larger the score, the higher the first user's preference for the image aesthetic tendency corresponding to the fourth image; the smaller the score, the lower the first user's preference for the image aesthetic tendency corresponding to the fourth image.

[0182] For example, the value range of the second scoring value may be [0, 9]. If the second scoring value of the first user for a fourth image is 0, it indicates that the first user dislikes the image aesthetic tendency corresponding to the fourth image the most; if the second scoring value of the first user for a fourth image is 9, it indicates that the first user likes the image aesthetic tendency corresponding to the fourth image the most.

[0183] Step 106: The server inputs the M groups of fourth images into the second scoring model, scores the M groups of fourth images, and outputs the M groups of third scoring scores.

[0184] Among them, the model parameters of the second scoring model can be used to characterize various image aesthetic tendencies.

[0185] In some embodiments of the present application, M groups of third scoring values ​​correspond one-to-one to M groups of fourth images, and a score in each group of third scoring values ​​corresponds to an image in a group of fourth images.

[0186] In some embodiments of the present application, the input data set of the second scoring model may include images in 6 public aesthetic datasets: AVA, Flicker-AES, TAD66K, PARA, KonIQ-10K and SPAQ, and the label data set may include the scoring values ​​corresponding to the images in the 6 aesthetic datasets. The images in the 6 aesthetic datasets correspond to the scoring values ​​of the images one by one, that is, one image corresponds to one scoring value.

[0187] In some embodiments of the present application, each aesthetic dataset has an image aesthetic tendency, and an image aesthetic tendency may include aesthetic tendencies such as clarity, color, and composition.

[0188] In some embodiments of the present application, the server may use the CLIP model structure, input the above-mentioned input data set into the model, use the loss function based on the above-mentioned label data set, calculate the loss value, and adjust the model parameters of the model to obtain a second scoring model.

[0189] Specifically, the server can use a loss function to calculate a loss value by presetting a learning rate, batch size, and epoch, and use an AdamW optimizer to train the model based on the loss value to obtain a trained second scoring model.

[0190] For example, the server can preset the learning rate value to 0.0001, the batch size value to 128, the epoch value to 100, use the MSE loss function to calculate the loss value, and based on the loss value, use the AdamW optimizer to train the model to obtain the second scoring model.

[0191] In some embodiments of the present application, the second scoring model can be trained with multiple aesthetic datasets of multiple image aesthetic tendencies as input datasets, and with scoring values ​​corresponding to the multiple aesthetic datasets as note datasets. Therefore, the model parameters of the second scoring model can be used to characterize multiple image aesthetic tendencies.

[0192] In some embodiments of the present application, for the specific training process of the second scoring model, reference may be made to the relevant description of the beautification model training process in the above embodiments, which will not be described here to avoid repetition.

[0193] Step 107: The server calculates a fourth loss value based on the second scoring value of the M group and the third scoring value of the M group.

[0194] In some embodiments of the present application, the server may calculate a fourth loss value based on the M groups of second scoring values, the M groups of third scoring values, and a loss function (such as a cross entropy loss function).

[0195] In some embodiments of the present application, the server may use a cross entropy loss function to calculate whether the scores of the two images predicted by the second scoring model (i.e., the corresponding scores of the two images input into the second scoring model for scoring and output) are consistent with the scores of the two images given by the first user. The cross entropy loss function L is represented by the following formula (4).

[0196]

[0197] Wherein, L represents the fourth loss value, and represents the prediction result of the second scoring model (i.e. the scoring value output by the second scoring model), x1 and x2 represent the input images (the two images in the fourth image), and y1 and y2 represent the scoring results of the input images by the first user (i.e. the second scoring values ​​corresponding to the two images in the fourth image).

[0198] Step 108: The server updates the model parameters of the second scoring model based on the fourth loss value to obtain the first scoring model.

[0199] In some embodiments of the present application, the server may, based on the fourth loss value, use the AdamW optimizer to update the model parameters of the second scoring model by presetting the learning rate, batchsize and fine-tuning steps to obtain the first scoring model.

[0200] For example, the server may update the model parameters of the second scoring model based on the fourth loss value, the preset learning rate value is 0.000001, the fine-tuning step number is 500 steps, the batch size value is 1, and the AdamW optimizer is used to update the model parameters of the second scoring model based on the fourth loss value. The batch size value of 1 can be understood as: inputting a group of images into the second scoring model at one time, and the group of images includes 2 fourth images.

[0201] It should be noted that the fine-tuning step of 500 can be understood as: an image is input into the second scoring model for scoring and the scoring value of the image is output as one step in the fine-tuning step. If the total number of fourth images in the M groups is 30, the 30th image is input into the second scoring model for scoring and the scoring value of the 30th image is output, which is the 30th step of the fine-tuning step. The 31st step of the fine-tuning step is that the first image of the M fourth images is input into the second scoring model for scoring and the scoring value is output. Similarly, the 30 images of the M groups of fourth images are cyclically input into the second scoring model for scoring and the scoring values ​​are output until the 500th step ends.

[0202] In some embodiments of the present application, a possible implementation method is that the server can use an AdamW optimizer to update the model parameters of the second scoring model based on a fourth loss value and a preset learning rate for each fine-tuning step until the 500th step, thereby obtaining a first beautification model.

[0203] In this way, the server can obtain the second scoring values ​​of the first user for the fourth image of the M group, and the second scoring values ​​can represent the image aesthetic tendency of the first user for the fourth image of the M group; the server can score the fourth image of the M group using the second scoring model, and output the third scoring values ​​of the M group, and can update the model parameters of the second scoring model based on the fourth loss value calculated by the third scoring values ​​of the M group and the second scoring values ​​of the M group, and obtain the first scoring model. Therefore, the model parameters of the first scoring model can represent the image aesthetic tendency of the first user for the image, and therefore, the server can improve the consistency between the image scoring result of the server and the image aesthetic tendency of the user for the image by using the first scoring model to score the image.

[0204] In some embodiments of the present application, before the server obtains the M groups of second scoring scores of the first user for the M groups of fourth images, the model parameter updating method provided in the embodiments of the present application may include the following steps 01 to 03.

[0205] Step 01: The server inputs G*Q images of the first data set into M beautification models, performs beautification processing, and outputs M groups of fourth images.

[0206] Step 02: The server sends M groups of fourth images to the terminal, and the first user scores the M groups of fourth images.

[0207] Step 03: The server receives the second scoring value sent by the terminal and obtains the second scoring value.

[0208] In some embodiments of the present application, the relevant descriptions of the above steps 01 to 03 can refer to the relevant descriptions in the above embodiments, and in order to avoid repetition, they are not repeated here.

[0209] In some embodiments of the present application, before the above-mentioned step 106, the model parameter updating method provided in the embodiment of the present application may also include the following step 109, and the "server inputs M groups of fourth images into the second scoring model" in the above-mentioned step 106 can be specifically implemented through the following step 110.

[0210] Step 109: The server updates the model parameters of the second scoring model based on the model parameters of the U third scoring models.

[0211] Wherein, U is an integer greater than 1, the model parameters of the third scoring model represent an image aesthetic tendency, and different third scoring model parameters can be used to represent different image aesthetic tendencies.

[0212] Step 110: The server inputs the M groups of fourth images into the second scoring model after updating the model parameters.

[0213] In some embodiments of the present application, U third scoring models are obtained by training U data sets, each of the U data sets can represent an image aesthetic tendency, and different data sets represent different image aesthetic tendencies.

[0214] For example, when U=6, six third scoring models can be trained by six data sets, which can be the six public aesthetic data sets mentioned above, wherein each aesthetic data set can represent an image aesthetic tendency. Thus, the model parameters of each third scoring model can represent an image aesthetic tendency represented by the corresponding aesthetic data set, and different third scoring models can be used to represent different image aesthetic tendencies.

[0215] In some embodiments of the present application, the server may use an image in one of U data sets as an input data set, use the scoring values ​​corresponding to the images in the data set as a label data set, and use a loss function to train a third scoring model.

[0216] Specifically, the server can first calculate the loss value based on the above-mentioned labeled data set using the loss function; then the server can train the model based on the loss value using the AdamW optimizer by presetting the learning rate, batch size and epoch to obtain a trained third scoring model.

[0217] For example, the server can use the images in the dataset AVA as the input dataset, and the scoring values ​​corresponding to the images in the AVA dataset as the label dataset, preset the learning rate value to 0.00005, the batch size value to 8, the epoch value to 5, use the MSE loss function to calculate the loss value, and based on the loss value, use the AdamW optimizer to train the model to obtain a third scoring model.

[0218] In some embodiments of the present application, for the training of other third scoring models, reference may be made to the training method using the AVA data set. To avoid repetition, this will not be described in detail in the embodiments of the present application.

[0219] In some embodiments of the present application, the server may update the model parameters of the second scoring model based on the model parameters of U third scoring models, and then input the M groups of fourth images into the second scoring model with updated model parameters, score the M groups of fourth images and output M groups of third scoring scores.

[0220] In this way, since the server can update the parameters of the second scoring model based on the model parameters of U third scoring models, the model parameters of U third scoring models can characterize the image aesthetic tendencies of different data sets. After the server inputs M groups of fourth images into the second scoring model after the model parameters are updated, the scoring values ​​output by the second scoring model can characterize the aesthetic tendencies of multiple images, thereby improving the accuracy of the server's scoring of images.

[0221] In some embodiments of the present application, the above step 109 can be specifically implemented through the following steps 109A and 109B.

[0222] Step 109A: The server subtracts the model parameters of the second scoring model from the model parameters of the U third scoring models to obtain U groups of model parameters.

[0223] In some embodiments of the present application, the server may subtract the model parameters of the second scoring model from the model parameters of the U third scoring models to obtain U groups of model parameters.

[0224] Specifically, Figure 3 As shown, the first group of model parameters ε1 in the U group of model parameters can be obtained by fine-tuning the model parameters θ of model 1 ft1 (Model parameters of the third scoring model 1) minus the model parameters θ of the pre-trained model pre (Model parameters of the second scoring model) are obtained, and the second group of model parameters ε2 in the U group of model parameters can be obtained by fine-tuning the parameters θ of model 2 ft2 (Model parameters of the third scoring model 2) minus the model parameters θ of the pre-trained model pre (Model parameters of the second scoring model) are obtained, and so on, until the Uth group of parameters in the U groups of model parameters are obtained. In this way, U groups of model parameters can be obtained.

[0225] In some embodiments of the present application, each set of model parameters can be calculated using the following formula (5).

[0226] ε=θ ft -θ pre(5)

[0227] Among them, ε is a set of model parameters in the U set of model parameters, θ ft is the model parameter of a third scoring model, θ pre are the model parameters of the second scoring model.

[0228] It can be understood that the U groups of model parameters are obtained by subtracting the model parameters of the U third scoring models from the model parameters of the second scoring model respectively.

[0229] In some embodiments of the present application, each group of model parameters in the U groups of model parameters can represent the model parameter difference between the second scoring model and a third scoring model. It can be understood that the model parameter difference can reflect the difference between the scoring standard of the second scoring model and the scoring standard of the third scoring model.

[0230] Step 109B: The server updates the model parameters of the second scoring model based on the U group of model parameters.

[0231] In some embodiments of the present application, the server may perform weighted addition of U groups of model parameters, and then add the basic pre-trained parameters (i.e., the model parameters of the second scoring model) to update the model parameters of the second scoring model. The updated model parameters of the second scoring model may be specifically calculated using the following formula (6).

[0232]

[0233] Among them, θ p is the updated model parameter of the second scoring model, θ pre is the model parameter of the second scoring model, α is the weight, and α is a parameter that can be continuously learned, and k is the number of the third scoring model (ie, U value).

[0234] In some embodiments of the present application, the server may use M groups of fourth images as training data sets, M groups of second scoring values ​​as label data sets, and update the model parameters of the second scoring model based on U groups of model parameters using a loss function.

[0235] In this way, since the server can subtract the model parameters of the second scoring model from the model parameters of U third scoring models respectively to obtain U groups of model parameters, the U groups of model parameters can characterize the difference between the third scoring model and the second scoring model. Therefore, the server updates the model parameters of the second scoring model based on the U groups of model parameters, so that the updated second scoring model can enhance the scoring ability for various image aesthetic tendencies, thereby more accurately obtaining the evaluation criteria for the image aesthetic tendencies of the first user, so that the scoring value output by the second scoring model can be more in line with the first user's personalized image aesthetic tendency rating criteria, thereby improving the server's image processing capabilities.

[0236] The model parameter updating method provided in the embodiment of the present application is described below with reference to specific examples.

[0237] For example, the second data set includes portrait images, landscape images, and animal images, and the number of images in each of the three types of images is 1000. The model parameter updating method provided in the embodiment of the present application may include the following steps 201 to 211.

[0238] Step 201: The cloud (ie, the server) sets 10 beautification models with different image aesthetic tendencies.

[0239] Step 202: The server uses the images retouched by professional photo retouchers as a label data set, and the 3,000 images included in the second data set as an input data set, and uses the MSE loss function to train the above 10 models respectively to obtain 10 beautification models.

[0240] Step 203: The server inputs 3 images different from the second data set (i.e., 3 images of 1 image in each of the 3 categories of images in the first data set) into 10 beautification models to generate 30 beautified images of different image styles and different image categories.

[0241] Step 204: The server merges the six public aesthetic datasets into a total dataset, and uses the CLIP model structure to train a basic scoring model (i.e., the second scoring model) based on the total dataset.

[0242] Step 205: The server uses each of the six aesthetic datasets to fine-tune the basic scoring model to obtain six tendency fine-tuned scoring models (ie, the third scoring model).

[0243] Step 206: The server sends the 30 beautified images to the terminal, and the owner user of the terminal (ie, the first user) scores the 30 beautified images.

[0244] Step 207, the server receives the scoring values ​​of the 30 beautified images sent by the terminal, creates a data set based on the scoring values ​​of the machine owner, creates data pairs corresponding to the 30 beautified images and the scoring values ​​of the machine owner, uses the scores of the machine owner for the 30 beautified images as a label data set, and uses the 30 beautified images as an input data set.

[0245] Step 208: The server fine-tunes the basic scoring model based on the data set in step 207 to obtain a personalized scoring model (ie, the first scoring model).

[0246] Step 209, the server inputs the 3000 images in the first data set into 10 beautification models for beautification processing, obtains and outputs 3000*10 beautified images, and groups the output beautified images, and groups the 10 output images of each input image as a group (that is, one image in the first data set is input into 10 beautification models for beautification processing, and the obtained and output images of 10 different image aesthetic tendencies are grouped as a group), to obtain 3000 groups of output images (i.e., M groups of first images).

[0247] Step 210: The server inputs 3000 groups of images into the personalized scoring model for scoring, retains the highest-scoring image (and the image scoring range is 7-9 points) and the lowest-scoring image (and the image scoring range is 0-2 points) in the K groups of images, and records the scoring value of the output image of each beautification model in the personalized scoring model, and calculates the total score of the output image of each beautification model.

[0248] Step 211: The server determines the beautification model with the highest total score as the base model (i.e., the first beautification model), retains the highest-scoring image and the lowest-scoring image as label data sets, and uses the original images corresponding to the high-scoring images and the original images corresponding to the low-scoring images as input data sets, and uses the MSE loss function and the contrastive learning loss function to fine-tune the base model to obtain the image beautification model (i.e., the second beautification model).

[0249] In some embodiments of the present application, for the detailed description of the above steps 201 to 211, reference may be made to the relevant descriptions in the above steps, and in order to avoid repetition, they are not described here.

[0250] The model parameter updating method provided in the embodiment of the present application can be executed by a model parameter updating device. In the embodiment of the present application, the method of executing the model parameter updating by the model parameter updating device is taken as an example to illustrate the model parameter updating device provided in the embodiment of the present application.

[0251] like Figure 4As shown, the embodiment of the present application further provides a model parameter updating device 400 , which may include: a processing module 401 , a selection module 402 , and an updating module 403 . The processing module 401 can be used to input M groups of first images into the first scoring model, score the M groups of first images, and output M groups of first scoring values, where the M groups of first images correspond one to one with the M beautification models, and the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user, where M is an integer greater than 1; the selection module 402 can be used to select the first beautification model from the M beautification models and the N second images from the M groups of first images based on the M groups of first scoring values ​​output by the processing module 401, where N is a positive integer; the processing module 403 can be used to calculate the first loss value based on the beautification parameters of the N second images and the beautification parameters of the N third images selected by the selection module 402, where S is a positive integer less than or equal to N; the updating module 403 can be used to update the model parameters of the first beautification model based on the first loss value calculated by the processing module 403, and obtain the second beautification model. Wherein, each third image is an image whose image content matches a second image in the beautification image corresponding to the first beautification model.

[0252] In some embodiments of the present application, the selection module 402 may be specifically used to: determine the scoring values ​​corresponding to the M beautification models according to the M groups of first scoring values; and determine the beautification model with the highest scoring value among the M beautification models as the first beautification model.

[0253] In some embodiments of the present application, the N second images include: T images with the highest score among the first images whose image contents match, and whose score is greater than or equal to a first score threshold; NT images with the lowest score among the first images whose image contents match, and whose score is less than or equal to a second score threshold. The first score threshold is greater than the second score threshold, T is an integer greater than or equal to 0, and T is less than or equal to N.

[0254] In some embodiments of the present application, the processing module 403 may be used for at least one of the following:

[0255] Calculating a second loss value based on the beautification parameters of the T images and the beautification parameters of the T third images;

[0256] Based on the beautification parameters of the NT images and the beautification parameters of the NT third images, a third loss value is calculated.

[0257] In some embodiments of the present application, the model parameter updating device further includes: an acquisition module. The acquisition module can be used to obtain the M groups of second scoring scores of the first user for the M groups of fourth images before the processing module 401 inputs the M groups of first images into the first scoring model, and the M groups of fourth images correspond one to one with the M beautification models; the processing module 401 can also be used to input the M groups of fourth images into the second scoring model, score the M groups of fourth images, and output the M groups of third scoring scores; the processing module 403 can also be used to calculate the fourth loss value based on the M groups of second scoring scores obtained by the acquisition module and the M groups of third scoring scores output by the processing module 401; the update module 403 can also be used to update the model parameters of the second scoring model based on the fourth loss value calculated by the processing module 403 to obtain the first scoring model.

[0258] The model parameters of the second scoring model are used to characterize various image aesthetic tendencies.

[0259] In some embodiments of the present application, the above-mentioned updating module 403 can also be used to update the model parameters of the second scoring model based on U model parameters of the third scoring models before the above-mentioned processing module 401 inputs the M groups of fourth images into the second scoring model, where U is an integer greater than 1; the above-mentioned processing module 401 can specifically be used to input the M groups of fourth images into the second scoring model after the model parameters are updated.

[0260] The model parameters of the third scoring model represent an image aesthetic tendency, and different third scoring model parameters can be used to represent different image aesthetic tendencies.

[0261] In some embodiments of the present application, the above-mentioned updating module 403 can be specifically used to: subtract the model parameters of the second scoring model from the model parameters of U third scoring models respectively to obtain U groups of model parameters; and update the model parameters of the second scoring model based on the U groups of model parameters.

[0262] In the model parameter updating device provided in some embodiments of the present application, since the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user, the first scoring model can score the M groups of first images based on the image aesthetic tendency of the first user, that is, the M groups of first scoring scores output by the first scoring model can characterize the user's aesthetic evaluation or aesthetic preference for the M groups of first images; thereby ensuring that the first beautification model and the N second images selected based on the M groups of first scoring scores can be consistent with the image aesthetic tendency of the first user. Therefore, based on the beautification parameters of the N second images and the beautification parameters of the S third images (the third image is the beautified image corresponding to the first beautification model, and the image content matches the second image), the first loss value is calculated, and based on the first loss value, the model parameters of the first beautification model are updated, so that the second beautification model whose image aesthetic tendency conforms to the user's aesthetics can be obtained, that is, the image aesthetic tendency corresponding to the image beautified by the second beautification model conforms to the image aesthetic tendency of the first user.

[0263] The model parameter updating device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0264] The model parameter updating device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0265] The model parameter updating device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0266] Alternatively, if Figure 5 As shown, an embodiment of the present application also provides an electronic device 500, including a processor 501 and a memory 502, wherein the memory 502 stores programs or instructions that can be executed on the processor 501, and when the program or instructions are executed by the processor 501, the various steps of the above-mentioned model parameter updating method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0267] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0268] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0269] The electronic device 1700 includes but is not limited to: a radio frequency unit 1701, a network module 1702, an audio output unit 1703, an input unit 1704, a sensor 1705, a display unit 1706, a user input unit 1707, an interface unit 1708, a memory 1709, and a processor 1710 and other components.

[0270] Those skilled in the art will appreciate that the electronic device 1700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1710 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0271] Wherein, the processor 1710 can be used to input the M groups of first images into the first scoring model, score the M groups of first images, and output the M groups of first scoring scores, the M groups of first images correspond one to one with the M beautification models, and the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user, where M is an integer greater than 1; the processor 1710 can be used to select the first beautification model from the M beautification models and the N second images from the M groups of first images based on the M groups of first scoring scores output by the processor 1710, where N is a positive integer; the processing module 403 can be used to calculate the first loss value based on the beautification parameters of the N second images and the beautification parameters of the N third images selected by the processor 1710, where S is a positive integer less than or equal to N; the processor 1710 can be used to update the model parameters of the first beautification model based on the first loss value calculated by the processing module 403, and obtain the second beautification model. Wherein, each third image is an image whose image content matches a second image in the beautification image corresponding to the first beautification model.

[0272] In some embodiments of the present application, the processor 1710 may be specifically used to: determine the scoring values ​​corresponding to the M beautification models according to the M groups of first scoring values; and determine the beautification model with the highest scoring value among the M beautification models as the first beautification model.

[0273] In some embodiments of the present application, the N second images include: T images with the highest score among the first images whose image contents match, and whose score is greater than or equal to a first score threshold; NT images with the lowest score among the first images whose image contents match, and whose score is less than or equal to a second score threshold. The first score threshold is greater than the second score threshold, T is an integer greater than or equal to 0, and T is less than or equal to N.

[0274] In some embodiments of the present application, the processing module 403 may be used for at least one of the following:

[0275] Calculating a second loss value based on the beautification parameters of the T images and the beautification parameters of the T third images;

[0276] Based on the beautification parameters of the NT images and the beautification parameters of the NT third images, a third loss value is calculated.

[0277] In some embodiments of the present application, the processor 1710 can be used to obtain M groups of second scoring scores of the first user for the M groups of fourth images before the processor 1710 inputs the M groups of first images into the first scoring model, and the M groups of fourth images correspond one-to-one to the M beautification models; the processor 1710 can also be used to input the M groups of fourth images into the second scoring model, score the M groups of fourth images, and output M groups of third scoring scores; the processing module 403 can also be used to calculate a fourth loss value based on the M groups of second scoring scores obtained by the processor 1710 and the M groups of third scoring scores output by the processor 1710; the processor 1710 can also be used to update the model parameters of the second scoring model based on the fourth loss value calculated by the processing module 403 to obtain the first scoring model.

[0278] The model parameters of the second scoring model are used to characterize various image aesthetic tendencies.

[0279] In some embodiments of the present application, the above-mentioned processor 1710 can also be used to update the model parameters of the second scoring model based on U model parameters of the third scoring models before the above-mentioned processor 1710 inputs the M groups of fourth images into the second scoring model, where U is an integer greater than 1; the above-mentioned processor 1710 can specifically be used to input the M groups of fourth images into the second scoring model after the model parameters are updated.

[0280] The model parameters of the third scoring model represent an image aesthetic tendency, and different third scoring model parameters can be used to represent different image aesthetic tendencies.

[0281] In some embodiments of the present application, the processor 1710 may be specifically used to: subtract the model parameters of the second scoring model from the model parameters of U third scoring models respectively to obtain U groups of model parameters; and update the model parameters of the second scoring model based on the U groups of model parameters.

[0282] In some embodiments of the present application, since the model parameters of the first scoring model can be used to characterize the image aesthetic tendency of the first user, the first scoring model can score the M groups of first images based on the image aesthetic tendency of the first user, that is, the M groups of first scoring scores output by the first scoring model can characterize the user's aesthetic evaluation or aesthetic preference for the M groups of first images; thereby ensuring that the first beautification model and the N second images selected based on the M groups of first scoring scores can be in line with the image aesthetic tendency of the first user. Therefore, based on the beautification parameters of the N second images and the beautification parameters of the S third images (the third image is the beautified image corresponding to the first beautification model, and the image content matches the second image), the first loss value is calculated, and based on the first loss value, the model parameters of the first beautification model are updated, so that the second beautification model whose image aesthetic tendency conforms to the user's aesthetics can be obtained, that is, the image aesthetic tendency corresponding to the image beautified by the second beautification model conforms to the image aesthetic tendency of the first user.

[0283] It should be understood that in the embodiment of the present application, the input unit 1704 may include a graphics processor (Graphics Processing Unit, GPU) 17041 and a microphone 17042, and the graphics processor 17041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1706 may include a display panel 17061, and the display panel 17061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1707 includes a touch panel 17071 and at least one of other input devices 17072. The touch panel 17071 is also called a touch screen. The touch panel 17071 may include two parts: a touch detection device and a touch controller. Other input devices 17072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0284] The memory 1709 can be used to store software programs and various data. The memory 1709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1709 may include a volatile memory or a non-volatile memory, or the memory 1709 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0285] The processor 1710 may include one or more processing units; optionally, the processor 1710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1710.

[0286] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned model parameter updating method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0287] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0288] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned model parameter updating method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0289] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0290] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned model parameter updating method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0291] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0292] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0293] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A model parameter updating method, characterized in that: The method comprises: The server inputs M groups of first images into a first scoring model, scores the M groups of first images, and outputs M groups of first scoring values, wherein the M groups of first images correspond one-to-one to the M beautification models, and the model parameters of the first scoring model are used to characterize the image aesthetic tendency of the first user, where M is an integer greater than 1; The server selects, based on the M groups of first scoring values, a first beautification model from the M beautification models and N second images from the M groups of first images, where N is a positive integer; The server calculates a first loss value based on the beautification parameters of the N second images and the beautification parameters of the S third images, where S is a positive integer less than or equal to N; The server updates the model parameters of the first beautification model based on the first loss value to obtain a second beautification model; Each of the third images is an image whose image content in the beautified image corresponding to the first beautification model matches one of the second images.

2. The method according to claim 1, characterized in that: The server selects a first beautification model from the M beautification models based on the M groups of first scoring scores, including: The server determines the scoring values ​​corresponding to the M beautification models according to the M groups of first scoring values; The server determines the beautification model with the highest score among the M beautification models as the first beautification model.

3. The method according to claim 1 or 2, characterized in that: The N second images include: T images with the highest scores among the first images with matching image contents and scores greater than or equal to the first score threshold; NT images with the lowest score among the first images with matching image contents and whose score is less than or equal to the second score threshold; The first score threshold is greater than the second score threshold, T is an integer greater than or equal to 0, and T is less than or equal to N.

4. The method according to claim 3, characterized in that: The server calculates a first loss value based on the beautification parameters of the N second images and the beautification parameters of the N third images, including at least one of the following: The server calculates a second loss value based on the beautification parameters of the T images and the beautification parameters of the T third images; The server calculates a third loss value based on the beautification parameters of the NT images and the beautification parameters of the NT third images.

5. The method according to claim 1, characterized in that Before the server inputs the M groups of first images into the first scoring model, the method further includes: The server obtains M groups of second scoring scores of the first user for the M groups of fourth images, where the M groups of fourth images correspond one-to-one to the M beautification models; The server inputs the M groups of fourth images into a second scoring model, scores the M groups of fourth images, and outputs the M groups of third scoring scores; The server calculates a fourth loss value based on the second scoring value of the M group and the third scoring value of the M group; The server updates the model parameters of the second scoring model based on the fourth loss value to obtain the first scoring model; The model parameters of the second scoring model are used to characterize various image aesthetic tendencies.

6. A model parameter updating device, characterized in that: The device comprises: a processing module, a selection module and an updating module; The processing module is used to input M groups of first images into a first scoring model, score the M groups of first images, and output M groups of first scoring values, wherein the M groups of first images correspond one-to-one to the M beautification models, and the model parameters of the first scoring model are used to characterize the image aesthetic tendency of the first user, and M is an integer greater than 1; The selection module is used to select a first beautification model from the M beautification models and N second images from the M groups of first images based on the M groups of first scoring values ​​output by the processing module, where N is a positive integer; The processing module is further used to calculate a first loss value based on the beautification parameters of the N second images and the beautification parameters of the S third images selected by the selection module, where S is a positive integer less than or equal to N; The updating module is used to update the model parameters of the first beautification model based on the first loss value calculated by the processing module to obtain a second beautification model; Each of the third images is an image whose image content in the beautified image corresponding to the first beautification model matches one of the second images.

7. The device according to claim 6, characterized in that The selection module is specifically used for: Determine the scoring values ​​corresponding to the M beautification models according to the M groups of first scoring values; The beautification model with the highest score among the M beautification models is determined as the first beautification model.

8. The device according to claim 6 or 7, characterized in that The N second images include: T images with the highest scores among the first images with matching image contents and scores greater than or equal to the first score threshold; NT images with the lowest score among the first images with matching image contents and whose score is less than or equal to the second score threshold; The first score threshold is greater than the second score threshold, T is an integer greater than or equal to 0, and T is less than or equal to N.

9. The device according to claim 8, characterized in that The processing module is specifically used for at least one of the following: Calculating a second loss value based on the beautification parameters of the T images and the beautification parameters of the T third images; Based on the beautification parameters of the NT images and the beautification parameters of the NT third images, a third loss value is calculated.

10. The device according to claim 6, characterized in that The device also includes: an acquisition module; The acquisition module is used to acquire, before the processing module inputs the M groups of first images into the first scoring model, the M groups of second scoring scores of the first user for the M groups of fourth images, wherein the M groups of fourth images correspond one-to-one to the M beautification models; The processing module is further used to input the M groups of fourth images into a second scoring model, score the M groups of fourth images, and output the M groups of third scoring scores; The processing module is further used to calculate a fourth loss value based on the M groups of second scoring values ​​obtained by the obtaining module and the M groups of third scoring values ​​output by the processing module; The updating module is further used to update the model parameters of the second scoring model based on the fourth loss value calculated by the processing module to obtain the first scoring model; The model parameters of the second scoring model are used to characterize various image aesthetic tendencies.

Images