Image processing method, device, computer equipment and computer readable storage medium

Abstract

The present application relates to an image processing method, apparatus, computer equipment and computer-readable storage medium. The method comprises: obtaining an image to be processed and a target user identifier; extracting image aesthetic attribute features corresponding to the image to be processed; determining a generalized aesthetic evaluation result of the image to be processed according to the image aesthetic attribute features; determining a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to the image aesthetic attribute features and the user personality features corresponding to the target user identifier; and determining a personalized aesthetic evaluation result corresponding to the target user identifier based on the generalized aesthetic evaluation result and the personalized aesthetic evaluation correction parameter. The use of this method can improve the accuracy of the aesthetic evaluation results.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, apparatus, computer device and computer-readable storage medium. Background Art

[0002] With the research and progress of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as image aesthetic evaluation. Image aesthetic evaluation uses computer systems to simulate human aesthetic perception of images and then conduct aesthetic evaluation of images. Image aesthetic evaluation can be specifically applied to image recommendation, image enhancement, image retrieval, and personal album management.

[0003] Traditional image aesthetic evaluation is mainly based on the aesthetic attributes of the image itself. However, due to differences in culture, age, gender, etc., different users have different aesthetic judgment standards for images. As a result, the aesthetic evaluation results obtained based on the aesthetic attribute evaluation of the image cannot cater to the aesthetic preferences of some users, resulting in inaccurate aesthetic evaluation results. Summary of the invention

[0004] Based on this, it is necessary to provide an image processing method, apparatus, computer device and computer-readable storage medium that can improve the accuracy of aesthetic evaluation results in response to the above technical problems.

[0005] An image processing method, the method comprising:

[0006] Obtain the image to be processed and the target user ID;

[0007] Extracting the image aesthetic attribute features corresponding to the image to be processed;

[0008] Determine the generalized aesthetic evaluation result of the image to be processed according to the image aesthetic attribute characteristics;

[0009] Determine a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to the image aesthetic attribute characteristics and the user personality characteristics corresponding to the target user identifier;

[0010] Based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter, a personalized aesthetic evaluation result corresponding to the target user identifier is determined.

[0011] An image processing device, the device comprising:

[0012] An acquisition module, used to acquire the image to be processed and the target user ID;

[0013] An extraction module, used for extracting the image aesthetic attribute features corresponding to the image to be processed;

[0014] A universal aesthetic evaluation module, used to determine the universal aesthetic evaluation result of the image to be processed according to the image aesthetic attribute characteristics;

[0015] A correction parameter determination module, used to determine a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to the image aesthetic attribute characteristics and the user personality characteristics corresponding to the target user identifier;

[0016] The personalized aesthetic evaluation module is used to determine the personalized aesthetic evaluation result corresponding to the target user identifier based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter.

[0017] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0018] Obtain the image to be processed and the target user ID;

[0019] Extracting the image aesthetic attribute features corresponding to the image to be processed;

[0020] Determine the generalized aesthetic evaluation result of the image to be processed according to the image aesthetic attribute characteristics;

[0021] Determine a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to the image aesthetic attribute characteristics and the user personality characteristics corresponding to the target user identifier;

[0022] Based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter, a personalized aesthetic evaluation result corresponding to the target user identifier is determined.

[0023] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0024] Obtain the image to be processed and the target user ID;

[0025] Extracting the image aesthetic attribute features corresponding to the image to be processed;

[0026] Determine the generalized aesthetic evaluation result of the image to be processed according to the image aesthetic attribute characteristics;

[0027] Determine a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to the image aesthetic attribute characteristics and the user personality characteristics corresponding to the target user identifier;

[0028] Based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter, a personalized aesthetic evaluation result corresponding to the target user identifier is determined.

[0029] The above-mentioned image processing method, device, computer equipment and computer-readable storage medium first determine a universal aesthetic evaluation result based on the image aesthetic attribute characteristics of the image to be processed, and determine a personalized aesthetic evaluation correction parameter based on the image aesthetic attribute characteristics and the user personality characteristics corresponding to the target user identifier, and then correct the universal aesthetic evaluation result based on the personalized aesthetic evaluation correction parameter to obtain a personalized aesthetic evaluation result corresponding to the target user identifier. In this way, since the universal aesthetic evaluation result is the result obtained by evaluating the aesthetic characteristics of the image itself, it reflects the aesthetic quality of the image at the objective level, and the universal aesthetic evaluation result is corrected according to the aesthetic deviation corresponding to the user's personality characteristics, the corrected personalized aesthetic evaluation result is close to the aesthetic evaluation result obtained by the user based on his or her own aesthetics, that is, the image is personalized aesthetically evaluated in combination with the user's subjective aesthetics and the objective aesthetic quality of the image, so that the personalized aesthetic evaluation result conforms to the aesthetic preferences of individual users, thereby improving the accuracy of the personalized aesthetic evaluation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A diagram showing an application environment of an image processing method in an embodiment;

[0031] Figure 2 is a schematic flow chart of an image processing method in one embodiment;

[0032] Figure 3 is a schematic diagram of a personalized aesthetic evaluation model in one embodiment;

[0033] Figure 4 A schematic diagram showing a comparison of personalized aesthetic evaluation performance in one embodiment;

[0034] Figure 5 A schematic diagram of a process for obtaining user personality characteristics in one embodiment;

[0035] Figure 6 is a schematic diagram of the structure of a personalized aesthetic evaluation model in one embodiment;

[0036] FIG. 7( a ) is a schematic diagram of training an image aesthetic attribute feature extraction network in one embodiment;

[0037] FIG7( b ) is a schematic diagram of training a generalized aesthetic evaluation network in one embodiment;

[0038] FIG7( c ) is a schematic diagram of training a personalized aesthetic evaluation correction network in one embodiment;

[0039] Figure 8 is a flowchart of an image processing method in yet another embodiment;

[0040] Fig. 9is a structural block diagram of an image processing device in one embodiment;

[0041] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0044] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0045] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0046] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, driverless cars, autonomous driving, drones, robots, smart medical care, smart customer service, image recommendation, image enhancement, image retrieval, personal album management, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0047] The solution provided in the embodiments of the present application involves artificial intelligence, machine learning and other technologies, which are specifically described by the following embodiments:

[0048] The image processing method provided in this application can be applied to Figure 1 In the application environment shown in FIG. 1 , the terminal 102 communicates with the server 104 through a network. The terminal 102 obtains the image to be processed and the target user identifier, and sends the image to be processed and the target user identifier to the server 104; then, the server 104 extracts the image aesthetic attribute features corresponding to the image to be processed; then, the server 104 determines the generalized aesthetic evaluation result of the image to be processed according to the image aesthetic attribute features; then, the server 104 determines the personalized aesthetic evaluation correction parameters corresponding to the target user identifier according to the image aesthetic attribute features and the user personality features corresponding to the target user identifier; then, the server 104 determines the personalized aesthetic evaluation result corresponding to the target user identifier based on the generalized aesthetic evaluation result and the personalized aesthetic evaluation correction parameters.

[0049] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud storage, network services, cloud communications, big data, and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected via wired or wireless communications, and this application does not limit this.

[0050] In one embodiment, Figure 2 As shown, an image processing method is provided. This embodiment mainly applies this method to the above Figure 1 Taking the computer device (terminal 102 or server 104) in FIG. 1 as an example, the following steps are included:

[0051] Step 202: Obtain the image to be processed and the target user identifier.

[0052] The image to be processed is an image to be aesthetically evaluated by the method provided in the embodiment of the present application.

[0053] In the present application, an aesthetic evaluation of an image can be performed on a specific user by simulating the user's aesthetic perception of the image, so that the aesthetic evaluation result obtained by the evaluation is close to the aesthetic evaluation result obtained by the user based on his or her own aesthetic taste, thereby improving the accuracy of the aesthetic evaluation result.

[0054] The target user ID is the user ID of the target user, which is used to describe the user's identity information and is unique. Specifically, it can be an application account, a mobile phone number, an ID number, etc. The aesthetic evaluation of the processed image is performed by simulating the target user's aesthetic perception of the image, so that the aesthetic evaluation result obtained by the evaluation is consistent with the aesthetic taste of the target user.

[0055] Specifically, an image processing application is running on the terminal, and the target user has a target user ID registered based on the image processing application. The terminal can start the image processing application based on the target user's operation, and the image processing application performs aesthetic evaluation on the image to be processed based on the aesthetic understanding of the target user.

[0056] In one embodiment, the source of the image to be processed is related to the image aesthetic evaluation task, that is, the image to be processed corresponding to the image aesthetic evaluation task is obtained. The image aesthetic evaluation task can be image recommendation, image enhancement, image retrieval, personal album management, etc. Taking image recommendation as an example, the computer device obtains multiple images to be recommended, and based on the aesthetic understanding of the target user, preferentially recommends images that match the aesthetics of the target user to the target user, thereby improving the accuracy of image recommendation.

[0057] Step 204: extracting the image aesthetic attribute features corresponding to the image to be processed.

[0058] Among them, the image aesthetic attribute features are data that reflect the aesthetic characteristics of the image. Image aesthetic characteristics are the characteristics of the image at the aesthetic level, such as composition characteristics, color characteristics, brightness characteristics, depth of field characteristics, content characteristics, etc. The composition characteristics describe the composition method of the image, which can be one-third composition, symmetrical composition, frame composition, center composition, guide line composition, diagonal composition, triangle composition, balanced composition, etc.; the color characteristics describe the color composition of the image, which can be color temperature, hue, color components, color coordination, etc.; the brightness characteristics describe the brightness of the image; the depth of field characteristics describe the clear range of the main object in the image and the degree of blurring of the background; the content characteristics describe the content contained in the image.

[0059] In one embodiment, the image aesthetic feature may include multiple feature dimensions, each of which corresponds to an image aesthetic characteristic. For example, the image aesthetic feature may include 11 feature dimensions, and the image aesthetic characteristics corresponding to each of the 11 feature dimensions may be: content of interest, subject object, content repetitiveness, brightness, color coordination, color vividness, depth of field, motion blur, one-third composition, symmetrical composition, and balanced composition. The feature value of each feature dimension can be used to represent the score of the image in the corresponding image aesthetic characteristic.

[0060] In one embodiment, step 204 includes: obtaining an image aesthetic attribute feature extraction network; inputting the image to be processed into the image aesthetic attribute feature extraction network, and outputting the image aesthetic attribute features through the image aesthetic attribute feature extraction network.

[0061] The image aesthetic attribute feature extraction network is a machine learning model trained with the ability to extract image aesthetic attribute features. The image aesthetic attribute feature extraction network can be trained through image samples and image aesthetic attribute feature training labels.

[0062] It can be understood that the image aesthetic attribute features extracted by a general machine learning model capable of extracting image aesthetic attribute features, such as a ResNet model, a ResNet-18 model, etc., meet the requirements of the embodiment of the present application for image aesthetic attribute features. The general machine learning model capable of extracting image aesthetic attribute features can be used as the image aesthetic attribute feature extraction network of the embodiment of the present application.

[0063] Specifically, the computer device obtains an image to be processed and an image aesthetic attribute feature extraction network, and the computer device inputs the image to be processed into the image aesthetic attribute feature extraction network to obtain image aesthetic attribute features output by the image aesthetic attribute feature extraction network.

[0064] Step 206: Determine a generalized aesthetic evaluation result of the image to be processed according to the image aesthetic attribute characteristics.

[0065] Among them, the generalized aesthetic evaluation result is the result obtained by evaluating the aesthetic characteristics of the image itself, which can reflect the popular aesthetics to a certain extent.

[0066] In the present application, an aesthetic evaluation of the image may be first performed based on the aesthetic attributes of the image itself, and the aesthetic evaluation result obtained based on the evaluation of the aesthetic attributes of the image reflects the aesthetic quality of the image at an objective level.

[0067] In one embodiment, the generalized aesthetic evaluation result may be a generalized aesthetic evaluation score. The generalized aesthetic evaluation score may be positively correlated with the aesthetic quality of the image at the objective level. The higher the generalized aesthetic evaluation score is, the higher the aesthetic quality of the image at the objective level is.

[0068] In one embodiment, step 206 includes: obtaining a generalized aesthetic evaluation network; inputting the image aesthetic attribute features into the generalized aesthetic evaluation network, and outputting a generalized aesthetic evaluation result through the generalized aesthetic evaluation network.

[0069] Among them, the generalized aesthetic evaluation network is a machine learning model trained with generalized aesthetic evaluation capabilities. The generalized aesthetic evaluation network can be trained through image aesthetic attribute feature samples and aesthetic evaluation score probability distribution training labels.

[0070] Specifically, the computer device obtains image aesthetic attribute features and a generalized aesthetic evaluation network, and the computer device inputs the image aesthetic attribute features into the generalized aesthetic evaluation network to obtain a generalized aesthetic evaluation result output by the generalized aesthetic evaluation network.

[0071] Step 208 : determining a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to the image aesthetic attribute characteristics and the user personality characteristics corresponding to the target user identifier.

[0072] Among them, the user personality characteristics are data reflecting the user's personality characteristics, and the user personality characteristics are correlated with the user's aesthetic feelings towards the image.

[0073] In this application, considering that different users have different aesthetic judgment standards for images, and user personality is an important factor affecting user aesthetic preferences, the user personality characteristic parameter is introduced. The user personality characteristic is a parameter associated with the user's aesthetic perception of the image. This application divides user groups based on user personality characteristics, studies the aesthetic deviation of different user personality characteristics relative to the universal aesthetic evaluation results, and then corrects the universal aesthetic evaluation results according to the aesthetic deviation of the user personality characteristics of the target user, so that the corrected personalized aesthetic evaluation results are close to the aesthetic evaluation results obtained by the target user based on his or her own aesthetics.

[0074] In one embodiment, the user's personality characteristics may include multiple characteristic dimensions, each of which corresponds to a personality trait. The characteristic value of each characteristic dimension can be used to represent the score of the user in the corresponding personality trait, which is related not only to the user's own personality trait but also to the user's aesthetic feeling of the image.

[0075] For example, user personality traits may include five characteristic dimensions, and the corresponding personality traits of each of these five characteristic dimensions may be openness, responsibility, extraversion, agreeableness, and neuroticism. These five personality traits are collectively referred to as the Big Five personality traits, which are a type of overall personality structure model. Openness is used to characterize the degree of curiosity, wisdom, imagination, and creativity; responsibility is used to characterize the sense of responsibility, reliability, and perseverance in work; extraversion is used to characterize the degree of good communication and self-confidence; agreeableness is used to characterize the degree of easygoingness, cooperation, and trustworthiness; neuroticism is used to characterize the degree of calmness, enthusiasm, and security.

[0076] Among them, the personalized aesthetic evaluation correction parameters are used to correct the universal aesthetic evaluation results, so that the corrected personalized aesthetic evaluation results are consistent with the aesthetic taste of the target users.

[0077] In one embodiment, the user personality characteristics corresponding to the target user identification may be predetermined, and the computer device may obtain the user personality characteristics corresponding to the input target user identification, or extract the user personality characteristics corresponding to the target user identification from pre-stored user personality characteristics.

[0078] In one embodiment, step 208 includes: obtaining a personalized aesthetic evaluation correction network; and obtaining personalized aesthetic evaluation correction parameters according to the image aesthetic attribute characteristics and the user's personality characteristics through the personalized aesthetic evaluation correction network.

[0079] Among them, the personalized aesthetic evaluation correction network is a machine learning model that is trained with the ability to calculate personalized aesthetic evaluation correction parameters. The personalized aesthetic evaluation correction network can be trained through image aesthetic attribute feature samples, user personality feature samples and personalized aesthetic evaluation correction parameter training labels.

[0080] In one embodiment, the computer device obtains personalized aesthetic evaluation correction parameters through a personalized aesthetic evaluation correction network according to a fusion result of the image aesthetic attribute characteristics and the user's personality characteristics.

[0081] Among them, feature fusion can be concatenating, adding, matrix multiplying, etc. For example, the image aesthetic attribute features of 11 feature dimensions are fused with the user personality features of 5 feature dimensions, and the fused features of 55 feature dimensions are obtained through matrix multiplication.

[0082] Specifically, the computer device obtains the image aesthetic attribute features, the user personality features and the personalized aesthetic evaluation correction network, the computer device first determines the fusion result of the image aesthetic attribute features and the user personality features, and then inputs the fusion result into the personalized aesthetic evaluation correction network to obtain the personalized aesthetic evaluation correction parameters output by the personalized aesthetic evaluation correction network. Alternatively, the computer device inputs the image aesthetic attribute features and the user personality features into the personalized aesthetic evaluation correction network, the personalized aesthetic evaluation correction network first determines the fusion result of the image aesthetic attribute features and the user personality features, and then outputs the personalized aesthetic evaluation correction parameters according to the fusion result.

[0083] Step 210 : determining a personalized aesthetic evaluation result corresponding to the target user identifier based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter.

[0084] Among them, the personalized aesthetic evaluation result is an aesthetic evaluation result obtained by simulating the target user's aesthetic appreciation of the processed image.

[0085] In one embodiment, the personalized aesthetic evaluation result may be a personalized aesthetic evaluation score, which may reflect the degree of fit between the user's subjective aesthetics and the image aesthetic quality. The higher the personalized aesthetic evaluation score, the more the image conforms to the user's subjective aesthetics.

[0086] Specifically, the computer device adjusts the universal aesthetic evaluation result according to the personalized aesthetic evaluation correction parameter to obtain the personalized aesthetic evaluation result corresponding to the target user identifier.

[0087] In one embodiment, referring to Figure 3 , Figure 3 Schematic diagram of a personalized aesthetic evaluation model in an embodiment. It can be seen that the personalized aesthetic evaluation model includes an image aesthetic attribute feature extraction network, a generalized aesthetic evaluation network, and a personalized aesthetic evaluation correction network. First, the image to be processed 302 is input into the image aesthetic attribute feature extraction network, and the image aesthetic attribute feature 304 is output through the image aesthetic attribute feature extraction network; then, the image aesthetic attribute feature 304 is input into the generalized aesthetic evaluation network, and the generalized aesthetic evaluation result 306 is output through the generalized aesthetic evaluation network; then, the personalized aesthetic evaluation correction network is used to obtain the personalized aesthetic evaluation correction parameter 310 according to the image aesthetic attribute feature 304 and the user personality feature 308 of the target user identifier; finally, based on the generalized aesthetic evaluation result 306 and the personalized aesthetic evaluation correction parameter 310, the personalized aesthetic evaluation result 312 corresponding to the target user identifier is determined.

[0088] This application selects two traditional personalized aesthetic evaluation methods and conducts personalized aesthetic evaluation performance tests on the FLICKR-AES test set. The Spearman Rank Order Correlation Coefficient (SROCC) is used to measure the personalized aesthetic evaluation performance of the three methods. SROCC is used to quantitatively measure the ranking correlation between personalized aesthetic evaluation results and true results. The larger the SROCC value, the better the personalized aesthetic evaluation performance.

[0089] Specifically, 10 or 100 images scored by each user in the FLICKR-AES test set were randomly selected as training samples, and the remaining images were used as test samples. Training and testing were performed using three methods. For one of the methods, in order to eliminate the random selection error, the above process was repeated 50 times, and the mean SROCC was taken as the personalized aesthetic evaluation performance corresponding to each user. Finally, the average result of the personalized aesthetic evaluation performance corresponding to each of the 37 users in the test set was taken as the test result of this method.

[0090] Reference Figure 4 , Figure 4 Schematic diagram of the comparison of the performance of personalized aesthetic evaluation in one embodiment. It can be seen that the SROCC value obtained by the present application for individual user testing is higher than the SROCC value obtained by the traditional personalized aesthetic evaluation method for individual user testing, indicating that the present application has a more superior performance in personalized aesthetic evaluation for individual users.

[0091] The personalized aesthetic evaluation model provided in this application can simulate the personalized aesthetic appreciation of individual users for images. Experiments have shown that it has better aesthetic evaluation performance than traditional personalized aesthetic evaluation methods and can be widely used in personalized image aesthetic analysis.

[0092] In the above-mentioned image processing method, a universal aesthetic evaluation result is first determined according to the image aesthetic attribute characteristics of the image to be processed, and a personalized aesthetic evaluation correction parameter is determined according to the image aesthetic attribute characteristics and the user personality characteristics corresponding to the target user identifier, and then the universal aesthetic evaluation result is corrected based on the personalized aesthetic evaluation correction parameter to obtain a personalized aesthetic evaluation result corresponding to the target user identifier. In this way, since the universal aesthetic evaluation result is the result obtained by evaluating the aesthetic characteristics of the image itself, it reflects the aesthetic quality of the image at the objective level, and the universal aesthetic evaluation result is corrected according to the aesthetic deviation corresponding to the user's personality characteristics, the corrected personalized aesthetic evaluation result is close to the aesthetic evaluation result obtained by the user based on his or her own aesthetics, that is, the image is personalized aesthetically evaluated in combination with the user's subjective aesthetics and the objective aesthetic quality of the image, so that the personalized aesthetic evaluation result conforms to the aesthetic preferences of individual users, thereby improving the accuracy of the personalized aesthetic evaluation result.

[0093] In one embodiment, there are multiple images to be processed; the personalized aesthetic evaluation result is a personalized aesthetic evaluation score; the method also includes: obtaining the personalized aesthetic evaluation score of the target user identifier for each image to be processed; selecting a preset number of target images from each image to be processed according to the personalized aesthetic evaluation score; and outputting the target images to the terminal where the target user identifier is located.

[0094] Among them, the source of the images to be processed is related to the image aesthetic evaluation task, and the image aesthetic evaluation task can be image recommendation, image enhancement, image retrieval, personal album management, etc.

[0095] Specifically, the computer device obtains multiple images to be processed corresponding to the image aesthetic evaluation task, obtains the personalized aesthetic evaluation score of the target user identifier for each image to be processed, selects a preset number of target images from high to low according to the personalized aesthetic evaluation score, and outputs the target images to the terminal where the target user identifier is located. In this way, in each image aesthetic evaluation task, the computer device gives priority to displaying images that meet the target user's aesthetic taste to the target user.

[0096] Taking image retrieval as an example, the computer device obtains the search keywords input by the target user identifier, retrieves multiple search images corresponding to the search keywords, determines the personalized aesthetic evaluation score of the target user identifier for each search image through the method provided in this embodiment, selects a preset number of images from high to low according to the personalized aesthetic evaluation scores as search results, and outputs the search results to the terminal where the target user identifier is located, so as to preferentially display images that meet the target user's aesthetic taste to the target user, thereby improving image retrieval efficiency.

[0097] The personalized aesthetic evaluation method provided in this application can be applied to various image aesthetic evaluation tasks to improve the execution efficiency of various image aesthetic evaluation tasks.

[0098] The present application involves user personality characteristics, which can be determined in advance or determined according to reference images in an application scenario. A method for obtaining user personality characteristics corresponding to a target user identifier is provided below.

[0099] In one embodiment, referring to Figure 5 , Figure 5 FIG. 1 is a flow chart of obtaining a user's personality characteristics in an embodiment. It can be seen that obtaining the user's personality characteristics corresponding to the target user identifier includes:

[0100] Step 502 , obtaining more than one reference image corresponding to the target user identifier, each reference image being marked with a personalized aesthetic evaluation result corresponding to the target user identifier; the personalized aesthetic evaluation result is a personalized aesthetic evaluation score.

[0101] In one embodiment, the computer device outputs more than one reference image to the target user identifier, and receives more than one reference image returned by the target user identifier and annotated with the personalized aesthetic evaluation result. Alternatively, the computer device directly receives more than one reference image corresponding to the target user identifier, and each reference image is annotated with the personalized aesthetic evaluation result corresponding to the target user identifier.

[0102] In one embodiment, only the personalized aesthetic evaluation score corresponding to the target user identifier may be marked on the reference image, or both the personalized aesthetic evaluation score corresponding to the target user identifier and the personalized aesthetic evaluation scores corresponding to other user identifiers may be marked.

[0103] Step 504, obtaining a preferred user personality feature extraction network, inputting more than one reference image into the preferred user personality feature extraction network respectively, and obtaining the preferred user personality features corresponding to each of the more than one reference image through the preferred user personality feature extraction network.

[0104] For a reference image, the preferred user personality feature is the personality feature of the preferred reference image. The preferred user personality feature may include multiple feature dimensions, each feature dimension corresponds to a personality characteristic, and the feature value of each feature dimension is used to represent the score of the user in the corresponding personality characteristic, and the score is only related to the personality characteristic of the user itself.

[0105] It can be understood that the characteristic value of the preferred user personality feature is only related to the personality characteristics of the user itself, and the characteristic value of the aforementioned user personality feature is not only related to the personality characteristics of the user itself, but also related to the user's aesthetic feeling of the image. For the sake of distinction, the personality features involved in the preferred user personality features are referred to as general personality features, and the personality features involved in the user personality features are referred to as aesthetic personality features.

[0106] The preferred user personality feature extraction network is a machine learning model trained to have the ability to extract preferred user personality features. The preferred user personality feature extraction network can be trained through image samples and preferred user personality feature training labels.

[0107] Specifically, the computer device obtains a reference image and a preferred user personality feature extraction network, and the computer device inputs more than one reference image into the preferred user personality feature extraction network respectively to obtain the preferred user personality features corresponding to the more than one reference images output by the preferred user personality feature extraction network.

[0108] Step 506 : Obtain the maximum value and the minimum value of the personalized aesthetic evaluation corresponding to each reference image and the target user identifier.

[0109] Specifically, the computer device selects a maximum personalized aesthetic evaluation value and a minimum personalized aesthetic evaluation value from the personalized aesthetic evaluation scores corresponding to each reference image and the target user identifier.

[0110] For example, five reference images corresponding to the target user identifier are obtained: reference image A, reference image B, reference image C, reference image D, and reference image E. The personalized aesthetic evaluation score corresponding to reference image A and the target user identifier is 1, the personalized aesthetic evaluation score corresponding to reference image B and the target user identifier is 2, the personalized aesthetic evaluation score corresponding to reference image C and the target user identifier is 3, the personalized aesthetic evaluation score corresponding to reference image D and the target user identifier is 4, and the personalized aesthetic evaluation score corresponding to reference image E and the target user identifier is 5. Then the maximum personalized aesthetic evaluation is 5, and the minimum personalized aesthetic evaluation is 1.

[0111] Step 508 : For one of the reference images, based on the personalized aesthetic evaluation score, the maximum personalized aesthetic evaluation value, and the minimum personalized aesthetic evaluation value corresponding to the target user identifier, obtain the preference degree of the target user identifier for the reference image.

[0112] In one embodiment, a computer device obtains a first difference between a maximum personalized aesthetic evaluation value and a minimum personalized aesthetic evaluation value, obtains a second difference between a personalized aesthetic evaluation score corresponding to a target user identifier on a reference image and the minimum personalized aesthetic evaluation value, and uses a ratio between the second difference and the first difference as a degree of preference of the target user identifier for the reference image.

[0113] Continuing with the above example, the difference between the maximum personalized aesthetic evaluation value and the minimum personalized aesthetic evaluation value is 4, then the target user identification's corresponding preference levels for reference image A, reference image B, reference image C, reference image D, and reference image E are: 0, 1 / 4, 1 / 2, 3 / 4, and 1, respectively.

[0114] Step 510 , based on the preference degree of the target user identifier to each reference image and the preferred user personality characteristics corresponding to each reference image, obtain the user personality characteristics corresponding to the target user identifier.

[0115] In the present application, the user's personality characteristics are adjusted according to the user's preference for the image, so that the characteristic value of the user's personality characteristics is related to the user's aesthetic feeling for the image.

[0116] In one embodiment, for one of the reference images, the computer device adjusts the feature values ​​of each dimension of the preferred user personality characteristic corresponding to the reference image according to the preference degree of the target user identifier for the reference image, and obtains the user personality characteristic of the target user identifier relative to the reference image. The computer device obtains the average value or weighted average value of each dimension of the user personality characteristic of the target user identifier relative to each reference image, and obtains the user personality characteristic corresponding to the target user identifier.

[0117] Continuing with the above example, the preferred user personality characteristics corresponding to reference image A are (a1, a2, a3, a4, a5), and the user personality characteristics of the target user identity relative to reference image A are (0, 0, 0, 0, 0); the preferred user personality characteristics corresponding to reference image B are (b1, b2, b3, b4, b5), and the user personality characteristics of the target user identity relative to reference image B are (1 / 4b1, 1 / 4b2, 1 / 4b3, 1 / 4b4, 1 / 4b5); the preferred user personality characteristics corresponding to reference image C are (c1, c2, c3, c4, c5), and the user personality characteristics of the target user identity relative to reference image C are (1 / 2c1, 1 / 2c2, 1 / 2c3, 1 / 2 c4,1 / 2c5); the preferred user personality features corresponding to reference image D are (d1,d2,d3,d4,d5), and the user personality features of the target user ID relative to reference image D are (3 / 4d1,3 / 4d2,3 / 4d3,3 / 4d4,3 / 4d5); the preferred user personality features corresponding to reference image E are (e1,e2,e3,e4,e5) and the user personality features of the target user ID relative to reference image E are (e1,e2,e3,e4,e5), then the user personality features corresponding to the target user ID are 1 / 5*(0+1 / 4b1+1 / 2c1+3 / 4d1+e1,……,0+1 / 4b5+1 / 2c5+3 / 4d5+e5).

[0118] In this embodiment, the user personality characteristics of the user are obtained based on the user's preference degree for the reference image and the preferred user personality characteristics of the reference image. The obtained user personality characteristics are related to the user's aesthetic feelings about the image. The aesthetic evaluation results are subsequently determined based on the user personality characteristics and the image aesthetic attribute characteristics, which can improve the accuracy of the aesthetic evaluation results.

[0119] In one embodiment, referring to Figure 6 , Figure 6 FIG. 1 is a schematic diagram of the structure of a personalized aesthetic evaluation model in an embodiment. It can be seen that the personalized aesthetic evaluation model includes an image aesthetic attribute feature extraction network, a preference user personality feature extraction network, a generalized aesthetic evaluation network, and a personalized aesthetic evaluation correction network.

[0120] Specifically, first, the computer device obtains more than one reference image 602 corresponding to the target user identifier, and each reference image 602 is marked with a personalized aesthetic evaluation score corresponding to the target user identifier. The computer device inputs the more than one reference image 602 into the preferred user personality feature extraction network, and obtains the preferred user personality features 604 corresponding to each of the more than one reference images through the preferred user personality feature extraction network. The computer device obtains the user personality features 606 corresponding to the target user identifier based on the personalized aesthetic evaluation scores corresponding to the target user identifier marked on each reference image and the preferred user personality features 604 corresponding to each reference image.

[0121] Secondly, after the computer device obtains the user personality characteristics 606 corresponding to the target user identifier, the image to be processed 608 is input into the image aesthetic attribute feature extraction network, and the image aesthetic attribute feature 610 is output through the image aesthetic attribute feature extraction network. Then, the image aesthetic attribute feature 610 is input into the generalized aesthetic evaluation network, and the generalized aesthetic evaluation network outputs the generalized aesthetic evaluation result 612. Then, through the personalized aesthetic evaluation correction network, the personalized aesthetic evaluation correction parameter 614 is obtained according to the image aesthetic attribute feature 610 and the user personality characteristics 606 of the target user identifier. Finally, based on the generalized aesthetic evaluation result 612 and the personalized aesthetic evaluation correction parameter 614, the personalized aesthetic evaluation result 616 corresponding to the target user identifier is determined.

[0122] In this embodiment, a personalized aesthetic evaluation model is constructed using both the objective attributes of the image and the subjective attributes of the user. That is, the difference between the personalized aesthetics and the popular aesthetics of the image is modeled using the personality characteristics of different users, so that the personalized aesthetic evaluation results obtained through the personalized aesthetic evaluation model are more in line with the aesthetic preferences of individual users.

[0123] The present application relates to a training method for a personalized aesthetic evaluation model, which is specifically described through the following embodiments.

[0124] First, for the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network, the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network can be trained separately, that is, the image aesthetic attribute feature extraction network is trained through image samples and image aesthetic attribute feature training labels, and the preferred user personality feature extraction network is trained through image samples and preferred user personality feature training labels. The image aesthetic attribute feature extraction network and the preferred user personality feature extraction network can also be trained together, and a joint training method is provided below.

[0125] In one embodiment, an image aesthetic attribute feature extraction network and a preferred user personality feature extraction network are jointly trained; the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network share a basic subnetwork and respectively include corresponding output subnetworks; the steps of jointly training the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network include: obtaining a first image sample set, a second image sample set, an image aesthetic attribute feature extraction network and a preferred user personality feature extraction network; each first image sample in the first image sample set has an image aesthetic attribute feature training label, and each second image sample in the second image sample set has a preferred user personality feature training label; inputting the first image sample into the image aesthetic attribute feature extraction network The method comprises the following steps: a first image sample is extracted from a basic sub-network of the image aesthetic attribute feature extraction network, and an image aesthetic attribute feature prediction result is outputted through an output sub-network of the image aesthetic attribute feature extraction network; a second image sample is inputted into a preference user personality feature extraction network, and a second image sample is extracted from a basic sub-network of the preference user personality feature extraction network, and a preference user personality feature prediction result is outputted through an output sub-network of the preference user personality feature extraction network; and the image aesthetic attribute feature extraction network and the preference user personality feature extraction network are jointly trained based on the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, as well as the preference user personality feature prediction result and the preference user personality feature training label.

[0126] Among them, the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network share a basic subnetwork and respectively include corresponding output subnetworks. The number of the basic subnetworks can be one or two. When the number of the basic subnetworks is one, the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network share a basic subnetwork, and the basic subnetwork is connected to two output subnetworks; when the number of the basic subnetworks is two, the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network each correspond to a basic subnetwork, and the two basic subnetworks are each connected to an output subnetwork, and the model structures of the two basic subnetworks are the same.

[0127] In one embodiment, the basic sub-network may use a convolutional network of ResNet18, and generate a fully connected network using a global average pooling operation. Both output sub-networks may use a fully connected network. For ease of calculation, the feature values ​​of the image aesthetic attribute features and the preferred user personality features may be normalized to [0, 1], so the Sigmoid function may be used as the activation function of the output sub-network.

[0128] Among them, each first image sample in the first image sample set has an image aesthetic attribute feature training label. The first image sample set can specifically adopt the AADB (Image Aesthetics Analysis Database) image dataset. The AADB image dataset contains 1,000 images, and each image is annotated by multiple sample users with 11 image aesthetic characteristics: content of interest, main object, content repeatability, brightness, color coordination, color vividness, depth of field, motion blur, one-third composition, symmetrical composition and balanced composition.

[0129] Each second image sample in the second image sample set has a preferred user personality characteristic training label. The second image sample set can specifically use the PsychoFlickr image dataset, which includes the preferred images of 300 sample users, and the number of preferred images of each sample user is 200. The general personality characteristics of each sample user are described by the Big Five personality traits (openness, responsibility, extraversion, agreeableness and neuroticism), and the general personality characteristics of each sample user are determined by the BFI-10 (10-Item Big Five Inventory) psychological test.

[0130] Specifically, taking two basic sub-networks as an example, the computer device inputs the first image sample into the image aesthetic attribute feature extraction network, extracts features of the first image sample through the basic sub-network of the image aesthetic attribute feature extraction network, and outputs the image aesthetic attribute feature prediction result through the output sub-network of the image aesthetic attribute feature extraction network. Then, the computer device inputs the second image sample into the preference user personality feature extraction network, extracts features of the second image sample through the basic sub-network of the preference user personality feature extraction network, and outputs the preference user personality feature prediction result through the output sub-network of the preference user personality feature extraction network. Then, based on the difference between the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, and the difference between the preference user personality feature prediction result and the preference user personality feature training label, the computer device jointly trains the image aesthetic attribute feature extraction network and the preference user personality feature extraction network.

[0131] In one embodiment, based on the image aesthetic attribute feature prediction results and the image aesthetic attribute feature training labels, as well as the preferred user personality feature prediction results and the preferred user personality feature training labels, jointly training an image aesthetic attribute feature extraction network and a preferred user personality feature extraction network includes: constructing a first loss function based on the difference between the image aesthetic attribute feature prediction results and the image aesthetic attribute feature training labels, and constructing a second loss function based on the difference between the preferred user personality feature prediction results and the preferred user personality feature training labels; obtaining a first network gradient parameter by minimizing the first loss function, and obtaining a second network gradient parameter by minimizing the second loss function; and jointly training the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network by combining the first network gradient parameter and the second network gradient parameter.

[0132] Among them, the network gradient parameter is the gradient of the back-propagation update of the model parameters, which can be determined by continuously reducing the loss function.

[0133] The first loss function is specifically expressed by the following formula:

[0134]

[0135] Among them, L1 is the first loss function; Train labels for image aesthetic attribute features; is the prediction result of the image aesthetic attribute feature; N is the number of first image samples; I is the feature dimension of the image aesthetic attribute feature.

[0136] The second loss function is specifically expressed by the following formula:

[0137]

[0138] Among them, L2 is the second loss function; Train labels for preferred user personality traits; is the prediction result of the preferred user's personality characteristics; N is the number of first image samples; J is the characteristic dimension of the preferred user's personality characteristics.

[0139] Specifically, the computer device constructs a first loss function based on the difference between the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, such as constructing the first loss function through the Euclidean distance between the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label; and constructs a second loss function based on the difference between the preference user personality feature prediction result and the preference user personality feature training label, such as constructing the second loss function through the Euclidean distance between the preference user personality feature prediction result and the preference user personality feature training label. Then, the computer device obtains the first network gradient parameter by minimizing the first loss function, and obtains the second network gradient parameter by minimizing the second loss function. Then, the computer device combines the first network gradient parameter and the second network gradient parameter to jointly train the image aesthetic attribute feature extraction network and the preference user personality feature extraction network, such as obtaining the sum of the first network gradient parameter and the second network gradient parameter, and uses the sum of the first network gradient parameter and the second network gradient parameter to back-propagate and update the model parameters of the image aesthetic attribute feature extraction network and the preference user personality feature extraction network.

[0140] In this embodiment, since the image aesthetic attribute feature extraction network and the preference user personality feature extraction network share a basic sub-network, the image aesthetic attribute feature extraction network and the preference user personality feature extraction network can be trained together, thereby saving computer resources and improving the model training speed; and obtaining the objective attributes of the image and the subjective attributes of the user in the same model has practical value for the study of personalized aesthetic evaluation.

[0141] Secondly, for the generalized aesthetic evaluation network, a training method for a generalized aesthetic evaluation network is provided below.

[0142] In one embodiment, the training steps of the generalized aesthetic evaluation network include: obtaining a fourth image sample set and a generalized aesthetic evaluation network; each fourth image sample in the fourth image sample set has an aesthetic evaluation score probability distribution training label; extracting image aesthetic attribute feature samples corresponding to the fourth image samples; obtaining a generalized aesthetic evaluation prediction result based on the image aesthetic attribute feature samples through the generalized aesthetic evaluation network; and training the generalized aesthetic evaluation network based on the generalized aesthetic evaluation prediction result and the aesthetic evaluation score probability distribution training label.

[0143] Among them, the fourth image sample set can specifically adopt the Flickr-AES (Flickr Images with Aesthetics Annotation Dataset) image dataset, which is an image dataset in which 173 sample users conduct aesthetic evaluations on 35,263 images. The aesthetic evaluation scores of 5 sample users are collected for each image, and the aesthetic evaluation scores are between 1 and 5 points. For one of the fourth image samples, an aesthetic evaluation score probability distribution is generated based on the aesthetic evaluation scores of each sample user, and the sum of the aesthetic evaluation score probabilities is 1. In one embodiment, the computer device uses a fully connected network to construct a generalized aesthetic evaluation network. In order to make the sum of the aesthetic evaluation score probabilities 1, the Sigmoid function can be used as the activation function of the generalized aesthetic evaluation network.

[0144] In one embodiment, the step of extracting an image aesthetic attribute feature sample corresponding to the fourth image sample includes: obtaining an image aesthetic attribute feature extraction network; inputting the fourth image sample into the image aesthetic attribute feature extraction network, and performing feature extraction on the fourth image sample through the image aesthetic attribute feature extraction network to obtain an image aesthetic attribute feature sample.

[0145] Specifically, the computer device inputs the fourth image sample into the image aesthetic attribute feature extraction network, and extracts the image aesthetic attribute feature sample corresponding to the fourth image sample through the image aesthetic attribute feature extraction network. Then, the computer device inputs the image aesthetic attribute feature sample into the generalized aesthetic evaluation network, and obtains the aesthetic evaluation score probability distribution prediction result through the generalized aesthetic evaluation network. Then, the computer device constructs a loss function based on the difference between the aesthetic evaluation score probability distribution prediction result and the aesthetic evaluation score probability distribution training label, such as constructing a loss function through the Euclidean distance between the aesthetic evaluation score probability distribution prediction result and the aesthetic evaluation score probability distribution training label; then, the computer device trains the generalized aesthetic evaluation network in the direction of minimizing the loss function.

[0146] The loss function of the generalized aesthetic evaluation network is specifically expressed by the following formula:

[0147]

[0148] Among them, L3 is the loss function of the generalized aesthetic evaluation network; represents the probability of the aesthetic evaluation score being l in the training label of the aesthetic evaluation score probability distribution; represents the probability of the aesthetic evaluation score being 1 in the prediction result of the probability distribution of the aesthetic evaluation score; N is the number of samples of the fourth image.

[0149] It can be understood that by taking the weighted sum of the prediction result of the probability distribution of the aesthetic evaluation score and the corresponding aesthetic evaluation score, a generalized aesthetic evaluation result can be obtained.

[0150] In this embodiment, the probability distribution of aesthetic evaluation scores is used to construct training labels, so that the prediction of aesthetic evaluation scores by the generalized aesthetic evaluation network is more in line with the actual situation, thereby improving the prediction accuracy of the generalized aesthetic evaluation network. In addition, for the personalized aesthetic evaluation correction network, a training method for a personalized aesthetic evaluation correction network is provided below.

[0151] In one embodiment, the training steps of the personalized aesthetic evaluation correction network include: obtaining a third image sample and a personalized aesthetic evaluation correction network corresponding to a sample user identifier; the third image sample has a personalized aesthetic evaluation correction parameter training label corresponding to the sample user identifier; extracting an image aesthetic attribute feature sample corresponding to the third image sample; obtaining a user personality feature sample corresponding to the sample user identifier; obtaining a personalized aesthetic evaluation correction parameter prediction result corresponding to the sample user identifier through the personalized aesthetic evaluation correction network based on the fusion result of the image aesthetic attribute feature sample and the user personality feature sample; and training the personalized aesthetic evaluation correction network based on the personalized aesthetic evaluation correction parameter prediction result and the personalized aesthetic evaluation correction parameter training label.

[0152] Among them, the third image sample has a personalized aesthetic evaluation correction parameter training label corresponding to the sample user identifier. The third image sample can specifically come from the Flickr-AES image dataset. The number of images aesthetically evaluated by each sample user is between 105 and 171. The aesthetic evaluation scores of 5 sample users are collected for each image, and the aesthetic evaluation scores range from 1 to 5.

[0153] In one embodiment, the computer device first constructs a personalized aesthetic evaluation correction network composed of a fully connected network, and then uses the third image sample set and personalized aesthetic evaluation correction parameter training labels to train the personalized aesthetic evaluation correction network.

[0154] In one embodiment, the third image sample corresponds to more than one sample user identifier; the step of obtaining the personalized aesthetic evaluation correction parameter training label includes: obtaining the personalized aesthetic evaluation result samples of each sample user identifier corresponding to the third image sample; obtaining the personalized aesthetic evaluation mean sample based on each personalized aesthetic evaluation result sample; obtaining the personalized aesthetic evaluation correction parameter sample corresponding to each sample user identifier based on each personalized aesthetic evaluation result sample and the personalized aesthetic evaluation mean sample, and using each personalized aesthetic evaluation correction parameter sample as the personalized aesthetic evaluation correction parameter training label corresponding to the third image sample and each sample user identifier.

[0155] Specifically, since the third image sample has an aesthetic evaluation score corresponding to a sample user identifier, for one of the third image samples, the mean aesthetic evaluation score corresponding to the third image sample is obtained, and for one of the sample user identifiers, the difference between the corresponding aesthetic evaluation score and the mean aesthetic evaluation score is the personalized aesthetic evaluation correction parameter of the sample user identifier.

[0156] Specifically, the computer device inputs the third image sample into the image aesthetic attribute feature extraction network, and extracts features from the third image sample through the image aesthetic attribute feature extraction network to obtain an image aesthetic attribute feature sample. Then, the computer device obtains a user personality feature sample corresponding to the sample user identifier. Then, the computer device obtains a personalized aesthetic evaluation correction parameter prediction result corresponding to the sample user identifier through the personalized aesthetic evaluation correction network based on the fusion result of the image aesthetic attribute feature sample and the user personality feature sample. Then, the computer device constructs a loss function based on the difference between the personalized aesthetic evaluation correction parameter prediction result and the personalized aesthetic evaluation correction parameter training label, and trains the personalized aesthetic evaluation correction network in the direction of minimizing the loss function.

[0157] The loss function of the personalized aesthetic evaluation correction network is specifically expressed by the following formula:

[0158]

[0159] Among them, L4 is the loss function of the personalized aesthetic evaluation correction network; e n Correcting parameter training labels for personalized aesthetic evaluation; f n is the prediction result of the correction parameter of personalized aesthetic evaluation; N is the number of third image samples.

[0160] In this embodiment, the correction parameters of personalized aesthetics relative to popular aesthetics are modeled using the aesthetic personality traits of individual users and the aesthetic quality of images, which also conforms to the objective fact that users with different general personality traits have aesthetic differences.

[0161] The present application relates to user personality characteristic samples, and a method for obtaining user personality characteristic samples corresponding to a sample user identifier is provided below.

[0162] In one embodiment, obtaining a user personality characteristic sample corresponding to a sample user identifier includes: obtaining a preferred user personality characteristic sample corresponding to a third image sample; obtaining a personalized aesthetic evaluation result sample corresponding to the sample user identifier; and obtaining a user personality characteristic sample corresponding to the sample user identifier based on the personalized aesthetic evaluation result sample and the preferred user personality characteristic sample.

[0163] In one embodiment, the sample user identifier corresponds to at least two third image samples; the personalized aesthetic evaluation result sample is a personalized aesthetic evaluation score sample; based on the personalized aesthetic evaluation result sample and the preferred user personality characteristic sample, a user personality characteristic sample corresponding to the sample user identifier is obtained, including: obtaining personalized aesthetic evaluation maximum samples and personalized aesthetic evaluation minimum samples of at least two third image samples; for one of the third image samples, based on the personalized aesthetic evaluation score sample, the personalized aesthetic evaluation maximum sample and the personalized aesthetic evaluation minimum sample corresponding to the sample user identifier, the preference degree of the sample user identifier for the third image sample is obtained; based on the preference degree of the sample user identifier for each third image sample, and the preferred user personality characteristic sample corresponding to each third image sample, the user personality characteristic sample corresponding to the sample user identifier is obtained.

[0164] In one embodiment, there are multiple characteristic dimensions of the preferred user personality characteristic samples; based on the preference degree and the preferred user personality characteristic samples, a user personality characteristic sample corresponding to the sample user identifier is obtained, including: according to the preference degree, adjusting the characteristic values ​​of the preferred user personality characteristic samples in each dimension to obtain the user personality characteristic sample corresponding to the sample user identifier.

[0165] The process of obtaining the user personality characteristic sample corresponding to the sample user identification may specifically refer to the method of obtaining the user personality characteristic corresponding to the target user identification, which will not be described in detail here.

[0166] This application aims to combine the user's subjective aesthetics and the objective aesthetic quality of the image to perform personalized aesthetic evaluation of the image, and the user's personality is an important factor affecting the user's aesthetic preferences. Therefore, it is conceived to introduce reference to user personality-related parameters in the personalized aesthetic evaluation task.

[0167] Currently, there are image sample sets that annotate both the personality traits and aesthetic evaluation scores of preferred users, such as the PsychoFlickr image dataset. The PsychoFlickr image dataset includes the preferred images of 300 sample users, with each sample user having 200 preferred images, and the sample users annotate the preferred images with aesthetic evaluation scores. The general personality traits of each sample user are described by the Big Five personality traits (openness, responsibility, extraversion, agreeableness, and neuroticism), and the general personality traits of each sample user are determined by the BFI-10 (10-Item Big Five Inventory) psychological test.

[0168] First, for sample users, the aesthetic evaluation scores of sample users for image samples are related to their own scoring standards. For example, two sample users may have the same aesthetic feelings for the same image sample, but there may be differences in the aesthetic evaluation scores. Therefore, the aesthetic evaluation scores cannot actually accurately reflect the aesthetic feelings of different sample users. In this application, for one of the sample users, the preference of the sample user for each image sample is determined based on the aesthetic evaluation scores corresponding to more than one image samples corresponding to the sample user. Since the preference eliminates the differences between the scoring standards of different sample users, the preference can accurately reflect the aesthetic feelings of different sample users.

[0169] Secondly, psychological tests also have a certain degree of subjectivity. By updating the general personality traits of sample users through the degree of preference, the updated aesthetic personality traits are associated with the user's aesthetic feelings, which reduces the subjective error of psychological tests and improves the accuracy of user personality traits at the level of aesthetic feelings.

[0170] The present application also provides an application scenario, which applies the above-mentioned image processing method. The application scenario can be an image recommendation scenario, an image enhancement scenario, an image retrieval scenario, a personal album management scenario, etc. Figure 8 , the application of the image processing method in this application scenario is as follows:

[0171] Step 802: Obtain the image to be processed and the target user identifier.

[0172] Step 804, obtaining an image aesthetic attribute feature extraction network, inputting the image to be processed into the image aesthetic attribute feature extraction network, and outputting the image aesthetic attribute features through the image aesthetic attribute feature extraction network.

[0173] Step 806, obtaining a generalized aesthetic evaluation network, inputting the image aesthetic attribute features into the generalized aesthetic evaluation network, and outputting a generalized aesthetic evaluation result through the generalized aesthetic evaluation network.

[0174] Step 808, obtaining a personalized aesthetic evaluation correction network, and obtaining personalized aesthetic evaluation correction parameters through the personalized aesthetic evaluation correction network according to the fusion result of the image aesthetic attribute characteristics and the user's personality characteristics.

[0175] Among them, the method for obtaining user personality characteristics is as follows: the computer device obtains more than one reference image corresponding to the target user identifier, and each reference image is marked with a personalized aesthetic evaluation score corresponding to the target user identifier; then, the computer device obtains a preferred user personality feature extraction network, and inputs more than one reference image into the preferred user personality feature extraction network respectively, and obtains the preferred user personality characteristics corresponding to each of the more than one reference images through the preferred user personality feature extraction network; then, the computer device obtains the maximum personalized aesthetic evaluation value and the minimum personalized aesthetic evaluation value corresponding to each reference image and the target user identifier; then, for one of the reference images, the computer device obtains the degree of preference of the target user identifier for the reference image based on the personalized aesthetic evaluation score, the maximum personalized aesthetic evaluation value and the minimum personalized aesthetic evaluation value corresponding to the target user identifier; then, the computer device obtains the user personality characteristics corresponding to the target user identifier based on the degree of preference of the target user identifier for each reference image and the preferred user personality characteristics corresponding to each reference image.

[0176] Step 810 : determining a personalized aesthetic evaluation result corresponding to the target user identifier based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter.

[0177] In one embodiment, referring to FIG. 7( a ), FIG. 7( a ) is a training diagram of an image aesthetic attribute feature extraction network in one embodiment. For the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network, a first image sample 702 is input into the image aesthetic attribute feature extraction network, and the image aesthetic attribute feature prediction result 704 is output through the image aesthetic attribute feature extraction network. A second image sample 706 is input into the preferred user personality feature extraction network, and the preferred user personality feature prediction result 708 is output through the preferred user personality feature extraction network. A first loss function is constructed based on the difference between the image aesthetic attribute feature prediction result 704 and the image aesthetic attribute feature training label, and a second loss function is constructed based on the difference between the preferred user personality feature prediction result 708 and the preferred user personality feature training label. A first network gradient parameter is obtained by minimizing the first loss function, and a second network gradient parameter is obtained by minimizing the second loss function. The image aesthetic attribute feature extraction network and the preferred user personality feature extraction network are jointly trained by combining the first network gradient parameter and the second network gradient parameter.

[0178] In one embodiment, referring to FIG. 7( b ), FIG. 7( b ) is a schematic diagram of training a generalized aesthetic evaluation network in one embodiment. The fourth image sample 710 is input into the image aesthetic attribute feature extraction network, and the image aesthetic attribute feature sample 712 is output through the image aesthetic attribute feature extraction network. Next, the image aesthetic attribute feature sample 712 is input into the generalized aesthetic evaluation network, and the aesthetic evaluation score probability distribution prediction result 714 is obtained through the generalized aesthetic evaluation network. Next, a loss function is constructed based on the difference between the aesthetic evaluation score probability distribution prediction result 714 and the aesthetic evaluation score probability distribution training label, and the generalized aesthetic evaluation network is trained in the direction of minimizing the loss function.

[0179] In one embodiment, referring to FIG. 7( c ), FIG. 7( c ) is a training diagram of a personalized aesthetic evaluation correction network in one embodiment. The third image sample 716 is input into the image aesthetic attribute feature extraction network, and the image aesthetic attribute feature sample 718 is output through the image aesthetic attribute feature extraction network. Next, the third image sample 716 is input into the preferred user personality feature extraction network, and the preferred user personality feature sample 720 is output through the preferred user personality feature extraction network, and the user personality feature sample 722 corresponding to the sample user identifier is obtained based on the preferred user personality feature sample 720. Next, through the personalized aesthetic evaluation correction network, according to the fusion result of the image aesthetic attribute feature sample 718 and the user personality feature sample 722, the personalized aesthetic evaluation correction parameter prediction result corresponding to the sample user identifier is obtained. Next, a loss function is constructed based on the difference between the personalized aesthetic evaluation correction parameter prediction result and the personalized aesthetic evaluation correction parameter training label, and the personalized aesthetic evaluation correction network is trained in the direction of minimizing the loss function.

[0180] It should be understood that although Figure 2 , Figure 5 , Figure 8 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 , Figure 5 , Figure 8 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0181] In one embodiment, Fig. 9As shown, an image processing device is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: an acquisition module 902, an extraction module 904, a universal aesthetic evaluation module 906, a correction parameter determination module 908 and a personalized aesthetic evaluation module 910, wherein:

[0182] An acquisition module 902 is used to acquire an image to be processed and a target user identifier;

[0183] An extraction module 904 is used to extract image aesthetic attribute features corresponding to the image to be processed;

[0184] A universal aesthetic evaluation module 906 is used to determine a universal aesthetic evaluation result of the image to be processed according to the image aesthetic attribute characteristics;

[0185] A correction parameter determination module 908 is used to determine a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to the image aesthetic attribute characteristics and the user personality characteristics corresponding to the target user identifier;

[0186] The personalized aesthetic evaluation module 910 is used to determine the personalized aesthetic evaluation result corresponding to the target user identifier based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter.

[0187] In one embodiment, the extraction module 904 is further used to: obtain an image aesthetic attribute feature extraction network; input the image to be processed into the image aesthetic attribute feature extraction network, and output the image aesthetic attribute features through the image aesthetic attribute feature extraction network.

[0188] In one embodiment, an image aesthetic attribute feature extraction network and a preferred user personality feature extraction network are jointly trained; the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network share a basic subnetwork and respectively include corresponding output subnetworks; the image processing device further includes a training module, the training module is used to: obtain a first image sample set, a second image sample set, an image aesthetic attribute feature extraction network and a preferred user personality feature extraction network; each first image sample in the first image sample set has an image aesthetic attribute feature training label, and each second image sample in the second image sample set has a preferred user personality feature training label; the first image sample is input into the image aesthetic attribute feature extraction network, and the image aesthetic attribute feature extraction network is obtained by the image aesthetic attribute feature extraction network. The basic sub-network of the image aesthetic attribute feature extraction network extracts features from the first image sample, and outputs the image aesthetic attribute feature prediction result through the output sub-network of the image aesthetic attribute feature extraction network; the second image sample is input into the preference user personality feature extraction network, and the basic sub-network of the preference user personality feature extraction network extracts features from the second image sample, and outputs the preference user personality feature prediction result through the output sub-network of the preference user personality feature extraction network; based on the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, as well as the preference user personality feature prediction result and the preference user personality feature training label, the image aesthetic attribute feature extraction network and the preference user personality feature extraction network are trained together.

[0189] In one embodiment, the training module is also used to: construct a first loss function based on the difference between the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, and construct a second loss function based on the difference between the preferred user personality feature prediction result and the preferred user personality feature training label; obtain a first network gradient parameter by minimizing the first loss function, and obtain a second network gradient parameter by minimizing the second loss function; combine the first network gradient parameter and the second network gradient parameter to jointly train the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network.

[0190] In one embodiment, the correction parameter determination module 908 is further used to: obtain a personalized aesthetic evaluation correction network; and obtain personalized aesthetic evaluation correction parameters according to the fusion result of the image aesthetic attribute characteristics and the user personality characteristics through the personalized aesthetic evaluation correction network.

[0191] In one embodiment, the training module is also used to: obtain a third image sample and a personalized aesthetic evaluation correction network corresponding to the sample user identifier; the third image sample has a personalized aesthetic evaluation correction parameter training label corresponding to the sample user identifier; extract the image aesthetic attribute feature sample corresponding to the third image sample; obtain the user personality feature sample corresponding to the sample user identifier; through the personalized aesthetic evaluation correction network, according to the fusion result of the image aesthetic attribute feature sample and the user personality feature sample, obtain the personalized aesthetic evaluation correction parameter prediction result corresponding to the sample user identifier; based on the personalized aesthetic evaluation correction parameter prediction result and the personalized aesthetic evaluation correction parameter training label, train the personalized aesthetic evaluation correction network.

[0192] In one embodiment, the third image sample corresponds to more than one sample user identifier; the training module is also used to: obtain personalized aesthetic evaluation result samples of each sample user identifier corresponding to the third image sample; obtain personalized aesthetic evaluation mean samples based on each personalized aesthetic evaluation result sample; obtain personalized aesthetic evaluation correction parameter samples corresponding to each sample user identifier based on each personalized aesthetic evaluation result sample and the personalized aesthetic evaluation mean sample, and use each personalized aesthetic evaluation correction parameter sample as a personalized aesthetic evaluation correction parameter training label corresponding to the third image sample and each sample user identifier.

[0193] In one embodiment, the training module is also used to: obtain a preferred user personality characteristic sample corresponding to the third image sample; obtain a personalized aesthetic evaluation result sample corresponding to the sample user identifier; and obtain a user personality characteristic sample corresponding to the sample user identifier based on the personalized aesthetic evaluation result sample and the preferred user personality characteristic sample.

[0194] In one embodiment, the training module is also used to: obtain an image aesthetic attribute feature extraction network and a preferred user personality feature extraction network; input the third image sample into the image aesthetic attribute feature extraction network, perform feature extraction on the third image sample through the basic sub-network of the image aesthetic attribute feature extraction network, and output the image aesthetic attribute feature sample through the output sub-network of the image aesthetic attribute feature extraction network; input the third image sample into the preferred user personality feature extraction network, perform feature extraction on the third image sample through the basic sub-network of the preferred user personality feature extraction network, and output the preferred user personality feature sample through the output sub-network of the preferred user personality feature extraction network.

[0195] In one embodiment, the sample user identifier corresponds to at least two third image samples; the personalized aesthetic evaluation result sample is a personalized aesthetic evaluation score sample; the training module is also used to: obtain personalized aesthetic evaluation maximum samples and personalized aesthetic evaluation minimum samples of at least two third image samples; for one of the third image samples, based on the personalized aesthetic evaluation score sample, the personalized aesthetic evaluation maximum sample and the personalized aesthetic evaluation minimum sample corresponding to the sample user identifier, obtain the preference degree of the sample user identifier for the third image sample; based on the preference degree of the sample user identifier for each third image sample, and the preferred user personality feature samples corresponding to each third image sample, obtain the user personality feature sample corresponding to the sample user identifier.

[0196] In one embodiment, the characteristic dimensions of the preferred user personality characteristic samples are multiple; the training module is also used to: adjust the characteristic values ​​of the preferred user personality characteristic samples in each dimension according to the degree of preference, and obtain the user personality characteristic samples corresponding to the sample user identification.

[0197] In one embodiment, there are multiple images to be processed; the personalized aesthetic evaluation result is a personalized aesthetic evaluation score; the image processing device also includes an application module, which is used to: obtain the personalized aesthetic evaluation score of the target user identifier for each image to be processed; select a preset number of target images from each image to be processed according to the personalized aesthetic evaluation score; and output the target images to the terminal where the target user identifier is located.

[0198] For the specific definition of the image processing device, please refer to the definition of the image processing method above, which will not be repeated here. Each module in the above image processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0199] In the above-mentioned image processing device, since the universal aesthetic evaluation result is obtained by evaluating the aesthetic characteristics of the image itself, it reflects the aesthetic quality of the image at the objective level, and the universal aesthetic evaluation result is corrected according to the aesthetic deviation corresponding to the user's personality characteristics, and the corrected personalized aesthetic evaluation result is close to the aesthetic evaluation result obtained by the user based on his or her own aesthetics, that is, the image is personalized aesthetically evaluated in combination with the user's subjective aesthetics and the objective aesthetic quality of the image, so that the personalized aesthetic evaluation result conforms to the aesthetic preferences of individual users, thereby improving the accuracy of the personalized aesthetic evaluation result.

[0200] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image processing data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an image processing method is implemented.

[0201] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0202] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0203] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0204] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0205] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0206] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0207] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An image processing method, characterized in that: The method comprises: Obtain the image to be processed and the target user ID; Extracting image aesthetic attribute features corresponding to the image to be processed; Determining a generalized aesthetic evaluation result of the image to be processed according to the image aesthetic attribute characteristics; Determining, through a personalized aesthetic evaluation correction network, a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to a fusion result of the image aesthetic attribute feature and the user personality feature corresponding to the target user identifier; the user personality feature is data reflecting the user's personality characteristics; Based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter, a personalized aesthetic evaluation result corresponding to the target user identifier is determined.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a plurality of reference images corresponding to the target user identifier, each reference image being marked with a personalized aesthetic evaluation score corresponding to the target user identifier; Inputting the plurality of reference images into a preferred user personality feature extraction network respectively, and obtaining the preferred user personality features corresponding to each of the plurality of reference images through the preferred user personality feature extraction network; the preferred user personality feature is a personality feature of the reference image corresponding to the preferred user personality feature; Obtaining a maximum personalized aesthetic evaluation value and a minimum personalized aesthetic evaluation value corresponding to each of the reference images and the target user identifier; For each reference image, based on the personalized aesthetic evaluation score corresponding to the target user identifier, the personalized aesthetic evaluation maximum value, and the personalized aesthetic evaluation minimum value, obtaining a preference degree of the target user identifier for the reference image; Based on the preference degree of the target user identifier to each reference image and the preferred user personality characteristics corresponding to each reference image, the user personality characteristics corresponding to the target user identifier are obtained.

3. The method according to claim 2, characterized in that The step of extracting the image aesthetic attribute features corresponding to the image to be processed includes: Obtaining an image aesthetic attribute feature extraction network; The image to be processed is input into the image aesthetic attribute feature extraction network, and the image aesthetic attribute feature is output through the image aesthetic attribute feature extraction network.

4. The method according to claim 3, characterized in that The image aesthetic attribute feature extraction network and the preferred user personality feature extraction network are trained together; the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network share a basic subnetwork and each includes a corresponding output subnetwork; The steps of jointly training the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network include: Obtain a first image sample set, a second image sample set, the image aesthetic attribute feature extraction network, and the preferred user personality feature extraction network; each first image sample in the first image sample set has an image aesthetic attribute feature training label, and each second image sample in the second image sample set has a preferred user personality feature training label; Inputting the first image sample into the image aesthetic attribute feature extraction network, performing feature extraction on the first image sample through a basic sub-network of the image aesthetic attribute feature extraction network, and outputting an image aesthetic attribute feature prediction result through an output sub-network of the image aesthetic attribute feature extraction network; Inputting the second image sample into the preferred user personality feature extraction network, performing feature extraction on the second image sample through the basic sub-network of the preferred user personality feature extraction network, and outputting the preferred user personality feature prediction result through the output sub-network of the preferred user personality feature extraction network; Based on the image aesthetic attribute feature prediction results and the image aesthetic attribute feature training labels, as well as the preferred user personality feature prediction results and the preferred user personality feature training labels, the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network are jointly trained.

5. The method according to claim 4, characterized in that The method of jointly training the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network based on the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, and the preferred user personality feature prediction result and the preferred user personality feature training label, comprises: constructing a first loss function based on the difference between the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, and constructing a second loss function based on the difference between the preferred user personality feature prediction result and the preferred user personality feature training label; Obtain a first network gradient parameter by minimizing the first loss function, and obtain a second network gradient parameter by minimizing the second loss function; The image aesthetic attribute feature extraction network and the preferred user personality feature extraction network are jointly trained by combining the first network gradient parameter and the second network gradient parameter.

6. The method according to claim 2, characterized in that The training steps of the personalized aesthetic evaluation correction network include: Acquire a third image sample corresponding to the sample user identifier and the personalized aesthetic evaluation correction network; the third image sample has a personalized aesthetic evaluation correction parameter training label corresponding to the sample user identifier; extracting an image aesthetic attribute feature sample corresponding to the third image sample; Obtaining a user personality characteristic sample corresponding to the sample user identifier; Obtaining a prediction result of a personalized aesthetic evaluation correction parameter corresponding to the sample user identifier through the personalized aesthetic evaluation correction network according to a fusion result of the image aesthetic attribute feature sample and the user personality feature sample; The personalized aesthetic evaluation correction network is trained based on the personalized aesthetic evaluation correction parameter prediction result and the personalized aesthetic evaluation correction parameter training label.

7. The method according to claim 6, characterized in that The third image sample corresponds to more than one sample user identification; The step of obtaining the personalized aesthetic evaluation correction parameter training label comprises: Obtaining a personalized aesthetic evaluation result sample of each sample user identifier corresponding to the third image sample; Obtaining a personalized aesthetic evaluation mean sample based on each of the personalized aesthetic evaluation result samples; Based on each of the personalized aesthetic evaluation result samples and the personalized aesthetic evaluation mean samples, a personalized aesthetic evaluation correction parameter sample corresponding to each of the sample user identifiers is obtained, and each of the personalized aesthetic evaluation correction parameter samples is used as a personalized aesthetic evaluation correction parameter training label corresponding to the third image sample and each of the sample user identifiers.

8. The method according to claim 6, characterized in that The step of obtaining a user personality characteristic sample corresponding to the sample user identifier includes: Obtaining a preferred user personality characteristic sample corresponding to the third image sample; Obtaining a personalized aesthetic evaluation result sample corresponding to the sample user identifier; Based on the personalized aesthetic evaluation result sample and the preferred user personality characteristic sample, a user personality characteristic sample corresponding to the sample user identifier is obtained.

9. The method according to claim 8, characterized in that The steps of acquiring the image aesthetic attribute feature sample and the preferred user personality feature sample corresponding to the third image sample include: Obtain image aesthetic attribute feature extraction network and preference user personality feature extraction network; Inputting the third image sample into the image aesthetic attribute feature extraction network, performing feature extraction on the third image sample through a basic sub-network of the image aesthetic attribute feature extraction network, and outputting the image aesthetic attribute feature sample through an output sub-network of the image aesthetic attribute feature extraction network; The third image sample is input into the preferred user personality feature extraction network, features of the third image sample are extracted through the basic sub-network of the preferred user personality feature extraction network, and the preferred user personality feature sample is output through the output sub-network of the preferred user personality feature extraction network.

10. The method according to claim 8, characterized in that The sample user identifier corresponds to at least two third image samples; the personalized aesthetic evaluation result sample is a personalized aesthetic evaluation score sample; The obtaining of the user personality characteristic sample corresponding to the sample user identifier based on the personalized aesthetic evaluation result sample and the preferred user personality characteristic sample includes: Obtaining a personalized aesthetic evaluation maximum value sample and a personalized aesthetic evaluation minimum value sample of at least two of the third image samples; For one of the third image samples, based on the personalized aesthetic evaluation score sample corresponding to the sample user identifier, the personalized aesthetic evaluation maximum sample, and the personalized aesthetic evaluation minimum sample, obtaining the preference degree of the sample user identifier for the third image sample; Based on the preference degree of the sample user identifier to each of the third image samples and the preferred user personality characteristic samples corresponding to each of the third image samples, the user personality characteristic samples corresponding to the sample user identifier are obtained.

11. The method according to claim 10, characterized in that The characteristic dimensions of the preferred user personality characteristic samples are multiple; The obtaining of the user personality characteristic sample corresponding to the sample user identifier based on the preference degree and the preferred user personality characteristic sample includes: According to the degree of preference, the characteristic values ​​of the preferred user personality characteristic samples in various dimensions are adjusted to obtain user personality characteristic samples corresponding to the sample user identifiers.

12. The method according to claim 1, characterized in that The number of images to be processed is multiple; the personalized aesthetic evaluation result is a personalized aesthetic evaluation score; The method further comprises: Obtaining a personalized aesthetic evaluation score of each of the to-be-processed images by the target user identifier; According to the personalized aesthetic evaluation score, selecting a preset number of target images from each of the images to be processed; The target image is output to the terminal where the target user identifier is located.

13. An image processing device, characterized in that: The device comprises: An acquisition module, used to acquire the image to be processed and the target user ID; An extraction module, used for extracting image aesthetic attribute features corresponding to the image to be processed; A universal aesthetic evaluation module, used to determine a universal aesthetic evaluation result of the image to be processed according to the image aesthetic attribute characteristics; a correction parameter determination module, configured to determine, through a personalized aesthetic evaluation correction network, a personalized aesthetic evaluation correction parameter corresponding to the target user identifier according to a fusion result of the image aesthetic attribute feature and the user personality feature corresponding to the target user identifier; the user personality feature is data reflecting the user's personality characteristics; The personalized aesthetic evaluation module is used to determine the personalized aesthetic evaluation result corresponding to the target user identifier based on the universal aesthetic evaluation result and the personalized aesthetic evaluation correction parameter.

14. The device according to claim 13, characterized in that The device also includes: The user personality feature acquisition module is used to acquire multiple reference images corresponding to the target user identifier, each reference image is marked with a personalized aesthetic evaluation score corresponding to the target user identifier; the multiple reference images are respectively input into a preferred user personality feature extraction network, and the preferred user personality features corresponding to each of the multiple reference images are obtained through the preferred user personality feature extraction network; the preferred user personality feature is a personality feature that prefers the reference image corresponding to the preferred user personality feature; the personalized aesthetic evaluation maximum value and the personalized aesthetic evaluation minimum value corresponding to each of the reference images and the target user identifier are obtained; for each reference image, based on the personalized aesthetic evaluation score corresponding to the target user identifier, the personalized aesthetic evaluation maximum value and the personalized aesthetic evaluation minimum value, the preference degree of the target user identifier for the reference image is obtained; based on the preference degree of the target user identifier for each reference image and the preferred user personality features corresponding to each reference image, the user personality feature corresponding to the target user identifier is obtained.

15. The device according to claim 14, characterized in that The extraction module is also used to obtain an image aesthetic attribute feature extraction network; input the image to be processed into the image aesthetic attribute feature extraction network, and output the image aesthetic attribute feature through the image aesthetic attribute feature extraction network.

16. The device according to claim 15, characterized in that The image aesthetic attribute feature extraction network and the preferred user personality feature extraction network are jointly trained; the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network share a basic subnetwork and respectively include corresponding output subnetworks; the device also includes a training module for acquiring a first image sample set, a second image sample set, the image aesthetic attribute feature extraction network, and the preferred user personality feature extraction network; Each first image sample in the first image sample set has an image aesthetic attribute feature training label, and each second image sample in the second image sample set has a preferred user personality feature training label; Inputting the first image sample into the image aesthetic attribute feature extraction network, performing feature extraction on the first image sample through a basic sub-network of the image aesthetic attribute feature extraction network, and outputting an image aesthetic attribute feature prediction result through an output sub-network of the image aesthetic attribute feature extraction network; The second image sample is input into the preferred user personality feature extraction network, the feature of the second image sample is extracted through the basic sub-network of the preferred user personality feature extraction network, and the preferred user personality feature prediction result is output through the output sub-network of the preferred user personality feature extraction network; based on the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, as well as the preferred user personality feature prediction result and the preferred user personality feature training label, the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network are jointly trained.

17. The device according to claim 16, characterized in that The training module is also used to construct a first loss function based on the difference between the image aesthetic attribute feature prediction result and the image aesthetic attribute feature training label, and to construct a second loss function based on the difference between the preferred user personality feature prediction result and the preferred user personality feature training label; obtain a first network gradient parameter by minimizing the first loss function, and obtain a second network gradient parameter by minimizing the second loss function; and jointly train the image aesthetic attribute feature extraction network and the preferred user personality feature extraction network by combining the first network gradient parameter and the second network gradient parameter.

18. The device according to claim 14, characterized in that The device further comprises a training module for acquiring a third image sample corresponding to the sample user identifier and the personalized aesthetic evaluation correction network; the third image sample has a personalized aesthetic evaluation correction parameter training label corresponding to the sample user identifier; extracting an image aesthetic attribute feature sample corresponding to the third image sample; Obtaining a user personality characteristic sample corresponding to the sample user identifier; Obtaining a prediction result of a personalized aesthetic evaluation correction parameter corresponding to the sample user identifier through the personalized aesthetic evaluation correction network according to a fusion result of the image aesthetic attribute feature sample and the user personality feature sample; The personalized aesthetic evaluation correction network is trained based on the personalized aesthetic evaluation correction parameter prediction result and the personalized aesthetic evaluation correction parameter training label.

19. The device according to claim 18, characterized in that The third image sample corresponds to more than one sample user identifier; the training module is further used to obtain personalized aesthetic evaluation result samples of each sample user identifier corresponding to the third image sample; and obtain personalized aesthetic evaluation mean samples based on each personalized aesthetic evaluation result sample; Based on each of the personalized aesthetic evaluation result samples and the personalized aesthetic evaluation mean samples, a personalized aesthetic evaluation correction parameter sample corresponding to each of the sample user identifiers is obtained, and each of the personalized aesthetic evaluation correction parameter samples is used as a personalized aesthetic evaluation correction parameter training label corresponding to the third image sample and each of the sample user identifiers.

20. The device according to claim 18, characterized in that The training module is also used to obtain a preferred user personality feature sample corresponding to the third image sample; obtain a personalized aesthetic evaluation result sample corresponding to the sample user identifier; and obtain a user personality feature sample corresponding to the sample user identifier based on the personalized aesthetic evaluation result sample and the preferred user personality feature sample.

21. The device according to claim 20, characterized in that The training module is also used to obtain an image aesthetic attribute feature extraction network and a preferred user personality feature extraction network; input the third image sample into the image aesthetic attribute feature extraction network, perform feature extraction on the third image sample through the basic sub-network of the image aesthetic attribute feature extraction network, and output the image aesthetic attribute feature sample through the output sub-network of the image aesthetic attribute feature extraction network; input the third image sample into the preferred user personality feature extraction network, perform feature extraction on the third image sample through the basic sub-network of the preferred user personality feature extraction network, and output the preferred user personality feature sample through the output sub-network of the preferred user personality feature extraction network.

22. The device according to claim 20, characterized in that The sample user identifier corresponds to at least two third image samples; the personalized aesthetic evaluation result sample is a personalized aesthetic evaluation score sample; The training module is further used to obtain personalized aesthetic evaluation maximum samples and personalized aesthetic evaluation minimum samples of at least two of the third image samples; for one of the third image samples, based on the personalized aesthetic evaluation score sample corresponding to the sample user identifier, the personalized aesthetic evaluation maximum sample and the personalized aesthetic evaluation minimum sample, obtain the preference degree of the sample user identifier for the third image sample; Based on the preference degree of the sample user identifier to each of the third image samples and the preferred user personality characteristic samples corresponding to each of the third image samples, the user personality characteristic samples corresponding to the sample user identifier are obtained.

23. The device according to claim 22, characterized in that The characteristic dimensions of the preferred user personality characteristic samples are multiple; The training module is also used to adjust the characteristic values ​​of the preferred user personality characteristic samples in various dimensions according to the preference degree, so as to obtain the user personality characteristic samples corresponding to the sample user identification.

24. The device according to claim 13, characterized in that The number of images to be processed is multiple; the personalized aesthetic evaluation result is a personalized aesthetic evaluation score; The device also includes: The application module is used to obtain the personalized aesthetic evaluation score of the target user identification for each of the images to be processed; select a preset number of target images from each of the images to be processed according to the personalized aesthetic evaluation score; and output the target images to the terminal where the target user identification is located.

25. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

26. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

27. A computer program product comprising computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

Images