Image beautifying method and device based on user portrait

Abstract

The embodiment of the invention provides an image beautifying method and device based on a user portrait. The method comprises the steps of obtaining first user portrait information of a first user, a first product image of a first product and first product information, wherein the first user portrait information comprises information used for reflecting preferences of the first user; and inputting the first user portrait information, the first product image and the first product information into an image generation model to obtain a first beautified image after the first product image is beautified, the first beautified image being used for displaying the first product to the first user in the business. The method can generate a beautified image which better accords with user preferences, and improves the interest of the user in the product.

Description

A method and apparatus for image enhancement based on user profiles Technical Field

[0001] The embodiments in this specification relate to the field of image processing technology, specifically to an image enhancement method and apparatus based on user profiles. Background Technology

[0002] With increasing competition among e-commerce platforms, product images, such as main product images and detail images, serve as crucial entry points for users to obtain product information and make purchasing decisions. Their visual presentation directly impacts user clicks and subsequent conversion rates. Currently, AIGC (Artificial Intelligence Generated Content)-based image enhancement and beautification solutions are widely used for e-commerce product image optimization. These solutions enhance the attractiveness of product images through methods such as background removal and replacement, filter and stylization processing, color and lighting correction, composition optimization, generating or replacing virtual models, and scene compositing, aiming to improve online metrics such as click-through rate (CTR) and conversion rate (CVR).

[0003] However, different user groups exhibit significant differences in aesthetic preferences, compositional tastes, and attention to detail (such as material representation, accessory display, and contextual information). Current mainstream image enhancement solutions largely rely on generic image generation algorithms, applying a one-size-fits-all approach to product images. This lack of dynamic adaptation to individual user preferences leads to the inefficient consumption of high-quality exposure resources and significantly limits the potential for improving CTR / CVR.

[0004] For example, as shown in Figure 1, current mainstream solutions use a single template to enhance product images and distribute the same enhanced image to all user groups. The results often only cater to the preferences of specific demographics. In this case, the style of the enhanced image only aligns with the aesthetics of young men who are tech enthusiasts, and is unsuitable for the aesthetics of middle-aged men who enjoy literature, young women who love fashion, and young women who appreciate home decor. After the campaign, it only achieved a 9% CTR among young men who love technology; the CTR among other user groups did not meet expectations. Summary of the Invention

[0005] This specification provides an image enhancement scheme based on user profiles, which can perform personalized image enhancement processing on product images according to user profiles, generating enhanced images that better meet user preferences.

[0006] In a first aspect, embodiments of this specification provide an image enhancement method based on user profiles, comprising: acquiring first user profile information of a first user, a first product image of a first product, and first product information, wherein the first user profile information includes information reflecting the preferences of the first user; inputting the first user profile information, the first product image, and the first product information into an image generation model to obtain a first enhanced image after enhancing the first product image, wherein the first enhanced image is used to display the first product to the first user in business operations.

[0007] In some embodiments, the step of inputting the first user profile information, the first product image, and the first product information into an image generation model to obtain a first beautified image after beautifying the first product image includes: inputting the first user profile information, the first product image, and the first product information into the image generation model multiple times to obtain multiple candidate images after beautifying the first product image; detecting the candidate images to obtain detection results, the detection results being used to indicate the quality and / or compliance of the candidate images, the compliance being used to indicate the degree to which the candidate images conform to preset rules; and determining the first beautified image to be displayed from the multiple candidate images based on the detection results of each candidate image.

[0008] In some embodiments, obtaining the first user profile information of the first user includes: obtaining the first user profile information sent by the first user's terminal, wherein the first user profile information includes de-identified information reflecting the preferences of the first user.

[0009] In some embodiments, the first user profile information is obtained on the terminal through the following processing: acquiring the attribute information and behavior information of the first user; encrypting and encoding the attribute information and behavior information to obtain the first user profile information.

[0010] In some embodiments, the method further includes: acquiring first display results of multiple first beautified images for multiple first users, the first display results including feedback data of the first users on the first beautified images; constructing a fine-tuning dataset based on the first user profile information, the first product image, the first product information, the first beautified image and the first display results, the fine-tuning dataset including multiple sample data, the sample labels in the sample data being the first beautified image where the first display results achieve a preset effect, and the sample inputs in the sample data including the first user profile information, the first product image and the first product information; and adjusting the model parameters of the image generation model based on the fine-tuning dataset.

[0011] In some embodiments, after adjusting the model parameters of the image generation model based on the fine-tuning dataset, the method further includes: obtaining second user profile information of a second user, a second product image of a second product, and second product information, wherein the second user profile information includes information reflecting the preferences of the second user; inputting the second user profile information, the second product image, and the second product information into the adjusted image generation model to obtain a second beautified image after beautifying the second product image, wherein the second beautified image is used to display the second product to the second user in business; obtaining a second display result of the second beautified image, wherein the second display result includes feedback data from the second user on the second beautified image; determining the performance evaluation result of the adjusted image generation model based on the second display result; and determining the scope of application of the adjusted image generation model based on the performance evaluation result.

[0012] In some embodiments, adjusting the model parameters of the image generation model based on the fine-tuning dataset includes: determining the sample weights corresponding to each sample data based on the display results of the first beautified image in each of the sample data, wherein the sample weights are positively correlated with a preset index in the first display results; beautifying the first product image based on the image generation model for the sample data in the fine-tuning dataset to obtain a predicted image corresponding to the first product image; determining a first loss term corresponding to each sample data based on the difference between the predicted image corresponding to the first product image and the sample labels in each of the sample data; weighting the first loss term corresponding to each sample data based on the sample weights corresponding to each sample data to obtain a first loss function, and adjusting the model parameters of the image generation model based on the first loss function.

[0013] In some embodiments, the feedback data in the display results includes at least one of the following metrics: exposure rate, click-through rate, number of visitors to the details page, add-to-cart rate, conversion rate, dwell time, and share rate.

[0014] Secondly, embodiments of this specification provide an image enhancement device based on user profiles, comprising: an information acquisition module, configured to acquire first user profile information of a first user, a first product image of a first product, and first product information, wherein the first user profile information includes information reflecting the preferences of the first user; and an image generation module, configured to input the first user profile information, the first product image, and the first product information into an image generation model to obtain a first enhanced image after enhancing the first product image, wherein the first enhanced image is used to display the first product to the first user in business operations.

[0015] Thirdly, embodiments of this specification provide a computing device including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in any of the implementations in the first aspect.

[0016] In the solutions provided by the above embodiments in this specification, product images are beautified by combining user profile information and product information to generate beautified images that better match user preferences, thereby increasing user interest in the product and achieving better product placement results. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the various embodiments disclosed in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only a few embodiments disclosed in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 is a schematic diagram of a product image enhancement scheme in the present art, as shown in this specification;

[0019] Figure 2 is a schematic diagram of an image enhancement scheme based on user profile in an embodiment of this specification.

[0020] Figure 3 is a flowchart of an image enhancement method based on user profile in an embodiment of this specification.

[0021] Figure 4 is a schematic diagram of the architecture of an image enhancement method based on user profile in an embodiment of this specification;

[0022] Figure 5 is a schematic diagram of the image enhancement device based on user profile in an embodiment of this specification. Detailed Implementation

[0023] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0024] As mentioned earlier, in e-commerce platforms, the aesthetic appeal of product display images, such as main product images and detail images, directly impacts user click-through rates and conversion rates. To enhance the visual attractiveness of products, it is usually necessary to beautify the original product display images, such as adjusting composition, correcting colors, replacing backgrounds, applying filters, and replacing models, in order to improve click-through rates and conversion rates.

[0025] However, most current product image enhancement functions rely on pre-trained static models or fixed enhancement algorithms. For example, one-click enhancement tools are used to process product images offline in batches and the same version is then distributed to all user groups. This lacks the ability to dynamically adapt to individual user preferences, resulting in the inefficient consumption of high-quality exposure resources and limited improvement of metrics such as CTR and CVR after enhancement.

[0026] Based on this, this specification proposes an image enhancement method based on user profiles, which can personalize product images according to the user profiles of different users, generate beautified images that better meet user preferences, and improve product launch metrics and user satisfaction.

[0027] Figure 2 is a schematic diagram of the image enhancement scheme based on user profiles proposed in the embodiments of this specification. As shown in Figure 2, based on the user profile information of different users (such as middle-aged men who love literature, young men who love technology, young women who love fashion, and young women who love home decor) and product information, the same original product image is subjected to targeted enhancement processing to obtain enhanced images that match the aesthetic preferences of various users. Specifically, for middle-aged men who love literature, retro tones, book elements, or poetic compositions are used to generate an elegant, calm, and more literary enhanced image 1; for young men who love technology, cool tones, geometric lines, and high-contrast design are used to generate a more technological and futuristic enhanced image 2; for young women who love fashion, bright colors, popular elements, and fashionable layouts are used to enhance visual appeal to generate a more trendy and refined enhanced image 3; for young women who love home decor, soft tones, natural materials, and home scene layouts are used to generate a more warm and comfortable enhanced image 4. Each beautified image obtained in this way meets the aesthetic needs of the corresponding user, thereby arousing greater user interest after being displayed to the user and achieving a higher CTR index.

[0028] The implementation steps of the above method are described below with reference to the embodiments.

[0029] Figure 3 is a flowchart illustrating an image enhancement method based on user profiles in an embodiment of this specification. The execution entity of this method can be any device, platform, server, or device cluster with computing and processing capabilities. As shown in Figure 3, the method includes steps 301-302 as follows.

[0030] In step 301, the first user profile information of the first user, the first product image of the first product, and the first product information are obtained.

[0031] In this specification, "first user" can refer to a specific user or a group of users with the same first user profile. For example, the first user can be user A, user B, or users C, D, and E, who are all young women who enjoy home decor.

[0032] The first user profile information includes information reflecting the preferences of the first user, providing contextual basis for the image generation model to judge its aesthetic inclinations, thereby enabling targeted visual style customization when beautifying product images. This specification does not limit the specific information included in the user profile information. For example, user profile information may include the following dimensions: demographic information; interests and hobbies; and behavioral data. Among them, demographic information may include identity, city, educational background, and occupation; interests and hobbies may include tags such as literature, technology, fashion, home decor, music, and gardening; and behavioral data may include browsing and clicking history, purchase records, collections, likes, and other online user interaction records.

[0033] In this specification, product images refer to images that primarily showcase a product, used to demonstrate its appearance, structure, details, or usage scenarios. A product is a tangible object with a recognizable appearance that can be presented visually; for example, it could be a product for sale on an e-commerce platform or a sample displayed on a product promotional website. Product images can be captured by photography equipment or generated through graphic rendering, and typically include the main product image, as well as relevant background information. For instance, product images can be the main product image on an e-commerce platform or promotional website, the main product image in a carousel, or a product detail image in a drop-down page.

[0034] Product information is information related to a product and can be used to describe the product's appearance, shape, function, and effects. For example, product information may include textual information such as product category, color, size, price range, product effects, and brand.

[0035] This embodiment does not limit the specific method of obtaining the first user's profile information. For example, the first user's profile information can be obtained from the personal information and preference settings filled in by the first user on the platform, provided that the user is clearly informed of the purpose of the data and an "agree / decline" option is provided. Alternatively, the first user's profile information can be obtained from the interests, areas of interest, or tags publicly disclosed by the first user on their social media platform, after the first user's authorization. Or, the first user's profile information can be obtained from the product categories browsed, ad clicks, content page dwell time, and other interactive behaviors of the first user on the platform, after the user's consent.

[0036] To further protect user privacy and avoid the risks of data leakage, in some embodiments, de-identified first user profile information can be generated on the first user's terminal firstly, and then the first user profile information sent by the first user's terminal can be obtained on the server or cloud. The first user profile information includes de-identified information reflecting the first user's preferences.

[0037] By desensitizing and encrypting user profile data containing privacy information on the user terminal side, the transmission of the original privacy data in the network is effectively avoided. Thus, while ensuring user privacy and security, the desensitized profile information can be securely applied to the personalized beautification of product images, achieving accurate and efficient visual customization while taking privacy protection into account.

[0038] This embodiment does not limit the specific method of de-identification processing on the terminal side. For example, localized feature calculation, federated learning, and differential privacy can be used to de-identify user profile information. The collection and generation of user profile information are completed locally on the user terminal. During network transmission, the original user profile information, plaintext behavioral data, or other privacy data that can identify an individual is not sent.

[0039] For example, the first user profile information is obtained on the terminal through the following processing: the terminal obtains the attribute information and behavior information of the first user, wherein the attribute information includes demographic information such as geographical location, occupation, user tags and interest dimension information, and the behavior information includes user interaction records such as browsing, clicking, searching, collecting, purchasing and dwell time; subsequently, the attribute information and behavior information are encrypted and encoded on the terminal to obtain the first user profile information.

[0040] Specifically, the attribute and behavioral information is first preprocessed and characterized, such as cleaning, denoising, normalization, discretization, bucketing, and serialization. Then, privacy protection is further carried out by encryption or encoding methods, such as hash mapping, homomorphic / symmetric encryption, tokenization, vectorization, or adding differential privacy noise, to convert the original identifiable information into an irreversible or difficult-to-restore feature representation, thereby obtaining the first user profile information after desensitization.

[0041] For example, a feature encoder can be used to encrypt attribute and behavioral information, outputting user features as the first de-identified user profile. Specifically, the user feature can be a string of numbers, such as a 256-bit or 1024-bit number, which does not involve user privacy.

[0042] In step 302, the first user profile information, the first product image, and the first product information are input into the image generation model to obtain the first beautified image after beautifying the first product image.

[0043] The first enhanced image is used to showcase the first product to the first user in the business process. This embodiment does not limit the specific display scenario. For example, in e-commerce platforms, thumbnails or original images of the enhanced image can be displayed in card format in locations such as homepage recommendations, search results pages, channel pages, event venues, and live streaming rooms to attract users to click and enter the details page. The enhanced image can also be displayed in the main image carousel area at the top of the details page or in the details page drop-down area to highlight selling points, parameters, usage scenarios, and texture. The enhanced image can also be displayed in advertising areas (such as splash screens and webpage pop-ups) on various online platforms to attract users.

[0044] In this embodiment, the image generation model is a neural network model with image processing capabilities. This embodiment does not limit the image generation model used; for example, it can be a convolutional neural network model, an encoder-decoder structure model, a generative adversarial network model, a variational autoencoder-based generative model, a diffusion model-based generative model, a Transformer-based generative model, or a model composed of one or more of the above models. In the process of beautifying product images, the image generation model can implement various image processing methods such as composition adjustment, color correction, background replacement, filter application, model replacement, and placement style adjustment.

[0045] After obtaining the initial beautified image, a customized beautified image can be displayed to the first user, achieving real-time personalized generation at the individual profile level. For the same product, beautified images tailored to the preferences and interests of different users can be generated, further improving the actual click-through rate and traffic conversion rate of product image advertising.

[0046] It should be noted that the automatically generated beautified images have a high degree of randomness, which may lead to inconsistent quality, copyright infringement risks, or other risks that could affect user experience. Therefore, the beautified images can be subjected to quality and / or compliance verification to obtain detection results. Then, in response to the detection results failing to meet preset standards, the first beautified image will not be displayed to the first user in the business process, thus ensuring a good user experience. The detection results indicate the quality and / or compliance of the beautified image, and the compliance level represents the degree to which the beautified image conforms to preset rules. These preset rules can be laws and regulations, platform policies, or ethical guidelines, etc.

[0047] Considering that the first beautified image generated each time may not meet the corresponding quality or compliance requirements, and thus cannot be used to obtain a beautified image, the first product image can be input into the image generation model multiple times to obtain multiple different first beautified images, ensuring that a beautified image that achieves the preset effect can be generated. In some embodiments, firstly, the user profile information, the first product image, and the first product information can be input into the image generation model multiple times to obtain multiple candidate images after beautifying the first product image; then, the multiple candidate images are detected separately to obtain detection results; next, based on the detection results of each candidate image, the first beautified image to be displayed is determined from the multiple candidate images.

[0048] This embodiment does not limit the dimensions of quality and compliance testing. Specifically, the testing of candidate images may include, but is not limited to, one or more of the following dimensions: sharpness detection, face recognition and sensitive content detection, consistency with product images, copyright and watermark recognition, and product integrity detection. Among them, the sharpness detection is used to assess whether the image is blurry, has blocky artifacts, or loses details due to excessive compression, scaling, or format conversion, ensuring that the output image meets the minimum sharpness and visual quality requirements; face recognition and sensitive content detection is used to identify whether the image contains identifiable natural human faces, and combined with privacy protection strategies, to determine whether there is a risk of unauthorized use of other people's portraits; consistency with product images is used to detect whether the beautified image, after enhancement and modification, still expresses the same product and the same key attributes as the original product image, and has not been beautified to the point of distortion, exaggeration, or tampering; copyright and watermark recognition is used to detect whether the image contains embedded third-party brand logos, copyrighted patterns, digital watermarks, or proprietary image features with unique identifiers, to prevent unauthorized commercial use or infringement; and product integrity detection can be used in e-commerce, advertising, and other business scenarios to verify whether the core product in the beautified image is fully presented, has not been cropped, distorted, or replaced, and ensures the authenticity and consistency of the product display.

[0049] This embodiment does not limit the algorithms and models used for detection. The above-mentioned detections can be completed collaboratively based on deep learning models, rule engines, and third-party compliance services, and the detection results can be output. The detection results can include scores for each detection dimension, with higher scores indicating higher quality or compliance. Then, based on the scores in the detection results, multiple candidate results can be sorted and filtered. For example, the candidate image with the highest score or a preset number of candidate images can be displayed as the first beautified image. Alternatively, the candidate image can be judged based on the score to determine whether it meets a preset standard. The preset standard can be a preset threshold or a combination of rules. For example, if the score of any dimension does not reach the preset threshold, the candidate image is determined to not meet the preset standard, and the candidate image is removed from the candidate image set. Then, the candidate images that meet the preset standard are determined as the first beautified images to be displayed.

[0050] In practical applications, image generation models are typically pre-trained using fine-tuned datasets, and then the pre-trained, fixed image generation model is used to enhance product images in online applications. However, user preferences drift over time, and a pre-trained, fixed model struggles to respond to these changing trends. Furthermore, for long-tail SKUs (Stock Keeping Units), i.e., products with low sales volume, low exposure, and limited data, the lack of sufficient historical data makes it difficult for the model to learn their patterns during pre-training. Additionally, when new products appear in the business, the model struggles to generate accurate images because it hasn't learned relevant product knowledge during pre-training. Considering these factors, this specification introduces a mechanism for fine-tuning the model based on online user feedback, building upon the aforementioned embodiments. Figure 4 illustrates the overall architecture of this mechanism.

[0051] The process involves several steps. Module A runs an image generation model to generate multiple candidate images based on the first user profile information, the first product information, and the first product image. Module B performs quality and compliance checks on the generated candidate images to determine the first beautified image to be displayed. After the first beautified image is displayed to the first user in the business logic, in the user interaction layer of Module C, first display results of the multiple beautified images are obtained for each of the first users. These first display results include user feedback data on the first beautified images. Then, in Module D, a fine-tuning dataset is constructed based on the first user profile information, the first product image, the first product information, the first beautified image, and the first display results. This fine-tuning dataset includes multiple sample data sets, where the sample labels represent the first beautified image whose first display results achieve the preset effect. The sample inputs include the first user profile information, the first product image, and the first product information. Finally, in Module E, the model parameters of the image generation model are adjusted based on the fine-tuning dataset.

[0052] Feedback data refers to a series of quantifiable metrics generated by the subsequent behaviors of different first users after they see the first beautified image. For example, feedback data includes at least one of the following metrics: exposure rate, click-through rate, product detail page entry rate, add-to-cart rate, conversion rate, dwell time, and sharing rate. In other examples, feedback data may also include at least one of the following auxiliary metrics: complaint rate, loading speed, and return rate.

[0053] This embodiment does not limit the method of obtaining the first display result of the first beautified image. For example, points can be embedded in the user interface. After the first beautified image is displayed in a preset position or traffic position, user behavior events such as the exposure of the first beautified image, the first user clicking on the area where the beautified image is located, jumping to the product details page, clicking the add to cart button, successful payment, clicking the share button and completing the sharing, as well as calculating the time difference between entering and exiting the product page, can be used to obtain the first display result. Alternatively, the first display result can be obtained through historical behavior data such as platform logs.

[0054] This embodiment does not limit the specific method for constructing the fine-tuning dataset. When the metrics in the first display result meet the preset metric requirements, it indicates that the first display result has achieved the preset effect. The first beautified image conforms to the aesthetics and preferences of the first user and can be used as a high-quality sample to train the model, so that the model can be optimized in the direction of adapting to real user preferences. This embodiment does not limit the preset metric requirements. For example, the preset metric requirements could be a click-through rate of 3% or a page view rate of 10%. In other examples, the construction of the fine-tuning dataset can also refer to auxiliary metrics. When the auxiliary metrics are unqualified, such as a high complaint rate, slow loading speed, or high return rate, the first beautified image will not be used as a sample label.

[0055] When it is determined that the display result of the first beautified image has reached the preset effect, the first beautified image is determined as the sample label in the sample data. This sample label will be used as the ground truth image during model training. The first product image, first product information and first user profile information used when generating the first beautified image will be used as the sample input in the sample data. This sample input will be input into the model during model training.

[0056] This embodiment does not restrict the specific method of adjusting model parameters based on the fine-tuning dataset. For example, it can choose to use global or LoRa tuning methods depending on the size of the fine-tuning dataset, and adjust the model parameters through supervised learning with sample labels as ground truth labels.

[0057] During the model parameter adjustment process, the following processing can be iteratively performed on each sample data in the fine-tuning dataset: The sample data is input into the image generation model for enhancement processing to obtain the predicted image corresponding to the first product image; then, the reconstruction loss corresponding to the sample data is calculated based on the difference between the predicted image and the sample label. This embodiment does not restrict the loss function used for reconstruction loss; for example, L1 (Mean Absolute Error), L2 (Mean Squared Error), or perceptual loss functions can be used. Next, with the goal of minimizing the reconstruction loss, the model parameters of the image generation model are adjusted. For example, through the backpropagation algorithm, the model parameters of the image generation model are updated in each iteration with the goal of minimizing the reconstruction loss, until the reconstruction loss value meets the preset loss value requirement, or the number of iterations reaches the preset number. In this way, the adjusted image generation model will learn the user's preference for enhanced images through user feedback data, resulting in output images that are not only more aesthetically pleasing but also more likely to lead to conversions, such as increased click-through rate and conversion rate. Afterwards, the adjusted image generation model can replace the currently deployed online image generation model to enhance product images.

[0058] In one implementation, sample weights can be introduced when adjusting model parameters to adjust the contribution of sample data from different display results to model training, giving greater weight to samples with higher online conversion rates, thus making the image generation model more inclined to produce styles that better match user preferences. Specifically, firstly, based on the first display result of the first beautified image in each sample data set, the sample weights corresponding to each sample data set can be determined. The sample weights are positively correlated with preset indicators in the first display result, such as click-through rate, add-to-cart rate, or dwell time. For the sample data in the fine-tuning dataset, the first product image is beautified based on the image generation model to obtain the predicted image corresponding to the first product image. Based on the difference between the predicted image corresponding to the first product image and the sample labels in each sample data set, the first loss term corresponding to each sample data set is determined. Based on the sample weights corresponding to each sample data set, the first loss term corresponding to each sample data set is weighted to obtain the first loss function, and the model parameters of the image generation model are adjusted based on the first loss function.

[0059] Considering that the performance improvement of the adjusted image generation model compared to the original model may be limited, or even degraded, in some embodiments, the adjusted model can be evaluated offline or online after it is deployed.

[0060] In one example, test user profile information of test users in the test set, product images and product information of the test product can be obtained. The test user profile information includes information reflecting the preferences of the test users. The test user profile, product image and product information of the test product are input into the adjusted image generation model to beautify the product image to obtain the evaluation image. Then, the performance of the model is evaluated based on the quality, compliance and other factors of the evaluation image.

[0061] In another example, the image generation model fine-tuned in module E is deployed on a small-scale, gray-scale basis to replace the image generation model in module A. Performance evaluation can be performed in module F by monitoring CTR, CVR, complaint rate, and return rate in real time, and the scope of model application can be determined based on the performance evaluation results. Specifically, after the model is deployed, the second user profile information, the second product image, and the second product information can be obtained. The second user profile information includes information reflecting the second user's preferences. This second user profile information, the second product image, and the second product information are input into the adjusted image generation model to obtain a second beautified image, which is used to display the second product to the second user in business operations. The second display result of the second beautified image is obtained, including feedback data from the second user. Based on the second display result, the performance evaluation result of the adjusted image generation model is determined.

[0062] Here, the second user can refer to a specific user or a group of users with the same second user profile information. The second product can be a product of the same category as the first product or a product of a different category. Regarding the second user profile information, the process of acquiring the second product image and information, the process of generating the second beautified image, and the process of acquiring the second display result, please refer to the description in the above embodiments, which will not be repeated here. Based on the various indicators in the second display result, the performance evaluation result of the model can be determined. Then, based on the performance evaluation result, the scope of application of the adjusted image generation model is determined.

[0063] If the performance evaluation results of the adjusted model are better than those of the previously deployed model—for example, if metrics such as click-through rate, add-to-cart rate, and conversion rate are all improved compared to the previously deployed model—the application scope of the adjusted model can be expanded, such as from canary releases to full deployment, or applied to more business scenarios and user groups. If the performance evaluation results of the adjusted model do not meet the preset performance requirements (such as key metrics failing to reach the baseline, exhibiting negative fluctuations, or showing degraded performance in certain user segments) or if the overall metric performance is lower than that of the previously deployed model, then the model should be rolled back to the stable model before the adjustment, triggering a new round of model parameter adjustment. This ensures that model updates are always geared towards improving user experience and business conversion, avoiding online performance degradation due to blind deployment.

[0064] The system shown in Figure 4 can personalize and enhance product images displayed by merchants, generating beautified images with different characteristics for target audiences with varying interests. This enables precise and personalized advertising, effectively improving the performance of product images and overall traffic conversion rates. Furthermore, it can build a fine-tuning dataset based on online feedback on campaign performance (such as click-through rate and add-to-cart rate), allowing for real-time adjustments to the online image generation model. This ensures that the generated beautified images better match the aesthetic preferences of specific user profiles. In addition, the system continuously monitors and evaluates the actual performance of the model after each fine-tuning. If performance degradation or a decline in user experience is detected, timely intervention will be implemented to prevent model rollback from negatively impacting business results.

[0065] Figure 5 is a schematic diagram of the image enhancement device based on user profiles in an embodiment of this specification. This device can be applied to any device, platform, or device cluster with computing and processing capabilities. The device includes:

[0066] The information acquisition module 501 is used to acquire the first user profile information of the first user, the first product image and the first product information of the first product. The first user profile information includes information reflecting the preferences of the first user.

[0067] The image generation module 502 is used to input the first user profile information, the first product image and the first product information into the image generation model to obtain the first beautified image after beautifying the first product image. The first beautified image is used to display the first product to the first user in business.

[0068] In some embodiments, the image generation module 502 is specifically used to input the first user profile information, the first product image, and the first product information into the image generation model multiple times to obtain multiple candidate images after beautifying the first product image; to detect the candidate images and obtain detection results, the detection results being used to indicate the quality and / or compliance of the candidate images, the compliance being used to indicate the degree to which the candidate images conform to preset rules; and based on the detection results of each candidate image, to determine the first beautified image to be displayed from the multiple candidate images.

[0069] In some embodiments, when the information acquisition module 501 is used to acquire the first user profile information of the first user, it is specifically used to acquire the first user profile information sent by the first user's terminal. The first user profile information includes de-identified information reflecting the preferences of the first user.

[0070] In some embodiments, the first user profile information is obtained on the terminal through the following processing: obtaining the attribute information and behavior information of the first user; encrypting and encoding the attribute information and behavior information to obtain the first user profile information.

[0071] In some embodiments, the device further includes a model fine-tuning module (not shown in the figure), used to acquire first display results of multiple first beautified images for multiple first users, the first display results including feedback data of the first users on the first beautified images; construct a fine-tuning dataset based on first user profile information, first product image, first product information, first beautified image and first display results, the fine-tuning dataset including multiple sample data, the sample label in the sample data being the first beautified image where the first display result achieves a preset effect, the sample input in the sample data including first user profile information, first product image and first product information; and adjust the model parameters of the image generation model based on the fine-tuning dataset.

[0072] In some embodiments, the model fine-tuning module is further configured to acquire second user profile information of the second user, second product image of the second product, and second product information, wherein the second user profile information includes information reflecting the preferences of the second user; input the second user profile information, the second product image, and the second product information into the adjusted image generation model to obtain a second beautified image after beautifying the second product image, wherein the second beautified image is used to display the second product to the second user in business; acquire a second display result of the second beautified image, wherein the second display result includes feedback data of the second user on the second beautified image; determine the performance evaluation result of the adjusted image generation model based on the second display result; and determine the scope of application of the adjusted image generation model based on the performance evaluation result.

[0073] In some embodiments, when the model fine-tuning module is used to adjust the model parameters of the image generation model based on the fine-tuning dataset, it is specifically used to determine the sample weights corresponding to each sample data based on the display results of the first beautified image in each sample data, wherein the sample weights are positively correlated with the preset indicators in the first display results; for the sample data in the fine-tuning dataset, the first product image is beautified based on the image generation model to obtain the predicted image corresponding to the first product image; based on the difference between the predicted image corresponding to the first product image and the sample labels in each sample data, the first loss term corresponding to each sample data is determined; based on the sample weights corresponding to each sample data, the first loss term corresponding to each sample data is weighted to obtain the first loss function, and the model parameters of the image generation model are adjusted based on the first loss function.

[0074] In some embodiments, the feedback data in the display results includes at least one of the following metrics: exposure rate, click-through rate, number of visits to the details page, add-to-cart rate, conversion rate, dwell time, and share rate.

[0075] This specification also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computer, it causes the computer to perform the method described in any of the above embodiments.

[0076] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in any of the above embodiments.

[0077] This specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any of the above embodiments.

[0078] Those skilled in the art will recognize that the functions described in the various embodiments disclosed in this specification in one or more of the examples above can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0079] It is understood that before or during the use of the technical solutions of the various embodiments in this specification, the user will be informed of the type, scope of use, and usage scenarios of the personal information involved in accordance with relevant laws and regulations, and the user's authorization will be obtained.

[0080] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to choose, based on the prompt message, whether to provide personal information to the software or hardware such as the electronic device, application program, server, or storage medium performing the operation of the technical solution described in this specification.

[0081] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0082] It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation method of this specification. Other methods that comply with relevant laws and regulations may also be applied to the implementation method of this specification.

[0083] In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0084] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of the multiple embodiments disclosed in this specification. It should be understood that the above descriptions are merely specific implementations of the multiple embodiments disclosed in this specification and are not intended to limit the protection scope of the multiple embodiments disclosed in this specification. Any modifications, equivalent substitutions, improvements, etc., made based on the technical solutions of the multiple embodiments disclosed in this specification should be included within the protection scope of the multiple embodiments disclosed in this specification.

Claims

1. An image enhancement method based on user profiles, the method comprising: Obtain first user profile information of a first user, first product image of a first product, and first product information of a first product. The first user profile information includes information reflecting the preferences of the first user. Input the first user profile information, the first product image, and the first product information into an image generation model to obtain a first beautified image after beautifying the first product image. The first beautified image is used to display the first product to the first user in business operations.

2. The method according to claim 1, wherein, The step of inputting the first user profile information, the first product image, and the first product information into an image generation model to obtain a first beautified image after beautifying the first product image includes: inputting the first user profile information, the first product image, and the first product information into the image generation model multiple times to obtain multiple candidate images after beautifying the first product image; detecting the candidate images to obtain detection results, the detection results being used to indicate the quality and / or compliance of the candidate images, the compliance being used to indicate the degree to which the candidate images conform to preset rules; and determining the first beautified image to be displayed from the multiple candidate images based on the detection results of each candidate image.

3. The method according to claim 1, wherein, The step of obtaining the first user profile information of the first user includes: obtaining the first user profile information sent by the first user's terminal, wherein the first user profile information includes de-identified information reflecting the preferences of the first user.

4. The method according to claim 3, wherein, The first user profile information is obtained on the terminal through the following processing: acquiring the attribute information and behavior information of the first user; encrypting and encoding the attribute information and behavior information to obtain the first user profile information.

5. The method according to claim 1, wherein, The method further includes: acquiring first display results of multiple first beautified images for multiple first users, the first display results including feedback data of the first users on the first beautified images; constructing a fine-tuning dataset based on the first user profile information, the first product image, the first product information, the first beautified image and the first display results, the fine-tuning dataset including multiple sample data, the sample labels in the sample data being the first beautified image where the first display results achieve a preset effect, the sample inputs in the sample data including the first user profile information, the first product image and the first product information; and adjusting the model parameters of the image generation model based on the fine-tuning dataset.

6. The method according to claim 5, wherein, After adjusting the model parameters of the image generation model based on the fine-tuning dataset, the method further includes: obtaining second user profile information of a second user, a second product image of a second product, and second product information, wherein the second user profile information includes information reflecting the preferences of the second user; inputting the second user profile information, the second product image, and the second product information into the adjusted image generation model to obtain a second beautified image after beautifying the second product image, wherein the second beautified image is used to display the second product to the second user in business operations; obtaining a second display result of the second beautified image, wherein the second display result includes feedback data from the second user on the second beautified image; determining the performance evaluation result of the adjusted image generation model based on the second display result; and determining the scope of application of the adjusted image generation model based on the performance evaluation result.

7. The method according to claim 5, wherein, The step of adjusting the model parameters of the image generation model based on the fine-tuning dataset includes: determining the sample weights corresponding to each sample data based on the display results of the first beautified image in each of the sample data, wherein the sample weights are positively correlated with a preset index in the first display results; beautifying the first product image based on the image generation model for the sample data in the fine-tuning dataset to obtain a predicted image corresponding to the first product image; determining a first loss term corresponding to each sample data based on the difference between the predicted image corresponding to the first product image and the sample labels in each of the sample data; weighting the first loss term corresponding to each sample data based on the sample weights corresponding to each sample data to obtain a first loss function, and adjusting the model parameters of the image generation model based on the first loss function.

8. The method according to any one of claims 5-7, wherein, The feedback data in the display results includes at least one of the following metrics: exposure rate, click-through rate, number of visitors to the details page, add-to-cart rate, conversion rate, dwell time, and share rate.

9. An image enhancement device based on user profile, the device comprising: The information acquisition module is used to acquire the first user profile information of the first user, the first product image and the first product information of the first product, wherein the first user profile information includes information reflecting the preferences of the first user; The image generation module is used to input the first user profile information, the first product image, and the first product information into the image generation model to obtain a first beautified image after beautifying the first product image. The first beautified image is used to display the first product to the first user in business operations.

10. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-8.

Images