Method and device for beautifying product image

Abstract

The embodiment of the invention provides a product image beautifying method and device. The method comprises the steps of obtaining a product image and product information of a target product, wherein the product information comprises information used for describing the target product; the multi-modal large model carries out analysis based on the product image and the product information, task arrangement is carried out based on product understanding information obtained through analysis, an image processing link is obtained, the product understanding information comprises key component information of the target product, and the image processing link comprises a plurality of image processing tasks; and beautifying the product image according to the image processing link to obtain a beautified image of the target product. According to the method, different image processing links can be customized for different product images, so that the display effect of the product images is improved.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of image processing technology, and more specifically, to a method and apparatus for enhancing product images. Background Technology

[0002] In e-commerce, product images, such as the main product image and detail images, serve as crucial entry points for users to obtain product information and make purchasing decisions. Their visual presentation directly impacts user clicks and subsequent conversion rates. Products typically consist of several identifiable components, especially for complex, multi-functional goods like clothing, footwear, bags, electronics, and furniture sets. These components include key elements such as zippers, buttons, interfaces, keycaps, and screen bezels. These components provide users with important information about a product's functionality, craftsmanship, design, compatibility, and applicable scenarios. Therefore, high-quality product display images not only need to possess excellent visual appeal but also ensure the clear and effective presentation of key components to fully support the user's decision-making process.

[0003] In recent years, AI (Artificial Intelligence) has been widely used in the field of product image enhancement. Major platforms and service providers generally offer one-click filters, background replacement, and automatic image retouching functions to facilitate merchants in enhancing product images. However, these solutions often use uniform models, templates, or processes to process product images, making it difficult to adapt to the differentiated display needs of various products. Moreover, they are prone to losing details of key components (such as zippers being cut off or interface shapes being blurred) during automatic cropping, occlusion, or smoothing, thereby affecting users' judgment of the product's functionality and quality and reducing the order rate. Summary of the Invention

[0004] This specification provides a product image enhancement scheme that can customize different image processing links for different product images through a multimodal large model, thereby improving the display effect of product images.

[0005] In a first aspect, embodiments of this specification provide a method for enhancing product images, comprising: acquiring a product image and product information of a target product, wherein the product information includes information describing the target product; parsing the product image and product information using a multimodal large model, and arranging tasks based on the parsed product understanding information to obtain an image processing link, wherein the product understanding information includes key component information of the target product, and the image processing link includes multiple image processing tasks; and enhancing the product image according to the image processing link to obtain an enhanced image of the target product.

[0006] In some embodiments, the parsing based on the product image and product information by the multimodal large model includes: obtaining the visual perception result of the product image, the visual perception result including image information of key components of the target product; and parsing based on the visual perception result and the product information by the multimodal large model.

[0007] In some embodiments, the visual perception result may further include segmentation mask information of the target product and / or contour information of the target product.

[0008] In some embodiments, the visual perception result is obtained by parsing the product image using a visual perception model invoked by the multimodal large model.

[0009] In some embodiments, after obtaining the beautified image of the target product, the method further includes: scoring the beautified image to obtain a scoring result of the beautified image; in response to the scoring result not meeting a preset requirement, the multimodal large model re-parses and re-arranges the product image and product information according to the scoring result to obtain a new image processing link.

[0010] In some embodiments, the scoring result includes an aesthetic scoring result; the scoring of the beautified image to obtain the scoring result of the beautified image includes: scoring the beautified image by an aesthetic scoring model based on preset aesthetic scoring rules to obtain the aesthetic scoring result of the beautified image.

[0011] In some embodiments, the scoring result includes an information integrity scoring result; scoring the beautified image to obtain the scoring result of the beautified image includes: comparing the information consistency between the beautified image and the product image to obtain the information integrity scoring result of the beautified image.

[0012] In some embodiments, the product understanding information further includes quality assessment focus information, which includes information on the visual elements of the target product; the step of scoring the beautified image to obtain a score result for the beautified image includes: based on the quality assessment focus information, comparing the visual elements in the beautified image and the product image to obtain a score result for the beautified image.

[0013] In some embodiments, the product information includes an inventory unit identifier.

[0014] In some embodiments, the product understanding information includes display priority information, which is used to indicate the display priority of visual elements in the target product.

[0015] In some embodiments, each image processing task includes an image processing tool and corresponding input instructions for the image processing tool.

[0016] Secondly, embodiments of this specification provide a product image enhancement device, comprising: an information acquisition module for acquiring a product image and product information of a target product, wherein the product information includes information describing the target product; a task orchestration module for parsing the product image and product information using a multimodal large model, and for orchestrating tasks based on the parsed product understanding information to obtain an image processing link, wherein the product understanding information includes key component information of the target product, and the image processing link includes multiple image processing tasks, each image processing task including an image processing tool and corresponding input instructions; and a link processing module for enhancing the product image according to the image processing link to obtain an enhanced image of the target product.

[0017] Thirdly, embodiments of this specification provide a computing device including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in any of the implementations in the first aspect.

[0018] In the above-mentioned solution provided in this specification, multimodal large model technology is used to combine the key components of different products to customize the image processing link for each product. This ensures the display effect of key components while beautifying the product image, enhances the recognizability of product functions and quality in the beautified image, and thus improves product conversion rate and other indicators and user satisfaction. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the various embodiments disclosed in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only a few embodiments disclosed in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram illustrating the aesthetic enhancement of existing solutions as shown in this specification;

[0021] Figure 2 This is a schematic diagram of a product image enhancement method according to an embodiment of this specification;

[0022] Figure 3 This is a flowchart of a product image enhancement method according to an embodiment of this specification;

[0023] Figure 4This is a schematic diagram of the output result of a multimodal large model in an embodiment of this specification;

[0024] Figure 5 This is a schematic diagram of the output result of another multimodal large model in the embodiments of this specification;

[0025] Figure 6 This is a schematic diagram of the processing procedure of a multi-granularity visual perception and understanding model in the embodiments of this specification;

[0026] Figure 7 This is a schematic diagram of the processing procedure of an image processing tool orchestration and execution module in an embodiment of this specification;

[0027] Figure 8 This is a schematic diagram of the processing procedure of another image processing tool orchestration and execution module in the embodiments of this specification;

[0028] Figure 9 This is a schematic diagram of the processing procedure of a dual-head evaluation module in an embodiment of this specification;

[0029] Figure 10 This is a comprehensive flowchart of a product image enhancement method according to an embodiment of this specification;

[0030] Figure 11 This is a schematic diagram of the structure of the product image enhancement device in the embodiments of this specification. Detailed Implementation

[0031] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0032] The following section will first explain some of the terms that may be used in this instruction manual.

[0033] SKU (Stock Keeping Unit) is generally the smallest identifier representing a specific product in an e-commerce system. It typically contains structured product information, such as product specifications, brand, model, color, and components. Each product has a unique SKU identifier. For a given product, if any of its attributes—brand, model, configuration, packaging capacity, unit, shelf life, usage, price, or place of origin—differences exist between it and other products, its SKU identifier will also be different.

[0034] AIGC (Artificial Intelligence Generated Content) refers to the technology of automatically generating or editing images using image generation models, such as the Stable Diffusion Model or Kontext (an image editing model). In this specification, it is used to enhance product images.

[0035] In this specification, the Large Language Model (LLM) may also be referred to simply as the Large Model. A Large Language Model is a natural language processing model based on deep learning techniques, typically with billions to hundreds of billions or even more parameters, possessing powerful language understanding and generation capabilities. Large Language Models can employ the Transformer architecture or its variants (such as GPT, BERT, etc.), which utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. Large Language Models learn the statistical features and semantic relationships of language through pre-training on large-scale corpora, enabling them to generalize. The core capabilities of Large Language Models include, but are not limited to: understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Its usage typically includes two modes: direct inference and fine-tuning. In direct inference mode, the user guides the Large Language Model to generate specific outputs by designing prompts. Cue words can be task descriptions or instructions in text form, used to stimulate the semantic understanding and generation capabilities of large language models. In fine-tuning mode, large language models are further trained on small-scale datasets in specific domains to optimize their performance on specific tasks. The powerful generalization ability and flexibility of large language models make them an important tool in the field of artificial intelligence, providing efficient and accurate solutions for automated text generation and understanding.

[0036] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, large language models can also be called multimodal large language models (MLLMs). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0037] It should be noted that the key technologies of large language models can be found in the detailed description in the paper "A Survey of Large Language Models" (paper number: arXiv:2303.18223v16, published on March 11, 2025, public link: https: / / doi.org / 10.48550 / arXiv.2303.18223), and will not be repeated here.

[0038] As mentioned earlier, AI has been widely applied in product image enhancement in recent years. To improve the aesthetics of product images, major platforms and service providers generally offer one-click filters, background replacement, and automatic image retouching capabilities. However, existing solutions often use uniform models, templates, or workflows to process product images. For example, a uniform enhancement template is used for clothing products, while a fixed retouching pipeline is used for daily chemical products. This approach ignores the differences in composition requirements and key components among different products, easily leading to the loss of crucial details during image processing. For instance, zipper positions may be cropped, button reflections may be smoothed out, and interface shapes may be blurred, thus misleading users' judgments about product functionality, compatibility, and quality, and reducing the order rate. At the same time, different products require different emphases in their composition. Some products need to highlight the overall outline and texture, some need to emphasize interfaces and screens, some need to showcase accessory combinations and details, and some need to emphasize application scenarios. A single, universal enhancement strategy is difficult to adapt to different product images.

[0039] The following is combined with Figure 1 The illustration shows a beautification template in an existing solution, illustrating the shortcomings of the current technology. This beautification template enhances product images by cutting out the image and replacing the background with a uniform one, such as... Figure 1As shown, the original product image 1 of the faucet splash filter included a universal connector (the yellow component highlighted by the red dotted box). However, in the beautified image 1 obtained using the unified template, this crucial component was cropped out, resulting in the loss of key product details and misleading users into believing that the product does not contain the universal connector, thus reducing user interest. Similarly, the original product image 2 of the squat toilet grout sealant included a corresponding squat toilet background. However, in the beautified image 2 obtained using the unified template, this squat toilet background was replaced, preventing users from intuitively understanding the actual usage scenario of the product and reducing click-through rates.

[0040] Based on this, this specification proposes a product image enhancement method, which combines key components of the product and uses a multimodal large model to customize different image processing links for different product images, thereby improving the display effect of product images. Figure 2 This is a schematic diagram of the product image enhancement system architecture proposed in the embodiments of this specification. Figure 2 As shown, after acquiring the product image and product information of the target product, the multimodal large model parses the product image and product information, and arranges tasks based on the product understanding information obtained from the parsing to obtain the image processing link. The product understanding information includes the key component information of the target product, and the image processing link includes multiple image processing tasks. Then, according to the image processing link, the product image is beautified to obtain the beautified image of the target product.

[0041] This method utilizes multimodal large model technology, combined with the key components of different products, to customize the image processing chain for each product. This allows for the enhancement of product images while ensuring the display effect of key components, improving the recognizability of product functions and quality in the enhanced images, and ultimately improving product conversion rates and user satisfaction.

[0042] The implementation steps of the above method are described below with reference to the embodiments.

[0043] Figure 3 This is a flowchart illustrating an image enhancement method based on user feedback, as described in an embodiment of this specification. The execution entity of this method can be any device, platform, server, or device cluster with computing and processing capabilities. Figure 3 As shown, the method includes steps 301-303 as shown below.

[0044] In step 301, product images and product information of the target product are obtained.

[0045] Product images refer to images that primarily showcase the target product, used to display its appearance, structure, details, or usage scenarios. A target product is a tangible object with a recognizable appearance that can be presented visually; for example, it could be a product for sale on an e-commerce platform or a sample displayed on a product promotion website. Product images can be captured by photography equipment or generated by graphic rendering, typically including the product itself and related background information. For example, a product image could be the main product image on an e-commerce platform or promotional website, the main product image in a carousel, or a product detail image in a dropdown menu. Product information includes information describing the target product. For example, product information may include, but is not limited to, at least one of the following: title, category, attributes, accessory list, model / interface information, material label, brand style requirements, and price range.

[0046] In one example, product information may also include SKU identifiers to guide subsequent visual processing and composition decisions for multimodal large-scale models using semantic and attribute information from the SKU, enabling customized strategy scheduling based on product SKUs. In another example, product information may include key product display information, which can be crucial information that influences user decisions. This key information can be customized by the merchant or user, or pre-set through other means; for example, it could be a specific component of the product, a unique feature, or the product's price.

[0047] In step 302, the multimodal large model parses the product image and product information, and arranges tasks based on the product understanding information obtained from the parsing to obtain the image processing link.

[0048] In this specification, the multimodal large model serves as the overall control module. It can integrate the input product image and product information to perform fine-grained semantic understanding of the target product, generating product understanding information. Based on this product understanding information, it can customize and orchestrate suitable image processing links. This embodiment does not limit the type of multimodal large model used; it can use open-source large models or other pre-trained and fine-tuned large models.

[0049] For example, this step could involve inputting product images and information into a multimodal large-scale model, which would then output product understanding information. Based on this product understanding information, the multimodal large-scale model would then customize and orchestrate the image processing chain for the target product. Alternatively, this step could involve inputting product images and information into the multimodal large-scale model, which would simultaneously output product understanding information and the orchestrated image processing chain. In this case, the multimodal large-scale model would concurrently utilize the product understanding information to assist in orchestrating the image processing chain during the reasoning process of generating the product understanding information.

[0050] Among them, product understanding information is content generated by the multimodal big model based on its understanding of the product. It is used to guide which visual elements to highlight during the beautification process, so as to more effectively convey the appearance and function of the target product to the user while improving the aesthetics of the image.

[0051] This embodiment does not limit the specific content of the product understanding information. For example, product understanding information includes an understanding of the product's style, purpose, and function. For instance, for products like fragrance oils, the product understanding information might be "fragrant oils for indoor use"; for products like women's coats, the product understanding information might be "pink coats targeting young women." Furthermore, product understanding information may also include information about the key components of the target product. Key components can refer to components that carry the key functions of the target product, or components that have a significant impact on user decisions, such as the camera module of a mobile phone, the screen and keyboard area of ​​a laptop, or the control panel or air vent of a home appliance. The key components in the key component information may include one or more. For instance, for products like fragrance oils, key components may include the bottle cap, bottle body, bottle text, and logo; for a women's coat, key components may include the coat's shape, collar, and cuffs. Additionally, product understanding information may include compositional requirements, i.e., specific guidance on the overall layout, style, and visual atmosphere of the image. For example, for fragrance essential oils, the composition requirement could be to enhance the aesthetics and highlight a sense of sophistication; for women's outerwear, the composition requirement could be to highlight a sense of fashion.

[0052] In other implementations, product understanding information can also include display priority information, which indicates the display priority of visual elements within the target product. Display priority information can include components or features that have the greatest impact on user decisions or best reflect the product's core value and should be highlighted first. For example, for eyeshadow, the color should be emphasized first; for data cables, the interface should be emphasized first. Display priority information can also include secondary details that can be appropriately downplayed. For example, when displaying an eyeshadow palette, the text on the back of the packaging or the internal tray structure can be appropriately downplayed; when presenting a data cable, the cable section can be simplified. Display priority information can optimize visual communication efficiency and improve users' perception and understanding of key product information.

[0053] In other implementations, when subsequent steps include scoring the beautified image, the product understanding information may also include quality assessment focus information, which includes information on the visual elements of the target product to guide the scoring direction of the beautified image.

[0054] In this embodiment, the image processing chain includes multiple image processing tasks, each task performing different enhancements to the product image. For example, different image processing tasks can be used to perform image cutout, composition calculation, close-up generation, background generation, color adjustment, lighting compositing, and local sharpening. In some cases, the multiple image processing tasks in the chain have clear dependencies and execution orders; for example, image cutout must be completed before background generation. In other scenarios, some image processing tasks can be executed in parallel or in any order; for example, color adjustment can be performed before local sharpening, or local sharpening before color adjustment, or color adjustment and local sharpening can be processed simultaneously. The specific image processing tasks and their execution order can be flexibly arranged according to actual needs; this embodiment does not impose any restrictions on this.

[0055] In some embodiments, each image processing task may specifically include an image processing tool and corresponding input instructions for that tool. When orchestrating the image processing chain, the multimodal large model can schedule appropriate image processing tools for each node's image processing task and generate suitable parameterized input instructions. Here, image processing tools refer to modules with specific image processing capabilities, including but not limited to neural network models, image algorithms, or dedicated application software. For example, it can call models such as Kontext or Stable Diffusion, including image inpainting models, ControlNet models, depth estimation models, and background generation models. It can also call deep learning-based matting and semantic segmentation models for matting or image segmentation, and call image sharpening or super-resolution reconstruction algorithms, noise reduction modules, color correction, and white balance adjustment tools to achieve corresponding image processing functions. Through semantic parsing of key component information, display priority information, and composition requirements in product understanding information, the multimodal large model not only selects the most suitable tool combination but also dynamically generates parameterized input instructions such as prompts, control conditions, intensity parameters, and region masks, thereby achieving efficient, controllable, and high-quality automated beautification processing of product images. For example, appropriate background generation prompts can be provided for the background generation model. For instance, for the target product bed sheet, the input instruction generated by the background generation model is: Please generate a warm family-style background image for the product bed sheet.

[0056] To better understand this step, Figure 4 and Figure 5 The diagrams illustrate the product understanding information and image processing chain generated by the multimodal large model for two different target products. For example... Figure 4As shown, the product image and information of the fragrance essential oil are input into a multimodal large model. The model's output 1 includes product understanding, composition requirements, key components, evaluation focus, and orchestration plan (i.e., image processing chain). The product information can include the product title and other product information, such as usage scenarios and functions. The orchestration plan sequentially includes image processing tasks such as product cutout, centering, background filling, and AI lighting. As an example, Figure 4 The documentation also provides parameterized versions of StableDiffusion and ControlNet, which are required for the background fill task, along with the corresponding input instructions (i.e., prompts): Please fill the background with white. For example... Figure 5 As shown, the product image and information of a women's coat are input into a multimodal large model. The model's output 2 includes product understanding, composition requirements, key components, evaluation priorities, and a staging plan (i.e., the image processing chain). The product information may include the product title and other product information, such as usage scenarios and functions. The staging plan sequentially includes image processing tasks such as image editing, beautification optimization, saturation adjustment, and image sharpening. As an example, Figure 5 The software also provides the Kontext required for image editing tasks in a parameterized form, along with the corresponding input instructions (i.e., prompts): Please put this coat on a young woman.

[0057] In some embodiments, visual perception results from product images can also be used to provide more fine-grained reference information input to the multimodal large model, assisting the multimodal large model in understanding the product image. When the multimodal large model parses the product image based on product information, it can obtain the visual perception results of the product image, and then perform parsing based on these results and product information. This embodiment does not limit the method of obtaining the visual perception results of the product image. For example, the visual perception results can be obtained by image processing algorithms or models with image processing capabilities performing image segmentation, semantic segmentation, or image detection. The visual perception results include image information of key components of the target product. This embodiment does not limit the form of the image information of key components; for example, it can be the location information of the area where the key component is located, a segmentation mask of the key component, or extracted image features of the key component. Then, the multimodal large model can parse the product image based on the visual perception results and product information to generate product understanding information and an image processing chain.

[0058] In other embodiments, the visual perception results also include segmentation mask information and / or contour information of the target product. The segmentation mask information of the target product is used to indicate the pixel region of the target product in the product image, helping the multimodal large model to better locate, identify, and analyze the target product. The contour information of the target product refers to the spatial shape description of the target product's outer boundary, which can be represented as an ordered sequence of two-dimensional coordinate points or edge curves. This information can effectively reflect the overall shape and structural characteristics of the target product, assisting the large model in composition planning.

[0059] In one implementation, the visual perception result can be obtained by parsing the product image using a visual perception model invoked by a multimodal large model. The visual perception model can be an image segmentation model, a semantic segmentation model, an object detection model, or a keypoint detection or edge extraction model, etc. The multimodal large model can dynamically select or combine multiple visual perception models. For example... Figure 6 As shown, after the product image is input into the multi-granularity visual perception and understanding module, the various visual perception models in the multi-granularity visual perception and understanding module can output the product cutout, the extracted product outline, and key components respectively.

[0060] In step 303, the product image is beautified according to the image processing link to obtain the beautified image of the target product.

[0061] In this step, different image processing tasks can be performed on different product images according to different image processing links. For example, different registered image processing tools can be scheduled to achieve personalized beautification processing and obtain beautified images. In this embodiment, the image processing link can be executed by the image processing tool orchestration and execution module.

[0062] For example, such as Figure 7 As shown, product images of fragrance essential oils and Figure 4 The generated arrangement plan is input into the image processing tool's arrangement execution module. The image processing tool's arrangement execution module sequentially calls the corresponding image processing tools to execute each task, ultimately obtaining a high-quality beautified image, in which the key components of the target product, such as the bottle cap, bottle body, bottle body text, and logo, are presented completely without obstruction.

[0063] like Figure 8 As shown, product images and Figure 5 The generated orchestration plan is input into the orchestration execution module of the image processing tool. The orchestration execution module of the image processing tool sequentially calls the corresponding image processing tools to execute each task, and finally obtains a fashionable and beautified image of a young woman (AI-generated virtual human) wearing a pink coat. The shape of the coat, the collar and the length of the cuffs of the target product are highlighted, while the back style, which is not important, is not shown.

[0064] Enhanced images are used to display to users in business operations, and this embodiment does not limit the specific display scenarios. For example, on e-commerce platforms, thumbnails or original images of enhanced images can be displayed as cards in locations such as homepage recommendations, search results pages, channel pages, event venues, and live streaming rooms to attract users to click and enter the details page. Enhanced images can also be displayed in the main image carousel at the top of the details page or in the details page drop-down area to highlight selling points, parameters, usage scenarios, and texture. Enhanced images can also be displayed in advertising areas (such as splash screens and webpage pop-ups) on various online platforms to attract users.

[0065] In the above embodiments, multimodal large model technology is used to combine the key components of different products and customize the image processing link for each product according to the composition requirements. This can beautify the product image while ensuring the display effect of key components, enhance the recognizability of product functions and quality in the beautified image, and thus improve product conversion rate and other indicators and user satisfaction.

[0066] In some embodiments, in order to ensure the quality and effect of the generated beautified image, after obtaining the beautified image of the target product in step 303, the beautified image can be scored to obtain the scoring result of the beautified image. In response to the scoring result not meeting the preset requirements, the multimodal large model re-parses and re-arranges the product image and product information according to the scoring result to obtain a new image processing link.

[0067] This embodiment does not limit the method of scoring the beautified image. For example, it can use a large model, neural network model, or algorithm to score the beautified image in multiple dimensions according to preset rules. Specifically, it can score from dimensions such as image resolution, clarity, aesthetics, and compliance. The higher the score in the scoring result, the higher the quality of the beautified image. The preset requirement can be a score threshold. When the scoring result meets the preset requirement, the beautified image can be displayed to the user. When the scoring result does not meet the preset requirement, it is necessary to re-parse and re-arrange the product image and product information according to the scoring result to obtain a new image processing link. The product image is then re-beautified according to the new image processing link, or the previously obtained beautified image can be modified and improved.

[0068] When the product understanding information also includes key quality assessment information, this embodiment can also refer to the key quality assessment information when scoring the beautified image. Specifically, based on the key quality assessment information, visual elements in the beautified image and the product image are compared to obtain the scoring result of the beautified image. The quality assessment information is used to indicate the visual elements of the target product that need to be focused on when scoring the beautified image. For example, Figure 4The quality assessment information in the model output shows that the key assessment points are: consistency of bottle shape and consistency between bottle text and logo; Figure 5 The quality assessment information in the model output shows that the assessment focus is on: consistency of coat color and shape, and consistency of cuff length.

[0069] In one example, the scoring results can include aesthetic scores. When scoring an enhanced image, the scoring model can use preset aesthetic scoring rules to evaluate the image and obtain its aesthetic score. Aesthetic scoring rules can cover multiple visual aesthetic dimensions, such as composition, lighting, color, and style consistency. The composition dimension assesses whether the layout of elements conforms to classic composition principles (such as the rule of thirds, symmetry, and visual balance), whether the subject is prominent, and whether the white space is reasonable. The lighting dimension judges whether the direction of light, contrast, and transitions between highlights and shadows are natural and layered, and whether they effectively shape the product's texture and three-dimensionality. The color dimension analyzes whether the color tone is harmonious, whether the saturation and brightness are appropriate, whether there are any glaring or discordant color schemes, and whether it matches the product's tone. Style consistency checks whether the overall visual style of the image (such as minimalism, retro, technological, or natural) is unified, and whether the background, props, and filters are consistent with the product positioning and brand tone.

[0070] This embodiment can use an aesthetic scoring model to score the beautified image. For example, when scoring the beautified image of a fragrance essential oil, the aesthetic scoring model may score according to the following rules: if the beautified image uses soft diffused light, low contrast tones, a simple background, and places the bottle near the golden ratio point of the image, while the overall style presents a sense of tranquility and sophistication, then it will obtain higher scores in dimensions such as 'light and shadow', 'color', 'composition', and 'style consistency'; conversely, if the background is cluttered, the colors are conflicting, or the product is too small to be easily identified, then the scores in the corresponding dimensions will decrease.

[0071] In one example, the rating result may include an information integrity rating result. When rating the beautified image and obtaining the rating result for the beautified image, the information consistency between the beautified image and the product image can be compared to obtain the information integrity rating result for the beautified image.

[0072] Information consistency between beautified images and product images can include the consistency between the main product in the beautified image and the main product in the product image. That is, whether the overall product presented in the beautified image is consistent with the target product in the original product image in terms of category, model, shape, and orientation, avoiding incorrect products, deformation, or structural distortion. Information consistency between beautified images and product images can also include the consistency between the key components of the product in the beautified image and the key components of the product in the product image. That is, whether the key components of the product that carry core functions or affect user decisions are correctly preserved, clearly presented, and without deviations in shape and structure in the beautified image.

[0073] Specifically, when comparing the consistency of information between beautified images and product images, comparisons can be made from multiple dimensions, including consistency of the product subject, visibility of key components, sharpness, geometric fidelity, occlusion violations, and coverage. Key component visibility is used to determine whether key components marked in the original image are visible in the beautified image, calculating the proportion of visible components to total key components. Key component sharpness is used to assess the sharpness, detail retention, and whether information is lost due to blurring, overexposure, or underexposure in the beautified image. Geometric fidelity measures whether the shape, proportion, and angle of the product subject and key components are consistent with the original image, and whether there is stretching, distortion, or perspective distortion. Occlusion violation detection identifies whether unreasonable occlusion has been introduced during the beautification process (such as decorative elements, text labels, or background objects covering key functional areas). Such occlusion is considered a violation if it affects product identification or functional expression. Coverage and integrity check whether the beautified image fully contains the main visible surfaces of the original product and whether there are any missing parts of the product due to excessive cropping or improper composition.

[0074] This embodiment uses an information integrity scoring model to assess the information consistency between the beautified image and the product image, obtaining an information integrity score for the beautified image. For example, the beautified image and the product image can be input into the information integrity scoring model. The model will extract image features of the target product from both the beautified and product images, perform feature matching to obtain the similarity between the two features, and thus obtain a comparison result for product consistency. The model can also extract features of key components of the target product from both the beautified and product images, perform feature matching to obtain similarity, and thus obtain a comparison result for key component consistency. For example, when evaluating a beautified image of a wireless Bluetooth headset, the model will verify whether the headset itself is fully presented, whether the charging contacts are visible, whether the brand logo is clearly identifiable, whether the earbuds are not covered by the background pattern, and whether the relative positions of the left and right earbud components are consistent with the original product image. If the beautification process blurs interface details or uses decorative graphics to obscure the microphone opening in pursuit of artistic appeal, points will be deducted in the dimensions of "key component visibility" and "obstruction violation," thereby lowering the overall information integrity score.

[0075] In one implementation, scoring can be based on both aesthetic and information integrity dimensions simultaneously.

[0076] In other embodiments, an evaluation result can be generated based on the aesthetic score and information integrity score in the scoring results. The evaluation result includes an evaluation conclusion and retry suggestions. The evaluation conclusion indicates whether the beautified image needs to be regenerated, while the retry suggestions provide references based on the score, such as prioritizing text information, increasing foreground weights, relaxing cropping, simplifying the background, and adjusting enhancement intensity, to instruct the multimodal large model to rearrange the image processing chain. The evaluation result can be output simultaneously by the aforementioned aesthetic scoring model and information integrity scoring model when outputting the score, or a large model can be set up to analyze the product image, beautified image, and score to output the evaluation result.

[0077] like Figure 9 As shown, the aforementioned aesthetic scoring model and information integrity scoring model can be integrated into the dual-head evaluation module. The product image, beautified image, key component information from product understanding information, and evaluation focus information (i.e. quality evaluation focus information) are input into the dual-head evaluation module. The evaluation results output by the dual-head evaluation module include evaluation conclusions and retry suggestions.

[0078] like Figure 10 As shown, Figure 10This paper illustrates a product image enhancement process. The product image and product information, including SKU identifiers, are input into a multimodal large-scale model for processing. This generates product understanding information, including key component information and evaluation priorities, and orchestrates an image processing chain. After the image processing tool's execution module enhances the product image according to this chain, an enhanced image is obtained. A dual-head evaluation module scores the enhanced image, and the evaluation result is returned to the multimodal large-scale model to instruct it to re-parse the product image and product information and re-arrange the tasks, resulting in a new image processing chain. Enhancement processing is then performed according to this new chain to obtain a new, optimized image.

[0079] Figure 10 The proposed solution utilizes a multimodal large model to perform fine-grained (e.g., SKU-level) semantic analysis of product information and images, generating key components and critical evaluation information. By defining the image processing path for product images with SKU as the degree of focus, the solution significantly improves the consistency of the generated beautification results and makes them more aligned with the actual application scenarios of SKU products. Furthermore, by introducing pre-processing SKU-level semantic information understanding capabilities and a post-processing quality assessment and retry workflow, the solution effectively mitigates the instability of generated results that may exist in existing solutions, such as failed image matting, illogical background generation, poor lighting and shadow effects, and low image quality. This solution addresses key issues to ensure higher stability in the generation of beautified images. Through a dual-head scoring mechanism, it comprehensively evaluates the aesthetics, product consistency, and consistency of key details in the generated beautified images, providing a more comprehensive evaluation. This not only guarantees the visual appeal of the beautified images but also ensures the complete visibility and readability of key components, supporting user judgments regarding size, workmanship, compatibility, and usage scenarios. Furthermore, for beautified images that fail to meet the standards, this solution will provide a retry option and offer guiding adjustments for the next retry, further improving the effectiveness of product image beautification generation.

[0080] Figure 11 This is a schematic diagram of the product image enhancement device in the embodiments of this specification. This device can be applied to any device, platform, or cluster of devices with computing and processing capabilities. The device includes:

[0081] Information acquisition module 101 is used to acquire product images and product information of the target product, including information describing the target product;

[0082] The task orchestration module 102 is used to parse the multimodal large model based on product images and product information, and to orchestrate tasks based on the product understanding information obtained from the parsing, so as to obtain the image processing link. The product understanding information includes the key component information of the target product, and the image processing link includes multiple image processing tasks.

[0083] The link processing module 103 is used to beautify the product image according to the image processing link to obtain the beautified image of the target product.

[0084] In some embodiments, the task orchestration module 102, when used for parsing based on product images and product information by a multimodal large model, is specifically used to obtain visual perception results of product images, the visual perception results including image information of key components of the target product; and to parse based on the visual perception results and product information by the multimodal large model.

[0085] In some embodiments, the visual perception results may also include segmentation mask information of the target product and / or contour information of the target product.

[0086] In some embodiments, the visual perception result is obtained by parsing the product image using a visual perception model invoked by a multimodal large model.

[0087] In some embodiments, the device further includes a quality assessment module (not shown in the figure), which is used to score the beautified image of the target product after obtaining the beautified image to obtain a score result of the beautified image; in response to the score result not meeting the preset requirements, the multimodal large model re-parses and re-arranges the product image and product information according to the score result to obtain a new image processing link.

[0088] In some embodiments, the scoring results include aesthetic scoring results; when the quality assessment module is used to score the beautified image and obtain the scoring results of the beautified image, it is specifically used to score the beautified image by an aesthetic scoring model based on preset aesthetic scoring rules and obtain the aesthetic scoring results of the beautified image.

[0089] In some embodiments, the scoring results include information integrity scoring results; when the quality assessment module scores the beautified image and obtains the scoring results of the beautified image, it is specifically used to compare the information consistency between the beautified image and the product image to obtain the information integrity scoring results of the beautified image.

[0090] In some embodiments, the product understanding information further includes quality assessment focus information, which includes information on the visual elements of the target product; when the quality assessment module is used to score the beautified image and obtain a score result for the beautified image, it is specifically used to compare the visual elements in the beautified image and the product image based on the quality assessment focus information to obtain a score result for the beautified image.

[0091] In some embodiments, product information includes inventory unit identifiers.

[0092] In some embodiments, product understanding information includes display priority information, which is used to indicate the display priority of visual elements in the target product.

[0093] In some embodiments, each image processing task includes an image processing tool and corresponding input instructions for the image processing tool.

[0094] This specification also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed in a computer, it causes the computer to perform the method described in any of the above embodiments.

[0095] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method described in any of the above embodiments.

[0096] This specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any of the above embodiments.

[0097] It is understood that before or during the use of the technical solutions of the various embodiments in this specification, the user will be informed of the type, scope of use, and usage scenarios of the personal information involved in accordance with relevant laws and regulations, and the user's authorization will be obtained.

[0098] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to choose, based on the prompt message, whether to provide personal information to the software or hardware such as the electronic device, application program, server, or storage medium performing the operation of the technical solution described in this specification.

[0099] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0100] It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation method of this specification. Other methods that comply with relevant laws and regulations may also be applied to the implementation method of this specification.

[0101] Those skilled in the art will recognize that the functions described in the various embodiments disclosed in this specification in one or more of the examples above can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0102] In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of the multiple embodiments disclosed in this specification. It should be understood that the above descriptions are merely specific implementations of the multiple embodiments disclosed in this specification and are not intended to limit the protection scope of the multiple embodiments disclosed in this specification. Any modifications, equivalent substitutions, improvements, etc., made based on the technical solutions of the multiple embodiments disclosed in this specification should be included within the protection scope of the multiple embodiments disclosed in this specification.

Claims

1. A method for enhancing product images, the method comprising: Acquire product images and product information of the target product, wherein the product information includes information describing the target product; The multimodal large model parses the product image and product information, and arranges tasks based on the parsed product understanding information to obtain an image processing link. The product understanding information includes key component information of the target product, and the image processing link includes multiple image processing tasks. According to the image processing link, the product image is beautified to obtain the beautified image of the target product.

2. The method according to claim 1, wherein, The process of parsing the product image and product information using a multimodal large model includes: Obtain the visual perception results of the product image, the visual perception results including image information of key components of the target product; The multimodal large model is analyzed based on the visual perception results and the product information.

3. The method according to claim 2, wherein, The visual perception results also include the segmentation mask information of the target product and / or the contour information of the target product.

4. The method according to claim 2, wherein, The visual perception result is obtained by parsing the product image using the visual perception model invoked by the multimodal large model.

5. The method according to claim 1, wherein, After obtaining the beautified image of the target product, the method further includes: The beautified image is scored to obtain the score result of the beautified image; In response to the scoring result not meeting the preset requirements, the multimodal large model re-analyzes and re-arranges the product image and product information based on the scoring result to obtain a new image processing link.

6. The method according to claim 5, wherein, The scoring results include aesthetic scoring results; The process of scoring the beautified image to obtain a score result includes: Based on preset aesthetic scoring rules, the aesthetic scoring model scores the beautified image to obtain the aesthetic score result of the beautified image.

7. The method according to claim 5, wherein, The scoring results include information integrity scoring results; The process of scoring the beautified image to obtain a score result includes: By comparing the information consistency between the beautified image and the product image, an information integrity score for the beautified image is obtained.

8. The method according to claim 5, wherein, The product understanding information also includes key quality assessment information, which includes information on the visual elements of the target product. The process of scoring the beautified image to obtain a score result includes: Based on the key information for quality assessment, the visual elements in the beautified image and the product image are compared to obtain the score result of the beautified image.

9. The method according to claim 1, wherein, The product information includes the SKU identifier.

10. The method according to claim 1, wherein, The product understanding information includes display priority information, which is used to indicate the display priority of visual elements in the target product.

11. The method according to claim 1, wherein, Each image processing task includes an image processing tool and corresponding input instructions for the image processing tool.

12. A product image enhancement device, the device comprising: An information acquisition module is used to acquire product images and product information of a target product, wherein the product information includes information describing the target product. The task orchestration module is used to parse the product image and product information based on the multimodal large model, and to orchestrate tasks based on the parsed product understanding information to obtain an image processing link. The product understanding information includes key component information of the target product, and the image processing link includes multiple image processing tasks. The link processing module is used to perform beautification processing on the product image according to the image processing link to obtain the beautified image of the target product.

13. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-11.

Images