E-commerce model optimization method based on industry experience data training

Abstract

The present invention discloses an e-commerce model optimization method based on industry experience data training, and specifically relates to the field of data processing technology; preliminary model training is performed through data set generalization training, and then high-quality images are refined and screened through high-quality data sets, and finally, refined training is performed in combination with application scenarios of actual e-commerce platforms, so that the model can optimize the layout, angle and content of images according to different product categories and display scenarios. In addition, in combination with the Chinese CLIP layer and DIT architecture, the model is optimized using Chinese prompt words, and it is ensured that the generated image meets the logical requirements of e-commerce product display; finally, personalized and attractive high-quality product images are generated by inputting product categories and display scenarios, thereby improving the product display effect of the e-commerce platform, enhancing the user's purchasing experience, and solving the problems of difficulty in personalized optimization and high training complexity in traditional methods.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an e-commerce model optimization method based on industry experience data training. Background Art

[0002] With the rapid development of e-commerce, the demand for product display images on e-commerce platforms is increasing, especially in terms of product display effects and user click-through rates, where high-quality product images play a vital role. Traditional product image generation methods often rely on manual shooting or fixed templates. These methods are usually unable to flexibly respond to the display needs of different categories of products, especially in the display of products such as large appliances and small daily necessities. It is difficult to optimize image layout and angle according to different application scenarios and product characteristics. In recent years, image generation technology based on deep learning has gradually developed, but it still lacks adaptability to the specific needs of the e-commerce industry, especially in generating high-quality images that meet the needs of e-commerce product display. In order to solve these problems, a method for optimizing e-commerce models based on industry experience data has emerged. By training models with massive high-quality product display images and combining innovative multi-stage training processes, it can accurately generate more attractive images that conform to the logic of e-commerce display according to product categories and scene requirements, thereby improving users' purchasing experience and the commercial benefits of the platform.

[0003] The prior art has the following deficiencies: Existing e-commerce product image generation methods often ignore the industry-specific needs of product display images, especially how to optimize the layout, angle and content of images according to different application scenarios and product characteristics. For example, how to extract features that are highly relevant to e-commerce display from massive product image data to ensure that the generated images can not only meet the display needs of the e-commerce industry, but also have visual appeal and practical marketing value. Traditional methods are difficult to achieve personalized optimization for different product categories (such as large appliances and small daily necessities), and the model training is complex and often cannot achieve the expected high-quality output effect with limited data and computing resources. Summary of the invention

[0004] The purpose of the present invention is to provide an e-commerce model optimization method based on industry experience data training to address the deficiencies in the background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solution: an e-commerce model optimization method based on industry experience data training, comprising the following steps: S1: Collect and construct a dataset containing millions of high-definition e-commerce product display images, including display pictures of different product categories, including large appliances and small daily necessities; S2: Using a deep learning framework, the data set is optimized through a three-stage training process, which includes: a data set generalization training stage for preliminary training of the model; a high-quality data set refinement stage for screening and using high-quality product display image data; an application scenario refinement stage for fine-tuning the model in combination with application scenarios in actual e-commerce platforms, so that the model can optimize the layout, angle and content of the image according to the product category and display scenario; S3: Combine the Chinese CLIP layer and DIT architecture during training, use Chinese prompt words to optimize the model, and determine whether the generated image meets the logical requirements of e-commerce product display; S4: For images that meet the logical requirements of e-commerce product display, based on the trained e-commerce optimization model, the product category and display scene are input to generate attractive product images that meet the e-commerce display needs. The images can be personalized and optimized according to changes in the logical requirements of e-commerce product display.

[0006] Preferably, in S2, the data set generalization training stage extracts local and global features of product images by using a convolutional neural network-based architecture; the high-quality data set refinement stage screens out product images with high image resolution, strong background distinction, and natural lighting for deep learning to ensure that the model can learn high-quality features of e-commerce products; the application scenario refinement stage optimizes the layout, angle, etc. of product images according to the specific display requirements of the e-commerce platform to ensure that the generated images meet the platform display requirements.

[0007] Preferably, in S3, determining whether the generated image meets the logic requirements for displaying e-commerce products specifically includes: After analyzing the accuracy of the product images that meet the requirements according to the actual needs of the e-commerce platform, the product image generation accuracy index is obtained. The method for obtaining the product image generation accuracy index is as follows: For the generated image set and the real image set, the Inception v3 network is used to extract features, and the penultimate layer of the Inception v3 network is used as the feature vector. The mean and covariance of the real image set and the generated image set are calculated, and the difference between the generated image and the real image is measured by calculating the Frechet distance of the two Gaussian distributions. The expression is: ; In the formula, are the mean vectors of the real image and generated image features, are the covariance matrices of the real image and generated image features, respectively. FID is the product image generation accuracy index. Tr is the sum of the diagonal elements of the matrix, which is used to measure the difference between the covariance matrices.

[0008] Preferably, in S3, the expected semantic consistency index is obtained after analyzing the semantic consistency between the generated image and the user's expectation. The method for obtaining the expected semantic consistency index is: First, the input image and the user's desired text are preprocessed to adapt them to the input format of the CLIP model: the image is resized to the size required by CLIP, the text is converted to the format of CLIP input, and processed through the text encoder of the CLIP model; The image is converted into a fixed-dimensional feature vector through CLIP's image encoder, and the text is converted into a fixed-dimensional feature vector through CLIP's text encoder; in CLIP's embedding space, the image and text are mapped into high-dimensional vectors respectively. and ,in The feature vector representing the image, The feature vector representing the text is used to calculate the cosine similarity between the image and text feature vectors to measure the semantic consistency between the two. The formula for cosine similarity is: ;in: is the embedding vector of the image, is the embedding vector of the text, is the cosine similarity between the image and text feature vectors. Let the user's expected text description be T and the generated image be I. The image and text features calculated by the CLIP model are and , then the expected semantic consistency index is obtained by calculating the cosine similarity between the image and the text: ; In the formula, is the expected semantic consistency index, the text feature vector The text is converted into a high-dimensional vector through CLIP's text encoder to represent the semantic information of the text.

[0009] Preferably, the product image generation accuracy index and the expected semantic consistency index are converted into a comprehensive feature vector, and the comprehensive feature vector is used as the input of the machine learning model. The machine learning model uses each group of comprehensive feature vectors to predict the generated image to meet the logical requirements analysis value label of e-commerce product display as the prediction target, and takes minimizing the sum of prediction errors of all generated images that meet the logical requirements analysis value label of e-commerce product display as the training target. The machine learning model is trained until the sum of prediction errors converges and the model training is stopped. The generated image meets the logical requirements analysis value of e-commerce product display based on the model output results, wherein the machine learning model is a polynomial regression model.

[0010] Preferably, in S3, the obtained analysis value of the generated image meeting the logical requirements of e-commerce product display is compared with a pre-set analysis reference threshold; if the analysis value of the generated image meeting the logical requirements of e-commerce product display is greater than or equal to the pre-set analysis reference threshold, it means that the generated image meets the logical requirements of e-commerce product display, and no image abnormality signal is generated at this time; if the analysis value of the generated image meeting the logical requirements of e-commerce product display is less than the pre-set analysis reference threshold, it means that the generated image does not meet the logical requirements of e-commerce product display, and an image abnormality signal is generated at this time.

[0011] Preferably, the image can be personalized and optimized according to the changes in the logic requirements of e-commerce product display, specifically including: The logic requirement analysis values ​​generated within a fixed time period that are greater than or equal to a preset analysis reference threshold are collected, and the mean and standard deviation of all logic requirement analysis values ​​are calculated. After analyzing them, it is determined whether the generated image needs to be further optimized.

[0012] Preferably, if the mean of the logical requirement analysis values ​​is greater than or equal to the reference threshold of the mean of the logical requirement analysis values, and the standard deviation of the logical requirement analysis values ​​is less than the reference threshold of the standard deviation of the logical requirement analysis values, it means that the image performance is stable and meets the e-commerce display requirements, and no further optimization is required; If the mean of the logical requirement analysis value is greater than or equal to the reference threshold of the logical requirement analysis value mean, and the standard deviation of the logical requirement analysis value is greater than or equal to the reference threshold of the standard deviation of the logical requirement analysis value, although most of the images meet the display logic requirements, there is volatility, which means that the image performance is unstable in different scenarios, and the image generation model needs to be further optimized to improve stability; If the mean of the logical requirement analysis value is less than the reference threshold of the logical requirement analysis value mean, and the standard deviation of the logical requirement analysis value is greater than or equal to the reference threshold of the standard deviation of the logical requirement analysis value, the image does not meet the display logic requirements as a whole, and there are fluctuations, and the image generation algorithm and scene adaptability need to be further optimized; If the mean of the logical requirement analysis values ​​is less than the reference threshold of the mean of the logical requirement analysis values, and the standard deviation of the logical requirement analysis values ​​is less than the reference threshold of the standard deviation of the logical requirement analysis values, the image as a whole does not meet the display requirements, but the volatility is small. At this time, optimize the image generation algorithm or adjust the specific display strategy.

[0013] In the above technical solution, the technical effects and advantages provided by the present invention are: 1. The e-commerce product image optimization method based on industry experience data training provided by the present invention can effectively solve the problem of insufficient optimization of e-commerce display images in the prior art. Through a three-stage refined training process, the method can perform personalized optimization for different product categories and application scenarios, ensuring that the generated product images not only meet the display requirements of the e-commerce industry, but also have high visual appeal and marketing value. By accurately screening high-quality data and combining the scene optimization required by the e-commerce platform, the generated images can adapt to different display layouts, angles and styles on different platforms, improving user experience and increasing the conversion rate of products.

[0014] 2. The present invention combines the optimization method of the Chinese CLIP layer and the DIT architecture to make the generated image more in line with the logical requirements of e-commerce product display, and quantitatively evaluates the quality and semantic consistency of the generated image through the product image generation accuracy index and the expected semantic consistency index, further ensuring that the generated image is consistent with the user's expectations. Combined with the training of the machine learning model, the present invention can determine in real time whether the generated image meets the display requirements, and continuously optimize according to the feedback data, improve the stability and accuracy of the image generation model, and ultimately provide high-quality and accurate product image display solutions for e-commerce platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0016] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] For examples, see Figure 1 As shown, the e-commerce model optimization method based on industry experience data training described in this embodiment includes the following steps: S1: Collect and construct a dataset containing millions of high-definition e-commerce product display images, including display pictures of different product categories, including large appliances and small daily necessities; S2: Using a deep learning framework, the data set is optimized through a three-stage training process, which includes: a data set generalization training stage for preliminary training of the model; a high-quality data set refinement stage for screening and using high-quality product display image data; an application scenario refinement stage for fine-tuning the model in combination with application scenarios in actual e-commerce platforms, so that the model can optimize the layout, angle and content of the image according to the product category and display scenario; S3: Combine the Chinese CLIP layer and DIT architecture during training, use Chinese prompt words to optimize the model, and determine whether the generated image meets the logical requirements of e-commerce product display; S4: For images that meet the logical requirements of e-commerce product display, based on the trained e-commerce optimization model, the product category and display scene are input to generate attractive product images that meet the e-commerce display needs. The images can be personalized and optimized according to changes in the logical requirements of e-commerce product display.

[0019] In this invention, the first step is to collect and build a dataset containing millions of high-definition e-commerce product display images. This step is the basis of the entire model optimization process. By widely collecting product display images from major e-commerce platforms, brands, suppliers and other e-commerce channels, the diversity and high quality of the dataset are ensured, thereby providing rich image samples for subsequent model training.

[0020] Diversity of product categories: The product display images in the dataset cover multiple categories, ensuring that they can be optimized for the needs of different e-commerce platforms. Specifically, they include but are not limited to the following categories: Large home appliances: such as televisions, refrigerators, washing machines, air conditioners and other large home appliances, usually have large sizes and unique display requirements, and need to show clear product appearance, operation interface and functional details in the image.

[0021] Small daily necessities: such as kitchen supplies, household items, personal care equipment and other small items. These products usually need to show their details, materials and usage scenarios, so the shooting angle, lighting and details of the image are particularly important.

[0022] Apparel and accessories: such as clothing, shoes, hats, watches, etc. These products usually pay more attention to the overall matching effect and material texture, so the style, color and fabric texture must be reflected in the image.

[0023] Beauty and skin care products: such as skin creams, cosmetics, etc. The display pictures of these products often require more delicate details and lighting effects to show the product packaging, texture and actual use effects.

[0024] Food and beverage products: such as packaged food, beverages, etc. The display of these products usually focuses on the outer packaging and product details, and the color and clarity of the image are required to be high.

[0025] Digital products and accessories: For digital products such as mobile phones, headphones, power banks, etc., the images need to highlight the product’s functions, appearance, and brand characteristics.

[0026] High-definition quality and diverse perspectives: The product display images in the dataset need to not only cover a variety of product categories, but also ensure high-definition quality. High-definition images can retain more details, which helps the model better understand the appearance of the product and its functional characteristics, thereby improving the quality of the generated images. Each product display image should have high resolution and be shot from a variety of angles to show different perspectives and important functions of the product.

[0027] For example, large home appliances may need to be displayed from multiple angles, including the front, side, back, and detailed close-ups, in order to fully present the product's design and functional features.

[0028] Small daily necessities may need to be photographed in combination with actual usage scenarios (such as kitchens, living rooms, bedrooms, etc.). The image display is not limited to a single angle, but should also include the interaction between the product and the surrounding environment.

[0029] Manual screening and classification: When constructing a dataset, all collected product display images must be manually screened and classified. Screening criteria include image quality (clarity, color authenticity, contrast, lighting, etc.), accuracy and completeness of product display (avoiding blurred or unrecognizable parts), and image suitability (whether it meets the display requirements of the e-commerce platform). Each image should be classified by product category, and key features in the image (such as brand logo, model, material, etc.) should be annotated so that subsequent model training can identify and optimize relevant features in a targeted manner.

[0030] The scale and scalability of the dataset: In order to ensure the generalization and accuracy of the training model, the scale of the dataset needs to reach the million level. Such a large-scale dataset can cover more product types, different display angles, various usage scenarios, and display styles of different e-commerce platforms, ensuring that the model can adapt to the needs of different e-commerce platforms. In actual operation, the dataset can be expanded by cooperating with e-commerce platforms, opening datasets, and automatically collecting through crawler technology.

[0031] Data enhancement and diversity processing: In order to improve the diversity of the data set and the training effect, data enhancement techniques can also be used, such as image rotation, scaling, cropping, flipping, color conversion, etc. These technologies can simulate product images in different shooting environments, further enrich the data set, and improve the robustness of the model and its adaptability in various application scenarios.

[0032] By collecting and building such a dataset of millions of high-definition e-commerce product display images, we can ensure that the e-commerce optimization model trained subsequently has the ability to accurately generate high-quality product images, especially in different product categories and application scenarios, thereby effectively improving the product display effect of the e-commerce platform and enhancing the user's purchasing experience.

[0033] S2: Using a deep learning framework, the data set is optimized through a three-stage training process, wherein the three-stage training process includes: 1. Dataset generalization training phase: The main goal of this phase is to conduct preliminary training on a large-scale, diverse product image dataset so that the model can learn the general characteristics of e-commerce product images, such as basic information such as product appearance, color, shape, size, and general rules in different display scenarios. Through this phase of training, the model can form a framework with basic understanding, laying the foundation for subsequent refined training.

[0034] Specific operations include: Dataset selection and preprocessing: In this phase, a dataset of millions of commodity images is used, including various commodities (such as home appliances, daily necessities, clothing, digital products, etc.). First, the data is uniformly preprocessed, including image size standardization, color normalization, image enhancement (such as rotation, cropping, flipping), etc., to ensure that the model can learn common features from a variety of input images.

[0035] Selection of basic network structure: Use mature convolutional neural network (CNN) architectures in modern deep learning frameworks (such as TensorFlow, PyTorch), such as ResNet, VGGNet, or EfficientNet, as basic models. These networks can efficiently extract local and global features from images.

[0036] Training process: In this stage, the network learns the common features of product images from the original data through multiple iterations of training, such as color distribution, lighting characteristics, texture characteristics, object shape, etc. The training focus of this stage is to learn how to process images of different categories of products without being biased towards a specific type, ensuring that the model has a wide range of generalization capabilities.

[0037] Loss function and optimization: Use the cross-entropy loss function for classification training, or the mean square error (MSE) loss function for regression problems, so that the model gradually adjusts the weight parameters to reduce the prediction error. In addition, use optimization algorithms such as the Adam optimizer for gradient descent so that the model is continuously optimized in each iteration.

[0038] Evaluation indicators: During the generalization training phase of the dataset, the accuracy, precision, recall and other indicators of the model are evaluated to ensure that the model can effectively learn the basic features of e-commerce product images.

[0039] 2. High-quality dataset refinement stage: The goal of this stage is to focus the model training on high-quality and highly relevant product display images to further improve its generation and understanding capabilities. By selecting high-quality datasets to fine-tune the model, the model can better identify and generate high-quality images that meet the needs of e-commerce display.

[0040] Specific operations include: High-quality dataset screening: From the collected million-level datasets, a portion of images are manually screened to ensure that they have the following characteristics: The image has high resolution and clarity, and no noticeable distortion or noise. The product display angle and layout are appropriate, highlighting the characteristics of the product (such as detailed pictures, panoramic pictures, etc.). The product is clearly distinguishable from the background, ensuring that the product itself is the visual focus in the image. The lighting in the image is natural and can accurately display the color and material of the product.

[0041] Data augmentation and cropping: In this stage, data augmentation techniques are used to transform high-quality images, such as lighting adjustment, contrast enhancement, blur processing, etc. Data augmentation can simulate the display style of images in different e-commerce platforms and improve the adaptability of the model.

[0042] Target task training: Use the selected high-quality images to train the model to conduct in-depth learning of product appearance features, packaging details, brand logos, functional descriptions, etc. Focus on the details of these images to improve the model's ability to generate details and image quality.

[0043] Loss function and optimization: At this time, more sophisticated loss functions, such as perceptual loss or adversarial loss, can be used to ensure that the model not only learns the basic features of the image, but also understands the visual effects and perceptual quality of the image.

[0044] Evaluation indicators: In this stage, the visual effects, detail restoration capabilities, and similarity of the model-generated images with real product images are evaluated. At the same time, the quality of image generation is measured using evaluation indicators related to the Generative Adversarial Network (GAN), such as the FID score.

[0045] 3. Application scenario refinement stage: The purpose of this stage is to train the model in a refined manner based on the needs of the actual e-commerce platform, combined with specific product categories and display scenarios. The model will be further optimized in this stage to ensure that the generated images meet the display requirements of specific e-commerce platforms, and can improve user experience, increase click-through rate, and improve product purchase conversion rate in the actual e-commerce environment.

[0046] Specific operations include: E-commerce platform demand analysis: Through research and data analysis, understand the specific requirements of different e-commerce platforms (such as Taobao, JD.com, Amazon, etc.) for product display images. For example, some platforms may prefer neat and concise product images, while other platforms may require more diverse display angles and scenes. Different categories of products (such as home appliances, clothing, digital products, etc.) may require different display styles and image content.

[0047] Scenario-customized training: Customize the model for different application scenarios (such as home page display, category page, promotional advertisement, etc.). For example, in promotional advertisements, it may be necessary to highlight the functional highlights or discount information of the product, while on the product details page, more attention is paid to the detailed display and usage scenarios of the product. According to the needs of different scenarios, the image generation style, layout, angle, etc. of the model need to be adjusted and optimized.

[0048] Product category-specific optimization: Different training strategies are used for different categories of products (such as large appliances, daily necessities, clothing, digital products, etc.). In the display of large appliances, the model should learn how to display the product's size, shape, function buttons, etc.; while in the display of clothing products, the model should focus on the overall matching effect, fabric texture and style details of the product. For each product category, the images generated by the model should conform to the display logic and aesthetic requirements of the category.

[0049] Display layout and angle optimization: For different display layouts in e-commerce platforms (such as waterfall flow, grid display, horizontal sliding, etc.), the model should be able to adaptively adjust the layout, proportion, display angle, etc. of product images. For example, in a horizontal sliding display, the image should present a wider viewing angle; in a grid display, the image should highlight the core features of the product as much as possible and avoid redundant background.

[0050] Loss function and optimization: In this stage, in addition to traditional classification and regression loss functions, custom loss functions may be introduced to better optimize the display effect in specific scenarios. For example, through data feedback based on user behavior, the attractiveness and click-through rate of generated images can be optimized, or a multi-task learning mechanism can be introduced to simultaneously optimize image generation and scene adaptation capabilities.

[0051] Evaluation indicators: Use comprehensive evaluation criteria and combine data from real e-commerce platforms (such as click-through rate, conversion rate, etc.) to measure the effect of generated images in actual display scenarios. In addition, use user behavior data (such as user stay time, interaction rate, etc.) to evaluate the attractiveness of images to users and further optimize model performance.

[0052] S3: Combine the Chinese CLIP layer and DIT architecture during training and use Chinese prompt words to optimize the model, including: 1. Function and design of the Chinese CLIP layer: The design purpose of the Chinese CLIP (Contrastive Language-Image Pretraining) layer is to associate Chinese language with image features through natural language processing technology, so that the model can accurately understand and apply Chinese prompt words when generating product images, thereby generating product images that meet the display requirements of Chinese e-commerce platforms.

[0053] The CLIP model was originally proposed by OpenAI, which aims to map language (text) and images (vision) to the same latent space through contrastive learning. The core idea of ​​the CLIP model is to enable the model to understand the similarities between text and images through learning text-image pairs. In this invention, an optimized Chinese CLIP model is used, which is specially trained on a large-scale Chinese corpus and e-commerce product image dataset.

[0054] During training, the Chinese CLIP layer accepts Chinese prompt words (such as "modern minimalist style TV" or "stylish casual men's sports shoes") as input. These prompt words will be converted into text embeddings. The model compares these embeddings with the features of the product images to ensure that the generated product images can faithfully reflect the product features described in the Chinese prompt words.

[0055] Through contrastive learning, the model can adjust the generated results during the image generation process to make them consistent with the semantic information related to the input prompt word. For example, when the prompt word "red mobile phone" is input, the model will not only understand the product category of "mobile phone", but also understand the color characteristics of "red style", and thus generate a mobile phone image of the corresponding color.

[0056] By training the Chinese CLIP layer, the model can understand the details of Chinese prompt words more accurately when generating images. It is not just a simple keyword match, but generates high-quality product display images that meet e-commerce needs through context understanding and semantic reasoning.

[0057] 2. The role and design of the DIT architecture: The DIT (Differentiable Image Transformer) architecture is a deep image generation architecture based on a transformer. It can make full use of the global context information of the image during the generation process to ensure that the generated product images can be accurately optimized in terms of content, style, layout, etc., especially in complex e-commerce product display scenarios.

[0058] The DIT architecture uses the self-attention mechanism to process global contextual information in images, understand the spatial relationship between products and backgrounds, products and other objects, color coordination, etc. This is crucial for e-commerce image generation, because e-commerce images require not only clear product details, but also coordination between products and backgrounds in terms of matching, layout, angle, etc., to improve the attractiveness and beauty of product display.

[0059] The DIT architecture takes advantage of the Transformer to transfer and process information in different areas of the image. Compared with traditional convolutional neural networks (CNNs), the Transformer model can better capture long-range dependencies, allowing the model to flexibly adjust the layout, angle, and details of the product during the generation process.

[0060] In the DIT architecture, the information generated by each image pixel or region not only depends on local information, but is also affected by other areas of the image. Through this global processing method, the model can more naturally adjust the spatial relationship in the image, such as adjusting the interaction between the product and the background, and optimizing the separation of the foreground and background of the object.

[0061] By combining with the Chinese CLIP layer, the DIT architecture can also optimize the environment and layout of product images based on the semantic information of Chinese prompt words. For example, if the prompt word is "modern refrigerator in the kitchen scene", the model not only needs to generate a refrigerator image, but also needs to combine the layout of the kitchen scene, the placement of furniture, the lighting, etc. to generate a high-quality image that conforms to the e-commerce display logic.

[0062] 3. Training process combining the Chinese CLIP layer with the DIT architecture: By combining the Chinese CLIP layer with the DIT architecture, the model can optimize the layout, angle, and style of the image based on the understanding of the Chinese prompt words and the global context information generated by the image, ensuring that the generated image meets the high-quality standards of e-commerce display and can enhance the user's purchasing experience.

[0063] During model training, users or data providers input Chinese prompts, such as "modern-style sofa" or "business men's shoes." These prompts are converted into text embeddings through the Chinese CLIP layer, and the model generates images based on these embeddings.

[0064] Through the Chinese CLIP layer, the model first models the relationship between the input Chinese prompt words and product images through contrastive learning. The product categories, features, and display scenes involved in the prompt words are converted into embedding vectors and compared with the visual features of the image to ensure that the generated image is semantically consistent with the prompt words.

[0065] Based on text-image contrast learning, the DIT architecture generates high-quality product images through a self-attention mechanism. By modeling the global information of the image, DIT ensures that the generated images have reasonable layout, angles, and details to meet the needs of e-commerce display.

[0066] In the actual training process, the Chinese CLIP layer and the DIT architecture are jointly trained, that is, the text embedding and the feedback information of the image generation interact with each other to optimize the objective function. When generating images, the model continuously adjusts the embedding vector of the Chinese prompt word and the generated image features, so that the generated image not only conforms to the semantics of the Chinese prompt word, but also has a higher visual appeal and e-commerce display effect.

[0067] Determine whether the generated image meets the logical requirements for e-commerce product display, including: By combining the Chinese CLIP layer and the DIT architecture, the model can more accurately understand the intent of the Chinese prompt words and generate product images that meet the requirements based on the actual needs of the e-commerce platform. This accuracy can significantly improve the attractiveness of product images in actual applications on e-commerce platforms, thereby increasing user click-through rates and purchase conversion rates.

[0068] After analyzing the accuracy of the product images that meet the requirements according to the actual needs of the e-commerce platform, the product image generation accuracy index is obtained. The method for obtaining the product image generation accuracy index is as follows: First, select a pre-trained deep learning model (usually Inception v3 network). This model will be used to extract high-level features of the image. For the generated image set and the real image set, use the Inception v3 network to extract features. Usually, only the second-to-last layer (i.e., the mixing layer, pool3 layer) or other suitable intermediate layer output of the Inception v3 network is used as the feature vector. This feature vector can represent the high-dimensional features of the image.

[0069] For the real image set and the generated image set, their respective means and covariances are calculated. The difference between the generated image and the real image is measured by calculating the Frechet distance of the two Gaussian distributions. The expression is: ; In the formula, are the mean vectors of the real image and generated image features, are the covariance matrices of the real image and generated image features, respectively. FID is the product image generation accuracy index. Tr is the sum of the diagonal elements of the matrix, which is used to measure the difference between the covariance matrices.

[0070] The smaller the FID value, the smaller the difference between the generated image and the real image, and the higher the quality of the generated image. The larger the FID value, the greater the difference between the generated image and the real image, and the lower the quality of the generated image.

[0071] After combining the Chinese CLIP layer and the DIT architecture, the generated images are not only more visually outstanding, but also can truly reproduce the details and descriptions in the Chinese prompt words, making the generated images highly consistent with user expectations.

[0072] After analyzing the semantic consistency between the generated image and the user's expectations, the expected semantic consistency index is obtained. The method for obtaining the expected semantic consistency index is as follows: First, the input image and the user's expected text are preprocessed to adapt them to the input format of the CLIP model: the image is resized to the size required by CLIP, usually 224x224 pixels, and normalized (for example, subtracting the mean and dividing by the standard deviation). The text is converted into a format suitable for CLIP input, usually by converting the text into tokens through a tokenizer and processing it through the text encoder of the CLIP model.

[0073] The preprocessed image and text are input into the CLIP model: the image is converted into a fixed-dimensional feature vector through CLIP's image encoder (usually a convolutional neural network, such as ResNet), and the text is converted into a fixed-dimensional feature vector through CLIP's text encoder (usually a Transformer network).

[0074] In the embedding space of CLIP, images and text are mapped into high-dimensional vectors respectively. and ,in The feature vector representing the image, The feature vector representing the text is used to calculate the cosine similarity between the image and text feature vectors to measure the semantic consistency between the two. The formula for cosine similarity is: ;in: is the embedding vector of the image, is the embedding vector of the text, is the cosine similarity between the image and text feature vectors, and the cosine similarity result range is [-1, 1]. A similarity of 1 means that the image and text are semantically identical. A similarity of 0 means that there is no semantic match between the image and text. A similarity of -1 means that the image and text are completely inconsistent.

[0075] Assume that the text description expected by the user is T, the generated image is I, and the image and text features calculated by the CLIP model are and , then the expected semantic consistency index is obtained by calculating the cosine similarity between the image and the text: ; In the formula, is the expected semantic consistency index, the text feature vector The text is converted into a high-dimensional vector through CLIP's text encoder (Transformer), which represents the semantic information of the text. The higher the expected semantic consistency index, the better the semantic match between the image and the text.

[0076] The product image generation accuracy index and the expected semantic consistency index are converted into comprehensive feature vectors, and the comprehensive feature vectors are used as the input of the machine learning model. The machine learning model uses each group of comprehensive feature vectors to predict the generated image to meet the logical requirements of e-commerce product display analysis value labels as the prediction target, and takes minimizing the sum of prediction errors of all generated images that meet the logical requirements of e-commerce product display analysis value labels as the training target. The machine learning model is trained until the sum of prediction errors converges and the model training is stopped. The generated image meets the logical requirements of e-commerce product display analysis value according to the model output results, wherein the machine learning model is a polynomial regression model.

[0077] The obtained analysis value of the generated image meeting the logical requirements of e-commerce product display is compared with the pre-set analysis reference threshold. If the analysis value of the generated image meeting the logical requirements of e-commerce product display is greater than or equal to the pre-set analysis reference threshold, it means that the generated image meets the logical requirements of e-commerce product display, and no image abnormality signal is generated at this time; if the analysis value of the generated image meeting the logical requirements of e-commerce product display is less than the pre-set analysis reference threshold, it means that the generated image does not meet the logical requirements of e-commerce product display, and an image abnormality signal is generated at this time.

[0078] S4: Through the trained e-commerce optimization model, personalized high-quality product images that meet e-commerce display requirements can be generated according to the input product categories and display scenarios. This process combines the characteristics of different products, display requirements, and user preferences to generate more attractive and more convertible product display images.

[0079] Product categories: Different categories of products (such as home appliances, daily necessities, clothing, food, etc.) usually have different display requirements. For example, large home appliances (such as TVs and refrigerators) need to emphasize functionality and grandeur, while small daily necessities (such as cosmetics and kitchen utensils) pay more attention to details and exquisiteness.

[0080] Display scene: Display scenes can be single product displays (products are displayed isolated on a white background), environmental scenes (products are placed in a specific home or commercial environment), or situational scenes (such as holiday promotions or specific event scenes). Each scene type has different requirements for image layout, angle, background, etc.

[0081] The process of generating product images: Product category features: According to the type of product, extract the typical visual elements of the product, such as size, shape, main features, etc. Display scene features: According to different display scene requirements, extract the visual elements of the corresponding scene, such as background environment, light intensity, item matching, etc. Personalized optimization: The trained e-commerce optimization model adjusts the layout, angle, light, background and other parameters of the image according to the product category and display scene through personalized adjustment, so that the generated product image not only meets the best display effect of the e-commerce platform, but also can attract consumers' attention and increase click-through rate and purchase rate.

[0082] Generate an image: With the help of the optimization model, the system will generate an image that is personalized and meets the needs of e-commerce display. Specifically: Image layout: According to the requirements of product category and display scene, adjust the spatial relationship between the product and the background to make the image visually attractive and easy to understand.

[0083] Image angle: The angle of the product is adjusted to the best display angle. For example, home appliance products may need to emphasize the display or operation panel, while small products can show detailed close-ups.

[0084] Background optimization: The background pattern and color will be adjusted according to the needs of the scene. For example, the display of electronic products may use a modern and simple background, while the display of home products may focus more on a warm and comfortable atmosphere.

[0085] The image can be personalized and optimized according to the changes in the logic requirements of e-commerce product display, including: The logic requirement analysis values ​​generated within a fixed time period that are greater than or equal to a preset analysis reference threshold are collected, and the mean and standard deviation of all logic requirement analysis values ​​are calculated. After analyzing them, it is determined whether the generated image needs to be further optimized.

[0086] If the mean of the logical requirement analysis values ​​is greater than or equal to the reference threshold of the mean of the logical requirement analysis values, and the standard deviation of the logical requirement analysis values ​​is less than the reference threshold of the standard deviation of the logical requirement analysis values, it means that the image performance is stable and meets the e-commerce display requirements, and no further optimization is required; If the mean of the logical requirement analysis value is greater than or equal to the reference threshold of the logical requirement analysis value mean, and the standard deviation of the logical requirement analysis value is greater than or equal to the reference threshold of the standard deviation of the logical requirement analysis value, although most of the images meet the display logic requirements, there is volatility, which means that the image performance is unstable in different scenarios, and the image generation model needs to be further optimized to improve stability; If the mean of the logical requirement analysis value is less than the reference threshold of the logical requirement analysis value mean, and the standard deviation of the logical requirement analysis value is greater than or equal to the reference threshold of the standard deviation of the logical requirement analysis value, the image does not meet the display logic requirements as a whole, and there are fluctuations, and the image generation algorithm and scene adaptability need to be further optimized; If the mean of the logical requirement analysis value is less than the reference threshold of the mean of the logical requirement analysis value, and the standard deviation of the logical requirement analysis value is less than the reference threshold of the standard deviation of the logical requirement analysis value, the image as a whole does not meet the display requirements, but the volatility is small; although the performance is not as expected, it is at least consistent to a certain extent. At this time, you can consider optimizing the image generation algorithm or adjusting the specific display strategy.

[0087] In this embodiment, first, a data set containing millions of high-definition e-commerce product display images is collected and constructed, covering different product categories, such as large appliances and small daily necessities. Then, the data set is optimized through a three-stage training process using a deep learning framework: first, data set generalization training is used to perform preliminary training on the model; then, high-quality data set refinement is performed to screen high-quality product display image data; finally, through the application scenario refinement stage, the model is refined in combination with the application scenarios of the actual e-commerce platform, so that it can optimize the layout, angle and content of the image according to the product category and display scenario. During the training process, the model is optimized through Chinese prompt words in combination with the Chinese CLIP layer and DIT architecture to determine whether the generated image meets the logical requirements of e-commerce product display. For images that meet the logical requirements, based on the trained e-commerce optimization model, by inputting the product category and display scenario, a product image that is both attractive and meets the e-commerce display requirements is generated, and personalized optimization can be performed according to the changes, thereby improving the display effect of the image and the user's shopping experience.

[0088] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0089] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0090] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects associated with each other are in an "or" relationship, but it may also indicate an "and / or" relationship, which can be understood by referring to the context. A person of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this article can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0091] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An e-commerce model optimization method based on industry experience data training, characterized by: The following steps are involved: S1: Collect and construct a dataset containing millions of high-definition e-commerce product display images, including display pictures of different product categories, including large appliances and small daily necessities; S2: Using a deep learning framework, the data set is optimized through a three-stage training process, which includes: a data set generalization training stage for preliminary training of the model; a high-quality data set refinement stage for screening and using high-quality product display image data; an application scenario refinement stage for fine-tuning the model in combination with application scenarios in actual e-commerce platforms, so that the model can optimize the layout, angle and content of the image according to the product category and display scenario; S3: Combine the Chinese CLIP layer and DIT architecture during training, use Chinese prompt words to optimize the model, and determine whether the generated image meets the logical requirements of e-commerce product display; S4: For images that meet the logical requirements of e-commerce product display, based on the trained e-commerce optimization model, the product category and display scene are input to generate attractive product images that meet the e-commerce display needs. The images can be personalized and optimized according to changes in the logical requirements of e-commerce product display.

2. The e-commerce model optimization method based on industry experience data training according to claim 1 is characterized by: In S2, the dataset generalization training phase extracts local and global features of product images by using a convolutional neural network-based architecture; In the high-quality dataset refining stage, product images with high image resolution, strong background differentiation, and natural lighting are screened out for deep learning to ensure that the model can learn the high-quality features of e-commerce products. In the application scenario refining stage, the layout, angle, etc. of the product images are optimized according to the specific display needs of the e-commerce platform to ensure that the generated images meet the platform display requirements.

3. The e-commerce model optimization method based on industry experience data training according to claim 1 is characterized by: In S3, it is determined whether the generated image meets the logical requirements for displaying e-commerce products, including: After analyzing the accuracy of the product images that meet the requirements according to the actual needs of the e-commerce platform, the product image generation accuracy index is obtained. The method for obtaining the product image generation accuracy index is as follows: For the generated image set and the real image set, the Inception v3 network is used to extract features, and the penultimate layer of the Inception v3 network is used as the feature vector. The mean and covariance of the real image set and the generated image set are calculated, and the difference between the generated image and the real image is measured by calculating the Frechet distance of the two Gaussian distributions. The expression is: ; In the formula, are the mean vectors of the real image and generated image features, are the covariance matrices of the real image and generated image features, respectively. FID is the product image generation accuracy index. Tr is the sum of the diagonal elements of the matrix, which is used to measure the difference between the covariance matrices.

4. The e-commerce model optimization method based on industry experience data training according to claim 3 is characterized by: In S3, the semantic consistency between the generated image and the user's expectation is analyzed to obtain the expected semantic consistency index. The method for obtaining the expected semantic consistency index is: First, the input image and the user's desired text are preprocessed to adapt them to the input format of the CLIP model: the image is resized to the size required by CLIP, the text is converted to the format of CLIP input, and processed through the text encoder of the CLIP model; The image is converted into a fixed-dimensional feature vector through CLIP's image encoder, and the text is converted into a fixed-dimensional feature vector through CLIP's text encoder; in CLIP's embedding space, the image and text are mapped into high-dimensional vectors respectively. and ,in The feature vector representing the image, The feature vector representing the text is used to calculate the cosine similarity between the image and text feature vectors to measure the semantic consistency between the two. The formula for cosine similarity is: in: is the embedding vector of the image, is the embedding vector of the text, is the cosine similarity between the image and text feature vectors. Let the user's expected text description be T and the generated image be I. The image and text features calculated by the CLIP model are and , then the expected semantic consistency index is obtained by calculating the cosine similarity between the image and the text: ; In the formula, is the expected semantic consistency index, the text feature vector The text is converted into a high-dimensional vector through CLIP's text encoder to represent the semantic information of the text.

5. The e-commerce model optimization method based on industry experience data training according to claim 4 is characterized by: The product image generation accuracy index and the expected semantic consistency index are converted into comprehensive feature vectors, and the comprehensive feature vectors are used as the input of the machine learning model. The machine learning model uses each group of comprehensive feature vectors to predict the generated image to meet the logical requirements of e-commerce product display analysis value labels as the prediction target, and takes minimizing the sum of prediction errors of all generated images that meet the logical requirements of e-commerce product display analysis value labels as the training target. The machine learning model is trained until the sum of prediction errors converges and the model training is stopped. The generated image meets the logical requirements of e-commerce product display analysis value according to the model output results, wherein the machine learning model is a polynomial regression model.

6. The e-commerce model optimization method based on industry experience data training according to claim 5 is characterized by: In S3, the obtained analysis value of whether the generated image meets the logical requirements of e-commerce product display is compared with the pre-set analysis reference threshold. If the analysis value of whether the generated image meets the logical requirements of e-commerce product display is greater than or equal to the pre-set analysis reference threshold, it means that the generated image meets the logical requirements of e-commerce product display, and no image abnormality signal is generated at this time; if the analysis value of whether the generated image meets the logical requirements of e-commerce product display is less than the pre-set analysis reference threshold, it means that the generated image does not meet the logical requirements of e-commerce product display, and an image abnormality signal is generated at this time.

7. The e-commerce model optimization method based on industry experience data training according to claim 6 is characterized by: The image can be personalized and optimized according to the changes in the logic requirements of e-commerce product display, including: The logic requirement analysis values ​​generated within a fixed time period that are greater than or equal to a preset analysis reference threshold are collected, and the mean and standard deviation of all logic requirement analysis values ​​are calculated. After analyzing them, it is determined whether the generated image needs to be further optimized.

8. The e-commerce model optimization method based on industry experience data training according to claim 7 is characterized by: If the mean of the logical requirement analysis values ​​is greater than or equal to the reference threshold of the mean of the logical requirement analysis values, and the standard deviation of the logical requirement analysis values ​​is less than the reference threshold of the standard deviation of the logical requirement analysis values, it means that the image performance is stable and meets the e-commerce display requirements, and no further optimization is required; If the mean of the logical requirement analysis value is greater than or equal to the reference threshold of the logical requirement analysis value mean, and the standard deviation of the logical requirement analysis value is greater than or equal to the reference threshold of the standard deviation of the logical requirement analysis value, although most of the images meet the display logic requirements, there is volatility, which means that the image performance is unstable in different scenarios, and the image generation model needs to be further optimized to improve stability; If the mean of the logical requirement analysis value is less than the reference threshold of the logical requirement analysis value mean, and the standard deviation of the logical requirement analysis value is greater than or equal to the reference threshold of the standard deviation of the logical requirement analysis value, the image does not meet the display logic requirements as a whole, and there are fluctuations, and the image generation algorithm and scene adaptability need to be further optimized; If the mean of the logical requirement analysis values ​​is less than the reference threshold of the mean of the logical requirement analysis values, and the standard deviation of the logical requirement analysis values ​​is less than the reference threshold of the standard deviation of the logical requirement analysis values, the image as a whole does not meet the display requirements, but the volatility is small. At this time, optimize the image generation algorithm or adjust the specific display strategy.

Images