E-commerce commodity virtual display method and device based on AI image generation and style migration

Abstract

The invention relates to the technical field of image processing and computer vision, and particularly discloses an E-commerce commodity virtual display method and device based on AI image generation and style migration. The core of the method is that semantic segmentation is carried out on an original commodity image, and a semantic segmentation map for marking a commodity main body and an internal key structure is obtained; the image, the original image and the style reference image are input into a double-branch style migration network together, the network is globally stylized through a first branch, and key structure features of the commodity are strengthened through a second branch according to the semantic segmentation image; then, in a self-adaptive fusion layer guided by a semantic segmentation map, carrying out weighted fusion on the two types of features, and decoding to generate a stylized commodity image; and finally, synthesizing the image with the stylized background into a virtual display image through illumination coordination fusion. According to the scheme, the problems that in the prior art, key features of commodities are prone to being lost in the style migration process, and the sense of reality of a synthesis result is insufficient are solved, and unification of high fidelity and strong stylization is achieved.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and more specifically, to a method and apparatus for virtual display of e-commerce products based on AI image generation and style transfer. Background Technology

[0002] In e-commerce, the visual presentation of product images is crucial for attracting consumers and driving purchasing decisions. Traditional product displays rely on physical photography, which is costly, inflexible, and difficult to adapt quickly to diverse marketing scenarios and themes.

[0003] Existing deep learning-based image style transfer techniques provide a foundation for virtual product displays, but they typically treat the product as a whole for stylization. This approach has significant drawbacks: First, during the transfer process, the product's core identifying features, such as specific structural outlines and key texture details, can easily become blurred or distorted due to increased style intensity, impairing the product's recognizability and realism. Second, a uniform style application mechanism struggles to differentiate between different areas of the product (such as rigid structural components and flexible fabrics), resulting in a stiff and unnatural stylization effect. Finally, most methods fail to adequately consider the physical consistency between the generated image and the virtual background in terms of lighting and shadow, leading to a lack of realism and immersion in the synthesized display, thus affecting the final display quality and credibility.

[0004] Therefore, this paper proposes a method and device for virtual display of e-commerce products based on AI image generation and style transfer. The technical problem to be solved is how to effectively maintain the key identification features of the products while adapting e-commerce product images to diverse target styles, and improve the realism and overall coordination of the stylization results. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a method and apparatus for virtual display of e-commerce products based on AI image generation and style transfer, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for virtual display of e-commerce products based on AI image generation and style transfer, comprising the following steps: S1: Obtain the original product image and the target style reference image; S2: Perform semantic segmentation on the original product image to obtain a semantic segmentation map that includes at least the outline of the main body of the product and the outline of its internal key structures; S3: Input the original product image, the semantic segmentation map, and the target style reference image into a two-branch style transfer network; S4: Strong stylistic features are extracted and transferred from the entire input image through the first branch of the dual-branch style transfer network; S5: Through the second branch of the dual-branch style transfer network, extract and enhance the geometric and texture features of the main outline and internal key structure of the product based on the semantic segmentation map, and generate enhanced and preserved features; S6: In the fusion layer of the dual-branch style transfer network, the strongly stylized features output by the first branch and the enhanced preservation features output by the second branch are subjected to adaptive weighted fusion guided by the semantic segmentation map to generate a fused feature map. In the product subject and key structural contour area indicated by the semantic segmentation map, the fusion weight is configured to tilt towards the features of the second branch. S7: Decode the fused feature map and output the final stylized product image.

[0007] Preferably, step S2 specifically includes: S2.1: Process the original product image using the product instance segmentation model to obtain the product subject binary mask; S2.2: Using an edge detection network, extract significant internal structural edge lines from the main product area of ​​the original product image to generate an internal edge line map; S2.3: Overlay the binary mask of the product body with the internal edge line map to generate the semantic segmentation map.

[0008] Preferably, in step S2.2, the internal structural edge line includes at least one of the following: collar line, cuff line, and seam line of clothing; zipper outline and seam line of bags; screen bezel line and button gap line of electronic products.

[0009] Preferably, in step S5, the operations performed by the second branch specifically include: S5.1: Encode the original product image using an encoder to extract multi-scale visual features; S5.2: In at least one intermediate feature layer, a spatial attention module is introduced, which takes the semantic segmentation map as input and generates a spatial attention map; S5.3: Modulate the multi-scale visual features using the spatial attention map to enhance the response intensity of the feature channels corresponding to the main outline of the product and the outline of the key internal structure in the semantic segmentation map, while suppressing the feature response of the background region, to obtain the enhanced and preserved features.

[0010] Preferably, the adaptive weighted fusion described in step S6 specifically includes: S6.1: Input the semantic segmentation map into a weight prediction network to predict and generate a dual-channel fusion weight map with the same spatial resolution as the feature map. The two channels of the dual-channel fusion weight map correspond to the pixel-level weights of the first branch feature and the second branch feature, respectively. S6.2: In the fusion layer, based on the dual-channel fusion weight map, the strongly stylized features from the first branch and the enhanced preserved features from the second branch are summed pixel by pixel to complete feature fusion.

[0011] Preferably, the weight prediction network is configured such that: for pixel positions marked as the main outline and internal key structures of the product in the semantic segmentation image, the weight value assigned to the second branch feature in the output weight vector is higher than a first preset threshold; for pixel positions marked as background, the weight value assigned to the first branch feature in the output weight vector is higher than a second preset threshold.

[0012] Preferably, after step S7, a virtual display compositing step is also included: S8: Extract the global style feature vector of the target style reference image; S9: Using the global style feature vector as a conditional input, generate a background image with a consistent style through the background generation model; S10: The final stylized product image is combined with the background image through lighting coordination and fusion to generate the final e-commerce product virtual display image.

[0013] Preferably, step S10 specifically includes: S10.1: Perform foreground segmentation on the final stylized product image to obtain a foreground layer with an opacity channel; S10.2: Estimate the main lighting tone and direction of the subject edge region in the final stylized product image; S10.3: Based on the estimated main light hue and direction, perform local color and brightness affine transformation on the area in the background image corresponding to the foreground projection; S10.4: Overlay and synthesize the processed background image with the foreground layer.

[0014] A virtual e-commerce product display device based on AI image generation and style transfer, comprising: The data acquisition module is used to acquire the original product image and the target style reference image; The semantic segmentation module is used to perform semantic segmentation on the original product image to obtain a semantic segmentation map that includes at least the outline of the main body of the product and the outline of its internal key structures. A dual-branch style transfer network module, connecting the data acquisition module and the semantic segmentation module, is used to receive the original product image, the semantic segmentation map, and the target style reference image; the dual-branch style transfer network module includes: The first branch processing unit is used to perform strong stylistic feature extraction and transfer on the entire input image; The second branch processing unit is used to extract and enhance the features of the key structure of the product based on the semantic segmentation graph; The feature fusion unit is used to adaptively and weightedly fuse the output features of the first branch processing unit and the second branch processing unit in a manner guided by the semantic segmentation map to generate a fused feature map. The image decoding module is used to decode the fused feature map and output the final stylized product image.

[0015] Preferably, it also includes a virtual display compositing module, the virtual display compositing module comprising: A style encoding unit is used to extract the global style feature vector of the target style reference image; The background generation unit, connected to the style encoding unit, is used to generate a background image with a consistent style based on the global style feature vector. The lighting fusion unit, connected to the image decoding module and the background generation unit, is used to perform lighting coordination and fusion between the final stylized product image and the background image to output the final virtual display image.

[0016] The technical effects and advantages of this invention are as follows: Compared to existing technologies, this invention achieves image generation by constructing a two-branch style transfer network structure guided by a semantic segmentation map. The method first performs fine semantic segmentation on the product image, distinguishing between key structural contours that need to be strongly preserved and regions that can be freely stylized. The two branches of the network work in parallel: the first branch focuses on extracting global style features from style references for transfer, while the second branch specifically enhances the key structural features of the product based on the semantic segmentation map. Finally, a fusion layer dynamically modulated by semantic information adaptively weights and fuses the stylized features and the preserved features. This approach enables the generated image to strongly reflect the target artistic style while ensuring that the product's own iconic contours, edges, and key details are clearly and accurately preserved, thus achieving an optimal balance between stylization and fidelity.

[0017] Existing methods often lack differentiated control over different semantic regions within a product when processing features. This invention introduces a semantically guided spatial attention mechanism to modulate network activation during feature extraction. Based on the semantic segmentation map, this mechanism automatically enhances the feature channel responses corresponding to the main body of the product and its key internal structures in the feature space, while suppressing interference from irrelevant background regions. Thus, in subsequent transfer and fusion calculations, the network can concentrate more computational resources and attention on the core components of the product that need to maintain their original appearance, effectively preventing the features of these regions from being over-covered or diluted during style transfer, and improving the robustness of features in key areas.

[0018] To address the issue of abrupt blending between foreground and background in virtual displays, this invention performs a post-processing workflow for lighting coordination after generating stylized products. This workflow analyzes the lighting information at the edges of the stylized product's main body, estimates the hue and direction of its primary light source, and then uses this information to perform affine color transformation and lighting adjustments on corresponding areas of the generated stylized background image. This step simulates the color reflection and shadow projection effects of objects in a real environment, ensuring that the synthesized product and background exhibit consistent physical logic in terms of lighting. This enhances the overall realism and visual harmony of the final virtual display image, achieving a superior immersive display effect. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the overall workflow of the method of the present invention.

[0020] Figure 2 This is a flowchart illustrating the core processing and feature fusion of the present invention.

[0021] Figure 3 This is a flowchart of the adaptive fusion and illumination coordination decision-making process of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1 As attached Figures 1 to 3The method and apparatus for virtual display of e-commerce products based on AI image generation and style transfer, as shown, are implemented as follows: First, the original product image to be processed and the target style reference image defining the desired visual effect are acquired simultaneously. The original product image is usually a high-resolution image taken against a solid color background to ensure the subject is clear; the target style reference image can be any painting, filter effect, or other product scene image, and its visual style will be extracted and transferred.

[0024] Subsequently, refined semantic segmentation was performed on the original product images. Delving into the product's interior, two types of regions with different meanings for subsequent processing were identified and separated: one type consists of "key structures" that constitute the product's basic shape and core recognizability, such as the object's outer contour, specific edges (e.g., the collar line of clothing, the screen bezel of electronic products), and distinctive details (e.g., zippers, seams, logos); the other type consists of "non-key surface areas" that can be heavily stylized without affecting recognition, such as the main fabric of clothing, the leather surface of bags, and the flat parts of appliance casings. The segmentation results were encoded into a semantic segmentation map.

[0025] Then, the visual transfer occurs in a specially designed two-branch style transfer network. This network receives the original product image, the target style reference image, and the aforementioned semantic segmentation map. Its internal structure is designed with two parallel and function-specific pathways: The first path is the stylization branch, whose main task is to decouple and learn the global and local visual patterns contained in the target style reference image, such as color distribution, brushstroke texture, and light and shadow characteristics, and to forcefully apply these stylistic features to the input image.

[0026] The second pathway is the feature preservation branch, which has the opposite task. Based on the semantic segmentation map, it focuses on locating, extracting, and enhancing the features of regions marked as "key structures" in the original product image, ensuring that the strength of these features is not diluted during the forward propagation of the network. The processing results of the two branches are combined at the feature level.

[0027] This is controlled by an adaptive feature fusion module. At its core is a lightweight weight prediction network that rereads the semantic segmentation map and dynamically predicts a set of spatially adaptive fusion weights. The key function of these weights is to perform fine-tuning at the pixel level: at pixels indicated as "key structures" in the semantic segmentation map, the output of the feature preservation branch is given higher weights, thereby minimizing the influence of stylization on these regions; while in "non-key surface areas" or background areas, the output of the stylization branch is given higher weights, allowing stylistic features to be fully expressed. Through this semantically adaptive weighting, the network can generate an intermediate image in a single forward pass that both conforms to the overall target artistic style and clearly preserves the key identifying features of the product.

[0028] Finally, based on the global features of the target style reference image, a style-coordinated background environment is generated or retrieved. Then, the generated stylized product image undergoes fine-tuning to obtain its foreground transparency channel. Illumination coordination calculations are performed: the system analyzes the surface lighting information of the stylized product subject in the image (such as the distribution of highlights and shadows), and accordingly performs simulated color and brightness adjustments on the corresponding areas of the background image, making the foreground product appear as if it realistically casts shadows and reflects ambient light onto the background. Finally, the illuminated background and the foreground product image with its transparency channel are composited to output a visually unified and realistic virtual e-commerce product display image.

[0029] The following is a further detailed implementation of the above process: Furthermore, the semantic segmentation of the original product image is specified in two steps. The first step is coarse segmentation at the product subject level. An instance segmentation model (e.g., MaskR-CNN, PointRend, etc.) that has been fully pre-trained on a large general object dataset (such as COCO) or a specialized product dataset is used to process the input image. This model locates the product subject and outputs a high-precision binary mask image, denoted as . In this mask, all pixels belonging to the main product are assigned a value of 1, while background pixels are assigned a value of 0.

[0030] The second step is fine-grained edge extraction targeting key internal structures. A lightweight convolutional neural network (such as HED, RCF, or a variant thereof) specializing in edge detection is employed. To ensure that edge extraction focuses on the product itself and reduces background noise interference, the processing scope of this network is strictly limited to the product mask obtained in the previous step. Within the defined area, identify the key internal lines that characterize the design, construction, and function of the original product from its visual appearance. These lines may correspond to material boundaries (such as the seams between leather and fabric), physical structures (such as the stitching on a bag or the seams on an electronic product), or functional components (such as zipper teeth or the edges of buttons).

[0031] The output of this step is an edge response map of the same size. In the image, pixel values ​​are higher at critical edges and lower in smooth areas. The edge response map is described below. The process of converting to a binary edge map involves a dynamic thresholding logic. This logic calculates the edge response map. In the product body mask The pixel value distribution statistics within the defined area (as before) quantiles), used as an adaptive threshold Subsequently, regarding the mask... Each pixel position within Execute the judgment: If Then assign a value to the binary edge map at that location. Determine if it is a critical edge point; otherwise, assign a value. This adaptive thresholding method, based on the statistical characteristics of the image content itself, can effectively address the differences in contrast and lighting conditions among different product images. Compared to a fixed threshold, it can extract significant internal structural lines more stably and consistently.

[0032] Finally, the main mask Binary graph of internal critical edges The data is then overlaid and fused to generate a composite semantic segmentation map. ,Right now This picture The area where the product exists is marked. By marking the internal edges, the areas that need to be protected and the areas that can be freely rendered are divided within the product body.

[0033] Furthermore, an adaptive edge enhancement strategy based on product category is introduced during the extraction of internal structural edges. The system maintains or has access to a lightweight product category-key component association knowledge base. This strategy is activated when the upstream system provides product category labels (such as "dress," "sneakers," "laptop") or automatically predicts the category through an image classification model. For example, for the category "men's leather shoes," the system instructs the edge detection network to focus on the "seam between the toe and the upper," the "edges of the lace holes," and the "joint line between the sole and the upper"; for "smartphones," it focuses on the "border between the screen and the bezel," the "camera module outline," and the "shape of the side buttons."

[0034] This focus is achieved by dynamically adjusting the post-processing parameters of the edge detection network or by attaching a category-guided attention mechanism to its front end. Specifically, different non-maximum suppression thresholds and edge connection length thresholds can be configured for different categories. For critical structures requiring strong protection, a more lenient threshold is used to ensure the continuity and integrity of their edge lines; for non-critical regions, a stricter threshold is used to filter out noise. This makes the generated internal edge map... It is no longer uniform, but rich in semantic information, with those edges that are strongly related to product function and recognition being highlighted and enhanced.

[0035] Furthermore, the branch responsible for feature preservation in the dual-branch network introduces a spatial attention modulation mechanism tightly coupled to the semantic segmentation map. This branch typically contains a backbone encoder (e.g., ResNet or a lightweight MobileNet) for feature extraction, used to extract multi-level visual features from the original product image. Assume that in the encoder's... The layer obtained a spatial dimension of The number of channels is Feature tensor .

[0036] At this point, the semantic segmentation graph from the front end... (Its original size is) ) is introduced. First, bilinear interpolation is used to... Downsampled to the same spatial size as the current feature layer ,get .then, It is fed into a tiny convolutional subnetwork, which typically consists of one or two... The convolutional layers are designed to map single-channel semantic segmentation information to features. Attention weights that match along the channel dimension. This mapping process can be represented as: ,in Represents the Sigmoid activation function, ensuring that the output value is within... The interval. The final result. It is a with Attention tensors of the same size.

[0037] The key feature modulation operation is implemented through element-wise multiplication: .here, This represents element-wise multiplication. Its working principle is as follows: in the semantic segmentation graph... The pixel locations marked as key structures (such as edge lines) in the middle. The value is close to 1. After convolution and Sigmoid, The values ​​for the corresponding positions and channels also tend to be close to 1. When compared with the original features... When multiplied, the feature values ​​at these locations are almost completely preserved or even relatively enhanced (due to network training). Conversely, in background or non-critical regions, The value is close to 0, resulting in The corresponding values ​​are close to 0, thus suppressing the feature responses in these regions. This mechanism does not change the network weights, but dynamically redistributes the energy of the feature maps during forward propagation, forcing the information flow through this branch to prioritize and transmit visual features related to the key structure of the product.

[0038] Furthermore, in the adaptive feature fusion process of the dual-branch network, the stylized branch outputs features. Rich in target style textures and color modes; features preserved after branch output modulation. It contains geometric and detailed information about the key structures of the product.

[0039] Therefore, this scheme designs an independent weight prediction network, which is a lightweight fully convolutional network (e.g., composed of several...). and (Constructed by convolutional layers), whose only input is the original semantic segmentation map. The network's task is to learn a mapping function from semantic labels to fused weights, outputting a two-channel weight graph. The spatial resolution of each channel and the feature map to be fused and They are identical and satisfy the pixel-wise weight normalization condition: For all spatial locations .

[0040] During network training, a specific loss function term is designed to constrain the weight prediction network. For example, a supervision signal can be constructed: in the semantic segmentation graph... The pixel locations explicitly marked as internal critical edges are expected to be... The value is close to 1. Approaching 0; in a pure background region, the expected value is... Approaching 1, Approaching 0; in the non-edge "surface area" within the main body of the commodity, the expected value is... and Both are around 0.5, indicating balanced integration.

[0041] This supervision enables the weight prediction network to internalize the fusion preferences of different semantic regions.

[0042] During the inference phase (i.e., in practical applications), the fusion operation is based on the predicted weight graph. Execute according to the following formula: in, These are spatial coordinates. It is the channel index. This formula means that for each pixel in the image, its final fused feature is... and The weighted sum is calculated by considering the semantic category of the pixel. This allows the edge points of a zipper tooth to be almost entirely represented by the weighted sum. The design maintains its sharp lines, while adjacent leather accents can primarily utilize... Features are used to depict watercolor dappled patterns, achieving pixel-level, semantically linked adaptive style intensity control.

[0043] Furthermore, after generating the stylized product image, a post-processing workflow simulating physical lighting consistency is performed, which begins with the generation or selection of the background. First, an image encoder (such as a CLIP image encoder or a bottleneck layer of an autoencoder) is used to extract the deep global style feature vector of the target style reference image. This vector is then used as a conditional input to a pre-trained generative model (such as a Conditional Generative Adversarial Network (CGAN) or a diffusion model) to generate an image that is similar in color tone and texture style to... Matching background images with abstract or contextual content. .

[0044] Obtaining the Prospects of Stylized Products And its transparency channel (Alpha channel) Subsequently, the core of lighting coordination lies in estimating ambient light interactions. The system first locates... The outer contour of the main object extends outward into a narrow band (e.g.) A 15px wide area is designated as the "interactive influence zone." Within this zone, the image is converted from RGB to the more perceptually uniform CIELAB color space. This is achieved by calculating the average value of the pixels in this zone across the ab chroma channels. This allows us to estimate the dominant color of light reflected from the foreground object into the environment. Simultaneously, by analyzing the gradient of the L channel (luminance) in this region, we can roughly infer the direction of the main light source (e.g., the side with higher luminance is considered the illuminated side).

[0045] Next, the background image needs to be added. Dynamically determine the area affected by light This process includes the following steps: First, based on the foreground transparency channel... Calculate the axially aligned bounding box of the foreground product. Then, map this bounding box onto the background image. The coordinate space of the bounding box is used as the initial candidate region. Then, with the center of the bounding box as the origin, a certain range around its perimeter is iterated (for example, each side of the bounding box is extended outwards by its length). to All pixels within the ring-shaped region. Determine if these pixels simultaneously satisfy the following two conditions: 1) Located in the area of Channels are subjected to morphological dilation operations (using) or Within the projection area generated by the structural element; 2) The Euclidean distance to the nearest edge of the foreground subject is less than the set maximum influence radius (e.g., 30 pixels). Pixels that meet the condition are included in the set, ultimately forming the illumination influence area. To simulate the color projection (i.e., ambient occlusion) and shadow effects of foreground objects onto the background, the area... A local affine transformation is applied to each pixel within the range.

[0046] This transformation can be represented by a simplified model: in, Represents pixel position, adjustment amount It is designed to decay with distance. One implementation is to use a two-dimensional Gaussian kernel. To simulate the diffusion of influence, its center is located at the center of the projected region, and the standard deviation is... Control the rate of decay. The adjustment amount is related to the estimated color of the foreground, and can be specifically expressed as: here, It is a LAB chromaticity A small transformation matrix for converting to RGB space increments. It is a global intensity coefficient (e.g., 0.1 to 0.3) used to control the strength of the light effect. This ensures that the further away from the projection center, the weaker the impact and the more natural the transition. Finally, the composite image is achieved through transparency blending. This series of calculations makes the originally independent background After overlaying the foreground, the color and brightness of the local area change in accordance with physical intuition based on the presence and apparent color of the foreground object, which greatly reduces the "pasting" feeling common in digital compositing and creates a realistic visual experience where the product and the virtual environment are integrated.

[0047] To make the technical details and collaborative working methods of this solution more concrete, an end-to-end process calculation is performed using a specific e-commerce marketing case. Suppose a brand needs to create a series of social media posters for its newly launched "black glossy leather women's handbag," suitable for different holiday themes. The task here is to incorporate a Qixi Festival (Chinese Valentine's Day) theme into the white background product image of the handbag, using an artistic illustration featuring soft pink and purple tones and shimmering starlight elements as the target style reference image.

[0048] 1. Input and Semantic Segmentation: The system loads the original image of the handbag. Qixi Festival themed illustrations First, the instance segmentation model accurately identifies the main body of the handbag and generates a mask. It accurately covers all parts of the bag, including the body and handle. Subsequently, the edge detection network... Within the designated area, based on knowledge of the "handbag" category, the following key structural lines were emphasized: the curved opening edge of the flap, the outline of the metal clasp on the front, the accordion pleats on the sides, and the connection between the handle and the bag body. These edges were combined to form... Ultimately, through Generate semantic segmentation graph In the image, the bag body area is marked as 1, and the internal edges, such as the metal clasp and stitching, are also clearly marked.

[0049] 2. Dual-branch processing and adaptive fusion: , and It was sent to a dual-branch network. The stylization branch focused on learning the pink-purple gradient colors and starlight brushstroke textures from the Qixi Festival illustrations. The feature-preserving branch, on the other hand, based on... Its spatial attention mechanism generates high weights at feature locations corresponding to metal buckles and stitching. This preserves and enhances features such as the metallic highlights of the buckle and the recessed details of the stitching in these locations, resulting in... Meanwhile, weighted prediction network analysis Predict the fusion weight graph .exist On the pixels displayed as the edges and seams of the metal buckle, the predicted Extremely high value (e.g., 0.9). The value is extremely low (0.1); in the smooth leather areas of the bag body, the predicted value is... The value is relatively high (e.g., 0.7). The value is low (0.3).

[0050] 3. Feature fusion calculation: In the fusion layer, according to the formula... Perform pixel-by-pixel fusion. For a single pixel on the edge of the latch, the calculation might be as follows: 0.1×[Pinkish Purple Texture] + 0.9×[Metallic Silver Highlight], the result primarily preserves metallic features. For a pixel in the adjacent leather area, the calculation is... 0.7 × [pinkish-purple texture] + 0.3 × [black leather base] results in a distinct pinkish-purple style. (Network decoding) Then, output a stylized handbag image. The main body of the bag has turned into a romantic pinkish-purple gradient with subtle starlight, while the metal clasp still maintains a clear silver-white color and three-dimensional shape, and the stitching structure is also clearly visible.

[0051] 4. Illumination-coordinated synthesis: System extraction The global features are used to generate a background with a pinkish-purple night sky and a soft halo. .right Obtain by cutting out the image .analyze A simulated sheen was observed on the left side of the handbag (hypothetically), suggesting that the main light source was coming from the upper left, and that the color of light reflected from the bag was primarily pinkish-purple. In the handbag Projection area on Applying the illumination coordination formula After calculation, The area on the lower right side of the package projection is darkened with a slight purple tint (simulating shadows), while the adjacent area on the upper left is slightly brightened and has a pinkish tint (simulating reflections). Finally, through... Synthesis complete.

[0052] This entire process demonstrates how semantically guided preprocessing, differentiated parallel processing and intelligent fusion, and physical simulation post-processing work together to solve the challenges of fidelity, naturalness, and realism in the virtual display of e-commerce products.

[0053] Finally, the following points should be noted: First, in the description of this application, it should be noted that, unless otherwise specified and limited, the terms "installation", "connection", and "linkage" should be interpreted broadly, and can be mechanical or electrical connections, or internal connections between two components, or direct connections. "Up", "down", "left", "right", etc. are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may change. Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for virtual display of e-commerce products based on AI image generation and style transfer, characterized in that, Includes the following steps: S1: Obtain the original product image and the target style reference image; S2: Perform semantic segmentation on the original product image to obtain a semantic segmentation map that includes at least the outline of the main body of the product and the outline of its internal key structures; S3: Input the original product image, the semantic segmentation map, and the target style reference image into a two-branch style transfer network; S4: Strong stylistic features are extracted and transferred from the entire input image through the first branch of the dual-branch style transfer network; S5: Through the second branch of the dual-branch style transfer network, extract and enhance the geometric and texture features of the main outline and internal key structure of the product based on the semantic segmentation map, and generate enhanced and preserved features; S6: In the fusion layer of the dual-branch style transfer network, the strongly stylized features output by the first branch and the enhanced preservation features output by the second branch are subjected to adaptive weighted fusion guided by the semantic segmentation map to generate a fused feature map. In the product subject and key structural contour area indicated by the semantic segmentation map, the fusion weight is configured to tilt towards the features of the second branch. S7: Decode the fused feature map and output the final stylized product image.

2. The e-commerce product virtual display method based on AI image generation and style transfer according to claim 1, characterized in that, Step S2 specifically includes: S2.1: Process the original product image using the product instance segmentation model to obtain the product subject binary mask; S2.2: Using an edge detection network, extract significant internal structural edge lines from the main product area of ​​the original product image to generate an internal edge line map; S2.3: Overlay the binary mask of the product body with the internal edge line map to generate the semantic segmentation map.

3. The e-commerce product virtual display method based on AI image generation and style transfer according to claim 2, characterized in that, In step S2.2, the internal structural edge lines include at least one of the following: collar lines, cuff lines, and seam lines for clothing; zipper outline lines and seam lines for bags; and screen bezel lines and button gap lines for electronic products.

4. The e-commerce product virtual display method based on AI image generation and style transfer according to claim 1, characterized in that, In step S5, the operations performed by the second branch specifically include: S5.1: Encode the original product image using an encoder to extract multi-scale visual features; S5.2: In at least one intermediate feature layer, a spatial attention module is introduced, which takes the semantic segmentation map as input and generates a spatial attention map; S5.3: Modulate the multi-scale visual features using the spatial attention map to enhance the response intensity of the feature channels corresponding to the main outline of the product and the outline of the key internal structure in the semantic segmentation map, while suppressing the feature response of the background region, to obtain the enhanced and preserved features.

5. The e-commerce product virtual display method based on AI image generation and style transfer according to claim 1, characterized in that, The adaptive weighted fusion described in step S6 specifically refers to: S6.1: Input the semantic segmentation map into a weight prediction network to predict and generate a dual-channel fusion weight map with the same spatial resolution as the feature map. The two channels of the dual-channel fusion weight map correspond to the pixel-level weights of the first branch feature and the second branch feature, respectively. S6.2: In the fusion layer, based on the dual-channel fusion weight map, the strongly stylized features from the first branch and the enhanced preserved features from the second branch are summed pixel by pixel to complete feature fusion.

6. The e-commerce product virtual display method based on AI image generation and style transfer according to claim 5, characterized in that, The weight prediction network is configured such that: for pixel positions marked as the main outline and internal key structure of the product in the semantic segmentation image, the weight value of the second branch feature in the output weight vector is higher than a first preset threshold; for pixel positions marked as background, the weight value of the first branch feature in the output weight vector is higher than a second preset threshold.

7. The e-commerce product virtual display method based on AI image generation and style transfer according to claim 1, characterized in that, Following step S7, a virtual presentation compositing step is also included: S8: Extract the global style feature vector of the target style reference image; S9: Using the global style feature vector as a conditional input, generate a background image with a consistent style through the background generation model; S10: The final stylized product image is combined with the background image through lighting coordination and fusion to generate the final e-commerce product virtual display image.

8. The e-commerce product virtual display method based on AI image generation and style transfer according to claim 7, characterized in that, Step S10 specifically includes: S10.1: Perform foreground segmentation on the final stylized product image to obtain a foreground layer with an opacity channel; S10.2: Estimate the main lighting tone and direction of the subject edge region in the final stylized product image; S10.3: Based on the estimated main light hue and direction, perform local color and brightness affine transformation on the area in the background image corresponding to the foreground projection; S10.4: Overlay and synthesize the processed background image with the foreground layer.

9. A virtual e-commerce product display device based on AI image generation and style transfer, characterized in that, include: The data acquisition module is used to acquire the original product image and the target style reference image; The semantic segmentation module is used to perform semantic segmentation on the original product image to obtain a semantic segmentation map that includes at least the outline of the main body of the product and the outline of its internal key structures. A dual-branch style transfer network module, connecting the data acquisition module and the semantic segmentation module, is used to receive the original product image, the semantic segmentation map, and the target style reference image; The dual-branch style transfer network module includes: The first branch processing unit is used to perform strong stylistic feature extraction and transfer on the entire input image; The second branch processing unit is used to extract and enhance the features of the key structure of the product based on the semantic segmentation graph; The feature fusion unit is used to adaptively and weightedly fuse the output features of the first branch processing unit and the second branch processing unit in a manner guided by the semantic segmentation map to generate a fused feature map. The image decoding module is used to decode the fused feature map and output the final stylized product image.

10. The e-commerce product virtual display device based on AI image generation and style transfer according to claim 9, characterized in that, It also includes a virtual display compositing module, which comprises: A style encoding unit is used to extract the global style feature vector of the target style reference image; The background generation unit, connected to the style encoding unit, is used to generate a background image with a consistent style based on the global style feature vector. The lighting fusion unit, connected to the image decoding module and the background generation unit, is used to perform lighting coordination and fusion between the final stylized product image and the background image to output the final virtual display image.

Images