Transparent image generation

Abstract

The present application relates to the technical field of computer vision, and discloses a transparent image generation method and apparatus, a storage medium, and a computer device. The method comprises: acquiring an original product image to be processed; using a preset two-class image segmentation model to segment all of a product body in the original product image, to obtain a segmented product body mask image, the product body comprising at least two individual bodies; on the basis of the segmented product body mask image, determining a transparency channel; combining the transparency channel and three color channels of the original product image, to obtain a transparent image of the product body. The present application can improve the segmentation accuracy of a product body having a complex structure, thereby ensuring that the product body in the generated transparent image is clear and obvious, and the product can be accurately displayed.

Description

Transparent image generation Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, storage medium and computer device for generating transparent images. Background Technology

[0002] There is a significant demand for generating transparent product images in various scenarios. For example, restaurant owners require clear and distinct transparent images of dishes to accurately display them for use in menus, brochures, and online ordering. Furthermore, businesses can utilize transparent images to accurately identify and categorize dishes, aiding restaurant management systems in inventory management and sales analysis.

[0003] Currently, when generating transparent images of products, each individual element in the original product image is typically segmented separately and then combined to create the transparent image. However, many products have complex structures, containing multiple elements, and are usually displayed as a whole. This existing method of segmenting and then combining the elements results in unclear edges on the product elements, failing to meet the edge precision requirements of transparent images. Consequently, the product elements in the transparent images are not clear and distinct, making it impossible to accurately display the product. Summary of the Invention

[0004] In view of this, this application provides a transparent image generation method, apparatus, storage medium, and computer equipment, the main purpose of which is to improve the segmentation accuracy of complex structured commodity subjects, thereby ensuring that the commodity subjects in the generated transparent images are clear and distinct, and enabling accurate display of the commodities.

[0005] According to a first aspect of this application, a method for generating a transparent image is provided. The method includes: acquiring an original product image to be processed; performing overall segmentation of the product subject in the original product image using a preset two-class image segmentation model to obtain a segmented product subject mask image, wherein the product subject includes at least two individuals; determining a transparency channel based on the segmented product subject mask image; and synthesizing the transparency channel with the three color channels of the original product image to obtain a transparent image of the product subject.

[0006] Optionally, before performing overall segmentation of the product subject in the original product image using a preset binary image segmentation model to obtain a segmented product subject mask image, the method further includes: inputting the original product image into a preset integrity evaluation model for integrity evaluation to obtain an integrity evaluation result corresponding to the original product image; the step of performing overall segmentation of the product subject in the original product image using the preset binary image segmentation model to obtain a segmented product subject mask image includes: if the original product image is determined to be complete according to the integrity evaluation result, then performing overall segmentation of the product subject in the original product image using the preset binary image segmentation model to obtain a segmented product subject mask image.

[0007] Optionally, the method further includes: if the original product image is determined to be incomplete based on the integrity assessment result, then the original product image is normalized to obtain a normalized product image; the normalized product image is extended using a preset image extension model to obtain an extended product image; the step of using a preset binary image segmentation model to perform overall segmentation of the product subject in the original product image to obtain a segmented product subject mask image includes: using a preset binary image segmentation model to perform overall segmentation of the product subject in the extended product image to obtain the segmented product subject mask image.

[0008] Optionally, after determining the transparency channel based on the segmented product subject mask image, the method further includes: detecting whether there are noise points at the edges and around the segmented product subject mask image; if noise points exist, performing dilation and / or erosion processing on the segmented product subject mask image to obtain a noise-removed product subject mask image; determining the transparency channel based on the segmented product subject mask image includes: determining the transparency channel based on the noise-removed product subject mask image.

[0009] Optionally, after combining the transparency channel with the three color channels of the original product image to obtain a transparent image of the product subject, the method further includes: determining the bounding box of the product subject and the position information of the product subject in the segmented product subject mask image based on the segmented product subject mask image; and processing the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the product subject proportion requirements.

[0010] Optionally, processing the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the proportion requirements of the product subject includes: determining the longest side in the bounding box and generating a target border based on the longest side; generating a transparent background based on the size of the target border and a preset ratio; determining the coordinates of the target border in the transparent image based on the position information; cropping the transparent image according to the coordinates of the target border in the transparent image to obtain a cropped transparent image; and compositing the cropped transparent image onto the transparent background to obtain the target transparent image.

[0011] Optionally, after processing the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the requirements for the proportion of the main body of the product, the method further includes: performing a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score for the target transparent image; determining whether the target transparent image meets the display requirements based on the comprehensive quality score; and if the target transparent image meets the display requirements, adding the target transparent image to a preset transparent image resource library.

[0012] Optionally, the step of performing a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score for the target transparent image includes: inputting the target transparent image into a preset quality scoring model for quality scoring to obtain a first quality score for the target transparent image; evaluating the visual effect, edge smoothness, and redundant noise of the target transparent image to obtain visual effect evaluation results, edge smoothness evaluation results, and redundant noise evaluation results; determining a second quality score, a third quality score, and a fourth quality score based on the visual effect evaluation results, the edge smoothness evaluation results, and the redundant noise evaluation results, respectively; and performing a weighted summation of the first quality score, the second quality score, the third quality score, and the fourth quality score to obtain a comprehensive quality score for the target transparent image.

[0013] Optionally, before performing overall segmentation of the main product in the original product image using a preset binary image segmentation model to obtain a segmented product mask image, the method further includes: acquiring a first sample training set and a second sample training set; constructing an initial ground truth encoder and an initial binary image segmentation model; training the initial ground truth encoder using the first sample training set to obtain a preset ground truth encoder; and training the initial binary image segmentation model based on the preset ground truth encoder and the second sample training set to obtain the preset binary image segmentation model.

[0014] Optionally, the first sample training set includes a first ground truth mask image corresponding to a first sample product image. The step of training the initial ground truth encoder using the first sample training set to obtain a preset ground truth encoder includes: encoding the deep intermediate features of the ground truth values ​​in the first ground truth mask image using the initial ground truth encoder to obtain a first ground truth code; constructing a binary cross-entropy loss function based on the first ground truth code and the ground truth values ​​in the first ground truth mask image; and updating and iterating the parameters in the initial ground truth encoder based on the binary cross-entropy loss function until a first preset training condition is met, at which point the iteration stops, and the preset ground truth encoder is output.

[0015] Optionally, the second sample training set includes a second sample product image and its corresponding second ground truth mask image. The step of training the initial binary image segmentation model based on the preset ground truth encoder and the second sample training set to obtain the preset binary image segmentation model includes: encoding the depth intermediate features of the ground truth in the second ground truth mask image using the preset ground truth encoder to obtain a second ground truth code; inputting the second sample product image into the initial binary image segmentation model for product segmentation to obtain the segmentation feature code corresponding to the second sample product image; constructing a mean squared error loss function based on the segmentation feature code and the second ground truth code; and updating and iterating the parameters in the initial binary image segmentation model according to the mean squared error loss function until the second preset training condition is met, at which point the iteration stops and the preset binary image segmentation model is output.

[0016] According to a second aspect of this application, a transparent image generation apparatus is provided, comprising: an acquisition unit for acquiring an original product image to be processed; a segmentation unit for segmenting the main body of the product in the original product image using a preset two-class image segmentation model to obtain a segmented product body mask image, wherein the product body includes at least two individuals; a determination unit for determining a transparency channel based on the segmented product body mask image; and a synthesis unit for synthesizing the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

[0017] Optionally, the device further includes: an integrity assessment unit, used to input the original product image into a preset integrity assessment model for integrity assessment, and obtain an integrity assessment result corresponding to the original product image; and a segmentation unit, specifically used to, if the original product image is determined to be complete according to the integrity assessment result, use a preset two-class image segmentation model to perform overall segmentation of the product body in the original product image, and obtain a segmented product body mask image.

[0018] Optionally, the device further includes: an extension unit, configured to, if the original product image is determined to be incomplete according to the integrity assessment result, perform normalization processing on the original product image to obtain a normalized product image; and extend the normalized product image using a preset image extension model to obtain an extended product image; the segmentation unit is further specifically configured to, using a preset two-class image segmentation model, perform overall segmentation of the product subject in the extended product image to obtain the segmented product subject mask image.

[0019] Optionally, the apparatus further includes: a denoising unit, configured to detect whether there are noise points at the edges and around the segmented product subject mask image; if noise points exist, to perform dilation and / or erosion processing on the segmented product subject mask image to obtain a noise-removed product subject mask image; and the determining unit, specifically configured to determine the transparency channel based on the noise-removed product subject mask image.

[0020] Optionally, the apparatus further includes: the determining unit, which is further configured to determine the bounding box of the product subject and the position information of the product subject in the segmented product subject mask image based on the segmented product subject mask image; and the post-processing unit, which is configured to process the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the product subject proportion requirements.

[0021] Optionally, the post-processing unit is specifically configured to: determine the longest side of the bounding box; generate a target border based on the longest side; generate a transparent background based on the size of the target border and a preset ratio; determine the coordinates of the target border in the transparent image based on the position information; crop the transparent image according to the coordinates of the target border in the transparent image to obtain a cropped transparent image; and composite the cropped transparent image onto the transparent background to obtain the target transparent image.

[0022] Optionally, the device further includes: a quality assessment unit, configured to perform a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score for the target transparent image; based on the comprehensive quality score, determine whether the target transparent image meets the display requirements; if the target transparent image meets the display requirements, then add the target transparent image to a preset transparent image resource library.

[0023] Optionally, the quality assessment unit is specifically configured to input the target transparent image into a preset quality scoring model for quality scoring to obtain a first quality score for the target transparent image; evaluate the visual effect, edge smoothness, and redundant noise of the target transparent image to obtain visual effect evaluation results, edge smoothness evaluation results, and redundant noise evaluation results; determine a second quality score, a third quality score, and a fourth quality score based on the visual effect evaluation results, the edge smoothness evaluation results, and the redundant noise evaluation results, respectively; and perform a weighted summation of the first quality score, the second quality score, the third quality score, and the fourth quality score to obtain a comprehensive quality score for the target transparent image.

[0024] Optionally, the apparatus further includes: the acquisition unit, which is further configured to acquire a first sample training set and a second sample training set; construct an initial ground truth encoder and an initial binary image segmentation model; a training unit, which is configured to train the initial ground truth encoder using the first sample training set to obtain a preset ground truth encoder; the training unit is further configured to train the initial binary image segmentation model based on the preset ground truth encoder and the second sample training set to obtain the preset binary image segmentation model.

[0025] Optionally, the first sample training set includes a first real mask image corresponding to a first sample product image; the training unit is specifically used to encode the deep intermediate features of the real values ​​in the first real mask image using the initial ground truth encoder to obtain a first ground truth code; construct a binary cross-entropy loss function based on the first ground truth code and the real values ​​in the first real mask image; update and iterate the parameters in the initial ground truth encoder based on the binary cross-entropy loss function until a first preset training condition is met, then stop the iteration and output the preset ground truth encoder.

[0026] Optionally, the second sample training set includes a second sample product image and its corresponding second ground truth mask image; the training unit is further specifically used to encode the depth intermediate features of the ground truth in the second ground truth mask image using the preset ground truth encoder to obtain the second ground truth code; input the second sample product image into the initial binary image segmentation model to segment the product subject to obtain the segmentation feature code corresponding to the second sample product image; construct a mean squared error loss function based on the segmentation feature code and the second ground truth code; update and iterate the parameters in the initial binary image segmentation model according to the mean squared error loss function until the second preset training condition is met, then stop the iteration and output the preset binary image segmentation model.

[0027] According to a third aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described transparent image generation method.

[0028] According to a fourth aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the above-described transparent image generation method.

[0029] By employing the above technical solutions, this application provides a transparent image generation method, apparatus, storage medium, and computer device. Compared with existing technologies, this method utilizes a preset two-class image segmentation model to segment the main body of the product in the original product image as a whole. Based on the mask image of the segmented main body, a transparency channel is determined. A transparent image of the main body is then generated based on this transparency channel and the three color channels of the original product image. Because this application segments the complex structure of the main body as a whole when generating the transparent image, there is no subsequent issue of recombining individual components. Therefore, it ensures that the segmentation edges of the main body are sufficiently clear, meeting the edge precision requirements of the transparent image. This guarantees that the generated transparent image has a complete and aesthetically pleasing main body, enabling accurate product display.

[0030] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0032] Figure 1 shows a flowchart of a transparent image generation method provided in an embodiment of this application;

[0033] Figure 2 shows a schematic diagram of the reasoning framework provided in an embodiment of this application;

[0034] Figure 3 shows a flowchart of another transparent image generation method provided in an embodiment of this application;

[0035] Figure 4 shows a comparison diagram of the cutting and pasting operations before and after the embodiments of this application;

[0036] Figure 5 shows a schematic diagram of the overall process of generating transparent images provided in the embodiments of this application;

[0037] Figure 6 shows a schematic diagram of the training process of the two-class image segmentation model provided in the embodiments of this application;

[0038] Figure 7 shows a schematic diagram of the training framework provided in an embodiment of this application;

[0039] Figure 8 shows a schematic diagram of a transparent image generation device provided in an embodiment of this application;

[0040] Figure 9 shows a schematic diagram of another transparent image generation device provided in an embodiment of this application. Detailed Implementation

[0041] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0042] Existing technologies result in insufficiently clear segmentation edges of the product subject, making it difficult to meet the edge precision requirements of transparent images. Consequently, the product subject in transparent images is not clear and distinct enough, making it impossible to accurately display the product.

[0043] To overcome the aforementioned technical problems, this embodiment provides a transparent image generation method, as shown in Figure 1. The method includes:

[0044] Step 101: Obtain the original product image to be processed.

[0045] The original product image to be processed contains interfering backgrounds. The main product in the original product image can be food, household goods, electronic products, sporting goods, etc., and the main product contains at least two individuals. For example, when the main product is food (a plate of stir-fried pork with chili peppers), the main product includes the individual individual of the stir-fried pork with chili peppers and the individual individual of the container. It should be noted that the embodiments of the present invention do not specifically limit the type of the main product.

[0046] In this embodiment of the invention, the main body of the product can be photographed or downloaded from the Internet to obtain the original product image to be processed.

[0047] Step 102: Use a preset two-class image segmentation model to perform overall segmentation of the main body of the product in the original product image to obtain the segmented product body mask image.

[0048] The product subject comprises at least two individuals. The preset binary image segmentation model is a pre-trained model capable of separating the product subject from the interfering background; specifically, it can be the DIS-Net model, or other binary image segmentation models, which are not specifically limited in this invention.

[0049] To avoid segmenting and recombining individual components of the product body, which would affect the segmentation results, this embodiment of the invention directly segments and encodes the product body as a whole, without separately segmenting each individual component. Specifically, the original product image is input into a preset two-class image segmentation model to segment the product body as a whole, resulting in a segmented product body mask image, i.e., the product body mask image.

[0050] This invention employs a pre-defined two-class image segmentation model to segment the main body of the product, ensuring the clarity of the segmentation edges and thus improving the segmentation effect of transparent images.

[0051] Step 103: Determine the transparency channel based on the segmented product body mask image.

[0052] In this embodiment of the invention, after obtaining the segmented product body mask image, the segmented product body mask image is directly used as the transparency channel, i.e., the alpha channel.

[0053] Step 104: Combine the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

[0054] The original product image has RGB channels for color and an alpha channel for transparency. The transparent image is an image with the background removed.

[0055] For the embodiments of the present invention, the overall reasoning process based on the preset two-class image segmentation model is shown in Figure 2. After determining the transparency channel, the three color channels and the transparency channel of the original product image are synthesized to obtain a four-channel transparent image, and the four-channel transparent image is used as the transparent image of the product body.

[0056] This embodiment provides a transparent image generation method. It utilizes a preset two-class image segmentation model to segment the main body of the product in the original product image. Based on the mask image of the segmented main body, a transparency channel is determined. A transparent image of the main body is then generated based on this transparency channel and the three color channels of the original product image. Because this embodiment segments the complex product structure as a whole when generating the transparent image, there is no subsequent issue of recombining individual components. Therefore, it ensures that the segmentation edges of the main body are sufficiently clear, meeting the edge precision requirements of the transparent image. This guarantees that the generated transparent image has a complete and aesthetically pleasing main body, enabling accurate product display.

[0057] Furthermore, as a refinement and extension of the above embodiments, this embodiment of the invention also provides another method for generating transparent images, as shown in Figure 3, including:

[0058] Step 201: Obtain the original product image to be processed.

[0059] In this embodiment of the invention, the main body of the product can be photographed or downloaded from the Internet to obtain the original product image to be processed, which contains interfering background.

[0060] Step 202: Input the original product image into the preset integrity assessment model for integrity assessment, and obtain the integrity assessment result corresponding to the original product image.

[0061] The preset completeness assessment model can be the Alpha-CLIP model, or other models capable of performing completeness assessments; this embodiment of the invention does not impose specific limitations on this. The completeness assessment result can include whether the original product image is complete or incomplete.

[0062] In this embodiment of the invention, before segmenting the original product image, it is necessary to first assess the integrity of the original product image to ensure that the subsequently generated transparent image can contain the complete product body.

[0063] Specifically, training images can be labeled as "complete" and "incomplete" based on the completeness of the product subject in the training images. Then, the initial completeness assessment model is trained using these labeled training images to obtain a preset completeness assessment model. During formal evaluation, this preset completeness assessment model can output the probability value of whether the original product image is complete or incomplete. By setting a probability threshold, the completeness of the original product image can be ultimately determined.

[0064] For example, the original product image is predicted to be complete with a probability of 0.7 from the completeness assessment model, and the predicted probability threshold is 0.5. Since the predicted probability of the original product image being complete is greater than the predicted probability threshold, the original product image is ultimately determined to be complete.

[0065] By assessing the integrity of the original product image, this invention ensures the integrity of the product itself, thereby guaranteeing that the subsequently generated transparent image accurately displays the product.

[0066] Step 203a: If the original product image is determined to be complete based on the completeness assessment result, the main body of the product in the original product image is segmented as a whole using a preset two-class image segmentation model to obtain a segmented product body mask image.

[0067] In this embodiment of the invention, if the original product image is determined to be complete based on the integrity assessment result, the original product image is input into a preset two-class image segmentation model for segmentation and encoding to obtain a segmented product subject mask image. This ensures the integrity of the segmented product subject.

[0068] Step 203b: If the original product image is determined to be incomplete based on the completeness assessment result, the original product image is normalized to obtain a normalized product image, and the normalized product image is extended using a preset image extension model to obtain an extended product image.

[0069] Specifically, the preset image extension model can be the Stable-Diffusion model. The Stable-Diffusion model is a generative model based on the diffusion process. Through progressive denoising, it can generate high-quality and detailed images. The preset image extension model can also be other extension models with diffusion capabilities; this embodiment of the invention does not impose specific limitations on this. The extended product image retains the complete product subject.

[0070] In this embodiment of the invention, to extend an incomplete original product image and ensure the integrity of the product's main body, an image extension model needs to be pre-trained. During training, a large number of complete and incomplete sample images can be used to train the initial image extension model, thereby obtaining a preset image extension model. After training the preset image extension model, if the original product image is determined to be incomplete based on the completeness assessment results, then the preset image extension model needs to be used to extend the original product image. Before extending the image, the original product image is first normalized, such as adjusting its size to the size required by the model, and simultaneously performing color normalization to ensure that the original product image is in the same spatial structure and color space. After obtaining the normalized product image, it is input into the preset image extension model to extend the graphic content, ensuring the integrity of the product's main body structure in the original product image. For example, for a plate of stir-fried pork with chili peppers, both the stir-fried pork and the container can be fully presented in the image.

[0071] This invention utilizes a preset image extension model to extend the original product image, ensuring that the main body of the product is fully presented in the transparent image, thereby achieving accurate display of the main body of the product.

[0072] Step 204: Use a preset two-class image segmentation model to perform overall segmentation of the main body of the product in the extended product image to obtain the segmented product main body mask image.

[0073] In this embodiment of the invention, after the original product image undergoes integrity assessment and image extension, the extended product image is input into a preset two-class image segmentation model to perform overall segmentation of the product body, thereby obtaining a segmented product body mask image.

[0074] Step 205: Determine the transparency channel based on the segmented product body mask image.

[0075] In this embodiment of the invention, to ensure the quality of the segmented product main body mask image, noise point detection is required. Specifically, step 205 includes: detecting whether noise points exist at the edges and around the segmented product main body mask image; if noise points exist, performing dilation and / or erosion processing on the segmented product main body mask image to obtain a noise-removed product main body mask image. Simultaneously, the transparency channel is determined based on the noise-removed product main body mask image.

[0076] Specifically, edge detection or contour detection algorithms can be used to detect whether there are redundant noise points, i.e., "islands", on the edges and around the segmented product main body mask image. If noise points exist, small noise points can be removed by dilation and / or erosion operations to ensure that the main body edge of the subsequently generated transparent image is smoother and has fewer irrelevant pixels.

[0077] Step 206: Combine the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

[0078] After determining the transparency channel, the three color channels of the original product image are combined with the transparency channel to obtain a four-channel transparent image, which is then used as the transparent image of the product body.

[0079] Step 207: Based on the segmented product body mask image, determine the bounding box of the product body and the position information of the product body in the segmented product body mask image.

[0080] To ensure the product's size meets requirements in the final transparent image, this embodiment of the invention requires cropping and pasting the image. During cropping and pasting, the bounding box of the product and its position within the image must first be determined. Specifically, the built-in function `cv2.boundingRect` can be used to treat the segmented product mask image as a set of points, thereby calculating the minimum bounding rectangle of the product, i.e., the bounding box. Simultaneously, based on the segmented product mask image, the product's position within the image can be determined.

[0081] Step 208: Based on the bounding box and the position information, process the transparent image to obtain a target transparent image that meets the requirements for the proportion of the main body of the product.

[0082] In this embodiment of the invention, in order to generate a target transparent image that meets the requirements for the proportion of the main body of the product, step 208 specifically includes: determining the longest side of the bounding box, and generating a target border based on the longest side; generating a transparent background based on the size of the target border and a preset ratio; determining the coordinates of the target border in the transparent image based on the position information; cropping the transparent image according to the coordinates of the target border in the transparent image to obtain a cropped transparent image; and compositing the cropped transparent image onto the transparent background to obtain the target transparent image.

[0083] The target border can be a square border or a border of other shapes. The preset ratio can be set according to actual needs.

[0084] Specifically, the target border is enlarged according to a preset ratio to generate a transparent background of appropriate size. Furthermore, based on the position information of the product in the image, the coordinates of the target border in the transparent image can be determined. Then, based on these coordinates, the transparent image is cropped and pasted onto the transparent background to obtain the final target transparent image. This image is saved in a uniform format to meet the needs of merchants displaying their products, as shown in Figure 4. The left image is the transparent image before cropping, and the right image is the transparent image after cropping.

[0085] This invention, through cropping and pasting the generated transparent image, enables the main body of the product to meet the proportion requirements within the transparent image, thereby achieving accurate product display.

[0086] In some embodiments, to ensure that the transparent image to be displayed meets the merchant's requirements for showcasing the main body of the product, the present invention can also perform a comprehensive quality assessment on the acquired target transparent image, as shown in Figure 5. The method for this process includes: performing a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score for the target transparent image; determining, based on the comprehensive quality score, whether the target transparent image meets the display requirements; and if the target transparent image meets the display requirements, adding the target transparent image to a preset transparent image resource library.

[0087] When determining the overall quality score of a target transparent image, the target transparent image is input into a preset quality scoring model for quality scoring to obtain a first quality score of the target transparent image; the visual effect, edge smoothness, and redundant noise of the target transparent image are evaluated to obtain visual effect evaluation results, edge smoothness evaluation results, and redundant noise evaluation results; based on the visual effect evaluation results, the edge smoothness evaluation results, and the redundant noise evaluation results, a second quality score, a third quality score, and a fourth quality score are determined respectively; the first quality score, the second quality score, the third quality score, and the fourth quality score are weighted and summed to obtain the overall quality score of the target transparent image.

[0088] The preset quality scoring model can be a binary classification model, such as a support vector machine or a neural network. This preset quality scoring model can classify a transparent target image into a high-quality image and a low-quality image. The weights corresponding to the first, second, third, and fourth quality scores can be set according to actual needs. In this embodiment of the invention, the weight of the first quality score can be set to the maximum.

[0089] Specifically, the target transparent image is input into a preset quality scoring model for quality averaging, yielding a probability value between 0 and 1 that indicates it belongs to a high-quality image. This probability value determines the first quality score. Simultaneously, the visual effect, edge smoothness, and redundant noise of the target transparent image are evaluated; better evaluation results in a higher score. Finally, all quality scores are weighted and summed to obtain the comprehensive quality score of the target transparent image.

[0090] After determining the overall quality score of the target transparent image, it is checked whether the overall quality score is greater than the preset score. If the overall quality score is greater than the preset score, the target transparent image is determined to meet the display requirements and is added to the preset transparent image resource library; conversely, if the overall quality score is less than or equal to the preset score, the target transparent image is determined to not meet the display requirements. The preset score can be set according to actual needs.

[0091] Therefore, by comprehensively scoring the target transparent images, high-quality transparent images that meet actual needs can be selected as transparent image library resources for merchants.

[0092] Furthermore, embodiments of the present invention also provide a training method for a preset two-class image segmentation model, as shown in Figure 6, including:

[0093] Step 301: Obtain the first sample training set and the second sample training set.

[0094] The first sample training set includes a first sample product image and its corresponding first real mask image, and the second sample training set includes a second sample product image and its corresponding second real mask image.

[0095] In this embodiment of the invention, the first sample training set and the second sample training set are different training sets, and the amount of data in the first sample training set is less than the amount of data in the second sample training set.

[0096] Step 302: Construct the initial ground truth encoder and the initial binary image segmentation model.

[0097] The initial binary image segmentation model can be either the DIS-Net model or other binary image segmentation models; this invention does not impose any specific limitations on it.

[0098] In this embodiment of the invention, the parameters in the truth encoder and the binary image segmentation model are initialized to construct the initial truth encoder and the initial binary image segmentation model.

[0099] Step 303: Train the initial ground truth encoder using the first sample training set to obtain a preset ground truth encoder.

[0100] This invention divides the entire training process into two stages. The first stage involves training the ground truth encoder separately, and the second stage involves training the binary image segmentation model based on the trained ground truth encoder, as shown in Figure 7. For the first stage of training, step 303 specifically includes: encoding the depth intermediate features of the ground truth values ​​in the first ground truth mask image using the initial ground truth encoder to obtain a first ground truth code; constructing a binary cross-entropy loss function (BCE) based on the first ground truth code and the ground truth values ​​in the first ground truth mask image; and updating and iterating the parameters in the initial ground truth encoder based on the binary cross-entropy loss function until a first preset training condition is met, at which point the iteration stops, and the preset ground truth encoder is output.

[0101] The first preset training condition can be set according to actual needs, such as stopping iteration when the preset number of iterations is reached.

[0102] Step 304: Based on the preset truth encoder and the second sample training set, train the initial binary image segmentation model to obtain the preset binary image segmentation model.

[0103] In this embodiment of the invention, for the second stage of the training process, step 304 specifically includes: encoding the depth intermediate features of the ground truth in the second ground truth mask image using the preset ground truth encoder to obtain the second ground truth encoding; inputting the second sample product image into the initial binary image segmentation model for product subject segmentation to obtain the segmentation feature encoding corresponding to the second sample product image; constructing a mean squared error loss function (MSE) based on the segmentation feature encoding and the second ground truth encoding; updating and iterating the parameters in the initial binary image segmentation model according to the mean squared error loss function until the second preset training condition is met, stopping the iteration, and outputting the preset binary image segmentation model.

[0104] The second preset training condition can be set according to actual needs, such as stopping iteration when the preset number of iterations is reached.

[0105] This invention introduces an intermediate-supervised segmentation model training method. It extracts deep intermediate features from the ground truth by pre-training a ground truth encoder and calculates the consistency loss between the deep intermediate features generated from the input image and the deep intermediate features from the ground truth during training, enabling feature synchronization of deep intermediate features. The diversity of deep intermediate features provides stronger regularity and more comprehensive supervision, thereby reducing the risk of overfitting.

[0106] Furthermore, as a specific implementation of the methods shown in Figures 1, 3 and 6, this embodiment provides a transparent image generation device, as shown in Figure 8. The device includes: an acquisition unit 31, a segmentation unit 32, a determination unit 33 and a synthesis unit 34.

[0107] The acquisition unit 31 can be used to acquire the original product image to be processed.

[0108] The segmentation unit 32 can be used to perform overall segmentation of the main body of the product in the original product image using a preset two-type image segmentation model to obtain a segmented product body mask image, wherein the main body of the product includes at least two individuals.

[0109] The determining unit 33 can be used to determine the transparency channel based on the segmented product body mask image.

[0110] The compositing unit 34 can be used to combine the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

[0111] In some embodiments, as shown in FIG9, the apparatus further includes an integrity assessment unit 35.

[0112] The integrity assessment unit 35 can be used to input the original product image into a preset integrity assessment model for integrity assessment, and obtain the integrity assessment result corresponding to the original product image.

[0113] The segmentation unit 32 can be specifically used to segment the main body of the product in the original product image as a whole using a preset two-class image segmentation model if the original product image is determined to be complete according to the integrity assessment result, so as to obtain the segmented product main body mask image.

[0114] In some embodiments, the device further includes an extension unit 36.

[0115] The extension unit 36 ​​can be used to perform normalization processing on the original product image if it is determined that the original product image is incomplete according to the integrity assessment result, to obtain a normalized product image; and to perform extension processing on the normalized product image using a preset image extension model to obtain an extended product image.

[0116] The segmentation unit 32 can also be specifically used to perform overall segmentation of the main body of the product in the extended product image using a preset two-type image segmentation model to obtain the segmented product main body mask image.

[0117] In some embodiments, the device further includes a noise reduction unit 37.

[0118] The denoising unit 37 can be used to detect whether there are noise points on the edges and around the segmented product main body mask image; if there are noise points, the segmented product main body mask image is subjected to dilation and / or erosion processing to obtain a noise-removed product main body mask image.

[0119] The determining unit 33 can be specifically used to determine the transparency channel based on the noise-removed product body mask image.

[0120] In some embodiments, the apparatus further includes a post-processing unit 38.

[0121] The determining unit 33 can also be used to determine the bounding box of the product body and the position information of the product body in the segmented product body mask image based on the segmented product body mask image.

[0122] The post-processing unit 38 can be used to process the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the requirements of the proportion of the main body of the product.

[0123] In some embodiments, the post-processing unit 38 may be specifically configured to: determine the longest side of the bounding box; generate a target border based on the longest side; generate a transparent background based on the size of the target border and a preset ratio; determine the coordinates of the target border in the transparent image based on the position information; crop the transparent image according to the coordinates of the target border in the transparent image to obtain a cropped transparent image; and composite the cropped transparent image onto the transparent background to obtain the target transparent image.

[0124] In some embodiments, the apparatus further includes a quality assessment unit 39.

[0125] The quality assessment unit 39 can be used to perform a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score for the target transparent image; based on the comprehensive quality score, determine whether the target transparent image meets the display requirements; if the target transparent image meets the display requirements, then add the target transparent image to the preset transparent image resource library.

[0126] In some embodiments, the quality assessment unit 39 may be specifically used to input the target transparent image into a preset quality scoring model for quality scoring to obtain a first quality score of the target transparent image; to evaluate the visual effect, edge smoothness, and redundant noise of the target transparent image to obtain visual effect evaluation results, edge smoothness evaluation results, and redundant noise evaluation results; to determine a second quality score, a third quality score, and a fourth quality score based on the visual effect evaluation results, the edge smoothness evaluation results, and the redundant noise evaluation results; and to perform a weighted summation of the first quality score, the second quality score, the third quality score, and the fourth quality score to obtain a comprehensive quality score of the target transparent image.

[0127] In some embodiments, the apparatus further includes a training unit 310.

[0128] The acquisition unit 31 can also be used to acquire the first sample training set and the second sample training set; and to construct the initial ground truth encoder and the initial binary image segmentation model.

[0129] The training unit 310 can be used to train the initial truth encoder using the first sample training set to obtain a preset truth encoder.

[0130] The training unit 310 can also be used to train the initial binary image segmentation model based on the preset ground truth encoder and the second sample training set to obtain the preset binary image segmentation model.

[0131] In some embodiments, the first sample training set includes a first ground truth mask image corresponding to a first sample product image. The training unit 310 can be specifically configured to encode the deep intermediate features of the ground truth values ​​in the first ground truth mask image using the initial ground truth encoder to obtain a first ground truth code; construct a binary cross-entropy loss function based on the first ground truth code and the ground truth values ​​in the first ground truth mask image; and update and iterate the parameters in the initial ground truth encoder based on the binary cross-entropy loss function until a first preset training condition is met, at which point the iteration stops and the preset ground truth encoder is output.

[0132] In some embodiments, the second sample training set includes a second sample product image and its corresponding second ground truth mask image. The training unit 310 may further be specifically configured to encode the depth intermediate features of the ground truth in the second ground truth mask image using the preset ground truth encoder to obtain a second ground truth code; input the second sample product image into the initial binary image segmentation model for product subject segmentation to obtain the segmentation feature code corresponding to the second sample product image; construct a mean squared error loss function based on the segmentation feature code and the second ground truth code; and update and iterate the parameters in the initial binary image segmentation model according to the mean squared error loss function until the second preset training condition is met, at which point the iteration stops and the preset binary image segmentation model is output.

[0133] It should be noted that other corresponding descriptions of the functional units involved in the transparent image generation device provided in this embodiment can be found in the corresponding descriptions in Figures 1, 3 and 6, and will not be repeated here.

[0134] Based on the methods shown in Figures 1, 3, and 6, this application embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the transparent image generation method shown in Figures 1, 3, and 6.

[0135] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0136] Based on the methods shown in Figures 1, 3, and 6, and the virtual device embodiments shown in Figures 8 and 9, in order to achieve the above objectives, this application also provides a computer device, which may be a tablet computer, smartphone, smartwatch, smart bracelet, or other network device, etc. The computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the transparent image generation method shown in Figures 1, 3, and 6.

[0137] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0138] Those skilled in the art will understand that the computer device structure provided in this embodiment does not constitute a limitation on such physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0139] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the two physical devices mentioned above, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0140] In this embodiment of the invention, the complex structure of the product is segmented as a whole when generating a transparent image. There is no need to reassemble the individual parts into a whole later. Therefore, the segmentation edges of the product are clear enough, which can meet the edge precision requirements of the transparent image. This ensures that the generated transparent image of the product is complete and has a beautiful outline, and can accurately display the product.

[0141] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0142] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

A method for generating transparent images, characterized in that, include: Obtain the original product image to be processed; The original product image is segmented as a whole using a preset two-class image segmentation model to obtain a segmented product mask image, wherein the product includes at least two individuals. Based on the segmented product body mask image, determine the transparency channel; The transparency channel is combined with the three color channels of the original product image to obtain a transparent image of the product body. The method according to claim 1, characterized in that, Before performing overall segmentation of the product subject in the original product image using a preset binary image segmentation model to obtain the segmented product subject mask image, the method further includes: The original product image is input into a preset completeness assessment model for completeness assessment, and the completeness assessment result corresponding to the original product image is obtained; The step of using a preset two-class image segmentation model to perform overall segmentation of the main product in the original product image to obtain a segmented product mask image includes: If the original product image is determined to be complete based on the completeness assessment result, then the main body of the product in the original product image is segmented as a whole using a preset two-class image segmentation model to obtain a segmented product body mask image. The method according to claim 2, characterized in that, The method further includes: If the original product image is determined to be incomplete based on the completeness assessment result, the original product image is then normalized to obtain a normalized product image. The normalized product image is extended using a preset image extension model to obtain an extended product image. The step of using a preset two-class image segmentation model to perform overall segmentation of the main product in the original product image to obtain a segmented product mask image includes: The extended product image is segmented as a whole using a preset two-class image segmentation model to obtain the segmented product mask image. The method according to claim 1, characterized in that, After determining the transparency channel based on the segmented product body mask image, the method further includes: Detect whether there are noise points at the edges and around the segmented product body mask image; If noise points exist, the segmented product main body mask image is subjected to dilation and / or erosion processing to obtain a noise-removed product main body mask image. The step of determining the transparency channel based on the segmented product body mask image includes: The transparency channel is determined based on the noise-removed mask image of the product. The method according to claim 1, characterized in that, After combining the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body, the method further includes: Based on the segmented product body mask image, determine the bounding box of the product body and the position information of the product body in the segmented product body mask image; Based on the bounding box and the location information, the transparent image is processed to obtain a target transparent image that meets the requirements for the proportion of the main body of the product. The method according to claim 5, characterized in that, The process of processing the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the requirements for the proportion of the main body of the product includes: Determine the longest edge in the bounding box, and generate the target bounding box based on the longest edge; A transparent background is generated based on the size and preset ratio of the target border; Based on the location information, the coordinates of the target bounding box in the transparent image are determined; Based on the coordinates of the target border in the transparent image, the transparent image is cropped to obtain the cropped transparent image; The cropped transparent image is composited onto the transparent background to obtain the target transparent image. The method according to claim 5, characterized in that, After processing the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the requirements for the proportion of the main body of the product, the method further includes: A comprehensive quality assessment is performed on the target transparent image to obtain a comprehensive quality score for the target transparent image; Based on the comprehensive quality score, it is determined whether the target transparent image meets the display requirements; If the target transparent image meets the display requirements, then the target transparent image is added to the preset transparent image resource library. The method according to claim 7, characterized in that, The comprehensive quality assessment of the target transparent image to obtain a comprehensive quality score for the target transparent image includes: The target transparent image is input into a preset quality scoring model for quality scoring to obtain the first quality score of the target transparent image; The visual effect, edge smoothness, and redundant noise of the target transparent image are evaluated to obtain the visual effect evaluation result, edge smoothness evaluation result, and redundant noise evaluation result. Based on the visual effect evaluation results, the edge smoothness evaluation results, and the redundant noise evaluation results, the second quality score, the third quality score, and the fourth quality score are determined respectively. The first quality score, the second quality score, the third quality score, and the fourth quality score are weighted and summed to obtain the comprehensive quality score of the target transparent image. The method according to any one of claims 1-8, characterized in that, Before performing overall segmentation of the product subject in the original product image using a preset binary image segmentation model to obtain the segmented product subject mask image, the method further includes: Obtain the first and second sample training sets; Construct an initial ground truth encoder and an initial binary image segmentation model; The initial truth encoder is trained using the first sample training set to obtain a preset truth encoder; Based on the preset truth encoder and the second sample training set, the initial binary image segmentation model is trained to obtain the preset binary image segmentation model. The method according to claim 9, characterized in that, The first sample training set includes a first ground truth mask image corresponding to a first sample product image. The step of training the initial ground truth encoder using the first sample training set to obtain a preset ground truth encoder includes: The initial ground value encoder is used to encode the depth intermediate features of the ground value in the first ground value mask image to obtain the first ground value code. Based on the first true value encoding and the true values ​​in the first true mask image, a binary cross-entropy loss function is constructed; Based on the binary cross-entropy loss function, the parameters in the initial truth encoder are updated and iterated until the first preset training condition is met, at which point the iteration stops and the preset truth encoder is output. The method according to claim 9, characterized in that, The second sample training set includes second sample product images and their corresponding second ground truth mask images. The step of training the initial binary image segmentation model based on the preset ground truth encoder and the second sample training set to obtain the preset binary image segmentation model includes: The preset truth encoder is used to encode the depth intermediate features of the truth values ​​in the second truth mask image to obtain the second truth value code. The second sample product image is input into the initial binary image segmentation model to segment the product subject, thereby obtaining the segmentation feature code corresponding to the second sample product image; Based on the segmentation feature encoding and the second true value encoding, a mean squared error loss function is constructed; The parameters in the initial binary image segmentation model are updated and iterated according to the mean squared error loss function until the second preset training condition is met, at which point the iteration stops and the preset binary image segmentation model is output. A transparent image generation device, characterized in that, include: The acquisition unit is used to acquire the original product image to be processed; The segmentation unit is used to perform overall segmentation of the main body of the product in the original product image using a preset two-type image segmentation model to obtain a segmented product body mask image, wherein the main body of the product includes at least two individuals; The determining unit is used to determine the transparency channel based on the segmented product body mask image; The compositing unit is used to combine the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body. The apparatus according to claim 12 is characterized in that, The device further includes: The integrity assessment unit is used to input the original product image into a preset integrity assessment model for integrity assessment, and obtain the integrity assessment result corresponding to the original product image; The segmentation unit is specifically used to perform overall segmentation of the main body of the product in the original product image using a preset two-class image segmentation model if the original product image is determined to be complete based on the integrity assessment result, thereby obtaining a segmented product main body mask image. The apparatus according to claim 13 is characterized in that, The device further includes: The extension unit is used to perform normalization processing on the original product image if it is determined that the original product image is incomplete according to the completeness assessment result, to obtain a normalized product image; and to extend the normalized product image using a preset image extension model to obtain an extended product image. The segmentation unit is further specifically used to perform overall segmentation of the main body of the product in the extended product image using a preset two-type image segmentation model, so as to obtain the segmented product main body mask image. The apparatus according to claim 12 is characterized in that, The device further includes: The noise reduction unit is used to detect whether there are noise points on the edges and around the segmented product main body mask image; if there are noise points, the segmented product main body mask image is subjected to dilation and / or erosion processing to obtain a noise-removed product main body mask image. The determining unit is specifically used to determine the transparency channel based on the noise-removed product body mask image. The apparatus according to claim 12 is characterized in that, The device further includes: The determining unit is further configured to determine the bounding box of the product body and the position information of the product body in the segmented product body mask image based on the segmented product body mask image; The post-processing unit is used to process the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the requirements for the proportion of the main body of the product. The apparatus according to claim 16 is characterized in that, The post-processing unit is specifically used to determine the longest side of the bounding box and generate a target border based on the longest side; generate a transparent background based on the size of the target border and a preset ratio; determine the coordinates of the target border in the transparent image based on the position information; crop the transparent image according to the coordinates of the target border in the transparent image to obtain a cropped transparent image; and composite the cropped transparent image onto the transparent background to obtain the target transparent image. The apparatus according to claim 16 is characterized in that, The device further includes: A quality assessment unit is used to perform a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score for the target transparent image; based on the comprehensive quality score, it determines whether the target transparent image meets the display requirements; if the target transparent image meets the display requirements, it adds the target transparent image to a preset transparent image resource library. The apparatus according to claim 18 is characterized in that, The quality assessment unit is specifically used to input the target transparent image into a preset quality scoring model for quality scoring, thereby obtaining a first quality score for the target transparent image; to evaluate the visual effect, edge smoothness, and redundant noise of the target transparent image, thereby obtaining visual effect evaluation results, edge smoothness evaluation results, and redundant noise evaluation results; to determine a second quality score, a third quality score, and a fourth quality score based on the visual effect evaluation results, the edge smoothness evaluation results, and the redundant noise evaluation results, respectively; and to perform a weighted summation of the first quality score, the second quality score, the third quality score, and the fourth quality score to obtain a comprehensive quality score for the target transparent image. The apparatus according to any one of claims 12-19 is characterized in that, The device further includes: The acquisition unit is also used to acquire the first sample training set and the second sample training set; and to construct the initial ground truth encoder and the initial binary image segmentation model. The training unit is used to train the initial truth encoder using the first sample training set to obtain a preset truth encoder. The training unit is also used to train the initial binary image segmentation model based on the preset ground truth encoder and the second sample training set to obtain the preset binary image segmentation model. The apparatus according to claim 20 is characterized in that, The first sample training set includes the first real mask image corresponding to the first sample product image; The training unit is specifically used to encode the depth intermediate features of the ground truth in the first ground truth mask image using the initial ground truth encoder to obtain the first ground truth code; to construct a binary cross-entropy loss function based on the first ground truth code and the ground truth in the first ground truth mask image; and to update and iterate the parameters in the initial ground truth encoder based on the binary cross-entropy loss function until the first preset training condition is met, at which point the iteration stops and the preset ground truth encoder is output. The apparatus according to claim 20 is characterized in that, The second sample training set includes the second sample product images and their corresponding second real mask images; The training unit is further specifically used to encode the depth intermediate features of the ground truth in the second ground truth mask image using the preset ground truth encoder to obtain the second ground truth encoding; input the second sample product image into the initial binary image segmentation model to segment the product subject to obtain the segmentation feature encoding corresponding to the second sample product image; construct a mean squared error loss function based on the segmentation feature encoding and the second ground truth encoding; update and iterate the parameters in the initial binary image segmentation model according to the mean squared error loss function until the second preset training condition is met, then stop the iteration and output the preset binary image segmentation model. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the transparent image generation method according to any one of claims 1 to 11. A computer device includes a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, it implements the transparent image generation method according to any one of claims 1 to 11.

Images