Transparent image generation method and device, storage medium and computer equipment

Abstract

The invention discloses a transparent image generation method and device, a storage medium and computer equipment, and relates to the technical field of computer vision. The method comprises the following steps: acquiring an original commodity image to be processed; integrally segmenting a commodity main body in the original commodity image by using a preset second-class image segmentation model to obtain a segmented commodity main body mask image, the commodity main body comprising at least two individuals; determining a transparency channel according to the segmented commodity main body mask image; and synthesizing the transparency channel and the three-color channel of the original commodity image to obtain a transparent image of the commodity main body. According to the method, the segmentation precision of the commodity main body with the complex structure can be improved, so that the commodity main body in the generated transparent image is clear and definite, and the commodity can be accurately displayed.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a transparent image generation method, device, storage medium and computer equipment. Background Art

[0002] In some scenarios, there are a lot of demands for transparent images of goods. For example, catering merchants need clear transparent images of the main body of the dishes to accurately display the dishes for use in menus, brochures, online ordering, etc. For another example, enterprises can use transparent images to accurately identify and classify dishes to help the catering management system perform tasks such as dish inventory management and sales analysis.

[0003] At present, when generating a transparent image of a product, each individual in the original product image is usually segmented separately, and then the individuals are combined to generate a transparent image. However, many products have a complex structure, including multiple individuals, and the entire product body is generally displayed as a whole when displayed. The existing method of first segmenting and then combining will result in unclear segmentation edges of the product body, which is difficult to meet the edge fineness requirements of the transparent image, resulting in unclear and unambiguous product bodies in the transparent image, and the inability to accurately display the product. Summary of the invention

[0004] In view of this, the present application provides a transparent image generation method, device, storage medium and computer equipment, the main purpose of which is to improve the segmentation accuracy of complex structure commodity bodies, so as to ensure that the commodity bodies in the generated transparent images are clear and can accurately display the commodities.

[0005] According to a first aspect of the present application, a method for generating a transparent image is provided, the method comprising:

[0006] Obtaining the original product image to be processed;

[0007] Using a preset two-category image segmentation model, the commodity body in the original commodity image is segmented as a whole to obtain a segmented commodity body mask image, wherein the commodity body includes at least two individuals;

[0008] Determining a transparency channel according to the segmented commodity main body mask image;

[0009] The transparency channel is synthesized with the three color channels of the original product image to obtain a transparent image of the product body.

[0010] Optionally, before using the preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole to obtain a segmented commodity body mask image, the method further includes:

[0011] Inputting the original product image into a preset integrity assessment model for integrity assessment, and obtaining an integrity assessment result corresponding to the original product image;

[0012] The method of using the preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole to obtain a segmented commodity body mask image includes:

[0013] If it is determined that the original product image is complete according to the completeness evaluation result, the product body in the original product image is segmented as a whole using a preset two-category image segmentation model to obtain a segmented product body mask image.

[0014] Optionally, the method further comprises:

[0015] If it is determined according to the completeness evaluation result that the original product image is incomplete, normalizing the original product image to obtain a normalized product image;

[0016] Using a preset image extension model to extend the normalized product image to obtain an extended product image;

[0017] The method of using the preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole to obtain a segmented commodity body mask image includes:

[0018] The commodity body in the extended commodity image is segmented as a whole using a preset two-category image segmentation model to obtain a segmented commodity body mask image.

[0019] Optionally, after determining the transparency channel according to the segmented commodity main body mask image, the method further includes:

[0020] Detecting whether there are noise points at the edge and around the segmented commodity main body mask image;

[0021] If there are noise points, dilation processing and / or corrosion processing are performed on the segmented commodity main body mask image to obtain a commodity main body mask image after noise removal;

[0022] The step of determining a transparency channel according to the segmented commodity main body mask image comprises:

[0023] The transparency channel is determined according to the mask image of the commodity body after noise is removed.

[0024] Optionally, after synthesizing the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body, the method further includes:

[0025] Determine, according to the segmented commodity body mask image, a boundary box of the commodity body and position information of the commodity body in the segmented commodity body mask image;

[0026] Based on the bounding box and the position information, the transparent image is processed to obtain a target transparent image that meets the requirement of product main body proportion.

[0027] Optionally, the processing of the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the requirement of the product main body proportion includes:

[0028] Determine the longest side in the bounding box, and generate a target border based on the longest side;

[0029] Generate a transparent background based on the size of the target frame and a preset ratio;

[0030] Based on the position information, determining the coordinates of the target frame in the transparent image;

[0031] Clipping the transparent image according to the coordinates of the target frame in the transparent image to obtain a clipped transparent image;

[0032] The cropped transparent image is synthesized onto the transparent background to obtain the target transparent image.

[0033] Optionally, after the transparent image is processed based on the bounding box and the position information to obtain a target transparent image that meets the requirement of the product main body proportion, the method further includes:

[0034] Performing a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score of the target transparent image;

[0035] Based on the comprehensive quality score, determining whether the target transparent image meets the display requirements;

[0036] If the target transparent image meets the display requirements, the target transparent image is added to the preset transparent image resource library.

[0037] Optionally, performing a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score of the target transparent image includes:

[0038] Inputting the target transparent image into a preset quality scoring model for quality scoring, to obtain a first quality score of the target transparent image;

[0039] Evaluate the visual effect, edge smoothness and excess noise of the target transparent image to obtain a visual effect evaluation result, an edge smoothness evaluation result and an excess noise evaluation result;

[0040] Determine a second quality score, a third quality score, and a fourth quality score respectively according to the visual effect evaluation result, the edge smoothness evaluation result, and the excess noise evaluation result;

[0041] A weighted sum is performed on the first quality score, the second quality score, the third quality score, and the fourth quality score to obtain a comprehensive quality score of the target transparent image.

[0042] Optionally, before using the preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole to obtain a segmented commodity body mask image, the method further includes:

[0043] Obtain a first sample training set and a second sample training set;

[0044] Construct an initial truth encoder and an initial two-class image segmentation model;

[0045] Using the first sample training set to train the initial truth value encoder to obtain a preset truth value encoder;

[0046] Based on the preset truth value encoder and the second sample training set, the initial two-category image segmentation model is trained to obtain the preset two-category image segmentation model.

[0047] Optionally, the first sample training set includes a first real mask image corresponding to the first sample product image, and the initial truth value encoder is trained using the first sample training set to obtain a preset truth value encoder, including

[0048] Encoding the deep intermediate features of the real value in the first real mask image using the initial real value encoder to obtain a first real value encoding;

[0049] constructing a binary cross entropy loss function according to the first true value encoding and the true value in the first true mask image;

[0050] Based on the binary cross entropy loss function, the parameters in the initial truth value encoder are updated and iterated until a first preset training condition is met, then the iteration is stopped and the preset truth value encoder is output.

[0051] Optionally, the second sample training set includes a second sample product image and a second real mask image corresponding thereto, and the initial two-category image segmentation model is trained based on the preset truth encoder and the second sample training set to obtain the preset two-category image segmentation model, including:

[0052] Encoding the deep intermediate features of the real value in the second real mask image using the preset real value encoder to obtain a second real value encoding;

[0053] Inputting the second sample product image into the initial two-category image segmentation model to perform product main body segmentation, and obtaining a segmentation feature code corresponding to the second sample product image;

[0054] Constructing a mean square error loss function based on the segmentation feature code and the second true value code;

[0055] According to the mean square error loss function, the parameters in the initial two-category image segmentation model are updated and iterated until a second preset training condition is met, then the iteration is stopped and the preset two-category image segmentation model is output.

[0056] According to a second aspect of the present application, a transparent image generating device is provided, comprising:

[0057] An acquisition unit, used for acquiring an original product image to be processed;

[0058] a segmentation unit, configured to segment the commodity body in the original commodity image as a whole by using a preset two-category image segmentation model to obtain a segmented commodity body mask image, wherein the commodity body includes at least two individuals;

[0059] A determination unit, configured to determine a transparency channel according to the segmented commodity body mask image;

[0060] The synthesis unit is used to synthesize the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

[0061] Optionally, the device further comprises:

[0062] An integrity evaluation unit, configured to input the original product image into a preset integrity evaluation model for integrity evaluation, and obtain an integrity evaluation result corresponding to the original product image;

[0063] The segmentation unit is specifically configured to, if it is determined according to the completeness evaluation result that the original product image is complete, use a preset two-category image segmentation model to segment the product body in the original product image as a whole to obtain a segmented product body mask image.

[0064] Optionally, the device further comprises:

[0065] an extension unit, configured to, if it is determined according to the completeness evaluation result that the original product image is incomplete, perform normalization processing on the original product image to obtain a normalized product image; and perform extension processing on the normalized product image using a preset image extension model to obtain an extended product image;

[0066] The segmentation unit is further specifically configured to use a preset two-category image segmentation model to perform overall segmentation on the commodity body in the extended commodity image to obtain the segmented commodity body mask image.

[0067] Optionally, the device further comprises:

[0068] a denoising unit, configured to detect whether there are noise points at the edge and around the segmented commodity main body mask image; if there are noise points, dilation processing and / or corrosion processing is performed on the segmented commodity main body mask image to obtain a commodity main body mask image after noise removal;

[0069] The determining unit is specifically configured to determine the transparency channel according to the mask image of the commodity body after noise is removed.

[0070] Optionally, the device further comprises:

[0071] The determining unit is further configured to determine, according to the segmented commodity body mask image, a boundary box of the commodity body and position information of the commodity body in the segmented commodity body mask image;

[0072] A post-processing unit is used to process the transparent image based on the boundary box and the position information to obtain a target transparent image that meets the requirements for the proportion of the main body of the product.

[0073] Optionally, the post-processing unit is specifically used to determine the longest side in the bounding box, and generate a target border based on the longest side; generate a transparent background based on the size of the target border and a preset ratio; determine the coordinates of the target border in the transparent image based on the position information; crop the transparent image according to the coordinates of the target border in the transparent image to obtain a cropped transparent image; and synthesize the cropped transparent image onto the transparent background to obtain the target transparent image.

[0074] Optionally, the device further comprises:

[0075] A quality assessment unit is used to perform a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score of the target transparent image; based on the comprehensive quality score, determine whether the target transparent image meets the display requirements; if the target transparent image meets the display requirements, add the target transparent image to a preset transparent image resource library.

[0076] Optionally, the quality assessment unit is specifically used to input the target transparent image into a preset quality scoring model for quality scoring to obtain a first quality score of the target transparent image; evaluate the visual effect, edge smoothness and excess noise of the target transparent image to obtain a visual effect evaluation result, an edge smoothness evaluation result and an excess noise evaluation result; determine a second quality score, a third quality score and a fourth quality score according to the visual effect evaluation result, the edge smoothness evaluation result and the excess noise evaluation result, respectively; and perform weighted summation of the first quality score, the second quality score, the third quality score and the fourth quality score to obtain a comprehensive quality score of the target transparent image.

[0077] Optionally, the device further comprises:

[0078] The acquisition unit is further used to acquire a first sample training set and a second sample training set; construct an initial truth value encoder and an initial two-category image segmentation model;

[0079] A training unit, configured to train the initial truth value encoder using the first sample training set to obtain a preset truth value encoder;

[0080] The training unit is further used to train the initial two-category image segmentation model based on the preset truth value encoder and the second sample training set to obtain the preset two-category image segmentation model.

[0081] Optionally, the first sample training set includes a first real mask image corresponding to a first sample product image; the training unit is specifically used to use the initial truth value encoder to encode the deep intermediate features of the real value in the first real mask image to obtain a first real value code; construct a binary cross entropy loss function based on the first real value code and the real value in the first real mask image; based on the binary cross entropy loss function, update and iterate the parameters in the initial truth value encoder until a first preset training condition is met, stop the iteration, and output the preset truth value encoder.

[0082] Optionally, the second sample training set includes a second sample product image and its corresponding second real mask image; the training unit is further specifically used to use the preset true value encoder to encode the deep intermediate features of the real value in the second real mask image to obtain a second real value code; the second sample product image is input into the initial two-category image segmentation model to perform product body segmentation to obtain a segmentation feature code corresponding to the second sample product image; based on the segmentation feature code and the second real value code, a mean square error loss function is constructed; according to the mean square error loss function, the parameters in the initial two-category image segmentation model are updated and iterated until the second preset training condition is met, the iteration is stopped, and the preset two-category image segmentation model is output.

[0083] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned transparent image generating method is implemented.

[0084] According to a fourth aspect of the present application, a computer device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned transparent image generation method when executing the computer program.

[0085] By means of the above technical solution, the transparent image generation method, device, storage medium and computer equipment provided by the present application, compared with the prior art, uses a preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole, and determines the transparency channel according to the mask image of the commodity body after the overall segmentation, and can generate a transparent image of the commodity body according to the transparency channel and the three-color channels of the original commodity image. Since the present application segments the commodity body with a complex structure as a whole when generating a transparent image, there is no problem of subsequent re-combination of the individual wholes, so it can ensure that the segmented edges of the commodity body are clear enough, that is, it can meet the edge fineness requirements of the transparent image, thereby ensuring that the generated transparent image commodity body is complete and beautiful in outline, and can accurately display the commodity.

[0086] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0088] Figure 1 A schematic diagram of a transparent image generation method provided in an embodiment of the present application is shown;

[0089] Figure 2 A schematic diagram of the reasoning framework provided by an embodiment of the present application is shown;

[0090] Figure 3 A schematic diagram showing a flow chart of another transparent image generation method provided in an embodiment of the present application;

[0091] Figure 4 A comparison diagram before and after the cutting and pasting operation provided in an embodiment of the present application is shown;

[0092] Figure 5 The overall process diagram of transparent image generation provided by the embodiment of the present application is shown;

[0093] Figure 6 A schematic diagram of the training process of the two-type image segmentation model provided in an embodiment of the present application is shown;

[0094] Figure 7 A schematic diagram of a training framework provided in an embodiment of the present application is shown;

[0095] Figure 8 A schematic structural diagram of a transparent image generating device provided in an embodiment of the present application is shown;

[0096] Fig. 9 A schematic structural diagram of another transparent image generating device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0097] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0098] The existing technology will cause the segmentation edge of the commodity body to be unclear, which makes it difficult to meet the edge fineness requirements of the transparent image, thereby causing the commodity body in the transparent image to be unclear and unambiguous, making it impossible to accurately display the commodity.

[0099] In order to overcome the above technical problems. This embodiment provides a transparent image generation method, such as Figure 1 As shown, the method includes:

[0100] Step 101: Obtain the original product image to be processed.

[0101] The original product image to be processed contains an interfering background, and the product subject in the original product image can be a dish, daily necessities, electronic products, sports goods, etc. The product subject includes at least two individuals. For example, when the product subject is a dish (a plate of fried pork with peppers), the product subject includes a fried pork with peppers individual and a container individual. It should be noted that the embodiment of the present invention does not specifically limit the type of the product subject.

[0102] According to the embodiment of the present invention, the commodity body can be photographed or downloaded from the Internet to obtain the original commodity image to be processed.

[0103] Step 102: Use a preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole to obtain a segmented commodity body mask image.

[0104] The commodity body includes at least two individuals. The preset two-class image segmentation model is a pre-trained model that can separate the commodity body from the interference background, which can be a DIS-Net model or other two-class image segmentation models, and the present invention does not make specific limitations on this.

[0105] In order to avoid segmenting and recombining the individuals contained in the commodity body, thereby affecting the segmentation effect, the embodiment of the present invention directly segments and encodes the commodity body as a whole, that is, each individual in the commodity body is no longer segmented separately. Specifically, the original commodity image is input into a preset two-class image segmentation model, and the commodity body is segmented as a whole to obtain a segmented commodity body mask image, that is, a mask image of the commodity body.

[0106] The embodiment of the present invention uses the preset two-type image segmentation model to perform overall segmentation on the commodity body, thereby ensuring the clarity of the segmentation edge of the commodity body, thereby improving the segmentation effect of the transparent image.

[0107] Step 103: Determine a transparency channel according to the segmented commodity main body mask image.

[0108] According to the embodiment of the present invention, after the segmented commodity main body mask image is obtained, the segmented commodity main body mask image is directly used as a transparency channel, that is, an alpha channel.

[0109] Step 104: synthesize the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

[0110] The three color channels of the original product image are RGB channels, and the transparency channel is the alpha channel. The transparent image is the image from which the interfering background has been removed.

[0111] For the embodiment of the present invention, the overall reasoning process based on the preset two-class image segmentation model is as follows: Figure 2 As shown, after the transparent channel is determined, the three color channels of the original product image are synthesized with the transparency channel to obtain a four-channel transparent image, and the four-channel transparent image is used as the transparent image of the product body.

[0112] The present embodiment provides a method for generating a transparent image, which uses a preset two-class image segmentation model to segment the main body of the product in the original product image as a whole, and determines the transparency channel according to the mask image of the main body of the product after the overall segmentation, and can generate a transparent image of the main body of the product according to the transparency channel and the three-color channels of the original product image. Since the embodiment of the present invention segments the main body of the product with a complex structure as a whole when generating a transparent image, there is no problem of subsequently re-combining the individual as a whole, so it can ensure that the segmented edge of the main body of the product is clear enough, that is, it can meet the edge fineness requirements of the transparent image, so that it can ensure that the main body of the generated transparent image is complete and beautiful in outline, and can accurately display the product.

[0113] Furthermore, as a refinement and extension of the above embodiment, the embodiment of the present invention also provides another transparent image generation method, such as Figure 3 As shown, including:

[0114] Step 201: Obtain the original product image to be processed.

[0115] According to the embodiment of the present invention, the commodity body can be photographed or downloaded from the Internet to obtain the original commodity image to be processed, and the original commodity image to be processed contains an interfering background.

[0116] Step 202: Input the original product image into a preset integrity assessment model to perform integrity assessment, and obtain an integrity assessment result corresponding to the original product image.

[0117] The preset integrity assessment model may be an Alpha-CLIP model or other models capable of integrity assessment, which is not specifically limited in the embodiment of the present invention. The integrity assessment result may include whether the original product image is complete or incomplete.

[0118] According to the embodiment of the present invention, before the original commodity image is segmented, it is necessary to firstly evaluate the integrity of the original commodity image to ensure that the subsequently generated transparent image can contain the complete commodity body.

[0119] Specifically, the training images can be marked as "complete" and "incomplete" based on the completeness of the commodity body in the training images, and then the initial completeness assessment model is trained based on the marked training images to obtain a preset completeness assessment model. When conducting a formal assessment, the preset completeness assessment model can output the probability value of whether the original commodity image is complete or incomplete, and can ultimately determine whether the original commodity image is complete through the set probability threshold.

[0120] For example, the probability that the original product image is complete output by the preset completeness assessment model is 0.7, and the preset probability threshold is 0.5. Since the predicted probability value of the original product image being complete is greater than the preset probability threshold, it is ultimately determined that the original product image is complete.

[0121] The embodiment of the present invention can ensure the integrity of the main body of the commodity by evaluating the integrity of the original commodity image, thereby ensuring that the subsequently generated transparent image can accurately display the main body of the commodity.

[0122] Step 203a: If it is determined that the original product image is complete according to the completeness evaluation result, the product body in the original product image is segmented as a whole using a preset two-category image segmentation model to obtain a segmented product body mask image.

[0123] In the embodiment of the present invention, if the original product image is determined to be complete according to the integrity evaluation result, the original product image is input into the preset two-category image segmentation model for segmentation encoding to obtain a segmented product main body mask image, thereby ensuring the integrity of the segmented product main body.

[0124] Step 203b: If it is determined that the original product image is incomplete according to the completeness evaluation result, the original product image is normalized to obtain a normalized product image, and the normalized product image is extended using a preset image extension model to obtain an extended product image.

[0125] The preset image extension model may be a Stable-Diffusion model, which is a generation model based on the diffusion process. It can generate high-quality and detailed images through a step-by-step denoising method. The preset image extension model may also be other extension models with diffusion capabilities, which are not specifically limited in the embodiments of the present invention. The main body of the product in the extended product image is complete.

[0126] For the embodiment of the present invention, in order to perform an extension operation on the incomplete original product image to ensure that the product body in the original product image is complete, it is necessary to pre-train the image extension model. During training, a large number of complete and incomplete sample images can be used to train the initial image extension model, so as to obtain a preset image extension model. After the preset image extension model is trained, if it is determined that the original product image is incomplete according to the completeness evaluation result, it is necessary to use the preset image extension model to perform extension processing on the original product image. Before the extension processing, the original product image is first normalized, such as adjusting the size of the original product image to the size required by the model, and performing color normalization at the same time to ensure that the original product image is in the same spatial structure and color space. After obtaining the product image after the normalization processing, it is input into the preset image extension model to extend the graphic content to ensure that the product body structure in the original product image is complete. For example, for a plate of fried pork with peppers, the fried pork with peppers and the container can be fully presented in the image.

[0127] The embodiment of the present invention uses a preset image extension model to extend the original product image, thereby ensuring that the product body is fully presented in the transparent image, so as to achieve accurate display of the product body.

[0128] Step 204: Use a preset two-category image segmentation model to segment the commodity body in the extended commodity image as a whole, and obtain a segmented commodity body mask image.

[0129] According to the embodiment of the present invention, after the original product image is evaluated for completeness and stretched, the stretched product image is input into a preset two-category image segmentation model to segment the product body as a whole, thereby obtaining a segmented product body mask image.

[0130] Step 205: Determine a transparency channel according to the segmented commodity main body mask image.

[0131] For the embodiment of the present invention, in order to ensure the quality of the segmented commodity main body mask image, it is necessary to perform noise point detection. For this process, step 205 specifically includes: detecting whether there are noise points at the edge and around the segmented commodity main body mask image; if there are noise points, dilating and / or corroding the segmented commodity main body mask image to obtain the commodity main body mask image after noise removal. At the same time, according to the commodity main body mask image after noise removal, the transparency channel is determined.

[0132] Specifically, an edge detection or contour detection algorithm can be used to detect whether there are redundant noise points, i.e., "island" points, at the edge and around the segmented product main body mask image. If there are noise points, the noise points in small areas can be removed by dilation and / or erosion operations to ensure that the main body edge of the subsequently generated transparent image is smoother and has fewer irrelevant pixels.

[0133] Step 206: synthesize the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

[0134] After the transparent channel is determined, the three color channels of the original product image are synthesized with the transparency channel to obtain a four-channel transparent image, and the four-channel transparent image is used as the transparent image of the product body.

[0135] Step 207: Determine the boundary box of the commodity body and the position information of the commodity body in the segmented commodity body mask image according to the segmented commodity body mask image.

[0136] In order to make the proportion of the main body of the product in the final generated transparent image meet the requirements, the embodiment of the present invention needs to cut and paste the image. When performing the cutting and pasting operations, it is necessary to first determine the bounding box of the main body of the product and the position information of the main body of the product in the image. Specifically, the cv2.boundingRect built-in function can be used to treat the segmented main body mask image as a point set, so that the minimum circumscribed rectangle of the main body of the product, that is, the bounding box, can be calculated. At the same time, according to the segmented main body mask image of the product, the position information of the main body of the product in the image can be determined.

[0137] Step 208: Based on the bounding box and the position information, the transparent image is processed to obtain a target transparent image that meets the requirement of the product main body proportion.

[0138] For the embodiment of the present invention, in order to generate a target transparent image that meets the requirements for the proportion of the main body of the product, step 208 specifically includes: determining the longest side in the boundary box, and generating a target border based on the longest side; generating a transparent background based on the size of the target border and a preset ratio; determining the coordinates of the target border in the transparent image based on the position information; cropping the transparent image according to the coordinates of the target border in the transparent image to obtain a cropped transparent image; and synthesizing the cropped transparent image onto the transparent background to obtain the target transparent image.

[0139] The target frame may be a square frame or a frame of other shapes. The preset ratio may be set according to actual needs.

[0140] Specifically, the target frame is enlarged according to a preset ratio to generate a transparent background with a suitable size ratio. In addition, according to the position information of the main body of the product in the image, the coordinates of the target frame in the transparent image can be determined, and then the transparent image is cropped based on the coordinates, and the cropped transparent image is pasted on the transparent background to obtain the final target transparent image, which is saved in a unified format to meet the needs of merchants to display products, such as Figure 4 As shown, the left image is the transparent image before clipping, and the right image is the transparent image after clipping.

[0141] The embodiment of the present invention can make the main body of the commodity meet the proportion requirement in the transparent image by cutting and pasting the generated transparent image, thereby realizing accurate display of the commodity.

[0142] In some embodiments, in order to make the transparent image to be displayed meet the merchant's display requirements for the main body of the product, the embodiment of the present invention can also perform a comprehensive quality assessment on the acquired target transparent image, such as Figure 5 For this process, the method includes: performing a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score of the target transparent image; based on the comprehensive quality score, determining whether the target transparent image meets the display requirements; if the target transparent image meets the display requirements, adding the target transparent image to a preset transparent image resource library.

[0143] When determining the comprehensive quality score of the target transparent image, the target transparent image is input into a preset quality scoring model for quality scoring to obtain a first quality score of the target transparent image; the visual effect, edge smoothness and excess noise of the target transparent image are evaluated to obtain a visual effect evaluation result, an edge smoothness evaluation result and an excess noise evaluation result; a second quality score, a third quality score and a fourth quality score are determined respectively according to the visual effect evaluation result, the edge smoothness evaluation result and the excess noise evaluation result; the first quality score, the second quality score, the third quality score and the fourth quality score are weightedly summed to obtain a comprehensive quality score of the target transparent image.

[0144] The preset quality scoring model may be a binary classification model, such as a support vector machine and a neural network. The preset quality scoring model may classify the target transparent image into a high-quality image and a low-quality image. The weights corresponding to the first quality score, the second quality score, the third quality score, and the fourth quality score may be set according to actual needs. In an embodiment of the present invention, the weight of the first quality score may be set to the maximum.

[0145] Specifically, the target transparent image is input into a preset quality scoring model for quality averaging, and a probability value of the target transparent image being a high-quality image is obtained, and the probability value range is between 0 and 1. The first quality score can be determined based on the probability value. At the same time, the visual effect, edge smoothness, and excess noise of the target transparent image are evaluated. The better the evaluation effect, the higher the corresponding score. Finally, all quality scores are weighted and summed to obtain a comprehensive quality score of the target transparent image.

[0146] After determining the comprehensive quality score of the target transparent image, it is determined whether the comprehensive quality score of the target transparent image is greater than a preset score. If the comprehensive quality score is greater than the preset score, it is determined that the target transparent image meets the display requirements and is added to the preset transparent image resource library; on the contrary, if the comprehensive quality score is less than or equal to the preset score, it is determined that the target transparent image does not meet the display requirements. The preset score can be set according to actual needs.

[0147] Therefore, by comprehensively scoring the quality of the target transparent images, high-quality transparent images that meet actual needs can be screened out as transparent image library resources for merchants.

[0148] Furthermore, the embodiment of the present invention also provides a training method for presetting two types of image segmentation models, such as Figure 6 As shown, including:

[0149] Step 301: Obtain a first sample training set and a second sample training set.

[0150] The first sample training set includes a first sample product image and a corresponding first real mask image, and the second sample training set includes a second sample product image and a corresponding second real mask image.

[0151] In the embodiment of the present invention, the first sample training set and the second sample training set are different training sets, and the amount of data in the first sample training set is smaller than the amount of data in the second sample training set.

[0152] Step 302: construct an initial truth encoder and an initial two-category image segmentation model.

[0153] Among them, the initial two-category image segmentation model can specifically be a DIS-Net model, or it can be other two-category image segmentation models, and the present invention does not make specific limitations on this.

[0154] According to the embodiment of the present invention, the parameters in the truth value encoder and the two-category image segmentation model are initialized, and the initial truth value encoder and the initial two-category image segmentation model are constructed.

[0155] Step 303: Use the first sample training set to train the initial truth value encoder to obtain a preset truth value encoder.

[0156] The embodiment of the present invention divides the entire training process into two stages. The first stage is to train the truth encoder separately. The second stage is to train the two-class image segmentation model based on the trained truth encoder. Figure 7 As shown. For the first stage of the training process, step 303 specifically includes: using the initial truth value encoder to encode the deep intermediate features of the real value in the first real mask image to obtain a first real value code; constructing a binary cross entropy loss function (BCE) according to the first real value code and the real value in the first real mask image; based on the binary cross entropy loss function, updating and iterating the parameters in the initial truth value encoder until the first preset training condition is met, stopping the iteration, and outputting the preset truth value encoder.

[0157] Among them, the preset first preset training condition can be set according to actual needs, such as stopping iteration when a preset number of iterations is reached.

[0158] Step 304: Based on the preset truth value encoder and the second sample training set, the initial two-category image segmentation model is trained to obtain the preset two-category image segmentation model.

[0159] For the embodiment of the present invention, for the second stage of the training process, step 304 specifically includes: using the preset truth value encoder to encode the deep intermediate features of the true value in the second real mask image to obtain a second true value code; inputting the second sample product image into the initial two-category image segmentation model to perform product body segmentation to obtain the segmentation feature code corresponding to the second sample product image; constructing a mean square error loss function (MSE) based on the segmentation feature code and the second true value code; according to the mean square error loss function, updating and iterating the parameters in the initial two-category image segmentation model until the second preset training condition is met, stopping the iteration, and outputting the preset two-category image segmentation model.

[0160] Among them, the preset second preset training condition can be set according to actual needs, such as stopping iteration when a preset number of iterations is reached.

[0161] The embodiment of the present invention introduces an intermediate supervised segmentation model training method, which can synchronize the deep intermediate features by pre-training a truth encoder to extract the deep intermediate features of the real value and calculating the consistency loss between the deep intermediate features generated by the model processing the input image during the training process and the deep intermediate features of the real value. The diversity of the deep intermediate features can provide stronger regularity and more comprehensive supervision, thereby reducing the risk of overfitting.

[0162] Further, as Figure 1 , Figure 3 and Figure 6 The specific implementation of the method shown in the embodiment provides a transparent image generating device, such as Figure 8 As shown, the device includes: an acquisition unit 31, a segmentation unit 32, a determination unit 33 and a synthesis unit 34.

[0163] The acquisition unit 31 may be used to acquire the original commodity image to be processed.

[0164] The segmentation unit 32 may be configured to segment the commodity body in the original commodity image as a whole using a preset two-category image segmentation model to obtain a segmented commodity body mask image, wherein the commodity body includes at least two individuals.

[0165] The determining unit 33 may be configured to determine a transparency channel according to the segmented commodity body mask image.

[0166] The synthesis unit 34 may be used to synthesize the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

[0167] In some embodiments, Fig. 9 As shown, the device further includes: an integrity evaluation unit 35.

[0168] The integrity evaluation unit 35 may be used to input the original product image into a preset integrity evaluation model to perform integrity evaluation, and obtain an integrity evaluation result corresponding to the original product image.

[0169] The segmentation unit 32 may be specifically configured to, if it is determined according to the completeness evaluation result that the original product image is complete, use a preset two-category image segmentation model to segment the product body in the original product image as a whole, and obtain a segmented product body mask image.

[0170] In some embodiments, the device further includes: an extension unit 36 ​​.

[0171] The extension unit 36 ​​can be used to normalize the original product image to obtain a normalized product image if it is determined that the original product image is incomplete according to the completeness evaluation result; and use a preset image extension model to extend the normalized product image to obtain an extended product image.

[0172] The segmentation unit 32 may also be specifically configured to segment the commodity body in the extended commodity image as a whole by using a preset two-category image segmentation model to obtain the segmented commodity body mask image.

[0173] In some embodiments, the device further includes: a denoising unit 37 .

[0174] The denoising unit 37 may be used to detect whether there are noise points at the edge and around the segmented commodity main body mask image; if there are noise points, the segmented commodity main body mask image is subjected to dilation processing and / or corrosion processing to obtain a commodity main body mask image after noise removal.

[0175] The determining unit 33 may be specifically configured to determine the transparency channel according to the product main body mask image after noise is removed.

[0176] In some embodiments, the device further includes: a post-processing unit 38 .

[0177] The determining unit 33 may also be configured to determine a boundary box of the commodity body and position information of the commodity body in the segmented commodity body mask image according to the segmented commodity body mask image.

[0178] The post-processing unit 38 may be configured to process the transparent image based on the boundary box and the position information to obtain a target transparent image that meets the requirements for the product main body ratio.

[0179] In some embodiments, the post-processing unit 38 can be specifically used to determine the longest side in the bounding box, and generate a target border based on the longest side; generate a transparent background based on the size of the target border and a preset ratio; determine the coordinates of the target border in the transparent image based on the position information; crop the transparent image according to the coordinates of the target border in the transparent image to obtain a cropped transparent image; and synthesize the cropped transparent image onto the transparent background to obtain the target transparent image.

[0180] In some embodiments, the apparatus further comprises: a quality assessment unit 39 .

[0181] The quality assessment unit 39 can be used to perform a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score of the target transparent image; based on the comprehensive quality score, determine whether the target transparent image meets the display requirements; if the target transparent image meets the display requirements, add the target transparent image to a preset transparent image resource library.

[0182] In some embodiments, the quality assessment unit 39 can be specifically used to input the target transparent image into a preset quality scoring model for quality scoring to obtain a first quality score of the target transparent image; evaluate the visual effect, edge smoothness and excess noise of the target transparent image to obtain a visual effect evaluation result, an edge smoothness evaluation result and an excess noise evaluation result; determine a second quality score, a third quality score and a fourth quality score respectively according to the visual effect evaluation result, the edge smoothness evaluation result and the excess noise evaluation result; and perform weighted summation of the first quality score, the second quality score, the third quality score and the fourth quality score to obtain a comprehensive quality score of the target transparent image.

[0183] In some embodiments, the apparatus further includes: a training unit 310 .

[0184] The acquisition unit 31 may also be used to acquire a first sample training set and a second sample training set; and construct an initial truth value encoder and an initial two-category image segmentation model.

[0185] The training unit 310 may be configured to train the initial truth value encoder using the first sample training set to obtain a preset truth value encoder.

[0186] The training unit 310 may also be configured to train the initial two-category image segmentation model based on the preset truth value encoder and the second sample training set to obtain the preset two-category image segmentation model.

[0187] In some embodiments, the first sample training set includes a first real mask image corresponding to the first sample product image. The training unit 310 can be specifically used to encode the deep intermediate features of the real value in the first real mask image using the initial true value encoder to obtain a first real value encoding; construct a binary cross entropy loss function based on the first real value encoding and the real value in the first real mask image; based on the binary cross entropy loss function, update and iterate the parameters in the initial true value encoder until the first preset training condition is met, stop the iteration, and output the preset true value encoder.

[0188] In some embodiments, the second sample training set includes a second sample product image and its corresponding second real mask image. The training unit 310 can also be specifically used to encode the deep intermediate features of the real value in the second real mask image using the preset true value encoder to obtain a second real value code; input the second sample product image into the initial two-category image segmentation model to perform product body segmentation to obtain the segmentation feature code corresponding to the second sample product image; construct a mean square error loss function based on the segmentation feature code and the second real value code; according to the mean square error loss function, update and iterate the parameters in the initial two-category image segmentation model until the second preset training condition is met, stop the iteration, and output the preset two-category image segmentation model.

[0189] It should be noted that for other corresponding descriptions of the functional units involved in the transparent image generating device provided in this embodiment, reference can be made to Figure 1 , Figure 3 and Figure 6 The corresponding description in will not be repeated here.

[0190] Based on the above Figure 1 , Figure 3 and Figure 6 The method shown in the embodiment of the present application is accordingly provided with a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 , Figure 3 and Figure 6 The transparent image generation method shown.

[0191] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0192] Based on the above Figure 1 , Figure 3 and Figure 6 The method shown, and Figure 8 and 9 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a computer device, which can be a tablet computer, a smart phone, a smart watch, a smart bracelet, or other network devices, etc. The computer device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 , Figure 3 and Figure 6The transparent image generation method shown.

[0193] Optionally, the above-mentioned physical devices may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0194] Those skilled in the art will appreciate that the computer device structure provided in this embodiment does not constitute a limitation on such physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0195] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the two physical devices mentioned above, and supports the operation of information processing programs and other software and / or programs. The network communication module is used to realize communication between the components inside the storage medium, and communication with other hardware and software in the information processing physical device.

[0196] When generating a transparent image, the embodiment of the present invention divides the main body of a product with a complex structure as a whole, and there is no problem of subsequently re-combining the individual parts as a whole. Therefore, it can ensure that the segmented edges of the main body of the product are clear enough, that is, it can meet the edge fineness requirements of the transparent image, thereby ensuring that the main body of the product in the generated transparent image is complete and has a beautiful outline, and can accurately display the product.

[0197] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0198] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A method for generating a transparent image, characterized in that: include: Obtaining the original product image to be processed; Using a preset two-category image segmentation model, the commodity body in the original commodity image is segmented as a whole to obtain a segmented commodity body mask image, wherein the commodity body includes at least two individuals; Determining a transparency channel according to the segmented commodity main body mask image; The transparency channel is synthesized with the three color channels of the original product image to obtain a transparent image of the product body.

2. The method according to claim 1, characterized in that: Before the method uses the preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole to obtain a segmented commodity body mask image, the method further includes: Inputting the original product image into a preset integrity assessment model for integrity assessment, and obtaining an integrity assessment result corresponding to the original product image; The method of using the preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole to obtain a segmented commodity body mask image includes: If it is determined that the original product image is complete according to the completeness evaluation result, the product body in the original product image is segmented as a whole using a preset two-category image segmentation model to obtain a segmented product body mask image.

3. The method according to claim 2, characterized in that The method further comprises: If it is determined according to the completeness evaluation result that the original product image is incomplete, normalizing the original product image to obtain a normalized product image; Using a preset image extension model to extend the normalized product image to obtain an extended product image; The method of using the preset two-category image segmentation model to segment the commodity body in the original commodity image as a whole to obtain a segmented commodity body mask image includes: The commodity body in the extended commodity image is segmented as a whole using a preset two-category image segmentation model to obtain a segmented commodity body mask image.

4. The method according to claim 1, characterized in that: After determining the transparency channel according to the segmented commodity main body mask image, the method further includes: Detecting whether there are noise points at the edge and around the segmented commodity main body mask image; If there are noise points, dilation processing and / or corrosion processing are performed on the segmented commodity main body mask image to obtain a commodity main body mask image after noise removal; The step of determining a transparency channel according to the segmented commodity main body mask image comprises: The transparency channel is determined according to the mask image of the commodity body after noise is removed.

5. The method according to claim 1, characterized in that: After synthesizing the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body, the method further includes: Determine, according to the segmented commodity body mask image, a boundary box of the commodity body and position information of the commodity body in the segmented commodity body mask image; Based on the bounding box and the position information, the transparent image is processed to obtain a target transparent image that meets the requirement of product main body proportion.

6. The method according to claim 5, characterized in that The step of processing the transparent image based on the bounding box and the position information to obtain a target transparent image that meets the requirements for the product main body ratio includes: Determine the longest side in the bounding box, and generate a target border based on the longest side; Generate a transparent background based on the size of the target frame and a preset ratio; Based on the position information, determining the coordinates of the target frame in the transparent image; Clipping the transparent image according to the coordinates of the target frame in the transparent image to obtain a clipped transparent image; The cropped transparent image is synthesized onto the transparent background to obtain the target transparent image.

7. The method according to claim 5, characterized in that After the transparent image is processed based on the bounding box and the position information to obtain a target transparent image that meets the requirement of the product main body proportion, the method further includes: Performing a comprehensive quality assessment on the target transparent image to obtain a comprehensive quality score of the target transparent image; Based on the comprehensive quality score, determining whether the target transparent image meets the display requirements; If the target transparent image meets the display requirements, the target transparent image is added to the preset transparent image resource library.

8. A transparent image generating device, characterized in that: include: An acquisition unit, used for acquiring an original product image to be processed; a segmentation unit, configured to segment the commodity body in the original commodity image as a whole by using a preset two-category image segmentation model to obtain a segmented commodity body mask image, wherein the commodity body includes at least two individuals; A determination unit, configured to determine a transparency channel according to the segmented commodity body mask image; The synthesis unit is used to synthesize the transparency channel with the three color channels of the original product image to obtain a transparent image of the product body.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the transparent image generating method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the transparent image generating method according to any one of claims 1 to 7 is implemented.

Images