Clothing image extraction method and device, equipment, medium and product

Abstract

The application relates to a clothing image extraction method and device, equipment, medium and product, the method comprising: acquiring an original image containing clothing content; transforming the original image into a specification image eliminating edge sawtooth in the image content; optimizing the contrast of the specification image to obtain an enhanced image; merging the specification image and the enhanced image into an image segmentation model to determine a target image mask corresponding to the depth of field of a visible wearing part of the clothing in the clothing content; and extracting a clothing main body image from the clothing content according to the target image mask. The application can accurately extract the clothing main body image of the visible wearing part from the original image containing the clothing content, the obtained clothing main body image is natural and delicate, can provide reliable materials for virtual fitting technology, has a wide application scenario, and has a low implementation cost.

Description

Technical Field

[0001] This application relates to virtual fitting technology, and more particularly to a method, apparatus, device, medium, and product for extracting clothing images. Background Technology

[0002] With the improvement of computing power, the application of virtual try-on technology is becoming increasingly widespread. A fundamental requirement of virtual try-on technology is to extract the main image of the visible part of the clothing from an image containing clothing content. This partial image can be more easily composited with an image of a model's body to obtain an image of the model wearing the clothing, showcasing the visual effect of the model wearing the corresponding clothes. Furthermore, these images of the model wearing the clothing can be used as product images for the corresponding clothing items and published on e-commerce platforms or advertising platforms. Users can then evaluate the style of the clothing based on the visual effect and decide whether to purchase it.

[0003] To obtain the main image of the clothing, image segmentation techniques can be applied to extract the corresponding main image of the clothing from an image containing clothing content. Traditionally, there are two main types of techniques for image segmentation of clothing content. The first type is based on multi-classification tasks to identify whether each pixel belongs to the foreground or background, using all the foreground clothing content as the target image for compositing with the model. The second type is based on pixel quantization, using image matting algorithms to determine the quantization value of each pixel belonging to the foreground pixels to be extracted, so that information of the corresponding pixels can be extracted appropriately based on the quantization value.

[0004] The first type of technique, which obtains the main image of clothing based on the discrete values ​​of 0 or 1 for each pixel in the binary classification result, produces obvious jagged edges and poor image quality, especially in high-resolution images. Furthermore, this technique is ineffective for complex classification scenarios. For example, some clothing images taken by simply laying the garment flat include not only the visible parts of the clothing worn by the person (e.g., the foreground image) but also the invisible parts that may be obscured by the neck, hands, or other body parts (e.g., the clothing behind the neck). Only the visible parts constitute the main image of the clothing. When such an image is combined with a model's body image, there is generally no need to crop the invisible parts, significantly improving the synthesis efficiency. The first type of technique is also inadequate for handling situations with multiple depths of field, including background, visible parts of the clothing, and invisible parts of the clothing.

[0005] While the second type of technique can make the obtained clothing subject image more natural, it cannot be trained for multi-classification tasks. Based on the goal of obtaining a quantized value for each pixel, its annotation cost is higher and requires fine annotation of edge information. Otherwise, it is easy to cause unclear image boundaries. Therefore, the second type of technique cannot be relied on alone to obtain a high-quality clothing subject image.

[0006] Therefore, it is necessary to comprehensively improve the existing methods for extracting clothing images in order to achieve good application results. Summary of the Invention

[0007] The purpose of this application is to solve the above-mentioned problems by providing a method for extracting clothing images and corresponding apparatus, devices, non-volatile readable storage media, and computer program products.

[0008] According to one aspect of this application, a method for extracting clothing images is provided, comprising:

[0009] Acquire the original image, which contains clothing content;

[0010] The original image is transformed into a standardized image with jagged edges removed from its image content;

[0011] Optimize the contrast of the specified image to obtain an enhanced image;

[0012] The specified image and the enhanced image are merged and input into the image segmentation model to determine the target image mask corresponding to the depth of field of the visible part of the clothing in the clothing content;

[0013] The main image of the clothing is extracted from the clothing content based on the target image mask.

[0014] According to another aspect of this application, a clothing image extraction device is provided, comprising:

[0015] The original image acquisition module is configured to acquire the original image, which contains clothing content;

[0016] The original image noise reduction module is configured to transform the original image into a standard image with jagged edges removed from its image content;

[0017] The original image enhancement module is configured to optimize the contrast of the specified image to obtain an enhanced image;

[0018] The segmentation execution module is configured to merge the specification image and the enhanced image into the image segmentation model to determine the target image mask corresponding to the depth of field of the visible part of the clothing in the clothing content;

[0019] The image extraction module is configured to extract the main image of the clothing from the clothing content based on the target image mask.

[0020] According to another aspect of this application, a clothing image extraction device is provided, including a central processing unit and a memory, wherein the central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the clothing image extraction method described in this application.

[0021] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the clothing image extraction method in the form of computer-readable instructions, wherein the computer program, when invoked by a computer, executes the steps included in the method.

[0022] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0023] Compared with existing technologies, this application has several technological advantages, including but not limited to:

[0024] First, the original image containing clothing content is scaled down to a standard image, and jagged edges that may form during the scaling process are eliminated. Then, the image contrast is optimized based on the standard image to make the light and dark features more obvious, resulting in an enhanced image. Next, the standard image and the enhanced image are merged and input into the image segmentation model. The image segmentation model directly predicts the target image mask of the depth of field corresponding to the visible part of the clothing. Based on this target image mask, the main image of the clothing corresponding to the visible part of the clothing content can be accurately extracted, and it can be ensured that the obtained main image of the clothing has high image quality.

[0025] Secondly, the image segmentation model is provided to generate target image masks corresponding to the visible parts of the clothing. It can determine the corresponding depth of field and thus the target image mask by comprehensively determining the depth of field based on the information of both the standard image and the enhanced image. Such a mask utilizes the deep semantics provided by the clear image after eliminating jagged edges, as well as the deep semantics provided by the enhanced image that further optimizes the contrast between light and dark areas based on the clear image. By using these two types of deep semantics for inference, each pixel in the obtained target image mask can accurately represent the corresponding depth quantization value. Using these values ​​to extract the main image of the clothing, the resulting image of the main image of the clothing has a more natural and delicate effect.

[0026] Furthermore, this application uses depth of field as a unit and employs an image segmentation model to process the segmentation task of clothing subject images, achieving effective separation of different depths of field. This can reduce the implementation cost of extracting clothing subject images and can be used in application scenarios such as virtual fitting applications and online virtual fitting on e-commerce platforms, achieving economies of scale and improving user experience. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of the network architecture of an exemplary application environment for the technical solution of this application;

[0029] Figure 2 This is a schematic flowchart of one embodiment of the clothing image extraction method of this application;

[0030] Figure 3 This is a schematic diagram of the process of training the image segmentation model in an embodiment of this application;

[0031] Figure 4 This is a schematic diagram illustrating the process of converting annotation information into a real mask group in an embodiment of this application;

[0032] Figure 5 This is a flowchart illustrating the process of calculating the total loss value of the image segmentation model in the embodiments of this application;

[0033] Figure 6 This is a schematic diagram of the process for synthesizing the target image mask in the embodiments of this application;

[0034] Figure 7 This is a schematic diagram illustrating the process of extracting the main image of clothing from a target image mask in an embodiment of this application;

[0035] Figure 8 This is a schematic block diagram of the clothing image extraction device of this application;

[0036] Figure 9 This is a schematic diagram of the structure of a clothing image extraction device used in this application. Detailed Implementation

[0037] Please see Figure 1The network architecture used in the exemplary application scenario of this application includes a terminal device 80, a business server 81, and an application server 82. The application server 82 can be used to deploy a clothing image extraction service and provide corresponding functional plugins to the downstream services implemented on the business server 81, so that the clothing image extraction service can be called through the functional plugins. When a user accesses the corresponding page of the downstream service of the business server 81 from his / her terminal device 80, he / she can upload an original image containing clothing content through the functional plugin, and then the business server 81 calls the clothing image extraction service opened by the application server 82 to extract the main image of clothing in the clothing content based on the original image extraction service.

[0038] The clothing image extraction service of this application can be implemented by executing the clothing image extraction method of this application. Specifically, the clothing image extraction method of this application can be implemented as a computer program product, installed on a corresponding device such as the application server, and after it is run, the method can be executed to open the clothing image extraction service.

[0039] The downstream business can be various, such as online store operations on e-commerce platforms, online sales in live streaming rooms, image cutout processing in image processing support platforms, etc. As a foundational technology, the clothing image extraction service implemented in this application can be called by any downstream business.

[0040] Of course, in another exemplary network architecture of this application, the clothing image extraction service of this application can be deployed on a terminal device or the application server, or any other smart device, as long as the device to which the application is deployed can provide sufficient hardware and software resources to support the implementation of the clothing image extraction service.

[0041] The models cited or potentially cited in this application, such as the image segmentation model and the matting model implemented using the matting algorithm, can be traditional machine learning models or deep learning models. Unless explicitly specified, these models can be deployed on a remote server and remotely invoked on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence can be obtained through transfer learning to reduce the requirements on the client's hardware resources and avoid excessive consumption of client hardware resources.

[0042] Please see Figure 2 According to a method for extracting clothing images provided in this application, in one embodiment, the method includes the following steps:

[0043] Step S1100: Obtain the original image, wherein the original image contains clothing content;

[0044] The original image generally contains clothing content, which can be recognized by the human eye as clothing products. The clothing content can be a flat lay image, which is an image of the clothing product fully displayed from one perspective, or a real-life image of the clothing being worn, which is an image of the clothing product being worn on a model's body and appearing together with the model's body.

[0045] In the original image, multiple preset depths of field are used as layers, which can be conceptually understood as containing at least multiple layers corresponding to preset depths of field. The specific number of preset depths of field can be determined as needed. In one embodiment, it can include three preset depths of field. The first depth of field indicates the background content in the image, referring to the depth of the background image behind the clothing content, generally understood as the depth corresponding to the background of the clothing content. The second depth of field indicates the invisible part of the clothing content in the image after it is worn. It is determined according to the clothing content and mainly refers to the depth of the part of the clothing content that will be obscured by the body after it is worn, called the invisible part after wearing. The third depth of field indicates the visible part of the clothing content in the image after it is worn. It is the foreground of the original image. For the clothing content, it refers to the depth of the part of the clothing content that is not obscured by the body after it is worn, called the visible part after wearing.

[0046] In this application, image segmentation can be performed for any of the aforementioned depths of field in order to obtain the image content of the corresponding layer. For example, for an original image containing clothing content, when the layer corresponding to the third depth of field is extracted, the obtained image is the main image of the clothing, that is, the main image of the clothing content that is visible to the human body and is not obscured by the human body parts after the clothing content is worn.

[0047] Step S1200: Transform the original image into a standard image with jagged edges removed from its image content;

[0048] Due to downstream business needs, original images containing clothing content typically have high resolution, such as 2K or higher. However, image segmentation models used to process clothing content often process input data at a smaller scale to improve processing efficiency. During the scaling process of the original image, various scaling algorithms often introduce noticeable jagged edges, leading to increased noise and decreased quality in the scaled image. Therefore, bilinear interpolation can be applied during the scaling process to reduce noise and eliminate jagged edges, resulting in a high-quality scaled image.

[0049] In mathematics, bilinear interpolation is a linear interpolation extension of an interpolation function with two variables. Its core idea is to perform linear interpolation once in each of the two directions. For example, suppose the original image is m*n and the target image is a*b. Then the side length ratios of the two images are m / a and n / b, respectively. Note that this ratio is usually not an integer and should be stored as a floating-point number in the program. The (i,j)th pixel of the target image can be mapped back to the original image using the side length ratio. Its corresponding coordinates are:

[0050]

[0051] Obviously, these corresponding coordinates are generally not integers, and non-integer coordinates cannot be used on discrete data like images. The bilinear interpolation algorithm calculates the value (grayscale or RGB value) of the pixel at those coordinates by finding the four nearest pixels in the original image. For example, if the corresponding coordinates of a given pixel are (2.5, 4.5), then the four nearest pixels are (2, 4), (2, 5), (3, 4), and (3, 5). Taking a grayscale image as an example, the grayscale value of pixel (i, j) can be calculated using the following formula:

[0052] f(i,j)=w1*p1+w2*p2+w3*p3+w4*p4

[0053] Where pi (i = 1, 2, 3, 4) are the four nearest pixels, and wi (i = 1, 2, 3, 4) are the corresponding weights of each pixel. Since f(i, j) is determined based on the pixel values ​​of the surrounding pixels, the scaled image obtained after scaling the original image will have a smoother image transition effect.

[0054] It's easy to understand that applying bilinear interpolation to transform the original image to obtain a corresponding image at the target scale can make the transition between pixels smoother, effectively eliminating jagged edges in the image content. This helps improve the cutout effect of clothing images, making the final image of the clothing subject more delicate and natural.

[0055] Step S1300: Optimize the contrast of the specified image to obtain an enhanced image;

[0056] In real-world photographic environments, factors such as lighting and minimal color difference between clothing and background (e.g., off-white clothing against a gray-white background) often result in insufficiently prominent boundary features between the foreground and background images, leading to image segmentation failures when applying image segmentation models. Therefore, this application enhances the contrast of the original image by improving its brightness, creating an enhanced image. This enhanced image is then used as an additional input to the image segmentation model, alongside the original image, to ensure improved accuracy and success rate.

[0057] In one embodiment, a dehazing algorithm is applied to the specified image to enhance its contrast, thereby obtaining an enhanced image. Available dehazing algorithms mainly include various types of dehazing algorithms based on image enhancement, image restoration, and convolutional neural networks.

[0058] Image enhancement-based dehazing algorithms highlight image details and improve contrast to make the image appear clearer. These algorithms have wide applicability. Specific algorithms include: Retinex algorithm, histogram equalization algorithm, partial differential equation algorithm, and wavelet transform algorithm. The Retinex algorithm, based on imaging principles, eliminates the influence of reflection components, achieving image enhancement and dehazing. The histogram equalization algorithm makes the pixel distribution of the image more uniform, amplifying image details. The partial differential equation algorithm treats the image as a partial differential equation, improving contrast by calculating the gradient field. The wavelet transform algorithm decomposes the image, amplifying the useful parts.

[0059] Image restoration-based dehazing algorithms are primarily based on atmospheric scattering physics models. By observing and summarizing a large number of hazy and hazy images, they identify certain mapping relationships. Then, they perform inverse operations based on the formation process of hazy images to restore clear images. The most classic example is the dark channel prior dehazing algorithm, which, through feature analysis of a large number of hazy images, finds prior relationships between hazy images and certain parameters in the atmospheric scattering model. This algorithm has low complexity and good dehazing effect.

[0060] Dehazing algorithms based on convolutional neural networks (CNNs) use an end-to-end model to recover a hazy image from a hazy one. Currently, there are two main approaches to dehazing algorithms using neural networks: using CNNs to generate certain parameters of an atmospheric scattering model and then using the atmospheric scattering model to recover a hazy image, or using CNNs to directly generate a clear, hazy image from a blurred image.

[0061] As can be seen, based on the advantages of various dehazing algorithms and to adapt to specific needs, any feasible dehazing algorithm can be adopted to obtain an enhanced image that improves the contrast between light and dark compared to the image of the specified size, making the image content clearer, the contrast between light and dark more obvious, and the boundary features more prominent.

[0062] Step S1400: Merge the specification image and the enhanced image into the image segmentation model to determine the target image mask corresponding to the depth of field of the visible part of the clothing in the clothing content;

[0063] The image segmentation model in this application is constructed using a deep learning model, which can be a U-net series model, including U-net, U... 2 -net, etc. According to the architecture of U-net, it internally implements an encoding path for downsampling images. The encoding path consists of multiple encoders corresponding to different scales cascaded together, thereby obtaining sampled feature maps of the input image at multiple scales. At the same time, a decoding path for upsampling the sampled feature maps is also implemented corresponding to the encoding path. The decoding path is also composed of multiple decoders corresponding to different scales cascaded together.

[0064] In the encoding path, higher-scale encoders are responsible for downsampling the image's feature maps at the corresponding scale, and then passing them to the next level encoder at a smaller scale for further downsampling. Simultaneously, through cross-layer connections, the feature maps obtained by this encoder are passed to the corresponding scale decoders in the decoding path. The encoding path samples feature maps at corresponding scales level by level through each encoder, obtaining the feature map corresponding to the smallest scale at the last encoder level. The features at the smallest scale are transformed and input into the decoding path as the underlying semantic information for upsampling.

[0065] In the decoding path, the higher-scale decoder receives the feature map at the next smaller scale at its corresponding scale. Then, referencing the feature map transmitted from the encoder at the corresponding scale via a cross-layer connection, it upsamples the features at the next smaller scale to reconstruct the feature map at the corresponding scale, which is then output to the higher-scale decoder for further decoding. This process continues until the last decoder. During decoding, each decoder can segment a set of scale masks corresponding to its scale.

[0066] Finally, by fully connecting the scale masks corresponding to different scales obtained by each decoder in the decoding path, the image mask set corresponding to the entire input image can be obtained. For a single input image, it is represented as a three-dimensional data structure of H*W*C, where H*W*C represents the height, width, and number of channels of the image. Similarly, its image mask set is also represented as a three-dimensional data structure of H*W*C. The difference between input and output is that the number of channels in the image mask set is generated according to multiple preset depths of field. Therefore, the feature map H*W corresponding to each channel actually constitutes a single image mask, and the entire image mask set is actually composed of C image masks of size H*W. From the perspective of a pixel, the values ​​of different channels at the same pixel constitute the element values ​​of the vector representation corresponding to the pixel falling into different preset depths of field.

[0067] The image segmentation model of this application has parallel processing capabilities. When two images are merged and input, they can be processed in parallel to obtain their respective corresponding image mask sets. Therefore, to improve computer execution efficiency, the standard image and the enhanced image of the original image can be channel-merged to obtain H*W*2C image data, which is then input into the image segmentation model. Correspondingly, H*W*2C image mask results are also obtained, including the image mask set of the standard image and the image mask set of the enhanced image. Both image mask sets contain multiple image masks corresponding to preset depths of field. That is, for the same depth of field, both the standard image and the enhanced image have their corresponding image masks. Therefore, for the target depth of field where the clothing subject image is located, the two image masks corresponding to the target depth of field can be directly called, namely, the image mask of the standard image corresponding to the target depth of field and the image mask of the enhanced image corresponding to the target depth of field, to comprehensively determine the target image mask corresponding to the clothing subject image.

[0068] In order to make comprehensive use of the advantageous features of the standard image and its enhanced image, when determining the target image mask, the two image masks corresponding to the target depth of field can be linearly fused. By linearly fusing, the advantageous semantic features of the standard image and its enhanced image can be utilized to obtain a target image mask that can more accurately indicate the image content of the clothing subject image.

[0069] Step S1500: Extract the main image of the clothing from the clothing content according to the target image mask.

[0070] The target image mask has the same scale as the standard image and the enhanced image. Depending on the original image from which the main clothing image is extracted, the target image mask can be scaled as needed. For example, when the original target image is the original image, the target image mask can be scaled to the same scale as the original image. In one embodiment, bilinear interpolation is also used for scaling when scaling the target image mask to avoid jagged edges in the obtained result.

[0071] The target image mask indicates the target area that constitutes the visible part of the clothing, and the depth quantization value of each pixel in the target area. By performing pixel-level smooth fusion with the target original image, that is, any one of the original image, the standard image, or the enhanced image, the main image of the clothing can be extracted from the target original image, thus achieving the purpose of image cutout.

[0072] In one embodiment, considering that the main image of the clothing is a continuous region, the target image mask can be filtered before image matting to delete smaller connected regions and retain larger regions, thereby avoiding the acquisition of the main image of the clothing from carrying unnecessary interference images.

[0073] As can be seen from the above embodiments, this application has multiple technical advantages, including but not limited to:

[0074] First, the original image containing clothing content is scaled down to a standard image, and jagged edges that may form during the scaling process are eliminated. Then, the image contrast is optimized based on the standard image to make the light and dark features more obvious, resulting in an enhanced image. Next, the standard image and the enhanced image are merged and input into the image segmentation model. The image segmentation model directly predicts the target image mask of the depth of field corresponding to the visible part of the clothing. Based on this target image mask, the main image of the clothing corresponding to the visible part of the clothing content can be accurately extracted, and it can be ensured that the obtained main image of the clothing has high image quality.

[0075] Secondly, the image segmentation model is provided to generate target image masks corresponding to the visible parts of the clothing. It can determine the corresponding depth of field and thus the target image mask by comprehensively determining the depth of field based on the information of both the standard image and the enhanced image. Such a mask utilizes the deep semantics provided by the clear image after eliminating jagged edges, as well as the deep semantics provided by the enhanced image that further optimizes the contrast between light and dark areas based on the clear image. By using these two types of deep semantics for inference, each pixel in the obtained target image mask can accurately represent the corresponding depth quantization value. Using these values ​​to extract the main image of the clothing, the resulting image of the main image of the clothing has a more natural and delicate effect.

[0076] Furthermore, this application uses depth of field as a unit and employs an image segmentation model to process the segmentation task of clothing subject images, achieving effective separation of different depths of field. This can reduce the implementation cost of extracting clothing subject images and can be used in application scenarios such as virtual fitting applications and online virtual fitting on e-commerce platforms, achieving economies of scale and improving user experience.

[0077] Based on any embodiment of this application, please refer to Figure 3 Before merging the standardized image and the enhanced image into the image segmentation model, the following steps are included:

[0078] Step S2100: Construct training samples and their corresponding real mask groups based on the specification images and / or enhanced images corresponding to the sample images containing clothing content, and store them in the training dataset. The real mask group contains three or more real masks, and each real mask corresponds to a single corresponding preset depth of field.

[0079] The image segmentation model of this application can be prepared through prior training. Before training it, corresponding training samples need to be constructed and stored in the training dataset. Subsequently, the image segmentation model can be iteratively trained using the training samples in the training dataset until it reaches a convergent state, making it suitable for online inference.

[0080] The training samples can be constructed based on pre-collected sample images. These sample images can be constructed using two types of materials: the first type can be the flat lay images of clothing mentioned earlier, and the second type can be the real-life images of clothing mentioned earlier. These materials are selected as needed, ensuring that they can comprehensively generalize the image features of clothing content, and all of these materials are then designated as sample images. By using these two types of materials to construct training samples for training the image segmentation model, the image segmentation model can segment not only the main clothing image from the flat lay images of clothing, but also the main clothing image from the real-life images of clothing.

[0081] Based on the determined sample images, each sample image can be further transformed into its corresponding specification image according to the method described above in this application. During the transformation process, bilinear interpolation algorithm can be applied to eliminate its jagged information, and an enhanced image can be further obtained based on its specification image.

[0082] When constructing each training sample, a single image can be used as the training sample, that is, the standard image of the sample image or the corresponding enhanced image can be used as the training sample, or the standard image of the sample image and the enhanced image generated on the standard image can be used together to form the training sample.

[0083] The sample images can be annotated manually. That is, the image content within the sample images is manually annotated, and category labels are assigned according to the different depths of field corresponding to different image content regions. During the manual annotation stage, these category labels can be represented using discrete data such as 0, 1, 2, etc., or they can be an enumerated value. Subsequently, a set of ground truth masks corresponding to the training samples can be generated based on this annotation information.

[0084] The training samples consist of a single image or multiple images. The corresponding real mask group is determined based on the annotation information of each image. Each real mask group contains multiple real masks, which are used to correspond to different preset depths of field.

[0085] Taking a real mask group for a single image as an example, it includes multiple real masks corresponding to multiple depths of field. The number of depths of field is three or more, arbitrarily determined as needed. In this application, in the real mask group generated based on the annotation information, each pixel value in each real mask is represented by a binary numerical value. Therefore, for each pixel value, a vector representation is formed by the binary numerical values ​​of multiple depths of field. For example, when there are three preset depths of field, its representation is {1,0,0}. Considering that the same pixel point can only belong to one depth of field, its vector representation actually constitutes a one-hot encoded vector, where each element value can provide supervision information for the prediction of each corresponding depth of field. Therefore, the real mask group can provide supervision labels for the image mask predicted by the image segmentation model based on multi-classification tasks. Of course, training supervision can also be implemented as needed based on the similarity, intersection-union ratio, etc., between the real mask group and the image mask group predicted by the image segmentation model.

[0086] After obtaining the training samples and their corresponding real mask sets, the two are constructed as a mapping relationship and stored in the training dataset. This makes the training dataset contain a massive number of training samples and their real mask sets sufficient to train the image segmentation model to a convergent state. It is expected that the rich sample features provided by the massive training samples can be used to quickly train the image segmentation model to a convergent state.

[0087] Step S2200: Input the training samples from the training dataset into the image segmentation model and train until convergence. For each training session, predict the image mask group corresponding to all preset depths of field for the training samples, and supervise the training using the real mask group corresponding to the training samples.

[0088] When training the image segmentation model is required, training samples can be successively called from the training dataset and input into the image segmentation model for training. After the training samples are input into the image segmentation model, the image segmentation model performs downsampling and upsampling on them to obtain the image mask group corresponding to each training sample. Then, the training loss of the image mask group can be calculated based on the real mask group corresponding to the training sample to determine the total loss value. Then, based on the total loss value, it is decided whether the image segmentation model has reached the convergence state. If it has not reached the convergence state, iterative training continues and the weight parameters of each link of the image segmentation model are corrected. This process is repeated until the image segmentation model is trained to the convergence state, after which it can be used for online inference.

[0089] According to the above embodiments, this application represents the labeled information of training samples as a set of real masks and provides multiple sets of real masks corresponding to preset depths of field as supervision labels for training samples. This enables the image segmentation model to adapt to more than three depths of field and generate image masks of corresponding depths of field. This allows the image segmentation model to perform image segmentation on training samples at multiple depths of field. When the image segmentation model is used for online inference, it can similarly perform image segmentation on the standard image and the enhanced image in the input image at multiple depths of field, accurately distinguishing pixels at different depths of field, so as to extract the image corresponding to the target depth of field, such as extracting the main image of clothing corresponding to the visible part of the clothing content in this application.

[0090] Based on any embodiment of this application, before constructing training samples and their corresponding real mask sets based on the specification images and / or enhanced images corresponding to sample images containing clothing content, the following steps are included:

[0091] Step S3100: Rotate the sample image containing clothing content or the specification image of the sample image.

[0092] By rotating the same image content at different angles, multiple training samples are obtained. These training samples enrich the image features, enabling the trained image segmentation model to learn stronger feature representation capabilities and preventing overfitting. For a training sample, considering that its standard image and augmented image generally require consistency in viewpoint, its sample image or the standard image obtained from the sample image can be rotated before further determining its corresponding augmented image based on the standard image.

[0093] As can be seen from the above embodiments, image rotation can expand the training samples, generalize sample features, achieve data augmentation, avoid model overfitting, and make the model more robust. The steps of this embodiment are introduced into the training sample creation process instead of image cropping as a data augmentation method because image cropping may destroy the integrity of the clothing content and change the depth relationship of the clothing content image. Using image rotation to enrich the training samples does not destroy this integrity, thus ensuring that higher quality training samples are obtained, and preventing the performance of the image segmentation model from being adversely affected by data augmentation.

[0094] Based on any embodiment of this application, before constructing training samples and their corresponding real mask sets based on the specification images and / or enhanced images corresponding to sample images containing clothing content, the following steps are included:

[0095] Step S3200: Identify the color-blocked area of ​​the clothing content from the sample image containing the clothing content or the specification image of the sample image, and replace the color in the color-blocked area.

[0096] Color blocking designs are common in clothing images, where two or more different colors are joined together. When the color blocking is near the transition zone between different depths of field, image segmentation models often have difficulty distinguishing this situation effectively and may mistakenly identify the color blocking area as a region of another depth of field. Therefore, data augmentation can be used to reduce the chance of image segmentation models misjudging this situation.

[0097] In this embodiment, image recognition can be performed on the sample image used to create training samples, or on the specification image of the sample image. The color-blocked area in the clothing content can be identified and the original image of the color-blocked area can be extracted. At the same time, the original image is subjected to image erosion processing to obtain an eroded image. Then, the difference between the eroded image and the original image is calculated to calculate the edge color band of the clothing. Then, the sample image or its specification image is filled with different colors corresponding to the edge color band, thereby expanding different training samples. This realizes the simulation of features that are prone to misidentification by the image segmentation model. After training the image segmentation model with these training samples, the image segmentation model can be improved to more accurately identify different depth areas of clothing content with color-blocked areas.

[0098] As can be seen from the above embodiments, by replacing the colors of the training samples with color-blocked areas, the features of color-blocking can be generalized, thereby expanding the training samples to a richer range. After training with these training samples, the image segmentation model can more accurately adapt to the color-blocking of clothing and accurately distinguish the depth of field of each pixel.

[0099] Based on any embodiment of this application, before constructing training samples and their corresponding real mask sets based on the specification images and / or enhanced images corresponding to sample images containing clothing content, any one of a variety of methods or a combination thereof can be used to obtain sample images. The methods described in the following two embodiments can both achieve the acquisition of sample images:

[0100] In one embodiment, the sample image is obtained by step S4100, which involves acquiring a flat lay image of the clothing and a model's body image corresponding to the clothing content, synthesizing a model's clothing image, and then using the model's clothing image as the foreground to synthesize into a random background image.

[0101] Specifically, to facilitate the creation of training samples, flat lay images of different clothing products and body images of different models in different poses can be prepared. Then, using any feasible image compositing method, each clothing flat lay image can be combined into a model body image to obtain the corresponding model dressed image. By cross-compositing multiple clothing flat lay images and multiple model body images, a large number of model dressed images can be obtained.

[0102] During the process of compositing the model's clothing image, it is ensured that the image effect of the model's clothing image conforms to the visual perspective law of the objective physical world. This ensures that after the clothing image is laid out and applied to the model's body image, the image content of the clothing area behind the model's background no longer exists. In fact, it is also to delete the image content of the corresponding invisible parts of the clothing, so that the image effect displayed by the entire model's body image is natural and reasonable.

[0103] Considering that the background of a model's body image is generally limited, in order to simulate the diversity of real-world scenes, a large number of background images can be prepared. One of these background images is randomly selected as the random background image, and then the model's clothing image is added to the random background image as the foreground to achieve image synthesis, thereby obtaining the corresponding sample image.

[0104] In another embodiment, step S4200 involves acquiring a real-life image of clothing, extracting the human clothing image from it, fusing it with the real-life image of clothing to obtain a composite sketch, and then using the composite sketch as the foreground to composite onto a random background image to obtain the sample image.

[0105] The real-life images of the clothing can be collected from publicly available datasets or taken by the photographer themselves. They are typically real-life images of actual people wearing the corresponding clothing. These real-life images are usually limited in number and have relatively little background content. Therefore, similarly, the background can be replaced, and more sample images can be synthesized by combining random background images with the clothing images from the real-life images.

[0106] Therefore, a matting model, such as the matting model, is first used to matte the real-life clothing image to obtain the human clothing image. Considering that matting models often lead to errors, resulting in incomplete clothing content in the extracted human clothing image, this embodiment uses a weighted fusion of the real-life clothing image and the human clothing image to obtain a composite sketch, which restores the clothing content as completely as possible. This composite sketch is then used as the foreground and combined with a randomly determined background image to obtain the corresponding sample image.

[0107] In the above embodiments, the flat lay images of clothing and the real-life images of the dressed figures are all pre-labeled with corresponding annotation information for each pixel. Typically, this annotation information is provided using a simple classification method, for example, the first region is labeled 0, the second region is labeled 1, the third region is labeled 3, and so on, indicating that different regions belong to different depths of field. After generating new sample images based on the flat lay images of clothing and the real-life images of the dressed figures, a mapping relationship can be established between the new sample images and the previously given annotation information. Subsequently, a mapping relationship can be established between the training samples constructed based on the sample images and the annotation information. Furthermore, based on the annotation information, the true mask set of the training samples can be obtained.

[0108] As can be seen from the above examples of obtaining sample images, two types of materials can be used as a basis: flat lay images of clothing with pre-annotated clothing subject image areas and real-life images of clothing. A large number of sample images can be prepared through technical means to achieve generalization of sample features, which greatly reduces the production cost of training samples. At the same time, the training samples made from these feature-generalized sample images can enable the image segmentation model to converge quickly, reducing the training cost.

[0109] Based on any embodiment of this application, please refer to Figure 4 Training samples and their corresponding set of ground truth masks are constructed based on the specification images and / or augmented images corresponding to the sample images containing clothing content, including:

[0110] Step S2110: Obtain the annotation information corresponding to the sample image, wherein the annotation information includes the category label corresponding to the depth of field of the pixel of the sample image;

[0111] Once the annotation information of a sample image is available, a set of real masks for training samples constructed based on the corresponding annotation information of the sample image can be generated.

[0112] In one embodiment, the image content of the sample image can be manually annotated based on earlier source material. Category labels are assigned according to the different depths of field corresponding to different image content areas. During the manual annotation stage, these category labels can be represented using discrete data such as 0, 1, 2, etc., or they can be enumerated values. For example, the main image of clothing in a sample image can be labeled "2", the background content can be labeled "0", and the mid-ground content between the foreground and background can be labeled "1". When the source material is scaled into a sample image using a bilinear interpolation algorithm, the corresponding labeled areas can also be scaled accordingly to ensure that the annotation correspondence remains essentially unchanged.

[0113] Step S2120: Based on the category label in the annotation information, construct the one-hot encoding vector of each pixel in the training sample. The training sample is constructed using the sample image. The elements at the same position of the one-hot encoding vectors of all pixels represent the corresponding depth-of-field ground mask. The ground mask group of the training sample is composed of multiple depth-of-field ground masks.

[0114] Although the annotation information of the sample image indicates the category of each region in the form of category labels, the category label of each region is actually the category label of each pixel within that region. Based on this principle, the annotation information can be transformed to represent the binary value of each pixel corresponding to different depths of field using a one-hot encoded vector, thus forming the multi-class label for that pixel. For example, suppose a sample image has the following pixels, its annotation information is represented as follows:

[0115] [1,0,0][1,0,0][0,1,0][0,1,0]……

[0116] [1,0,0][0,1,0][0,0,1][0,0,1]……

[0117] ...

[0118] [0,1,0][0,1,0][0,1,0][0,1,0]……

[0119] [1,0,0][1,0,0][0,1,0][0,1,0]……

[0120] Each vector corresponds to a pixel, and each vector has three elements, corresponding to three depths. By viewing the binarized values ​​at the same position according to the pixel correspondence, it's easy to understand that this results in three binary feature maps. Each feature map is essentially a ground truth mask corresponding to its respective depth. For example, the following is a feature map showing the pixel value distribution of the ground truth mask corresponding to the first element position in the example above:

[0121] 1, 1, 0, 0...

[0122] 1, 0, 0, 0...

[0123] ...

[0124] 0, 0, 0, 0……

[0125] 1, 1, 0, 0...

[0126] As can be seen from the examples above, all the real masks in each real mask group together represent the multi-class labels corresponding to the pixels of the image in the corresponding training sample. The multi-class labels constitute a one-hot encoded vector, where each element value represents whether the pixel corresponding to the multi-class label belongs to the depth of field corresponding to the real mask where the element value is located.

[0127] It is important to note that the annotation information used to create the real mask set is designed to maintain the same scale correspondence with the standard or enhanced images after the sample images have been converted into standard images and / or enhanced images to form training samples. In other words, when the annotation information corresponds to the source material used to generate the sample images or is provided in relation to the sample images, it should first be converted into annotation information of the same scale as the standard and enhanced images, and then the corresponding real mask set should be constructed based on the conversion result.

[0128] As can be seen from the above embodiments, image regions can be conveniently labeled based on sample images or earlier materials to obtain their corresponding labeling information. Then, the labeling information is encoded to obtain data represented by one-hot encoded vectors, which is equivalent to obtaining multiple real masks corresponding to multiple depths of field. The region labeling information is converted into the representation of the mask, and it is convenient to represent it as a three-dimensional array in the data structure, making the operation more efficient. Such a representation result allows the real mask group to be provided to the image segmentation model as a supervision label when performing multi-classification tasks during the training phase, making the image segmentation model more feasible for multi-classification tasks and improving training efficiency.

[0129] Based on any embodiment of this application, please refer to Figure 5Each training iteration predicts a set of image masks corresponding to all preset depths of field for the training samples, and supervises training using the real mask set corresponding to the training samples, including:

[0130] Step S2210: Extract the feature maps corresponding to the edge information in the image mask group as enhancement weights;

[0131] After the image segmentation model predicts an image mask set for a training sample, such as the specified image and / or the enhanced image, a pooling layer can be applied to the image mask set to perform a pooling operation, obtaining a pooled mask set to characterize the edge information. Then, the pooled mask set is subtracted from the corresponding image mask set until the feature map is obtained, which is used as the enhancement weight w for subsequent calculation of the model loss, as shown in the following formula:

[0132] w = (pool(mask) - mask)

[0133] Here, mask represents an image mask group, and pool represents a pooling layer.

[0134] Step S2220: Calculate two or more loss values ​​of the image mask group relative to the real mask group by associating the enhancement weights, and summarize them into a total loss value;

[0135] In order to determine whether the image segmentation model has reached convergence and to implement gradient updates for the image segmentation model, it is necessary to calculate the total loss value corresponding to each training sample of the image segmentation model. The total loss value can be determined by using two or more loss value calculation methods.

[0136] The first type of loss value is the cross loss value. Since the real mask set in this application provides multi-class labels for each pixel, it is suitable for multi-class training of the image segmentation model to improve training efficiency. Therefore, the cross-entropy loss value of each pixel can be calculated based on the corresponding multi-class label, thereby obtaining the cross-entropy loss value of the entire training sample. Its formula is expressed as follows:

[0137]

[0138] Among them, y c This represents the image mask set used for training samples. This represents the corresponding ground truth mask set for the training samples, where H and W are the height and width dimensions of the image, respectively, and i and j indicate the coordinates of the pixels. ij This indicates the corresponding enhancement weight for each pixel. The subsequent formulas follow the same principle.

[0139] Cross-entropy loss can be calculated for each pixel, allowing for fine-grained constraints on the model's inference capabilities and enabling the model to classify the depth of field for each pixel as accurately as possible.

[0140] The second type of loss value is the crossover ratio (CUP) loss value. It determines the loss value by calculating the ratio of the intersection region to the union region between the predicted image mask group and the real mask group, as shown in the following formula:

[0141]

[0142] The crossover ratio loss can be determined based on the regional relationship between the image mask set and the real mask set, thus enabling the model to take into account the overall regional prediction effect.

[0143] The third type of loss value is the similarity loss value. It determines the loss value by calculating the similarity between the predicted image mask set and the real mask set, as shown in the following formula:

[0144]

[0145] The similarity loss value can be calculated for each pixel to determine the similarity loss between the image mask group and the real mask group. Therefore, it can further improve the model's prediction accuracy of the depth of field to which each pixel belongs.

[0146] Considering that the real mask group itself is a discrete numerical representation, and the real mask group is suitable for calculating the similarity loss value using probability values, a transformation can be performed on the real mask group. By utilizing the weighting characteristics of the bilinear interpolation algorithm, interpolation can be performed on the original real mask group to convert the values ​​of each pixel point of each depth of field into probability values ​​and adjust them to the numerical representation range of the corresponding image mask group, such as [0,1], thereby achieving label softening and making it usable for calculating the similarity loss value.

[0147] In one embodiment, the total loss value can be determined by weighted fusion of cross-entropy loss value and cross-union ratio loss value or similarity loss value. In another embodiment, the total loss value can be determined by weighted fusion of the above three loss values. By using multiple loss values ​​to jointly constrain the training of the image segmentation model, it can be ensured that the image segmentation model obtains a more accurate image segmentation capability.

[0148] In the process of calculating various loss values, the enhancement weights are introduced to weight the corresponding loss values. Since the enhancement weights characterize the edge information in the image, the image segmentation model can pay more attention to the edge features in the image, thereby improving the accuracy of edge recognition.

[0149] Based on the calculation principle of the total loss value corresponding to the training sample, it can be seen that for the supervision process of each training sample, with the help of the real mask group, each pixel is converted into a multi-classification task. Using the multi-classification labels provided by the real mask group for each pixel, the corresponding cross-entropy loss value can be determined based on the multi-classification task. In addition, the cross-union loss value can be determined as needed based on the real mask group, and the similarity loss value can be calculated based on the result of label softening by the real mask group. Finally, the total loss value corresponding to the training sample is determined as needed. By taking multiple measures, the training of the image segmentation model is constrained from multiple aspects, enabling the image segmentation model to achieve a more accurate image segmentation effect.

[0150] Step S2230: Determine whether the image segmentation model has converged based on the total loss value. If it has not converged, continue iterative training and correct the weight parameters of the image classification model until convergence is achieved.

[0151] Once the total loss value is determined, a target threshold is used to determine whether the image segmentation model has reached convergence. The total loss value is compared with this target threshold. When the total loss value reaches the target threshold, the image segmentation model is considered to have reached convergence, the training task can be terminated, and the image segmentation model can be used in the online inference stage. When the total loss value does not reach the target threshold, the image segmentation model is considered not to have reached convergence. Gradient updates are performed on the image segmentation model based on the total loss value, its weight parameters are corrected through backpropagation, and then iterative training continues by calling the next training sample from the training dataset until the image segmentation model is trained to convergence.

[0152] In some embodiments, the image segmentation model can be trained in batches. Each time, multiple training samples from the same batch are input into the image segmentation model to predict the corresponding image mask set. The corresponding total loss value of each training sample is calculated using the corresponding real mask set. However, instead of judging whether the model has converged for each training sample individually, the total loss values ​​of multiple training samples are combined into a single total loss value and compared with the target threshold to judge whether the model has converged. This can reduce the variance during parameter updates, making the image segmentation model converge more stably. On the other hand, it can make high use of matrix operations to perform effective gradient calculation, thereby improving the training speed.

[0153] As can be seen from the above embodiments, during the training process of the image segmentation model, feature maps corresponding to edge information are depicted based on the image mask group predicted by the image segmentation model based on the training samples. These feature maps are used as weights to calculate various loss values. Then, by combining various loss values, the total loss value corresponding to the training samples is determined as needed, considering factors such as pixel depth classification accuracy, regional classification accuracy, and image segmentation accuracy. Based on the total loss value, a decision is made on whether the image segmentation model has converged or needs gradient updates. This allows the image segmentation model to be trained to a convergent state quickly, efficiently, and at low cost, achieving excellent multi-depth image segmentation results. With the help of training samples containing clothing content, the image segmentation model is suitable for accurately and delicately segmenting the clothing subject image corresponding to a certain depth from the original image containing clothing content.

[0154] Based on any embodiment of this application, please refer to Figure 6 The specified image and the enhanced image are merged and input into an image segmentation model to determine a target image mask corresponding to the depth of field of the visible part of the clothing in the clothing content, including:

[0155] Step S1410: Merge the standard image and the enhanced image into the image segmentation model to obtain image mask groups corresponding to the standard image and the enhanced image respectively. Each image mask group contains multiple image masks corresponding to multiple preset depths of field, and the number of the multiple preset depths of field is three or more.

[0156] In this embodiment, the image segmentation model is pre-trained to predict image masks corresponding to three depths of field, thereby obtaining an image mask group composed of image masks of three depths of field. The three depths of field, according to the viewing angle depth of the real-world space represented by their respective images, correspond to the foreground, midground, and background. For the clothing content segmentation task in this application, the foreground can be the portion of the clothing visible to the human eye after it is worn; the midground can be the portion visible before the clothing is worn but becomes invisible due to partial occlusion by the body after being worn; and the background can be purely background content.

[0157] In other embodiments, there can be more than three arbitrary and determined depths of field. The image segmentation model can be trained accordingly to acquire the corresponding reasoning ability as needed.

[0158] After converting the original image into a standard image and then converting it into an enhanced image of the same scale, the standard image and the enhanced image can be processed into image data corresponding to the red, green, and blue channels, respectively. Then, they are merged together in the channel direction to obtain the image data corresponding to the entire training sample. This image data is then input into the image segmentation model to start inference.

[0159] The image segmentation model predicts the corresponding image mask set by sequentially downsampling and upsampling the image data. Similarly, the image mask set also consists of six channels: three channels are image mask sets generated for the standard image, and the other three channels are image mask sets generated for the enhanced image. Unlike the input image data, each channel in the image mask set of the same image corresponds to a different depth of field among the multiple preset depths of field. Therefore, taking three depths of field as an example, each depth of field—foreground, midground, and background—corresponds to two images, each yielding a corresponding image mask, forming the same pair. Since these two images are homologous in image content, the image masks in the same pair have semantic mutual reference meaning.

[0160] Step S1420: Perform weighted fusion on the two image masks corresponding to the depth of field of the visible part of the clothing to obtain the target image mask.

[0161] The clothing content image segmentation task applicable to this application requires segmenting the main clothing image from a given image. The main clothing image refers to the clothing content that constitutes the visible part of the clothing. Correspondingly, it is necessary to obtain the same pair of image masks corresponding to the depth of field where the main clothing image is located. For example, the main clothing image usually refers to the foreground in three preset depths of field. Therefore, the image masks corresponding to the foreground in the standard image and the enhanced image can be called respectively. Then, the two image masks corresponding to the foreground can be weighted and fused together by combining preset weights to merge them into the same target image mask.

[0162] In one embodiment, the following formula can be applied to weight two image masks with the same depth of field:

[0163] mask total =α*mask x0 +(1-α)*mask y0

[0164] Among them, mask totaal mask for the target image x0 A mask is an image mask corresponding to the foreground of a standard image. y0To enhance the image mask corresponding to the foreground of the image, α is the weight used to implement smoothing.

[0165] As can be seen, the target image mask achieves the comprehensive utilization of the semantics of two image masks at the same depth of field, which helps to simultaneously leverage the image quality advantages of the standard image with anti-aliasing and the enhanced image with optimized contrast. Thus, based on the target image mask, image extraction can be performed on any one of the original image, standard image, and enhanced image to obtain a high-quality clothing subject image.

[0166] Based on any embodiment of this application, please refer to Figure 7 Extracting the main image of the clothing from the clothing content based on the target image mask includes:

[0167] Step S1510: Transform the target image mask to the same scale as the original image;

[0168] Typically, the original image has a higher resolution and therefore better image quality. To obtain a higher quality and larger-scale image of the clothing subject, the clothing subject image can be extracted based on the original image. However, to perform image extraction, the target image mask needs to be transformed to the same scale as the original image.

[0169] The target image mask can be scaled using bilinear interpolation. Since the image mask predicted by the image segmentation model provides a probability value for each pixel belonging to the corresponding depth of field, which is a definite value within a specific range such as [0,1], the target image mask obtained is naturally also a probability value representation. Therefore, after scaling the target image mask using bilinear interpolation, it can still be ensured that the final target image mask has a flexible depth quantization value representation. Using this representation of the target image mask for image segmentation can achieve a more natural and delicate segmentation effect.

[0170] Step S1520: Calculate all connected components in the target image mask after scaling, and delete the connected components with smaller areas to obtain the main image mask.

[0171] Considering that the area of ​​the main clothing image usually accounts for a large proportion of the entire image and is a continuous region, other continuous regions, although they may also be represented as foreground, may be the result of misprediction. Therefore, we can first calculate all connected components based on the target image mask after scaling. Then, according to a preset small threshold, such as 0.1, all connected components smaller than this small threshold are deleted, while other connected components are retained to achieve a filtering effect. The mask obtained by filtering is used as the main image mask.

[0172] Step S1530: Extract the clothing subject image with corresponding depth of field from the clothing content of the original image based on the subject image mask.

[0173] After determining the subject image mask, the original image is extracted using the subject image mask to extract the clothing subject image.

[0174] As can be seen from the above embodiments, the target image mask generated based on the results predicted by the image segmentation model, after scaling and filtering out smaller connected components, yields a high-quality subject image mask. Since the subject image mask inherits the technical advantages obtained in each step of the processing process of this application, it can accurately extract a high-quality clothing subject image from the original image. The obtained clothing subject image is the visually visible part of the corresponding clothing content after it is worn on the human body, excluding the invisible parts after wearing. Therefore, it can be used to provide basic clothing images for various downstream tasks, such as virtual try-on services, and has a wide range of application scenarios.

[0175] Please see Figure 8 According to one aspect of this application, a clothing image extraction apparatus includes an original image acquisition module 1100, an original image denoising module 1200, an original image enhancement module 1300, a segmentation execution module 1400, and an image extraction module 1500. The original image acquisition module 1100 is configured to acquire an original image containing clothing content. The original image denoising module 1200 is configured to transform the original image into a standardized image with jagged edges removed from its image content. The original image enhancement module 1300 is configured to optimize the contrast of the standardized image to obtain an enhanced image. The segmentation execution module 1400 is configured to merge the standardized image and the enhanced image and input them into an image segmentation model to determine a target image mask corresponding to the depth of field of the visible portion of the clothing content that constitutes the clothing. The image extraction module 1500 is configured to extract the main image of the clothing from the clothing content based on the target image mask.

[0176] Based on any embodiment of this application, prior to the segmentation execution module 1400, the clothing image extraction device of this application further includes: a sample construction module, configured to construct training samples and their corresponding real mask groups based on the specification images and / or enhanced images corresponding to the sample images containing clothing content, and store them in the training dataset, wherein the real mask group contains three or more real masks, each real mask corresponding to a single corresponding preset depth of field; and a model training module, configured to use the training samples in the training dataset as input to the image segmentation model for training until convergence, wherein each training session predicts the image mask group corresponding to all preset depths of field for the training samples, and uses the real mask group corresponding to the training samples for supervised training.

[0177] Based on any embodiment of this application, prior to the sample construction module, the clothing image extraction device of this application further includes any one or more of the following modules: a rotation enhancement module, configured to rotate the sample image containing clothing content or the specification image of the sample image; and a color band enhancement module, configured to identify the color-blocked area of ​​clothing content in the sample image containing clothing content or the specification image of the sample image, and replace the color in the color-blocked area.

[0178] Based on any embodiment of this application, prior to the sample construction module, the clothing image extraction device of this application further includes: a first preparation module, configured to acquire a flat lay image of clothing and a model body image corresponding to the clothing content, synthesize a model clothing image, and synthesize the model clothing image as a foreground into a random background image to obtain the sample image; and / or, a second preparation module, configured to acquire a real-life clothing image, extract the human clothing image therein and fuse it with the real-life clothing image to obtain a composite sketch, and synthesize the composite sketch as a foreground into a random background image to obtain the sample image.

[0179] Based on any embodiment of this application, the sample construction module includes: a label acquisition unit, configured to acquire annotation information corresponding to the sample image, the annotation information including category labels corresponding to the depth of field of the pixels in the sample image; and a label conversion unit, configured to construct a one-hot encoded vector for each pixel in the training sample according to the category labels in the annotation information, the training sample being constructed using the sample image, wherein the corresponding depth-corresponding ground mask is represented by the same position elements of the one-hot encoded vectors of all pixels, and the ground mask group of the training sample is composed of multiple depth-corresponding ground masks.

[0180] Based on any embodiment of this application, the plurality of preset depths of field include a first depth of field indicating background content in the image, a second depth of field indicating the invisible portion of clothing content in the image after it is worn, and a third depth of field indicating the visible portion of clothing content in the image after it is worn.

[0181] Based on any embodiment of this application, all the real masks in each real mask group jointly represent the multi-class labels corresponding to the pixels of the image in the corresponding training sample. The multi-class labels constitute a one-hot encoded vector, wherein each element value represents whether the pixel corresponding to the multi-class label belongs to the depth of field corresponding to the real mask where the element value is located.

[0182] Based on any embodiment of this application, the model training module includes: a weight extraction unit, configured to extract feature maps corresponding to the edge information in the image mask group as enhancement weights; a loss calculation unit, configured to associate the enhancement weights to calculate two or more loss values ​​of the image mask group relative to the real mask group, and summarize them into a total loss value; and an iterative decision unit, configured to decide whether the image segmentation model has converged based on the total loss value, and if it has not converged, to continue iterative training and correct the weight parameters of the image classification model until convergence is achieved.

[0183] Based on any embodiment of this application, the segmentation execution module 1400 includes: an inference execution unit, configured to merge the standard image and the enhanced image into an image segmentation model to obtain image mask groups corresponding to the standard image and the enhanced image respectively, each image mask group containing multiple image masks corresponding to multiple preset depths of field, the number of the multiple preset depths of field being three or more; and a fusion processing unit, configured to perform weighted fusion of two image masks corresponding to the depth of field where the visible part of the clothing is located to obtain the target image mask.

[0184] Based on any embodiment of this application, the image extraction module 1500 includes: a scale transformation unit configured to transform the target image mask to the same scale as the original image; a filtering optimization unit configured to calculate all connected components in the target image mask after scale transformation, and delete the connected components with smaller areas to obtain the main image mask; and an extraction execution unit configured to extract the clothing main image with the corresponding depth of field from the clothing content of the original image based on the main image mask.

[0185] Another embodiment of this application provides a clothing image extraction device. For example... Figure 9The diagram shows the internal structure of a garment image extraction device. This device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database stores information sequences, and when executed by the processor, these computer-readable instructions enable the processor to implement a garment image extraction method.

[0186] The processor of the garment image extraction device provides computing and control capabilities, supporting the operation of the entire device. The device's memory can store computer-readable instructions, which, when executed by the processor, cause the processor to perform the garment image extraction method of this application. The network interface of the garment image extraction device is used for communication with a terminal.

[0187] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the clothing image extraction device to which the present application is applied. A specific clothing image extraction device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0188] In this embodiment, the processor is used to execute... Figure 8 The specific functions of each module are described, and the memory stores the program code and various data required to execute the above modules or sub-modules. The network interface is used to realize data transmission between user terminals or servers. In this embodiment, the non-volatile readable storage medium stores the program code and data required to execute all modules in the clothing image extraction device of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0189] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the clothing image extraction method of any embodiment of this application.

[0190] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.

[0191] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0192] In summary, this application can accurately extract the main image of the clothing subject, which is visible in the clothing content, from the original image containing clothing content. The obtained main image of the clothing subject is natural and delicate, providing reliable material for virtual try-on technology. It has a wide range of applicable scenarios and has a low implementation cost.

Claims

1. A method for extracting clothing images, characterized in that, include: Acquire the original image, which contains clothing content; The original image is transformed into a standardized image with jagged edges removed from its image content; Optimize the contrast of the specified image to obtain an enhanced image; The process of merging the standard image and the enhanced image and inputting them into an image segmentation model to determine a target image mask corresponding to the depth of field of the visible part of the clothing in the clothing content includes: merging the standard image and the enhanced image and inputting them into an image segmentation model to obtain image mask groups corresponding to the standard image and the enhanced image respectively, each image mask group containing multiple image masks corresponding to multiple preset depths of field, the number of preset depths of field being three or more; and performing weighted fusion on two of the image masks corresponding to the depth of field of the visible part of the clothing to obtain the target image mask. The main image of the clothing is extracted from the clothing content based on the target image mask.

2. The clothing image extraction method according to claim 1, characterized in that, Before merging the standardized image and the enhanced image into the image segmentation model, the following steps are included: Training samples and their corresponding real mask sets are constructed based on the specification images and / or enhanced images corresponding to the sample images containing clothing content, and stored in the training dataset. The real mask set contains three or more real masks, and each real mask corresponds to a single corresponding preset depth of field. The image segmentation model is trained until convergence by inputting training samples from the training dataset. Each training iteration predicts a set of image masks corresponding to all preset depths of field for the training samples, and the training is supervised by the set of real masks corresponding to the training samples.

3. The clothing image extraction method according to claim 2, characterized in that, Before constructing training samples and their corresponding set of ground truth masks based on the specification images and / or augmented images of sample images containing clothing content, the process includes any one or more of the following steps: The sample image containing clothing content or the specification image of the sample image is rotated. Identify the color-blocked areas of the clothing content from the sample image containing the clothing content or the specification image of the sample image, and replace the colors in the color-blocked areas.

4. The clothing image extraction method according to claim 2, characterized in that, Before constructing training samples and their corresponding set of ground truth masks based on sample images containing clothing content, the following steps are included: Obtain a flat lay image of the clothing and a model's body image corresponding to the clothing content, synthesize a model's clothing image, and use the model's clothing image as the foreground to synthesize it into a random background image to obtain the sample image; And / or, A real-life image of a person dressed in clothing is acquired, and the image of the person dressed in clothing is extracted from it and merged with the real-life image of the person dressed in clothing to obtain a composite sketch. The composite sketch is then used as the foreground and composited into a random background image to obtain the sample image.

5. The clothing image extraction method according to claim 2, characterized in that, Training samples and their corresponding set of ground truth masks are constructed based on the specification images and / or augmented images corresponding to sample images containing clothing content, including: Obtain the annotation information corresponding to the sample image, wherein the annotation information includes the category label corresponding to the depth of field of the pixel in the sample image; Based on the category labels in the annotation information, a one-hot encoding vector is constructed for each pixel in the training sample. The training sample is constructed using the sample image. The corresponding depth-of-field ground mask is represented by the elements at the same position of the one-hot encoding vectors of all pixels. The ground mask group of the training sample is composed of multiple depth-of-field ground masks.

6. The clothing image extraction method according to claim 2, characterized in that: The multiple preset depths of field include a first depth of field indicating background content in the image, a second depth of field indicating the portion of clothing content in the image that is not visible after being worn, and a third depth of field indicating the portion of clothing content in the image that is visible after being worn.

7. The clothing image extraction method according to claim 2, characterized in that: All the real masks in each set of real masks together represent the multi-class labels corresponding to the pixels of the image in the corresponding training sample. The multi-class labels form a one-hot encoded vector, where each element value represents whether the pixel corresponding to the multi-class label belongs to the depth of field corresponding to the real mask where the element value is located.

8. The clothing image extraction method according to claim 2, characterized in that, Each training iteration predicts a set of image masks corresponding to all preset depths of field for the training samples, and supervises training using the real mask set corresponding to the training samples, including: The feature maps corresponding to the edge information in the image mask group are extracted as enhancement weights; The enhancement weights are associated with two or more loss values ​​of the image mask group relative to the real mask group, and then summarized into a total loss value. The image segmentation model is determined based on the total loss value. If it does not converge, the training continues iteratively and the weight parameters of the image segmentation model are corrected until convergence is achieved.

9. The method for extracting clothing images according to any one of claims 1 to 8, characterized in that, Extracting the main image of the clothing from the clothing content based on the target image mask includes: The target image mask is transformed to the same scale as the original image; Calculate all connected components in the target image mask after scaling, and obtain the main image mask by deleting the connected components with smaller areas. Based on the subject image mask, the clothing subject image with the corresponding depth of field is extracted from the clothing content of the original image.

10. A clothing image extraction device, characterized in that, include: The original image acquisition module is configured to acquire the original image, which contains clothing content; The original image noise reduction module is configured to transform the original image into a standard image with jagged edges removed from its image content; The original image enhancement module is configured to optimize the contrast of the specified image to obtain an enhanced image; The segmentation execution module is configured to merge the standard image and the enhanced image and input them into an image segmentation model to determine a target image mask corresponding to the depth of field of the visible part of the clothing in the clothing content. This includes: merging the standard image and the enhanced image and inputting them into the image segmentation model to obtain image mask groups corresponding to the standard image and the enhanced image respectively, each image mask group containing multiple image masks corresponding to multiple preset depths of field, the number of preset depths of field being three or more; and performing weighted fusion on two of the image masks corresponding to the depth of field of the visible part of the clothing to obtain the target image mask. The image extraction module is configured to extract the main image of the clothing from the clothing content based on the target image mask.

11. A clothing image extraction device, comprising a central processing unit and a memory, characterized in that, The central processing unit is configured to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 9.

12. A non-volatile readable storage medium, characterized in that, It stores a computer program in the form of computer-readable instructions, which, when invoked by a computer, performs the steps included in the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes a computer program / instruction, which, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 9.

Images