Clothes display image extraction method, control device and medium

Abstract

The invention relates to a clothes management technology, and particularly provides a clothes display image extraction method, a control device and a medium. The method comprises the following steps: obtaining a clothes shot picture of a user; the clothes image in the photo is extracted based on a pre-established matting model, a first display image is obtained, and the matting model is established based on a matting algorithm. According to the method, the vivid clothes image with the soft edge is obtained, the extracted clothes image is used for the display image of the clothes, the quality of the clothes display image is improved, and the display image can be subsequently classified and stored in the cloud wardrobe for clothes management.

Description

Technical Field

[0001] The present invention relates to clothing management technology, and specifically provides a method for extracting clothing display images, a control device, and a medium. Background Art

[0002] With the continuous improvement of living standards, consumers' demand for clothing purchases is increasing day by day. They not only pay attention to the quality and style of clothing but also keenly choose to purchase corresponding clothing according to the change of seasons and fashion trends. In winter, they will buy warm sweaters and down jackets, pursuing the warmth performance and comfort of clothing; while in summer, they will choose cool short-sleeved shirts and shorts to enjoy the comfort of lightness and breathability. This consumption behavior demonstrates consumers' pursuit of quality of life and their keen insight into fashion trends, and also makes clothing purchases a pleasant life experience.

[0003] With the rapid development of clothing design and smart home industries, as well as the wide popularity of online shopping, consumers' demand for clothing is gradually becoming personalized, which undoubtedly provides broad development space for image processing technology in the clothing field. Against this background, cloud wardrobes have emerged. As a tool for online management of personal clothing, they can better meet consumers' personalized clothing needs.

[0004] The applications of cloud wardrobes in clothing management mainly include the following points:

[0005] Improve clothing utilization rate: Through cloud wardrobes, consumers can view, select, and update their clothing at any time, enabling better utilization of clothing during storage, improving clothing utilization rate, and reducing waste.

[0006] Personalized customization: Cloud wardrobes can help consumers quickly find and purchase clothing suitable for themselves according to their preferences, body types, colors, etc. This can not only improve consumers' shopping experience but also provide market information for clothing brands to better meet consumers' personalized needs.

[0007] Facilitate cleaning and drying: Cloud wardrobes can achieve remote cleaning and drying of clothing. Consumers can view the cleaning status of clothing at any time to ensure the cleanliness and hygiene of clothing. In addition, cloud wardrobes can also provide drying techniques and drying tools to help consumers better dry clothing and improve drying efficiency.

[0008] Data analysis and intelligent recommendation: Cloud wardrobes can collect consumers' clothing management data and provide personalized clothing management suggestions through data analysis and intelligent recommendation. For example, based on data such as consumers' purchase records and laundry habits, cloud wardrobes can recommend suitable clothing brands, styles, and colors for consumers.

[0009] Social interaction: The cloud wardrobe can provide social interaction functions, allowing consumers to share clothing photos, comments, and likes, thereby building a social network and increasing user stickiness and activity.

[0010] The first step in realizing the above functions of the cloud wardrobe is to extract the image of the clothes. However, existing clothing extraction technologies often have some problems, such as incomplete extraction of clothes, clothes folded together, blurred extraction of clothing boundaries, or the appearance of edge artifacts. These problems may lead to a chaotic and unaesthetic clothing management interface in the cloud wardrobe, inaccurate intelligent clothing recommendations, and poor virtual fitting effects.

[0011] Therefore, it is crucial to improve the accuracy of clothing extraction.

[0012] To solve these problems, we need to develop a new clothing extraction method that can accurately extract the image of the clothes. Summary of the Invention

[0013] To overcome the above defects, the present invention proposes a method, control device, and medium for extracting a clothing display image, which improve the quality of the clothing display image.

[0014] In a first aspect, the present invention provides a method for extracting a clothing display image, including:

[0015] Obtaining a photo of the user's clothing taken;

[0016] Extracting the clothing image in the photo based on a pre-established matting model to obtain a first display image, wherein the matting model is established based on the matting algorithm.

[0017] In a technical solution of the above method for extracting a clothing display image, the method for establishing the pre-established matting model includes:

[0018] Obtaining a number of sample photos of the user's clothing taken;

[0019] Respectively obtaining the corresponding matting sample images of the sample photos;

[0020] Training the matting model based on the matting algorithm according to the sample photos and the matting sample images.

[0021] In a technical solution of the above method for extracting a clothing display image, the training of the matting model based on the matting algorithm according to the sample photos and the matting sample images includes:

[0022] Taking the matting sample image as the ground truth, and calculating the loss of the matting image generated by the model through a preset loss function;

[0023] Backpropagation is performed based on the loss to update the parameters of the matting model, completing the current round of iterative training.

[0024] In one technical solution of the above-mentioned method for extracting clothing display images, after the step of extracting the clothing image in the photo based on a pre-established matting model to obtain a first display image, the method further includes:

[0025] Determine whether there is clothing mutilation and / or clothing deformation in the first display image;

[0026] If there is clothing mutilation, complete the mutilated clothing in the first display image. If there is clothing deformation, correct the deformed clothing in the first display image;

[0027] Obtain a second display image.

[0028] In one technical solution of the above-mentioned method for extracting clothing display images, the determination of whether there is clothing mutilation and / or clothing deformation in the first display image includes:

[0029] By extracting the key points of the clothing in the first display image, determine whether there is clothing mutilation and / or clothing deformation.

[0030] In one technical solution of the above-mentioned method for extracting clothing display images, the completion of the mutilated clothing in the first display image includes:

[0031] Based on an image generation algorithm, complete the mutilated clothing in the first display image.

[0032] The correction of the deformed clothing in the first display image includes:

[0033] Based on an image generation algorithm, correct the deformed clothing in the first display image.

[0034] In one technical solution of the above-mentioned method for extracting clothing display images, after the step of obtaining the second display image, the method further includes:

[0035] Change the color of the clothing in the second display image to obtain images of multiple clothing colors.

[0036] In one technical solution of the above-mentioned method for extracting clothing display images, the determination of whether there is clothing mutilation and / or clothing deformation in the first display image further includes:

[0037] According to a pre-established clothing mutilation recognition model, determine whether there is clothing mutilation in the first display image;

[0038] Determine whether there is clothing deformation in the first display image according to a pre-established clothing deformation recognition model.

[0039] In a second aspect, the present invention provides a control device, including a processor and a storage device. The storage device is adapted to store multiple program codes, and the program codes are adapted to be loaded and run by the processor to execute the method for extracting the clothing display image.

[0040] In a third aspect, the present invention provides a computer-readable storage medium, in which multiple program codes are stored, and the program codes are adapted to be loaded and run by a processor to execute the method for extracting the clothing display image.

[0041] One or more of the above technical solutions of the present invention have at least one or more of the following beneficial effects:

[0042] The present invention uses a matting model established based on the matting algorithm to extract the clothing image from the clothing photo taken by the user as the clothing display image, so that after the clothing photo taken by the user independently is cropped, a clothing image with soft and realistic edges can be obtained.

[0043] If the extracted clothes are still incomplete or deformed, the generation technology can be used for automatic generation to complete or correct the deformation; if you want to change the color of the extracted clothes, color processing can also be performed to obtain multiple images of the same style of clothes in different colors.

[0044] Finally, the extracted clothing image is used as the display image of the clothing. The present invention improves the quality of the clothing display image, and subsequently, the display image can be classified and stored in the cloud wardrobe for clothing management. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Referring to the accompanying drawings, the disclosure of the present invention will become easier to understand. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the protection scope of the present invention. In addition, similar numbers in the figures are used to represent similar components, where:

[0046] Figure 1 is a schematic flowchart of the main steps of the method for extracting the clothing display image according to an embodiment of the present invention;

[0047] Figure 2 is a schematic diagram of a wedding dress photo in an application scenario of the present invention;

[0048] Figure 3 is a schematic diagram of the extracted wedding dress in an application scenario of the present invention;

[0049] Figure 4It is the contour diagram of a wedding dress in an application scenario according to the present invention;

[0050] Figure 5 It is a schematic diagram of a photo taken of a down jacket in an application scenario according to the present invention;

[0051] Figure 6 It is a schematic diagram of a wedding dress cut out in an application scenario according to the present invention;

[0052] Figure 7 It is the contour diagram of a wedding dress in an application scenario according to the present invention;

[0053] Figure 8 It is a schematic flowchart of a method for establishing a matting model established in advance according to an embodiment of the present invention;

[0054] Figure 9 It is a schematic flowchart for determining whether there is clothing damage and / or clothing deformation in the first display image according to an embodiment of the present invention;

[0055] Figure 10 It is a schematic overall flowchart of an application scenario according to the present invention. Detailed implementation manners

[0056] The following describes some embodiments of the present invention with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0057] In the description of the present invention, "module" and "processor" may include hardware, software, or a combination of both. A module may include a hardware circuit, various suitable sensors, communication ports, memories, and may also include a software part, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, in hardware, or in a combination of both. The non-transitory computer-readable storage medium includes any suitable medium for storing program code, such as magnetic disks, hard disks, optical discs, flash memories, read-only memories, random access memories, and the like. The term "A and / or B" represents all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a meaning similar to "A and / or B" and may include only A, only B, or A and B. The singular terms "a" and "this" may also include the plural form.

[0058] The present invention provides a method for extracting a clothing display image. Refer to Figure 1 , including:

[0059] S1. Obtain a photo of the user's clothing taken.

[0060] S2. Extract the clothing image in the photo based on a pre - established matting model to obtain a first display image, where the matting model is established based on the matting algorithm.

[0061] Image Matting refers to extracting the foreground object of interest from an image while filtering out the background part. An image can be simply regarded as composed of two parts, namely the foreground and the background. Simply put, matting is to distinguish the foreground and background of a given image.

[0062] Image Matting technology: This technology aims to extract the foreground object from an image while retaining the details and texture of the original image. Matting is a technology for extracting the foreground object from the background by calculating the color and transparency of the image, aiming to estimate the transparency and distinguish the foreground and background. For clothes such as furry collars, skin - revealing clothes, and wedding dresses that are difficult to process, feathering processing can be performed, and the extraction effect is more natural.

[0063] The matting model of the present invention is established based on the matting algorithm, so that after the clothing photo taken by the user independently is extracted, a clothing image with soft and realistic edges can be obtained.

[0064] In an application scenario, when the user takes a photo of a wedding dress, during the shooting, the user or the model wears the wedding dress and takes a photo. Referring to Figure 2 ..., it is necessary to extract the wedding dress separately to obtain a picture with only the wedding dress, removing the person and the background. The obtained wedding dress picture is shown in Figure 3 In addition, the contour map of the first display image can also be obtained by extracting the clothing image in the photo based on a pre - established matting model. Referring to Figure 4 ..., and the overall shape of the clothing can be understood through the contour.

[0065] In an application scenario, when the user takes a photo of a down jacket, first lay the down jacket flat on a plane, and the user takes a photo of the down jacket. Referring to Figure 5 ..., then perform matting on the photo of the down jacket taken based on the matting algorithm to obtain an image of the down jacket alone after removing the background. Referring to Figure 6 In addition, the contour map of the first display image can also be obtained by extracting the clothing image in the photo based on a pre - established matting model. Referring to Figure 7 ..., and the overall shape of the down jacket can be understood through the contour.

[0066] The matting algorithm can be used to finely extract wedding dresses, which have transparent edges and more design details.

[0067] The image extracted by the present invention is the first display image, which is used as the display image of the clothing corresponding to the photographed photo and is used in the cloud wardrobe to facilitate clothing management.

[0068] The present invention adopts image matting technology, which can extract images more precisely than image segmentation technology. The traditional machine learning method of extracting clothes has the following disadvantages: the extracted clothes are incomplete, the background in the captured clothes image is regarded as part of the clothes, the clothes boundary extraction is rough and not delicate enough and has a jagged structure, and the wedding dress and some transparent clothes are extracted rough. The present invention aims to solve these problems. The segmentation technology based on deep learning used to extract clothes also has these problems.

[0069] An important difference between image segmentation technology and Matting is that segmentation focuses on the semantic understanding of each pixel, and pixels with the same semantics are divided into the same part. The result is several blocks. Whether the blocks are naturally connected is not considered. That is, it returns the pixel classification label, which is a binary classification problem. The pixel return value is 0 and 1, and the result is integer data. Matting returns the probability P of belonging to the foreground or background. The pixel return value is between 0 and 1 (including 0 and 1), so as to produce a gradient effect in the interaction area between the foreground and the background, making the cutout more natural. After the Matting model is trained, a value representing the transparency of the foreground will be generated for each pixel at the original image position, called Alpha. The set of all Alpha values ​​in the image is called Alpha Matte. Finally, using Alpha to perform pixel-wise operations on the pixels at the corresponding positions of the original image, a fine foreground can be achieved.

[0070] In one embodiment, referring to Figure 8 , the method for establishing the cutout model pre-established in S2 includes:

[0071] S21, obtaining a number of sample photos of the user's clothing;

[0072] S22, respectively obtaining cutout sample images corresponding to the sample photos;

[0073] S23, training a matting model based on the matting algorithm according to the sample photos and the matting sample images.

[0074] In one embodiment, the S23, training the matting model based on the matting algorithm according to the sample photo and the matting sample image, includes:

[0075] Using the matte sample image as the ground truth, calculate the loss for the matte image generated by the model through a preset loss function;

[0076] Based on the loss, perform backpropagation to update the parameters of the matte model, completing the current round of iterative training.

[0077] After the matte model is trained, input the captured clothing photo of the user into the trained matte model, and output the clothing image after matte extraction. The output clothing image is the first display image.

[0078] In one embodiment, after the step S2 of extracting the clothing image in the photo based on a pre-established matte model to obtain the first display image, refer to Figure 9 , the method further includes:

[0079] S3, determine whether there is clothing damage and / or clothing deformation in the first display image;

[0080] S4, if there is clothing damage, complete the damaged clothing in the first display image, and if there is clothing deformation, correct the deformed clothing in the first display image;

[0081] S5, obtain the second display image.

[0082] The present invention improves the accuracy and authenticity of clothing extraction in the cloud wardrobe. First, use image matting technology to extract clothes, further optimize the extracted clothes, use image generation technology to complete the damage, and improve the quality of the clothing display image.

[0083] In one embodiment, the step S3 of determining whether there is clothing damage and / or clothing deformation in the first display image includes:

[0084] By extracting the key points of the clothing in the first display image, determine whether there is clothing damage and / or clothing deformation.

[0085] There can be several key points selected, such as 6 - 12. Adopt the key point or feature point recognition method in the field of image recognition technology, such as using Py-OpenCV (SIFT key points) to realize the recognition of whether there is damage to the clothing in the clothing image. The key points can be selected as the collar, sleeves, cuffs, shoulders, hem, etc.

[0086] In one embodiment, in S4, the step of completing the damaged clothing in the first display image includes:

[0087] Based on the image generation algorithm, complete the damaged clothing in the first display image.

[0088] The correction of the deformed clothing in the first display image includes:

[0089] Correcting the deformed clothing in the first display image based on an image generation algorithm.

[0090] In an application scenario, the GAN algorithm and the fusion algorithm are used to implement image completion and correction.

[0091] In one embodiment, after step S5 of obtaining the second display image, the method further includes:

[0092] S6, changing the color of the clothing in the second display image to obtain images of various clothing colors.

[0093] In an application scenario, Photoshop is used to achieve color change, so that images of the same style of clothing presented in different colors can be pushed to the user. For example, a white shirt can be changed to colors such as blue, black, and yellow, and the user can consider clothing matching and purchase plans based on the images after color change.

[0094] In one embodiment, the determination of whether there is clothing mutilation and / or clothing deformation in the first display image further includes:

[0095] Judging whether there is clothing mutilation in the first display image according to a pre-established clothing mutilation recognition model;

[0096] Judging whether there is clothing deformation in the first display image according to a pre-established clothing deformation recognition model.

[0097] In an application scenario, collect clothing complete image samples and mutilated image samples of clothing with different degrees of mutilation. According to the mutilated image samples and their corresponding complete image samples, referring to the training process of the matting model, using the complete image samples as the ground truth, calculate the loss for the complete image samples generated by the clothing mutilation recognition model through a preset loss function;

[0098] Based on the loss, perform backpropagation to update the parameters of the matting model and complete the current round of iterative training.

[0099] After the training of the clothing mutilation recognition model is completed, input the mutilated clothing image to be processed (corresponding to the first display image), and output the complete clothing image.

[0100] In an application scenario, collect clothing complete image samples and deformed image samples of clothing with different degrees of deformation. According to the deformed image samples and their corresponding complete image samples, referring to the training process of the matting model, using the complete image samples as the ground truth, calculate the loss for the complete image samples generated by the clothing deformation recognition model through a preset loss function;

[0101] Backpropagation is performed based on the loss to update the parameters of the matting model, completing the current round of iterative training.

[0102] After the clothing deformation recognition model is trained, the deformed clothing image to be processed (corresponding to the first display image) is input, and the complete clothing image is output.

[0103] The second display image is an image that is neither incomplete nor deformed.

[0104] If there are clothing defects and clothing deformation in the first display image, the first display image is sequentially input into the clothing defect recognition model and the clothing deformation recognition model. The input order of the two models is not limited. It can be input into the clothing defect recognition model first and then the clothing deformation recognition model, or it can be input into the clothing deformation recognition model first and then the clothing defect recognition model, and finally the second display image is obtained.

[0105] If there is one of clothing defects and clothing deformation in the first display image, it is only necessary to input the first display image into the corresponding recognition model.

[0106] The clothing defect recognition model and the clothing deformation recognition model can also select existing generalized large models.

[0107] The present invention is described in combination with an application scenario. Refer to Figure 10 , the present invention first takes pictures of clothing images through the camera of the cloud wardrobe, then collects these clothing images, manually processes each picture with Photoshop software, saves the alpha map, and puts it into the image matting algorithm for training. Then when the user uses the cloud wardrobe and takes pictures and uploads them, the present invention will use the image matting technology to extract the clothes from the taken pictures, obtaining clothes with delicate extraction and soft and realistic edges. If the extracted clothes are still incomplete, the generation technology can be used for automatic generation and completion; if you want to change the color of the extracted clothes, the color can also be changed through the generation technology to obtain multiple pieces of the same style of clothes in different colors. Finally, the extracted clothes are classified and stored in the cloud wardrobe for management.

[0108] The present invention also provides a control device. In an embodiment of the control device according to the present invention, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the method for extracting a clothing display image in the above method embodiment. The processor can be configured to execute the program in the storage device, and the program includes, but is not limited to, the program for executing the method for extracting a clothing display image in the above method embodiment. For ease of description, only the parts related to the embodiments of the present invention are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present invention. The control device can be a control device formed by various electronic devices.

[0109] The present invention also provides a computer-readable storage medium. In an embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for executing the method for extracting a clothing display image in the above method embodiment. The program can be loaded and run by a processor to implement the above-mentioned washing machine clothing clamping detection method. For ease of description, only the parts related to the embodiments of the present invention are shown. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0110] It should be noted that although the above steps are described in a specific order in the above embodiments, those skilled in the art can understand that in order to achieve the effects of the present invention, it is not necessary for different steps to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these changes are within the protection scope of the present invention.

[0111] Those skilled in the art can understand that all or part of the processes in the method of the above-mentioned embodiment of the present invention can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0112] Furthermore, it should be understood that since the setting of each module is only to illustrate the functional units of the device of the present invention, the corresponding physical devices of these modules can be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only illustrative.

[0113] Those skilled in the art can understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principle of the present invention. Therefore, the technical solutions after splitting or combining will all fall within the protection scope of the present invention.

[0114] So far, the technical solution of the present invention has been described in combination with the preferred embodiments shown in the drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the protection scope of the present invention.

Claims

1. A method for extracting clothing display images, characterized in that: include: Get photos of the user's clothing; The clothing image in the photo is extracted based on a pre-established matting model to obtain a first display image, wherein the matting model is established based on a matting algorithm.

2. The method according to claim 1, characterized in that The method for establishing the pre-established cutout model comprises: Obtain several sample photos of the user's clothing; Respectively obtain cutout sample images corresponding to the sample photos; According to the sample photos and the cutout sample images, a cutout model is trained based on a matting algorithm.

3. The method according to claim 2, characterized in that The step of training the matting model based on the matting algorithm according to the sample photo and the matting sample image comprises: The cutout sample image is taken as the true value, and the loss of the cutout image generated by the model is calculated using a preset loss function; Back propagation is performed based on the loss to update the parameters of the cutout model to complete the current round of iterative training.

4. The method according to claim 1, characterized in that: After the step of extracting the clothing image in the photo based on the pre-established cutout model to obtain the first display image, the method further includes: Determining whether the clothing is damaged and / or deformed in the first display image; If there are incomplete clothes, the incomplete clothes in the first display image are completed; if there are deformed clothes, the deformed clothes in the first display image are corrected; A second display image is obtained.

5. The method according to claim 4, characterized in that The determining whether the first display image contains damaged clothing and / or deformed clothing comprises: By extracting key points of the clothes in the first display image, it is determined whether the clothes are damaged and / or deformed.

6. The method according to claim 4, characterized in that The completing the incomplete clothing in the first display image comprises: The incomplete clothing in the first display image is completed based on an image generation algorithm. The correcting the deformed clothing in the first display image comprises: The deformed clothing in the first display image is corrected based on an image generation algorithm.

7. The method according to claim 4, characterized in that After the step of obtaining the second display image, the method further includes: The clothing in the second display image is changed in color to obtain images of multiple clothing colors.

8. The method according to claim 4, characterized in that The determining whether the first display image contains damaged clothing and / or deformed clothing further comprises: Determining whether there are any clothing defects in the first display image according to a pre-established clothing defect recognition model; It is determined whether there is clothing deformation in the first display image according to a pre-established clothing deformation recognition model.

9. A control device, comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the method for extracting a clothing display image according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the method for extracting a clothing display image according to any one of claims 1 to 8.

Images