Clothing information acquisition method, intelligent wardrobe and control method thereof, and storage medium

Through the bilateral segmentation network, the acquisition of clothing color information is simplified, and the problems of complicated acquisition of clothing color information and complex network models in the prior art are solved, thereby achieving efficient identification of clothing color information.

CN119068066APending Publication Date: 2024-12-03QINGDAO HAIER WASHING MASCH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310638349.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the prior art, the process of obtaining clothing color information is cumbersome, and the network model for identifying clothing color information is complicated and difficult to debug.

Method used

Using a bilateral segmentation network, including detail branches and semantic branches, the clothing pictures are processed through a pre-trained segmentation network, pattern information and color categories are obtained, and the proportional information of each color category in the clothing is calculated, which simplifies the acquisition of clothing color information. process.

Benefits of technology

Directly identifying the color of each pattern of the clothing, eliminating the image steps after color space recognition and segmentation, optimizing the network model structure for identifying the color information of the clothing, and simplifying the process of obtaining the clothing color information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068066A_ABST
    Figure CN119068066A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of artificial intelligence, aims to solve the technical problems that in the prior art, the acquisition process of clothes color information is tedious, and a network model for identifying the clothes color information is also complex, and provides a clothes information acquisition method, an intelligent wardrobe, a control method of the intelligent wardrobe and a storage medium. Comprising the following steps: acquiring a clothes-containing picture; inputting the picture into a pre-trained segmentation network for processing to obtain pattern information on the clothes and a color category of each pattern; and obtaining the proportion information of each color category in the clothes according to the pattern information and the color category of each pattern. According to the invention, segmentation of the patterns of the clothes and identification of the color of each pattern are completed through the to-be-segmented network, the acquisition process of the clothes color information is simplified, and the structure of the network model for identifying the clothes color information is optimized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically provides a method for obtaining clothing information, a smart wardrobe and a control method thereof, and a storage medium. Background Art

[0002] As society continues to develop, people's requirements for quality of life are constantly increasing, and the requirements for the intelligence of smart furniture such as smart wardrobes are constantly increasing. At the same time, for clothes, the identification and analysis of the color characteristics of clothes can effectively promote the development of smart furniture.

[0003] At present, in the field of smart device technology, the color of an object is generally identified through color space. When the color of each pattern on a piece of clothing needs to be identified, a photo of the clothing is first obtained, and then the photo is sent to a network segmentation model for segmentation, and finally the segmented image data is subjected to color recognition to obtain the color information of the clothing.

[0004] However, the above technical solution is relatively cumbersome in obtaining clothing color information, and the network model for identifying clothing color information is also relatively complex. In the field of deep learning, the more complex the network model is, the more difficult it is to train the network model, and it is also more difficult to debug.

[0005] Accordingly, the art needs a new technical solution to solve the above problems. Application Contents

[0006] In order to solve the above-mentioned problems in the prior art, that is, to solve the technical problems in the prior art that the process of acquiring clothing color information is relatively cumbersome and the network model for identifying clothing color information is also relatively complex.

[0007] The first aspect of the present application provides a method for obtaining clothing color information, comprising:

[0008] Get pictures containing clothing;

[0009] Inputting the image into a pre-trained segmentation network for processing to obtain pattern information on the clothing and a color category of each pattern;

[0010] The proportion information of each color category in the clothing is obtained according to the pattern information and the color category of each pattern.

[0011] In one embodiment, the segmentation network is a bilateral segmentation network, which includes a detail branch and a semantic branch;

[0012] The step of inputting the image into a pre-trained segmentation network for processing to obtain pattern information on the clothing and a color category of each pattern includes:

[0013] Obtaining pattern information on the clothing through the detail branch;

[0014] The color category of each of the patterns is obtained through the semantic branch.

[0015] In one embodiment, obtaining the proportion information of each color category in the image according to the pattern information and the color category of each pattern includes:

[0016] The area of ​​each of the patterns is calculated to obtain the area ratio of the pattern in the image, and the area ratios of the patterns of the same color category are added together to obtain the proportion information of each color category in the clothing.

[0017] In one embodiment, the training method of the segmentation network includes:

[0018] Get a dataset including original pictures of clothing, labeled pictures, and label files, where

[0019] The original image includes a background and clothing, the label image includes pattern information on the clothing and a color category of each pattern, and the label file includes all color categories of the original image;

[0020] The segmentation network is trained based on the dataset.

[0021] A second aspect of the present application provides a control method for a smart wardrobe, the control method comprising:

[0022] The clothing color information acquisition method described in the first aspect is used to acquire the proportion information of each color category in each piece of clothing.

[0023] In one embodiment, the control method includes a method for storing clothes in a smart wardrobe, and the method for storing clothes in a smart wardrobe includes:

[0024] Get the picture of the clothing taken by the terminal;

[0025] Obtaining information about the proportion of each color category in the clothing;

[0026] Matching information of a vacant position in the smart closet according to information of the proportion of each color category in the clothing;

[0027] Establishing a mapping relationship between the ratio information and the free position;

[0028] The mapping relationship is saved.

[0029] In one embodiment, the control method further includes a method for a smart closet to retrieve clothing, and the method for a smart closet to retrieve clothing includes:

[0030] Searching for information of a position corresponding to the proportion information according to the mapping relationship;

[0031] Displays information about the location.

[0032] The third aspect of the present application also provides a smart wardrobe, comprising:

[0033] A camera module, which is used to obtain pictures containing clothing;

[0034] A processing module, which is used to execute the control method described in the second aspect;

[0035] A display module is used to display the location information.

[0036] In one embodiment, the processing module includes a sensor unit, and the sensor unit is used to obtain information about vacant positions in the smart wardrobe.

[0037] The fourth aspect of the present application provides a computer-readable storage medium, which stores multiple program codes, and the program codes are suitable for being loaded and run by a processor to execute the clothing color information acquisition method described in the first aspect, or to execute the smart wardrobe control method described in the second aspect.

[0038] Those skilled in the art can understand that the technical solution provided by the present application processes the image of clothing through a segmentation network, and after the image of clothing is segmented, directly identifies the color of each pattern of clothing. Compared with the prior art, it omits the step of identifying the segmented image through color space, simplifies the process of acquiring clothing color information, and optimizes the structure of the network model for identifying clothing color information. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings, in which:

[0040] Figure 1 A flow chart showing a method for obtaining clothing color information is shown;

[0041] Figure 2 Flowchart showing the training method of the segmentation network

[0042] Figure 3 A flow chart showing a method for storing clothes in a smart wardrobe;

[0043] Figure 4A flow chart of a method for taking clothes from a smart wardrobe is shown;

[0044] Figure 5 A schematic diagram of the front side of a T-shirt with a pattern is shown;

[0045] Figure 6 A schematic diagram of a T-shirt after the text pattern is removed is shown;

[0046] Figure 7 A schematic diagram of the structure of a smart wardrobe is shown.

[0047] List of reference numerals:

[0048] 1. Outer frame; 2. Vertical partition; 3. Horizontal partition; 4. Cabinet; 5. Pressure plate; 6. Pressure sensor; 7. Infrared sensor; 8. Clothesline; 9. T-shirt; 91. Soft curve; 92. Hollow area; 93. White pattern. DETAILED DESCRIPTION

[0049] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0050] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0051] In view of the technical problems in the prior art that the process of acquiring clothing color information is relatively cumbersome and the network model for identifying clothing color information is also relatively complex, the first aspect of the present application provides a method for acquiring clothing color information, and based on the clothing color information acquisition method, provides a smart wardrobe and a control method thereof, as well as a computer-readable storage medium.

[0052] First, some terms mentioned in this application are explained as follows.

[0053] Pattern information: The pattern information mentioned in this application refers to all the patterns on a piece of clothing. For example, for a white T-shirt 9 with patterns, referring to Figure 5 , Figure 5 , a front schematic view of a patterned T-shirt 9 is shown. The T-shirt 9 is printed with a pattern of the words "Haier". Among them, the two characters "Haier" are surrounded by smooth curves, and a hollow area 92 is formed in the middle of the font. When the smooth curves surrounding the two characters "Haier" are black, the black pattern of this T-shirt includes the smooth curves surrounding the character "Hai" and the smooth curves surrounding the character "Er". And the white pattern 93 of this T-shirt, referring to Figure 6 , the white pattern 93 includes the entire white pattern 93 after removing the two characters "Haier". For the several hollow areas 92 formed between the characters "Hai" and "Er", each hollow area 92 is a small pattern. Those skilled in the art can understand that the pattern mentioned in this application refers to a continuous area with the same color. Therefore, Figure 5 the pattern information of the T-shirt 9 in Figure 6 includes the white pattern 93 after removing the two characters "Haier" in

[0054] Color category: The color category in this application refers to the color of the pattern. For example, for the T-shirt 9 in Figure 5 , the color category corresponding to the white pattern 93 is white, and the color category corresponding to the black pattern is black.

[0055] Color information: The color information mentioned in this application refers to the proportion information of each color category in the clothing. It should be noted that the proportion information of each color in the clothing does not refer to the actual proportion of this color in the whole piece of clothing, but refers to the proportion of each color on the clothing in the picture containing the clothing. For example, Figure 5The T-shirt 9 in the figure only has the front side, so when calculating the color information of this T-shirt 9, only the proportion of each color on the front side is considered. Assuming that the white pattern 93 on the front side of the T-shirt 9 occupies 95% of the front side of the T-shirt 9 and the area of ​​the black pattern occupies 5% of the front side of the T-shirt 9, then the color information of this T-shirt 9 is - white: 95%, black: 5%. In addition, if the back side of this T-shirt 9 is only white, then when observing from the back side of this T-shirt 9, the color information of this T-shirt 9 is - white: 100%, that is, if a photo of the back side of this T-shirt 9 is taken, then the color information obtained from this back side photo is - white: 100%. Therefore, for the same piece of clothing, the color information of this clothing obtained will be different if the shooting angle is different.

[0056] The following is a detailed description of the method for obtaining clothing color information provided by the present application. The method for obtaining clothing color information can be used to process pictures or photos containing clothing to obtain the color information of the clothing.

[0057] This application first proposes a method for obtaining clothing color information to obtain a trained network model, and then uses the trained network model to process the original image.

[0058] The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The execution subject of the clothing color information acquisition method may be, for example, any electronic device with processing capabilities, such as a terminal or a server.

[0059] See also Figure 1 The method for obtaining clothing color information provided by this application includes the following steps:

[0060] Step 101, obtaining a picture containing clothing.

[0061] Specifically, the picture containing clothes includes clothes and images (the clothing image mentioned below refers to the clothes in the picture, and the background image mentioned below refers to the background in the picture).

[0062] The clothing image can be the front side of the clothing or the back side of the clothing. Since the patterns on the front and back sides of the clothing are usually not exactly the same, the color complexity of the front and back sides of the clothing is also different. Selecting the side with fewer color categories of clothing can make the segmentation result more accurate. However, when the color categories of clothing are fewer, it is possible that the color information of two pieces of clothing is the same. Therefore, the user can also select the side with more color categories of clothing according to the actual situation to increase the difference in color information between different clothing.

[0063] The background image can be a natural environment, a building, other objects or a simple solid color background. It is understandable that the acquired clothing pictures will inevitably have some background images, so the clothing can be made more prominent in the picture by selecting or adjusting the background. By processing the background image or selecting the background image (if the picture is obtained by taking a photo, the background of the photo can be selected as needed), the contrast, clarity and visibility of the clothing can be enhanced, so that the clothing image in the picture containing the clothing is more prominent.

[0064] The pictures containing the clothes can be taken in real time by the camera of the terminal device, or the pictures of the same model of clothes can be downloaded by the terminal device from the shopping platform. The terminal device can be a smart device such as a mobile phone, a tablet computer, a smart camera, a smart wardrobe, etc. The clothes are photographed by the cameras on these smart devices to obtain pictures containing the clothes. For example, the clothes can be photographed by the camera on the mobile phone, and then the photographed photos are sent to a pre-trained network segmentation model for processing, or the clothes can be photographed by the camera on the smart wardrobe to obtain pictures containing the clothes. In addition, it is also possible to log in to the shopping platform through the mobile phone, find the picture information of the clothes on the shopping platform, and then save the picture information of the clothes and send it to the pre-trained network segmentation model for processing.

[0065] Step 102: input the image into a pre-trained segmentation network for processing to obtain pattern information on the clothing and the color category of each pattern.

[0066] Segmentation Network (SegNet) is a deep convolutional neural network that can segment images. Segmentation network can identify and segment different regions in an image, for example, separating the background from the foreground objects. The working principle of segmentation network is based on deep convolutional neural network (CNN) and deconvolution operation. During the training process, segmentation network learns how to segment an image into different regions from a large number of annotated images.

[0067] In the present application, the segmentation network can segment the pattern on the clothes in the picture according to different colors to obtain the pattern information of the clothes, and then identify the color category of each pattern. Specifically, the segmentation network obtains the pattern information of the clothing image by extracting the position, outline, edge, shape of the clothes, and the position, outline, edge, shape and other information of the pattern on the clothes. Then, because the pixel values ​​of different colors are different, after the segmentation network undergoes convolution operation, the values ​​of the pixels of different colors after convolution are also different, and the pixels of the same color will fall into a specific value range after convolution. Therefore, by training the segmentation network to learn the relationship between the result after convolution and the color, the segmentation network distinguishes the color category of each pattern.

[0068] In one embodiment, the segmentation network used in the present application is a bilateral segmentation network, which includes a detail branch and a semantic branch. In a preferred embodiment of the present application, the bilateral segmentation network uses BiSeNetV2.

[0069] BiSeNetV2 includes a backbone network and an aggregation layer. The backbone network includes a detail branch and a semantic branch. The detail branch can obtain the pattern information on the clothes, and the semantic branch can obtain the color category of each pattern. Then, the aggregation layer aggregates the feature maps of the two branches to generate the final segmentation result, so that the final segmentation result includes the pattern information on the clothes and the color category of each pattern. In other words, different patterns are divided based on the position, contour, edge, and shape features segmented by the detail branch, and then the color category of each pattern is determined according to the value of each pixel after segmentation by the semantic branch.

[0070] Step 103: Obtaining information on the proportion of each color category in the clothing pattern according to the pattern information and the color category of each pattern.

[0071] The proportion information of each color category in the clothing refers to the proportion of the sum of the areas of the patterns of the same color category to the total area of ​​the clothing image. For example, when calculating the proportion information, first, the area of ​​each pattern on the clothing image is calculated to obtain the area proportion of each pattern in the clothing image, and then the area proportions of the patterns of the same color category are added together to obtain the proportion information of each color category.

[0072] In some embodiments of the present application, the area ratio of each pattern on the clothing image can be calculated by an integral method. It can be understood that the pattern on each piece of clothing may be regular or irregular, so the calculation of the area of ​​the pattern needs to be performed according to the irregular pattern.

[0073] Specifically, a rectangular coordinate system is established on the image containing clothing, and then each pattern on the clothing is divided into multiple rectangles of equal size. Finally, the area of ​​each pattern is calculated, and the area of ​​all rectangles in the pattern is added together to obtain the area of ​​the pattern. Similarly, the area of ​​the clothing image is calculated in the same way. First, the clothing image is divided into multiple rectangles of equal size, and then the sum of the areas of all rectangles is calculated to obtain the total area of ​​the clothing image. Finally, the areas of patterns of the same color category are added together and divided by the total area of ​​the clothing image to obtain the proportion of each color category in the clothing. Among them, the length and width of each rectangle can be obtained according to the coordinates of the vertices of the rectangle in the coordinate system. For example, assuming that the coordinates of the vertices of a rectangle are (X1, Y1), (X2, Y1), (X1, Y2), (X2, Y2), assuming X2>X1, Y2>Y1, then the length of the rectangle in the X direction is X=X2-X1, and the width in the Y direction is Y=Y2-Y1, so the area of ​​the rectangle is S=XY.

[0074] It should be noted that the above calculation of the area of ​​the clothing image and the area of ​​each pattern on the clothing is not the actual area, but the relative area. Since the size of the clothing image in the picture containing the clothing is proportionally scaled relative to the actual clothing, the ratio information of the pattern on the clothing and the clothing image is unchanged. Therefore, it is only necessary to calculate the ratio between the two in the same dimension to obtain the actual ratio information.

[0075] It is worth noting that the above ratio information can also be calculated through pixel information, that is, the above ratio information can be obtained by calculating the ratio of the number of pixels in each pattern to the number of pixels in the clothing image.

[0076] In one embodiment, see Figure 2 , the training method of the segmentation network includes the following steps:

[0077] Step 201, obtaining a data set including original pictures of clothing, label pictures and label files.

[0078] Step 202: training a segmentation network based on the data set.

[0079] The original image is equivalent to the image containing clothes mentioned above, which includes the background and clothes. In other words, the original image consists of the clothing image and the background image. The label image corresponds to the original image, and the clothing image in the label image is labeled, and the clothing image is divided into different patterns according to color, and the color category of each pattern is labeled, for example, Figure 1The black pattern in the label is labeled as "black" and the white pattern is labeled as "white". It is worth mentioning that although the background image of the label image may also be a color image, the training method provided by itself does not label the color categories of the background image one by one, but uniformly labels the background image as "background". Since the dataset contains a large number of original images and label images that correspond to the original images one by one, the label file contains the color categories in all label images and the labels of the background. For example, when the color categories marked in the label image in the dataset include red, yellow, blue, and green, the labels in the label file include (red, yellow, blue, green, background).

[0080] The original image can be obtained by shooting or downloading from a shopping platform. The clothes in the original image include T-shirts, sweaters, hoodies, down jackets, shorts, trousers, long skirts and other clothes. Since the shapes of different types of clothes are quite different, by adding pictures of multiple types of clothes to the dataset to increase the richness of the dataset, the segmentation network can be trained to adapt to the shapes of different types of clothes, making the recognition of the trained segmentation network more accurate. In addition, you can also take pictures of clothes under different light intensities, and then obtain pictures of each piece of clothing under different light intensities, so as to train the segmentation network to distinguish the colors of clothes under different lighting conditions.

[0081] It is worth mentioning that when obtaining the original image, the clothes in the image need to be in an unfolded state, and the image of the clothes in the image should correspond to the front or back of the clothes, so that the clothes have more pattern information and color information. Specifically, when the original image is taken, the clothes can be laid flat, hung, or worn on the body before taking the photo.

[0082] After obtaining a certain number of original images by shooting or downloading, we can rotate and scale all the original images to obtain different forms of the original images and expand the data set. For example, we can rotate the original image of a piece of clothing by 30°, 60°, 90°, and 120° respectively, and obtain the images rotated by 30°, 60°, 90°, and 120° as new original images, and add these new original images to the data set.

[0083] After expanding the data set, all the original images in the data set are normalized to control the range of the input vector. Through normalization, the data can be kept in the same range, eliminating the impact of scale, making data with different features or dimensions comparable and processable, and improving the speed and stability of model training.

[0084] Then, label images are created based on the original images in the expanded data set. The clothes in each original image are divided into different patterns according to color, and the color category of each pattern is marked. Finally, the background image in each original image is marked as "background", thereby completing the creation of label images. It should be pointed out that the label images are labeled after the color of each pattern of the clothes is manually identified, and then the segmentation network learns the characteristics of these colors. Since people are more adaptable to lighting conditions, the situation where large recognition errors are caused by differences in lighting conditions is avoided.

[0085] Create a label file according to the color categories involved in the label image, that is, add the color categories used when annotating the label image to the label file, and add the "background" label to the label file.

[0086] When training the segmentation network, the original image and the labeled image can be divided into two parts, one of which is used as a training set to train the segmentation model, and the other is used as a validation set to validate the trained model. The trained segmentation network is validated by the validation set, and the segmentation network with the best segmentation intersection-over-union ratio and the most accurate color category prediction is selected for obtaining clothing information.

[0087] The present application also provides a smart wardrobe. The smart wardrobe provided by the present application is described in detail below.

[0088] The smart wardrobe provided in this application includes a camera module, a processing module, and a display module.

[0089] The camera module includes a camera for obtaining pictures containing clothing. The camera can transmit data to the processing module through a wired connection, or the camera can be an independent device connected through a wireless transmission method such as Bluetooth, WiFi, etc.

[0090] The processing module includes an execution unit and a storage unit. The execution unit can execute some control methods of the smart wardrobe. The storage unit is used to store some data generated by the execution unit, and the execution unit can complete the control of the smart wardrobe by calling the data in the storage unit. The display module includes a display screen for human-computer interaction, for example, displaying the internal image of the smart wardrobe.

[0091] In one embodiment, the processing module further comprises a sensor unit, which comprises a pressure sensor 6 .

[0092] Specifically, see Figure 7The smart wardrobe includes an outer frame 00481, which defines an internal space for storing clothes. Vertical partitions 2 and transverse partitions 3 are arranged in the internal space, and the vertical partitions 2 and transverse partitions 3 divide the internal space into a number of different cabinets 4. A pressure plate 5 is arranged on the upper part of the transverse partition 3 of each cabinet 4, and a pressure sensor 6 is arranged between the pressure plate 5 and the transverse partition 3. When the amount of stored clothes in the cabinet 4 increases or decreases, the pressure on the pressure plate 5 changes. A clothes drying rod 8 can also be arranged in the cabinet 4, and the clothes drying rod 8 is used to hang clothes in the smart wardrobe, and then a pressure sensor 6 is arranged at the end or middle of the clothes drying rod 8 to detect the change of the pressure on the clothes drying rod 8.

[0093] The pressure sensor 6 can convert the change in pressure into an electrical signal and transmit it to the processing unit, and then the processing unit determines the increase or decrease in the amount of clothing stored in the corresponding cabinet 4 based on the change in pressure (when the pressure detected by the pressure sensor 6 increases, the amount of clothing in the corresponding cabinet 4 increases; when the pressure detected by the pressure sensor 6 decreases, the amount of clothing in the corresponding cabinet 4 decreases); the pressure sensor 6 converts this change into an electrical signal and transmits it to the processing unit, so that the processing unit executes the control method of the smart wardrobe based on the change.

[0094] In particular, since the weight of each piece of clothing is different, it is impossible to judge whether the cabinet 4 is full of clothing by the weight of the clothing in the cabinet 4. Therefore, the sensor unit also includes an infrared sensor 7. By arranging an infrared sensor 7 on the upper part of each cabinet 4, the height of the clothing in the cabinet 4 can be judged. When the clothing in the cabinet 4 is full, the infrared sensor 7 can detect the clothing. When the cabinet 4 is not full of clothing or no clothing is placed, the infrared sensor 7 cannot detect the clothing. In other words, when the infrared sensor 7 detects the information of clothing, it means that the clothing in the cabinet 4 is full. When the infrared sensor 7 does not detect the information of clothing, it means that the cabinet 4 is not full of clothing or no clothing is placed. When the infrared sensor 7 detects the information of clothing, a "full" signal is transmitted to the processing unit. When the infrared sensor 7 does not detect the information of clothing, a "not full" signal is transmitted to the processing unit. The processing unit can display the internal image of the smart wardrobe on the display screen according to the feedback result of the infrared sensor in each cabinet 4, and highlight the "not full" cabinet 4 on the internal image.

[0095] The present invention also provides a method for controlling a smart wardrobe, which adopts the clothing color information acquisition method in the first aspect to acquire the proportion information of each color category of each piece of clothing in the clothing.

[0096] Specifically, the control method includes a method for the smart closet to store clothes, and a method for the smart closet to take clothes.

[0097] In one embodiment, see Figure 3 ,The method for storing clothes in a smart wardrobe includes the following steps:

[0098] Step 301, obtaining a picture of clothing taken by a terminal.

[0099] The terminal may be a mobile terminal, such as a smart phone, a tablet computer, a smart wearable device, a smart camera, etc., which takes pictures of the clothes to be stored through the camera end of these terminals, and then sends the pictures to the smart wardrobe. Alternatively, the terminal may be the smart wardrobe itself, which takes pictures of the clothes through the camera set on the smart wardrobe. Finally, the terminal sends the pictures to the processing module of the smart wardrobe.

[0100] Step 302, obtaining information on the proportion of each color category in the clothing.

[0101] The processing module of the smart wardrobe executes the clothing color information acquisition method provided in the first aspect of the present application, thereby obtaining the proportion information of each color category in the clothing to be stored, for example, white: 95%, black: 5%.

[0102] Step 303, matching information of an empty position in the smart closet according to information of the proportion of each color category in the clothing.

[0103] First of all, it should be noted that the smart wardrobe includes several compartments, the positions mentioned above correspond to the compartments in the smart wardrobe, and the empty positions refer to the compartments that are not full of clothes or have no clothes. Therefore, the empty compartments mentioned below refer to the empty positions mentioned above.

[0104] After the processing module of the smart closet obtains the color information of the clothes, it displays the internal image of the smart closet on the display screen of the smart closet, as shown in the figure. At this time, the empty compartments on the internal image of the smart closet can be highlighted, for example, by increasing the brightness of the empty compartments, or displaying prompts such as "not full" in the empty compartments, so that users can determine the information of the empty compartments through the internal image of the smart closet displayed on the display screen, which is convenient for users to find empty compartments that can store clothes.

[0105] In an optional embodiment, the user can select an empty compartment to store the clothes. Specifically, the user can first select an empty compartment and then confirm to store the clothes in the compartment. For example, when the display screen is a touch screen, the empty compartment can be selected by a single click, and then the selected empty compartment can be double-clicked to match the color information of the clothes with the empty compartment.

[0106] In another optional embodiment, a pressure sensor for detecting the weight of clothes in each compartment of the smart wardrobe can be provided. After the user selects an empty compartment according to the internal image of the smart wardrobe on the display screen, the clothes can be directly stored in the empty compartment. When the weight of the clothes in the empty compartment changes, the pressure sensor can send the change to the processing module, so that the processing module can match the compartment with the clothes.

[0107] Step 304: establish a mapping relationship between the ratio information and the free position.

[0108] Step 305: save the mapping relationship.

[0109] Specifically, after the processing module matches the color information of the clothing with an empty cabinet compartment, it establishes a mapping relationship between the two and stores the mapping relationship in a storage unit of the processing module.

[0110] In one embodiment, see Figure 4 , the methods of using clothes in smart wardrobe include:.

[0111] Step 401: searching for information of a position corresponding to the ratio information according to the mapping relationship.

[0112] Step 402: Display the location information.

[0113] In an optional embodiment, when the display screen of the smart wardrobe is a touch screen, the color information of the clothing can be input on the display screen, for example, white: 95%, black: 5%. The processing unit then searches for a mapping relationship containing the color information in the storage unit based on the color information, and obtains information about the position where the color information matches when the clothing is stored based on the mapping relationship. Finally, the internal image of the smart wardrobe is displayed on the display screen, and the compartment corresponding to the position of the internal image is highlighted.

[0114] In another optional embodiment, if the smart wardrobe does not have a display module, but has a voice interaction module. At this time, it is necessary to name each cabinet in the smart wardrobe so that each cabinet has a different name, for example, the first cabinet, the second cabinet... When the user takes the clothes, he / she tells the smart wardrobe the color information of the corresponding obligation, such as "95% white, 5% black clothes". Then, the voice interaction module transmits the user's voice to the processing unit. The processing unit searches for the mapping relationship containing the color information according to the voice, and obtains the information of the position where the color information matches when the clothes are stored according to this mapping relationship. Finally, the voice interaction module broadcasts the specific location of the clothes by voice, for example, "the clothes you want to take are in the first cabinet".

[0115] This application associates the color information of clothing with the position of the clothing in the smart closet, so that when taking clothing, the user can quickly find the cabinet where the clothing is located, reducing the time the user spends searching for clothing in the closet and improving the user experience.

[0116] The present application also provides a computer-readable storage medium, in which a plurality of program codes are stored, and the program codes are suitable for being loaded and run by a processor to execute the clothing color information acquisition method described in the first aspect, or to execute the smart wardrobe control method described in the second aspect. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0117] It should be noted that the present application is not limited thereto. Without departing from the principles of the present application, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A method for obtaining clothing color information, characterized in that: include: Get pictures containing clothing; Inputting the image into a pre-trained segmentation network for processing to obtain pattern information on the clothing and a color category of each pattern; The proportion information of each color category in the clothing is obtained according to the pattern information and the color category of each pattern.

2. The method for obtaining clothing color information according to claim 1, characterized in that: The segmentation network is a bilateral segmentation network, which includes a detail branch and a semantic branch; The step of inputting the image into a pre-trained segmentation network for processing to obtain pattern information on the clothing and a color category of each pattern includes: Obtaining pattern information on the clothing through the detail branch; The color category of each of the patterns is obtained through the semantic branch.

3. The method for obtaining clothing color information according to claim 1, characterized in that: The obtaining, according to the pattern information and the color category of each pattern, the proportion information of each color category in the clothing comprises: The area of ​​each of the patterns is calculated to obtain the area ratio of the pattern in the clothing, and the area ratios of the patterns of the same color category are added together to obtain the proportion information of each color category in the clothing.

4. The method for obtaining clothing color information according to claim 1, characterized in that: The training method of the segmentation network comprises: Get a dataset including original pictures of clothing, labeled pictures, and label files, where The original image includes a background and clothing, the label image includes pattern information on the clothing and a color category of each pattern, and the label file includes all color categories of the original image; The segmentation network is trained based on the dataset.

5. A method for controlling a smart wardrobe, characterized in that: The control method comprises: The clothing color information acquisition method according to any one of claims 1 to 4 is used to acquire the proportion information of each color category in each piece of clothing.

6. The control method according to claim 5, characterized in that: The control method includes a method for storing clothes in a smart wardrobe, and the method for storing clothes in a smart wardrobe includes: Get the picture of the clothing taken by the terminal; Obtaining information about the proportion of each color category in the clothing; Matching information of a vacant position in the smart closet according to information of the proportion of each color category in the clothing; Establishing a mapping relationship between the ratio information and the free position; The mapping relationship is saved.

7. The control method according to claim 6, characterized in that: The control method also includes a method for the smart closet to take clothes, and the method for the smart closet to take clothes includes: Searching for information of a position corresponding to the proportion information according to the mapping relationship; Displays information about the location.

8. A smart wardrobe, characterized in that: include: A camera module, which is used to obtain pictures containing clothing; A processing module, which is used to execute the control method described in any one of claims 5 to 7; A display module is used to display the location information.

9. The smart wardrobe according to claim 8, characterized in that: The processing module includes a sensor unit, and the sensor unit is used to obtain information about vacant positions in the smart wardrobe.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the clothing color information acquisition method described in any one of claims 1 to 4, or to execute the smart wardrobe control method described in any one of claims 5 to 7.