Intelligent washing control method, device and washing machine based on clothing image recognition

Abstract

The present invention discloses an intelligent washing control method, device and washing machine based on clothing image recognition, the method comprising the steps of: obtaining original photos of clothing in different postures in a washing cabin; preprocessing the obtained original photos to obtain a local sample set; sending the local sample set to a pre-trained neural network for a pre-estimation classification of the local sample set; performing secondary classification on the estimated result using a pre-built decision tree model to obtain a clothing material identification classification result; and determining a corresponding washing program according to the material identification classification result. The present invention starts from the local and takes the global into consideration, that is, it ensures the information provided by a higher resolution, and uses local images for model learning, which accelerates the image processing efficiency and reduces the hardware requirements. It does not require a large video memory to cope with the large matrix caused by high resolution. Through secondary classification, more accurate clothing material identification can be obtained, and a built-in targeted washing program can be applied according to the material.

Description

Technical Field

[0001] The present invention belongs to the technical field of laundry equipment, and in particular relates to an intelligent washing control method and device based on clothing image recognition, and a washing machine. Background Art

[0002] Most existing washing machines require users to select the washing mode on their own, and users are often not familiar with the washing mode that should be used for clothes. The existing technologies that determine the subsequent washing mode based on clothing recognition mainly include RFID recognition, multi-spectral recognition, image recognition, etc. In the existing solutions using image recognition, only a single-layer recognition logic is used, supplemented by some verification methods, which has the disadvantages of low input image clarity, poor recognition rate, large amount of calculation, and high hardware requirements. Summary of the invention

[0003] Technical purpose: In response to the above technical problems, the present invention provides an intelligent washing control method, device and washing machine based on clothing image recognition. It starts from the local and through secondary classification and recognition, it can obtain a more accurate clothing material recognition effect, provide built-in targeted washing plans according to the material, and the washing control process is simple and easy to operate.

[0004] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution:

[0005] An intelligent washing control method based on clothing image recognition, characterized in that it comprises the steps of:

[0006] Obtain multiple original photos of the current clothes in the washing compartment of the washing machine in different postures;

[0007] Preprocessing the acquired original photo to obtain multiple local samples corresponding to the current clothing to form a local sample set of the current clothing;

[0008] All the local samples obtained are sent to the pre-trained neural network for the first classification to obtain the estimated classification results of each local sample of the current clothing;

[0009] The estimated classification results of the local samples of the current clothing are counted to obtain the number of different types of materials into which the local samples of the current clothing are classified. The percentage of local samples of different types of materials in the local sample set of the current clothing is calculated, and all corresponding percentages are sent to the pre-built decision tree model for a second classification to obtain the material recognition and classification results of the current clothing.

[0010] Preferably, the neural network adopts a ResNet model, and the training set and test set of the neural network include multiple local sample sets corresponding to multiple materials, and the local sample set corresponding to each material is obtained by the following steps:

[0011] Select a clothing sample of a certain material, place it in the laundry cabin, and take multiple original photos of the clothing sample in different postures;

[0012] Preprocessing the original photos of the clothing samples to obtain multiple partial pictures of the clothing samples;

[0013] According to the purchase information or label information of the clothing samples, the partial images are marked with material labels, and the partial images constitute a partial sample set of the corresponding material.

[0014] Preferably, the input vector of the decision tree model is the percentage of local samples classified as different types of materials in each local sample set obtained after the first classification, the vector length is the same as the number of material types, and the decision tree model outputs the corresponding material number.

[0015] Preferably, the pretreatment comprises the steps of:

[0016] The original photo is cropped into a partial image of a preset pixel size, wherein the partial image includes more than one clothing sample, and the original photo is randomly flipped upside down, left to right, or rotated 90 degrees or 180 degrees during the cropping process;

[0017] The acquired local images are converted into RGB three-channel storage format, and the brightness of each local image is calculated according to the intensity values ​​of the RGB three channels;

[0018] Filter out the local image with the highest brightness of each clothing sample, and calculate the brightness threshold of the corresponding clothing sample according to the brightness of the filtered local image;

[0019] According to the brightness threshold of each clothing sample, all local images of each clothing sample are processed to filter out the local sample set of each clothing sample;

[0020] Among them, the local sample set of each clothing sample is screened out in the following way of supplementing brightness:

[0021] Compare the brightness of the local image of the clothing sample with the brightness threshold of the corresponding clothing sample, and if it is greater than or equal to the brightness threshold, put the local image into the local sample set;

[0022] If it is less than the brightness threshold, the local image is enlarged in proportion, and it is determined whether any intensity value of the RGB three channels of the corresponding local image after enlargement is greater than 255. If it is greater, the local image is discarded, otherwise the local image is placed in the local sample set.

[0023] Preferably, the pre-processing step further comprises:

[0024] After filtering the local sample sets of each clothing sample in a way of supplementing brightness, the sobel edge operator is used to calculate the sharpness of the local image, and the sharpness of the local image of the local sample set of each clothing sample itself is calculated as an outlier, and the local images with sharpness less than the sharpness threshold are eliminated.

[0025] Preferably, the pre-processing step further comprises: removing partial images in which the laundry compartment occupies an area exceeding a preset ratio on the image.

[0026] Preferably, the original photo comes from a terminal device with a photo taking function, and the terminal device performs the following steps:

[0027] Open the APP client on the terminal device, and the terminal device is connected to the control system network of the washing machine;

[0028] Placing the terminal device in a dedicated device slot on the washing machine compartment door;

[0029] After the APP client receives the interactive start signal sent by the control system, it starts the standard shooting process. During the shooting process, the laundry compartment rotates, and the terminal device obtains multiple original photos of the current clothes in the laundry compartment in different postures;

[0030] The standard shooting process includes the following steps:

[0031] The control system drives the laundry compartment to rotate so that the posture of the current clothes in the laundry compartment is updated multiple times, and the interval between two updates is a preset time;

[0032] After each update of the posture of the current clothing, the terminal device focuses and takes one or more high-resolution photos of the current clothing within the preset time interval, and transmits them to the control system.

[0033] Preferably, the washing machine is provided with a sensor for sensing whether the terminal device is placed in the dedicated device slot. After the sensor is triggered, it sends a confirmation signal to the control system. After the control system receives the confirmation signal sent by the sensor set on the washing machine, it sends an interactive start signal to the APP client.

[0034] An intelligent washing control device based on clothing image recognition, characterized by comprising:

[0035] A photo acquisition module is used to acquire multiple original photos of the current clothes in the washing compartment of the washing machine in different postures;

[0036] A local sample set acquisition module is used to pre-process the acquired original photo to obtain multiple local samples corresponding to the current clothing to form a local sample set of the current clothing;

[0037] The first classification module is used to send all the local sample sets obtained into the pre-trained neural network for the first classification to obtain the estimated classification results of each local sample set of the current clothing;

[0038] The second classification module is used to count the estimated classification results of the local samples of the current clothing, obtain the number of local samples of the current clothing classified into different types of materials, calculate the percentage of local samples of different types of materials in the local sample set of the current clothing, and send all corresponding percentages to the pre-built decision tree model for a second classification to obtain the material recognition and classification results of the current clothing;

[0039] The decision module is used to determine the corresponding washing program according to the material identification and classification results.

[0040] Preferably, the photo acquisition device is an external terminal device with a photo taking function, and an APP client is provided in the terminal device.

[0041] A washing machine is characterized by being provided with a control system and the intelligent washing control device based on clothing image recognition.

[0042] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects:

[0043] The present invention uses the model obtained by deep learning of neural networks for classification processing, and uses a decision tree to perform secondary classification processing on the results obtained. It starts from the local and takes the global into consideration, ensuring the information provided by higher resolution, that is, the global advantage, and uses local images for model learning, which accelerates the image processing efficiency, obtains better clothing material recognition effect, reduces memory consumption for storing large images, speeds up the overall processing speed and reduces hardware requirements, and realizes intelligent washing control according to the material classification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of the intelligent washing control method based on clothing image recognition of the present invention;

[0045] Figure 2 This is a flow chart of the intelligent washing control method for taking original photos using a mobile phone in Embodiment 1 of the present invention;

[0046] Figure 3 This is a standard shooting flow chart of the mobile phone shooting photos of the current clothes in the laundry cabin in the first embodiment;

[0047] Figure 4 This is a schematic diagram of the structure of a washing machine door. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0049] Embodiment 1

[0050] Figure 1 As shown, this embodiment proposes an intelligent washing control method based on clothing image recognition, comprising the steps of:

[0051] Obtain multiple original photos of the current clothes in the washing compartment of the washing machine in different postures;

[0052] Preprocessing the acquired original photo to obtain multiple local samples corresponding to the current clothing to form a local sample set of the current clothing;

[0053] All the local samples obtained are sent to the pre-trained neural network for the first classification to obtain the estimated classification results of each local sample of the current clothing;

[0054] The estimated classification results of the local samples of the current clothing are counted to obtain the number of different types of materials into which the local samples of the current clothing are classified. The percentage of local samples of different types of materials in the local sample set of the current clothing is calculated, and all corresponding percentages are sent to the pre-built decision tree model for a second classification to obtain the material recognition and classification results of the current clothing.

[0055] Specifically, the neural network adopts the ResNet model. The training set and test set of the neural network include multiple local sample sets corresponding to multiple materials. The local sample set corresponding to each material is obtained by the following steps:

[0056] Select a clothing sample of a certain material, place it in the laundry cabin, and take multiple original photos of the clothing sample in different postures;

[0057] Preprocessing the original photos of the clothing samples to obtain multiple partial pictures of the clothing samples;

[0058] According to the purchase information or label information of the clothing samples, the partial images are marked with material labels, and the partial images constitute a partial sample set of the corresponding material.

[0059] Specifically, the pretreatment comprises the steps of:

[0060] The original photo is cropped into a partial image of a preset pixel size, wherein the partial image includes more than one clothing sample, and the original photo is randomly flipped upside down, left to right, or rotated 90 degrees or 180 degrees during the cropping process;

[0061] The acquired local images are converted into RGB three-channel storage format, and the brightness of each local image is calculated according to the intensity values ​​of the RGB three channels;

[0062] Filter out the local image with the highest brightness of each clothing sample, and calculate the brightness threshold of the corresponding clothing sample according to the brightness of the filtered local image;

[0063] According to the brightness threshold of each clothing sample, all local images of each clothing sample are processed to filter out the local sample set of each clothing sample;

[0064] Among them, the local sample set of each clothing sample is screened out in the following way of supplementing brightness:

[0065] Compare the brightness of the local image of the clothing sample with the brightness threshold of the corresponding clothing sample, and if it is greater than or equal to the brightness threshold, put the local image into the local sample set;

[0066] If it is less than the brightness threshold, the local image is enlarged in proportion, and it is determined whether any intensity value of the RGB three channels of the corresponding local image after enlargement is greater than 255. If it is greater, the local image is discarded, otherwise the local image is placed in the local sample set.

[0067] Furthermore, after the local sample sets of each clothing sample are screened out in a manner of supplementing brightness, the sobel edge operator is used to calculate the sharpness of the local image, and the sharpness of the local image of the local sample set of each clothing sample itself is calculated as an outlier, and the local images with sharpness less than the sharpness threshold are eliminated.

[0068] Specifically, multiple original photos of the current clothes in the washing compartment of the washing machine in different postures can be obtained by setting a camera module in the washing machine to collect the photos, or by using an external terminal device with a camera function. For example, in this embodiment, a dedicated device slot is set on the washing machine for placing terminal devices such as mobile phones, and the door of the washing machine is made of transparent material. The washing machine is provided with a sensor for sensing whether the terminal device is placed in the dedicated device slot. After the sensor is triggered, it sends a confirmation signal to the control system. After receiving the confirmation signal sent by the sensor set on the washing machine, the control system sends an interactive start signal to the APP client.

[0069] The terminal device performs the following steps to capture the original photo:

[0070] Open the APP client on the terminal device, and connect the APP client to the control system network of the washing machine;

[0071] Placing the terminal device in a dedicated device slot on the washing machine compartment door;

[0072] After the APP client receives the interactive start signal sent by the control system, it starts the standard shooting process. During the shooting process, the laundry compartment rotates, and the terminal device obtains multiple original photos of the current clothes in the laundry compartment in different postures;

[0073] Among them, the standard shooting process includes the following steps:

[0074] The control system drives the laundry compartment to rotate so that the posture of the current clothes in the laundry compartment is updated multiple times, and the interval between two updates is a preset time;

[0075] After each update of the posture of the current clothing, the terminal device focuses and takes one or more high-resolution photos of the current clothing within the preset time interval, and transmits them to the control system.

[0076] like Figure 2 As shown, taking a mobile phone as an example of a terminal device, the method of this embodiment is described in detail.

[0077] 1. Model parameter acquisition part:

[0078] S1.1. The user opens the client APP on the mobile phone and connects to the washing machine control system via Wi-Fi;

[0079] S1.2, the user puts the mobile phone into the dedicated mobile phone slot 2 on the door 1 of the drum washing compartment, such as Figure 4 As shown;

[0080] S1.3, the gravity sensor 3 in the mobile phone slot 2 on the door 1 is triggered, and sends a confirmation signal to the control system of the washing machine;

[0081] S1.4, the control system of the washing machine interacts with the mobile phone and starts shooting the standard process, such as Figure 3 As shown, several high-resolution photos of the clothes in different postures (up to 10 photos) are obtained, that is, the picture materials of each material of the clothes. Multiple high-resolution photos can provide more sample information, which is conducive to improving the effect of the model;

[0082] S1.5. Repeat the above steps S1.1-S1.4 for different clothing materials, accumulate enough (more than 5,000 clothing samples of different materials, and the number of clothing samples of each material is roughly the same), and form a picture material set of clothing samples for training subsequent models;

[0083] S1.6. For the clothing image material, the original material ratio is maintained, and it is randomly cropped into a local image of 224*224 pixels. In the process of processing, in order to reduce overfitting and increase robustness, random (with a probability of 50%) upside down, left-right flipping, 90 degree or 180 degree rotation and other operations are performed; the cropping process uses two random center point selections, the purpose is to try to make the obtained local image located in the center of the image;

[0084] S1.7. Convert the obtained partial image into RGB three-color channel storage format (R, G, B), where the value range of R, G, B is 0-255, and calculate the brightness Y of each partial image according to the formula Y (brightness) = (0.299*R) + (0.587*G) + (0.114*B). Calculate the brightness value of the partial image with the highest brightness in the partial image of each clothing sample itself, and record it as Y_max. Amplify the RGB values ​​in (R, G, B) of all partial images of the clothing whose brightness values ​​are less than 1 / 8*Y_max in equal proportion, and the amplification multiple is the ratio of (1 / 8*Y_max) to its brightness Y_i. Small i represents the number of the partial image of each material clothing. For example, there are 1000 partial images of the material clothing numbered 200, so the value range of i in Y_i should be 1-1000. If a certain RGB value is greater than 255 after amplification, the partial image is discarded. This operation is to increase the brightness for edge detection;

[0085] S1.8. Under the premise of supplementing the brightness algorithm, the sobel edge operator is used to calculate the sharpness of the local image, and the sharpness of the local sample set of each clothing sample is calculated as an outlier (the most common parameter is 1.5), that is, the image with a sharpness less than Q1-1.5*(Q3-Q1) is regarded as a blurred local image and is removed. Q1 represents the 25th percentile in the outlier description, and Q3 represents the 75th percentile. This operation is to obtain clearer partial samples to provide reliable and effective information, rather than being disturbed by blurred images;

[0086] S1.9. Eliminate local image samples where metal bins occupy more than 65% of the local image. This operation is because the metal bins will reflect the clothing samples, but the image is blurred and cannot provide effective information. They can be eliminated manually or using a neural network with a pre-trained metal bin recognition algorithm.

[0087] S1.10. A large number of clothing samples of different materials are subjected to the above operations to obtain a partial image sample set. At the same time, the material classification of the clothing samples is confirmed based on the purchase information and label information of these clothing samples. The partial samples of clothing are regarded as the same material as the clothing, so a partial image sample set with material labels is obtained;

[0088] S1.11. After randomly shuffling the above local image sample set, 50% of the materials are used as the sample set and 50% of the materials are used as the test set. Under the premise of ensuring that the local samples of the same clothing sample only exist in one of the sample set and the test set, the model parameter batch is set to 64, and the ResNet model is used for training, and the learning rate function is set to the dynamic learning rate function noisy_linear_cosine_decay. Multiple experiments are performed to ensure that the number of training cycles is sufficient to stabilize the accuracy of multiple sample sets. Obtain the recognition model and model parameters of the local samples.

[0089] S1.12. According to the classification results of the local samples of the clothing samples, obtain the percentage of each clothing's local sample set that is classified into different types of materials. Use these percentage values ​​as new input vectors. For example, there are 21 different material classifications in total. In the local sample set of a certain clothing, 80% are considered to be material 1, 17% are considered to be material 2, and 3% are considered to be material 4. Then our input vector is (0.8, 0.17, 0, 0.03, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), and the vector length is the same as the total number of material classifications, which is 21. Then, use the decision tree model as the classifier for model training to obtain the relevant model parameters of the decision tree model. The output result of the decision tree model is the corresponding material number 4.

[0090] 2. Clothing identification and usage:

[0091] S1. The user opens the matching APP on the mobile phone and connects to the washing machine control system via wifi;

[0092] S2, the user puts the mobile phone into the dedicated mobile phone slot 2 on the door 1 of the drum washing compartment;

[0093] S3, the gravity sensor 3 in the mobile phone slot 2 on the door 1 is triggered, and sends a confirmation signal to the control system of the washing machine;

[0094] S4, the control system of the washing machine interacts with the mobile phone and starts shooting the standard process, such as Figure 2 As shown, a number of high-resolution photos of clothes in different postures are obtained, that is, picture materials of clothes of each material;

[0095] S5. For the clothing image material, the original material ratio is maintained, and it is randomly cropped into a local image of 224*224 pixels. In the process of processing, in order to reduce overfitting and increase robustness, random (with a probability of 50%) upside down flipping, left-right flipping, 90 degree or 180 degree rotation and other operations are performed; the cropping process uses two random center point selections, the purpose is to try to make the obtained local image located in the center of the image;

[0096] Convert the obtained partial image into RGB three-color channel storage format (R, G, B), where the value range of R, G, B is 0-255, and calculate the brightness Y of each partial image according to the formula Y (brightness) = (0.299*R) + (0.587*G) + (0.114*B). Calculate the brightness value of the partial image with the highest brightness in the partial images of each clothing sample itself, recorded as Y_max. Proportionally amplify the RGB values ​​in (R, G, B) of all partial images of the clothing whose brightness values ​​are less than 1 / 8*Y_max, and the amplification multiple is the ratio of (1 / 8*Y_max) to its brightness Y_i. If a certain RGB value is greater than 255 after amplification, the partial image is discarded. This operation is to increase the brightness for edge detection;

[0097] On the premise of supplementing the brightness algorithm, the sobel edge operator is used to calculate the sharpness of the local image, and the sharpness of the local sample set of each clothing sample is calculated as an outlier (the most common parameter is 1.5). Images with sharpness less than Q1-1.5*(Q3-Q1) are considered blurred local images and are removed;

[0098] In the local images, the local image samples where the metal warehouse occupies more than 65% are eliminated.

[0099] S6. The local sample set obtained from the clothing is identified and classified using the ResNet trained in step 1.11 in the model parameter acquisition part to obtain an estimated classification for each local sample.

[0100] S7. Based on the estimated classification results of the local samples of the clothing samples, the percentage of each piece of clothing classified into different types of materials is obtained, and these percentage values ​​are used as new input vectors. The decision tree model obtained in the model parameter acquisition part S1.12 is used for classification to obtain the final estimated classification of this clothing sample, and the classification result is passed to the control system of the washing machine.

[0101] This embodiment uses the rear camera of a mobile phone to take high-resolution clothing pictures, cooperates with the washing machine control system, and is processed into a local sample set through an algorithm. The model obtained by deep learning of a neural network is then used for the first classification process, and the results obtained are used for the second classification process using a decision tree. Finally, the material of the clothing is identified, and a built-in targeted washing plan is provided according to the material, including but not limited to the type of detergent, soaking time, washing time, forward and reverse intervals during washing, washing temperature, maximum spin speed, etc. Selecting a targeted washing plan can not only achieve a better washing effect for the clothing, but also achieve the purpose of energy saving and consumption reduction.

[0102] The overall algorithm uses the deep learning neural network ResNet and decision tree as the core model and classifier, starting from the local to achieve better recognition results. At the same time, because the high-resolution image is cropped before processing, it not only retains more information brought by the high resolution and avoids the loss caused by compression, but also uses small images for recognition processing, which speeds up the overall processing speed and reduces hardware requirements.

[0103] Embodiment 2

[0104] An intelligent washing control device based on clothing image recognition, comprising:

[0105] A photo acquisition module is used to acquire multiple original photos of the current clothes in the washing compartment of the washing machine in different postures;

[0106] A local sample set acquisition module is used to pre-process the acquired original photo to obtain multiple local samples corresponding to the current clothing to form a local sample set of the current clothing;

[0107] The first classification module is used to send all the local sample sets obtained into the pre-trained neural network for the first classification to obtain the estimated classification results of each local sample set of the current clothing;

[0108] The second classification module is used to count the estimated classification results of the local samples of the current clothing, obtain the number of local samples of the current clothing classified into different types of materials, calculate the percentage of local samples of different types of materials in the local sample set of the current clothing, and send all corresponding percentages to the pre-built decision tree model for a second classification to obtain the material recognition and classification results of the current clothing;

[0109] The decision module is used to determine the corresponding washing program according to the material identification and classification results.

[0110] In another embodiment of the present invention, a washing machine is provided, which is provided with a control system and the intelligent washing control device based on clothing image recognition described in the second embodiment.

[0111] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent washing control method based on clothing image recognition, characterized in that: Includes steps: Obtain multiple original photos of the current clothes in the washing compartment of the washing machine in different postures; Preprocessing the acquired original photo to obtain multiple local samples corresponding to the current clothing to form a local sample set of the current clothing; All the local samples obtained are sent to the pre-trained neural network for the first classification to obtain the estimated classification results of each local sample of the current clothing; Count the estimated classification results of the local samples of the current clothing, obtain the number of different types of materials that the local samples of the current clothing are classified into, calculate the percentage of local samples of different types of materials in the local sample set of the current clothing, and send all the corresponding percentages to the pre-built decision tree model for a second classification to obtain the material recognition and classification results of the current clothing; Determining a corresponding washing program according to the material identification and classification result; The neural network adopts a ResNet model. The training set and test set of the neural network include multiple local sample sets corresponding to multiple materials. The local sample set corresponding to each material is obtained by the following steps: Select a clothing sample of a certain material, place it in the laundry cabin, and take multiple original photos of the clothing sample in different postures; Preprocessing the original photos of the clothing samples to obtain multiple partial pictures of the clothing samples; According to the purchase information or label information of the clothing samples, the partial images are marked with material labels, and the partial images constitute a partial sample set of the corresponding material.

2. The intelligent washing control method based on clothing image recognition according to claim 1 is characterized in that: The input vector of the decision tree model is the percentage of local samples classified as different types of materials in each local sample set obtained after the first classification. The vector length is the same as the number of material types, and the decision tree model outputs the corresponding material number.

3. The intelligent washing control method based on clothing image recognition according to claim 1 is characterized in that: The pre-processing comprises the steps of: The original photo is cropped into a partial image of a preset pixel size, wherein the partial image includes more than one clothing sample, and the original photo is randomly flipped upside down, left to right, or rotated 90 degrees or 180 degrees during the cropping process; The acquired local images are converted into RGB three-channel storage format, and the brightness of each local image is calculated according to the intensity values ​​of the RGB three channels; Filter out the local image with the highest brightness of each clothing sample, and calculate the brightness threshold of the corresponding clothing sample according to the brightness of the filtered local image; According to the brightness threshold of each clothing sample, all local images of each clothing sample are processed to filter out the local sample set of each clothing sample; Among them, the local sample set of each clothing sample is screened out in the following way of supplementing brightness: Compare the brightness of the local image of the clothing sample with the brightness threshold of the corresponding clothing sample, and if it is greater than or equal to the brightness threshold, put the local image into the local sample set; If it is less than the brightness threshold, the local image is enlarged in proportion, and it is determined whether any intensity value of the RGB three channels of the corresponding local image after enlargement is greater than 255. If it is greater, the local image is discarded, otherwise the local image is placed in the local sample set.

4. The intelligent washing control method based on clothing image recognition according to claim 1, characterized in that: The pre-processing further comprises: After filtering the local sample sets of each clothing sample in a way of supplementing brightness, the sobel edge operator is used to calculate the sharpness of the local image, and the sharpness of the local image of the local sample set of each clothing sample itself is calculated as an outlier, and the local images with sharpness less than the sharpness threshold are eliminated.

5. The intelligent washing control method based on clothing image recognition according to claim 1, characterized in that: The pre-processing further includes: removing partial images in which the laundry compartment occupies an area exceeding a preset ratio on the image.

6. The intelligent washing control method based on clothing image recognition according to claim 1, characterized in that: The original photo comes from a terminal device with a photo taking function, and the terminal device performs the following steps: Open the APP client on the terminal device, and the terminal device is connected to the control system network of the washing machine; Placing the terminal device in a dedicated device slot on the washing machine; After the APP client receives the interactive start signal sent by the control system, it starts the standard shooting process. During the shooting process, the laundry compartment rotates, and the terminal device obtains multiple original photos of the current clothes in the laundry compartment in different postures; The standard shooting process includes the following steps: The control system drives the laundry compartment to rotate so that the posture of the current clothes in the laundry compartment is updated multiple times, and the interval between two updates is a preset time; After each update of the posture of the current clothing, the terminal device focuses and takes one or more high-resolution photos of the current clothing within the preset time interval, and transmits them to the control system.

7. The intelligent washing control method based on clothing image recognition according to claim 6 is characterized in that: The washing machine is provided with a sensor for sensing whether a terminal device is placed in a dedicated device slot. When the sensor is triggered, it sends a confirmation signal to the control system. After receiving the confirmation signal sent by the sensor provided on the washing machine, the control system sends an interactive start signal to the APP client.

8. An intelligent washing control device based on clothing image recognition, characterized in that: include: A photo acquisition module is used to acquire multiple original photos of the current clothes in the washing compartment of the washing machine in different postures; A local sample set acquisition module is used to pre-process the acquired original photo to obtain multiple local samples corresponding to the current clothing to form a local sample set of the current clothing; The first classification module is used to send all the local sample sets obtained into the pre-trained neural network for the first classification to obtain the estimated classification results of each local sample set of the current clothing; The second classification module is used to count the estimated classification results of the local samples of the current clothing, obtain the number of different types of materials that the local samples of the current clothing are classified into, calculate the percentage of local samples of different types of materials in the local sample set of the current clothing, and send all corresponding percentages to the pre-built decision tree model for a second classification to obtain the material recognition and classification results of the current clothing; A decision module, used to determine a corresponding washing program according to the material identification and classification result; The neural network adopts a ResNet model. The training set and test set of the neural network include multiple local sample sets corresponding to multiple materials. The local sample set corresponding to each material is obtained by the following steps: Select a clothing sample of a certain material, place it in the laundry cabin, and take multiple original photos of the clothing sample in different postures; Preprocessing the original photos of the clothing samples to obtain multiple partial pictures of the clothing samples; According to the purchase information or label information of the clothing samples, the partial images are marked with material labels, and the partial images constitute a partial sample set of the corresponding material.

9. The intelligent washing control device based on clothing image recognition according to claim 8, characterized in that: The method comprises the following steps: the photo acquisition module is an external terminal device with a photo taking function, and an APP client is arranged in the terminal device.

10. A washing machine, characterized in that: A control system and an intelligent washing control device based on clothing image recognition as claimed in claim 8 or 9 are provided.

Images