Laundry treating apparatus

Abstract

The embodiment of the invention provides clothes treating equipment. The clothes treating equipment comprises a motor; a door body and an inner cylinder; the camera is arranged on the door body and used for collecting pictures containing information of clothes in the inner drum; the fingerprint simulation sensor is arranged on the inner cylinder wall; the controller is configured to control the camera to execute image taking operation so as to obtain images collected by the camera, and the number information and color information of clothes in the inner barrel are determined based on the images. A motor is controlled to drive an inner barrel to rotate, so that the clothes in the inner barrel are in contact with a fingerprint simulation sensor; after the fingerprint simulation sensor makes contact with the clothes in the inner barrel, a clothes contact signal detected by the fingerprint simulation sensor is obtained, and material information of the clothes in the inner barrel is recognized through the clothes contact signal; and based on the number information, the color information and the material information of the clothes in the inner drum, matching a corresponding clothes processing program so as to control the clothes processing equipment to work according to the clothes processing program. The intelligent degree of the clothes processing equipment can be improved.

Description

Technical Field

[0001] This application relates to the technical field of clothing treatment, and particularly relates to a clothing treatment device. Background Art

[0002] A washing machine is a cleaning appliance that uses electrical energy to generate mechanical action to wash clothes. Currently, a load sensor is often equipped in a washing machine to measure the weight of the clothes in the washing tub, and then match a corresponding washing program according to the weight of the clothes. For example, when it is detected that the weight of the clothes in the washing tub is large, a strong washing program can be matched, and a higher washing water level and a longer washing duration can be set; when it is detected that the weight of the clothes in the washing tub is small, a gentle washing program can be matched, and a lower washing water level and a shorter washing duration can be set.

[0003] However, only matching a corresponding washing program according to the weight of the clothes in the washing tub, the matched washing program is often not accurate enough in washing parameters, and it is easy to cause damage to the clothes in the washing tub or low cleanliness, which is likely to cause dissatisfaction among users.

[0004] Therefore, there is an urgent need for a clothing treatment device that can accurately match a clothing treatment program. Summary of the Invention

[0005] To solve the above technical problems, an embodiment of the present application provides a clothing treatment device.

[0006] According to one aspect of the embodiment of the present application, the clothing treatment device provided by the embodiment of the present application includes: a motor; a door body and an inner tub; a camera disposed on the door body for collecting a picture containing information of the clothes in the inner tub; a fingerprint simulation sensor disposed on the inner tub wall; a controller electrically connected to the camera, the motor, and the fingerprint simulation sensor respectively, and the controller is configured to perform the following steps: controlling the camera to perform a picture-taking operation to obtain the picture collected by the camera, and determining the quantity information and color information of the clothes in the inner tub based on the picture; controlling the motor to drive the inner tub to rotate so that the clothes in the inner tub come into contact with the fingerprint simulation sensor; after the fingerprint simulation sensor comes into contact with the clothes in the inner tub, obtaining the clothes contact signal detected by the fingerprint simulation sensor to identify the material information of the clothes in the inner tub through the clothes contact signal; and matching a corresponding clothing treatment program based on the quantity information, color information, and material information of the clothes in the inner tub, and controlling the clothing treatment device to work according to the clothing treatment program.

[0007] In the above embodiments, by controlling the camera provided on the door body to perform an image-taking operation to collect pictures containing information about the clothes in the inner drum, the quantity information and color information of the clothes in the inner drum can be determined based on the pictures; and by controlling the motor to drive the inner drum to rotate, the clothes in the inner drum can be brought into contact with the fingerprint simulation sensor provided on the inner drum wall to obtain the clothes contact signal detected by the fingerprint simulation sensor, so as to accurately identify the material information of the clothes in the inner drum through the clothes contact signal; compared with only matching the corresponding clothes treatment program based on the weight of the clothes, the present application matches the clothes treatment program based on the quantity information, color information and material information of the clothes, making the matched clothes treatment program more accurate.

[0008] In an embodiment of the present application, based on the foregoing solution, the controller is further configured to perform the following steps: determine the quantity information of the clothes in the inner drum by distinguishing different clothes areas in the picture; determine the color information of each piece of clothes in the inner drum by performing color recognition on different clothes areas in the picture.

[0009] In an embodiment of the present application, based on the foregoing solution, the controller is further configured to perform the following steps: convert the color mode of the picture into the LAB mode, and calculate the color data of different clothes in the picture within the color space corresponding to the LAB mode; compare the calculated color data of different clothes with the color data in a preset standard color table to determine the color information of different clothes in the inner drum respectively.

[0010] In the above embodiments, by calculating the color data of different clothes in the picture within the color space corresponding to the LAB mode and comparing the calculated color data of different clothes with the color data in a preset standard color table, the color with the smallest color difference from the preset standard color table is obtained, and the color with the smallest color difference is used as the color information of the clothes, so that the color information of different clothes in the inner drum can be accurately identified.

[0011] In an embodiment of the present application, based on the foregoing solution, the controller is further configured to perform the following steps: obtain the clothes contact signal generated by the fingerprint simulation sensor when the clothes in the inner drum contact the fingerprint simulation sensor; determine the static pressure and high-frequency vibration of the clothes in the inner drum that contact the fingerprint simulation sensor based on the clothes contact signal; identify the material information of the clothes in the inner drum that contact the fingerprint simulation sensor based on the static pressure and the high-frequency vibration.

[0012] In the above embodiments, since the static pressure and high-frequency vibration on the surfaces of different clothing materials are different, therefore, based on the static pressure and high-frequency vibration of the clothing in the inner cylinder in contact with the fingerprint simulation sensor, the material information of the clothing in the inner cylinder in contact with the fingerprint simulation sensor can be accurately identified.

[0013] In an embodiment of the present application, based on the foregoing solution, the controller is further configured to perform the following steps: perform wavelet transform on the static pressure and high-frequency vibration of the clothing in the inner cylinder in contact with the fingerprint simulation sensor to obtain the frequency-domain characteristics of the clothing in the inner cylinder; based on the frequency-domain characteristics, identify the material information of the clothing in the inner cylinder in contact with the fingerprint simulation sensor.

[0014] In an embodiment of the present application, based on the foregoing solution, the controller is further configured to perform the following steps: classify the frequency-domain characteristics of the clothing in the inner cylinder in contact with the fingerprint simulation sensor to obtain at least one characteristic type of clothing height, clothing spacing, and clothing stiffness to which the frequency-domain characteristics belong; project the frequency-domain characteristics into the two-dimensional spaces corresponding to the different characteristic types to which they belong respectively to obtain the characteristic representations of the frequency-domain characteristics in each two-dimensional space respectively; determine the material information of the clothing in the inner cylinder in contact with the fingerprint simulation sensor according to the characteristic representations of the frequency-domain characteristics in each two-dimensional space.

[0015] In the above embodiments, by classifying the frequency-domain characteristics of the clothing in the inner cylinder in contact with the fingerprint simulation sensor and projecting the frequency-domain characteristics into the two-dimensional spaces corresponding to the different characteristic types to which they belong respectively, the characteristic representations of the frequency-domain characteristics in each two-dimensional space can be visualized, and at the same time, the global and local structures of the frequency-domain characteristics are retained, so that the material information of the clothing in the inner cylinder in contact with the fingerprint simulation sensor can be accurately determined according to the characteristic representations of the frequency-domain characteristics in each two-dimensional space.

[0016] In an embodiment of the present application, based on the foregoing solution, the clothing treatment device further includes a lighting device, and the lighting device is used to emit light when the camera performs an image-taking operation to enhance the ambient light in the inner cylinder.

[0017] In the above embodiments, by providing a lighting device in the clothing treatment device, there is good ambient light when the camera performs an image-taking operation, so that the pictures taken by the camera can have good clarity.

[0018] In an embodiment of the present application, based on the foregoing solution, the clothing treatment device further includes a display screen, and the display screen is used to display at least one of the material information of the clothing in the inner cylinder, washing and care, drying, and storage information.

[0019] In the above embodiments, by providing a display screen on the laundry treatment device, and the display screen can display at least one of the material information of the laundry in the inner drum, the washing, drying and storage information, it can be used for the user to learn this information, so as to better protect the laundry.

[0020] In one embodiment of the present application, based on the foregoing solution, the laundry treatment device is bound to a mobile terminal, and the mobile terminal is used to display at least one of the material information of the laundry in the inner drum, the washing, drying and storage information.

[0021] In the above embodiments, by binding the laundry treatment device to a mobile terminal, and the mobile terminal can display at least one of the material information of the laundry in the inner drum, the washing, drying and storage information, it can be used for the user to learn this information, so as to better protect the laundry.

[0022] In one embodiment of the present application, based on the foregoing solution, the laundry treatment device is a washing machine, and the controller is further configured to perform the following steps: based on the quantity information of the laundry in the inner drum, match the corresponding washing water level and detergent dosage; based on the color information of the laundry in the inner drum, match the corresponding water temperature; based on the material information of the laundry in the inner drum, match the corresponding washing duration and the rotation speed of the inner drum; generate a laundry treatment program suitable for the laundry in the inner drum according to the matched washing water level, detergent dosage, water temperature, washing duration and rotation speed of the inner drum.

[0023] In the above embodiments, matching the corresponding washing water level and detergent dosage according to the quantity information of the laundry in the inner drum can avoid wasting water resources and detergents, and prevent affecting the washing effect of the laundry; matching the corresponding water temperature according to the color information of the laundry in the inner drum can avoid color bleeding of the laundry caused by washing multiple colors of laundry together; matching the corresponding washing duration and the rotation speed of the inner drum according to the material information of the laundry in the inner drum can ensure the washing efficiency of the laundry; in summary, a laundry treatment program suitable for the laundry in the inner drum can be accurately generated according to the matched washing water level, detergent dosage, water temperature, washing duration and rotation speed of the inner drum.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. Description of the Drawings

[0025] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application. Obviously, the accompanying drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts. In the accompanying drawings:

[0026] Figure 1 is a schematic structural diagram of a laundry treatment device provided by an exemplary embodiment of this application;

[0027] Figure 2 is a schematic diagram of a camera provided by an exemplary embodiment of this application;

[0028] Figure 3 is a schematic diagram of a picture collected by a camera provided by an exemplary embodiment of this application;

[0029] Figure 4 is a schematic diagram of the composition of a simulation sensor provided by an exemplary embodiment of this application;

[0030] Figure 5 is a schematic diagram of the division of the clothing area in a picture provided by an exemplary embodiment of this application;

[0031] Figure 6 is a schematic diagram of the feature representation of the clothing height feature in a two-dimensional space provided by an exemplary embodiment of this application;

[0032] Figure 7 is a schematic diagram of the feature representation of the clothing spacing feature in a two-dimensional space provided by an exemplary embodiment of this application;

[0033] Figure 8 is an overall flowchart of a matching laundry treatment program provided by an exemplary embodiment of this application. Detailed Description of the Embodiments

[0034] Here, the exemplary embodiments will be described in detail, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. On the contrary, they are merely examples of devices and methods consistent with some aspects of this application as detailed in the appended claims.

[0035] The flowcharts shown in the accompanying drawings are merely illustrative and not necessarily include all content and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be further decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation. It should also be noted that: in this application, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0036] The terms "first", "second", "third", "fourth", etc. in the description, claims, and the accompanying drawings of this application are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0037] Currently, washing machines often come with load sensors. The weight of the clothes in the washing tub is measured by the load sensor, and then the corresponding washing program is matched according to the weight of the clothes. For example, when it is detected that the weight of the clothes in the washing tub is large, a strong washing program can be matched, and a higher washing water level and a longer washing duration can be set; while when it is detected that the weight of the clothes in the washing tub is small, a gentle washing program can be matched, and a lower washing water level and a shorter washing duration can be set.

[0038] However, only matching the corresponding washing program according to the weight of the clothes in the washing tub, the matched washing program is often not precise enough in terms of washing parameters, and it is very easy to cause damage to the clothes in the washing tub or low cleanliness, which is likely to lead to dissatisfaction among users.

[0039] To solve the above technical problems, this application proposes a laundry treatment device. The laundry treatment device can be a washing machine, a dryer, or a washing and drying integrated machine. Among them, the washing machine includes but is not limited to drum washing machines, pulsator washing machines, and agitator washing machines. The dryer includes but is not limited to exhaust dryers, condensation dryers, and heat pump dryers. This application does not limit the type of the laundry treatment device. Figure 1 It is a schematic structural diagram of a laundry treatment device provided by an exemplary embodiment of this application. As Figure 1 shown, the laundry treatment device 100 includes a motor 101, a door body 102, an inner tub 103, a camera 104, a fingerprint simulation sensor 105, and a controller (not shown in the figure). Each part in the laundry treatment device 100 will be described in detail below.

[0040] The motor 101, as the core component of the laundry treatment device 100, is responsible for providing power to the laundry treatment device 100 to achieve functions such as washing, rinsing, dewatering, and drying; the motor 101 can be connected to the inner drum 103 through a belt to transmit power to the inner drum 103 to make the inner drum 103 work, or can be directly connected to the inner drum 103 to control the inner drum 103 to work. The housing of the motor 101 is usually made of metal materials, such as cast iron or aluminum alloy, to ensure the firmness and durability of the structure of the motor 101; the coil of the motor 101 is usually wound with copper wire, which generates a magnetic field after being energized to drive the motor 101 to rotate. The motor 101 can be a DC brushless motor, a DC brushed motor, a variable frequency motor, a stepper motor, etc.

[0041] The door body 102 is usually made of materials such as metal (such as stainless steel), plastic (such as ABS, PC + ABS, etc.), and glass. It can not only ensure the safe enclosure of the inner drum 103 during the processes of washing, rinsing, dewatering, and drying, preventing water, clothes, and detergent from leaking out, but also protect the inner drum 103, the motor 101, and other internal components from being invaded by external dust, dirt, and other impurities. Usually, a transparent window or viewing window is provided on the door body 102, which is convenient for users to observe the state of the clothes in the inner drum 103. At the same time, an operation interface, such as an operation panel, a door handle, etc., is also provided. Users can select washing programs, drying programs, set parameters, start and pause, etc. through the operation panel, and open or close the door body 102 through the door handle.

[0042] The inner drum 103 is usually made of ceramic, stainless steel, plastic, and carbon fiber, and is arranged inside the door body 102 for placing clothes and performing washing and drying; usually, a washing trough and drum ribs are also provided inside the inner drum 103 to clean the clothes through friction and impact, which helps to improve the washing effect of the clothes.

[0043] The camera 104 is arranged on the door body 102 and can be a high-definition ordinary camera, an infrared camera, a 3D camera, etc. As Figure 2 shown, Figure 2 is a schematic diagram of the camera provided by an exemplary embodiment of the present application. The camera 104 can perform an image capture operation to collect pictures containing information about the clothes in the inner drum. For the collected pictures, please refer to Figure 3 shown, Figure 3It is a schematic diagram of a picture captured by a camera provided by an exemplary embodiment of the present application. The ways for the camera 104 to capture pictures include but are not limited to real-time capture, intermittent capture, and capture after reaching pre-set capture conditions; in an exemplary embodiment, the camera 104 can also be used to detect whether the clothes in the inner cylinder 103 are entangled during the washing process and take corresponding measures for processing, such as adjusting the washing parameters or reminding the user to intervene for processing. It should be noted that the number of cameras 104 can be one or more, and the present application does not limit the number of cameras 104.

[0044] The fingerprint simulation sensor 105 is provided on the wall of the inner cylinder 103 and can identify and sense objects by simulating the characteristics of human fingerprints. Thus, when it comes into contact with the clothes in the inner cylinder 103, it can obtain the detected clothing contact signal to identify the material information of the clothes in the inner cylinder 103 through the clothing contact signal. In an exemplary embodiment, the fingerprint simulation sensor 105 can be made of artificial fingerprints, flexible electrodes, and ionic gels. Specifically, please refer to Figure 4 , Figure 4 It is a schematic diagram of the composition of the simulation sensor provided by an exemplary embodiment of the present application. As Figure 4 shown, the fingerprint simulation sensor 105 can be composed of an artificial fingerprint made of polydimethylsiloxane (PDMS), a polyvinyl alcohol (PVA)-phosphoric acid (H3PO4, 8.3 wt.%) ionic gel layer, two flexible electrodes, a gold (Au) thin film on polyethylene terephthalate (PET), and a planar PDMS film for encapsulation. Similarly, the number of fingerprint simulation sensors 103 can be one or more, and the present application does not limit the number of fingerprint simulation sensors 103. Considering the use cost of the fingerprint simulation sensor and the material identification effect of the clothes in the inner cylinder 103, preferably, 3 fingerprint simulation sensors are evenly spaced on the wall of the inner cylinder 103.

[0045] In an exemplary embodiment, the laundry treatment device further includes a lighting device, which is used to emit light when the camera performs an image-taking operation to enhance the ambient light in the inner cylinder.

[0046] Furthermore, the lighting device can be an adjustable light source, an infrared lamp, and an LED light strip. The position of the lighting device can be set at the top, side of the inner cylinder, and near the camera. The lighting device is used to provide sufficient lighting for the inner cylinder so that there is good ambient light when the camera performs an image-taking operation, facilitating the camera to accurately capture the clothing information in the inner cylinder, and thus enabling the pictures taken by the camera to have good clarity.

[0047] The controller is electrically connected to the motor 101, the camera 104, and the fingerprint simulation sensor 105 respectively, and the controller is configured to perform the following steps:

[0048] S210, control the camera to perform an image acquisition operation to obtain the images captured by the camera, and determine the quantity information and color information of the clothes in the inner cylinder based on the images.

[0049] S220, control the motor to drive the inner cylinder to rotate so that the clothes in the inner cylinder come into contact with the fingerprint simulation sensor.

[0050] S230, after the fingerprint simulation sensor comes into contact with the clothes in the inner cylinder, obtain the clothes contact signal detected by the fingerprint simulation sensor to identify the material information of the clothes in the inner cylinder through the clothes contact signal.

[0051] S240, based on the quantity information, color information and material information of the clothes in the inner cylinder, match the corresponding clothes processing program to control the clothes processing device to work according to the clothes processing program.

[0052] The following will describe these 4 steps in detail respectively.

[0053] In S210, in order to avoid that the images captured by the camera during the image acquisition operation do not contain the clothes information in the inner cylinder, for example, there are no clothes in the inner cylinder in the images, preferably, set the preset condition for the camera to perform the image acquisition operation as detecting that there are clothes in the inner cylinder, that is, when the sensor detects that there are clothes in the inner cylinder, the camera can be controlled to perform the image acquisition operation. During the detection that there are clothes in the inner cylinder, the camera can capture images in real time or intermittently. For example, control the camera to perform the image acquisition operation every 10 s or 20 s. In order to avoid that too many images captured in real time affect the processing of the images, preferably, control the camera to acquire images of the clothes in the inner cylinder according to the set acquisition interval.

[0054] The images captured by the camera contain the clothes information in the inner cylinder, and based on the images, the quantity information and color information of the clothes in the inner cylinder can be determined. In an exemplary embodiment, the decision module can be configured to perform the following steps to determine the quantity information and color information of the clothes in the inner cylinder:

[0055] S310, determine the quantity information of the clothes in the inner cylinder by distinguishing different clothes regions in the image;

[0056] S320, determine the color information of each piece of clothes in the inner cylinder by performing color recognition on different clothes regions in the image.

[0057] The following will describe these 2 steps in detail respectively.

[0058] In S310, different clothing regions can be recognized through image recognition technology; in addition, based on deep learning technology, especially convolutional neural network (CNN) and image segmentation models, different clothing regions in the picture can also be recognized. Please refer to Figure 5 , Figure 5 which is a schematic diagram of the division of clothing regions in the picture provided by an exemplary embodiment of the present application. As Figure 5 shown, there are three clothing regions divided in the picture, namely clothing region 1, clothing region 2, and clothing region 3, and different clothing regions correspond to different clothes. Therefore, according to Figure 5 the schematic diagram of the divided clothing regions, it can be determined that the number of clothes in the inner drum is three pieces.

[0059] However, considering that the pictures collected by the camera at different times are different, and the distribution of clothes in the pictures may also be different. For example, at time t1, the camera performs an image acquisition operation to collect Picture 1. By analyzing Picture 1, it is recognized that there are two clothing regions in Picture 1, then it can be determined that the number of clothes in the inner drum is two pieces; while at time t2, the camera performs an image acquisition operation to collect Picture 2. By analyzing Picture 2, it is recognized that there are three clothing regions in Picture 2, then it can be determined that the number of clothes in the inner drum is three pieces. Therefore, in an exemplary embodiment, the number of clothes in the inner drum determined by different pictures can be compared, and the largest determined number of clothes is selected as the number information of the clothes in the inner drum. Exemplarily, if at most four clothing regions are divided in the picture collected by the camera, it can be determined that the number of clothes in the inner drum is four pieces; if at most five clothing regions are divided in the picture collected by the camera, it can be determined that the number of clothes in the inner drum is five pieces.

[0060] In another exemplary embodiment, the clothing regions divided in the picture collected by the camera can also be combined with the weight of the clothes detected by the sensor in the inner drum, and the weight range of the clothes can be set according to the number information of the clothes in the inner drum. For example, when it is determined that the number of clothes in the inner drum is one piece, the corresponding weight range of the clothes should be 0 kg - 2 kg; when it is determined that the number of clothes in the inner drum is two pieces, the corresponding weight range of the clothes should be 2 kg - 4 kg, and so on. Therefore, when it is recognized through picture analysis that there are two clothing regions in the picture and the weight of the clothes detected in the inner drum is also within the range of 2 kg - 4 kg, it can be accurately determined that the number of clothes in the inner drum is two pieces.

[0061] It can be understood that the weight range of the clothes increases according to the number of clothes in the inner tub. For each additional piece of clothing, the corresponding weight range will increase accordingly. However, the actual situation may be adaptively adjusted according to different types of clothes because the weights of different types of clothes often vary.

[0062] In S320, color recognition of different clothing areas can be performed through the CIE (International Commission on Illumination) algorithm. In addition, the RGB channel information of the picture can also be used to perform color recognition of different clothing areas. In an exemplary embodiment, the decision module can be configured to perform the following steps to determine the color information of different clothes in the inner tub:

[0063] S410, convert the color mode of the picture to the LAB mode, and calculate the color data of different clothes in the picture within the color space corresponding to the LAB mode;

[0064] S420, compare the calculated color data of different clothes with the color data in the preset standard color table to determine the color information of different clothes in the inner tub.

[0065] Furthermore, fully considering the sensitivity of the human eye to different colors, the CIE (International Commission on Illumination) algorithm is adopted in this embodiment to eliminate color differences to achieve the best color difference elimination effect. Specifically, the color mode of the picture is converted to the LAB mode to make up for the deficiencies of the two color modes of RGB (Red, Green, Blue) and CMYK (Cyan, Magenta, Yellow, and K represents Black). There are three channels in the LAB mode, namely the lightness (L) channel, the a channel, and the b channel. The lightness (L) channel is only used to record the brightness information in the picture, and the brightness of the pixel is represented by the value 0-100; the a channel only contains (cyan) green and (magenta) red information, which can be simply called the "red-green channel", and the (cyan) green is represented by the value 0-128, and the (magenta) red is represented by the value 0-127. When the value is 0, it represents neutral gray, that is, no red and no green; the b channel only contains blue and yellow information, which can be simply called the "yellow-blue channel", and the blue is represented by the value 0-128, and the yellow is represented by the value 0-127. When the value is 0, it represents neutral gray, that is, no yellow and no blue.

[0066] The color data may include the LAB values of pixel points, the average color value of the clothing area, color distribution statistics, etc.; there are 408 colors in the preset standard color table, and 281 colors in the reduced version. After the color mode of the picture is converted to the LAB mode, calculations can be performed within the color space corresponding to the LAB mode (i.e., the LAB color space) to obtain the LAB value of each pixel in the picture. Compare the LAB value of each pixel in the picture with the color data in the preset standard color table, calculate the color difference, and filter out the color with the smallest color difference from the preset standard color table, so as to identify the color category of the clothing area, and then determine the respective color information of different clothes in the inner drum. The color information may include color depth, color coordinate space, color brightness, etc.

[0067] It can be understood that the analysis and processing of the picture to determine the quantity information and color information of the clothes in the inner drum can be integrated into the control board of the clothing processing device or processed in the cloud server.

[0068] As can be seen from the above, by calculating the color data of different clothes in the picture within the color space corresponding to the LAB mode, and comparing the calculated color data of different clothes with the color data in the preset standard color table, the color with the smallest color difference from the preset standard color table can be obtained, and using this color with the smallest color difference as the color information of the clothes, the respective color information of different clothes in the inner drum can be accurately identified.

[0069] In S220, by controlling the motor to drive the inner drum to rotate, the clothes in the inner drum are continuously flipped, so that the clothes can come into contact with the fingerprint simulation sensor on the inner drum wall during the flipping process. The rotation speed of driving the inner drum by the motor should not be too high or too low. If the rotation speed of the inner drum is too low, the flipping amplitude of the clothes in the inner drum may be too small, and it is difficult to come into contact with the fingerprint simulation sensor on the inner drum wall. If the rotation speed of the inner drum is too high, the rolling amplitude of the clothes in the inner drum may be too large and they may fall from the high altitude in the inner drum, which is likely to cause damage to the clothes.

[0070] Therefore, a large number of experiments can be carried out in advance to set the maximum rotation speed value and the minimum rotation speed value of the inner drum. By controlling the rotation speed of the motor to drive the inner drum to be between the set maximum rotation speed value and the minimum rotation speed value of the inner drum, the situation where the clothes in the inner drum are difficult to come into contact with the fingerprint simulation sensor on the inner drum wall and the situation of clothing damage are effectively avoided.

[0071] In S230, after the fingerprint simulation sensor comes into contact with the clothes in the inner cylinder, the fingerprint simulation sensor can detect a clothes contact signal. The clothes contact signal is the signal generated when the fingerprint simulation sensor comes into contact with the clothes, and can reflect the static pressure and high-frequency vibration of the contacted clothes. The material information of the clothes refers to the type of material or fabric used for the clothes, including but not limited to material types such as wool, polyester, cotton linen, and nylon.

[0072] In an exemplary embodiment, the decision module may be configured to perform the following steps to identify the material information of the clothes in the inner cylinder that are in contact with the fingerprint simulation sensor:

[0073] S510, obtain the clothes contact signal generated by the fingerprint simulation sensor when the clothes in the inner cylinder come into contact with the fingerprint simulation sensor;

[0074] S520, determine the static pressure and high-frequency vibration of the clothes in the inner cylinder that are in contact with the fingerprint simulation sensor based on the clothes contact signal;

[0075] S530, identify the material information of the clothes in the inner cylinder that are in contact with the fingerprint simulation sensor based on the static pressure and high-frequency vibration.

[0076] Specifically, considering that humans can gently slide their fingers on the surface of an object and capture the static pressure and high-frequency vibration of the object, and the static pressure and high-frequency vibration on the surfaces of clothes of different materials are often different, a fingerprint simulation sensor is provided on the inner cylinder wall in this embodiment. The fingerprint simulation sensor is used to identify and sense the clothes by simulating the characteristics of human fingerprints. The readout circuit board can be used to collect the clothes contact signal generated when the fingerprint simulation sensor comes into contact with the clothes. Among them, the circuit board can be composed of five parts: a power module, a single-chip microcomputer module (STM32) that processes data and makes decisions as the central processing unit, an input / output interface module for communicating with external devices, and a signal processing module responsible for signal processing. The fingerprint simulation sensor can respond to static and dynamic stimuli (0 - 400 Hz), has a high spatial resolution of 15 μm pitch and 6 μm height, and a high-frequency resolution of 0.02 Hz at 400 Hz, and can accurately distinguish fine surface features with high precision. Through a large number of experiments, it has been proved that the sensory system integrated on the prosthetic fingertip can identify a variety of different commercial textiles, with an accuracy of 100.0% at a fixed sliding rate and 98.9% at a random sliding rate.

[0077] Moreover, it is also possible to control the camera to perform an image capture operation when the fingerprint simulation sensor comes into contact with the clothes in the inner tub, that is, the image captured by the camera can include specific clothing information in contact with the fingerprint simulation sensor. Exemplarily, at time t1, if the fingerprint simulation sensor is in contact with clothing 1, control the camera to perform an image capture operation to capture image 1, and determine the quantity information and color information of the clothes in the inner tub by identifying image 1, and identify the material information of clothing 1 through the fingerprint simulation sensor; similarly, at time t2, if the fingerprint simulation sensor is in contact with clothing 2, control the camera to perform an image capture operation to capture image 2, and determine the quantity information and color information of the clothes in the inner tub by identifying image 2, and identify the material information of clothing 2 through the fingerprint simulation sensor. And so on, the camera can capture multiple images, and summarize the quantity information, color information, and material information of the clothes in the inner tub recognized from each image, so as to determine accurate quantity information, color information, and material information, and then more accurately match the corresponding clothing treatment program.

[0078] Furthermore, the decision-making module can be configured to perform the following steps to identify the material information of the clothes in the inner tub in contact with the fingerprint simulation sensor:

[0079] S610, perform wavelet transform on the static pressure and high-frequency vibration of the clothes in the inner tub in contact with the fingerprint simulation sensor to obtain the frequency-domain characteristics of the clothes in the inner tub;

[0080] S620, based on the frequency-domain characteristics, identify the material information of the clothes in the inner tub in contact with the fingerprint simulation sensor.

[0081] Specifically, perform wavelet transform on the static pressure signal and high-frequency vibration signal of the clothes in contact with the fingerprint simulation sensor, so as to facilitate analysis in the time domain and frequency domain, and thus calculate the frequency-domain characteristics of the clothes in the inner tub. The frequency-domain characteristics of the clothes in the inner tub include but are not limited to clothing height characteristics, clothing spacing characteristics, and clothing stiffness characteristics.

[0082] Furthermore, the decision-making module can be configured to perform the following steps to determine the material information of the clothes in the inner tub in contact with the fingerprint simulation sensor:

[0083] S710, classify the frequency-domain characteristics of the clothes in the inner tub in contact with the fingerprint simulation sensor to obtain at least one characteristic type of clothing height, clothing spacing, and clothing stiffness to which the frequency-domain characteristics belong;

[0084] S720, project the frequency-domain characteristics into the two-dimensional spaces corresponding to the different characteristic types to which they belong respectively, so as to obtain the characteristic representations of the frequency-domain characteristics in each two-dimensional space respectively;

[0085] S730 determines the material information of the clothing in the inner cylinder that contacts the fingerprint analog sensor according to the feature representations in each two-dimensional space in the frequency domain.

[0086] Specifically, considering that the complex factors (height, spacing, stiffness, etc.) of the clothing in the inner cylinder will affect the clothing contact signal, machine learning is used to classify these complex features. For example, the Bagging ensemble learning method can be used to solve the classification problem. The generalization ability of the model and the overall classification performance are improved through classification. The classification models (or classifiers) used in the ensemble include K-Nearest Neighbor, Random Forest, Logistic Regression Algorithm, and Decision Tree.

[0087] After classifying the frequency domain features of the clothing in the inner cylinder that contacts the fingerprint analog sensor, the T-distributed Stochastic Neighbor Embedding (t-SNE) is used to project the frequency domain features into the two-dimensional (2D) spaces corresponding to the different feature types respectively to visualize the data while retaining its global and local structures. For example, the height feature of the clothing in the inner cylinder is projected into the two-dimensional space corresponding to the height feature type; the spacing feature of the clothing in the inner cylinder is projected into the two-dimensional space corresponding to the spacing feature type; the stiffness feature of the clothing in the inner cylinder is projected into the two-dimensional space corresponding to the stiffness feature type.

[0088] Please refer to Figure 6 , Figure 6 which is a schematic diagram of the feature representation of the clothing height feature in the two-dimensional space provided by an exemplary embodiment of the present application. As Figure 6 shown, the height features projected by clothing of different materials are often different. The surface height of some clothing may be 10 μm, the surface height of some clothing may be 30 μm, and the surface height of some clothing may be 50 μm. Please continue to refer to Figure 7 , Figure 7 which is a schematic diagram of the feature representation of the clothing spacing feature in the two-dimensional space provided by an exemplary embodiment of the present application. Figure 7 The clothing spacing feature represented in it is about 275 μm.

[0089] After obtaining the feature representations of the frequency domain features in each two-dimensional space, they are compared with the feature representations of clothing of different preset materials, so as to determine the feature representation with the smallest difference, and then the material information of the clothing can be accurately determined.

[0090] It can be understood that the analysis and processing of the clothing contact signal detected by the fingerprint analog sensor to obtain the clothing material information can be integrated into the control board of the clothing processing device or processed in the cloud server.

[0091] As can be seen from S710 - S730 above, by classifying the frequency - domain features of the clothes in the inner drum that are in contact with the fingerprint - simulation sensor and projecting the frequency - domain features into the two - dimensional spaces corresponding to their respective different feature types, the feature representations of the frequency - domain features in each two - dimensional space can be visualized, while retaining the global and local structures of the frequency - domain features, so that the material information of the clothes in the inner drum that are in contact with the fingerprint - simulation sensor can be accurately determined according to the feature representations of the frequency - domain features in each two - dimensional space.

[0092] In S240, for example, if the quantity information of the clothes in the inner drum is 3 pieces, the color information is a mixed color, and the material information is cotton - linen blend, then the corresponding first clothes - treatment program can be matched. The parameters in the first clothes - treatment program can include a medium washing water level, a relatively small amount of detergent, a water temperature of 30°C - 40°C, a washing duration set between 30 minutes and 45 minutes, and the rotation speed of the inner drum set between 600 revolutions per minute and 800 revolutions per minute; if the quantity information of the clothes in the inner drum is 2 pieces, the color information is white, and the material information is wool, then the corresponding second clothes - treatment program can be matched. The parameters in the second clothes - treatment program can include a low washing water level, a relatively small amount of detergent, cold water or a maximum water temperature of 30°C, a washing duration set between 15 minutes and 30 minutes, and the rotation speed of the inner drum set between 400 revolutions per minute and 600 revolutions per minute.

[0093] After matching the corresponding clothes - treatment program according to the quantity information, color information, and material information of the clothes in the inner drum, control the clothes - treatment device to work according to the matched clothes - treatment program.

[0094] In summary, as can be seen from S210 - S240, by controlling the camera on the door body to perform the image - taking operation to collect the pictures containing the clothes information in the inner drum, the quantity information and color information of the clothes in the inner drum can be determined based on the pictures; and by controlling the motor to drive the inner drum to rotate, the clothes in the inner drum can be made to contact the fingerprint - simulation sensor provided on the inner - drum wall to obtain the clothes - contact signal detected by the fingerprint - simulation sensor, so that the material information of the clothes in the inner barrel can be accurately identified through the clothes - contact signal. Compared with only matching the corresponding clothes - treatment program based on the weight of the clothes, this application matches the clothes - treatment program based on the quantity information, color information, and material information of the clothes, making the matched clothes - treatment program more accurate.

[0095] Please refer to Figure 8 , Figure 8 which is the overall flowchart of matching the clothes - treatment program provided by an exemplary embodiment of this application. As Figure 8As shown, by collecting the images of the clothes in the inner cylinder installed on the door body, and then using the image segmentation algorithm to identify the specific colors and quantities of the clothes in the clothes image; and, by obtaining the static pressure and high-frequency vibration of the clothes in the inner cylinder through the fingerprint simulation sensor installed on the inner cylinder wall, and then performing wavelet transform on the static pressure and high-frequency vibration to obtain the frequency domain characteristics of the clothes in the inner cylinder, and classifying the frequency domain characteristics to identify the materials of the clothes in the inner cylinder; finally, combining the quantity information, color information, and material information of the clothes in the inner cylinder, accurately matching the clothes treatment program, so as to control the clothes treatment device to work according to the matched clothes treatment program.

[0096] In an exemplary embodiment, the clothes treatment device is a washing machine, and the controller is further configured to perform the following steps:

[0097] S810, based on the quantity information of the clothes in the inner cylinder, match the corresponding washing water level and detergent dosage;

[0098] S820, based on the color information of the clothes in the inner cylinder, match the corresponding water temperature;

[0099] S830, based on the material information of the clothes in the inner cylinder, match the corresponding washing duration and the rotation speed of the inner cylinder;

[0100] S840, according to the matched washing water level, detergent dosage, water temperature, washing duration, and the rotation speed of the inner cylinder, generate a clothes treatment program suitable for the clothes in the inner cylinder.

[0101] The following describes these 4 steps in detail.

[0102] In S810, the quantity information of the clothes in the inner cylinder to a certain extent reflects the height of the required washing water level and the amount of detergent used. It can be understood that if the quantity of the clothes in the inner cylinder is small, the corresponding washing water level matched is also low, and the corresponding detergent dosage matched is also small; if the quantity of the clothes in the inner cylinder is large, the corresponding washing water level matched is also high, and the corresponding detergent dosage matched is also large.

[0103] In another exemplary embodiment, the weight information of the clothes in the inner cylinder can be determined based on a weight sensor; and based on the quantity information and weight information of the clothes in the inner cylinder, the corresponding washing water level and detergent dosage are matched. Thereby, the weight information of the clothes is fully considered, avoiding the situation where the quantity of the clothes in the inner cylinder is small but the clothes are all large pieces, yet a low washing water level and a small amount of detergent are used; and also avoiding the situation where the quantity of the clothes in the inner cylinder is large but the clothes are all small pieces, yet a high washing water level and a large amount of detergent are used; thus making the matching of the corresponding washing water level and detergent dosage more accurate, effectively avoiding wasting water resources and detergent, and preventing affecting the washing effect of the clothes.

[0104] In S820, exemplarily, for dark-colored clothes such as black, dark blue, or dark green, cold water or cool water can be used for washing. Low-temperature water can help reduce the risk of color fading and bleeding of the clothes; for light-colored clothes such as light yellow, light blue, or pastel colors, cold water or warm water can be used for washing, and overheated water temperature should be avoided to prevent the clothes from becoming lighter in color or bleeding; for white clothes, hot water is usually used. High temperature can better remove stains and sterilize, but it is necessary to ensure that white clothes are not washed together with colored clothes to prevent staining problems.

[0105] In S830, exemplarily, if the clothes in the inner drum contain wool material, and since wool material is an extremely fragile fiber and is not suitable for a long washing time, the corresponding washing duration can be matched between 15 minutes and 30 minutes. And to avoid damage to the wool caused by too high a rotation speed of the inner drum, the corresponding rotation speed of the inner drum can be matched between 400 revolutions per minute and 600 revolutions per minute; if the clothes in the inner drum are made of cotton and linen blended material, since the cotton and linen blended clothes are relatively tough, the corresponding washing duration can be matched between 30 minutes and 45 minutes, and a moderate rotation speed of the inner drum is beneficial to thoroughly clean the clothes, so the corresponding rotation speed of the inner drum can be matched between 600 revolutions per minute and 800 revolutions per minute.

[0106] In S840, exemplarily, if the clothes in the inner drum are made of wool material, a wool wash program can be generated. The specific parameters of the wool wash program can include the washing water level set to a low water level, cold water or a water temperature of at most 30 °C, the washing duration set between 15 minutes and 30 minutes, and the rotation speed of the inner drum set between 400 revolutions per minute and 600 revolutions per minute; if the clothes in the inner drum are made of cotton and linen blended material, a cotton and linen wash program can be generated. The specific parameters of the cotton and linen wash program can include the washing water level set to a medium water level, a water temperature of 30 °C - 40 °C, the washing duration set between 30 minutes and 45 minutes, and the rotation speed of the inner drum set between 600 revolutions per minute and 800 revolutions per minute.

[0107] In an exemplary embodiment, the laundry treatment device further includes a display screen for displaying at least one of the material information of the clothes in the inner drum, washing, drying, and storage information.

[0108] Further, a display screen is provided on the laundry treatment device. The display screen can be a liquid crystal display screen, a touch display screen, a digital display screen, etc. Exemplarily, the material information of the clothes in the inner drum can be "cotton T-shirt", "silk pajamas", "wool sweater", etc. Through this clothing material identification function, it can help users distinguish the authenticity of the clothing material; the washing and care information of the clothes in the inner drum can be "gentle washing, water temperature not exceeding 30°C", "only dry cleaning allowed", etc.; the drying information of the clothes in the inner drum can be "avoid direct sunlight exposure", "hang flat with a hanger"; the storage information of the clothes in the inner drum can be "store in a ventilated and dry place", "avoid moisture", "store in the wardrobe to avoid direct sunlight", etc. By displaying at least one of the material information, washing and care, drying, and storage information of the clothes in the inner drum on the display screen, it can be provided for users to learn about this information, and thus better protect the clothes.

[0109] It should be understood that the display content in the above examples is only for illustration to facilitate understanding of the application scenario of the present application, but should not be construed as a specific limitation on the display content of the display screen.

[0110] In another exemplary embodiment, the laundry treatment device is bound to a mobile terminal, and the mobile terminal is used to display at least one of the material information, washing and care, drying, and storage information of the clothes in the inner drum.

[0111] Further, the mobile terminal can be a smart phone, a tablet computer, a smart watch, etc. The mobile terminal can be bound to the laundry treatment device through Bluetooth, Wi-Fi or other wireless communication technologies. After the laundry treatment device is bound to the mobile terminal, the mobile terminal can display at least one of the material information, washing and care, drying, and storage information of the clothes in the inner drum, which can be provided for users to learn about this information, and thus better protect the clothes.

[0112] Similarly, the material information of the clothes displayed on the mobile terminal can be "gentle washing, water temperature not exceeding 30°C", "only dry cleaning allowed", etc.; the drying information of the clothes displayed on the mobile terminal can be "avoid direct sunlight exposure", "hang flat with a hanger"; the storage information of the clothes displayed on the mobile terminal can be "store in a ventilated and dry place", "avoid moisture", "store in the wardrobe to avoid direct sunlight", etc.

[0113] Those skilled in the art will readily think of other implementation manners of the present application after considering the specification and practicing the disclosed embodiments herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include the common general knowledge or conventional technical means in the technical field not disclosed in the present application.

[0114] It should be understood that the above content is only a preferred exemplary embodiment of the present application and is not used to limit the implementation of the present application. Those of ordinary skill in the art can easily make corresponding adaptations or modifications according to the main concept and spirit of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope required by the claims.

Claims

1. A laundry treatment device, characterized in that, Including: A motor; A door body and an inner drum; A camera disposed on the door body for collecting pictures containing information about the clothes in the inner drum; A fingerprint simulation sensor disposed on the inner drum wall; A controller electrically connected to the camera, the motor and the fingerprint simulation sensor respectively, and the controller is configured to perform the following steps: Control the camera to perform an image acquisition operation to obtain the pictures collected by the camera, and determine the quantity information and color information of the clothes in the inner drum based on the pictures; Control the motor to drive the inner drum to rotate so that the clothes in the inner drum come into contact with the fingerprint simulation sensor; After the fingerprint simulation sensor comes into contact with the clothes in the inner drum, obtain the clothes contact signal detected by the fingerprint simulation sensor to identify the material information of the clothes in the inner drum through the clothes contact signal; Based on the quantity information, color information and material information of the clothes in the inner drum, match a corresponding clothes treatment program to control the clothes treatment device to work according to the clothes treatment program.

2. The laundry treatment device according to claim 1, wherein The controller is further configured to perform the following steps: Determine the quantity information of the clothes in the inner drum by distinguishing different clothes areas in the pictures; Determine the color information of each piece of clothes in the inner drum by performing color recognition on different clothes areas in the pictures.

3. The laundry treating apparatus according to claim 2, wherein The controller is further configured to perform the following steps: Convert the color mode of the pictures into the LAB mode, and calculate the color data of different clothes in the pictures within the color space corresponding to the LAB mode; Compare the calculated color data of different clothes with the color data in a preset standard color table to determine the color information of different clothes in the inner drum respectively.

4. The laundry treatment device according to claim 1, characterized in that, The controller is further configured to perform the following steps: Obtain the clothes contact signal generated by the fingerprint simulation sensor when the clothes in the inner drum contact the fingerprint simulation sensor; Determine the static pressure and high-frequency vibration of the clothes in the inner drum that come into contact with the fingerprint simulation sensor based on the clothes contact signal; Identify the material information of the clothes in the inner drum that come into contact with the fingerprint simulation sensor based on the static pressure and the high-frequency vibration.

5. The laundry treatment device according to claim 4, characterized in that, The controller is further configured to perform the following steps: Perform wavelet transform on the static pressure and high-frequency vibration of the clothes in the inner drum that come into contact with the fingerprint simulation sensor to obtain the frequency domain characteristics of the clothes in the inner drum; Identify the material information of the clothes in the inner drum that come into contact with the fingerprint simulation sensor based on the frequency domain characteristics.

6. The laundry treating apparatus according to claim 5, wherein The controller is further configured to perform the following steps: Classify the frequency domain characteristics of the clothes in the inner drum that come into contact with the fingerprint simulation sensor to obtain at least one characteristic type among the clothes height, clothes spacing and clothes stiffness to which the frequency domain characteristics belong; Project the frequency domain characteristics into the two-dimensional spaces corresponding to the different characteristic types to which they belong respectively to obtain the characteristic representations of the frequency domain characteristics in each two-dimensional space; Based on the feature representations of the fingerprint analog sensor in each two-dimensional space, determine the material information of the clothing in contact with the fingerprint analog sensor in the inner cylinder.

7. The laundry treating apparatus according to claim 1, wherein The clothing treatment device further includes a lighting device, which is configured to emit a light source when the camera performs an image capturing operation to enhance the ambient light in the inner cylinder.

8. The laundry treating apparatus according to claim 1, wherein The clothing treatment device further includes a display screen, which is configured to display at least one of the material information of the clothing in the inner cylinder, washing, drying, and storage information.

9. The laundry treating apparatus according to claim 1 or 8, wherein The clothing treatment device is bound to a mobile terminal, and the mobile terminal is configured to display at least one of the material information of the clothing in the inner cylinder, washing, drying, and storage information.

10. The laundry treating apparatus according to claim 1, wherein The clothing treatment device is a washing machine, and the controller is further configured to perform the following steps: Based on the quantity information of the clothing in the inner cylinder, match the corresponding washing water level and detergent dosage; Based on the color information of the clothing in the inner cylinder, match the corresponding water temperature; Based on the material information of the clothing in the inner cylinder, match the corresponding washing duration and the rotation speed of the inner cylinder; Generate a clothing treatment program suitable for the clothing in the inner cylinder according to the matched washing water level, detergent dosage, water temperature, washing duration, and rotation speed of the inner cylinder.

Images