Control method for smart home system

By using the ResNet18 model in the smart home system to determine whether the user has clothes in his hands, and using the deep learning model to identify clothes attributes on the server side, the problem of direct misjudgment of clothing processing equipment is solved, and the accurate recommendation of washing procedures is achieved, which improves the User experience.

CN114657744BActive Publication Date: 2024-11-19QINGDAO HAIER WASHING MASCH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011550504.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-24
Publication Date
2024-11-19
Estimated Expiration
2040-12-24

AI Technical Summary

Technical Problem

The clothes processing equipment of the existing smart home system directly determines the attribute information of the clothes based on the captured images, resulting in the recommended washing procedures inaccurate. Especially when there is no clothes in the user's hands, it is easy to misjudgment the clothes worn on the user. For the attribute information of the laundry to be washed.

Method used

By analyzing the user's image using the ResNet18 model in the clothing processing device, determining whether the user has clothes in his hand, and selectively sending the image to the server based on the judgment results. The server uses a deep learning model to identify the attribute information of the clothes to determine the washing procedure.

Benefits of technology

It avoids the accidental determination of the washing procedures when the user has no clothes in his hands, ensures accurate identification and recommendation of the attribute information of the laundry to be washed, and improves the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114657744B_ABST
    Figure CN114657744B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of smart home technology, and specifically provides a control method for a smart home system, aiming to solve the problem that the clothing processing device of the existing smart home system directly determines the attribute information of clothing based on the captured image, resulting in inaccurate recommended washing and care programs. To this end, the control method of the present invention includes the following steps: the clothing processing device obtains the image of the user; determines whether the user has clothing in his hand based on the image; based on the judgment result of whether the user has clothing in his hand, the server selectively determines the washing and care program based on the image, thereby avoiding determining the washing and care program based on the image when the user has no clothing in his hand, thereby avoiding mistakenly determining the attribute information of the clothing worn by the user as the attribute information of the clothing to be washed, being able to accurately determine the attribute information of the clothing to be washed, being able to accurately recommend washing and care programs, providing reasonable washing and care for the clothing, and thereby improving the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of smart home, and specifically provides a control method for a smart home system. Background Art

[0002] In daily life at home, taking a washing machine as an example, the washing machine is equipped with a camera, which captures images of clothes and determines the attribute information of the clothes, such as material and category, based on the captured images. It also recommends a suitable washing program based on the attribute information of the clothes, which can not only clean the clothes but also prevent them from being damaged.

[0003] However, in actual use, as long as a user passes by the washing machine, the camera will perform a shooting operation and directly determine the attribute information of the clothes based on the captured image to recommend a washing program. However, some users only pass by the washing machine without holding any clothes to be washed. In this case, the attribute information of the clothes determined based on the captured image is the attribute information of the clothes worn by the user, rather than the attribute information of the clothes to be washed, which results in inaccurate recommended washing programs, making it difficult to provide reasonable washing and care for the clothes to be washed, and reducing the user experience.

[0004] Therefore, the art needs a new control method for a smart home system to solve the above problems. Summary of the invention

[0005] In order to solve the above-mentioned problems in the prior art, that is, to solve the problem that the clothing processing device of the existing smart home system directly determines the attribute information of the clothing according to the captured image, resulting in inaccurate recommended washing and care programs, the present invention provides a control method for a smart home system, the smart home system comprises a clothing processing device and a server, the clothing processing device can communicate with the server; the control method comprises the following steps: the clothing processing device obtains an image of a user; determines whether the user has clothing in his hands according to the image; and according to the judgment result of whether the user has clothing in his hands, the server selectively determines the washing and care program according to the image.

[0006] In the preferred technical solution of the above control method, the clothing processing device has a ResNet18 model; the step of "determining whether the user has clothes in his hands based on the image" specifically includes: calling a preset model to analyze the image; and determining whether the user has clothes in his hands based on the analysis results.

[0007] In the preferred technical scheme of the above control method, the step of "calling a preset model to analyze the image" specifically includes: intercepting multiple sub-images from the image according to a preset method; inputting all the sub-images into the preset model; the preset model calculates and obtains the feature value of the image based on all the sub-images; the step of "determining whether the user has clothes in his hands based on the analysis results" specifically includes: determining whether the feature value is greater than a preset value; if the feature value is greater than the preset value, it is determined that the user has clothes in his hands.

[0008] In the preferred technical solution of the above control method, the step of "determining whether the user has clothes in his hands according to the analysis results" also includes: if the characteristic value is less than or equal to the preset value, it is determined that the user has no clothes in his hands.

[0009] In the preferred technical solution of the above control method, the preset model is pre-stored in the clothing processing device; the step of "based on the judgment result of whether the user has clothes in his hands, the server selectively determines the washing and care program according to the image" specifically includes: if the user has clothes in his hands, the clothing processing device sends the image to the server, so that the server determines the washing and care program according to the image.

[0010] In the preferred technical solution of the above control method, the server has a deep learning model; the step of "the clothing processing device sends the image to the server, so that the server determines the washing and care program according to the image" specifically includes: the clothing processing device sends the image to the server; the server uses the deep learning model to recognize the image and identify the attribute information of the clothes in the user's hands; the server determines the washing and care program according to the attribute information.

[0011] In the preferred technical solution of the above control method, after "the server determines the washing and care program according to the clothing attribute information", the control method also includes: the server sends the washing and care program to the clothing processing device; when the clothing processing device receives the washing and care program, the clothing processing device runs the washing and care program.

[0012] In the preferred technical solution of the above control method, the step of "based on the judgment result of whether the user has clothes in his hands, the server selectively determines the washing and care program based on the image" also includes: if the user has no clothes in his hands, the clothing processing device does not send the image to the server.

[0013] In the preferred technical solution of the above control method, before "the clothing processing device obtains the image of the user", the control method also includes: the clothing processing device detects whether there is a user within a preset range; if a user enters the preset range, the clothing processing device executes the operation of obtaining the image.

[0014] In a preferred technical solution of the above control method, the control method further comprises: if no user enters the preset range, the clothing processing device does not perform the operation of acquiring the image.

[0015] Those skilled in the art can understand that, in the preferred technical solution of the control method of the present invention, the smart home system includes a washing machine and a server; the washing machine obtains an image of the user; determines whether the user has clothes in his hands based on the image; and based on the judgment result of whether the user has clothes in his hands, the server selectively determines a washing and care program based on the image.

[0016] Compared with the technical solution in the prior art that the washing machine directly determines the attribute information of clothes based on the captured image, after the washing machine obtains the image of the user, the server of the present invention selectively determines the washing and care program based on the image according to the judgment result of whether the user has clothes in his hands, thereby avoiding determining the washing and care program based on the image when the user has no clothes in his hands, thereby avoiding mistakenly determining the attribute information of the clothes worn by the user as the attribute information of the clothes to be washed, and can accurately determine the attribute information of the clothes to be washed, can accurately recommend washing and care programs, provide reasonable washing and care for the clothes to be washed, and thus improve the user experience.

[0017] Furthermore, if the user has clothes in his hands, it means that the user wants to wash the clothes in his hands instead of passing them through the washing machine. The user's image can reflect the attribute information of the clothes to be washed. The server can accurately determine the attribute information and washing and care program of the clothes to be washed based on the image, provide reasonable washing and care for the clothes, and thus improve the user experience.

[0018] Furthermore, if the user has no clothes in his hands, it means that the user is passing through the washing machine instead of washing the clothes. The image of the user reflects the attribute information of the clothes he is wearing, instead of the attribute information of the clothes to be washed. Therefore, the attribute information of the clothes to be washed cannot be determined based on the user's image, and the washing program for the clothes to be washed cannot be recommended based on the user's image. In order to ensure the accuracy of the recommended washing program, the server does not determine the attributes of the clothes and recommend the washing program based on the user's image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The smart home system and control method of the present invention will be described below with reference to the accompanying drawings and in combination with a washing machine, wherein:

[0020] Figure 1 is a structural diagram of the smart home system of the present invention;

[0021] Figure 2 It is the main flow chart of the control method of the present invention;

[0022] Figure 3 is a flow chart of a control method for determining whether a user has clothes in his hand according to the present invention;

[0023] Figure 4 It is a flow chart of a control method for calling a ResNet18 model to analyze an image of the present invention;

[0024] Figure 5 is a flow chart of a control method for judging whether a user has clothes in his hand according to the analysis result of the present invention;

[0025] Figure 6 is a flow chart of a control method for a server of the present invention to selectively determine a washing and care program according to an image;

[0026] Figure 7 is a flow chart of a control method of the washing machine of the present invention sending an image to a server so that the server determines a washing and care program according to the image;

[0027] Figure 8 is a complete flow chart of the control method of the present invention;

[0028] Fig. 9 It is a logic diagram of the control method of the present invention.

[0029] Reference numerals list

[0030] 1. Washing machine; 2. Server. DETAILED DESCRIPTION

[0031] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. For example, although the present application is described in conjunction with a washing machine, the technical solution of the present invention is not limited thereto, and the smart home system and control method can obviously also be applied to other clothing processing equipment such as dryers, integrated washing and drying machines, shoe washing machines, and care machines, and such changes do not deviate from the principles and scope of the present invention.

[0032] It should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0033] Based on the technical problems raised in the background technology, the present invention provides a control method for a smart home system, which aims to, after the washing machine obtains the image of the user, according to the judgment result of whether the user has clothes in his hands, the server selectively determines the washing and care program based on the image, thereby avoiding determining the washing and care program based on the image when the user has no clothes in his hands, thereby avoiding mistakenly determining the attribute information of the clothes worn by the user as the attribute information of the clothes to be washed, and can accurately determine the attribute information of the clothes to be washed, can accurately recommend washing and care programs, provide reasonable washing and care for the clothes, and thus improve the user experience.

[0034] See first Figure 1 , the smart home system of the present invention is described. Among them, Figure 1 It is a structural diagram of the smart home system of the present invention.

[0035] like Figure 1 As shown, the smart home system of the present invention includes a washing machine 1 and a server 2, wherein the washing machine 1 can communicate with the server 2, wherein the washing machine 1 includes a detection module, an image acquisition module, a preset model and a control module, wherein the detection module, the image acquisition module and the preset model are all connected to the control module, the detection module is used to detect whether there is a user within a preset range of the washing machine 1, the image acquisition module is used to capture the image of the user, the preset model is used to analyze the image of the user, the control module is used to control the image acquisition module to capture the image of the user according to the detection result of the detection module, and control the preset model to analyze the image of the user captured by the image acquisition module, and judge whether there is clothing in the user's hand according to the analysis result, and selectively send the user's image to the server 2 according to the judgment result. A deep learning model is provided on the server 2, and the deep learning model is used to recognize the user's image to identify the attribute information of the clothing in the user's hand.

[0036] Among them, the detection module is an infrared sensor, which can accurately detect whether a user enters the preset range of the washing machine 1, thereby improving the accuracy of the detection module. Of course, the detection module can also be a radar sensor or a laser sensor. Regardless of which sensor is used to detect whether a user enters the preset range of the washing machine 1, the specific detection method corresponding to any sensor should not limit the present invention.

[0037] The image acquisition module may be, but is not limited to, a camera, etc.

[0038] In terms of physical form, the control module can be any type of controller, such as a programmable controller, a combinational logic controller, etc.

[0039] The server 2 may be, but is not limited to, a cloud server 2 and a backend server 2 .

[0040] Preferably, server 2 can receive an image of the user from washing machine 1, and determine a washing and care program based on the image of the user. For example, server 2 uses a deep learning model to recognize the image, identifies the attribute information of the clothes in the user's hands, and server 2 determines the washing and care program based on the attribute information. Server 2 can also send the washing and care program to washing machine 1, so that washing machine 1 runs the washing and care program after receiving the washing and care program.

[0041] Among them, the deep learning model can be but is not limited to a CNN model, a ResNet101 model, a DeeplabV3+ model, a ResNeXt model, and a HRNet model.

[0042] Preferably, the washing machine 1 can communicate with the server 2 via a wide area network, a local area network, WiFi, "WiFi+a router connected to the Internet", Bluetooth, ZIGBEE, NFC, GPRS or other communication methods.

[0043] Preferably, the preset model is a ResNet18 model. Of course, the preset model may also be a ResNet50 model, a ResNet101 model or other models. No matter which model is adopted, as long as the user's image can be analyzed to determine whether the user has clothes in his hands.

[0044] It should be noted that, although the above example is that the preset model is pre-stored in the washing machine 1, and the washing machine 1 performs the work of "judging whether the user has clothes in his hands according to the image", specifically "calling the preset model to analyze the image; judging whether the user has clothes in his hands according to the analysis result", this is not restrictive, but only exemplary, and the preset model can also be pre-stored in the server 2, and the server 2 performs the work of "judging whether the user has clothes in his hands according to the image". Those skilled in the art can flexibly adjust and set the execution subject of the above work, and the present invention does not impose any limitation on this.

[0045] The following takes the preset model as the ResNet18 model and the washing machine 1 as the execution subject of “determining whether the user has clothes in his hand according to the image” as an example to further illustrate the control method for the smart home system of the present invention.

[0046] See also Figure 2 , the control method for the smart home system of the present invention is described. Among them, Figure 2 It is the main flow chart of the control method of the present invention.

[0047] like Figure 2 As shown, the control method for the smart home system of the present invention comprises the following steps:

[0048] S100, the washing machine obtains an image of the user;

[0049] S200, the washing machine determines whether the user has clothes in his hand according to the image;

[0050] S300. Based on the result of determining whether the user has clothes in his hands, the server selectively determines a washing and care program based on the image.

[0051] In step S100, the washing machine captures an image of the user through an image acquisition module such as a camera or a still camera.

[0052] Refer to the following Figures 3 to 5 , the control method of judging whether a user has clothes in his hand is described. Figure 3 is a flow chart of a control method for determining whether a user has clothes in his hand according to the present invention; Figure 4 It is a flow chart of a control method for calling a ResNet18 model to analyze an image of the present invention; Figure 5 It is a flow chart of a control method for judging whether a user has clothes in his hands according to analysis results of the present invention.

[0053] like Figure 3 As shown, the ResNet18 model is pre-stored on the washing machine; in step S200, the step of "the washing machine determines whether the user has clothes in his hand according to the image" specifically includes:

[0054] S210, the washing machine calls the ResNet18 model to analyze the image;

[0055] S220: The washing machine determines whether the user has any clothes in his hands based on the analysis result.

[0056] Among them, a possible network structure corresponding to the ResNet18 model is shown in Table 1.

[0057] Table 1. Network structure of ResNet18 model

[0058]

[0059]

[0060] Among them, the layers of the ResNet18 model use nonlinear connections, so the ResNet18 model as a whole is also nonlinearly connected. It should be noted that the calculation method and operation principle of the ResNet18 model are common knowledge in the field and will not be repeated here. Of course, other models such as the ResNet50 model and the ResNet101 model can also be used to analyze the user's image to determine whether the user has clothes in his hands.

[0061] like Figure 4 As shown, in step S210, the step of "the washing machine calls the ResNet18 model to analyze the image" specifically includes:

[0062] S211, the washing machine captures a plurality of sub-images from the image according to a preset method;

[0063] S212, the washing machine inputs all sub-images into the ResNet18 model;

[0064] S213, ResNet18 model calculates and obtains the feature values ​​of the image based on all sub-images.

[0065] The preset method may be to set different sliding frames, extract the image in each sliding frame, and use the extracted image as a sub-image; or the preset method may be to divide the user's image into N parts according to the size of the user's image, for example, 5, 10, 15 or 20 parts, where N is a positive integer, and extract each image separately, and use the extracted image as a sub-image. Of course, the preset method is not limited to the above-mentioned methods, and no matter which method is adopted, as long as multiple sub-images can be extracted from the image.

[0066] like Figure 5 As shown, in step S220, the step of "the washing machine determines whether the user has clothes in his hand according to the analysis result" specifically includes:

[0067] S221, the washing machine determines whether the characteristic value is greater than a preset value;

[0068] S222: If the characteristic value is greater than a preset value, it is determined that the user has clothes in his hand;

[0069] S223: If the characteristic value is less than or equal to the preset value, it is determined that the user has no clothes in his hands.

[0070] In step S222, if the characteristic value is greater than the preset value, for example, the preset value is 0.5, and the characteristic value calculated in step S213 is 0.95, which is greater than the preset value, it means that the user has clothes in his hands, and it is determined that the user has clothes in his hands.

[0071] In step S223, if the characteristic value is less than or equal to the preset value, for example, the preset value is 0.5, and the characteristic value calculated in step S213 is 0.05, which is less than the preset value, indicating that the user has no clothes in his hands, it is determined that the user has no clothes in his hands.

[0072] It should be noted that the preset values ​​listed above are only exemplary and not restrictive. Those skilled in the art can flexibly adjust and set the preset values ​​according to the accuracy of judging whether the user has clothes in his hands. For example, the preset values ​​can also be 0.7, 0.8, 0.9 or 1. No matter how the preset values ​​are set, as long as it can accurately judge whether the user has clothes in his hands, it will be fine.

[0073] It should also be noted that in the above process, step S222 and step S223 have no order of precedence and are parallel, and are only related to the judgment result of whether the characteristic value is greater than the preset value. The corresponding steps can be executed according to different judgment results.

[0074] Refer to the following Figure 6 and Figure 7 , describes the control method of the present invention for selectively sending an image to a server so that the server determines a washing and care program according to the image. Figure 6 is a flow chart of a control method for a server of the present invention to selectively determine a washing and care program according to an image; Figure 7 It is a flow chart of a control method in which a washing machine of the present invention sends an image to a server so that the server determines a washing and care program according to the image.

[0075] like Figure 6 As shown, in step S300, the step of "according to the result of judging whether the user has clothes in his hand, the server selectively determines the washing and care program according to the image" specifically includes:

[0076] S311, if the user has no clothes in his hand, the washing machine does not send the image to the server;

[0077] S312: If the user has clothes in his hands, the washing machine sends the image to the server so that the server determines the washing program according to the image.

[0078] In step S311, if the user has no clothes in his hands, it means that the user is passing by the washing machine instead of washing clothes. The image of the user reflects the attribute information of the clothes he is wearing, not the attribute information of the clothes to be washed. Therefore, the attribute information of the clothes to be washed cannot be determined based on the image of the user, and the washing program for the clothes to be washed cannot be recommended based on the image of the user. In order to ensure the accuracy of the recommended washing program, the washing machine does not send the image to the server, so as to not determine the attributes of the clothes and recommend the washing program based on the image of the user.

[0079] like Figure 7 As shown, the server has a deep learning model; in step S312, the step of "the washing machine sends the image to the server so that the server determines the washing program according to the image" specifically includes:

[0080] S321, the washing machine sends the image to the server;

[0081] S322: The server uses a deep learning model to recognize the image and identify attribute information of the clothes in the user's hand;

[0082] S323. The server determines the washing and care program according to the attribute information.

[0083] Among them, the attribute information includes category, material and color. The category can be divided into T-shirts, shirts, windbreakers, coats, suits, shorts, cropped pants, trousers, etc. The category of clothing can also be divided into men's clothing, women's clothing, and neutral clothing. The clothing categories can be classified according to different standards, and the present invention does not impose any restrictions on this. Of course, the attribute information can also only include category, material or color, or include any two of category, material and color. Those skilled in the art can flexibly adjust and set the clothing information included in the attribute information according to actual usage requirements.

[0084] The following further describes step S322 by taking the use of a deep learning model to identify category, material, color and other information as an example.

[0085] First, let’s take the example of identifying the category of clothing to further explain:

[0086] A CNN deep learning model is used to obtain the category of clothing, wherein the CNN deep learning model is trained based on a data set obtained by cutting the image using a sliding window, rotating it, and then cutting it using an initial sliding window. The initial sliding window is used to cut the user's image, and the sub-image obtained by cutting the image is input into the CNN deep learning model for category recognition, and the category information of the clothing is obtained based on the recognition result.

[0087] Secondly, let’s take the identification of clothing materials as an example to further explain:

[0088] A CNN deep learning model is used to obtain the material of the clothing, wherein the CNN deep learning model has a clothing material database, which is a database established based on images; the CNN deep learning model performs image retrieval in the clothing material database based on the user's image to determine the first clothing information A and the target position of the clothing in the user's hand, wherein the first clothing information A includes a first material set A 1 , the first probability A 2 and the number of clothes N, the target position is the position of the clothes in the user's hand. The CNN deep learning model performs image retrieval in the clothing material database according to the local feature map corresponding to the target position, and determines the second clothing information B of the clothes in the user's hand. The second clothing information B includes the second material set B 1 And the corresponding second probability B 2 , the weighted method can be used to normalize and merge the first clothing information A and the second clothing information B to obtain the final clothing material. For example, the weight of the first clothing information A is set to a, the weight of the second clothing information B is set to b, and the clothing material is C, then C = A 1 ×A 2 ×a+B 1 ×B 2 ×b.

[0089] Alternatively, in an alternative implementation, a ResNet101 deep learning model is used to obtain the material of the clothing. The ResNet101 deep learning model has a clothing material database and a classifier, and the classifier is used to classify the material of the clothing. The ResNet101 deep learning model calculates the distance between the material of the clothing in the user's hand and all the materials in the clothing material database through cosine distance. The classifier identifies the material of the clothing according to the calculated distance, thereby determining the material of the clothing. For example, all the calculated distances are compared, and the material corresponding to the smallest distance is determined as the material of the clothing.

[0090] Among them, a possible calculation method corresponding to the cosine distance is as follows:

[0091] The user's image is input into the ResNet101 deep learning model, and the classification data output by the ResNet101 deep learning model in response to the input of the user's image is obtained. The ResNet101 deep learning model uses the loss function as a constraint condition to limit the cosine distance of the intra-class features in the classification data to tend to the Euclidean distance, wherein the loss function includes: a cross entropy loss function, a Centerloss loss function based on cosine distance, and a Centerloss loss function based on Euclidean distance. Limiting the cosine distance of the intra-class features in the classification data to tend to the Euclidean distance includes: screening the classification data by the loss function based on cosine distance in the joint loss function to maximize the cosine distance in the classification data, obtaining the classification data, and understanding the content of the target image to be tested based on the classification data.

[0092] The forward propagation process of the above loss function is described as:

[0093]

[0094] Among them, L is the total loss function; Lcrossentropy is expressed as the cross entropy loss function, (f(x i )-c i ) 2 is the centerloss loss function based on Euclidean distance, Expressed as a centerloss loss function based on cosine distance, c i represents the average feature within the class, λ 1 and λ 2 To adjust the hyperparameters of the loss function weight, f(x i ) represents the image features extracted by the deep learning model.

[0095] Of course, the method for calculating the cosine distance is not limited to the above-listed methods, and other methods may also be used to calculate the cosine distance. No matter which method is used, as long as the cosine distance can be calculated.

[0096] Again, let’s take the example of identifying the color of clothing to further illustrate:

[0097] The DeepLabV3+ deep learning model is used to obtain the color of the clothes. First, the user's image is segmented, and the segmented image is masked, and the masked image is mapped to the RGB color space for color feature extraction, that is, the R, G, and B values ​​corresponding to the color are extracted, and the color features of the clothes mapped to the RGB color space are classified using the K-means clustering algorithm, and finally the color of the clothes in the user's hand is obtained. For example, RGB (0, 0, 0) is black, RGB (127, 127, 127) is gray, RGB (255, 0, 0) is red, RGB (0, 255, 0) is green, RGB (0, 0, 255) is blue, etc., and the present invention does not impose any restrictions on this.

[0098] It should be noted that the deep learning model used to identify categories, materials and colors is not limited to the models listed above. Other deep learning models can also be used to identify categories, materials and colors, such as ResNeXt model and HRNet model. Although the above examples use different deep learning models to identify categories, materials and colors respectively, this is not restrictive, but only exemplary. The same deep learning model can be used to identify categories, materials and colors, or one deep learning model can be used to identify any two of the categories, materials and colors, and another deep learning model can be used to identify the other of the categories, materials and colors. Those skilled in the art can flexibly adjust and set according to actual usage requirements.

[0099] It should also be noted that the method for deep learning models to identify categories, materials, colors and other information is not limited to the methods listed above. Other methods can also be used to identify categories, materials, colors and other information. No matter what method is used to identify categories, materials, colors and other information, as long as it can identify categories, materials, colors and other information.

[0100] In the above method, a deep learning model is used to recognize images, which can accurately identify the attribute information of the clothes in the user's hands, so as to accurately determine the washing and care program, provide reasonable washing and care for the clothes, and thus improve the user experience.

[0101] Continue reading Figure 7 , after step S323, the control method further includes:

[0102] S324, the server sends the washing program to the washing machine;

[0103] S325. After the washing machine receives the washing and care program, the washing machine runs the washing and care program.

[0104] Among them, the washing and care program is the washing and care program determined in step S323. Since the washing and care program is determined based on information such as the category, material, color, etc. of the clothes in the user's hands, it is more matched with the clothes to be washed (i.e., the clothes in the user's hands) and can provide reasonable washing and care for the clothes to be washed, thereby improving the user experience.

[0105] In step S325, after the washing machine receives the washing and care program, if the user is satisfied with the washing and care program received by the washing machine, the washing machine can directly run the received washing and care program; if the user is not satisfied with the washing and care program received by the washing machine, the user can also reset the washing and care program by himself, or make adjustments based on the received washing and care program, so that the washing machine can run the reset or adjusted washing and care program.

[0106] Refer to the following Figure 8 , the control method of the present invention is further described. Among them, Figure 8 It is a complete flow chart of the control method of the present invention.

[0107] like Figure 8 As shown, before step S100, the control method further includes:

[0108] S010, the washing machine detects whether a user enters the preset range; if not, execute step S020; if yes, execute step S100;

[0109] S020: The washing machine does not perform an operation of acquiring an image;

[0110] S100: The washing machine obtains an image of a user.

[0111] In step S010, the detection module may detect in real time whether a user enters the preset range; or the detection module may detect at preset time intervals whether a user enters the preset range.

[0112] Among them, the preset time interval can be 10 minutes, 20 minutes or 30 minutes, etc. The above preset time interval is only exemplary and not restrictive. Those skilled in the art can flexibly adjust and set the preset time interval according to the frequency of users entering the washing machine placement location in actual applications. No matter how the preset time interval is adjusted and set, it is sufficient as long as it can accurately detect whether a user enters the preset range.

[0113] Among them, the preset range can be the detection range of the detection module, such as 1.2m, 1.5m or 2.0m; the preset range can also be a range set by technical personnel in the field based on experiments or experience, such as 1.0m, 1.3m or 1.6m. Technical personnel in the field can flexibly adjust and set the preset range.

[0114] If a user enters the preset range, the user may want to wash clothes and hold clothes in his hands. Of course, the user may also be passing by the washing machine. In order to determine the user's purpose, the user's image is obtained, and based on the user's image, it is determined whether the user has clothes in his hands, thereby determining the user's purpose.

[0115] If no user enters the preset range, it means that no user wants to wash clothes and there is no need to determine the washing program based on the user's image. The washing machine does not perform the image acquisition operation and continues to detect whether there is a user entering the preset range until it detects that a user has entered the preset range.

[0116] It should be noted that in the above process, step S020 and step S100 have no order of precedence and are parallel, and are only related to the judgment result of whether a user enters the preset range. The corresponding steps can be executed according to different judgment results.

[0117] Refer to the following Fig. 9 , a possible control process of the present invention is introduced. Among them, Fig. 9 is a logic diagram of the control method of the present invention. Fig. 9 The control method of the present invention is further described by taking an example in which the detection module is an infrared sensor, the image acquisition module is a camera, and the ResNet18 model is pre-stored in a washing machine.

[0118] like Fig. 9 As shown, a possible complete process of the control method of the present invention is:

[0119] S401, turn on the infrared sensor;

[0120] S402, the infrared sensor detects whether a user enters the preset range; if so, execute step S403; if not, continue to execute step S402 until the infrared sensor detects that a user enters the preset range;

[0121] S403, the camera acquires an image of the user;

[0122] S404, the washing machine calls the ResNet18 model to analyze the image;

[0123] S405, the washing machine determines whether the user has clothes in his hand according to the analysis result; if not, execute step S406; if yes, execute step S407;

[0124] S406: The washing machine does not send the image to the server, and returns to step S402 until the infrared sensor detects that a user has entered the preset range again and that the user has clothes in his hand;

[0125] S407, the washing machine sends the image to the server;

[0126] S408: The server uses a deep learning model to recognize the image and identify attribute information of the clothes in the user's hand;

[0127] S409, the server determines a washing and care program according to the attribute information;

[0128] S410, the server sends the washing program to the washing machine;

[0129] S411. After the washing machine receives the washing and care program, the washing machine runs the washing and care program.

[0130] It should be pointed out that the above embodiment is only a preferred implementation of the present invention, which is only used to illustrate the principle of the method of the present invention, and is not intended to limit the scope of protection of the present invention. In practical applications, those skilled in the art can allocate the above functions to different steps as needed, that is, decompose or combine the steps in the embodiment of the present invention. For example, the steps of the above embodiment can be combined into one step, or further divided into multiple sub-steps to complete all or part of the functions described above. The names of the steps involved in the embodiments of the present invention are only for distinguishing the steps and are not regarded as limitations of the present invention.

[0131] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A control method for a smart home system, It is characterized in that The smart home system includes a clothes processing device and a server, and the clothes processing device can communicate with the server; The control method comprises the following steps: The clothing processing device acquires an image of a user; determining whether the user has clothes in his hands according to the image; According to the result of judging whether the user has clothes in his hands, the server selectively determines a washing and care program according to the image; The step of "determining whether the user has clothes in his hands according to the image" specifically includes: Calling a preset model to analyze the image; According to the analysis results, determine whether the user has clothes in his hand; The preset model is pre-stored in the clothes processing device; The step of "according to the result of determining whether the user has clothes in his hands, the server selectively determines the washing and care program according to the image" specifically includes: If the user has clothes in his hands, the clothes processing device sends the image to the server, so that the server determines a washing and care program according to the image; If the user has no clothes in his hands, the clothes treating apparatus does not send the image to the server.

2. The control method according to claim 1, It is characterized in that The step of "calling a preset model to analyze the image" specifically includes: intercepting a plurality of sub-images from the image according to a preset method; Inputting all the sub-images into the preset model; The preset model calculates and obtains the feature value of the image based on all the sub-images; The steps of "determining whether the user has clothes in his hand according to the analysis result" specifically include: Determine whether the characteristic value is greater than a preset value; If the characteristic value is greater than the preset value, it is determined that the user has clothes in his hands.

3. The control method according to claim 2, It is characterized in that The step of "determining whether the user has clothes in his hand according to the analysis result" also includes: If the characteristic value is less than or equal to the preset value, it is determined that the user has no clothes in his hands.

4. The control method according to claim 1, It is characterized in that The server has a deep learning model; The step of "the clothing processing device sends the image to the server so that the server determines the washing and care program according to the image" specifically includes: The laundry processing device sends the image to the server; The server uses the deep learning model to recognize the image and identify attribute information of the clothes in the user's hand; The server determines the washing and care program according to the attribute information.

5. The control method according to claim 4, It is characterized in that After "the server determines the washing and care program according to the clothing attribute information", the control method further includes: The server sends the washing program to the clothing processing device; After the clothes treating device receives the washing and care program, the clothes treating device runs the washing and care program.

6. The control method according to claim 1, It is characterized in that Before "the laundry processing device acquires the image of the user", the control method further includes: The clothing processing device detects whether a user enters a preset range; The clothing processing device will only execute the operation of acquiring the image if a user enters the preset range.

7. The control method according to claim 6, It is characterized in that The control method further comprises: If no user enters the preset range, the laundry treating device does not perform the operation of acquiring the image.