Control method and device of home appliance, home appliance and readable storage medium

Abstract

The application provides a control method and device of a household appliance, the household appliance and a readable storage medium, wherein the control method of the household appliance comprises: acquiring multiple frames of environment images; determining a target brightness according to the multiple frames of environment images; and controlling an illuminating device of the household appliance to work according to the target brightness. In the embodiment of the application, on one hand, the illuminating device of the household appliance can provide a more suitable brightness, which is convenient for the user to use the household appliance and improves the use effect; on the other hand, the multiple frames of images are used for statistical recognition, so that the instability of the recognition relying on a single frame of image in the related art is overcome, and the judgment of the current environment brightness is more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of household appliances, in particular to a control method of a household appliance, a control device of a household appliance, a household appliance and a readable storage medium. BACKGROUND

[0002] With the vigorous development of artificial intelligence technology, users have increasingly strong demand for smart home. A household appliance with artificial intelligence function can greatly improve product quality and user experience.

[0003] However, in the related art, the light brightness provided by the household appliance is fixed during the use of the household appliance by the user, which can result in unsatisfactory cooking effect and user eye fatigue. SUMMARY

[0004] The present application aims to at least solve one of the problems in the prior art or related art.

[0005] To this end, one aspect of the present application provides a control method of a household appliance.

[0006] Another aspect of the present application provides a control device of a household appliance.

[0007] Still another aspect of the present application provides a household appliance.

[0008] Yet another aspect of the present application provides a readable storage medium.

[0009] Therefore, according to one aspect of the present application, a control method of a household appliance is provided, which comprises: acquiring multiple frames of environment images; determining a target brightness according to the multiple frames of environment images; and controlling a lighting device of the household appliance to work according to the target brightness.

[0010] In this technical solution, the lighting device is installed on the household appliance, which can provide light to illuminate the environment around the household appliance. The image acquisition device is installed on the household appliance, which can acquire multiple frames of environment images, or the communication device is installed on the household appliance, which can receive multiple frames of environment images provided by the external image acquisition device.

[0011] Based on the acquired multiple frames of environment images, the environment brightness of the environment where the household appliance is located is determined, and then the target brightness for controlling the lighting device is obtained. Further, the lighting device adjusts the light brightness to the target brightness to adapt to the environment brightness of the environment where the household appliance is located.

[0012] The embodiment of the present application can control the lighting device of the household appliance to provide appropriate brightness, facilitate the use of the household appliance by the user, and improve the use effect. In addition, the multi-frame image is used for statistical recognition, the instability of the single-frame image recognition in the related art is overcome, and the judgment of the current environment brightness is more accurate.

[0013] The control method of the household appliance according to the present application can further have the following additional technical features.

[0014] In the above technical solution, the multi-frame environment image is obtained by collecting a first environment video and obtaining the multi-frame environment image from the first environment video at a preset time interval.

[0015] In the related art, the single-frame image is used for environment brightness recognition, which is prone to recognition errors due to detection errors. Unlike the related art, the present application requires processing multiple frames of images at a time, which are continuously extracted from the video frames (i.e., the first environment video) collected by the image acquisition device at a preset time interval.

[0016] By using multiple frames of images for environment brightness recognition, the method is more robust and accurate than the single-frame image method, thereby ensuring the accuracy of the brightness adjustment of the lighting device.

[0017] In any of the above technical solutions, the first environment video is collected by obtaining the working state of the household appliance and collecting the first environment video based on the working state of the household appliance.

[0018] In this technical solution, it is determined whether the household appliance is working, and the first environment video is collected when the household appliance is working, and the collection of the first environment video is stopped when the household appliance is not working.

[0019] In this way, the image acquisition can be automatically controlled based on the working state of the household appliance, without the need for manual operation by the user, thereby improving the intelligence of the household appliance.

[0020] In any of the above technical solutions, the target brightness is determined according to the multi-frame environment image, including determining the target brightness according to the multi-frame environment image and a brightness prediction model.

[0021] In this technical solution, the final target brightness is output according to the multi-frame environment image and the brightness prediction model. Specifically, the brightness prediction model can identify the environment brightness of the environment where the household appliance is located based on the multi-frame environment image, and then determine the target brightness corresponding to the environment brightness for final output.

[0022] In the technical solution of the application, the brightness prediction model pre-trained is used to directly obtain the final target brightness output from the input, thereby improving the speed and accuracy of target brightness acquisition.

[0023] In any of the above technical solutions, the target brightness is determined according to the multi-frame environment image and the brightness prediction model, comprising: performing convolution processing on the multi-frame environment image to extract image features; converting the image features into a target vector; inputting the target vector into the brightness prediction model to output the target brightness.

[0024] In the technical solution, the multi-frame environment image is continuously subjected to convolution processing to extract image features, wherein the image features include shape, texture, color, brightness and the like.

[0025] The extracted image features are then stretched into a one-dimensional vector (i.e., a target vector) through a reshape function or global pooling, and finally the final target brightness is output through the brightness prediction model.

[0026] It should be noted that, since it is a multi-input detection model architecture, the convolution processing of the multi-input image can adopt a two-dimensional (2D) convolution or a three-dimensional (3D) convolution processing scheme.

[0027] In the above manner, the multi-frame environment image is sequentially subjected to convolution processing and vector conversion processing to obtain a one-dimensional vector, realizing an end-to-end framework, i.e., the input to the output of the brightness prediction model, thereby directly obtaining the final target brightness, improving the speed and accuracy of target brightness acquisition.

[0028] In any of the above technical solutions, the multi-frame environment image is subjected to convolution processing to extract image features, comprising: performing image stitching processing on the multi-frame environment image to generate a target image; performing two-dimensional convolution processing on the target image to extract image features.

[0029] In the technical solution, the multi-frame environment image is directly subjected to image stitching in the channel dimension, for example, if the input image size is WxHx3, wherein W represents the width of the image, H represents the height of the image, and 3 represents the RGB three channels of the image acquisition device, then after stitching of N images, the target image becomes WxHx3xN, and then the target image is subjected to 2D convolution processing, thereby achieving the purpose of extracting image features.

[0030] It should be noted that the multi-frame environment image includes a visible light image and an image of different spectrum, and the input of the 2D convolution processing is the width W and the height H of the image.

[0031] By the above manner, the image features are extracted by using the 2D convolution processing scheme, the 2D convolution processing method is simpler and is easy to be deployed to the household appliance, and the processing speed of the household appliance is improved.

[0032] In any of the above technical solutions, the image features are extracted by performing convolution processing on the multiple environment images, including: performing two-dimensional convolution processing on the multiple environment images respectively to extract multiple sub-image features; and performing feature splicing processing on the multiple sub-image features to obtain the image features.

[0033] In this technical solution, the multiple environment images are subjected to 2D convolution processing respectively, and the sub-image features of each image are extracted continuously, and then the multiple sub-image features are subjected to feature splicing in the channel dimension to obtain the image features.

[0034] It should be noted that the input of the 2D convolution processing is the width W and the height H of the image.

[0035] By the above manner, the image features are extracted by using the 2D convolution processing scheme, the 2D convolution processing method is simpler and is easy to be deployed to the household appliance, and the processing speed of the household appliance is improved.

[0036] In any of the above technical solutions, the image features are extracted by performing convolution processing on the multiple environment images, including: performing two-dimensional convolution processing on the multiple environment images respectively to extract multiple sub-image features; and performing feature splicing processing on the multiple sub-image features to obtain the image features.

[0037] In this technical solution, the multiple environment images are subjected to 2D convolution processing respectively, and the sub-image features of each image are extracted continuously, and then the multiple sub-image features are subjected to feature splicing in the channel dimension to obtain the image features.

[0038] It should be noted that the input of the 2D convolution processing is the width W and the height H of the image.

[0039] By the above manner, the image features are extracted by using the 2D convolution processing scheme, the 2D convolution processing method is simpler and is easy to be deployed to the household appliance, and the processing speed of the household appliance is improved.

[0040] In any of the above technical solutions, the method further includes: collecting a second environment video; obtaining at least one group of sampling images in the second environment video according to a preset time interval; and establishing a brightness prediction model according to the at least one group of sampling images.

[0041] In this technical solution, the second environment video is obtained in advance, and at least one group of sampling images in the second environment video is obtained at a preset time interval, that is, a sample set for model training is obtained, and the sample set for model training is input into the network for training to obtain the brightness prediction model.

[0042] By the above manner, the precise brightness prediction model is established, and thus the precise control of the target brightness of the lighting device of the household appliance is realized.

[0043] In any of the above technical solutions, the luminance prediction model is established according to the at least one set of sample images, including: obtaining luminance label information of the at least one set of sample images; and establishing the luminance prediction model according to the at least one set of sample images and the corresponding luminance label information.

[0044] In this technical solution, it is assumed that the lighting device of the household appliance has N different luminance levels, which will be used as the final classification output of the image.

[0045] The sample set obtained by training the above model is labeled with luminance, and luminance label information is obtained. The luminance prediction model is obtained by training the model based on at least one set of sample images and corresponding luminance label information.

[0046] For example, when the household appliance is a range hood, a series of range hood scene videos under different environments are recorded, and sample image sets are randomly extracted at a predetermined time interval, for example, 4 images are taken every 1 second, and the number of images in each set is the same, that is, the time interval span is the same.

[0047] Manual labeling is performed, that is, the luminance level of the sample image set under different environments is manually judged according to experience. In this way, each sample image set corresponds to a luminance level, thereby forming a sample set for model training. Then, a batch of image sets are randomly shuffled and input into the network for training. The target loss function of the training output is the cross-entropy loss, that is, the cross-entropy of the output luminance level of the network and the manually labeled luminance level (i.e., luminance label information). The entire model is trained according to the target loss function, so that the target loss function of the model finally reaches an ideal threshold or a specified number of rounds, and the model is finally saved.

[0048] In the above manner, a precise luminance prediction model is established, thereby realizing precise control of the target luminance of the lighting device of the household appliance.

[0049] According to another aspect of the present application, a control device of a household appliance is provided, including: an acquisition module configured to acquire a plurality of environmental images; a determination module configured to determine a target luminance according to the plurality of environmental images; and a control module configured to control a lighting device of the household appliance to work according to the target luminance.

[0050] In this technical solution, the lighting device is installed on the household appliance, and the lighting device can provide light to illuminate the environment around the household appliance. The image acquisition device is installed on the household appliance, and the image acquisition device can acquire a plurality of environmental images, or the communication device is installed on the household appliance, and the communication device can receive a plurality of environmental images provided by an external image acquisition device.

[0051] Based on the obtained multiple frames of environment images, the environment brightness of the environment where the household appliance is located is determined, and then the target brightness corresponding to the control of the lighting device is obtained. Further, the lighting device is controlled to adjust the light brightness to the target brightness to adapt to the environment brightness of the environment where the household appliance is located.

[0052] In an embodiment of the present application, on the one hand, the lighting device of the household appliance can provide a more appropriate brightness, facilitating the use of the household appliance by the user and improving the use effect; on the other hand, the multi-frame image is used for statistical recognition, overcoming the instability of the single-frame image recognition in the related art, and the judgment of the current environment brightness is more accurate.

[0053] According to still another aspect of the present application, a household appliance is provided, comprising: a lighting device; a memory storing a program or instructions; and a processor, which implements the steps of the control method of the household appliance according to any one of the above technical solutions when executing the program or instructions.

[0054] The household appliance provided by the present application comprises the lighting device, the memory and the processor, and the program or instructions are executed by the processor to implement the steps of the control method of the household appliance according to any one of the above technical solutions, so that the household appliance comprises all the beneficial effects of the control method of the household appliance according to any one of the above technical solutions.

[0055] According to still another aspect of the present application, a readable storage medium is provided, which stores a program or instructions, and the program or instructions are executed by the processor to implement the steps of the control method of the household appliance according to any one of the above technical solutions.

[0056] The readable storage medium provided by the present application, the program or instructions are executed by the processor to implement the steps of the control method of the household appliance according to any one of the above technical solutions, so that the readable storage medium comprises all the beneficial effects of the control method of the household appliance according to any one of the above technical solutions.

[0057] Additional aspects and advantages of the present application will become apparent from the following description with reference to specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0058] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0059] Figure 1 Fig. 1 shows one of the flow diagrams of the control method of the household appliance according to an embodiment of the present application;

[0060] Figure 2 Fig. 2 shows a structural diagram of the range hood according to an embodiment of the present application;

[0061] Figure 3A schematic diagram of determining multi-frame images of an embodiment of the present application is shown.

[0062] Figure 4 A flowchart of a control method of an appliance of an embodiment of the present application is shown.

[0063] Figure 5 A schematic diagram of a target brightness determination method of an embodiment of the present application is shown.

[0064] Figure 6 A logic diagram of a 2D convolution processing scheme of an embodiment of the present application is shown.

[0065] Figure 7 A logic diagram of a 2D convolution processing scheme of an embodiment of the present application is shown.

[0066] Figure 8 A logic diagram of a 3D convolution processing scheme of an embodiment of the present application is shown.

[0067] Figure 9 A flowchart of a control method of a range hood of an embodiment of the present application is shown.

[0068] Figure 10 A schematic block diagram of a control device of an appliance of an embodiment of the present application is shown.

[0069] Figure 11 A schematic block diagram of an appliance of an embodiment of the present application is shown.

[0070] Figure 12 A schematic block diagram of an appliance of an embodiment of the present application is shown.

[0071] Wherein, Figure 2 The correspondence between the reference signs and the component names in the accompanying drawings is as follows:

[0072] 200 range hood, 202 LED lamp, 204 camera. DETAILED DESCRIPTION

[0073] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work fall within the protection scope of the present application.

[0074] It should be noted that all the direction indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the direction indications will also change accordingly.

[0075] In addition, the descriptions such as "first", "second" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can be explicitly or implicitly included at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0076] In the present application, unless otherwise explicitly specified and limited, the terms "connection", "fixation" and the like should be understood in a broad sense, for example, "fixation" can be fixed connection, or detachable connection, or integral; can be mechanical connection, or electrical connection; can be direct connection, or indirect connection through intermediate medium; can be internal communication of two elements or interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0077] In addition, the technical solutions of each embodiment of the present application can be combined with each other, but it must be based on the fact that a person skilled in the art can realize it, and when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor is it within the scope of protection required by the present application.

[0078] The control method of the household appliance, the control device of the household appliance, the household appliance and the readable storage medium provided by the embodiments of the present application will be described in detail below in combination with the drawings, specific embodiments and application scenarios.

[0079] Embodiment one

[0080] The embodiments of the present application provide a control method of a household appliance, Figure 1 A flowchart of a control method of a household appliance according to an embodiment of the present application is shown. The method comprises:

[0081] Step 102, collecting multiple frames of environment images;

[0082] Step 104, predicting target brightness based on the multiple frames of environment images;

[0083] Step 106, controlling the lighting device of the household appliance to work according to the target brightness.

[0084] In the technical solution, the lighting device is installed on the household appliance, and the lighting device can provide light to illuminate the environment around the household appliance. The image acquisition device is installed on the household appliance, and the image acquisition device can acquire multiple frames of environment images, or the communication device is installed on the household appliance, and the communication device can receive multiple frames of environment images provided by the external image acquisition device.

[0085] Based on the acquired multiple frames of environment images, the environment brightness of the environment where the household appliance is located is determined, and then the target brightness for controlling the lighting device is correspondingly obtained. Further, the lighting device is controlled to adjust the light brightness to the target brightness to adapt to the environment brightness of the environment where the household appliance is located.

[0086] It should be noted that the household appliance includes a range hood, a refrigerator, an induction cooker and the like, and the lighting device includes an LED (Light Emitting Diode) lamp, an incandescent lamp and the like.

[0087] For example, when the household appliance is a range hood, as shown in Figure 2 The range hood 200 is installed with an LED lamp 202 and a camera 204, and the brightness of the LED lamp 202 is divided into three brightness levels of high, medium and low. Multiple frames of environment images of the range hood are acquired by the camera 204, and the current environment brightness is determined to be dark (for example, the current weather is rainy) by using the multiple frames of environment images, and then it is determined that the brightness level of the LED lamp 202 should be high, so that the LED lamp 202 is adjusted to the highest brightness, so that the user can clearly see the cooking degree of the food material, and the eye fatigue caused by the too low environment brightness is avoided.

[0088] In the embodiment of the present application, on the one hand, the lighting device of the household appliance can provide a more appropriate brightness, which is convenient for the user to use the household appliance and improves the use effect; on the other hand, multiple frames of images are used for statistical recognition, which overcomes the instability of the related art relying on single frame image recognition, and the judgment of the current environment brightness is more accurate.

[0089] Embodiment two

[0090] In the embodiment of the present application, the step of acquiring multiple frames of environment images specifically includes: acquiring a first environment video; and acquiring multiple frames of environment images in the first environment video at a preset time interval.

[0091] In the related art, the single frame image is used for environment brightness recognition, which is easy to cause recognition error due to detection error. Unlike the related art, in the technical solution of the present application, multiple frames of images need to be processed at a time, as shown in Figure 3 The multiple frames of images are continuously extracted from the video frames (i.e. the first environment video) collected by the image acquisition device at a preset time interval.

[0092] By using multiple frames of images to identify the ambient brightness, the method is more robust and accurate than the single frame image method, thereby ensuring the accuracy of the brightness adjustment of the lighting device.

[0093] Embodiment three

[0094] In the embodiment of the present application, the step of acquiring the first ambient video specifically comprises: detecting the current working state of the home appliance; and acquiring the first ambient video when the home appliance is working.

[0095] In the technical solution, it is determined whether the home appliance is working, the first ambient video is collected when the home appliance is working, and the collection of the first ambient video is stopped when the home appliance is not working.

[0096] In the above manner, the automatic control of image collection can be realized based on the working state of the home appliance, without manual operation of the user, and the intelligence of the home appliance is improved.

[0097] Embodiment four

[0098] In the embodiment of the present application, the step of predicting the target brightness based on multiple ambient images specifically comprises: predicting the target brightness based on the multiple ambient images and a pre-established brightness prediction model.

[0099] In the technical solution, the final target brightness is output through the brightness prediction model according to the multiple ambient images. Specifically, the brightness prediction model can identify the ambient brightness of the environment where the home appliance is located based on the multiple ambient images, and then determine the target brightness corresponding to the ambient brightness for final output.

[0100] In the technical solution of the present application, the last target brightness output is directly obtained from the input by using the pre-trained brightness prediction model, thereby improving the speed and accuracy of target brightness acquisition.

[0101] Embodiment five

[0102] In the embodiment of the present application, Figure 4 A flowchart of a control method of a home appliance according to an embodiment of the present application is shown. The method comprises:

[0103] Step 402: collect multiple ambient images;

[0104] Step 404: perform convolution processing on the multiple ambient images to extract image features of the multiple ambient images;

[0105] Step 406: perform vector conversion on the extracted image features to generate a target vector;

[0106] Step 408, input the target vector into the brightness prediction model, so as to output the target brightness;

[0107] Step 410, control the lighting device of the household appliance to work according to the target brightness.

[0108] According to the multi-frame environment image and the brightness prediction model, the target brightness is determined, including: performing convolution processing on the multi-frame environment image to extract image features; converting the image features into a target vector; inputting the target vector into the brightness prediction model to output the target brightness.

[0109] In the technical solution, as shown in the figure, Figure 5 The determination method of the target brightness includes image input, CNN (Convolutional Neural Network) feature extraction, Softmax classification, and target brightness output.

[0110] Specifically, the multi-frame environment image is continuously subjected to convolution processing to extract image features, and the image features include shape, texture, color, brightness, and the like.

[0111] The extracted image features are stretched into a one-dimensional vector (i.e., a target vector) through a reshape function or global pooling, and finally the target brightness is output through a brightness prediction model (Softmax).

[0112] It should be noted that, since it is a multi-input detection model architecture, the convolution processing of the multi-input image can adopt a two-dimensional (2D) convolution or a three-dimensional (3D) convolution processing scheme.

[0113] In the above manner, the multi-frame environment image is sequentially subjected to convolution processing and vector conversion processing to obtain a one-dimensional vector, an end-to-end framework is realized, i.e., the input to the output of the brightness prediction model, so that the final target brightness is directly obtained, and the speed and accuracy of obtaining the target brightness are improved.

[0114] Embodiment six

[0115] In the embodiment of the application, the step of performing convolution processing on the multi-frame environment image to extract image features of the multi-frame environment image specifically includes: performing image splicing processing on the multi-frame environment image to obtain a target image, and then performing 2D convolution processing on the target image to extract final image features.

[0116] In the technical solution, as shown in the figure, Figure 6As shown, the multi-frame environment image is directly spliced in the channel dimension, for example, if the input image size is WxHx3, wherein W represents the width of the image, H represents the height of the image, and 3 represents the RGB three channels of the image acquisition device, then N images are spliced into a target image of WxHx3xN, and then the target image is subjected to 2D convolution processing, so as to realize the purpose of extracting image features.

[0117] The extracted image features are stretched into a one-dimensional vector (i.e., a target vector) through a reshape function or global pooling, and finally the final target brightness is output through a brightness prediction model (Softmax).

[0118] It should be noted that the multi-frame environment image includes a visible light image and an image of different spectra, and the input of the 2D convolution processing is the width W and the height H of the image.

[0119] In the above manner, the 2D convolution processing scheme is used to extract image features, the 2D convolution processing method is simpler, can be easily deployed to a household appliance, and improves the processing speed of the household appliance.

[0120] Embodiment Seven

[0121] In the embodiment of the application, the multi-frame environment image is subjected to convolution processing, and the step of extracting image features of the multi-frame environment image includes: respectively performing 2D convolution processing on the multi-frame environment image to extract sub-image features of each frame of environment image, thereby obtaining a plurality of sub-image features; and performing feature splicing processing on the plurality of sub-image features to generate image features.

[0122] In the technical solution, as shown, Figure 7 The multi-frame environment image is subjected to 2D convolution processing, sub-image features of each frame of image are continuously extracted, and then the plurality of sub-image features are subjected to feature splicing in the channel dimension to obtain image features.

[0123] The extracted image features are stretched into a one-dimensional vector (i.e., a target vector) through a reshape function or global pooling, and finally the final target brightness is output through a brightness prediction model (Softmax).

[0124] It should be noted that the input of the 2D convolution processing is the width W and the height H of the image.

[0125] In the above manner, the 2D convolution processing scheme is used to extract image features, the 2D convolution processing method is simpler, can be easily deployed to a household appliance, and improves the processing speed of the household appliance.

[0126] Embodiment eight

[0127] In the embodiment of the present application, the step of performing convolution processing on the multiple frames of environment images to extract image features of the multiple frames of environment images specifically includes: performing 3D convolution processing on the multiple frames of environment images to extract image features of the multiple frames of environment images.

[0128] In the technical solution, as shown in the technical solution, Figure 8 the multiple frames of environment images are taken as a whole, and 3D convolution processing is used to continuously extract image features.

[0129] Then, the extracted image features are stretched into a one-dimensional vector (i.e., a target vector) through a reshape function or global pooling, and finally the final target brightness is output through a brightness prediction model (Softmax).

[0130] It should be noted that the input of the 3D convolution processing is the width W, the height H of the image, and the collection time of the image.

[0131] In the above manner, the image features are extracted by using the 3D convolution processing scheme, and the 3D convolution processing method is more accurate, so that the target brightness is more accurate.

[0132] It should be noted that based on the above embodiment six, embodiment seven and embodiment eight, the embodiment of the present application uses different processing methods to fuse a group of images to form a complete and fast end-to-end detection framework, which is more accurate than the detection method in the related art, thereby improving the function judgment of the household appliance and providing more accurate judgment and experience for the user.

[0133] Embodiment nine

[0134] In the embodiment of the present application, the method further includes: collecting a second environment video; obtaining at least one group of sampling images in the second environment video according to a preset time interval; and establishing a brightness prediction model according to the at least one group of sampling images.

[0135] In the technical solution, the second environment video is obtained in advance, and at least one group of sampling images in the second environment video is obtained at a preset time interval, that is, a sample set for model training is obtained, and then the sample set for model training is input into the network for training to obtain the brightness prediction model.

[0136] In the above manner, the precise brightness prediction model is established, so that the precise control of the target brightness of the lighting device of the household appliance is realized.

[0137] Embodiment ten

[0138] The embodiment of the present application establishes a brightness prediction model according to at least one set of sample images, including: obtaining brightness label information of at least one set of sample images; and establishing a brightness prediction model according to the at least one set of sample images and the corresponding brightness label information.

[0139] In the technical solution, it is assumed that the light of the lighting device of the household appliance has N different brightness levels, which will be used as the final classification output of the image.

[0140] The sample set obtained by training the above model is labeled with brightness, and brightness label information is obtained. The model is trained based on at least one set of sample images and the corresponding brightness label information, and a brightness prediction model is obtained.

[0141] Exemplarily, when the household appliance is a range hood, a series of range hood scene videos under different environments are recorded, and sample image sets are randomly extracted at preset time intervals, for example, 4 images are taken every 1 second, and the number of images in each set is the same, that is, the time interval span is the same.

[0142] Manual labeling is performed, that is, the brightness level of the sample image set under different environments is manually judged according to experience. In this way, each sample image set corresponds to a brightness level, thereby forming a sample set for model training. Then, a batch of image sets are randomly shuffled and input into the network for training. The output target loss function of the training is cross-entropy loss, that is, the cross-entropy of the output brightness level of the network and the manually labeled brightness level (i.e. brightness label information). The entire model is trained according to the target loss function, so that the target loss function of the model finally reaches an ideal threshold or a specified number of rounds, and the model is finally saved.

[0143] In the above manner, a precise brightness prediction model is established, thereby realizing precise control of the target brightness of the lighting device of the household appliance.

[0144] Embodiment eleven

[0145] In the embodiment of the present application, the household appliance is a range hood, as shown in the drawings, the control method of the range hood comprises: Figure 9 As shown in the drawings, the control method of the range hood comprises:

[0146] Step 902, turn on the hob switch of the range hood, load the brightness prediction model, initialize the camera, start the photographing mode, and obtain a video;

[0147] Step 904, select an image set in the video at a time interval same as the time interval for training the brightness prediction model;

[0148] Step 906, input the image set as input into the brightness prediction model;

[0149] Step 908, according to the output category of the brightness prediction model, the brightness adjustment instruction of the light is sent.

[0150] Step 910, the brightness of the lamp of the range hood is adjusted according to the brightness adjustment instruction.

[0151] Step 912, it is judged whether the stove is off or not, if not, return to step 904, if off, enter step 914.

[0152] Step 914, turn off the camera and release the brightness prediction model.

[0153] The scheme solves the automatic adjustment method of the lamp brightness of the range hood in the complex cooking scene, and the single picture is not used to judge the light brightness, so that the instability or error of the judgment is avoided. Several pictures can make the network more stable and accurate. Deep learning and the scene are organically combined together, the light brightness can be accurately controlled, the iterative optimization strategy is continuously circulated, and a complete decision control process is given.

[0154] Embodiment twelve

[0155] The embodiment of the application provides a control device of a household appliance, Figure 10 A schematic block diagram of the control device 1000 of the household appliance is shown, and the control device 1000 of the household appliance comprises an acquisition module 1002, a determination module 1004 and a control module 1006.

[0156] The acquisition module 1002 can collect multiple frames of environment images, the determination module 1004 can predict target brightness based on the multiple frames of environment images, and the control module 1006 can control the lighting device of the household appliance to work according to the target brightness.

[0157] In the technical scheme, the lighting device is installed on the household appliance, and the lighting device can provide light to illuminate the environment around the household appliance. The image acquisition device is installed on the household appliance, and the image acquisition device can collect multiple frames of environment images, or the communication device is installed on the household appliance, and the communication device can receive multiple frames of environment images provided by the external image acquisition device.

[0158] Based on the obtained multiple frames of environment images, the environment brightness of the environment where the household appliance is located is determined, and then the target brightness for controlling the lighting device is correspondingly obtained. Further, the lighting device is controlled to adjust the light brightness to the target brightness, so as to adapt to the environment brightness of the environment where the household appliance is located.

[0159] In the embodiment of the present application, on the one hand, the lighting device of the household appliance can be controlled to provide a more appropriate brightness, facilitating the use of the household appliance by the user and improving the use effect; on the other hand, a plurality of frames of images are used for statistical recognition, overcoming the instability of the related art relying on single-frame image recognition, and the judgment of the current environmental brightness is more accurate.

[0160] In this embodiment, the acquisition module 1002 is specifically configured to process a plurality of frames of images at a time, and the plurality of frames of images are continuously extracted at a preset time interval from a continuous video frame (i.e., a first environmental video) collected by an image acquisition device. By using the plurality of frames of images to identify the environmental brightness, the method is more robust and accurate than the single-frame image method, thereby ensuring the accuracy of the brightness adjustment of the lighting device.

[0161] In this embodiment, the acquisition module 1002 is specifically configured to determine whether the household appliance is working, and in the case that the household appliance is working, the first environmental video is collected, and in the case that the household appliance is not working, the collection of the first environmental video is stopped. Through the above-mentioned manner, the automatic control of image acquisition can be realized based on the working state of the household appliance, without the need for manual operation by the user, thereby improving the intelligence of the household appliance.

[0162] In this embodiment, the determination module 1004 is specifically configured to output the final target brightness through a brightness prediction model according to the plurality of frames of environmental images. Specifically, the brightness prediction model can identify the environmental brightness of the environment in which the household appliance is located based on the plurality of frames of environmental images, and then determine the target brightness corresponding to the environmental brightness for final output. In the technical solution of the present application, the brightness prediction model trained in advance is used to directly obtain the final target brightness output from the input, thereby improving the speed and accuracy of target brightness acquisition.

[0163] In this embodiment, the determination module 1004 is specifically configured to extract image features through continuous convolution processing of the plurality of frames of environmental images, wherein the image features include shape, texture, color, brightness and the like. Then, the extracted image features are stretched into a one-dimensional vector (i.e., a target vector) through a reshape function or global pooling, and finally the final target brightness is output through the brightness prediction model. Through the above-mentioned manner, the plurality of frames of environmental images are sequentially subjected to convolution processing and vector conversion processing to obtain a one-dimensional vector, realizing an end-to-end framework, i.e., the input to the output of the brightness prediction model, thereby directly obtaining the final target brightness, thereby improving the speed and accuracy of target brightness acquisition.

[0164] In this embodiment, the determining module 1004 is specifically configured to directly perform image stitching on the channel dimension of the multiple frames of environment images to obtain a target image, and then perform 2D convolution processing on the target image, so as to achieve the purpose of extracting image features.

[0165] In this embodiment, the determining module 1004 is specifically configured to perform 2D convolution processing on the multiple frames of environment images respectively, constantly extract sub-image features of each frame of image, and then perform feature stitching on the channel dimension of the multiple sub-image features to obtain image features.

[0166] In this embodiment, the determining module 1004 is specifically configured to take the multiple frames of environment images as a whole, and constantly extract image features by using 3D convolution processing.

[0167] In this embodiment, the control device 1000 of the household appliance further includes a model establishing module, configured to pre-acquire a second environment video, acquire at least one set of sampling images in the second environment video at a preset time interval, that is, obtain a sample set for model training, and input the sample set for model training into a network to perform training, to obtain a brightness prediction model.

[0168] In this embodiment, the model establishing module is configured to perform brightness labeling on the sample set for model training obtained above to obtain brightness labeling information, and perform model training based on the at least one set of sampling images and the corresponding brightness labeling information, to obtain the brightness prediction model.

[0169] Embodiment Thirteen

[0170] In the embodiments of the present application, a household appliance is provided, Figure 11 A schematic block diagram of a household appliance 1100 is shown, which includes an illumination device 1102, a memory 1104, and a processor 1106.

[0171] The memory 1104 stores programs or instructions, and the processor 1106 implements the steps of the control method of the household appliance according to any of the above technical solutions when executing the programs or instructions. The memory 1104 and the processor 1106 can be connected by a bus or other means. The processor 1106 can include one or more processing units, and the processor 1106 can be a central processing unit (CPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like chip.

[0172] The home appliance 1100 provided by the present application comprises the lighting device 1102, the memory 1104 and the processor 1106, and the program or the instruction is executed by the processor 1106 to realize the steps of the control method of the home appliance of any of the above technical solutions, so the home appliance 1100 comprises all the beneficial effects of the control method of the home appliance of any of the above technical solutions.

[0173] In the above technical solution, the home appliance 1100 further comprises an image acquisition device configured to acquire the first environment video.

[0174] In the above technical solution, the home appliance 1100 further comprises an image acquisition device configured to acquire the first environment video.

[0175] Embodiment fourteen

[0176] The embodiment of the present application provides a home appliance, Figure 12 A schematic block diagram of a home appliance 1200 is shown, which comprises a lighting device 1202 and the control device 1000 of the home appliance according to the above embodiment.

[0177] The home appliance 1200 provided by the present application comprises the lighting device 1202 and the control device 1000 of the home appliance according to the above technical solution, so the home appliance 1200 comprises all the beneficial effects of the control device of the home appliance according to any of the above technical solutions.

[0178] In the above technical solution, the home appliance 1200 further comprises an image acquisition device configured to acquire the first environment video.

[0179] In the above technical solution, the home appliance 1200 further comprises an image acquisition device configured to acquire the first environment video.

[0180] Embodiment fifteen

[0181] The embodiment of the present application provides a readable storage medium, which stores a program or an instruction, and the program or the instruction is executed by a processor to realize the steps of the control method of the home appliance according to any of the above technical solutions.

[0182] The readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disc or an optical disc, etc.

[0183] The readable storage medium, the program or the instruction provided by the application realize the steps of the control method of the household appliance of any of the above technical solutions when executed by the processor, and therefore the readable storage medium includes all the beneficial effects of the control method of the household appliance of any of the above technical solutions.

[0184] The above only describes the preferred embodiments of the application and is not intended to limit the application. The application can be variously changed and modified by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A control method of a home appliance, characterized by, The method comprises: acquiring a plurality of environment images; determining a target brightness according to the plurality of environment images; controlling a lighting device of the household appliance to work according to the target brightness; wherein, based on the acquired plurality of environment images, the ambient brightness of the environment where the household appliance is located is determined, and then the target brightness for controlling the lighting device is correspondingly obtained; the determination of the target brightness according to the plurality of environment images comprises: performing convolution processing on the plurality of environment images to extract image features; converting the image features into a target vector; inputting the target vector into a brightness prediction model to output the target brightness; wherein, the brightness prediction model identifies the ambient brightness of the environment where the household appliance is located based on the plurality of environment images, and then determines the target brightness corresponding to the ambient brightness; the convolution processing on the plurality of environment images to extract image features comprises: performing image stitching processing on the plurality of environment images to generate a target image; performing two-dimensional convolution processing on the target image to extract the image features.

2. The method of claim 1, wherein, The acquisition of the plurality of environment images comprises: collecting a first environment video; acquiring a plurality of environment images in the first environment video at a preset time interval.

3. The method of claim 2, wherein, The collection of the first environment video comprises: acquiring the working state of the household appliance; based on the working of the household appliance, collecting the first environment video.

4. The method of claim 1, wherein, The convolution processing on the plurality of environment images to extract image features comprises: performing two-dimensional convolution processing on the plurality of environment images respectively to extract a plurality of sub-image features; performing feature stitching processing on the plurality of sub-image features to obtain the image features.

5. The method of claim 1, wherein, The convolution processing on the plurality of environment images to extract image features comprises: performing three-dimensional convolution processing on the plurality of environment images to extract the image features.

6. The method according to any one of claims 1 to 3, characterized in that, Further comprising: collecting a second environment video; acquiring at least one group of sampling images in the second environment video at a preset time interval; establishing the brightness prediction model according to the at least one group of sampling images.

7. The method of claim 6, wherein, The establishment of the brightness prediction model according to the at least one group of sampling images comprises: acquiring brightness labeling information of the at least one group of sampling images; establishing the brightness prediction model according to the at least one group of sampling images and the corresponding brightness labeling information.

8. A control device of a home electric appliance, characterized by comprising: The method comprises: an acquisition module for acquiring a plurality of environment images; a determination module for determining a target brightness according to the plurality of environment images; a control module for controlling a lighting device of the household appliance to work according to the target brightness; wherein, based on the acquired plurality of environment images, the ambient brightness of the environment where the household appliance is located is determined, and then the target brightness for controlling the lighting device is correspondingly obtained; The determining module is configured to perform convolution processing on the multiple frames of environment images to extract image features, convert the image features into a target vector, input the target vector into a brightness prediction model, and output the target brightness. The convolution processing on the multiple frames of environment images to extract image features includes image stitching processing on the multiple frames of environment images to generate a target image, and two-dimensional convolution processing on the target image to extract the image features. The brightness prediction model identifies the ambient brightness of the environment in which the home appliance is located based on the multiple frames of environment images, and determines the target brightness corresponding to the ambient brightness.

9. An electric home appliance characterized by comprising: The home appliance comprises: a lighting device; a memory storing a program or instructions; a processor configured to execute the program or instructions to implement the steps of the control method of the home appliance according to any one of claims 1 to 7.

10. A readable storage medium, on which a program or instructions are stored, characterized in that, The program or instructions, when executed by the processor, implement the steps of the control method of the home appliance according to any one of claims 1 to 7.

Images