Image acquisition device, household appliance and smart home system

Abstract

The invention relates to the technical field of intelligent household electrical appliances, and discloses an image acquisition device, comprising: a camera module configured to acquire an image of a target area; a first communication module configured to receive detection data; and the information processing module is configured to input the acquired image and the received detection data into a target information processing model, and control the operation of target equipment according to a control instruction output by the target information processing model. According to the device, the target information processing model is localized, and quick response and timely control can be realized in household appliance control. In particular, during cooking, potential risks can be prevented or intervened, the intelligent degree of cooking can be improved, and the cooking time investment of a user is reduced. The invention further discloses a household appliance and a smart home system.

Description

Technical Field

[0001] The present application relates to the technical field of smart home appliances, for example, to an image acquisition device, home appliance equipment and a smart home system. Background Art

[0002] With the continuous advancement of technology and the popularization of smart scenarios, users have higher and higher requirements for intelligent control of home appliances. Especially for home appliances in cooking scenarios, users expect to be able to pay attention to the cooking progress in time without having to wait in the kitchen for a long time or being busy with other things in the kitchen. This can avoid cooking risks such as dry pots and overflowing pots, as well as safety hazards caused by cooking risks.

[0003] The related art discloses an integrated stove control system with an anti-burn function, comprising: an image acquisition device, used to acquire images near the integrated stove head and transmit them to a wireless transceiver module; a wireless transceiver module, used to transmit images and instructions between a cloud server; a cloud server, used to receive and forward images from the wireless transceiver module or recognition results from a remote image recognition AI system, and output control instructions based on the recognition results; a remote image recognition AI system, used to perform human hand recognition on the received images, and return the recognition results to the cloud server; a controller, used to receive instructions sent by the cloud server and control an actuator; an actuator, used to adjust the size of the flame or turn off the flame according to the instructions sent by the controller.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art:

[0005] The related art only discloses the anti-burn function, but does not involve how to prevent or intervene in potential risks in the cooking process. Summary of the invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The embodiments of the present disclosure provide an image acquisition device, a household appliance, and a smart home system to prevent or intervene in potential risks in the cooking process and reduce the time investment of users in cooking.

[0008] In some embodiments, the image acquisition device includes: a camera module, configured to acquire images of a target area; a first communication module, configured to receive detection data; an information processing module, configured to input the acquired images and the received detection data into a target information processing model, and control the operation of the target device according to the control instructions output by the target information processing model.

[0009] In some embodiments, the household appliance includes: an image acquisition device as described above.

[0010] In some embodiments, the smart home system includes: a first household appliance, including the image acquisition device as described above; a second household appliance, communicatively connected to the first household appliance, for executing control instructions of the image acquisition device, and / or for sending detection data to the image acquisition device.

[0011] The image acquisition device, home appliance and smart home system provided by the embodiments of the present disclosure can achieve the following technical effects:

[0012] In the disclosed embodiment, the image acquisition device is provided with an information processing module, and the information processing module is provided with a target information processing model. In actual application, the acquired images and detection data can be input into the target information processing model to generate control instructions to control the home appliance. In this way, the target information processing model is localized, and rapid response and timely control can be achieved in the control of the home appliance. In particular, in cooking, potential risks can be prevented or intervened, which helps to improve the intelligence of cooking and reduce the time investment of users in cooking.

[0013] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0015] Figure 1 is a structural schematic diagram of an image acquisition device provided by an embodiment of the present disclosure;

[0016] Figure 2 is a structural schematic diagram of another image acquisition device provided by an embodiment of the present disclosure;

[0017] Figure 3 is a structural schematic diagram of a household appliance provided by an embodiment of the present disclosure;

[0018] Figure 4 is a schematic structural diagram of a range hood provided by an embodiment of the present disclosure;

[0019] Figure 5 is a structural schematic diagram of another household appliance provided by an embodiment of the present disclosure;

[0020] Figure 6 Schematic diagram of a smart home system provided by an embodiment of the present disclosure.

[0021] Reference numerals:

[0022] 10: image acquisition device; 11: camera module; 12: first communication module; 13: information processing module; 20: target device; 22: cooking device; 21: exhaust device; 30: baking device; 311: third communication module; 312: temperature adjustment module; 40: server; 100: household appliance; 101: first household appliance; 102: second household appliance; 211: second communication module; 212: display screen; 213: main control module; 200: smart home system. DETAILED DESCRIPTION

[0023] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0024] The terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0025] Unless otherwise stated, the term "plurality" means two or more.

[0026] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0027] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0028] In the embodiments of the present disclosure, home appliances include smart home appliances. Smart home appliances refer to home appliances that are formed by introducing microprocessors, sensor technology, and network communication technology into home appliances, and have the characteristics of intelligent control, intelligent perception, and intelligent application. The operation process of smart home appliances often relies on the application and processing of modern technologies such as the Internet of Things, the Internet, and electronic chips. For example, smart home appliances can be connected to electronic devices to enable users to remotely control and manage smart home appliances.

[0029] Combination Figure 1 , 2 As shown, the image acquisition device 10 includes a camera module 11, a first communication module 12 and an information processing module 13. The camera module 11 is configured to acquire an image of a target area. The first communication module 12 is configured to receive detection data. The information processing module 13 is configured to input the acquired image and the received detection data into a target information processing model, and control the operation of the target device 20 according to the control instructions output by the target information processing model.

[0030] Here, the camera module 11 can be a camera or a video camera, and the captured image can be a picture or a video. The target area is set based on the control requirements. For example, the target area in the cooking scene is generally the area where the cooking equipment is located; the target area in the elderly guarding scene is generally the living room area and / or the bedroom area. The first communication module 12 communicates with the home appliance and / or the detection device (such as a sensor, etc.) to receive the detection data. The information processing module 13 stores a target information processing model, which analyzes and processes the captured images and the received detection data to output control instructions. The target information processing model depends on the application scenario, and is generally a model corresponding to a scene with higher effectiveness requirements, such as an overflow model in a cooking scene, a fall detection model in an elderly guarding scene, etc. The target information processing model 13 has an image processing function and can process image data. Compared with sending image data to the cloud or a local server for processing, the timeliness is higher and the data is more reliable. The target device is a home appliance. In the kitchen scene, the target device 20 includes a cooking device 22 and / or an exhaust device 21. The image acquisition device of the embodiment of the present disclosure helps to improve the real-time and accuracy of home appliance control.

[0031] As an example, the camera device is a camera, the first communication module includes a WIFI module and / or a Bluetooth module, etc.; the information processing module includes an integrated circuit with edge computing capabilities, such as a SOC (System on Chip). In the cooking scenario, the collected images and detection data are input into the overflow model to determine whether overflow occurs, and then the kitchen appliances are controlled to solve the overflow problem.

[0032] The image acquisition device provided by the embodiment of the present disclosure is provided with an information processing module, and the information processing module is provided with a target information processing model. In actual application, the acquired images and detection data can be input into the target information processing model to generate control instructions to control the home appliance. In this way, the target information processing model is localized, and rapid response and timely control can be achieved in the control of the home appliance. In particular, in cooking, potential risks can be prevented or intervened, which helps to improve the intelligence of cooking and reduce the time investment of users in cooking.

[0033] Optionally, the target area comprises a cooking area.

[0034] The disclosed embodiments are mainly aimed at controlling kitchen appliances, so the target area is the cooking area. That is, based on the cooking information, the appliances are controlled. Specifically, the cooking area is the cooking equipment area; cooking equipment includes gas stoves, induction cookers, etc. As an example, the area where the gas stove is located is taken as the target area; specifically, the area of ​​the gas stove in the image is greater than or equal to the preset proportion. The preset proportion can be 85% to 100%. In this way, not only can the cooking information be obtained, but also the interference of other areas in the judgment of the cooking area information can be avoided.

[0035] Optionally, the detection data includes: the on / off state of the cooking device, and / or the fire power of the cooking device, and / or the wind speed level of the exhaust device.

[0036] Here, the on / off state of the cooking device means whether the cooking device is in the on state or the off state. The on state indicates that cooking is in progress, and the off state indicates that cooking is in progress or idle. The firepower of the cooking device means the firepower level of the cooking device, such as high fire, medium fire or low fire. Exhaust equipment includes range hoods, air purifiers, exhaust fans, centrifugal fans, etc. The wind speed level may include high, medium, low, etc. Among them, the detection data is determined based on various control requirements in the cooking scene. As an example, when controlling oil smoke in a cooking scene, the detection data may include the firepower of the cooking device and the wind speed level of the exhaust device. When controlling the overflow of the pot in a cooking scene, the detection data may include the on / off state of the cooking device.

[0037] Optionally, the target information processing model is determined based on the collected first image and first detection data; wherein the first image is used to determine the cooking state, and the first detection data is used to determine the switch state of the cooking device.

[0038] Here, the first image includes the image features of the pot, and the image features of the pot include the image features of the pot body and / or the image features of the food in the pot. That is, the image features of the pot body can be used to determine the type of pot, such as a steamer, a frying pan, a ceramic pot, etc. The image features of the food in the pot can be used to determine the recipe, the cooking method, etc. In this way, the cooking state can be determined based on the first image. In combination with the cooking state and the switch state of the cooking device, the target information processing model can be determined.

[0039] Specifically, the cooking status of the pot includes uncooked, cooked with water, cooked with a lid, and cooked without water. Specifically, if the image feature of the pot shows that there is no food in the pot, it is determined to be uncooked. If the image feature of the pot shows that there is food in the pot and the food is soaked in water, it is determined to be cooked with water. If the image feature of the pot shows that there is food in the pot and there is no water or a small amount of water, it is determined to be cooked without water. If the image feature of the pot shows that the pot has a lid and the food cannot be identified, it is determined to be cooked with a lid.

[0040] Furthermore, when the cooking state is water cooking or lid cooking, and the switch state of the cooking device is on, less oil smoke is generated during the cooking process, which mainly involves the control of the cooking process and the anti-overflow control. Therefore, in this scenario, the target information processing model is a model for overflow control. When the cooking state is waterless cooking and the switch state of the cooking device is on, more oil smoke is generated during the cooking process, which mainly involves the control of the cooking process and oil smoke control. Therefore, in this scenario, the target information processing model is a model for oil smoke control.

[0041] In this way, the corresponding target information processing model can be determined based on the cooking state and the on / off state of the cooking device. The target information processing model is adapted to the cooking scene, so as to accurately control the target device. This avoids image recognition errors during waterless cooking, and improper adjustment of the target device due to model mismatch, which causes the fire to be turned off. At the same time, it also avoids overflowing during water cooking, and improper adjustment of the target device due to model mismatch, which affects the cooking experience.

[0042] Optionally, the first image is a collection of continuous multiple-frame images. Here, the first image includes multiple images. By analyzing and processing the continuous multiple-frame images, the accuracy of the analysis can be guaranteed, and the difficulty of the analysis and processing can be reduced. In addition, the second image, the third image, etc. below are all collections of continuous multiple-frame images.

[0043] Optionally, when the first image and the first detection data satisfy a first condition, the target information processing model is a first model for oil fume control; and / or, when the first image and the first detection data satisfy a second condition, the target information processing model is a second model for overflow control.

[0044] Here, the first condition and the second condition are set for the first model and the second model respectively, so as to determine whether the first image and the first detection data satisfy the first condition or the second condition, thereby determining the target information processing model.

[0045] Optionally, the first condition includes: the cooking state determined by the image feature of the pot in the first image is waterless cooking, and the stove state determined by the first detection data is on.

[0046] Here, as mentioned above, when the cooking state is waterless cooking and the switch state of the cooking device is on, the cooking method is generally stir-frying. At this time, cooking generates more oil smoke, so based on this, the first condition is set for the first model for oil smoke control.

[0047] Optionally, the second condition includes: the cooking state determined by the image feature of the pot in the first image is cooking with water or cooking with a lid, and the stove state determined by the first detection data is on.

[0048] Here, as mentioned above, when the cooking state is cooking with water or cooking with a lid, and the switch state of the cooking device is on, the cooking method is generally stewing, boiling and steaming. At this time, cooking produces less oil smoke but there is a risk of overflowing, so based on this, the second condition is set for the second model for overflowing control.

[0049] Optionally, the image acquisition device determines that the household appliance corresponding to the target information processing model is the target device based on the second mapping relationship.

[0050] Here, when the target information processing model is determined, the corresponding target device can be determined based on the second mapping relationship. As an example, when the target information processing model is the first model for oil fume control, the target device is an exhaust device. That is, the oil fume control is achieved by controlling the exhaust device. When the target information processing model is the second model for pot overflow control, the target device is a cooking device. That is, the pot overflow problem is solved by controlling the cooking device.

[0051] Optionally, the image acquisition device determines, based on the second mapping relationship, that the household appliance corresponding to the target information processing model is the target device, including:

[0052] When the target information processing model is the first model for oil fume control, the image acquisition device takes the exhaust device as the target device; or, when the cooking detection model is the second model for pot overflow control, the image acquisition device takes the exhaust device and the cooking device as the target devices; wherein the exhaust device is used to exhaust oil fume and the cooking device is used to adjust the firepower.

[0053] Here, when dealing with oil smoke, the exhaust equipment is the main control object. When dealing with pot overflow, the cooking equipment and the exhaust equipment are the main control objects. In this way, when stir-frying, the wind force of the exhaust equipment is timely controlled to exhaust the oil smoke. When stewing, the cooking equipment and the exhaust equipment are timely controlled to avoid pot overflow; or, when the pot overflows, the fire is adjusted to solve the pot overflow problem and the exhaust wind force is adjusted to exhaust the smoke generated when the pot overflows in time.

[0054] Optionally, the second model for overflow control includes a first overflow model and a second overflow model; the image acquisition device determines the target second model, including:

[0055] The image acquisition device determines the type of the cookware according to the first image;

[0056] The image acquisition device determines the target second model according to the type of the cookware.

[0057] As mentioned above, the first image includes the image features of the pot, and the image features of the pot include the image features of the pot body. Therefore, based on the image features of the pot body, the type of pot can be determined. The types of pots include pots without lids and pots with lids, and pots with lids include transparent pot lids, translucent pot lids, opaque pot lids, etc. It can be understood that the opaque pot lid cannot obtain the bubble situation in the pot, while other pots can obtain the bubble situation through image analysis. Therefore, the overflow models corresponding to the above two situations are different.

[0058] Optionally, the image acquisition device determines the target second model according to the type of the cookware, including:

[0059] When the pot type is a pot without a lid, or the pot type is a pot with a lid and the lid is transparent, the image acquisition device determines that the target overflowing pot model is the first overflowing pot model.

[0060] When the pot type is a pot with a lid and the lid is opaque, the image acquisition device confirms that the target overflowing pot model is the second overflowing pot model.

[0061] Here, the first overflow model mainly judges the overflow status by the bubbles inside the pot, while the second overflow model mainly judges the overflow status by the soup around the pot. In this way, the overflow judgment of the pot where the bubbles are not visible is realized, and then the automatic control of the home appliances in the overflow scenario is realized. When the user is not in the kitchen or is busy with other things and does not pay attention to the overflow, the home appliances can be adjusted in time to improve the intelligence of cooking.

[0062] Optionally, when the target information processing model is the second model, the image acquisition device controls the operation of the target device according to the control instruction output by the target information processing model, including:

[0063] The image acquisition device uses the target overflow model to analyze and process the acquired images to determine the overflow status.

[0064] The image acquisition device outputs control instructions for controlling the wind speed level of the exhaust device and the fire speed level of the cooking device according to the overflow status of the pot, and controls the target device to execute.

[0065] Here, the overflowing pot status includes overflowing pot and about to overflow pot. Based on the overflowing pot status, the wind speed level of the exhaust device and the fire power level of the cooking device are controlled. Usually, when the pot has overflowed, the fire power level of the cooking device is set to a lower level and the fire is turned off, and the wind speed level of the exhaust device is set to a higher level. When the pot is about to overflow, the fire power level of the cooking device is adjusted to a lower level, and the wind speed level of the exhaust device can remain unchanged. In this way, according to different cooking scenarios, matching home appliances are adjusted so that the regulation of the home appliances meets the needs of the cooking scenarios.

[0066] Optionally, the image acquisition device outputs a control instruction for controlling the wind speed level of the exhaust device and the fire speed level of the cooking device according to the overflow state, including:

[0067] When it is determined that the overflow state is about to overflow, or has overflowed and the overflow duration is less than the first duration, the control instruction output by the image acquisition device is to control the fire level of the cooking equipment to be adjusted to the minimum fire level, and to control the wind level of the exhaust equipment to be adjusted to a medium / high level.

[0068] When it is determined that the overflow state is overflow and the overflow duration is greater than or equal to the first duration, the control instruction output by the image acquisition device is to control the cooking device to shut down and control the wind speed level of the exhaust device to a high level.

[0069] Here, based on the overflow state and the overflow duration, the exhaust device and the cooking device are adjusted. Specifically, when the pot is about to overflow (the overflow indicates that the overflow has not occurred but is about to overflow), or when the pot has just overflowed (the pot has overflowed and the overflow duration is less than the first duration), the cooking device is controlled to be adjusted to the minimum firepower, and the wind force of the exhaust device is controlled to be adjusted to the medium / high level. At this time, the overflow has not occurred or the time of occurrence is short. Timely reducing the firepower can avoid overflow or avoid long-term overflow; at the same time, the wind force is adjusted to discharge the smoke / odor generated by water vapor or short-term overflow. In the case of overflow and the overflow duration is greater than or equal to the first duration, the cooking device is controlled to be turned off and the wind force of the exhaust device is the highest wind force. At this time, the overflow occurs for a period of time. In order to eliminate safety hazards and avoid more overflows, the cooking device is turned off, and the smoke / odor generated by the overflow is discharged at the highest wind force. Among them, the first duration can be 3 seconds to 10 seconds. In this way, based on the overflow state, the corresponding home appliances are controlled; so that the regulation of home appliances matches the cooking scene.

[0070] Optionally, after the image acquisition device determines the target information processing model according to the acquired first image and the first detection data, it further includes:

[0071] The image acquisition device determines the position change information of the cookware according to the acquired images.

[0072] When the position change information of the cookware indicates that the cookware has been displaced and the displacement is greater than the first distance, the image acquisition device re-determines the target device.

[0073] Here, the captured image refers to the image captured by the image acquisition device after the target device is determined, which is used to determine the position change information of the pot. It can be understood that in the process of controlling the target device, if the pot is displaced, the cooking type may change or the current cooking is ended. Therefore, in the process of controlling the target device, the position change of the pot is judged based on the real-time captured image. If the position change information of the pot is that the pot is displaced, and the displacement is greater than the first distance, the target device is re-determined. Here, the target device is re-determined based on the third image of the new pot or the original pot on the cooking device after the displacement. In this way, when the cooking type changes, the corresponding detection model is matched so as to adaptively control the target device. Among them, the first distance is greater than the diameter of the pot, and the value can be set based on user needs.

[0074] Optionally, the information processing module is further configured to analyze and process the first image to identify the type of cookware or stove, so as to generate a display instruction, and control the target device to display the image of the cookware or stove according to the display instruction.

[0075] Here, the first image refers to the image captured by the image acquisition device in response to the instruction within the first time period. As mentioned above, the image features of the pot include the image features of the pot body, and the image features of the pot body can be used to determine the type of pot, such as a steamer, a frying pan, a ceramic pot, etc. In the early stage of cooking, there is a situation where the pot has not been placed on the cooking equipment. At this time, the first image includes the image features of the stove, so the type of stove is determined based on the image features of the stove. In this way, when the type of pot or stove is identified, a corresponding display instruction is generated. The target device is controlled to display the image of the pot or the image of the stove according to the display instruction. Among them, the target device can also be an exhaust device, such as a range hood; it can also be other household appliances with screens in the kitchen.

[0076] It should be noted that, in some embodiments, in order to ensure the processing speed and processing accuracy of the image acquisition device, only the target information processing model is stored in the image acquisition device. For non-real-time controlled image processing, it is completed on the server. That is, when image display is required, if the identification of the type of pot or stove needs to be based on the pot identification model or the stove identification model, these two models are stored in the server. The first image collected is sent to the server for processing and the processing result is fed back to the image acquisition device. The image acquisition device generates a display instruction based on the feedback result and controls the target device to execute the display instruction. In this way, it is possible to avoid being limited by the computing power and memory of the image acquisition device.

[0077] In other embodiments, the image acquisition device can determine the type of cookware or stove after processing and analyzing the first image, without the need for a cookware recognition model or stove recognition model, and the image acquisition device completes the image processing and analysis locally.

[0078] Optionally, the information processing module is further configured to analyze and process the third image to determine the position change information of the cookware; when the position change information of the cookware indicates that the cookware has shifted and the displacement is greater than the first distance, the image acquisition device re-determines the target information processing model.

[0079] The third image refers to the image captured by the image acquisition device after the target device is determined, and is used to determine the position change information of the pot. It can be understood that during the control of the target device, if the pot is displaced, the cooking type may change or the current cooking is ended. Therefore, during the control of the target device, the position change of the pot is judged based on the third image. If the position change information of the pot is that the pot is displaced, and the displacement is greater than the first distance, the target device is re-determined. Here, the target device is re-determined based on the third image of the new pot or the original pot on the cooking device after the displacement. In this way, when the cooking type changes, the corresponding detection model is matched so as to adaptively control the target device. Among them, the first distance is greater than the diameter of the pot, and the value can be set based on user needs.

[0080] Optionally, the information processing module is also configured to analyze and process the first image to identify ingredient information; and, based on the ingredient information, call the target recipe to generate a first instruction to control the target device to display the target recipe, and / or, control the operation of the target device according to the first instruction.

[0081] Here, the information processing model also recognizes the food information based on the first image, and then calls the target recipe stored in the server based on the food information; then the target recipe can be displayed to assist the user in cooking. At the same time, a first instruction is generated based on the target recipe to control the target device. For example, the firepower of the cooking device can be controlled at each stage of the cooking process according to the target recipe. In some embodiments, the target device also includes a baking device, and the baking temperature and baking time of the baking device can be controlled based on the first instruction generated by the target recipe.

[0082] Optionally, the first communication module is also configured to send a second instruction output by the information processing module so that the target household appliance responds to the second instruction; wherein the second instruction is generated by the target information processing model based on the second image and the second detection data, and the second instruction includes cooking process information and / or cooking abnormality information.

[0083] Here, the second image and the second detection data are the images captured by the camera module after the target information processing model is determined and the detection data received by the first communication module. During the cooking process, it is necessary not only to control the oil smoke and overflowing pot, but also to prompt the cooking progress or cooking abnormalities, that is, to generate a second instruction. Among them, the second instruction is executed by the target home appliance. The target home appliance includes the above-mentioned target device, and may also include home appliances or terminal devices with display screens or voice functions outside the kitchen. The target home appliance is used to display and / or play the second instruction. The second instruction is such as dry pot, overflowing pot, cooking end, etc. In this way, the user can also obtain the cooking progress and abnormalities in time when he is not in the kitchen or busy with other things in the kitchen.

[0084] Combination Figure 3 , 4 As shown, the embodiment of the present disclosure provides a household appliance 100, including: a household appliance body, and the above-mentioned image acquisition device 10. The image acquisition device is installed on the household appliance body. The installation relationship described here is not limited to being placed inside the household appliance body, but also includes installation connections with other components of the household appliance 100, including but not limited to physical connections, electrical connections or signal transmission connections. It can be understood by those skilled in the art that the image acquisition device can be adapted to a feasible household appliance body, thereby realizing other feasible embodiments.

[0085] It should be noted that when the image acquisition device is provided in a household appliance, especially in a range hood, the range hood can communicate with the image acquisition device through the first communication module (i.e., the range hood also has the first communication module), and can also be connected with the information processing module 13 of the image acquisition device through the second communication module 211 of the range hood. As an example, the first communication module is WiFi and / or Bluetooth; the second communication module is a UART (Universal Asynchronous Receiver / Transmitter) interface. That is, the information processing module includes a UART interface.

[0086] The household appliance further includes a main control module 213, which is used to control the execution module to execute the instruction issued by the image acquisition device. As an example, the household appliance is a range hood, and the execution module is a fan. When the main control module receives the instruction to adjust the wind speed level, the instruction is sent to the fan, and the fan executes the instruction.

[0087] Optionally, the household appliance includes a range hood and / or a baking appliance.

[0088] Here, when the household appliance includes a range hood, the image acquisition device is installed on the range hood (e.g. Figure 4), to collect images of the cooking area below the range hood. When the household appliance includes a baking device, the image acquisition device can be installed on the outside of the baking device to collect images of the cooking area. When the household appliance includes a range hood and a baking device, the image acquisition device is installed on the range hood.

[0089] Optionally, the household appliance further includes: a display screen 212 for displaying a stove image or a pot image or a recipe; wherein the stove image or the pot image or the recipe is obtained by processing the image captured by the image capture device.

[0090] Here, the household appliance further includes a display screen to display a stove image or a pot image or a recipe. When displaying the stove image or the pot image, the image after style conversion can be displayed to improve the aesthetics of the image. The stove image or the pot image is determined according to the stove type or the pot type after processing the captured image.

[0091] Combination Figure 5 Optionally, when the household appliance includes a baking device 30, it also includes: a temperature adjustment module 312, which is used to adjust the oven preheating temperature and / or baking temperature; wherein the oven preheating temperature and / or baking temperature is obtained by processing the image captured by the image acquisition device.

[0092] Here, when the household appliance includes a baking device, the image acquisition device processes the acquired image to determine the food information. When the food can be a baking food, the target recipe is determined based on the food information, and then whether the oven needs to be preheated, the preheating temperature, and the baking temperature, etc. are confirmed according to the target recipe. In this way, the preheating temperature and baking temperature of the oven can be adjusted. Compared with installing a camera inside the baking device, the process of putting the baked food into the oven for identification and then taking it out can be omitted, making the cooking steps simpler. In addition, the baking device includes a third communication module 311, which is connected to the first communication module of the image acquisition device 10. Among them, the first communication module and the third communication module can be WiFi modules respectively.

[0093] Optionally, when the household appliance includes a baking device 30, it also includes: a time module 312, which is used to control the preheating time and / or baking time; wherein the oven preheating temperature and / or baking temperature is obtained by processing the image captured by the image acquisition device.

[0094] Here, when the household appliance includes a baking device, not only the preheating temperature and the baking temperature can be adjusted, but also the preheating time and the baking time can be controlled, thereby improving the intelligence of the baking device.

[0095] Combination Figure 6As shown, the smart home system 200 includes: a first home appliance 101 and a second home appliance 102. The first home appliance 101 includes the image acquisition device as described above. The second home appliance 102 is connected to the image acquisition device and is used to execute the control instructions of the image acquisition device and / or to send detection data to the image acquisition device.

[0096] Here, the first household appliance may be an exhaust device, such as a range hood. The second household appliance may be a cooking device such as a stove or an oven. The second household appliance includes a second communication module, which is connected to the first communication module of the image acquisition device to receive instructions from the image acquisition device and send detection information to the image acquisition device. That is, the image acquisition device serves as the control center of the smart home system.

[0097] In addition, in some embodiments, the smart home system also includes a server 40 and other home appliances / terminal devices. The image acquisition device is connected to the server through the first communication module so that the second instruction in the above text can be sent to other home appliances through the server, so that other home appliances can output cooking process information or cooking abnormality information. Other home appliances include home appliances with screens or voice functions. In addition, the terminal device can also execute the second instruction. The images collected and processed by the image acquisition device are stored in the server to reduce the memory usage of the image acquisition device. The server also stores other models other than the target information processing model so that they can be called from the server when used.

[0098] The terminal device refers to an electronic device with a wireless connection function. The terminal device can communicate with the above-mentioned smart home appliances by connecting to the Internet, or can communicate with the above-mentioned smart home appliances directly through Bluetooth, WiFi, etc. In some embodiments, the terminal device is, for example, a mobile device, a computer, or a vehicle-mounted device built into a hover car, or any combination thereof. The mobile device may include, for example, a mobile phone, a smart home device, a wearable device, a smart mobile device, a virtual reality device, etc., or any combination thereof, wherein the wearable device may include, for example, a smart watch, a smart bracelet, a pedometer, etc.

[0099] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes.

[0100] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0101] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0102] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0103] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. An image acquisition device, characterized in that: include: A camera module configured to capture an image of a target area; A first communication module is configured to receive detection data; The information processing module is configured to input the collected images and the received detection data into the target information processing model, and control the operation of the target device according to the control instructions output by the target information processing model.

2. The device according to claim 1, characterized in that The target area includes: a cooking area.

3. The device according to claim 1, characterized in that The detection data includes: the switch status of the cooking device, and / or the fire power of the cooking device, and / or the wind speed level of the exhaust device.

4. The device according to any one of claims 1 to 3, characterized in that The target information processing model is determined based on the collected first image and first detection data; The first image is used to determine the cooking status, and the first detection data is used to determine the switch status of the cooking device.

5. The device according to claim 4, characterized in that When the first image and the first detection data satisfy the first condition, the target information processing model is a first model for oil fume control; and / or, When the first image and the first detection data satisfy the second condition, the target information processing model is the second model for overflow control.

6. The device according to claim 5, characterized in that The first condition includes: The cooking state determined by the image feature of the pot in the first image is waterless cooking, and the cooking device state determined by the first detection data is turned on.

7. The device according to claim 5, characterized in that The second condition includes: The cooking state determined by the image feature of the pot in the first image is cooking with water or cooking with a lid, and the cooking device state determined by the first detection data is turned on.

8. The device according to claim 4, characterized in that The information processing module is further configured to analyze and process the first image to identify the type of the cookware or the type of the stove, so as to generate a display instruction, and control the target device to display the image of the cookware or the image of the stove according to the display instruction.

9. The device according to claim 4, characterized in that The information processing module is also configured to analyze and process the first image to identify the ingredient information; and, based on the ingredient information, call the target recipe to generate a first instruction to control the target device to display the target recipe, and / or, control the operation of the target device according to the first instruction.

10. The device according to claim 4, characterized in that The first communication module is also configured to send a second instruction output by the information processing module so that the target household appliance responds to the second instruction; wherein the second instruction is generated by the target information processing model based on the second image and the second detection data, and the second instruction includes cooking process information and / or cooking abnormality information.

11. A household appliance, characterized in that: include: An image acquisition device as claimed in any one of claims 1 to 10.

12. The household appliance according to claim 11, characterized in that: The household electrical appliances include range hoods and / or baking equipment.

13. The household appliance according to claim 11, characterized in that: Also includes: A display screen is used to display a stove image or a pot image or a recipe; wherein the stove image or the pot image or the recipe is obtained by processing the image captured by the image capture device.

14. The household appliance according to any one of claims 11 to 13, characterized in that: In the case where the household appliance includes baking equipment, it also includes: The temperature adjustment module is used to adjust the preheating temperature and / or baking temperature of the oven; wherein the preheating temperature and / or baking temperature of the oven is obtained by processing the images collected by the image acquisition device.

15. The household appliance according to any one of claims 11 to 13, characterized in that: In the case where the household appliance includes baking equipment, it also includes: The time module is used to control the preheating time and / or baking time; wherein the oven preheating temperature and / or baking temperature is obtained by processing the images acquired by the image acquisition device.

16. A smart home system, characterized in that: include: A first household appliance, comprising the image acquisition device according to any one of claims 1 to 10; The second household appliance is connected to the image acquisition device and is used to execute the control instructions of the image acquisition device and / or to send detection data to the image acquisition device.

Images