Intelligent rice heating and cooking assistant

By running graphical recognition and machine learning algorithms on mobile devices, identifying food ingredients and estimating heating time, the problem that existing equipment cannot accurately estimate food heating time is solved, and convenient and accurate food heating assistance is achieved, reducing costs and Human error.

CN119087856AInactive Publication Date: 2024-12-06陈大威
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411169757.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-25
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing food heating and cooking temperature control equipment cannot quickly and accurately estimate the heating time of different types of food. It relies on user manpower estimation and traditional detectors, and the cost is high, and the market lacks auxiliary heating and cooking equipment for intelligently estimating time.

Method used

Run a graph recognition algorithm on a mobile computing device, identify the main ingredients and proportions of food through a camera, use a machine learning regression algorithm to estimate the heating time, and prompt users through a TTS voice module or LED display screen to reduce equipment costs and energy consumption.

Benefits of technology

It realizes fast and accurate estimation of food heating time, improves dining experience and life efficiency, reduces labor estimation errors and equipment costs, and is suitable for household popularization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119087856A_ABST
    Figure CN119087856A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent rice heating and cooking assistant which is composed of a hardware part and a software part. The hardware part comprises an MPU or MCU mobile chip, an NPU neural network processor chip, an RAM memory, a FLASH memory, a camera, a TTS voice module, an LED display screen, an LED light supplement lamp and a power supply module. And the software part comprises a Tensorflow Lite or PyTorch Mobile machine learning framework, an image recognition module and a heating time estimation module. The method has the advantages that the pattern recognition algorithm is operated on the mobile computing device to recognize main components and proportions of food, heating properties of different types of food can be distinguished, the food heating and cooking time is predicted through the machine learning regression algorithm, then a user is prompted quickly, food changes can be monitored in real time in the food heating and cooking process, and the user experience is improved. When necessary, an alarm is given to assist cooking, and safety is guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of smart home, in particular to an intelligent rice heating and cooking assistant. Background Art

[0002] With the development of smart home technology, people's requirements for quality of life and safety are increasing.

[0003] The food heating and cooking temperature control equipment currently used on the market have the following problems: 1. It is impossible to quickly and accurately estimate the heating time of different types of food and assist in heating and cooking food; 2. Some rely on the user's manual estimation and control of time and temperature, which makes the use inconvenient and the experience unsatisfactory; 3. Some rely on traditional detectors, such as infrared radiation detectors, and some rely on remote servers for auxiliary calculations, which are costly and not conducive to popularization; 4. There is no similar device on the market that can intelligently estimate time and assist in heating and cooking.

[0004] Specifically, A vehicle-mounted intelligent lunch box intelligent temperature control method and system, patent number: CN118349048A Method: The camera, temperature sensor, weight sensor and remote server in the on-board smart lunch box are used to classify meat and vegetarian dishes to estimate the heating time of the overall type of food in the smart lunch box, and intelligently control the heat preservation or heating; Disadvantages: Using multiple sensors other than cameras and remote servers to estimate food heating time is costly. In addition, the Fast R-CNN algorithm is relatively simple and only roughly classifies food types into meat and vegetarian dishes. It is unable to identify the specific ingredients and proportions of the food in detail, and the estimated heating time is not accurate. Moreover, it is only used for heating smart lunch boxes on trucks, so its application scope is limited.

[0005] A flexible multi-purpose intelligent food temperature control device, patent number: CN113576263A Method: Use thermal infrared sensor probe to sense the real-time temperature of food, and keep or cool the food according to the user's manual setting; Disadvantages: It uses thermal infrared sensing probes, which are expensive and have no intelligent heating time estimation function. It can only rely on users to manually set the temperature to keep warm or cool down to a certain temperature, and its applicable scenarios are limited.

[0006] Intelligent temperature control system for efficient electromagnetic cookers, patent number: CN115789717A Method: By monitoring the vibration of the boiling liquid food in the pot, the heating temperature of the stove can be controlled to prevent overflow; Disadvantages: It is only used to heat porridge, soup and other liquid foods that need to be heated to boiling. The applicable scenarios are limited, and the temperature adjustment will only be triggered after the boiling food produces vibrations for a period of time, which is not safe and clean enough.

[0007] Food heating temperature control methods on the market: Use thermal infrared sensor probes, thermosensitive elements, vibration frequency monitors, smart cameras and remote server-assisted computing to keep food warm and heat it; Disadvantages: It cannot intelligently and accurately estimate the heating time of different types and ingredients of food to assist in heating and cooking, and its applicable scenarios are limited. In addition, some detection components and equipment are expensive and cannot be popularized in households. Summary of the invention

[0008] The purpose of the present invention is to provide an intelligent rice heating and cooking assistant, which runs a graphic recognition algorithm on a mobile computing device to identify the main ingredients and proportions of food, and uses this data as input to estimate the appropriate heating and cooking time for such food through a machine learning regression algorithm, and prompts the user to assist in heating and cooking rice and provide convenience.

[0009] To achieve the above object, the present invention adopts the following technical solutions: An intelligent rice heating and cooking assistant, comprising a hardware part and a software part; The hardware part includes an MPU or MCU mobile chip and an NPU neural network processor chip matched with the MPU or MCU mobile chip, a RAM memory, a FLASH memory, a camera, a TTS voice module, an LED display, an LED fill light, and a power module; The software part includes an image recognition module and a heating time estimation module; The MPU or MCU mobile chip cooperates with the NPU neural network processor, RAM memory and FLASH memory to run Tensorflow Lite or PyTorch Mobile to support the machine learning framework of mobile devices with limited computing power; The real-time images in the stove, oven or microwave oven are read through the device's camera, and then the MobileNetV2 SSD, Tiny YOLOv3 or MobileNetV2 image recognition machine learning algorithms are run on the TensorflowLite or PyTorch Mobile machine learning framework to classify and identify the surface components of the food to be heated in the stove, oven or microwave oven, and the area ratio of each component is counted. For example, 30% green leafy vegetables, 30% noodles, 30% water, 5% meat, 5% vegetable oil, etc., can be the composition of green vegetable noodles. This data is passed to the heating time estimation module to predict and estimate the overall heating time required, and then the user is prompted with a heating time suggestion through the TTS voice module or LED display to assist the user in heating and cooking rice; Specifically: S1: Using machine learning algorithms, on mobile chips and ultra-small devices with limited computing power and low cost, the surface ingredients of food in pots, ovens or microwave ovens captured by cameras are identified, classified and the area ratio of each ingredient is counted; S2: The machine learning algorithm runs on the Tensorflow Lite or PyTorch Mobile framework, supporting mobile and ultra-small devices with limited computing power, reducing device costs and energy consumption; S3: In machine learning algorithms, Tiny YOLOv3 or MobileNetV2 SSD algorithms are used to identify and classify the main components of food images taken by the camera. This algorithm is fast and has high recognition accuracy, and is suitable for mobile chips and components with limited computing power; S4: In the image recognition algorithm, the Tiny YOLOv3 or MobileNetV2 SSD algorithm identifies and classifies the main components of the food image taken by the camera to obtain the pixel area position data of each component in the image, and the area ratio of each component is obtained from the data statistics; S5: In the heating time estimation module, the classification of food surface components and the area ratio of each component are used as input parameters, and the machine learning regression algorithm model is used to predict the heating time, and the user is prompted with the recommended heating time through the TTS voice module or the LED display; S6: During the operation of the algorithm, the Tiny YOLOv3 or MobileNetV2 SSD algorithm can be used to continuously identify the real-time status of the food. If the food has been heated or cooked to the appropriate degree or exceeds the necessary degree, the user can be prompted or alarmed through the TTS voice module or LED display.

[0010] Beneficial effects of the present invention: 1. Using Tensorflow Lite or PyTorch Mobile machine learning frameworks and machine learning algorithms such as MobileNetV2 SSD, Tiny YOLOv3, and regression algorithms, it can run on devices with limited computing power, with a fast speed, real-time recognition, and high recognition and estimation accuracy. It can use MPU or MCU mobile chips such as the recently launched STM32F7 (STMicroelectronics chip) to achieve lower costs; 2. The recognition and estimation accuracy is high, which can replace the errors caused by manual estimation operations, help users heat and cook food successfully in one go, improve dining experience, improve life efficiency, avoid manual estimation errors, waste of time caused by multiple heating, or dining experience degradation and safety and health hazards caused by overheating; 3. Use TTS voice and LED display to prompt the user with the estimated results at the same time, which can assist the visually impaired, the visually impaired or the hearing impaired, the hearing impaired and the elderly users. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a hardware composition structure diagram of the present invention; Figure 2 It is a work flow chart of the present invention; Figure 3 It is a schematic diagram of the use of the present invention; Figure 4 Schematic diagram of the algorithm principle of the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0013] In the description of the embodiments of the present invention, it should be noted that if the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0014] In addition, the terms "horizontal", "vertical", "overhanging" and the like do not mean that the components must be absolutely horizontal or overhanging, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0015] In the description of the embodiments of the present invention, "plurality" means at least 2.

[0016] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. Example

[0017] An intelligent rice cooking assistant, comprising a hardware part and a software part; Hardware: MPU or MCU mobile chip, as the main computing and control unit of the system; NPU neural network processor chip, as an acceleration component to assist AI computing; RAM memory and FLASH memory are used to store data and run programs; A camera for capturing real-time images of food in the oven; TTS voice module, used to issue voice prompts; LED display screen, used to display prompt information; LED fill light, used to fill light for camera shooting; The power module provides a stable power supply for the system.

[0018] Software part: Tensorflow Lite or PyTorch Mobile machine learning frameworks for running machine learning algorithms on mobile devices; Image recognition module, including machine learning algorithms such as MobileNetV2 SSD, Tiny YOLOv3 or MobileNetV2, used to classify, identify and count images taken by the camera; The heating time estimation module includes a machine learning regression algorithm for predicting and estimating the heating time.

[0019] Working principle: Use a camera to capture real-time images of food in the stove; Run image recognition algorithms on the Tensorflow Lite or PyTorch Mobile machine learning framework to classify and identify images, and count the main components of the food surface in the stove and the area ratio of each component, such as 65% lean meat, 25% fat, 8% sauce, 1% green onion, and 1% ginger; Run a regression algorithm on the Tensorflow Lite or PyTorch Mobile machine learning framework, using the data output by the image recognition algorithm module, such as the main ingredients of the food and the proportion of each ingredient, to predict the heating and cooking time; Prompts users with recommended heating and cooking times through TTS voice module or LED display; During the algorithm operation, the Tiny YOLOv3 or MobileNetV2 SSD algorithm can be used to continuously identify the real-time status of food. If the food has been heated or cooked to the appropriate degree or exceeds the necessary degree, the user can be prompted or alarmed through the TTS voice module or LED display.

[0020] Specifically: S1: Using machine learning algorithms, on mobile chips and ultra-small devices with limited computing power and low cost, the surface ingredients of food in pots, ovens or microwave ovens captured by cameras are identified, classified and the area ratio of each ingredient is counted; S2: The machine learning algorithm runs on the Tensorflow Lite or PyTorch Mobile framework, supporting mobile and ultra-small devices with limited computing power, reducing device costs and energy consumption; S3: In machine learning algorithms, Tiny YOLOv3 or MobileNetV2 SSD algorithms are used to identify and classify the main components of food images taken by the camera. This algorithm is fast and has high recognition accuracy, and is suitable for mobile chips and components with limited computing power; S4: In the image recognition algorithm, the Tiny YOLOv3 or MobileNetV2 SSD algorithm identifies and classifies the main components of the food image taken by the camera to obtain the pixel area position data of each component in the image, and the area ratio of each component is obtained from the data statistics; S5: In the heating time estimation module, the classification of food surface components and the area ratio of each component are used as input parameters, and the machine learning regression algorithm model is used to predict the heating time, and the user is prompted with the recommended heating time through the TTS voice module or the LED display; S6: During the operation of the algorithm, the Tiny YOLOv3 or MobileNetV2 SSD algorithm can be used to continuously identify the real-time status of the food. If the food has been heated or cooked to the appropriate degree or exceeds the necessary degree, the user can be prompted or alarmed through the TTS voice module or LED display.

[0021] in, The image recognition and calculation part of heating time estimation can be performed off-device; A Wifi module can be added. On the device, the camera first takes pictures, which are then transmitted to the cloud server or personal computer or mobile phone via Wifi. Machine learning algorithms are used for image recognition, statistics, and prediction. After the estimated results are obtained, they are transmitted back to the device for voice prompts or LED display prompts. The advantages are: cloud servers or private computers have higher computing power, and can use more accurate image recognition algorithms such as YOLOv5, YOLOv8, ResNet50, etc., and can also apply more complex regression algorithms and record and refer to user historical data to improve the accuracy of heating time prediction. However, this will lead to higher time delays, more complex implementation plans, and the possibility of instability and user privacy and security risks.

[0022] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. An intelligent rice cooking assistant, characterized in that: It consists of hardware and software parts; The hardware part includes an MPU or MCU mobile chip and an NPU neural network processor chip matched with the MPU or MCU mobile chip, a RAM memory, a FLASH memory, a camera, a TTS voice module, an LED display, an LED fill light, and a power module; The software part includes an image recognition module and a heating time estimation module; The MPU or MCU mobile chip cooperates with the NPU neural network processor, RAM memory and FLASH memory to run Tensorflow Lite or PyTorch Mobile to support the machine learning framework of mobile devices with limited computing power; The real-time images in the stove, oven or microwave oven are read through the device's camera, and then the MobileNetV2 SSD, Tiny YOLOv3 or MobileNetV2 image recognition machine learning algorithms are run on the Tensorflow Lite or PyTorch Mobile machine learning framework to classify and identify the surface components of the food to be heated in the stove, oven or microwave oven, and count the area proportions of each component, such as 30% green leafy vegetables, 30% noodles, 30% water, 5% meat, 5% vegetable oil, etc., which can be the composition of green vegetable noodles. This data is passed to the heating time estimation module to predict and estimate the overall heating time required, and then the user is prompted with a heating time suggestion through the TTS voice module or LED display to assist the user in heating and cooking rice; Specifically: S1: Using machine learning algorithms, on mobile chips and ultra-small devices with limited computing power and low cost, the surface ingredients of food in pots, ovens or microwave ovens captured by cameras are identified, classified and the area ratio of each ingredient is counted; S2: The machine learning algorithm runs on the Tensorflow Lite or PyTorch Mobile framework, supporting mobile and ultra-small devices with limited computing power, reducing device costs and energy consumption; S3: In machine learning algorithms, Tiny YOLOv3 or MobileNetV2 SSD algorithms are used to identify and classify the main components of food images taken by the camera. This algorithm is fast and has high recognition accuracy, and is suitable for mobile chips and components with limited computing power; S4: In the image recognition algorithm, the Tiny YOLOv3 or MobileNetV2 SSD algorithm identifies and classifies the main components of the food image taken by the camera to obtain the pixel area position data of each component in the image, and the area ratio of each component is obtained from the data statistics; S5: In the heating time estimation module, the classification of food surface components and the area ratio of each component are used as input parameters, and the machine learning regression algorithm model is used to predict the heating time, and the user is prompted with the recommended heating time through the TTS voice module or the LED display; S6: During the operation of the algorithm, the Tiny YOLOv3 or MobileNetV2 SSD algorithm can be used to continuously identify the real-time status of the food. If the food has been heated or cooked to the appropriate degree or exceeds the necessary degree, the user can be prompted or alarmed through the TTS voice module or LED display.