Intelligent heating control method and system for microwave oven

Through the deep learning network model, the problem of intelligent microwave ovens not being able to identify new food and dynamically adjust the heating time is solved, and the precise heating and safe cooking of food is achieved, which improves the user experience and equipment intelligence.

CN118890734AInactive Publication Date: 2024-11-01THE FIRST AFFILIATED HOSPITAL HENGYANG MEDICAL SCHOOL UNIV OF SOUTH CHINA
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410725441.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-06
Publication Date
2024-11-01
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When heating food, existing smart microwave ovens have problems such as not being able to identify new food types and being unable to dynamically adjust the heating time according to food status and user needs, resulting in incomplete heating or overheating, which poses public safety hazards.

Method used

Through communication between the server and the client, the deep learning network model is used to optimize the cloud training model in combination with user personalized settings and experience, collect food images and pressure sensor data, identify food types and calculate reasonable heating time, and optimize the heating time calculation model.

Benefits of technology

It realizes the accurate heating time calculation of different foods, avoids the risks of burning and combustion, improves the intelligence level and public safety of microwave ovens, and provides a convenient and efficient cooking experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118890734A_ABST
    Figure CN118890734A_ABST
Patent Text Reader

Abstract

The invention provides a microwave oven intelligent heating control method and system.The method comprises the steps that an intelligent microwave oven server side is initialized, an initial food heating time calculation model is established, and the method further comprises the following steps that an intelligent microwave oven client side collects image data and feedback information when food is put into a microwave oven, uploading to an intelligent microwave oven server side; the food heating time calculation model identifies food types according to the image data and the feedback information, retrieves a heating time equation, calculates six types of reasonable food heating time based on the weight information of the food, and returns the food heating time to the intelligent microwave oven client; the display screen displays the food pictures, names and calculated heating time from top to bottom according to the similarity according to the six returned heating programs of the food with the highest similarity with the pictures of the food put into the microwave oven by the user; and after the heating program is finished, the intelligent microwave oven client side collects and packs data in the heating program and sends the data to the intelligent microwave oven server side.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent electrical appliances, and in particular to an intelligent heating control method and system for a microwave oven. Background Art

[0002] In the current medical service environment, the public microwave ovens in the inpatient department are designed to provide convenience for patients and their families to heat food. However, due to the large number of patients in the hospital, some family members are not familiar with the correct use of microwave ovens, and improper operation problems occur during actual use, such as improper setting of time, which leads to incomplete heating or overheating of food, and even causes scorching or burning, posing a public safety hazard.

[0003] Traditional cooking equipment such as smart microwave ovens and smart ovens usually use food type recognition technology to analyze food types by taking photos and searching in a preset food database to determine the corresponding heating time. However, the application of this technology has obvious limitations. First, for novel food types outside the database, the device cannot automatically identify and set the appropriate heating time, resulting in users still needing to make manual adjustments when using it. Secondly, the device cannot dynamically specify a specific heating time based on the specific food state and the user's personalized needs. In other words, all foods of the same type use a unified heating time, which obviously ignores the impact of individual differences in food on the heating effect. In addition, once the heating time setting for a certain food in the database is inaccurate or unreasonable, the device cannot automatically adjust and optimize, which limits the improvement of the device's adaptability and intelligence level.

[0004] The usual smart microwave heating scheme is as follows:

[0005] 1. User input: Users need to manually input relevant parameters according to the type, weight and desired doneness of food, and the device calculates and sets the heating time based on this information. However, this method requires users to have a certain understanding of food and the working principle of microwave ovens in order to reasonably set the heating time, otherwise the food may not be heated sufficiently, or even burn or catch fire.

[0006] 2. Preset Programs: Manufacturers preset a series of standard heating programs in the device, suitable for different types of food. Users only need to select the program that matches the desired food, and the device will automatically adjust the heating time and power. However, this method has limitations, such as the preset program cannot be changed and is only applicable to common food types. For food types that are not stored in the local database, this solution cannot be used.

[0007] 3. Food recognition: The built-in camera or scanner is used to identify the appearance of food, intelligently determine the type of food, and search the database for the corresponding heating time. It may also combine parameters such as food weight to calculate the appropriate heating time through an algorithm. However, this method also has limitations, such as the local database covers a limited number of food types, and the recognition accuracy of specific foods needs to be improved.

[0008] 4. Dynamic adjustment: The smart device dynamically adjusts the heating time according to food feedback (such as internal temperature) to ensure that the food is heated to the best state. Although this method can adjust the time according to the actual heating situation, the ultimate goal is still to automatically identify or manually input the type of food, obtain the appropriate temperature of the food stored in the database, and dynamically adjust the heating time based on this standard. Although the above-mentioned solutions have solved the problem of how to set the appropriate time for microwave ovens to a certain extent, there are still certain problems.

[0009] In summary, although the existing intelligent microwave oven heating solutions have solved the problem of setting the appropriate heating time to a certain extent, they still have certain limitations and shortcomings.

[0010] The Chinese invention patent application with publication number CN116518429A discloses a microwave oven control method, device, equipment and storage medium, which includes: discovering a microwave oven, a camera and a user terminal equipped with the Hongmeng system in the same local area network; reading microwave oven data, obtaining current food information in the microwave oven data and matching the current food information with the pre-stored food information; when the current food information matches the pre-stored food information successfully, sending the heating scheme corresponding to the pre-stored food information to the user terminal through the distributed soft bus; receiving the control instruction sent by the user terminal according to the heating scheme through the distributed soft bus, and sending the control instruction to the microwave oven; detecting whether the abnormal microwave oven event reported by the sensor is received; when the abnormal microwave oven event is received, obtaining the video of the surrounding environment of the microwave oven in the camera through the distributed soft bus, and sending the video of the surrounding environment of the microwave oven to the user terminal. The disadvantage of this method is that in the process of executing the food heating scheme, the system relies on the pre-stored food information file to match and determine the appropriate heating program. However, if the detected food type is not in the database, the system will not be able to generate a heating scheme for the specific food type, nor can it automatically add this new type of food information and the corresponding heating scheme to the database. This reflects a limitation of the current system, which is the lack of ability to generate heating plans for unrecorded food types. Summary of the invention

[0011] In order to solve the above technical problems, the present invention proposes a microwave oven intelligent heating control method and system, which receives uploaded data through communication between the server and the client, uses a deep learning network model to calculate the appropriate heating time, and optimizes the cloud training model based on the user's personalized settings and experience.

[0012] The first object of the present invention is to provide a microwave oven intelligent heating control method, comprising initializing an intelligent microwave oven server and establishing an initial food heating time calculation model, and further comprising the following steps:

[0013] Step 1: The smart microwave oven client collects image data when food is put into the microwave oven and feedback information from the pressure sensor, and uploads it to the smart microwave oven server;

[0014] Step 2: The food heating time calculation model identifies the type of food based on the image data and the feedback information through accurate image recognition technology, retrieves the corresponding heating time equation, calculates six types of reasonable food heating time based on the weight information of the food, and returns it to the smart microwave oven client;

[0015] Step 3: The display screen of the smart microwave oven client displays the food pictures, names and calculated heating time in descending order according to the heating programs of the six types of food with the highest similarity to the pictures of food put into the microwave oven by the user returned by the smart microwave oven server;

[0016] Step 4: After the heating program is completed, the smart microwave oven client collects and packages the data in this heating program and sends it to the smart microwave oven server, so that the smart microwave oven server can adjust and / or optimize the food heating time calculation model.

[0017] Preferably, the training center of the food heating time calculation model uses ResNet as a pre-training framework to build an initial database as a model training data set.

[0018] In any of the above schemes, preferably, the model training data set includes a food picture set, food specific heat capacity and food heating target temperature.

[0019] In any of the above schemes, preferably, the food heating time calculation model is used to calculate the ideal heating time of the food based on data such as food pictures and food weight.

[0020] In any of the above schemes, preferably, the food heating time calculation model adopts two contrastive learning methods for feature extraction and encoder optimization, including using the anchor point and its enhancement as positive samples for contrastive learning and using samples of the same category of the anchor point as positive examples for contrastive learning.

[0021] In any of the above schemes, preferably, the expression of the contrastive learning is:

[0022]

[0023] Among them, λ and τ are hyperparameters, f(·) is the encoder network, and x i is the input of the th sample, x′ i is the enhanced sample input of the i-th sample, |P(i)| is the number of samples in the same sample set as the i-th sample, P(i) is the same category set as the i-th sample, x p is the value of the i-th sample x i Samples of the same category, x j Divide x i Other samples except L is a labeled dataset used for pre-training, and j is the sample number.

[0024] In any of the above solutions, it is preferred that the final loss function L of the pre-trained model is pre-train The calculation formula is

[0025] L pre-train =L CL +CrossEntropy(y r ,y p )

[0026] Among them, CrossEntropy(y r ,y p ) is the cross entropy loss, y r is the true label of the input image, y p is the predicted label of the model.

[0027] In any of the above schemes, preferably, the cross entropy loss CrossEntropy(y r ,y p ) is calculated as

[0028]

[0029] Among them, H(y r ,y p ) is the cross entropy loss function, and n is the total number of input samples.

[0030] In any of the above schemes, preferably, after the model pre-training is completed, the heating time calculation center calculates the heating time of the food, and the calculation formula is:

[0031]

[0032] Among them, Φ(·) is the heating time calculation model, m is the weight of the food, T i is the initial temperature of the food, c k is the specific heat capacity of the kth type of food, T tk is the target temperature for heating the kth type of food, P is the power of the client's microwave oven, and t is the heating time.

[0033] In any of the above solutions, preferably, the data in the current heating program includes the heating program selected by the user, the customized heating time and the temperature of the food when the heating is completed.

[0034] In any of the above schemes, preferably, the optimization method of the food heating time calculation model is to use the heating time as the loss function according to the data in the current heating program to optimize the performance of the food heating time calculation model, and the formula is:

[0035] L optimize =|t p -t r |+CrossEntropy(y r ,y p )+max(Csafe-C,0)

[0036] Among them, L optimize is the loss function, t p The default heating time set for the server, t r is the actual heating time fed back by the client, Csafe is the safe smoke concentration value, and C is the smoke concentration value uploaded by the user when the heating ends.

[0037] The second object of the present invention is to provide a microwave oven intelligent heating control system, comprising an intelligent microwave oven server and an intelligent microwave oven client.

[0038] The intelligent microwave oven server includes a data collection center, a model training center and a heating time calculation center.

[0039] The data collection center is used to receive and store data sent by the client in real time.

[0040] The model training center is responsible for deep learning and feature extraction of the collected data, and training and optimizing the food heating time calculation model.

[0041] The heating time calculation center is used to calculate the ideal heating time of the food and transmit the time to the smart microwave oven client.

[0042] The system uses the method described in the first purpose to perform intelligent heating control on the microwave oven.

[0043] Preferably, the intelligent microwave oven client includes a temperature sensor, a pressure sensor, a smoke sensor, a camera module, a network communication module and a touch screen.

[0044] In any of the above schemes, it is preferred that each time the smart microwave oven client is started, image data and pressure sensor data are collected and uploaded to the smart microwave oven server. The smart microwave oven server uses the weight information obtained by converting the image data recognition result and the pressure sensor data as the main input parameter, and inputs it into the trained food heating time calculation model for data processing.

[0045] The present invention proposes an intelligent heating control method for a microwave oven, which ensures that food is not only heated to a suitable temperature but also avoids the risks of scorching and burning through technical means, thereby ensuring public safety and improving the dining experience of patients and their families. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The present invention is a flowchart of a preferred embodiment of the intelligent heating control method for a microwave oven according to the present invention.

[0047] Figure 2 It is a module diagram of a preferred embodiment of the intelligent heating control system of a microwave oven according to the present invention.

[0048] Figure 3 It is a schematic diagram of comparative learning effect according to a preferred embodiment of the intelligent heating control method for a microwave oven of the present invention.

[0049] Figure 4 The figure is a schematic diagram of the pre-trained model structure according to a preferred embodiment of the microwave oven intelligent heating control method of the present invention.

[0050] Figure 5 A schematic diagram of the composition of an intelligent microwave oven client system according to a preferred embodiment of the intelligent heating control method for a microwave oven of the present invention.

[0051] Figure 6 The present invention is a flowchart of an intelligent microwave oven client according to a preferred embodiment of the intelligent heating control method of a microwave oven of the present invention.

[0052] Figure 7 The figure is a schematic diagram of the model structure during tuning according to a preferred embodiment of the intelligent heating control method for a microwave oven of the present invention.

[0053] Figure 8 The figure is a schematic diagram of the composition of the intelligent microwave oven server-side module according to a preferred embodiment of the microwave oven intelligent heating control method of the present invention.

[0054] Fig. 9A system schematic diagram of a preferred embodiment of the intelligent heating control system for a microwave oven according to the present invention. DETAILED DESCRIPTION

[0055] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0056] Embodiment 1

[0057] like Figure 1 As shown, a microwave oven intelligent heating control method executes step 100, initializes the intelligent microwave oven server and establishes an initial food heating time calculation model, the training center of the food heating time calculation model uses ResNet as a pre-training skeleton, and builds an initial database as a model training data set, and the model training data set includes a food picture set, food specific heat capacity and food heating target temperature.

[0058] The food heating time calculation model is used to calculate the ideal heating time of food based on food images, food weight and other data. Two contrastive learning methods are used for feature extraction and encoder optimization, including using the anchor point and its enhancement as positive samples for contrastive learning and using samples of the same category of the anchor point as positive examples for contrastive learning.

[0059] The expression of contrastive learning is:

[0060]

[0061] Among them, λ and τ are hyperparameters, f(·) is the encoder network, and x i is the input of the th sample, x′ i is the enhanced sample input of the i-th sample, |P(i)| is the number of samples in the same sample set as the i-th sample, P(i) is the same category set as the i-th sample, x p is the value of the i-th sample x i Samples of the same category, x j Divide x i Other samples except L is a labeled dataset used for pre-training, and j is the sample number.

[0062] The final loss function L of the pre-trained model pre-train The calculation formula is

[0063] L pre-train =L CL +CrossEntropy(y r ,y p )

[0064] Among them, CrossEntropy(y r ,yp ) is the cross entropy loss, y r is the true label of the input image, y p is the predicted label of the model.

[0065] The cross entropy loss CrossEntropy(y r ,y p ) is calculated as

[0066]

[0067] Among them, H(y r ,y p ) is the cross entropy loss function, and n is the total number of input samples.

[0068] Execute step 110, the smart microwave oven client collects image data when food is put into the microwave oven and feedback information from the pressure sensor, and uploads it to the smart microwave oven server;

[0069] Execute step 120, the food heating time calculation model identifies the type of food based on the image data and the feedback information through precise image recognition technology, retrieves the corresponding heating time equation, calculates six types of reasonable food heating times based on the weight information of the food, and returns the calculated heating times to the smart microwave oven client.

[0070] The heating time calculation center calculates the heating time of food using the following formula:

[0071]

[0072] Among them, Φ(·) is the heating time calculation model, m is the weight of the food, T i is the initial temperature of the food, c k is the specific heat capacity of the kth type of food, T tk is the target temperature for heating the kth type of food, P is the power of the client's microwave oven, and t is the heating time.

[0073] Executing step 130, the display screen of the smart microwave oven client displays the food pictures, names and calculated heating times in descending order of similarity according to the heating programs of the six foods with the highest similarity to the pictures of the food put into the microwave oven by the user returned by the smart microwave oven server;

[0074] Execute step 140. After the heating program ends, the smart microwave oven client collects and packages the data in this heating program and sends it to the smart microwave oven server, so that the smart microwave oven server can adjust and / or optimize the food heating time calculation model. The data in this heating program includes the heating program selected by the user, the customized heating time, and the temperature of the food at the end of heating.

[0075] The optimization method of the food heating time calculation model is to use the heating time as the loss function according to the data in the current heating program to optimize the performance of the food heating time calculation model. The formula is:

[0076] L optimize =|t p -t r |+CrossEntropy(y r ,y p )+max(Csafe-C,0)

[0077] Among them, L optimize is the loss function, t p The default heating time set for the server, t r is the actual heating time fed back by the client, Csafe is the safe smoke concentration value, and C is the smoke concentration value uploaded by the user when the heating ends.

[0078] Embodiment 2

[0079] like Figure 2 As shown, a microwave oven intelligent heating control system includes an intelligent microwave oven server 200 and an intelligent microwave oven client 210.

[0080] The intelligent microwave oven server 200 includes a data collection center 201, a model training center 202, and a heating time calculation center 203.

[0081] The data collection center 210 receives and stores the data sent by the client in real time.

[0082] The model training center 202 is responsible for deep learning and feature extraction of the collected data, and training and optimizing the food heating time calculation model.

[0083] The heating time calculation center 203 is used to calculate the ideal heating time of the food and transmit the time to the smart microwave oven client 210.

[0084] The smart microwave oven client 210 includes a temperature sensor 211 , a pressure sensor 212 , a smoke sensor 213 , a camera module 214 , a network communication module 215 and a touch screen 216 .

[0085] Each time the smart microwave oven client 210 is started, image data and pressure sensor data are collected and uploaded to the smart microwave oven server 200. The smart microwave oven server 200 uses the image data recognition result and the weight information converted from the pressure sensor data as the main input parameter, and inputs it into the trained food heating time calculation model for data processing.

[0086] Embodiment 3

[0087] The present invention aims to provide a safe and convenient microwave oven use solution for densely populated and complex places such as hospitals. Even users who know nothing about the operation of a microwave oven can easily enjoy excellent heating services through the one-touch heating function. At the same time, the present invention also provides a customized heating function for users who are familiar with the characteristics of ingredients and the working principle of microwave ovens. The server can learn the user's customized settings so as to provide the user with a heating time that better meets his or her expectations next time.

[0088] As a household appliance with a relatively single function, a microwave oven needs the cooperation of multiple sensors and other modules to provide the foundation of intelligence in order to enable it to have intelligent functions. In order to realize the intelligent microwave oven, the present invention adopts temperature sensors, pressure sensors, smoke sensors, camera modules, and network communication modules as the information basis of intelligent equipment. In addition, a server is provided to exchange data with a microwave oven client through a communication module, so as to intelligently issue a reasonable heating plan according to the user's customized needs and the characteristics of the food, which not only greatly improves the intelligence level of the microwave oven, but also greatly expands its functionality and application range, bringing users a more convenient, safe and efficient cooking experience.

[0089] The server-side system is designed with a highly specialized modular architecture to achieve precise control of the microwave heating process. The main modules of the system are as follows: Figure 1As shown in Figure 1, it consists of three parts: data collection center, model training center, and heating time calculation center. The data collection center is the entry point of the system, responsible for receiving and storing the data stream sent by the client in real time after the client device is started. These data are the cornerstone of continuous model training and tuning, ensuring that the system can adapt to changing usage scenarios and user needs. The model training center is the core of the system, responsible for deep learning and feature extraction of the collected data to train and optimize the model. The model is expected to be able to perform high-precision analysis of image data transmitted through the client and identify food types. In addition, the model will combine the food weight information collected from the pressure sensor and calculate the accurate heating time through complex calculation logic. The heating time calculation center is the core component of the continuous operation of the server. Every time the client microwave oven is started, image data and pressure sensor data are collected and uploaded to the server. The server uses the weight information obtained by converting the image data recognition result and the pressure sensor data as the main input parameter and inputs it into the trained model. After processing, the model calculates the ideal heating time of the food and transmits the time to the client to guide the microwave oven to perform the most appropriate heating program.

[0090] 1. Initialization of the smart microwave oven server

[0091] During the initialization process of the smart microwave oven server, in order to ensure the accuracy and reliability of the data collection center, the server will perform a one-time data cleaning operation to remove possible noise data, thereby avoiding interference in subsequent model training and decision-making processes.

[0092] The model training center uses ResNet as the pre-training framework and builds an initial database as the model training data set, including food image sets (ChineseFoodNet can be used), food specific heat capacity, and food heating target temperature. The structure of this database is shown in Table 1.

[0093] Table 1 Database structure

[0094] Field Type Null Key COMMENT FoodID INT No PRI Food No. Name VARCHAR(255) No Food Name SpecificHeat FLOAT No Specific heat capacity of food TargetTemperature FLOAT No Heating target temperature Image URLs TEXT No Food Photo Gallery

[0095] Contrastive learning is a very powerful feature extraction tool that can effectively improve the accuracy of pre-training for classification tasks. In this paper, two contrastive learning methods are selected for feature extraction and encoder optimization. When performing contrastive learning, not only the anchor point and its enhancement are used as positive samples for contrastive learning, that is, L CL The first item of the loss, and also regards the samples of the same category of the anchor point as positive examples for comparative learning, that is, L CL The second term of the loss, for the first term, the negative example pairs are all samples except the anchor sample itself, and for the second term, the negative example pairs are all labeled samples except the anchor sample itself. Figure 3 As shown, in the feature space, after two contrastive learning, the anchor point sample ( Figure 3 The upper left side) is not only similar to its enhanced sample ( Figure 3 The distance between the left middle and the same sample ( Figure 3 The lower left side is close enough to the sample of the same category, and far enough from the samples of other categories. This enables the model to accurately distinguish similar foods.

[0096] Formula (1) is the expression of contrastive learning in the pre-training process,

[0097]

[0098] Wherein, f(·) is the encoder network, which refers to the ResNet18 encoder in this embodiment, and extracts the input as; x i is the input of the i-th sample, x′ i is the data-augmented input of the i-th image; |P(i)| is the number of samples in the same sample set of the i-th sample; x p is the sample belonging to P(i), that is, the sample x i Samples of the same category; x j Divide x i Other samples outside; ∑(·) is the summation function; exp(·) is the exponential function, exp(x) = e x ;

[0099] log(·) logarithmic function; λ is a hyperparameter, which is used as a weight to control the strength of the two contrastive learnings and is set to 0.5; τ is the temperature parameter in the hyperparameter, which is set to 0.01 to make the optimization process smoother; x i ′ is the data augmented input of the i-th image, B is the dataset used for pre-training, and B L It is a labeled dataset used for pre-training, P(i) is the set of samples of the same category as the i-th sample, that is, the anchor set of the anchor image.

[0100] L pre-train =L CL +CrossEntropy (y r ,y p ) (2)

[0101] Formula (2) is the final loss function of the pre-trained model. r With y p They are respectively for the input image, the true label and the predicted label of the model.

[0102] CrossEntropy(y r ,y p) is y r With y p The cross entropy of is shown in formula (3).

[0103]

[0104] like Figure 4 As shown in the figure, during the pre-training process, the image tensor is used as input into the encoder (ResNet18) to obtain a set of embeddings, and contrastive learning is performed on the embeddings. The trained embeddings enter the softmax layer and the linear classification layer for prediction and classification. In the process of reducing the contrast loss and the prediction loss, the parameters of the encoder are optimized to obtain a better encoder that can predict the food type according to the food type, so as to perform subsequent heating time calculation.

[0105] After the model pre-training is completed, the heating time calculation center calculates the heating time of the food using formula (4):

[0106]

[0107] Among them, Φ(·) is the heating time calculation model, t is the heating time (unit: second), c k is the specific heat capacity of the kth type of food (unit: joule / kg·degrees Celsius), m is the weight of the food (unit: kilogram), T tk The target temperature for heating the kth food (unit: Celsius), T i is the initial temperature of the food (unit: Celsius), and P is the power of the client's microwave oven (unit: Watt).

[0108] Get the user-uploaded picture x, identify the food category through the model, search the database for the corresponding food specific heat capacity and food heating target temperature through the food category label, combine the food weight m uploaded by the user and the current initial temperature T i , and finally the heating time is calculated by formula (1).

[0109] The heating time calculation center will use the trained and optimized model to process and calculate the data uploaded by the client, obtain the ideal heating time for the food, and send this result back to the microwave oven client so that the client device can execute the corresponding heating program based on this information.

[0110] 2. Smart Microwave Oven Client Initialization

[0111] During the startup process, the smart microwave client will execute initialization procedures on multiple hardware modules to ensure the normal operation and safety of the system. Figure 5As shown, the hardware module includes a temperature sensor, a pressure sensor, a smoke sensor, a camera module, a network communication module and a touch screen.

[0112] When the microwave oven client is started, each hardware module needs to be initialized according to its operating manual. Each sensor needs to be optimized for the acquisition frequency. The temperature sensor and the smoke sensor need to ensure a relatively high frequency of acquisition during operation so that the power-off protection program can take effect in real time when a dangerous situation occurs. The pressure sensor needs to ensure a low acquisition frequency. After the heated food is placed in the heating space, its pressure data is collected multiple times to obtain the average value to ensure the robustness of the system. The camera module needs to ensure the quality of the pictures taken, and collect picture data multiple times in a short period of time to ensure the accuracy of the model's food type recognition. The software configuration of the network communication module needs to be customized according to the conditions of the use scenario. For example, in a hospital environment, the security of the hospital intranet must be guaranteed first, followed by the stability of data transmission.

[0113] The main responsibility of the temperature sensor and smoke sensor is to monitor the internal environment of the microwave oven in real time. They ensure that the power-off protection program can be quickly triggered when the temperature or smoke concentration exceeds the predetermined threshold, thereby preventing potential safety risks. In addition, the temperature sensor is also responsible for collecting ambient temperature data at the end of heating. This data will be sent to the data collection center on the server side along with other information as feedback on whether the heating time calculated by the server has heated the food to a reasonable temperature, which will be used for subsequent optimization of the model and improvement of user experience.

[0114] Smoke sensors are similar to temperature sensors and are deployed inside microwave ovens to monitor the concentration of smoke particles in the microwave oven heating space in real time. When the smoke concentration inside the microwave oven exceeds a certain threshold, power-off protection is activated to avoid safety hazards caused by overheating of food and to ensure the taste and freshness of the food to a certain extent.

[0115] The pressure sensor is located in the lower layer of the microwave oven's heating space. When food is placed in the oven and the oven door is closed, it measures the pressure change generated by the food. By comparing it with the reading when the oven is empty, the pressure sensor can calculate the weight of the food and upload this data to the server. The server calculates the heating time based on this important parameter.

[0116] The camera module is deployed on the front upper side of the microwave client and is responsible for capturing images of the food to be heated. These image data are uploaded to the server for food type recognition. The server calculates the appropriate heating time based on the recognition results and food weight, and decides whether to store the image data for subsequent model tuning as needed.

[0117] The touch screen is located on the front right side of the microwave oven client and is always on standby after the microwave oven is powered on. After the microwave oven starts working, the screen will display the corresponding operation interface according to the system status for the user to operate.

[0118] The network communication module is usually deployed at the lower back of the client. There are two implementation schemes for the network communication module. One is to use wireless communication, combined with methods such as esp8266 or wireless network cards to connect to the network and then communicate with the server; the other is to use the RJ45 network port for wired network connection. For example, for use scenarios such as hospitals, the former solution uses wireless networks in crowded scenes such as hospitals, which are easily interfered by other wireless devices, easily causing data packet loss, etc., affecting the user experience, so it is more recommended to communicate by wired networking.

[0119] 3. Smart Microwave Operation

[0120] After the server completes the initialization deployment, it will open the port for communication. After the client completes the initialization and adjustment of each hardware module, it enters the standby state and waits for the user's operation. The flowchart of the user side is as follows Figure 6 As shown. When the user uses the microwave oven, the network communication module will establish a communication connection with the server. Subsequently, the camera module is responsible for taking pictures of the food, and the pressure sensor starts to weigh the food placed in the microwave. After the weighing is completed, the weighing data and the food image data are sent to the server through the established communication connection. After receiving the data, the server will process the weighing data and identify the image data. During the identification process, the six food labels with the highest matching degree will be selected. Combined with the weight data processed in the previous article, these labels and data will be input into the food heating time model to predict the exact time required for this heating when the six food labels and weight data are specified.

[0121] After calculating the appropriate heating time, the server will pass the data down through the established communication channel. The data received by the microwave client will be displayed on the display screen, which will include six food images and their corresponding heating times, and a custom button will also be provided for the user. The food images, names and calculated heating times will be arranged from top to bottom according to similarity, and the custom button will be at the end of the list. While displaying the heating strategy set by the server, in order to improve the user experience, the current page will automatically start the countdown function and prompt the user within 15 seconds. If the user does not operate within 15 seconds, the system will default to selecting the heating program for the food type with the highest food identification match, that is, the food label of the first item in the list on the screen and its corresponding heating time, and start the heating process. This design is intended to provide users with a barrier-free experience. Users only need to put the food in the microwave and close the door, then leave and return to take the food after the heating countdown ends. If the user selects any food label in the list and its corresponding heating time information within the 15-second countdown, the system will immediately start the heating plan selected by the user. This process ensures the efficiency and convenience of user operation, making the use of the microwave oven simple and intuitive.

[0122] If the user clicks the custom button on the current page, it means that the existing six heating plans do not meet the user's needs. The system will guide the user to the custom heating plan page. On this page, the user needs to select the food label of the food to be heated and set the heating time by himself. The labels provided are coarse-grained classifications, such as pasta, meat, vegetables, etc., so that users can easily choose. After the settings are completed, the system will start the heating program according to the user-defined heating plan.

[0123] After the heating program is started, the user will be redirected to the heating page, which will display the countdown for the remaining heating time. At the same time, the smoke sensor and temperature sensor will start working according to the preset sampling frequency, continuously monitoring whether the smoke concentration and temperature exceed the set threshold. Once the threshold is detected to be exceeded, the system will start the power-off protection measure to prevent the occurrence of safety accidents. If the power-off protection is not triggered until the end of the countdown during the heating process, it means that the heating has been completed. At this time, the system will automatically collect the ambient smoke concentration readings, the heating scheme selected by the user, and the custom heating parameters, and package and send these data to the data collection center on the server for subsequent processing and analysis.

[0124] 4. Server operation and optimization

[0125] The model tuning on the server side adopts an incremental learning method, that is, each time the tuning training is performed, not only the newly collected user feedback data is used, but also some previous user feedback data stored in the database are randomly extracted and mixed in a certain proportion to form a data set for this tuning training. When the model tuning training is performed on such a data set, the model can not only make accurate predictions when new food types appear, but also retain and remember the previously learned knowledge for a long time, and will not forget any food types that have been learned. This significantly improves the scalability and generalization performance of the model. Since it is impossible to include all food types in daily life in the data set in the pre-training stage, the incremental learning method can ensure that the intelligent microwave oven of the present invention can continue to adapt to the needs of users and increasingly conform to the average usage habits of users.

[0126] The user feedback dataset required for model optimization is composed of user feedback and related heating data collected by the server during the daily use of the microwave oven client. It includes:

[0127] For example, if the user does not make any selection, the heating time recommended by the system is accepted by default, indicating that the time calculated by the model is reasonable and therefore does not need to be adjusted. If the user selects another heating program in the list, this indicates that the user is not satisfied with the default heating time and thinks that the other heating program selected is more reasonable. If the user selects a custom heating time, this means that none of the six heating programs calculated by the server meets the user's expectations. In this case, the server regards the user-defined heating time as a reasonable heating time. In addition, the food smoke concentration data at the end of each microwave heating is also used to evaluate whether the heating time setting heats the food to the appropriate temperature. The server will adjust the model parameters based on the user's selection and the data collected at the end of microwave heating, so as to provide users with more accurate heating time recommendations in future calculation recommendations.

[0128] According to the above tuning strategy, we use the equation in (5) as the loss function in the model tuning training process, based on the collected user feedback data, and use Figure 7 The model structure shown continuously optimizes model performance.

[0129] L optimize =|t p -t r |+CrossEntropy (y r ,y p )+max(Csafe-C,0) (5)

[0130] Among them, CrossEntropy(y r ,y p ) is the cross entropy loss function, t pThe default heating time set for the server (in seconds), t r is the actual heating time reported by the client (in seconds), r The real label of the picture uploaded by the user, y p is the label predicted by the server model based on the user-uploaded image, Csafe is the safe smoke concentration value (unit: milligrams per cubic meter), and C is the smoke concentration value at the end of heating uploaded by the user (unit: milligrams per cubic meter).

[0131] During the tuning process, where: c k is the specific heat capacity T of type k food tk : The target heating temperature for type k food. We evaluate the rationality of the heating time setting by comparing the difference between the actual heating time and the default heating time set by the server. If the difference is too large, it indicates that the heating time may not be set appropriately. Furthermore, we determine whether the heating time is appropriate by monitoring whether the smoke concentration exceeds the safety threshold after the food is heated. If the smoke concentration exceeds the safety value, it means that the heating time needs to be adjusted accordingly; conversely, if it is lower than the safety value, the difference is set to zero, and the system no longer adjusts the heating time according to the smoke concentration.

[0132] In order to make the predicted food types closer to the actual situation, our model minimizes the cross entropy loss function CrossEntropy(·) so that y r With y p Closer. The learning goal of the model is to minimize the loss function L through the back propagation algorithm. optimize The value of the ResNet structure parameters and the target temperature T of this type of food are updated tk , in order to achieve more reasonable heating time for different foods. This process aims to improve the prediction accuracy of the model and ensure that the heating process is both safe and efficient.

[0133] The data involved above is stored in the data collection center on the server side, which not only facilitates incremental learning and model tuning training of the model, but also ensures the continuous optimization and upgrading of the model accuracy. Figure 8 shown.

[0134] The server side consists of three modules: data collection center, model training center, and heating time calculation center. After the data collection center is enabled on the client side, it receives and stores the data sent by the client. The model training center conducts incremental learning of the model and continuously optimizes the model. The heating time calculation center uses the model to calculate the time required for each food heating.

[0135] like Figure 5As shown in the figure, the microwave oven client includes multiple modules: temperature sensor, pressure sensor, smoke sensor, camera module, network communication module, and touch screen. The temperature sensor, pressure sensor, smoke sensor, and camera module are used to collect data, and the network communication module uploads the data to the server for processing and calculation, and the results are sent back to the screen through the communication channel established by the network communication module for display.

[0136] The server establishes a connection with the client's communication module, continuously receives user data, and formulates a reasonable heating strategy and sends it back to the client. At the same time, the client also receives feedback data from the user when the heating is finished, which is used to continuously optimize the performance of the model, making the heating time set for each food gradually more reasonable. The overall schematic diagram of the microwave oven intelligent heating control system based on cloud collaboration and deep learning is shown in the figure. Fig. 9 shown.

[0137] Embodiment 4 The technical solution of this embodiment is implemented as follows:

[0138] Step 1: Initialize the smart microwave oven server

[0139] An initial food heating time calculation model is established on the server side. The model has been trained using the self-built initial data set as the benchmark for the entire task. The model will be responsible for calculating the ideal heating time of food based on data such as food images and food weight.

[0140] Step 2: The client is responsible for collecting image data of food when it is placed in the microwave and feedback information from the pressure sensor, and uploading this data to the cloud server. The server-side model will use this data to identify the type of food through precise image recognition technology and retrieve the corresponding heating time equation. Based on the weight information of the food, the model will calculate six reasonable food heating times and return them to the client.

[0141] Step 3: The server returns six heating programs for foods that are most similar to the pictures of the food that the user puts in the microwave oven, and arranges the food pictures, names, and calculated heating times from top to bottom on the display screen according to the similarity. The user does not need to perform any operation, and the microwave oven automatically executes the heating program corresponding to the food category with the highest matching degree calculated by the server. Of course, the user can also customize the heating program. If the user feels that the first heating program is unreasonable, he can also choose the following five options. If the customer is not satisfied with the above six heating programs, he can click the last one: custom heating. That is, the user can completely customize the heating time of the program. After the user completes the settings, the heating program starts, the safety program starts synchronously, and the temperature sensor and smoke sensor start working to prevent the food from being overheated and burned. Step 4: After the heating program ends, the client collects and packages various data, including the heating program selected by the user, the customized heating time, and the temperature of the food at the end of heating. These data will be sent to the server to feedback the effect of the heating program provided this time, so that the server can adjust and optimize the model.

[0142] Step 5: The server regularly collects and parses data packets fed back by the user. User operation feedback is crucial to the adjustment of the model. If the user does not make any selection, it is deemed that the system-recommended heating time is accepted by default, which means that the time calculated this time is more reasonable; if the user selects other heating programs in the list, it means that the user is not satisfied with the system default heating time, but thinks that the selected heating program is more reasonable; if the user selects a custom heating time, it means that the six heating programs calculated by the server do not meet the user's expectations. At this time, the server regards the user-defined heating time as a reasonable time for this heating. The server will optimize the model parameters according to the user's choice through the back propagation algorithm during model tuning. In order to provide users with more appropriate heating time recommendations in future calculation recommendations.

[0143] In order to better understand the present invention, the above is described in detail in conjunction with the specific embodiments of the present invention, but it is not intended to limit the present invention. Any simple modifications made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention. Each embodiment in this specification focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referenced to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A microwave oven intelligent heating control method, comprising initializing an intelligent microwave oven server and establishing an initial food heating time calculation model, characterized in that: The following steps are also included: Step 1: The smart microwave oven client collects image data when food is put into the microwave oven and feedback information from the pressure sensor, and uploads it to the smart microwave oven server; Step 2: The food heating time calculation model identifies the type of food according to the image data and the feedback information, retrieves the corresponding heating time equation, calculates six types of reasonable food heating times based on the weight information of the food, and returns the calculation to the smart microwave oven client; Step 3: The display screen of the smart microwave oven client displays the food pictures, names and calculated heating time in descending order according to the heating programs of the six types of food with the highest similarity to the pictures of food put into the microwave oven by the user returned by the smart microwave oven server; Step 4: After the heating program is completed, the smart microwave oven client collects and packages the data in this heating program and sends it to the smart microwave oven server, so that the smart microwave oven server can adjust and / or optimize the food heating time calculation model.

2. The intelligent heating control method for a microwave oven according to claim 1, characterized in that: The training center of the food heating time calculation model uses ResNet as a pre-training framework and builds an initial database as a model training data set.

3. The intelligent heating control method for a microwave oven according to claim 2, characterized in that: The food heating time calculation model is used to calculate the ideal heating time of food based on data such as food pictures and food weight.

4. The intelligent heating control method for a microwave oven according to claim 3, characterized in that: The food heating time calculation model adopts two contrastive learning methods for feature extraction and encoder optimization, including using an anchor point and its enhancement as positive samples for contrastive learning and using samples of the same category of the anchor point as positive examples for contrastive learning.

5. The intelligent heating control method for a microwave oven according to claim 4, characterized in that: The expression of contrastive learning is: Among them, λ and τ are hyperparameters, f(·) is the encoder network, and x i is the input of the th sample, x i ′ is the enhanced sample input of the i-th sample, |P(i)| is the number of samples in the same sample set of the i-th sample, P(i) is the same category set of the i-th sample, x p is the value of the i-th sample x i Samples of the same category, x j Divide x i Other samples except L is a labeled dataset used for pre-training, and j is the sample number.

6. The intelligent heating control method for a microwave oven according to claim 5, characterized in that: The final loss function L of the pre-trained model pre-train The calculation formula is L pre-train =L CL +CrossEntropy(y r ,y p ) Among them, CrossEntropy(y r ,y p ) is the cross entropy loss, y r is the true label of the input image, y p is the predicted label of the model.

7. The intelligent heating control method for a microwave oven according to claim 6, characterized in that: The cross entropy loss CrossEntropy(y r ,y p ) is calculated as Among them, H(y r ,y p ) is the cross entropy loss function, and n is the total number of input samples.

8. The intelligent heating control method for a microwave oven according to claim 7, characterized in that: After the model pre-training is completed, the heating time calculation center calculates the heating time of the food. The calculation formula is Among them, Φ(·) is the heating time calculation model, m is the weight of the food, T i is the initial temperature of the food, c k is the specific heat capacity of the kth type of food, T tk is the target temperature for heating the kth type of food, P is the power of the client's microwave oven, and t is the heating time.

9. The intelligent heating control method for a microwave oven according to claim 8, characterized in that: The optimization method of the food heating time calculation model is to use the heating time as the loss function according to the data in the current heating program to optimize the performance of the food heating time calculation model. The formula is: L optimize =|t p -t r |+CrossEntropy(y r ,y p )+max(Csafe-C,0) Among them, L optimize is the loss function, t p The default heating time set for the server, t r is the actual heating time fed back by the client, Csafe is the safe smoke concentration value, and C is the smoke concentration value uploaded by the user when the heating ends.

10. A microwave oven intelligent heating control system, comprising an intelligent microwave oven server and an intelligent microwave oven client, characterized in that: The intelligent microwave oven server includes a data collection center, a model training center and a heating time calculation center. The data collection center is used to receive and store data sent by the client in real time. The model training center is responsible for deep learning and feature extraction of the collected data, and training and optimizing the food heating time calculation model. The heating time calculation center is used to calculate the ideal heating time of the food and transmit the time to the smart microwave oven client. The system uses the method as claimed in claim 1 to perform intelligent heating control on the microwave oven.

Citation Information

Cited By

  • AI rice heating and cooking auxiliary solution

    CN119888721A