Air fryer capable of improving user experience

Abstract

The invention relates to an air fryer capable of improving user experience, which comprises a machine body, a main control board arranged in the machine body and a cooking cavity used for containing food, and is characterized in that a camera assembly is further arranged in the machine body, is electrically connected with the main control board and is used for shooting images of the food in the cooking cavity at intervals, and the main control board is electrically connected with the camera assembly. The main control board obtains a shot food image and uploads the food image to the cloud server, and the cloud server generates a video file based on the food image to display key change points of the cooking process. According to the invention, the food type and total amount can be automatically identified, the cooking process of the air fryer can be intelligently controlled, the short video capable of displaying the key change point of the cooking process can be automatically generated according to the user demand, and the user can edit and share the automatically generated short video at the terminal equipment, so that the user experience is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent air fryers, and in particular to an air fryer capable of improving user experience. Background Art

[0002] Existing air fryers can basically only be controlled by the knob or touch screen on the air fryer. The cooking status of food during the operation of the air fryer can only be observed by pulling out the frying basket, or through the visual window on the frying basket or the body. The cooking status of food cannot be checked through a mobile terminal, cooking strategies cannot be automatically generated, and users cannot be allowed to participate in the food cooking process. The degree of intelligence is low and the user experience is poor. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an air fryer that improves user experience.

[0004] The technical solution adopted by the present invention to solve its technical problem is: to provide an air fryer for improving user experience, including a body, a main control board and a cooking cavity for holding food, characterized in that a camera assembly is also provided in the body, the camera assembly is electrically connected to the main control board, and is used to take images of food in the cooking cavity at intervals, the main control board obtains the taken food images and uploads them to a cloud server, and the cloud server generates a video file based on the food images to show key changing points of the cooking process.

[0005] Furthermore, the generating of a video file based on the food image to show key changing points of the cooking process includes:

[0006] Preprocessing the food image;

[0007] Using a classification model based on a convolutional neural network to determine the maturity state of the food in the food image, and obtaining the cooking stage of the food image;

[0008] Calculating the image similarity between each food image and other food images in the cooking stage to which it belongs, and dynamically adjusting its importance score in the cooking stage according to the image similarity;

[0009] Acquire the food image with the highest importance score in each cooking stage as the best image;

[0010] All of the best images are arranged in chronological order and a video file is generated therefrom.

[0011] Furthermore, dynamically adjusting the importance scores of the food images in each cooking stage according to the image similarities includes:

[0012] Initializing the importance scores of all the food images in the current cooking stage;

[0013] Adjusting the image similarity based on the importance score to obtain a weighted similarity, and then iteratively calculating the importance score of each of the food images based on the weighted similarity;

[0014] If the number of iterations reaches a set value and / or the importance score change is less than a set threshold, the next step is executed; otherwise, the process returns to the step of adjusting the picture similarity based on the importance score to obtain a weighted similarity.

[0015] Furthermore, the method also includes calculating a picture quality score based on image clarity and color saturation, and using the picture quality score to adjust the importance score corresponding to the food image obtained in each iteration.

[0016] Furthermore, the cloud server is also used to feed back information about food maturity to the main control board according to the cooking stage of the food image, and the main control board ends the cooking process according to the received information.

[0017] Furthermore, the cloud server is also used to identify the type of food to be cooked based on the food image, and after calculating the weight of the food using a volume projection algorithm, the obtained data is sent to the main control board. The main control board sets the menu category, as well as the recommended cooking time and cooking temperature based on the type of food and the weight of the food, and displays them through a display screen arranged on the surface of the machine body.

[0018] Furthermore, the shooting time interval of the camera assembly is greater than 2s and less than 8s.

[0019] Furthermore, the cloud server is also used to send the generated video file to the mobile terminal for playback and / or forwarding and sharing.

[0020] Furthermore, the mobile terminal is also used to edit the title of the received video file, insert text, insert background music or one or more of the above.

[0021] Furthermore, the mobile terminal is connected to the camera assembly via the cloud server and the main control board in sequence, so as to view the cooking status of food in the cooking cavity in real time.

[0022] Beneficial Effects

[0023] Due to the adoption of the above technical scheme, the present invention has the following advantages and positive effects compared with the prior art: the present invention adds a camera to take images of food in the cooking cavity of the air fryer at intervals, and uploads the food images to the cloud server for processing and identification to obtain the food status, and then intelligently controls the cooking process according to the food status, and automatically generates a short video that can show the key change points of the cooking process according to user needs and sends it to the terminal device, and the user can edit, forward and share the automatically generated short video on the terminal device, which greatly improves the user experience; when generating a short video, the present invention dynamically adjusts the weight of each picture in the same cooking stage according to the picture similarity, and uses the picture weight to update its picture similarity. After multiple iterations, the final picture weight is calculated, and finally the multiple pictures with the highest weight are combined into a short video, so that the generated short video can better show the key change points in the cooking process; the present invention also automatically identifies the type and total amount of food through the cloud server, and then generates and displays the menu category, as well as the recommended cooking time and cooking temperature, thereby improving the intelligence of the air fryer. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the present invention;

[0025] Figure 2 is a flow chart of an AIS algorithm according to an embodiment of the present invention;

[0026] Figure 3 4 is a flow chart of importance score calculation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0028] A first embodiment of the present invention relates to a cooking device for improving user experience, comprising a body, a main control board being provided in the body, and a cooking cavity for holding food, wherein a camera assembly is also provided in the body, the camera assembly is electrically connected to the main control board, and is used for taking images of food in the cooking cavity at intervals, the main control board obtains the taken food images and uploads them to a cloud server, and the cloud server generates a video file based on the food images to show key changing points of the cooking process.

[0029] The cooking equipment in this embodiment includes but is not limited to air fryers, ovens, microwave ovens, and rice cookers, etc. Taking the air fryer as an example, the cooking process of the air fryer is dynamically monitored in this embodiment, and images are collected in real time and uploaded to the cloud server for image processing. The video processing specifically involves the detection of food color, food volume, food type, and short video generation.

[0030] like Figure 1 As shown, after the cooking device is started, it begins to collect image information of the cooking process through the camera. The collected images are uploaded to the cloud server, and then analyzed after preprocessing. Images that can show the key changing points of the cooking process are extracted to generate short videos.

[0031] The specific analysis process is as follows Figure 2 As shown, including:

[0032] Build a classification model to determine the maturity of food in the collected food images and obtain the cooking stage of the food images;

[0033] Calculate the image similarity of each food image with other food images in the cooking stage to which it belongs, and dynamically adjust its importance score in the cooking stage according to the image similarity;

[0034] Obtain the food image with the highest importance score in each cooking stage as the best image;

[0035] All the best images are arranged in chronological order and a video file is created from them.

[0036] The following is a further explanation of the above analysis process using a classification model based on a convolutional neural network (CNN) as an example. In this embodiment, an adaptive importance scoring (AIS) method is introduced to dynamically adjust the importance scores of representative images at each stage, thereby improving the quality and fluency of the final synthesized video. The specific steps include:

[0037] Step 1: Image Preprocessing

[0038] Use tools such as OpenCV to perform basic preprocessing steps on all input images, including resizing and color space conversion, in preparation for subsequent analysis.

[0039] Step 2: CNN model training

[0040] Pre-train a CNN model that is specifically designed to identify specific types of food and their cooking states. For example, for roasted chicken legs, a large number of labeled sample data sets can be used to train the model so that it can learn to determine whether the chicken legs are frozen, thawed, raw, medium-rare, rare, medium-well, etc. based on their appearance features.

[0041] Step 3: Application of Adaptive Importance Scoring (AIS) algorithm

[0042] For each image correctly classified by CNN as a specific cooking stage, the AIS algorithm calculates its importance score relative to other candidate images of the same stage. This score is based on a combination of factors, such as the degree of difference between other images or image clarity and color saturation. By comparing the importance scores of all candidate images in the same stage, the one with the highest score is selected as the best image representing that stage.

[0043] Step 4: Short video synthesis

[0044] Once the best images at each stage are determined, the next step is to connect them in chronological order to form a continuously played video clip; in this application, in order to ensure the coherence and fluency of the image-synthesized video clip, the FLIM frame interpolation model or the image-synthesized video algorithm based on OpenCV can be used in the process of generating the video. In addition, users can further add personalized elements to the formed video clips through the APP, such as titles, text descriptions or background music.

[0045] like Figure 3 As shown in Figure 3, the core idea of ​​the AIS algorithm in step 3 is to dynamically adjust the relative importance of each image in its category. First, the importance scores of all images are initialized, and then these values ​​are continuously updated in the iterative process until convergence. This method provides a novel and effective means to solve the problem of selecting the most suitable short video from a large number of candidate images, while maintaining a high degree of flexibility and can be adjusted according to different types of food and application scenarios.

[0046] The specific implementation of the Adaptive Importance Scoring (AIS) algorithm involves several key steps, including initialization, similarity calculation, weight update, and iterative optimization. The following are several specific implementations of the AIS algorithm.

[0047] Embodiment 1:

[0048] ① Initialization

[0049] Input: A set of food images taken in chronological order I = {I1, I2, ..., I n}.

[0050] Output: The importance score of each image s = {s1,s2,...,s n}, initially all scores are set to the same value, for example, s i =1 / n.

[0051] ②Similarity calculation

[0052] In order to calculate the similarity between any two images, we can use the feature vector distance technique, that is, use a pre-trained convolutional neural network to extract the feature vector of each image, and then calculate the Euclidean distance or cosine similarity between the two images as the similarity score. Then for each pair of images (I i ,I j ), we get a similarity score similarity(i,j).

[0053] ③Weight update

[0054] Based on the formula mentioned above, we can define the weight update rule. In each iteration, the importance score of each image is adjusted according to its average similarity with other images. The specific formula is as follows:

[0055] The value is set to a value close to 1, with a value of 0.9<α<1, such as 0.95, to ensure that the score changes will not be too drastic. Similarity(i,j) represents the similarity between images i and j, and weighted_similarity(i,j) represents weighted similarity, which is combined with the current importance score of the jth image in each iteration, that is:

[0056]

[0057] ④Iterative optimization

[0058] The stopping condition may be that a certain number of iterations is reached, for example, the number of iterations t in the air fryer scenario of the present invention satisfies: t≥10; or the score change is less than a certain threshold, or both stopping conditions may be judged at the same time.

[0059] Execute loop: repeat the weight update step until the stopping condition is met.

[0060] ⑤Select the best image

[0061] After the final score was determined, the image with the highest score was selected from each cooking stage as the representative image of that stage.

[0062] Embodiment 2:

[0063] ① Initialization

[0064] Input: A set of food images taken in chronological order I = {I1, I2, ..., I n}.

[0065] Output: Initialize the importance score of each image s = {s1, s2, ..., s n}, set the initial value to the image quality score quality i , this score is calculated by taking into account image clarity and color saturation.

[0066] ②Similarity calculation

[0067] In order to calculate the similarity between any two images, we can use the feature vector distance technique, that is, use a pre-trained convolutional neural network to extract the feature vector of each image, and then calculate the Euclidean distance or cosine similarity between the two images as the similarity score. Then for each pair of images (I i ,I j ), we get a similarity score similarity(i,j).

[0068] ③Weight update

[0069] Based on the formula mentioned above, we can define the weight update rule. In each iteration, the importance score of each image is adjusted according to its average similarity with other images. The specific formula is as follows:

[0070]

[0071] The value is set to a value close to 1, with a value of 0.9<α<1, such as 0.95, to ensure that the score changes will not be too drastic. Similarity(i,j) represents the similarity between images i and j, and weighted_similarity(i,j) represents weighted similarity, which is combined with the current importance score of the jth image in each iteration, that is:

[0072]

[0073] ④Iterative optimization

[0074] The stopping condition may be that a certain number of iterations is reached, for example, the number of iterations t in the air fryer scenario of the present invention satisfies: t≥10; or the score change is less than a certain threshold, or both stopping conditions may be judged at the same time.

[0075] Execute loop: repeat the weight update step until the stopping condition is met.

[0076] ⑤Select the best image

[0077] After the final score was determined, the image with the highest score was selected from each cooking stage as the representative image of that stage.

[0078] Embodiment 3:

[0079] ① Initialization

[0080] Input: A set of food images taken in chronological order I = {I1, I2, ..., I n}.

[0081] Output: The importance score of each image s = {s1,s2,...,s n}, initially all scores are set to the same value, for example, s i =1 / n.

[0082] ②Similarity calculation

[0083] In order to calculate the similarity between any two images, we can use the feature vector distance technique, that is, use a pre-trained convolutional neural network to extract the feature vector of each image, and then calculate the Euclidean distance or cosine similarity between the two images as the similarity score. Then for each pair of images (I i ,I j ), we get a similarity score similarity(i,j).

[0084] ③Weight update

[0085] Based on the formula mentioned above, we can define the weight update rule. In each iteration, the importance score of each image is adjusted according to its average similarity with other images, and the quality score of the image is introduced. i Let's iterate together. The specific formula is as follows:

[0086]

[0087] The value is set to a value close to 1, such as 0.95, to ensure that the score changes will not be too drastic. β is the weight coefficient; quality i is the quality score of the i-th image, which takes into account the factors of image clarity and color saturation. This score is pre-calculated before each iteration; similarity(i,j) represents the similarity between images i and j, and weighted_similarity(i,j) represents the weighted similarity, which is combined with the current importance score of the j-th image in each iteration, that is:

[0088]

[0089] ④Iterative optimization

[0090] The stopping condition may be that a certain number of iterations is reached, for example, the number of iterations t in the air fryer scenario of the present invention satisfies: t≥10; or the score change is less than a certain threshold, or both stopping conditions may be judged at the same time.

[0091] Execute loop: repeat the weight update step until the stopping condition is met.

[0092] ⑤Select the best image

[0093] After the final score was determined, the image with the highest score was selected from each cooking stage as the representative image of that stage.

[0094] In the specific embodiments of the above three AIS algorithms, image clarity refers to the sharpness of edges and details in the image. Higher clarity means that the boundaries of objects in the image are clearer and the details are richer, which helps the audience better understand the image content. Specifically, an edge detection algorithm such as the Laplacian operator or the Sobel operator can be used, and then the variance of the edge pixels can be calculated as the image clarity score. The larger the variance, the clearer the image. Color saturation reflects the purity or intensity of the color. Highly saturated colors appear brighter and more vivid, while low saturation colors appear duller and bland. For the display of the cooking process, good color saturation can make the food look more appetizing. Specifically, it can be measured in the HSV (hue-saturation-value) color space, or the average value of the saturation channel in the entire image can be calculated as the color saturation score. quality i It can be obtained by directly summing the image clarity score and the color saturation score, or it can be calculated by weighted summing these two indicators to achieve more prominent one of the image clarity and color saturation factors.

[0095] A second embodiment of the present invention relates to a cooking device for improving user experience, wherein images of food in a cooking cavity are captured at intervals by a camera assembly disposed in the body, and the captured food images are processed to generate video files that can display key changing points of the cooking process.

[0096] The camera assembly in this embodiment has an AI recognition function, so the image processing process can be performed on the cloud server or locally. The camera automatically identifies the type of food placed in the pot through a large model and algorithm, calculates the weight of the food through a volume algorithm, and then automatically controls the cooking temperature and time through an algorithm with one click, eliminating the tedious manual button operation, and supports DIY to independently set the cooking temperature and time. Connecting to the server through a mobile APP allows remote operation and viewing, as well as manual operation on site. The camera and camera module can be integrated into one according to the use space, or they can be separated and connected through copper foil cables.

[0097] Taking the air fryer as an example, its control principle is: the switch power board provides power to the main control board and camera module, heating component, and motor component. The main control board can be equipped with multiple function buttons to meet the needs of manual control when it is not intelligent and automatic. The main control board unit and the camera module, high-precision NTC, motor component, heating component, overheating protection component, and mechanical protection component are controlled according to the communication protocol. The camera uses a high-pixel image recognition unit, which must meet the requirements of temperature resistance and food recognition. The camera module integrates Bluetooth and WIFI functions for device search and networking.

[0098] When the power is turned on, the machine is connected to the Internet through the mobile APP. The device can quickly connect to the APP through WIFI or WIFI hotspot and Bluetooth assistance to add devices. When food is put into the fryer, the automatic cooking function is enabled, the lighting in the fryer cavity is turned on, the camera turns on the recognition function, and through the preset large model and the food feature algorithm, one or more of the following functions are realized:

[0099] Identify food types;

[0100] Calculate the weight of the food by using a volume projection algorithm;

[0101] Identify food characteristics such as color and doneness.

[0102] According to the identified food type and weight, the menu category and cooking time and recommended temperature can be displayed on the device screen. When the cooking function is started, the device screen can alternately display the remaining cooking time and the current cooking temperature.

[0103] In some preferred embodiments, the identified food color and other features can be compared with pre-stored reference features, and if they are consistent, it is determined that the food has been cooked. At this time, the device will stop heating and enter the standby state, and the device interface will display that the food is cooked.

[0104] When the user needs to take pictures of the cooking process, he or she can enable this function through the display interface or APP. The camera will take pictures of the cooked food at 5 seconds or other intervals. The photos will be uploaded to the cloud server via the Internet for storage, and the user can view the generated photos through the APP.

[0105] If you need to generate a short video, you can enable this function through the APP interface of the smart terminal device. The mobile APP sends a command to the cloud server. The cloud server compares the cooking color, doneness and other characteristics of the food at different stages in the preset large model according to the picture sequence. The cloud server uses the advantages of high-speed computing and image processing to exclude blurry pictures or pictures that do not meet the preset characteristics, and selects pictures with high clarity that meet the preset characteristics in the middle stage of the process. The cloud server uses the frame extraction function to perform short video synthesis operations.

[0106] Users can edit short video titles, insert text or background music on the mobile APP. Users can also choose the duration of the synthesized video. The APP user sends instructions to the cloud server, and the cloud server will selectively extract frames to synthesize videos of corresponding duration based on the number of photos and video duration. The corresponding video can be downloaded to the mobile APP through the network for sharing, or saved on the cloud server for link sharing. During the cooking process, you can turn on the real-time viewing function of the camera through the smart terminal APP to view the cooking status and color of the food in the pot in real time. You can also take intermittent photos of food and upload the photos to the cloud server. On the server side, extract frames to synthesize short videos, which saves cloud storage space and network bandwidth compared to directly shooting videos with the camera and uploading them to the server and downloading them. It is low-cost and has lower requirements for camera heads and processors.

Claims

1. An air fryer that improves user experience, characterized in that: The invention comprises a body, wherein a main control board and a cooking cavity for holding food are arranged in the body, and the character is characterized in that a camera assembly is also arranged in the body, and the camera assembly is electrically connected to the main control board and is used for taking images of food in the cooking cavity at intervals, the main control board obtains the taken food images and uploads them to a cloud server, and the cloud server generates a video file based on the food images to show the key changing points of the cooking process.

2. The air fryer according to claim 1, characterized in that: The step of generating a video file based on the food image to show key changing points of the cooking process includes: Preprocessing the food image; Using a classification model based on a convolutional neural network to determine the maturity state of the food in the food image, and obtaining the cooking stage of the food image; Calculating the image similarity between each food image and other food images in the cooking stage to which it belongs, and dynamically adjusting its importance score in the cooking stage according to the image similarity; Acquire the food image with the highest importance score in each cooking stage as the best image; All of the best images are arranged in chronological order and a video file is generated therefrom.

3. The air fryer according to claim 2, characterized in that: The dynamically adjusting the importance scores of the food images in each cooking stage according to the image similarity includes: Initializing the importance scores of all the food images in the current cooking stage; Adjusting the image similarity based on the importance score to obtain a weighted similarity, and then iteratively calculating the importance score of each of the food images based on the weighted similarity; If the number of iterations reaches a set value and / or the importance score change is less than a set threshold, the next step is executed; otherwise, the process returns to the step of adjusting the picture similarity based on the importance score to obtain a weighted similarity.

4. The air fryer according to claim 3, characterized in that: The method also includes calculating a picture quality score based on image clarity and color saturation, and using the picture quality score to adjust the importance score corresponding to the food image obtained in each iteration.

5. The air fryer according to claim 1, characterized in that: The cloud server is also used to feed back information about food maturity to the main control board according to the cooking stage of the food image, and the main control board ends the cooking process according to the received information.

6. The air fryer according to claim 1, characterized in that: The cloud server is also used to identify the type of food to be cooked based on the food image, and after calculating the weight of the food using a volume projection algorithm, send the obtained data to the main control board. The main control board sets the menu category, as well as the recommended cooking time and cooking temperature based on the type of food and the weight of the food, and displays them through a display screen arranged on the surface of the machine body.

7. The air fryer according to claim 1, characterized in that: The shooting time interval of the camera assembly is greater than 2s and less than 8s.

8. The air fryer according to claim 1, characterized in that: The cloud server is also used to send the generated video file to the mobile terminal for playback and / or forwarding and sharing.

9. The air fryer according to claim 8, characterized in that: The mobile terminal is also used to edit the title of the received video file, insert text, insert background music or one or more of the above.

10. The air fryer according to claim 8, characterized in that: The mobile terminal is connected to the camera assembly via the cloud server and the main control board in turn, so as to view the cooking status of food in the cooking cavity in real time.

Images