Recipe recommendation method and device of intelligent steam oven

Abstract

The embodiment of the application provides a recipe recommendation method and device of an intelligent steaming oven, the recipe recommendation method comprises the following steps: obtaining a face image of a user; performing region segmentation processing on the face image to obtain a tongue image; performing feature quantization processing on the tongue image and the face image based on a constitution recognition model to obtain tongue features and face features; performing feature fusion processing on the tongue features and the face features to obtain fusion features; performing constitution recognition processing on the fusion features based on the constitution recognition model to obtain constitution information of the user; and performing food material matching processing and recipe recommendation processing on a food material database automatically according to the constitution information to generate a recommended recipe. The intelligent steaming oven is controlled by using the recipe recommendation method, constitution diagnosis can be implemented according to the face image of the user, recipe recommendation is implemented, and seamless connection between traditional Chinese medicine constitution recognition and intelligent cooking is achieved.

Description

Technical Field

[0001] This application relates to the field of smart home, specifically to a recipe recommendation method and device for a smart steam oven. Background Technology

[0002] With the advancement of technology, smart home appliances are becoming increasingly common in daily life. Among them, smart steam ovens have attracted much attention due to their automatic cooking and recipe recommendations. However, currently, smart steam ovens can only allow users to manually select ingredients or perform simple nutritional analyses (such as low-fat or high-protein), and cannot recommend suitable recipes and cooking methods based on the user's physical condition.

[0003] Therefore, there is an urgent need for a recipe recommendation method for smart steam ovens that can diagnose the user's physical condition and recommend recipes based on that condition. This would ensure that the smart steam oven adjusts the recipes and cooking methods according to the individual's physical condition, achieving more intelligent, personalized, and automated cooking. Summary of the Invention

[0004] To address the aforementioned technical problems, this application proposes a recipe recommendation method and apparatus for an intelligent steam oven.

[0005] On one hand, embodiments of this application provide a recipe recommendation method for a smart steam oven, the method including: Obtain the user's facial image; The facial image is segmented to obtain a tongue image; The tongue image and the facial image are subjected to feature quantization processing based on the body constitution recognition model to obtain tongue features and facial features respectively; the body constitution recognition model is trained based on different preset tongue images and different preset facial images; The tongue features and facial features are fused together to obtain fused features; Based on the aforementioned constitution recognition model, the fused features are processed for constitution recognition to obtain the user's constitution information; Based on the physical condition information, the system automatically performs ingredient matching and recipe recommendation processing in the ingredient database to generate recommended recipes; the ingredient database stores preset recommended ingredients and preset recommended recipes corresponding to different preset physical condition information.

[0006] Furthermore, the method also includes: In response to the ingredient information input by the user, the ingredient information is processed for ingredient properties and flavor based on the ingredient database to obtain the ingredient properties and flavor corresponding to the ingredient information; Based on the physical condition information and the ingredient information, the corresponding preset recommended recipes are selected from the ingredient database; Based on the physical constitution information and the corresponding flavor and properties of the ingredients, the cooking parameters of the preset recommended recipe are adjusted to obtain the adjusted preset recommended recipe, and the adjusted preset recommended recipe is used as the recommended recipe.

[0007] Furthermore, the method also includes: When the duration of the physical fitness information reaches the first preset time and / or in response to user feedback, repeat the operation of acquiring the user's facial image until the user's physical fitness information is obtained, and new physical fitness information is obtained. Based on the new physical condition information, the food ingredient database is used for ingredient matching and recipe recommendation to obtain new recommended recipes.

[0008] Further, the step of performing region segmentation processing on the facial image to obtain a tongue image includes: The facial image is subjected to key point detection and tongue verification to obtain the first detection result; The facial image is subjected to background removal processing to obtain a second detection result; Based on the first detection result and the second detection result, the facial image is segmented to obtain the tongue image.

[0009] Furthermore, the constitution recognition model includes a tongue image analysis module and a facial analysis module. The constitution recognition model performs feature quantization processing on the tongue image and the facial image respectively to obtain tongue features and facial features, including: The tongue image is quantized based on the tongue image analysis module to obtain the tongue features; The facial features are obtained by performing facial feature quantization processing on the facial image based on the facial analysis module.

[0010] Further, the tongue image is subjected to tongue feature quantization processing based on the tongue image analysis module to obtain the tongue features, including: Based on the tongue image analysis module, the tongue image is segmented to obtain the tongue body region; The tongue region is subjected to color space conversion, grayscale processing, and wavelet crack processing to obtain tongue coating features; the tongue coating features include at least one of the following: color features, grayscale features, directional information, high-frequency energy distribution ratio, and crack distribution map. The aspect ratio of the tongue region is processed and the edge curvature is detected to obtain the tongue features; The tongue features are constructed based on the tongue coating features and the tongue body features.

[0011] Further, the facial feature quantization processing of the facial image based on the facial analysis module to obtain the facial features includes: The facial image is partitioned based on facial key points to obtain a partitioned facial image. Based on the facial analysis module, a contrast analysis is performed on the partitioned facial image to obtain the first facial feature; Based on the facial analysis module, local binary processing is performed on the peri-eye region in the partitioned facial image to obtain peri-eye region features; Based on the facial analysis module, the facial color feature is processed in the partitioned facial image to obtain facial color features; The facial features are constructed based on the first facial features, the periorbital region features, and the facial color features.

[0012] Furthermore, the method also includes: When the smart steam oven reaches the second preset time, a preset number of new facial images are acquired as training data; the new facial images carry corresponding physical information, and the new facial image data is acquired via network connection. The training data is subjected to incremental learning training and data balancing to obtain an updated physical fitness analysis model. The system acquires historical physical constitution information from the intelligent steam oven, and iteratively trains the updated physical constitution analysis model based on this information to obtain a personalized physical constitution analysis model.

[0013] Furthermore, the method also includes: In response to the start signal of the smart steam oven, historical physical condition information and the determination time of the historical physical condition information are acquired even without acquiring the facial image. Get historical recommended recipes and the current installation location; If the determination time of the historical physical condition information is within a third preset time period, obtain the historical recommended recipes corresponding to the historical physical condition information, and use the historical recommended recipes as the current recommended recipes; If the time of determining the historical physical condition information is not within the third preset time period, a new recommended recipe is generated based on the historical recommended recipe and the current position.

[0014] On the other hand, this application embodiment also provides a recipe recommendation device for a smart steam oven, the device comprising: The acquisition module is used to acquire the user's facial image; The segmentation module is used to perform region segmentation processing on the facial image to obtain a tongue image; The feature quantization processing module is used to perform feature quantization processing on the tongue image and the facial image respectively based on the constitution recognition model to obtain tongue features and facial features; the constitution recognition model is trained based on different preset tongue images and different preset facial images; The feature fusion module is used to perform feature fusion processing on the tongue features and the facial features to obtain fused features; The identification module is used to perform body constitution identification processing on the fused features based on the body constitution identification model to obtain the user's body constitution information; The recommendation module is used to automatically perform ingredient matching and recipe recommendation processing in the ingredient database based on the physical condition information, and generate recommended recipes; the ingredient database stores preset recommended ingredients and preset recommended recipes corresponding to different preset physical condition information.

[0015] Furthermore, the device also includes: The ingredient properties and flavor verification module is used to respond to the ingredient information input by the user, perform ingredient properties and flavor verification processing on the ingredient information based on the ingredient database, and obtain the ingredient properties and flavors corresponding to the ingredient information. The filtering module is used to filter out corresponding preset recommended recipes from the ingredient database based on the physical condition information and the ingredient information. The parameter adjustment module is used to adjust the cooking parameters of the preset recommended recipe based on the physical condition information and the flavor and taste of the ingredients corresponding to the ingredient information, to obtain the adjusted preset recommended recipe, and to use the adjusted preset recommended recipe as the recommended recipe.

[0016] Furthermore, the device also includes: The re-acquisition module is used to repeat the operation of acquiring the user's facial image until the duration of the physical information reaches a first preset time and / or in response to user feedback, so as to obtain new physical information. The second recommendation module is used to perform ingredient matching and recipe recommendation processing in the ingredient database based on the new physical condition information to obtain new recommended recipes.

[0017] Furthermore, the segmentation module includes: The first detection result acquisition unit is used to perform key point detection and tongue verification processing on the facial image to obtain the first detection result. The second detection result acquisition unit is used to perform background removal processing on the facial image to obtain a second detection result. The tongue image acquisition unit is used to perform region segmentation processing on the facial image based on the first detection result and the second detection result to obtain the tongue image.

[0018] Furthermore, the constitution recognition model includes a tongue image analysis module and a facial analysis module, and the feature quantification processing module includes: The tongue feature quantization processing unit is used to perform tongue feature quantization processing on the tongue image based on the tongue image analysis module to obtain the tongue features; The facial feature quantization processing unit is used to perform facial feature quantization processing on the facial image based on the facial analysis module to obtain the facial features.

[0019] Furthermore, the tongue feature quantization processing unit includes: The tongue region segmentation unit is used to perform region segmentation processing on the tongue image based on the tongue image analysis module to obtain the tongue region. The tongue coating feature determination unit is used to perform color space conversion, grayscale processing, and wavelet crack processing on the tongue body region to obtain tongue coating features; the tongue coating features include at least one of the following features: color features, grayscale features, direction information, high-frequency energy distribution ratio, and crack distribution map. The tongue feature determination unit is used to process the aspect ratio and detect the edge curvature of the tongue region to obtain tongue features. A tongue feature construction unit is used to construct the tongue features based on the tongue coating features and the tongue body features.

[0020] Furthermore, the facial feature quantization processing unit includes: A facial image partitioning processing unit is used to partition the facial image based on facial key points to obtain a partitioned facial image. A contrast determination unit is used to perform contrast analysis on the partitioned facial image based on the facial analysis module to obtain a first facial feature. The periocular region feature determination unit is used to perform local binary processing on the periocular region in the partitioned facial image based on the facial analysis module to obtain periocular region features. A facial color feature determination unit is used to perform facial color feature processing on the partitioned facial image based on the facial analysis module to obtain facial color features; A facial feature construction unit is used to construct the facial features based on the first facial features, the periorbital region features, and the facial color features.

[0021] Furthermore, the device also includes: The training data acquisition unit is used to acquire a preset number of new facial images as training data when the running time of the intelligent steam oven reaches a second preset time; the new facial images carry corresponding physical information, and the new facial image data is acquired through a network connection. The model update unit is used to perform incremental learning training and data balancing on the training data to obtain an updated physical fitness analysis model. A personalized constitution analysis model construction unit is used to acquire historical constitution information from the smart steam oven, and to iteratively train the updated constitution analysis model based on the historical constitution information to obtain a personalized constitution analysis model.

[0022] Furthermore, the device also includes: The first historical data acquisition unit is used to acquire historical physical information and the determination time of historical physical information in response to the start signal of the smart steam oven, without acquiring the facial image. The second historical data acquisition unit is used to acquire historical recommended recipes and the current installation location; The recipe recommendation unit is used to obtain historical recommended recipes corresponding to the historical physical condition information when the determination time of the historical physical condition information is within a third preset time period, and use the historical recommended recipes as the current recommended recipes; when the determination time of the historical physical condition information is not within the third preset time period, it generates new recommended recipes based on the historical recommended recipes and the current position.

[0023] On the other hand, embodiments of this application also provide an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program. The processor loads and executes the at least one instruction or at least one program to implement the recipe recommendation method of the intelligent steam oven as described above.

[0024] On the other hand, embodiments of this application also provide a computer-readable storage medium storing at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the recipe recommendation method of the intelligent steam oven as described above.

[0025] On the other hand, this application also provides a computer program product, which, when executed by a processor, implements the recipe recommendation method for the intelligent steam oven as described above.

[0026] On the other hand, this application also provides a smart steam oven, which uses the recipe recommendation method of the smart steam oven described above to recommend recipes.

[0027] This application provides a method and apparatus for recommending recipes in a smart steam oven. The method includes acquiring a user's facial image; performing region segmentation processing on the facial image to obtain a tongue image; performing feature quantization processing on the tongue image and the facial image based on a constitution recognition model to obtain tongue features and facial features; the constitution recognition model is trained based on different preset tongue images and different preset facial images; performing feature fusion processing on the tongue features and the facial features to obtain fused features; performing constitution recognition processing on the fused features based on the constitution recognition model to obtain the user's constitution information; automatically performing ingredient matching processing and recipe recommendation processing in an ingredient database based on the constitution information to generate recommended recipes; the ingredient database stores preset recommended ingredients and preset recommended recipes corresponding to different preset constitution information. This application embodiment directly acquires a user's facial image and segments the tongue image from it. Facial and tongue features are extracted from both images for constitution diagnosis. Then, a pre-trained constitution recognition model is used to identify the user's constitution based on these features, obtaining the user's constitution information. Finally, based on this constitution information, corresponding recipes are matched against a food database to generate recommended recipes. This enables the smart steam oven to perform constitution recognition based on the user's facial image, allowing for recipe recommendations based on the user's constitution information. This achieves a seamless integration of traditional Chinese medicine constitution recognition and intelligent cooking, further enhancing the intelligence and personalization capabilities of the smart steam oven. Attached Figure Description

[0028] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a recipe recommendation method for an intelligent steam oven provided in an embodiment of this application.

[0030] Figure 2 This is a flowchart illustrating a method for determining a tongue image provided in an embodiment of this application.

[0031] Figure 3 This is a flowchart illustrating a method for determining tongue and facial features according to an embodiment of this application.

[0032] Figure 4 This is a flowchart illustrating a method for determining tongue features provided in an embodiment of this application.

[0033] Figure 5 This is a flowchart illustrating a method for determining facial features provided in an embodiment of this application.

[0034] Figure 6 This is a flowchart illustrating a method for determining fusion features provided in an embodiment of this application.

[0035] Figure 7 This is a flowchart illustrating a method for dynamically adjusting cooking parameters based on physical condition, as provided in an embodiment of this application.

[0036] Figure 8 This is a flowchart illustrating a method for providing feedback and adjusting recommended recipes according to an embodiment of this application.

[0037] Figure 9 This is a flowchart illustrating a feedback optimization rule provided in an embodiment of this application.

[0038] Figure 10 This is a flowchart illustrating a method for updating a body constitution classification model and a method for training a personalized body constitution analysis model, as provided in an embodiment of this application.

[0039] Figure 11 This is a flowchart illustrating the updating process of a system analysis model provided in an embodiment of this application.

[0040] Figure 12 This is a flowchart illustrating a personalized recommendation method provided in an embodiment of this application.

[0041] Figure 13 This is another schematic diagram of a recipe recommendation method for an intelligent steam oven provided in this application embodiment.

[0042] Figure 14 This is a structural block diagram of a recipe recommendation device for an intelligent steam oven provided in an embodiment of this application.

[0043] Figure 15 This is a hardware structure block diagram of a server for a recipe recommendation method for an intelligent steam oven provided in an embodiment of this application. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0045] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of the embodiments of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the present application described herein can be implemented in orders other than those illustrated or described herein. Thus, features defined with "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments, unless otherwise stated, "a plurality of" means two or more. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0046] On the one hand, embodiments of this application provide a recipe recommendation method for an intelligent steam oven. Figure 1 This is a flowchart illustrating a recipe recommendation method for a smart steam oven according to an embodiment of this application. This application provides the operational steps described in the embodiments or flowchart, but based on conventional or non-inventive labor, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown... Figure 1 As shown, the method may include: S101: Obtain the user's facial image.

[0047] In this embodiment, the facial image refers to the user's facial image, which can be captured by a camera installed on the smart steam oven, or by other smart devices and then synchronized or uploaded to the control system of the smart steam oven. For example, other smart devices refer to devices with camera functionality that are wirelessly or otherwise connected to the smart steam oven. The initial format of the facial image in this embodiment is an RGB image.

[0048] In one specific embodiment, the acquired facial image must possess the following characteristics: the user's tongue in the facial image must be in a naturally extended state, so that subsequent image segmentation can be performed to obtain the tongue image. Optionally, if the acquired facial image does not possess the required characteristics, the smart steam oven can proactively issue a notification indicating that it does not meet the requirements.

[0049] In one specific embodiment, the smart steam oven uses OpenCV (Open Source Computer Vision Library) for facial image detection. For example, it uses the Haar feature cascade classifier in OpenCV to automatically capture and store facial information from a camera. Optionally, during the saving process, reflection interference in the image is removed to make the saved image clearer. Methods for removing reflection interference include, but are not limited to, color space-based methods, morphological operation-based methods, histogram-based methods, and the use of polarizing filters.

[0050] In one specific embodiment, during facial image acquisition, to improve the detection rate in low-light environments, dynamic illumination supplementation is employed based on the aforementioned Haar feature cascade classifier. Specifically, during image acquisition, the illumination intensity of the image background is detected to obtain the current illumination intensity, and the detection threshold of the Haar feature cascade classifier is adjusted according to the current illumination intensity. For example, the detection threshold is lowered in low light and raised in high light. The detection threshold is used to determine whether the input features belong to the target category; that is, the detection threshold determines the detection accuracy.

[0051] Optionally, the smart steam oven is equipped with an image acquisition unit for capturing facial images. Specifically, the smart steam oven has a built-in camera that supports IR infrared illumination to eliminate ambient light interference, and the camera supports a resolution of 5 megapixels or higher.

[0052] S103: Perform region segmentation processing on the facial image to obtain a tongue image.

[0053] In this embodiment of the application, the region segmentation process refers to dividing the facial image into multiple meaningful regions, which correspond to multiple regions of interest in traditional Chinese medicine constitution diagnosis, including the tongue region. The image of the tongue region is saved separately as one of the main bases for determining constitution information, that is, the tongue image is obtained.

[0054] In one specific embodiment, the above-mentioned region segmentation process first determines the mouth region based on 68 facial key points (Dlib library), and then extracts the tongue region from the mouth region.

[0055] S105: Based on the constitution recognition model, the tongue image and the facial image are subjected to feature quantization processing to obtain tongue features and facial features respectively; the constitution recognition model is trained based on different preset tongue images and different preset facial images.

[0056] In this embodiment, the constitution recognition model is pre-set in a smart steam oven. It is trained using a large number of tongue and facial images, along with constitution annotations on these images. Specific training methods can include traditional feature-based methods, deep learning-based methods, metric learning-based methods, etc. Facial features here include features of various facial regions, specifically the shape, color, and texture of each region. Tongue features refer to the color, shape, and distribution of the tongue and tongue coating, which can be used for constitution assessment.

[0057] The constitution recognition model established in this application embodiment can extract features from facial and tongue images, extract facial and tongue features for constitution diagnosis, such as dark complexion and cracked tongue coating, and can quantify these indicators. That is, the constitution recognition model also includes the indicator level corresponding to each feature, so as to output quantified tongue and facial features.

[0058] S107: Perform feature fusion processing on the tongue features and the facial features to obtain fused features.

[0059] In this embodiment of the application, in order to comprehensively judge the user's physical condition and avoid the deviation caused by a single feature in confirming the physical condition information, the tongue feature and facial feature are fused together. The fused feature is used as the basis for subsequent determination of physical condition information. Here, the feature fusion processing can adopt a weighted fusion method, for example, using one as the main feature and the other as the auxiliary feature, and fusing them according to different weights. The corresponding weights need to be judged according to different features.

[0060] In a specific embodiment, during feature fusion, TCM correlation matching is first performed on the constitution recognition model based on individual features to obtain the TCM correlation of facial features and the TCM correlation of tongue features. Then, the weights corresponding to facial features and tongue features are determined according to the TCM correlation of facial features and the TCM correlation of tongue features. Feature fusion is then performed according to the weights to obtain fused features.

[0061] S109: Perform body constitution recognition processing on the fused features based on the body constitution recognition model to obtain the user's body constitution information.

[0062] In this embodiment, after obtaining the fused features, they are matched based on a constitution recognition model to obtain the user's constitution information. Here, the constitution information corresponds to one of the nine constitution classifications in Traditional Chinese Medicine, namely, balanced constitution, qi deficiency constitution, yang deficiency constitution, yin deficiency constitution, phlegm-dampness constitution, damp-heat constitution, blood stasis constitution, qi stagnation constitution, and special constitution.

[0063] S111: Based on the physical condition information, automatically perform ingredient matching and recipe recommendation processing in the ingredient database to generate recommended recipes; the ingredient database stores preset recommended ingredients and preset recommended recipes corresponding to different preset physical condition information.

[0064] In this embodiment, the ingredient database stores preset recommended recipes and ingredients corresponding to different preset constitution information. Therefore, after obtaining the constitution information, the database automatically performs ingredient matching and recipe recommendation processing to obtain recommended recipes corresponding to the constitution information, which are then automatically sent to the steam oven control terminal. This method, by processing facial images, enables the intelligent cooking device to recommend recipes based on the user's constitution. This not only solves the problem of traditional steam ovens being unable to integrate constitution recommendations, but also achieves a seamless connection between traditional Chinese medicine constitution recognition and intelligent cooking, further enhancing the intelligence and personalization level of the intelligent steam oven.

[0065] In an optional embodiment, in step S103 above, as follows: Figure 2 As shown, the above-described region segmentation process of the facial image to obtain the tongue image includes: S1031: Perform key point detection and tongue verification processing on the facial image to obtain the first detection result.

[0066] In this embodiment of the application, the mouth region is first extracted from the face region by key point detection, and then the tongue region is extracted and the tongue is verified in the mouth region to verify whether the tongue is included in the mouth region, and a first detection result is obtained. The first detection result represents the detection result of the tongue region.

[0067] In a specific embodiment, the tongue region extraction process is as follows: After the mouth region is extracted based on key point detection, the tongue is first extracted from the mouth region by HSV color space threshold segmentation, that is, the area within the HSV color space threshold of the mouth region is extracted as the tongue region. The HSV color space is a color model based on human visual perception, which divides color into three main components: hue, saturation, and value. Optionally, the threshold can be H∈[0,15], S∈[20,255], V∈[50,255].

[0068] In one specific embodiment, keypoint detection is based on lightweight Dlib keypoints. Specifically, 30 upper-layer neurons that are not related to the tongue are removed from the Dlib 68-point model (while retaining the chin / mouth-related layers).

[0069] In one specific embodiment, the above tongue verification process is based on HSV saturation comparison, tongue aspect ratio comparison, and region continuity comparison. Specifically, the standards and anomaly handling methods for each verification index are shown in Table 1. Table 1. Indicators for tongue verification processing

[0070] The following is a detailed explanation of the tongue verification process: First, determine the average saturation value of the aforementioned tongue region. Specifically, extract the saturation components of the tongue region separately to form a saturation grayscale image, i.e., the S channel in the table above. Obtain the average value of the S channel, i.e., compare it with the average saturation threshold. If it is lower than the average saturation threshold, it is determined to be lips / background; if it is higher than the average saturation threshold, the verification process is considered passed. Optionally, the average saturation threshold is 60.

[0071] Secondly, determine the aspect ratio of the tongue region. If the aspect ratio is within the normal range of the tongue, the verification process is considered successful. If it is greater than or less than the normal range of the tongue, it is considered that there may be a false detection, such as an open mouth. A second verification is performed. If the results of the second verification are consistent, it is marked as an abnormal aspect ratio of the tongue.

[0072] Finally, the continuity of the tongue region is verified to determine the percentage of the largest connected component in the tongue region. If the percentage of the largest connected component is greater than a threshold, the verification process is considered successful. Optionally, the threshold for the percentage of the largest connected component is 80%. Regions with a percentage less than the threshold are considered fragmented and thus considered interference.

[0073] The embodiments of this application can improve the speed of calculation and reduce the false negative rate of tongue detection through tongue body verification processing.

[0074] S1033: Perform background removal processing on the facial image to obtain a second detection result.

[0075] In this embodiment of the application, the background removal of facial images is used to solve the problem of misjudgment in complex backgrounds in traditional 2D detection. Specifically, the background is removed by using a depth estimation method to eliminate interference from non-face areas. The second detection result represents the background removal result.

[0076] Background removal based on depth estimation is achieved as follows: First, a depth map is extracted from the facial image (each pixel corresponds to the actual distance, in meters or millimeters), and a depth threshold is pre-set (e.g., 0-3 meters is foreground, and the rest is background). Then, a mask is generated, where areas with a depth less than the threshold are set to 1 (foreground), and others are set to 0 (background). The mask is then multiplied pixel by pixel with the original image to remove the background.

[0077] It should be noted that S1033 and S1031 are performed simultaneously, without any specific order.

[0078] S1035: Based on the first detection result and the second detection result, perform region segmentation processing on the facial image to obtain the tongue image.

[0079] In this embodiment of the application, the first detection result and the second detection result are fused in this step to segment the tongue image from the facial image. Specifically, the fusion method can be either intersection fusion or weighted fusion. Intersection fusion means retaining the overlapping part of the first detection result and the second detection result, while weighted fusion means combining the first detection result and the second detection result according to a preset weight ratio.

[0080] This application embodiment obtains a tongue image by performing key point recognition and background removal on the acquired facial image, and reduces the false judgment rate, thereby improving the accuracy of tongue feature recognition and laying the foundation for recipe recommendation based on body constitution information.

[0081] As an optional embodiment, the constitution recognition model includes a tongue image analysis module and a facial analysis module, such as... Figure 3 As shown, in step S105 above, the tongue image and the facial image are subjected to feature quantization processing based on the constitution recognition model to obtain tongue features and facial features, including: S1051: Based on the tongue image analysis module, perform tongue feature quantization processing on the tongue image to obtain the tongue features; S1053: Based on the facial analysis module, perform facial feature quantization processing on the facial image to obtain the facial features.

[0082] In this embodiment of the application, the constitution recognition model includes a tongue image analysis module and a face analysis module. The tongue image analysis module is used to analyze the tongue image to obtain tongue features, and the face analysis module is used to analyze the face image to obtain face features.

[0083] In one specific embodiment, the constitution recognition model uses an improved ResNet-18 architecture, including at least a tongue image analysis module and a face analysis module. The input layer of the tongue image analysis module is a standardized tongue image obtained from the tongue image in step S1035 above, after standardization processing. Optionally, the standardized tongue image is a 224×224 pixel tongue image (3-channel RGB). 3-channel RGB means that each pixel of the image consists of three color channels: red, green, and blue.

[0084] The specific process of standardizing tongue images is as follows: (1) Trim the minimum bounding rectangle of the tongue body. Specifically, use OpenCV’s cv2.minAreaRect to obtain the minimum bounding rectangle of the tongue body. cv2.minAreaRect is a function in OpenCV used to calculate the minimum bounding rectangle of a given set of points (usually a contour).

[0085] (2) Scaling to a preset pixel using bicubic interpolation: Bicubic interpolation is an advanced interpolation method for image scaling. It estimates the value of the target pixel by calculating a weighted average of the surrounding 16 pixels (4×4 grid). Compared to bilinear interpolation, bicubic interpolation produces smoother images, especially when zooming in. Optionally, bicubic interpolation can be implemented using the cv2.resize function in OpenCV. The preset pixel size can be 224×224.

[0086] In one specific embodiment, the tongue image normalization process still maintains the 3-channel RGB of the tongue image. This process allows the input size of the tongue image to be compatible with standard networks such as ResNet, while preserving color information, which is crucial for tongue color analysis.

[0087] The following is a detailed explanation of the body constitution identification model: The physical fitness identification model adopts an improved ResNet-18 architecture. ResNet (Residual Network) is a deep convolutional neural network architecture, and ResNet-18 is a variant of the ResNet architecture, containing 18 layers (including input and output layers). The main improvements of the improved ResNet-18 in this embodiment include the following aspects: (1) Add an SE (Squeeze-Excitation) attention module after the fourth residual block; (2) The output layer is changed to 9 neurons (corresponding to 9 types of TCM constitution).

[0088] Furthermore, the training data for the tongue image analysis module in the constitution recognition model consists of at least 10,000 tongue images carrying constitution labels. These tongue images are standardized tongue images that have undergone the aforementioned standardization process, and data augmentation is performed during training to allow the model to see more diverse images in each iteration. This increases the model's generalization ability, enabling it to perform better in the face of different variations. For example, data augmentation can employ random rotation (±15°) and HSV perturbation (ΔH=±10), where HSV perturbation (ΔH=±10) refers to perturbing the hue of the tongue image, with a perturbation range of ΔH=±10.

[0089] Furthermore, the tongue image analysis module in the body constitution recognition model uses Focal Loss as its loss function. Focal Loss is a loss function used for classification tasks, particularly suitable for handling class imbalance problems. The formula for Focal Loss is: ; in, This is a parameter that balances the weights of positive and negative samples, and is usually set to... When y=1; When y=0, y is the true label (0 or 1), pt is the probability of a positive sample predicted by the model, and γ is the exponent of the adjustment factor, controlling the rate of weight decay of easily classified samples. In this embodiment, γ=2 to address sample imbalance.

[0090] The evaluation metric is recall in the confusion matrix. Recall is calculated for a specific category (usually the positive class) and measures the model's ability to correctly identify positive samples. Optionally, the confusion matrix can be used to show that the recall rate for the constitution category is greater than a preset threshold as the evaluation metric. For example, the confusion matrix shows a recall rate of 92.3% for the damp-heat constitution.

[0091] As an optional embodiment, in step S1051 above, such as Figure 4 As shown, the tongue image is subjected to tongue feature quantization processing based on the tongue image analysis module to obtain the tongue features, including: S10511: Based on the tongue image analysis module, perform region segmentation processing on the tongue image to obtain the tongue body region.

[0092] In this embodiment, to obtain tongue features, the tongue body region is first extracted from the tongue image based on the tongue image analysis module. Here, the tongue body region refers to the image of the tongue body region. Specifically, the extraction process of the tongue body region is the same as in step S1031, and will not be repeated here. The white area in the extracted tongue body region is identified as tongue coating.

[0093] S10513: Perform color space conversion, grayscale processing, and wavelet crack processing on the tongue region to obtain tongue coating features; the tongue coating features include at least one of the following: color features, grayscale features, direction information, high-frequency energy distribution ratio, and crack distribution map.

[0094] In this embodiment of the application, in order to extract tongue coating features, the RGB pixels of the tongue image are first converted into LAB values. The three-dimensional axes of LAB include L*: brightness axis, that is, the brightness of the color (usually from 0 (black) to 100 (white)); a*: red-green axis, that is, the color change from green to red (usually from -128 (green) to +127 (red)); b*: yellow-blue axis, that is, the color change from blue to yellow (usually from -128 (blue) to +127 (yellow)).

[0095] Among them, the a* value in the LAB value directly reflects the difference between red and green tones and can be used to characterize the color characteristics of the tongue coating.

[0096] After converting the tongue image to LAB values, the mean value of the a* channel of the tongue coating region in the tongue body area is calculated, which is the mean value of the a* channel of the white region in the tongue body area. The mean value of the a* channel is then used as the color feature of the tongue coating. Based on the color feature of the tongue coating, the possible constitution type can be preliminarily judged, as shown in Table 2.

[0097] Table 2 Correspondence between color characteristics and possible body types

[0098] The following explains why the a* value was chosen for color feature analysis: (1) The a* value has the advantage of resisting light interference. The L channel of LAB separates the brightness information, making the a* value more stable. (2) The a* value has the advantage of human eye alignment, and the direction of the a* axis is consistent with the observation dimension of "pale red tongue color" in traditional Chinese medicine.

[0099] After acquiring color features, the contrast is calculated using the Gray-Level Co-occurrence Matrix (GLCM), which extracts gray-level features and directional information from the tongue coating characteristics. First, let's introduce the Gray-Level Co-occurrence Matrix (GLCM). GLCM is a statistical method for quantifying image texture features. By analyzing the spatial distribution of pixel gray values, it extracts features such as contrast, energy, and homogeneity. For example, in traditional Chinese medicine tongue diagnosis, GLCM contrast can be used to identify the roughness of the tongue coating (typical value >30) in individuals with a damp-heat constitution.

[0100] The specific calculation process includes the following steps: Step 1: Input Information Acquisition

[0101] In one specific embodiment, the input information includes a grayscale image and spatial relationship parameters.

[0102] A grayscale image refers to a single-channel grayscale image (8-bit, 0-255) of the tongue coating area. (8-bit, 0-255) means that each pixel value is represented by 8 bits, with a value range from 0 to 255. This type of image has only one color channel, specifically used to represent brightness information, and no color information. It is obtained by extracting the brightness (L) component separately from the LAB format tongue coating area. Specifically, the brightness (L) component is first extracted from the LAB image, then normalized and scaled. That is, the value of the L component is normalized and scaled to the range of 0-255 to fit the 8-bit image format, resulting in the processed L component. Finally, the processed L component is converted into an 8-bit image.

[0103] Spatial relationship parameters refer to the orientation and distance of specified pixel pairs (e.g., 0° orientation, distance = 1 pixel). The specific parameters of the spatial relationship parameters are determined according to the contrast analysis requirements. Optional, commonly used orientations: 0°, 45°, 90°, 135°; distance: 1~5 pixels.

[0104] Step 2: Calculate contrast and orientation information.

[0105] 1. Construct a co-occurrence matrix: Calculate the frequency P(i,j) of a pixel with gray value i co-occurring with its neighboring pixel with gray value j in a specified direction (e.g., horizontal 0°) and at a distance (e.g., 1 pixel).

[0106] For example, for a 3x3 grayscale image: [50, 50, 60]; [50, 60, 60]; [60, 60, 70]; GLCM (partial) in the horizontal direction (distance=1):

[0107] 2. Normalized matrix: Divide the frequency P(i,j) by the total number of pixel pairs to obtain the probability matrix p(i,j).

[0108] 3. Calculate contrast: The formula for calculating contrast is as follows: ; Where N is the number of gray levels (N=256 for an 8-bit image), and p(i,j) is the element in the i-th row and j-th column of the co-occurrence matrix, representing the probability that gray levels i and j appear simultaneously.

[0109] The physical meaning of calculating contrast is that the larger the contrast value, the coarser the texture of the tongue coating area (for example, cracks or granular textures are common in the tongue coating of people with damp-heat constitution).

[0110] Step 3: Output the calculation results.

[0111] Specifically, the contrast value is output as the grayscale feature of the tongue coating area according to the above calculation process. Its value range is usually 0~100, among which the tongue coating of the damp-heat constitution is typically >30.

[0112] Secondly, it can also output contrast values ​​in multiple directions, such as 0°, 45°, etc., to comprehensively evaluate the texture anisotropy of the tongue coating area, summarize the texture anisotropy, and map it to the texture characteristics in traditional Chinese medicine diagnosis to obtain the directional information. The formula for calculating texture anisotropy can be expressed as: ; Where D is the number of directions considered (e.g., 0°, 45°, 90°, 135°), GLCM i,j It is an element of GLCM in a specific direction. This represents normalizing each element in GLCM by dividing it by the sum of all elements, thus representing it as a probability value.

[0113] In one specific embodiment, Table 3 shows the texture and contrast features of some typical body types.

[0114] Table 3. Texture and contrast characteristics of typical body types

[0115] In one specific embodiment, the tongue coating features also include a crack distribution map. Wavelet transform can decompose a signal or image into components of different frequencies and scales, making it particularly suitable for identifying and extracting crack features from images. The crack distribution can be extracted using wavelet transform, specifically including the following steps: Step 1: Input Information Acquisition

[0116] 1. The crack distribution map is also obtained by analyzing grayscale images, namely the single-channel grayscale image (8-bit, 0~255) of the tongue coating area mentioned above. The acquisition process will not be described in detail here.

[0117] 2. Selection of wavelet basis functions.

[0118] In this embodiment, the Daubechies(db4) or Symlets(sym5) functions are used because these two functions have a strong ability to capture the edge features of irregular cracks, thus obtaining more accurate tongue coating features. Specifically, the Daubechies(db4) wavelet is a specific wavelet basis, and "sym" in the Symlets wavelet series represents symmetry, with the number (such as 5) representing the number of vanishing moments. Symlets wavelets are widely used in signal processing and image processing, especially in applications requiring good symmetry.

[0119] Optionally, when choosing a specific function, if the crack features are complex and the requirements for capturing signal details are high, Daubechies (db4) may be more suitable because it can handle the high-frequency details of the signal better; while when the crack distribution is more symmetrical and a smoother processing effect is required, Symlets (sym5) is more suitable because it has good symmetry, can reduce boundary effects, and provide more stable analysis results.

[0120] 3. The number of decomposition layers should be set to 3 to 4 (too many layers will lead to excessive fragmentation of high-frequency information).

[0121] Step 2: Obtaining the crack distribution map.

[0122] In this embodiment of the application, taking a decomposition layer of 3 layers as an example, obtaining the crack distribution map using wavelet transform may include the following process: 1. Wavelet decomposition: This includes a first-level decomposition and iterative layer-by-layer decomposition. The first-level decomposition divides the tongue image into LL1 (low-frequency approximation, preserving the overall tongue color), LH1 (horizontal high-frequency, capturing vertical cracks), HL1 (vertical high-frequency, capturing horizontal cracks), and HH1 (diagonal high-frequency, capturing diagonal cracks). Iterative layer-by-layer decomposition refers to continuing the decomposition of LL1 to the third level, obtaining multi-scale high-frequency sub-images (LH2 / HL2 / HH2, LH3 / HL3 / HH3).

[0123] 2. High-frequency energy calculation: First, extract all high-frequency components, including LH / HL / HH; then calculate the high-frequency energy percentage using the formula shown below: ; 3. Crack visualization processing: First, the high-frequency components are fused, that is, the LH / HL / HH layers are superimposed to enhance the crack features. Then, a crack distribution map, also known as a heat map, is generated. In the heat map, the crack density is represented by a color gradient.

[0124] In summary, by processing the tongue region in a tongue image using wavelet transform, we can obtain the high-frequency energy ratio and crack distribution map. The high-frequency energy ratio is used to characterize the fine structural features of the tongue cracks, and thus correlate them with the functional state of the internal organs that Traditional Chinese Medicine (TCM) focuses on. For example, when the high-frequency energy ratio is greater than 65%, it is identified as a cracked tongue, which may indicate a Yin deficiency constitution. The crack distribution map can intuitively show the location and density of the cracks, and the direction of the main cracks can be extracted from the crack distribution map. Different directions of the main cracks may correspond to different organ syndromes.

[0125] It should be noted that the above-mentioned judgment on grayscale features and directional information is mainly used to determine whether the user has a damp-heat constitution, and the judgment on crack distribution map and high-frequency energy ratio is mainly used to determine whether the user has a yin deficiency constitution. Therefore, the above features can be used for partial judgment. For example, if it is determined that the user is likely to have a damp-heat constitution based on grayscale features and directional information, the acquisition of crack distribution map and high-frequency energy ratio can be temporarily suspended.

[0126] S10515: Perform aspect ratio processing and edge curvature detection on the tongue region to obtain tongue features.

[0127] In this embodiment of the application, tongue features refer to the outline features of the tongue, including at least one of the following: length-to-width ratio, number of teeth marks, and distribution of teeth marks.

[0128] In one specific embodiment, aspect ratio processing of the tongue region yields the tongue's aspect ratio and area, thereby determining whether the overall shape of the tongue is abnormal. Specifically, the aspect ratio calculation process includes: Step 1: Acquire the contour point set of the tongue. Specifically, the grayscale image of the tongue region is first thresholded to obtain a binary image. Then, `cv2.findContours` is used to acquire the contour point set of the tongue from the binary image. The contour point set refers to the set of discrete points on the tongue contour extracted from the tongue image. These contour point sets can be used for further analysis of the tongue's shape, symmetry, boundary features, etc. `cv2.findContours` is a very important function in the OpenCV library for extracting image contours. It can detect and extract contours from a binary image and return these contours as a set of contour points.

[0129] Step 2: Preprocess the collected contour point set of the tongue.

[0130] The tongue's contour point set is thinned to obtain a simplified tongue contour point set. Thinning refers to retaining the main points in the contour point set, simplifying the complex contour point set into fewer points while preserving the main shape features of the contour. This is very useful when processing tongue contours, as it can reduce computation while retaining key information. Optionally, the Douglas-Peucker algorithm is applied to the tongue contour using cv2.approxPolyDP, a function in OpenCV for contour point set simplification that implements the Douglas-Peucker algorithm. The Douglas-Peucker algorithm is a classic algorithm for simplifying polygons or polylines. It approximates the original contour by recursively selecting key points while preserving its main shape features. The core idea of ​​the algorithm is: on a given contour, find the point farthest from the line segment; if this distance is greater than a certain threshold (called epsilon), then retain the point and recursively process the remaining part; otherwise, discard the point.

[0131] Step 3: Minimum bounding rectangle fitting. Using OpenCV's `cv2.minAreaRect()`, the minimum bounding rectangle of the tongue is obtained from the simplified tongue contour point set. The ratio of the longer side (width) to the shorter side (height) of the rectangle is then calculated to obtain the tongue's aspect ratio. This aspect ratio is compared with a preset aspect ratio threshold to analyze whether there are any abnormalities. Optionally, the preset aspect ratio thresholds include: a normal tongue's aspect ratio range of 1.5~2.0; and an aspect ratio >2.5 for a swollen tongue, which corresponds to a phlegm-dampness constitution in Traditional Chinese Medicine. `cv2.minAreaRect()` is an OpenCV function used to calculate the minimum area rectangle for a given set of points (usually a contour). This rectangle may not be axis-aligned but can be rotated, thus better fitting contours of arbitrary shapes.

[0132] Step 4: Calculate the area of ​​the tongue region. Specifically, count the total number of white pixels in the mask, then convert the total area to physical dimensions using an actual ruler (e.g., a reference object placed during shooting), to obtain the area of ​​the tongue region. By combining the tongue region area with the tongue contour point set, the contour symmetry can be obtained. The tongue shape is determined through edge curvature detection.

[0133] In one specific embodiment, the edge features of the tongue are further determined by edge curvature detection of the tongue region, specifically through the following method: 1. Calculate the curvature K of each point in the simplified tongue contour point set, using the following formula: ; in, , The first derivative, , Let x be the second derivative, and y be the x and y coordinates of the point, respectively.

[0134] The curvature of each point is calculated using the above formula to obtain the curvature set of the tongue contour.

[0135] 2. Determine the presence of teeth marks.

[0136] First, extreme curvature points are determined from the curvature set of the tongue contour. A tooth mark is identified when any extreme curvature point is greater than a curvature threshold, the corresponding depression depth is greater than a depression depth threshold, and the corresponding tooth mark width meets the minimum tooth mark width requirement. Optionally, the curvature threshold is a value between 0.03 and 0.07. An extreme curvature point refers to a point on a curve or contour where the curvature value reaches a local maximum. Specifically, a local maximum occurs when the curvature value of a point is greater than the curvature values ​​of its neighboring points within a certain neighborhood. Therefore, there may be multiple extreme curvature points on the tongue. The depression depth typically refers to the vertical distance from the extreme curvature point to its neighboring points. Optionally, the depression depth threshold is a value greater than 5 pixels. Optionally, the minimum tooth mark width is any value between 3 and 8 pixels.

[0137] It should be noted that setting the curvature threshold is used to filter out smooth contour segments, setting the indentation depth threshold is used to exclude slight fluctuations, and setting the minimum tooth mark width is used to avoid noise interference.

[0138] Next, the tooth marks are counted to obtain the total number of tooth marks. Specifically, a continuous group of indented points is counted as one tooth mark. A continuous group of indented points refers to a group of consecutive points on a curve or contour that exhibit significant indentation features. These points usually correspond to local indented areas of the contour, such as cracks, notches, or other indented parts of shapes.

[0139] Based on the above steps, the characteristics of the tongue are obtained.

[0140] S10517: Construct the tongue features based on the tongue coating features and the tongue body features.

[0141] In this embodiment, tongue features are obtained based on tongue coating and tongue body characteristics, allowing for a comprehensive analysis of constitution based on the various features within the tongue characteristics. Table 4 shows some of the indicators and abnormality criteria corresponding to the tongue features; these are merely examples, and more are included in actual applications.

[0142] Table 4. Indicators and Abnormality Judgments Corresponding to Some Tongue Features

[0143] The above is just an example, representing that during the feature acquisition process, the feature will be compared with the preset value in the physical fitness recognition model, and the judgment result of the item will be given, including the abnormality and the possible reasons for the abnormality.

[0144] It should be noted that the above analysis of tongue features is based on a body constitution recognition model. Specifically, some features are completed in the shallow convolution kernels of ResNet18, such as color features and contour symmetry, while some features are completed in the deep convolution kernels, such as the aspect ratio of the tongue, the number of teeth marks, and the distribution map of cracks. The SE attention module is used to enhance the focus on key areas (such as the thick and greasy parts of the tongue coating), and no specific limitations are made in this application.

[0145] This application embodiment constructs a constitution recognition model based on ResNet18, thereby enabling feature recognition of tongue images, obtaining tongue coating and tongue body features, and judging the features. This realizes the integration of TCM diagnosis tongue function into a smart steam oven, laying the foundation for more intelligent recipe recommendations.

[0146] As an optional embodiment, in step S1053 above, such as Figure 5 As shown, the facial features obtained by performing facial feature quantization processing on the facial image based on the facial analysis module include: S10531: The facial image is partitioned based on facial key points to obtain a partitioned facial image.

[0147] In this embodiment, both the original image for facial feature quantization and the original image for tongue feature quantization are facial images obtained in step S101 above. During facial feature quantization, the facial region is first segmented to obtain a partitioned facial image. Specifically, the facial region is located based on 68 facial keypoints from Dlib, and the partitioning rules include: (1) Forehead: The area above the line connecting key points 18-26; (2) Nose wing: The area surrounded by key points 31-35; (3) Eye perimeter: The elliptical area surrounded by key points 37-48.

[0148] S10533: Based on the facial analysis module, perform contrast analysis on the partitioned facial image to obtain the first facial feature.

[0149] In this embodiment, the first facial feature refers to the shine of the forehead / nasal alar region, which is characterized by contrast. Specifically, the grayscale of the forehead region is cropped from the ROI extracted, and then the GLCM contrast at 0° (distance = 3 pixels) is calculated. The purpose of calculating the contrast is mainly to determine whether the person has a damp-heat constitution. The threshold for determining a damp-heat constitution is a contrast > 25 and an energy < 0.1. Here, energy is used to measure the uniformity of the texture in the forehead / nasal alar region. The higher the energy, the more uniform the texture of the image. The specific calculation method is the same as the application of GLCM in the feature extraction of the tongue region, and will not be repeated here.

[0150] S10535: Based on the facial analysis module, perform local binary processing on the peri-eye region in the partitioned facial image to obtain peri-eye region features.

[0151] In this embodiment, Local Binary Patterns (LBP) is a simple and effective feature descriptor for texture analysis. It extracts texture information by comparing the grayscale values ​​of the center pixel with those of its neighboring pixels to generate a binary pattern. Specifically, it includes the following process: (1) Select the center pixel: For each pixel p in the image corresponding to the periphery region c , and use it as the center pixel.

[0152] (2) Compare neighboring pixels: Compare the gray value of the center pixel with the gray values ​​of the P neighboring pixels. If the gray value of a neighboring pixel is greater than or equal to the gray value of the center pixel, assign it a value of 1; otherwise, assign it a value of 0.

[0153] (3) Generate binary pattern: Arrange the comparison results in order to form a binary number.

[0154] (4) Calculate the LBP value: Convert the binary number to a decimal number and use it as the LBP value of the center pixel.

[0155] (5) Calculate the LBP variance at each point.

[0156] The main purpose of calculating LBP variance is to determine whether there is kidney yang deficiency. When LBP variance > 85, it is considered abnormal and marked as kidney yang deficiency.

[0157] S10537: Based on the facial analysis module, perform facial color feature processing on the partitioned facial image to obtain facial color features.

[0158] In this embodiment, facial color feature processing is mainly performed on the cheekbone area. The sampling area is a circular ROI with a diameter of 20 pixels on each side of the cheekbone. The feature calculation is performed using the following formula: ; Among them, L mean b represents the mean value of L* within the cheekbone area. mean a represents the mean value of b* within the cheekbone area. mean This represents the mean value of a* within the cheekbone area.

[0159] S10539: Construct the facial features based on the first facial features, the periorbital region features, and the facial color features.

[0160] In this embodiment of the application, facial features are constructed based on the first facial features, the periorbital area features, and the facial color features. When any one or more of these features are in an abnormal state, the abnormal features are marked when outputting features, and the possible causes of the abnormality are output accordingly, serving as the basis for subsequent physical constitution analysis.

[0161] It should be noted that the training method for the facial feature analysis module in the body constitution analysis model is similar to that for the tongue image analysis module, the difference being that the training data uses facial images instead of tongue images.

[0162] This application embodiment achieves the integration of traditional Chinese medicine tongue diagnosis function into a smart steam oven by performing feature recognition on facial images, obtaining facial features, and judging the features, thus laying the foundation for more intelligent recipe recommendations.

[0163] In an optional embodiment, in step S107 above, as follows: Figure 6 As shown, the above-described feature fusion processing of the tongue features and facial features yields fused features, including: S1071: Identify abnormal features in tongue and facial features; S1073: Obtain the cause of the abnormality corresponding to the abnormal feature; S1075: Determine the weights of facial features and tongue features based on the causes of anomalies.

[0164] S1077: Based on the weights of the facial features and the tongue features, perform feature fusion processing on the tongue features and the facial features to obtain fused features.

[0165] In this embodiment of the application, the steps S1071 and S1073 refer to the process of analyzing tongue and facial features, during which features exceeding the threshold are marked as abnormal features, and possible causes of abnormality are also output.

[0166] In step S1075 above, in Traditional Chinese Medicine, different constitution classifications correspond to different diagnostic criteria. Therefore, the constitution classification model determines the main analytical features and auxiliary analytical features, as well as their respective weights, based on abnormal characteristics. For example, if the abnormal characteristics include tongue coating features, such as a yellow and greasy tongue coating (a*>145), it is speculated that the constitution may be damp-heat. In this case, it is necessary to combine the first facial feature in the facial features for analysis. An oily forehead (GLCM>30) may indicate a damp-heat constitution. The joint judgment rule is that the confidence level is increased by 25% only when both meet this requirement. When the abnormal characteristics include tongue coating features, such as teeth marks on the tongue (≥3), it is speculated that the constitution may be spleen deficiency. Then, based on the constitution classification model, it is judged whether the facial features corresponding to spleen deficiency meet the requirement of a sallow complexion (b*<110). At this time, since the main manifestation of spleen deficiency is the tongue appearance, the weight of the tongue appearance is set at 70% and the weight of the complexion at 30% when performing feature fusion. When abnormal features include tongue coating characteristics, such as a pale and swollen tongue (a*<105), it may indicate kidney yang deficiency. In this case, it is necessary to combine the facial features, specifically the periocular area features, for analysis. Since facial features and tongue features are equally important for kidney yang deficiency, the periocular area features and tongue coating features should be combined with each feature having a 50% weighting. By combining different features according to their corresponding weights for different possible constitutions, more accurate constitution information can be obtained, leading to more accurate recipe recommendations.

[0167] As an optional embodiment, this application also provides a method for dynamically adjusting cooking parameters based on physical condition, such as... Figure 7 As shown, the method includes: S001: In response to the ingredient information input by the user, the ingredient information is processed for ingredient properties and flavor verification based on the ingredient database to obtain the ingredient properties and flavors corresponding to the ingredient information.

[0168] In this embodiment of the application, the method provided in steps S101-S109 does not consider the user's desired ingredients when recommending recipes; it only recommends recipes based on the user's physical constitution. When the user inputs ingredient information, the method queries the ingredient database to determine the properties and flavors of the input ingredient information. Ingredient properties and flavors refer to the nature and taste of the ingredients. The "nature" of an ingredient refers to its four properties: cold, hot, warm, and cool; and the "flavor" refers to its five tastes: sour, bitter, sweet, spicy, and salty.

[0169] In a specific embodiment, the food ingredient property and flavor verification process, namely the aforementioned property and flavor query, can be implemented in two ways. The first is automatic matching in the food ingredient database, which is achieved based on semantic association of knowledge graphs. The second is to support manual correction, allowing users to manually adjust the property and flavor labels for special varieties / origin differences.

[0170] S003: Based on the physical condition information and the ingredient information, the corresponding preset recommended recipes are selected from the ingredient database.

[0171] In this embodiment of the application, after obtaining the ingredient information, the corresponding preset recommended recipes are simultaneously selected from the ingredient database based on the physical condition information and the ingredient information. When the physical condition is different, the recommended recipes may be different. However, for the same recipe, the cooking parameters are based on the fixed method in the ingredient data at this time, which may not be suitable for the current user.

[0172] S005: Based on the physical constitution information and the flavor and properties of the ingredients corresponding to the ingredient information, adjust the cooking parameters of the preset recommended recipe to obtain the adjusted preset recommended recipe, and use the adjusted preset recommended recipe as the recommended recipe.

[0173] In this embodiment, to further optimize cooking parameters based on physical constitution information to achieve better nutritional benefits, the parameters of the cooking recipes in the preset recommended recipes are adjusted according to the physical constitution information and the corresponding flavor and properties of the ingredients. This results in an adjusted preset recommended recipe, which is then sent to the terminal of the smart steam oven. Specific parameter adjustments may include cooking methods, cooking time, cooking temperature, steam intensity, etc. Optionally, the cooking parameter adjustments not only include adjustments during recipe recommendation but also dynamic adjustments to the cooking parameters during the cooking process based on the set cooking parameter method.

[0174] In one specific embodiment, the cooking temperature and humidity are monitored in real time using an NTC temperature sensor and a humidity sensor (model SHT35), and then dynamically compensated using a PID control algorithm. For example, if the patient's constitution is described as damp-heat type and the ingredient is winter melon, the preset recommended cooking method is 100℃ / 15min. The adjusted preset recommended cooking method increases the temperature by 5℃ every 2 minutes to accelerate moisture evaporation. This can be achieved by monitoring the cooking temperature and humidity in real time using the NTC temperature sensor and humidity sensor (model SHT35), and then dynamically adjusting the temperature using a PID control algorithm. When the patient's constitution is described as yin-deficient type and the ingredient is white fungus, the preset recommended cooking method is steaming at 100℃. However, high temperatures destroy the yin-nourishing effect of white fungus. Therefore, the adjusted preset recommended cooking method is slow steaming at 80℃, and extending the cooking time by 5 minutes when the humidity is >85% to allow the gelatinous substance to fully dissolve. At this point, the humidity sensor can monitor the gelatinous substance seepage status of the white fungus and dynamically extend the steaming time accordingly. When the body constitution information is damp-heat constitution, the preset recommended recipe is steamed fish (95℃). In order to accelerate the excretion of moisture, it is adjusted to high temperature fast steaming at 105℃, and the strong exhaust mode is turned on during the cooking process to reduce the retention of condensation.

[0175] In one specific embodiment, cooking parameter adjustments are based on a decision tree.

[0176] Optionally, when a user inputs not only ingredient information but also cooking method or recipe, the cooking method or recipe can be used as a preset recommended recipe, and the cooking parameters can also be optimized using the methods described above.

[0177] The method provided in this application can optimize the health benefits of ingredients, enabling recommended recipes to dynamically adjust temperature, humidity, and time based on the properties (cold / hot) of the ingredients and individual needs. It can also avoid discomfort caused by "one-size-fits-all" cooking and maximize the preservation of the medicinal properties of ingredients through parameter optimization (e.g., increasing the retention rate of polysaccharides in tremella by 20%).

[0178] As an optional embodiment, this application also provides a method for feedback adjustment of recommended recipes, such as... Figure 8 As shown, the method includes: S201: When the duration of the physical condition information reaches the first preset time and / or in response to user feedback, repeat the operation of acquiring the user's facial image until the user's physical condition information is obtained, and obtain new physical condition information.

[0179] In this embodiment of the application, the purpose of adjusting the recommended recipes is to conduct a second consultation to evaluate the effectiveness of the diet and optimize the recommendations. The triggering mechanisms include, but are not limited to, the following: (1) User feedback, such as changes in body feeling after eating or objections to recommended recipes, can be initiated through the human-computer interaction interface of the smart steam oven's APP or control terminal.

[0180] (2) Regular health check-ups: Once the duration of the physical condition information reaches the first preset time, a second consultation will be triggered. Here, the duration of the physical condition information refers to the time during which the physical condition information is generated in the smart steam oven and has not been updated. The first preset time can be customized by the user. When the first preset time is reached, the smart steam oven will send a push notification via the mobile APP and provide a smart reminder when the smart steam oven is turned on.

[0181] (3) System intelligent reminder. There are two possible triggering situations here. The first is based on key time nodes, such as when the user uses the same type of recipe a certain number of times in a row. The second is based on the intelligent wearable device associated with the intelligent device detecting abnormal physiological books, that is, reminding the user to have a second face-to-face consultation.

[0182] Once the user confirms the face-to-face consultation, repeat steps S101-S107 above to obtain the updated constitution information, i.e., the new constitution information mentioned above.

[0183] S203: Based on the new physical condition information, perform ingredient matching and recipe recommendation processing in the ingredient database to obtain new recommended recipes.

[0184] In this embodiment of the application, after obtaining new physical condition information, the recipe is re-recommended based on the new user physical condition information to obtain new recommended recipes, so as to avoid long-term use of a single diet.

[0185] Specifically, the recommended menu after the second consultation can be analyzed individually based on the new constitution information, or it can be comprehensively adjusted by combining the new constitution information and user feedback. For example, the feedback optimization rules are based on both the diagnostic results and dynamic feedback data, forming a dual verification mechanism. Taking Yang deficiency constitution as an example, for instance... Figure 9 As shown in Table 4.

[0186] Table 4. An exemplary feedback optimization rule

[0187] In other words, during the process of users re-determining their physical condition information, when making recommendations based on the new physical condition information, the recommendations can be made by comprehensively considering parameters such as the physical condition information, the new physical condition information, user physiological feedback, and environmental factors, as well as the corresponding weights of each parameter, rather than making recommendations solely based on the physical condition information. This makes the recommendation process more intelligent, improves the user experience, and can form a personalized health management closed loop, avoiding the side effects of long-term mono-dietary diets.

[0188] As an optional embodiment, this application also provides a method for updating a body constitution classification model and a method for training a personalized body constitution analysis model, such as... Figure 10 As shown, the method includes: S301: When the running time of the intelligent steam oven reaches the second preset time, a preset number of new facial images are acquired as training data; the new facial images carry corresponding physical information, and the new facial image data are acquired through a network connection.

[0189] In a specific embodiment, the TCM constitution classification needs to adapt to individual differences and long-term constitution changes. Therefore, the constitution classification model adopts a dynamic update strategy, which includes the following two types of updates: The first type involves periodic updates. When the smart steam oven reaches a second preset running time, it automatically integrates a preset number of new facial images from the network as training data. These new facial images carry corresponding physical condition information tags. Optionally, the second preset time is one quarter. Optionally, the preset number is ≥1000.

[0190] The second type is emergency updates, which are triggered when the accuracy of a certain type of physical condition recognition drops by more than 5% for one consecutive month. The training data for emergency updates can be the original training data, or, like regular updates, a preset number of new facial images can be automatically integrated from the network as training data.

[0191] S303: Incremental learning training and data balancing are performed on the training data to obtain an updated physical fitness analysis model.

[0192] In the embodiments of this application, such as Figure 11 As shown, the update process of the system analysis model includes acquiring the above-preset number of new facial images, data cleaning and annotation, incremental training, model validation, and test set threshold validation.

[0193] In one specific embodiment, the data cleaning and labeling steps first involve organizing and correcting the data to ensure its quality and consistency. The purpose of data cleaning is to remove noise, correct errors, and fill in missing values, making the data more suitable for model training. Optionally, data labeling involves adding labels to new facial images; that is, the physical information carried in the new facial images is used as data labels.

[0194] In one specific embodiment, incremental learning employs the Elastic Weight Consolidation (EWC) algorithm to prevent new data from overwriting old knowledge and performs data balancing to oversample rare traits (such as special traits) and avoid bias.

[0195] In one specific embodiment, the test set threshold validation here adopts F1 threshold validation. The F1 score is the harmonic mean of precision and recall, which can balance the accuracy and completeness of the model in positive class identification. By adjusting the decision threshold, an optimal threshold can be found that maximizes the F1 score. The specific steps include using the reinforcement learning-adjusted model to predict the test set, obtaining the predicted probability or score for each sample; then comparing the test set F1 with a preset threshold. If the test set F1 is greater than the preset threshold, the updated physical fitness identification model is obtained; if the test set F1 is less than the preset threshold, manual intervention is performed. The strategies for manual intervention include, but are not limited to, the following aspects: 1. Data-level interventions: These include data cleaning (checking for mislabeled and outlier values ​​in the test set, correcting or deleting them, and ensuring that the data distribution of the test set is consistent with that of the training set), data augmentation, and resampling.

[0196] 2. Model-level intervention: This includes adjusting model hyperparameters, such as learning rate and regularization parameters; checking whether the currently used features are reasonable, and trying to add or delete features.

[0197] 3. Threshold-level intervention: This includes readjusting the decision threshold to find a threshold that is more suitable for the current test set; dynamically adjusting the threshold according to the distribution of the test set, such as adjusting the threshold according to the category ratio, etc.

[0198] S305: Obtain historical physical constitution information from the intelligent steam oven, and iteratively train the updated physical constitution analysis model based on the historical physical constitution information to obtain a personalized physical constitution analysis model.

[0199] In this embodiment, the model can also be fine-tuned to construct a user-specific model. Specifically, historical body constitution information stored in the smart steam oven is obtained. This historical body constitution information carries user feedback label correction information. Using this historical body constitution information as training data, the updated body constitution analysis model is iteratively trained to obtain a personalized body constitution analysis model. The iterative training method here is the same as the model update method. Optionally, the amount of historical body constitution information must be greater than a certain amount before iterative training can be performed, for example, greater than 20 pieces.

[0200] In one specific implementation, when three consecutive constitution classification errors occur, or when a user actively reports an error, local fine-tuning is initiated to correct the model. Specifically, when three consecutive constitution classification errors occur, historical constitution information is used for retraining. When a user actively reports an error, a correction tag is created based on the user's feedback, and the corrected model is used to fine-tune the constitution analysis model to obtain a personalized constitution analysis model. For example, if user A's tongue appearance is misclassified as "balanced constitution" when it is actually "yin deficiency constitution," the tag can be manually corrected. The smart steam oven uses the corrected tag to fine-tune the model, which can be done using a learning rate of 0.001 and five iterations to obtain a personalized constitution analysis model.

[0201] It should be noted that user data and personalized physical fitness analysis models are not uploaded to the cloud. Fine-tuning is completed on local devices, and only the model parameter gradients are uploaded, with central servers aggregating and updating them.

[0202] In this embodiment, the body constitution analysis model is updated periodically and trained based on the user's historical body constitution information to obtain personalized body constitution analysis. This enables personalized settings, improves the accuracy of body constitution information judgment, and thus facilitates more accurate recipe recommendations.

[0203] As an optional embodiment, such as Figure 12 As shown in the embodiments of this application, a personalized recommendation method is also provided, the method comprising: S401: In response to the start signal of the smart steam oven, without acquiring the facial image, acquire historical physical condition information and the determination time of the historical physical condition information; S403: Retrieve historical recommended recipes and the current installation location.

[0204] In this embodiment of the application, the activation signal includes the user pressing the power button, voice interaction, etc. When the activation signal is recognized, if the user's facial image is obtained, historical physical condition information and the determination time of the historical physical condition information are obtained. Here, not all historical physical condition information is obtained, but only the historical physical condition information most recent to the activation signal time is obtained.

[0205] Furthermore, it retrieves historical recommended recipes and the current installation location of the smart steam oven.

[0206] In one specific embodiment, this application also discloses a method for constructing a user profile, specifically including: First, obtain historical physical condition information, historical cooking recipes, and the climate conditions of the current installation location. The climate conditions include, but are not limited to, climate, season, and current weather.

[0207] Secondly, user profiles are constructed based on historical physical condition information, historical cooking recipes, and the climate conditions of the current installation location. The specific process includes: Data processing: The above-mentioned historical physical condition information, historical cooking recipes, and current installation location climate conditions are cleaned and feature extracted; User profile building: First, build a physical profile, a cooking preference profile, and a climate condition profile, and then conduct a comprehensive analysis to obtain a user profile.

[0208] During the construction of the body constitution profile, the user's main body constitution type is determined based on historical body constitution information. Then, based on the body constitution type, the user's potential health needs are inferred. For example, a user with Qi deficiency may need to replenish energy, while a user with Yin deficiency may need ingredients that nourish Yin and moisten dryness. The construction of the cooking preference profile includes extracting the user's preferred ingredients based on historical recipes; extracting the user's preferred flavors, such as spicy, sweet, and salty; extracting the user's preferred cooking methods, such as stir-frying, boiling, steaming, and roasting; and extracting the user's preferred dish types, such as staple foods, soups, and snacks. The climate condition profile includes determining the current season based on the current date; determining the current weather conditions (such as sunny, rainy, or snowy) based on meteorological data; and recording the current temperature and humidity.

[0209] Then, based on the user's physical condition profile and cooking preference profile, dishes that meet the user's health needs and taste are recommended. The recommended dishes are adjusted according to the current climate conditions. For example, warm soups are recommended in the cold winter, and refreshing salads in the hot summer. These steps establish the user profile.

[0210] Optionally, user profiles can be updated periodically to reflect changes in user physical condition and preferences, and recommendations can be dynamically adjusted based on real-time weather conditions.

[0211] S405: If the determination time of the historical physical condition information is within a third preset time period, obtain the historical recommended recipes corresponding to the historical physical condition information and use the historical recommended recipes as the current recommended recipes; if the determination time of the historical physical condition information is not within the third preset time period, generate new recommended recipes based on the historical recommended recipes and the current position.

[0212] In this embodiment, when the user's facial image is not recognized before recipe recommendation, the historical physical condition information and the corresponding historical recommended recipes are loaded based on the time interval between the determination time of the historical physical condition information and the current time. If the time interval is within a third preset time period, the historical physical condition information and the corresponding historical recommended recipes are sent directly to the terminal of the smart steam oven as recommended recipes. Optionally, the third preset time period can be 30-90 days.

[0213] When the time interval exceeds the third preset time, recommendations are made based on the user profile, that is, based on data such as historical recommended recipes and current location.

[0214] This application embodiment sets up a method to determine whether to recommend recipes based on historical physical condition information or user profile when the user's physical condition cannot be determined in real time during the current cooking process. This is beneficial for recommending recipes based on historical habits and regional characteristics when the user's physical condition is unclear, thereby improving the user experience and enhancing the intelligence level of recipe recommendations.

[0215] In one specific embodiment, to reduce the difficulty of interaction and make the smart steam oven suitable for more users, addressing the barriers for middle-aged and elderly users to operate smart devices, the smart steam oven's mobile app supports recipe recommendations via AR scanning of ingredients. The operation process is as follows: the user scans a winter melon, the AR interface displays "cooling in nature, suitable for those with damp-heat constitution," and the recipe "Steamed Winter Melon with Shrimp" automatically pops up. The user taps "start," and if the steam oven recognizes that the ingredients match the recommended recipe, it will automatically cook. Alternatively, the smart steam oven integrates a voice control device that can recognize voice control commands, such as the command: "Steam a little, make a dinner suitable for those with yin deficiency," and then the system automatically selects "Steamed Lily Bulbs with Snow Pear" and starts cooking.

[0216] It should be noted that the tongue features described in the embodiments of this application are equivalent to tongue features.

[0217] The following provides a general description of the recipe recommendation method for the intelligent steam oven provided in the embodiments of this application, such as... Figure 13 As shown: After starting the smart steam oven, the TCM facial diagnosis module is activated. The image acquisition device is used to acquire facial images. Then, based on the constitution recognition model, facial features and tongue features are extracted from the acquired facial images. Finally, the constitution is classified based on the facial and tongue features to obtain constitution information.

[0218] Then, based on the ingredient database, a recommended recipe is automatically generated. Subsequently, when the user puts in the ingredients corresponding to the recommended recipe, the steam oven control module is automatically controlled to cook.

[0219] If the user enters an ingredient that does not match the recommended recipe or enters ingredient information, the system queries the ingredient database for the properties and flavors of the ingredient information. Then, it matches the ingredient database with the user's constitution information and the corresponding preset recommended recipe. The system then adjusts the preset recommended recipe based on the properties and flavors of the ingredients and the user's constitution information to obtain the adjusted preset recommended recipe. The adjusted preset recommended recipe is then used as the recommended recipe, and the cooking process is then executed.

[0220] After cooking is completed, when the triggering conditions for a second face-to-face consultation are met, image information is acquired again to analyze new constitution information. Based on the new constitution information, the effectiveness of the recommended recipe is evaluated, and a decision is made on whether to maintain the original plan or optimize the recipe parameters.

[0221] On the other hand, such as Figure 14 As shown in the illustration, this application also provides a recipe recommendation device for a smart steam oven, the device comprising: The acquisition module 501 is used to acquire the user's facial image; The segmentation module 503 is used to perform region segmentation processing on the facial image to obtain a tongue image; The feature quantization processing module 505 is used to perform feature quantization processing on the tongue image and the facial image respectively based on the constitution recognition model to obtain tongue features and facial features; the constitution recognition model is trained based on different preset tongue images and different preset facial images; The feature fusion module 507 is used to perform feature fusion processing on the tongue features and the facial features to obtain fused features; The identification module 509 is used to perform body constitution identification processing on the fused features based on the body constitution identification model to obtain the user's body constitution information; The recommendation module 511 is used to automatically perform ingredient matching and recipe recommendation processing in the ingredient database based on the physical condition information, and generate recommended recipes; the ingredient database stores preset recommended ingredients and preset recommended recipes corresponding to different preset physical condition information.

[0222] Furthermore, the device also includes: The ingredient properties and flavor verification module is used to respond to the ingredient information input by the user, perform ingredient properties and flavor verification processing on the ingredient information based on the ingredient database, and obtain the ingredient properties and flavors corresponding to the ingredient information. The filtering module is used to filter out corresponding preset recommended recipes from the ingredient database based on the physical condition information and the ingredient information. The parameter adjustment module is used to adjust the cooking parameters of the preset recommended recipe based on the physical condition information and the flavor and taste of the ingredients corresponding to the ingredient information, to obtain the adjusted preset recommended recipe, and to use the adjusted preset recommended recipe as the recommended recipe.

[0223] Furthermore, the device also includes: The re-acquisition module is used to repeat the operation of acquiring the user's facial image until the duration of the physical information reaches a first preset time and / or in response to user feedback, so as to obtain new physical information. The second recommendation module is used to perform ingredient matching and recipe recommendation processing in the ingredient database based on the new physical condition information to obtain new recommended recipes.

[0224] Furthermore, the segmentation module includes: The first detection result acquisition unit is used to perform key point detection and tongue verification processing on the facial image to obtain the first detection result. The second detection result acquisition unit is used to perform background removal processing on the facial image to obtain a second detection result. The tongue image acquisition unit is used to perform region segmentation processing on the facial image based on the first detection result and the second detection result to obtain the tongue image.

[0225] Furthermore, the constitution recognition model includes a tongue image analysis module and a facial analysis module, and the feature quantification processing module includes: The tongue feature quantization processing unit is used to perform tongue feature quantization processing on the tongue image based on the tongue image analysis module to obtain the tongue features; The facial feature quantization processing unit is used to perform facial feature quantization processing on the facial image based on the facial analysis module to obtain the facial features.

[0226] Furthermore, the tongue feature quantization processing unit includes: The tongue region segmentation unit is used to perform region segmentation processing on the tongue image based on the tongue image analysis module to obtain the tongue region. The tongue coating feature determination unit is used to perform color space conversion, grayscale processing, and wavelet crack processing on the tongue body region to obtain tongue coating features; the tongue coating features include at least one of the following features: color features, grayscale features, direction information, high-frequency energy distribution ratio, and crack distribution map. The tongue feature determination unit is used to process the aspect ratio and detect the edge curvature of the tongue region to obtain tongue features. A tongue feature construction unit is used to construct the tongue features based on the tongue coating features and the tongue body features.

[0227] Furthermore, the facial feature quantization processing unit includes: A facial image partitioning processing unit is used to partition the facial image based on facial key points to obtain a partitioned facial image. A contrast determination unit is used to perform contrast analysis on the partitioned facial image based on the facial analysis module to obtain a first facial feature. The periocular region feature determination unit is used to perform local binary processing on the periocular region in the partitioned facial image based on the facial analysis module to obtain periocular region features. A facial color feature determination unit is used to perform facial color feature processing on the partitioned facial image based on the facial analysis module to obtain facial color features; A facial feature construction unit is used to construct the facial features based on the first facial features, the periorbital region features, and the facial color features.

[0228] Furthermore, the device also includes: The training data acquisition unit is used to acquire a preset number of new facial images as training data when the running time of the intelligent steam oven reaches a second preset time; the new facial images carry corresponding physical information, and the new facial image data is acquired through a network connection. The model update unit is used to perform incremental learning training and data balancing on the training data to obtain an updated physical fitness analysis model. A personalized constitution analysis model construction unit is used to acquire historical constitution information from the smart steam oven, and to iteratively train the updated constitution analysis model based on the historical constitution information to obtain a personalized constitution analysis model.

[0229] Furthermore, the device also includes: The first historical data acquisition unit is used to acquire historical physical information and the determination time of historical physical information in response to the start signal of the smart steam oven, without acquiring the facial image. The second historical data acquisition unit is used to acquire historical recommended recipes and the current installation location; The recipe recommendation unit is used to obtain historical recommended recipes corresponding to the historical physical condition information when the determination time of the historical physical condition information is within a third preset time period, and use the historical recommended recipes as the current recommended recipes; when the determination time of the historical physical condition information is not within the third preset time period, it generates new recommended recipes based on the historical recommended recipes and the current position.

[0230] It should be noted that the recipe recommendation device embodiment of the intelligent steam oven provided in this application is based on the same inventive concept as the recipe recommendation method embodiment of the intelligent steam oven described above.

[0231] This application also provides an electronic device for recommending recipes for a smart steam oven. The electronic device includes a processor and a memory. The memory stores at least one instruction or at least one program. The processor loads and executes the at least one instruction or at least one program to implement the recipe recommendation method for the smart steam oven provided in any of the above embodiments.

[0232] On the other hand, this application also provides a smart steam oven, which uses the recipe recommendation method of the smart steam oven described above to recommend recipes.

[0233] Embodiments of this application also provide a computer-readable storage medium that can be disposed in a terminal to store at least one instruction or at least one program for implementing the recipe recommendation method of a smart steam oven in the method embodiments. The at least one instruction or at least one program is loaded and executed by a processor to implement the recipe recommendation method of the smart steam oven provided in the above method embodiments.

[0234] Optionally, in the embodiments of this specification, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0235] The memory described in this specification can be used to store software programs and modules. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for functions, etc.; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0236] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the recipe recommendation method for the intelligent steam oven provided in the above-described method embodiments.

[0237] The methods and embodiments provided in this application can be executed on a terminal, computer terminal, server, or similar computing device. Taking running on a server as an example, Figure 15 This is a hardware structure block diagram of a server for a recipe recommendation method for an intelligent steam oven, provided according to an exemplary embodiment. For example... Figure 15As shown, the server 400 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 410 (CPUs 410 may include, but are not limited to, microprocessors (MCUs) or programmable logic devices (FPGAs), a memory 430 for storing data, and one or more storage media 420 (e.g., one or more mass storage devices) for storing application programs 423 or data 422. The memory 430 and storage media 420 may be temporary or persistent storage. The program stored in the storage media 420 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the CPU 410 may be configured to communicate with the storage media 420 and execute the series of instruction operations stored in the storage media 420 on the server 400. Server 400 may also include one or more power supplies 460, one or more wired or wireless network interfaces 450, one or more input / output interfaces 440, and / or one or more operating systems 421, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, etc.

[0238] The input / output interface 440 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of server 400. In one example, the input / output interface 440 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 440 may be a radio frequency (RF) module used for wireless communication with the Internet.

[0239] Those skilled in the art will understand that Figure 15 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 400 may also include... Figure 15 The more or fewer components shown, or having the same Figure 15 The different configurations shown.

[0240] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0241] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and server embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0242] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0243] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A recipe recommendation method for an intelligent steam oven, characterized in that, The method includes: Obtain the user's facial image; The facial image is segmented to obtain a tongue image; The tongue image and the facial image are subjected to feature quantization processing based on the body constitution recognition model to obtain tongue features and facial features respectively; the body constitution recognition model is trained based on different preset tongue images and different preset facial images; The tongue features and facial features are fused together to obtain fused features; Based on the aforementioned constitution recognition model, the fused features are processed for constitution recognition to obtain the user's constitution information; Based on the physical condition information, the system automatically performs ingredient matching and recipe recommendation processing in the ingredient database to generate recommended recipes; the ingredient database stores preset recommended ingredients and preset recommended recipes corresponding to different preset physical condition information.

2. The recipe recommendation method according to claim 1, characterized in that, The method further includes: In response to the ingredient information input by the user, the ingredient information is processed for ingredient properties and flavor based on the ingredient database to obtain the ingredient properties and flavor corresponding to the ingredient information; Based on the physical condition information and the ingredient information, the corresponding preset recommended recipes are selected from the ingredient database; Based on the physical constitution information and the corresponding flavor and properties of the ingredients, the cooking parameters of the preset recommended recipe are adjusted to obtain the adjusted preset recommended recipe, and the adjusted preset recommended recipe is used as the recommended recipe.

3. The recipe recommendation method according to claim 1, characterized in that, The method further includes: When the duration of the physical fitness information reaches the first preset time and / or in response to user feedback, repeat the operation of acquiring the user's facial image until the user's physical fitness information is obtained, and new physical fitness information is obtained. Based on the new physical condition information, the food ingredient database is used for ingredient matching and recipe recommendation to obtain new recommended recipes.

4. The recipe recommendation method according to claim 1, characterized in that, The process of performing region segmentation on the facial image to obtain a tongue image includes: The facial image is subjected to key point detection and tongue verification to obtain the first detection result; The facial image is subjected to background removal processing to obtain a second detection result; Based on the first detection result and the second detection result, the facial image is segmented to obtain the tongue image.

5. The recipe recommendation method according to claim 1, characterized in that, The constitution recognition model includes a tongue image analysis module and a facial analysis module. The model performs feature quantization processing on the tongue image and the facial image respectively to obtain tongue features and facial features, including: The tongue image is quantized based on the tongue image analysis module to obtain the tongue features; The facial features are obtained by performing facial feature quantization processing on the facial image based on the facial analysis module.

6. The recipe recommendation method according to claim 5, characterized in that, The tongue image is subjected to tongue feature quantization processing based on the tongue image analysis module to obtain the tongue features, including: Based on the tongue image analysis module, the tongue image is segmented to obtain the tongue body region; The tongue region is subjected to color space conversion, grayscale processing, and wavelet crack processing to obtain tongue coating features; the tongue coating features include at least one of the following: color features, grayscale features, directional information, high-frequency energy distribution ratio, and crack distribution map. The aspect ratio of the tongue region is processed and the edge curvature is detected to obtain the tongue features; The tongue features are constructed based on the tongue coating features and the tongue body features.

7. The recipe recommendation method according to claim 5, characterized in that, The facial feature quantization process performed on the facial image based on the facial analysis module to obtain the facial features includes: The facial image is partitioned based on facial key points to obtain a partitioned facial image. Based on the facial analysis module, a contrast analysis is performed on the partitioned facial image to obtain the first facial feature; Based on the facial analysis module, local binary processing is performed on the peri-eye region in the partitioned facial image to obtain peri-eye region features; Based on the facial analysis module, the facial color feature is processed in the partitioned facial image to obtain facial color features; The facial features are constructed based on the first facial features, the periorbital region features, and the facial color features.

8. The recipe recommendation method according to any one of claims 1-7, characterized in that, The method further includes: When the smart steam oven reaches the second preset time, a preset number of new facial images are acquired as training data; the new facial images carry corresponding physical information, and the new facial image data is acquired via network connection. The training data is subjected to incremental learning training and data balancing to obtain an updated physical fitness analysis model. The system acquires historical physical constitution information from the intelligent steam oven, and iteratively trains the updated physical constitution analysis model based on this information to obtain a personalized physical constitution analysis model.

9. The recipe recommendation method according to any one of claims 1-7, characterized in that, The method further includes: In response to the start signal of the smart steam oven, historical physical condition information and the determination time of the historical physical condition information are acquired even without acquiring the facial image. Get historical recommended recipes and the current installation location; If the determination time of the historical physical condition information is within a third preset time period, obtain the historical recommended recipes corresponding to the historical physical condition information, and use the historical recommended recipes as the current recommended recipes; If the time of determining the historical physical condition information is not within the third preset time period, a new recommended recipe is generated based on the historical recommended recipe and the current position.

10. A recipe recommendation device for an intelligent steam oven, characterized in that, The device includes: The acquisition module is used to acquire the user's facial image; The segmentation module is used to perform region segmentation processing on the facial image to obtain a tongue image; The feature quantization processing module is used to perform feature quantization processing on the tongue image and the facial image respectively based on the constitution recognition model to obtain tongue features and facial features; the constitution recognition model is trained based on different preset tongue images and different preset facial images; The feature fusion module is used to perform feature fusion processing on the tongue features and the facial features to obtain fused features; The identification module is used to perform body constitution identification processing on the fused features based on the body constitution identification model to obtain the user's body constitution information; The recommendation module is used to automatically perform ingredient matching and recipe recommendation processing in the ingredient database based on the physical condition information, and generate recommended recipes; the ingredient database stores preset recommended ingredients and preset recommended recipes corresponding to different preset physical condition information.

Images