Automatic cooking method and device and intelligent cooking equipment

Abstract

The invention discloses an automatic cooking method and device and intelligent cooking equipment. The method comprises the steps that food material feature data of a target food material are acquired, and the food material feature data comprise food material visual data, food material freshness data and food material weight data; the target food material is recognized according to the food material feature data through a food material state recognition model, a food material recognition result of the target food material is obtained, the food material state recognition model is a model obtained through machine learning training by using multiple sets of training data, and the food material recognition result comprises a food material name, a food material state and a cuisine label; according to the food material identification result, the historical cooking record and the selectable food materials stored in the intelligent refrigerator, a recommended cuisine is determined; and performing cooking operation on the target food material according to the recommended cuisine. According to the invention, the technical problem that the intelligent cooking equipment in the prior art is low in intelligent degree and cannot fully meet the requirements of individuation, intelligence and health in family cooking is solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent home appliance control technology, and more specifically, to an automatic cooking method and apparatus, and intelligent cooking equipment. Background Technology

[0002] In the field of smart home devices, the development of smart cooking equipment has greatly facilitated home cooking. Traditional cooking equipment is limited to providing basic heating and cooking functions, while modern smart cooking equipment integrates various intelligent functions such as recipe recommendations, automatic seasoning addition, and ingredient recognition, aiming to improve cooking efficiency and user experience. However, existing smart cooking equipment still has shortcomings in terms of recipe recommendations and automatic seasoning addition.

[0003] 1) Limitations of Ingredient Status Recognition and Recipe Recommendation: Current smart cooking devices largely rely on users manually inputting ingredient information or using image recognition technology for basic ingredient identification. However, this method often overlooks the freshness and cut shape of the ingredients, factors that significantly impact the final taste and flavor of the dish. For example, fresh potatoes are suitable for stir-frying, while dehydrated potatoes are better suited for stews. Furthermore, when recommending recipes, the devices often fail to adequately consider the availability of other ingredients in the household, potentially resulting in recipes lacking essential ingredients and thus impracticality.

[0004] 2) Standardization issues in automatic seasoning addition: Existing intelligent cooking equipment often uses standardized recipe proportions for seasoning addition, neglecting the differences in taste preferences among family members. Family members may have different preferences for salty, sweet, spicy, etc. Standardized seasoning addition cannot meet individual needs, resulting in cooking results that may not suit the tastes of some family members. In addition, environmental factors, such as changes in temperature and humidity, and the user's physical condition, such as high blood pressure, diabetes, etc., may also affect the appropriate amount of seasoning to add, but existing equipment does not fully consider these factors.

[0005] 3) Lack of intelligent vegetable cutting function: Although some intelligent cooking devices are equipped with vegetable cutting functions, most are limited to fixed-form cutting, such as simple slicing or dicing, and cannot flexibly adjust cutting parameters according to the needs of different cuisines, such as the thickness of potato shreds or the size of potato chunks. This lack of flexible vegetable cutting function limits the level of intelligence of cooking devices and cannot meet users' personalized needs for the taste and shape of dishes.

[0006] 4) Single dimension of recipe recommendation: The recipe recommendation of existing devices is mainly based on the ingredients input by users or historical cooking records, but it does not fully combine multimodal data (such as ingredient images, weight, smell and other information) for comprehensive analysis, which limits the accuracy and richness of the recommendation results and makes it difficult to meet users' needs for exploring diverse cuisines.

[0007] Therefore, existing technologies have shortcomings in areas such as ingredient status recognition, personalized recipe recommendations, intelligent seasoning control, and flexible vegetable cutting functions, failing to fully meet the personalized, intelligent, and healthy needs of home cooking. Thus, a new intelligent cooking equipment control method is needed to overcome these problems and achieve a more intelligent and personalized home cooking experience.

[0008] There is currently no effective solution to the above problems. Summary of the Invention

[0009] This invention provides an automatic cooking method and apparatus, and an intelligent cooking device, to at least solve the technical problem that the intelligent cooking devices in the related art have a low level of intelligence and cannot fully meet the personalized, intelligent and healthy needs of home cooking.

[0010] According to one aspect of the present invention, an automatic cooking method is provided, comprising: acquiring ingredient feature data of a target ingredient, wherein the ingredient feature data includes: ingredient visual data, ingredient freshness data, and ingredient weight data; identifying the target ingredient based on the ingredient feature data using an ingredient state recognition model to obtain an ingredient recognition result for the target ingredient, wherein the ingredient state recognition model is a model trained by machine learning using multiple sets of training data, and the ingredient recognition result includes: ingredient name, ingredient state, and cuisine label; determining a recommended cuisine based on the ingredient recognition result, historical cooking records, and optional ingredients stored in a smart refrigerator; and performing a cooking operation on the target ingredient according to the recommended cuisine.

[0011] Optionally, acquiring the ingredient characteristic data of the target ingredient includes: capturing the visual data of the target ingredient through an image acquisition device, wherein the visual data includes the following data of the target ingredient: morphological data, color data, and texture data; acquiring the ingredient weight data of the target ingredient through a pressure sensor; and acquiring the ingredient freshness data of the target ingredient through an electronic nose sensor.

[0012] Optionally, acquiring the visual data of the target ingredient includes: performing spatiotemporal feature encoding on multiple frames of images of the target ingredient to obtain ingredient encoding data; extracting geometric topological change information during the deformation process of the target ingredient based on the ingredient encoding data; enhancing the deformation region of the target ingredient based on the geometric topological change information to obtain an enhancement processing result; performing spatial alignment processing on the multiple frames of images to obtain a spatial alignment processing result; and obtaining the visual data of the ingredient based on the enhancement processing result and the spatial alignment processing result.

[0013] Optionally, determining the recommended cuisine based on the ingredient identification results, historical cooking records, and optional ingredients stored in the smart refrigerator includes: determining a user cuisine mapping relationship based on the historical cooking records, wherein the user cuisine mapping relationship is used to record the number of times each user selects each cuisine; determining the associated cuisine of the target ingredient based on the ingredient status and the indicated ingredient name in the ingredient identification results, wherein the associated cuisine is the dish corresponding to the target ingredient; determining the optional ingredients in the smart refrigerator; and determining the recommended cuisine based on the user cuisine mapping relationship, the associated cuisine, and the optional ingredients.

[0014] Optionally, determining the user's cuisine mapping relationship based on the historical cooking records includes: obtaining the original data of the historical cooking records, wherein the original data includes: user identifier and cuisine; performing matrix processing on the original data to obtain the user's cuisine mapping relationship, wherein the rows of the matrix in the matrix processing are the user identifier, and the columns of the matrix are cuisines.

[0015] Optionally, determining the recommended cuisine based on the user cuisine mapping relationship, the associated cuisine, and the optional ingredients includes: determining the similarity between the target and each user in each of the historical cooking records; selecting similar users whose similarity is higher than a similarity threshold; determining an initial recommended cuisine based on the user cuisine mapping relationship and the associated cuisine corresponding to the similar users; and filtering the initial recommended cuisine based on the optional ingredients to obtain the recommended cuisine.

[0016] Optionally, performing a cooking operation on the target ingredient according to the recommended cuisine includes: performing a cutting operation on the target ingredient according to the recommended cuisine to obtain a cut target ingredient; obtaining the taste information of family members; determining the seasoning information required for the cooking operation based on the taste information; and adding seasonings to the cut target ingredient according to the seasoning information to perform the cooking operation on the target ingredient.

[0017] Optionally, determining the seasoning information required for the cooking operation based on the taste information includes: obtaining physiological indicator data of the family members; determining the health risk score of the family members based on the physiological indicator data; determining the upper limit value of the seasoning based on the health risk score; and determining the seasoning information based on the upper limit value of the seasoning and the taste information.

[0018] Optionally, determining the seasoning information required for the cooking operation based on the taste information includes: obtaining environmental parameters of the room where the family member is located; and determining the seasoning information based on the environmental parameters and the taste information.

[0019] Optionally, the automatic cooking method further includes: detecting the current flavor value during the cooking operation of the target ingredient according to the recommended cuisine; and adjusting the seasoning addition method according to the current flavor value.

[0020] According to another aspect of the present invention, an automatic cooking device is also provided, comprising: an acquisition unit for acquiring ingredient feature data of a target ingredient, wherein the ingredient feature data includes: ingredient visual data, ingredient freshness data, and ingredient weight data; an identification unit for identifying the target ingredient based on the ingredient feature data using an ingredient state identification model to obtain an ingredient identification result of the target ingredient, wherein the ingredient state identification model is a model trained by machine learning using multiple sets of training data, and the ingredient identification result includes: ingredient name, ingredient state, and cuisine label; a determination unit for determining a recommended cuisine based on the ingredient identification result, historical cooking records, and optional ingredients stored in a smart refrigerator; and a cooking unit for performing a cooking operation on the target ingredient according to the recommended cuisine.

[0021] Optionally, the acquisition unit includes: a capture module, configured to capture the visual data of the target ingredient using an image acquisition device, wherein the visual data includes the following data of the target ingredient: morphological data, color data, and texture data; a first acquisition module, configured to acquire the ingredient weight data of the target ingredient using a pressure sensor; and a second acquisition module, configured to acquire the ingredient freshness data of the target ingredient using an electronic nose sensor.

[0022] Optionally, the acquisition unit includes: an encoding module for performing spatiotemporal feature encoding on multiple frames of images of the target ingredient to obtain ingredient encoding data; an extraction module for extracting geometric topological change information during the deformation process of the target ingredient based on the ingredient encoding data; an enhancement module for enhancing the deformation region of the target ingredient based on the geometric topological change information to obtain an enhancement result; an alignment module for performing spatial alignment processing on the multiple frames of images to obtain a spatial alignment result; and a third acquisition module for obtaining visual data of the ingredient based on the enhancement result and the spatial alignment result.

[0023] Optionally, the determining unit includes: a first determining module, configured to determine a user cuisine mapping relationship based on the historical cooking records, wherein the user cuisine mapping relationship is used to record the number of times each user selects each cuisine; a second determining module, configured to determine the associated cuisine of the target ingredient based on the ingredient status and the indicated ingredient name in the ingredient identification result, wherein the associated cuisine is the dish corresponding to the target ingredient; a third determining module, configured to determine the optional ingredients in the smart refrigerator; and a fourth determining module, configured to determine the recommended cuisine based on the user cuisine mapping relationship, the associated cuisine, and the optional ingredients.

[0024] Optionally, the first determining module includes: a first acquisition submodule, used to acquire the original data of the historical cooking records, wherein the original data includes: user identifier and cuisine; and a matrix processing submodule, used to perform matrix processing on the original data to obtain the user cuisine mapping relationship, wherein the rows of the matrix in the matrix processing are the user identifier, and the columns of the matrix are cuisines.

[0025] Optionally, the fourth determining module includes: a first determining submodule, used to determine the similarity between the target and each user in each of the historical cooking records; a selecting submodule, used to select similar users whose similarity is higher than a similarity threshold; a second determining submodule, used to determine an initial recommended cuisine based on the user cuisine mapping relationship corresponding to the similar users and the associated cuisine; and a filtering submodule, used to filter the initial recommended cuisine based on the optional ingredients to obtain the recommended cuisine.

[0026] Optionally, the cooking unit includes: a cutting module for performing a cutting operation on the target ingredient according to the recommended cuisine to obtain the cut target ingredient; a third acquisition module for acquiring the taste information of family members; a fifth determination module for determining the seasoning information required for the cooking operation based on the taste information; and an adding module for adding seasonings to the cut target ingredient according to the seasoning information to perform the cooking operation on the target ingredient.

[0027] Optionally, the fifth determining module includes: a second acquiring submodule for acquiring physiological indicator data of the family member; a third determining submodule for determining a health risk score of the family member based on the physiological indicator data; a fourth determining submodule for determining a seasoning upper limit value based on the health risk score; and a fifth determining submodule for determining the seasoning information based on the seasoning upper limit value and the taste information.

[0028] Optionally, the fifth determining module includes: a third acquiring submodule, used to acquire environmental parameters of the room where the family member is located; and a sixth determining submodule, used to determine the seasoning information based on the environmental parameters and the taste information.

[0029] Optionally, the automatic cooking device further includes: a detection unit for detecting the current flavor value during the cooking operation of the target ingredient according to the recommended cuisine; and an adjustment unit for adjusting the seasoning addition method according to the current flavor value.

[0030] According to another aspect of the present invention, an intelligent cooking device is also provided, which uses the automatic cooking method described in any one of the above embodiments.

[0031] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein the program executes any one of the automatic cooking methods described herein.

[0032] According to another aspect of the present invention, a processor is also provided, the processor being configured to run a program, wherein the program, when running, executes the automatic cooking method described in any of the preceding embodiments.

[0033] According to another aspect of the present invention, a computer program product is also provided, including computer instructions that, when executed by a processor, perform the automatic cooking method described in any one of the above embodiments.

[0034] In this embodiment of the invention, the ingredient feature data of the target ingredient is obtained, including: ingredient visual data, ingredient freshness data, and ingredient weight data. The target ingredient is identified using an ingredient state recognition model based on the ingredient feature data, yielding an ingredient identification result. This ingredient state recognition model is trained using multiple sets of training data through machine learning. The ingredient identification result includes: ingredient name, ingredient state, and cuisine label. A recommended cuisine is determined based on the ingredient identification result, historical cooking records, and available ingredients stored in the smart refrigerator. Cooking operations are then performed on the target ingredient according to the recommended cuisine. The technical solution provided by this invention achieves the goal of identifying the type, state, and cuisine label of ingredients, and determining a recommended cuisine based on the identification result combined with historical cooking records and available ingredients stored in the smart refrigerator. This enables the cooking equipment to proactively identify and recommend suitable cuisines, and to adjust according to the food reserves in the refrigerator. It also improves the intelligence level of the cooking equipment, providing more personalized and comprehensive cuisine recommendations, improving the user experience, and thus solving the technical problem that related technologies often have low levels of intelligence in smart cooking equipment, failing to fully meet the personalized, intelligent, and healthy needs of home cooking. Attached Figure Description

[0035] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0036] Figure 1 This is a hardware structure block diagram of a mobile terminal for an automatic cooking method according to an embodiment of the present invention.

[0037] Figure 2 This is a flowchart of an automatic cooking method according to an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram illustrating the construction process of a corruption knowledge graph according to an embodiment of the present invention;

[0039] Figure 4 This is a flowchart of food ingredient status recognition and intelligent cuisine recommendation according to an embodiment of the present invention;

[0040] Figure 5 This is a flowchart of the seasoning process according to an embodiment of the present invention;

[0041] Figure 6 This is a flowchart of an optional seasoning process according to an embodiment of the present invention;

[0042] Figure 7 This is a schematic diagram of an automatic cooking device according to an embodiment of the present invention.

[0043] The above figures include the following reference numerals:

[0044] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0045] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0047] As described in the background section, existing intelligent cooking devices have a low level of intelligence and cannot fully meet the personalized, intelligent, and healthy needs of home cooking. This invention provides an automatic cooking method and apparatus, an intelligent cooking device, a computer-readable storage medium, a processor, and a computer program product.

[0048] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0049] The methods and embodiments provided in this invention can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for an automatic cooking method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0050] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the automatic cooking method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0051] Example 1

[0052] According to an embodiment of the present invention, an embodiment of an automatic cooking method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0053] Figure 2 This is a flowchart of an automatic cooking method according to an embodiment of the present invention, such as... Figure 2 As shown, the automatic cooking method includes the following steps:

[0054] Step S202: Obtain the ingredient feature data of the target ingredient, wherein the ingredient feature data includes: ingredient visual data, ingredient freshness data, and ingredient weight data.

[0055] Optionally, the target ingredients here refer to any food ingredients that the user plans to use for cooking, including but not limited to vegetables, meat, fish, beans, grains, etc.

[0056] Optionally, the aforementioned food characteristic data may include, but is not limited to: food visual data, food freshness data, and food weight data.

[0057] The aforementioned visual data of the ingredients can be captured by a high-resolution RGB-D camera (at least 200fps) to determine the basic shape (such as potato chunks or shredded potatoes), color details, and surface texture of the ingredients, in order to determine their condition (fresh, dehydrated, rotten, etc.). For example, the surface of fresh potatoes may appear uniformly light yellow, while dehydrated potatoes may be darker and have cracks.

[0058] The above-mentioned food freshness data can be obtained by using an electronic nose sensor (containing at least elements sensitive to spoilage markers such as ammonia, hydrogen sulfide, and ketones) to detect the odor of the food, with a threshold set at a concentration change of 0.05%, which can promptly identify whether the food has spoiled.

[0059] The weight data of the ingredients can be accurately measured by an integrated array of four corner pressure sensors, each with an accuracy of 0.1g, to identify the size and quantity of the ingredients and thus determine whether they are suitable for a specific cuisine.

[0060] Step S204: The target ingredient is identified by the ingredient status recognition model based on the ingredient feature data to obtain the ingredient recognition result. The ingredient status recognition model is a model trained by machine learning using multiple sets of training data. The ingredient recognition result includes: ingredient name, ingredient status and cuisine label.

[0061] Optionally, the aforementioned food ingredient status recognition model is a model trained using deep learning technology. It can identify the status of food ingredients based on collected multimodal data (visual, odor, weight), including ingredient name, status (freshness, cutting shape, etc.), and cuisine recommendation tags. Its training method involves learning from multiple sets of training data. This data needs to cover a wide range of food types, different statuses, and corresponding cuisines to ensure the model has broad applicability and a high recognition rate. Model output: For each food ingredient, the model output includes, but is not limited to, tags such as "potato chunks - braised" and "shredded potatoes - stir-fried," helping the device accurately recommend cuisines.

[0062] Furthermore, deep learning frameworks (such as CNN+LSTM) are used for model training, and transfer learning strategies are employed to reduce reliance on large amounts of training data, particularly optimizing recognition performance in small sample scenarios. For example, a multi-state database of ingredients with labeled samples (covering cutting shape, spoilage level, etc.) is constructed using deep learning CNN+Transformer. Transfer learning is then used to optimize small sample scenarios, outputting ingredient names, states, and available cuisine labels (e.g., "potato chunks - braised" and "shredded potatoes - stir-fried"). The model must include three sub-tasks: ingredient name recognition, state analysis (fresh, dehydrated, etc.), and cuisine label prediction, ensuring the fusion processing of multimodal data. After ingredient recognition, collaborative filtering algorithms are used to analyze users' historical cooking records, combined with smart refrigerator data. A multi-objective optimization model comprehensively considers ingredient states, user preferences, and refrigerator inventory to generate a recommended cuisine list. Each cuisine must have at least three ingredients with a matching degree exceeding 80% to ensure the feasibility of the recommended cuisines.

[0063] Step S206: Determine the recommended cuisine based on the ingredient recognition results, historical cooking records, and optional ingredients stored in the smart refrigerator.

[0064] Optionally, the system will collect and store data such as the user's past recipe selections, seasoning preferences, and cooking frequency to obtain the aforementioned historical cooking records: and build a user taste preference model using this data.

[0065] For example, collaborative filtering or matrix factorization techniques can be used to analyze a user's preference for a specific cuisine. If user U1 selects Sichuan cuisine 150 times, accounting for 60% of the total cooking times, it indicates that the user may have a high preference for Sichuan cuisine.

[0066] Furthermore, in this embodiment of the invention, the smart refrigerator can realize the linkage analysis of food data. The smart refrigerator is equipped with IoT sensors, which can monitor and record the type, quantity and freshness of the food in the refrigerator in real time.

[0067] Furthermore, the smart cooking equipment and smart refrigerator are connected via wireless network to exchange ingredient information in real time, ensuring that the recommended recipes not only suit your taste but also take into account the amount of ingredients you have at home, thus avoiding the recommendation of dishes that are not practical to cook.

[0068] In this embodiment, taking into account the food identification results (name, status), historical cooking records (user taste preferences), and optional food stored in the smart refrigerator, a personalized and feasible cuisine recommendation list is generated through a multi-objective optimization sorting algorithm.

[0069] For example, cuisine recommendations can be made through the following process: 1) Ingredient matching: Based on the ingredient identification results, select cuisines that match the ingredient status. 2) User preference weighting: Use historical cooking records to weight the cuisines, prioritizing those frequently chosen by the user. 3) Inventory consideration: Link with smart refrigerator data to prioritize recipes that can be made using existing ingredients. 4) Comprehensive ranking: Through a multi-objective optimization model, rank the matched cuisines based on various factors (such as user preference, ingredient matching degree, health considerations, etc.), and output the final recommendation list.

[0070] By employing deep learning and multimodal data fusion technologies, the accuracy of ingredient recognition and cuisine recommendation in smart cooking equipment has been significantly improved, enabling automated and personalized cooking operations.

[0071] Step S208: Perform cooking operations on the target ingredients according to the recommended cuisine.

[0072] In this embodiment, the intelligent cooking device can automatically adjust cooking parameters, such as cooking time, temperature, and amount of seasonings, according to the recommended cuisine to match the characteristics of the selected cuisine and ensure the taste and flavor of the cooking result.

[0073] As described above, in this embodiment of the invention, the target ingredient's characteristic data is obtained, including visual data, freshness data, and weight data. The target ingredient is then identified using an ingredient state recognition model based on this characteristic data, yielding an ingredient identification result. This model is trained using multiple sets of training data via machine learning, and the identification result includes the ingredient name, state, and cuisine label. A recommended cuisine is determined based on the identification result, historical cooking records, and available ingredients stored in the smart refrigerator. Cooking is then performed on the target ingredient according to the recommended cuisine. This process achieves the goal of identifying the ingredient's type, state, and cuisine label, and determining the recommended cuisine based on the identification result, historical cooking records, and available ingredients stored in the smart refrigerator. This allows the cooking equipment to proactively identify and recommend suitable cuisines, adjust its operation based on the refrigerator's food reserves, and enhances the intelligence of the cooking equipment, providing more personalized and comprehensive cuisine recommendations and improving the user experience.

[0074] Therefore, the technical solution provided by the embodiments of the present invention solves the technical problem that the intelligent cooking equipment in the related art has a low level of intelligence and cannot fully meet the personalized, intelligent and healthy needs of home cooking.

[0075] According to the above embodiments of the present invention, obtaining the ingredient characteristic data of the target ingredient includes: capturing visual data of the target ingredient through an image acquisition device, wherein the visual data includes the following data of the target ingredient: morphological data, color data, and texture data; obtaining the ingredient weight data of the target ingredient through a pressure sensor; and obtaining the ingredient freshness data of the target ingredient through an electronic nose sensor.

[0076] Optionally, the aforementioned image acquisition device may include, but is not limited to: a high-resolution camera, or a dual camera consisting of a visible light and a near-infrared (NIR) camera.

[0077] Specifically, high-resolution cameras can capture the shape (such as potato chunks / shreds / mashed), color, and texture of ingredients; pressure sensors can determine the softness or hardness of ingredients (such as fresh / dehydrated potatoes); and electronic nose sensors can detect the freshness of ingredients (such as whether they have spoiled).

[0078] For example, high-resolution cameras, pressure sensors, and electronic nose sensors can be integrated into the system to ensure comprehensive and accurate data collection. The camera must support a frame rate of at least 200fps to capture details of dynamic changes in the ingredients; the pressure sensor must have an accuracy of at least 0.1g to distinguish weight differences between different ingredients; and the electronic nose sensor should be highly sensitive enough to detect VOCs (volatile organic compounds) with a concentration change of 0.01% to accurately assess the freshness of the ingredients.

[0079] Therefore, in this embodiment, visual data acquisition is performed using an RGB-D camera to ensure the acquisition of the shape, color, and depth information of the ingredients, with a frame rate of at least 100fps to capture dynamic changes in the ingredients. Pressure sensor integration involves installing four high-precision (0.1g) pressure sensors below the ingredient placement area of ​​the smart cooking device, forming a sensor array to improve the accuracy of weight data through averaging. Freshness data detection utilizes an electronic nose sensor equipped with elements sensitive to specific spoilage markers, such as alcohols and aldehydes, with a detection threshold set at a 0.05% change in VOC concentration to achieve real-time monitoring of ingredient freshness.

[0080] This integrated multimodal data acquisition scheme ensures the comprehensiveness of ingredient characteristic data, providing high-quality data input for subsequent ingredient status identification. By detecting ingredient freshness in real time, the intelligent cooking equipment can adjust recipe recommendations promptly, avoiding the use of stale ingredients and ensuring dish quality.

[0081] According to the above embodiments of the present invention, obtaining visual data of a target food ingredient includes: performing spatiotemporal feature encoding on multiple frames of images of the target food ingredient to obtain food ingredient encoding data; extracting geometric topological change information during the deformation process of the target food ingredient based on the food ingredient encoding data; performing enhancement processing on the deformation region of the target food ingredient based on the geometric topological change information to obtain enhancement processing results; performing spatial alignment processing on multiple frames of images to obtain spatial alignment processing results; and obtaining food ingredient visual data based on the enhancement processing results and the spatial alignment processing results.

[0082] In this embodiment, dynamic deformation modeling captures the shape, color, and texture of food ingredients. Spatiotemporal feature encoding is performed on multiple frames of images continuously acquired by a high-resolution camera to extract the geometric topological changes during the deformation process of the food ingredients. An attention weight calculation layer based on material mechanics priors (such as curvature-sensitive convolutional kernels) is embedded in the network to enhance the local features of key deformation areas. Data is simultaneously acquired by two cameras, one visible light and one near-infrared (NIR). The feature space is aligned using a cross-modal contrast loss function to eliminate the interference of illumination changes on color recognition, thereby achieving the acquisition of shape, color, and texture data.

[0083] Specifically, by calculating key mechanical quantities (curvature, strain) on the surface of the food material, these are transformed into a spatial attention weight map and applied to the network feature map, forcing the model to focus on the key deformation regions indicated by physical laws. The processing procedure of the attention weight calculation layer for the above-mentioned material mechanics prior is as follows:

[0084] Step 1: Calculate curvature and strain / strain energy density using the input image data (usually depth map, point cloud, or pre-processed geometric feature map): (1) Curvature: Apply differential geometry methods (such as fitting a surface based on neighborhood points) to calculate the Gaussian curvature, mean curvature, or principal curvature at each point or pixel location, generating a curvature feature map with the same spatial dimensions as the input image, where the numerical value represents the degree of curvature at that location. (2) Strain / strain energy density: Calculate the optical flow field or a more precise deformation displacement field using consecutive frame images. By spatially differentiating the displacement field (calculating the displacement gradient), the surface strain tensor can be estimated, thereby obtaining the strain magnitude or strain energy density and generating a dynamic strain feature map.

[0085] Step 2: Input the physical feature maps (curvature maps, strain maps, etc.) calculated in Step 1 into a dedicated attention weight generation module: map the original values ​​of the physical features to a suitable range, and apply an activation function (such as Sigmoid) to transform the physical feature values ​​into attention weight values. The key is to design the mapping relationship so that regions with high physical quantity values ​​(high curvature, high strain) receive higher attention weights (close to 1), and regions with low physical quantity values ​​receive lower weights (close to 0), achieving a nonlinear mapping.

[0086] Step 3: Apply attention weights to the feature map: Obtain the image feature map (containing spatial and channel information) learned by the current layer of the network, and apply the physical prior-based attention weight map (with the same spatial size as the feature map) generated in Step 2 to the feature map and multiply it element by element. That is, the feature vector of each spatial location on the feature map is multiplied by the attention weight value of the corresponding location.

[0087] Furthermore, the process of enhancing the local features of key deformation regions is as follows:

[0088] Step 1: Locate key regions using attention masks: Based on the physical prior attention weight map, set a threshold (or directly use weight values) to generate a binary attention mask, which clearly identifies the "deformation key regions" (high weight regions) that the network considers to be key regions.

[0089] Step 2: Multi-scale Feature Extraction and Fusion (for Key Regions): Features are extracted from different layers of the neural network (or the same layer using different parameters). Shallow networks typically capture detailed textures (small-scale features), while deep networks capture more abstract semantics and overall structure (large-scale features). Using the attention mask from Step 1, the network is guided to perform more refined multi-scale feature extraction in key regions, applying additional, denser convolutional operations in these regions. Alternatively, during feature fusion (as in the Feature Pyramid Network (FPN), special emphasis is placed on multi-scale feature fusion in key regions, effectively combining shallow features describing micro-texture changes with deep features describing macro-deformation within these regions.

[0090] Step 3: Local Feature Contrast Learning: Construct positive and negative sample pairs and design a loss function with the goal of maximizing the similarity between features of positive sample pairs (bringing them closer) while minimizing the similarity between features of negative sample pairs.

[0091] Step 4: Cross-modal feature alignment: Design two parallel network branches (or branches sharing some weights) to process RGB and NIR images respectively. The RGB branch focuses on extracting appearance features such as color and surface texture. The NIR branch is insensitive to changes in illumination and can penetrate the surface to obtain information closely related to deformation, such as the internal structure and moisture distribution of the food. Key region feature alignment is achieved by defining a cross-modal contrastive loss function or a feature distance minimization loss.

[0092] By performing spatiotemporal feature encoding on multiple frames of food images, specific convolutional kernels (such as curvature-sensitive convolutional kernels) are used to enhance the features of key deformation regions. The food state recognition model needs to learn from at least 1000 types of food to ensure its broad applicability and high recognition rate. Here, the spatiotemporal feature encoding and key region enhancement of the deep learning model significantly improve the accuracy and speed of food state recognition, providing crucial information for subsequent cuisine recommendations.

[0093] In addition, in this embodiment of the invention, the multimodal tactile food hardness detection is mainly achieved through a multimodal tactile sensing array and an impedance transfer learning module. The surface of the biomimetic tactile sensing array is covered with a microstructure biomimetic coating to enhance the sensitivity to microscopic deformation of the food surface. A food mechanical impedance spectrum library is constructed, which includes parameters such as pressure-deformation curves and frequency response. A meta-learning framework is used to adapt the softness and hardness assessment of food with different densities.

[0094] The process of electronic nose sensor detecting the freshness of food (e.g., whether it has spoiled) is as follows: (1) Target VOCs are adsorbed through microfluidic gas enrichment channels, and the separation column is modulated by temperature difference to achieve material stratification. (2) Differentiated sensitive coatings are designed for sulfides, amines, aldehydes, etc., and different VOCs are stratified according to boiling point differences. (3) Based on the type of food input by the user, the spoilage path of related data is extracted from the pre-constructed spoilage knowledge graph: oxidative spoilage → increase in trimethylamine (TMA) and dimethylamine (DMA) concentrations. (4) Graph neural network (GNN) receives the original data from the sensor and the topology of the knowledge graph to realize node feature updates and edge weight adjustments. Gated graph attention layer (GGAT) is used to suppress the propagation weight of irrelevant material nodes (e.g., ethanol). Finally, the spoilage probability is output based on the multi-index fusion score: Where, ω i C represents the material weights output by the GNN. i For the actual concentration, C i，thresh The threshold for the knowledge graph.

[0095] The processing procedure for the aforementioned corruption knowledge graph is as follows:

[0096] Step 1: Quantification of putrefaction pathways: Transforming abstract concepts such as "oxidative putrefaction" into: [temperature and humidity thresholds] + [VOCs concentration curves] + [microbial growth models];

[0097] Step 2: Cross-modal data fusion: Establishing a joint decision boundary between morphological parameters (local circularity) and VOCs concentration;

[0098] Step 3: Dynamic scalability When adding new food types (such as artificial meat), simply inject experimental data → automatically expand the graph topology.

[0099] Figure 3 This is a schematic diagram illustrating the construction process of a corruption knowledge graph according to an embodiment of the present invention, such as... Figure 3 As shown, for different types of food, their spoilage paths and morphological characteristics can be determined. After determining the spoilage path, key spoilage bacteria and characteristic VOCs are identified. Key spoilage bacteria can be determined based on optimal temperature and humidity. Characteristic VOCs can be detected using threshold detection. Morphological characteristics are determined by local roundness to obtain the burr index.

[0100] Furthermore, it should be noted that in this embodiment of the invention, after obtaining the multimodal dataset, preprocessing of the multimodal dataset is required. This mainly includes: image denoising: nonlocal means denoising algorithm based on OpenCV; background segmentation: Mask R-CNN segmentation of the main food region; morphology quantization: calculation of area, perimeter, and roundness (distinguishing between blocky / sheet-like shapes).

[0101] Morphological quantification can be achieved through the following methods:

[0102] Step 1: Segment the main food region using Mask R-CNN and output a binary mask;

[0103] Step 2: Calculate the area, perimeter, and radius of the target region, and perform polygonal approximation smoothing on the contour. The calculation is as follows: Area: The total number of pixels in the target region in the binary image, reflecting the physical size of the food. The calculation formula is: Area = ∑ (x,y)∈Mask 1 (pixel level); Perimeter: Based on the Douglas-Peucker algorithm, the contour polygon approximation balances accuracy and efficiency. Alternative algorithms include the simple chain code method (for 8-connected contours, horizontal / vertical step size is 1, diagonal is √2, and the step size is accumulated) or the Freeman chain code method (using directional encoding (0-7) to represent adjacent pixel positions, and then calculating the total length after decoding). Roundness: Measures how close the target shape is to a circle. Blocky foods (such as potatoes) have a roundness close to 1, while sliced ​​foods (such as cucumber slices) have a roundness close to 0. The calculation formula is as follows: The area is defined as follows: the total number of pixels enclosed within the target contour (or the actual physical area). It is calculated by using the contour pixel coordinates to calculate the polygon area (e.g., the Shoelace formula) or by counting the number of foreground pixels in a binary image. The perimeter is the total length of the closed boundary of the target contour. It is calculated by approximating the contour as a polygon and then summing the edge lengths. Simultaneously, for ingredients with different slice thicknesses (e.g., cross / vertical slices of carrots), local roundness is calculated and the median is used for multi-scale analysis. For ingredients with burrs on the edges (e.g., broccoli), a closing operation is performed to smooth the contour, achieving morphological preprocessing.

[0104] According to the above embodiments of the present invention, determining a recommended cuisine based on the ingredient identification result, historical cooking records, and optional ingredients stored in the smart refrigerator includes: determining a user cuisine mapping relationship based on historical cooking records, wherein the user cuisine mapping relationship is used to record the number of times each user selects each cuisine; determining the associated cuisine of the target ingredient based on the ingredient status and the indicated ingredient name in the ingredient identification result, wherein the associated cuisine is the dish corresponding to the target ingredient; determining optional ingredients in the smart refrigerator; and determining a recommended cuisine based on the user cuisine mapping relationship, the associated cuisine, and the optional ingredients.

[0105] In this embodiment, a user-cuisine matrix is ​​established based on the user's historical cooking records. Matrix factorization techniques such as SVD (Singular Value Decomposition) or NMF (Non-negative Matrix Factorization) are used to mine the user's preference patterns for different cuisines. In other words, the user-cuisine mapping relationship is matrixed.

[0106] For example, collaborative filtering algorithms can be used to analyze users' historical cooking records (such as frequently selected "Sichuan cuisine" and "stewed dishes"); knowledge graph reasoning can be used to associate cuisines based on the state of ingredients (such as "potato chunks + ribs → braised / stewed soup"); and information about other ingredients in the refrigerator can be obtained through IoT interfaces to prioritize recipes that can be fully prepared, i.e., refrigerator inventory can be linked. Dynamic cuisine recommendations can be achieved through these methods.

[0107] According to the above embodiments of the present invention, determining the user's cuisine mapping relationship based on historical cooking records includes: obtaining the original data of the historical cooking records, wherein the original data includes: user identifier and cuisine; performing matrix processing on the original data to obtain the user's cuisine mapping relationship, wherein the rows of the matrix in the matrix processing are user identifiers, and the columns of the matrix are cuisines.

[0108] In this embodiment, a user-cuisine interaction matrix is ​​constructed and weighted. That is, the user-cuisine mapping relationship can be accomplished as follows: The raw data of the user's historical cooking records is obtained and processed into a matrix, where rows represent users (u∈U) and columns represent cuisine labels (c∈C, such as "Sichuan cuisine", "Cantonese cuisine", "stewed dishes"). Matrix elements: R u，c = log(number of times the user selected cuisine c + 1). Assuming user U1 has cooked Sichuan cuisine 3 times and stew 1 time, then...

[0109] The process employs a cold start approach for new users and new cuisines: new users are supplemented using a knowledge graph (e.g., users fill in their taste preferences → mapped to the initial weight of the cuisine); new cuisines are assigned initial values ​​based on the correlation between ingredients (e.g., if "pickled fish" does not appear, the weight of "Sichuan cuisine" is associated with "fish + pickled cabbage").

[0110] Optionally, the recommended cuisine is determined based on the user's cuisine mapping relationship, associated cuisines, and available ingredients, including: determining the similarity between the target and each user in each historical cooking record; selecting similar users with similarity higher than a similarity threshold; determining the initial recommended cuisine based on the user's cuisine mapping relationship and associated cuisines corresponding to the similar users; and filtering the initial recommended cuisine based on available ingredients to obtain the final recommended cuisine.

[0111] In this embodiment, by calculating the similarity between the user and users in the historical records, users with a similarity greater than 0.7 are selected as references. Combining the cuisine tags of the target ingredients and the food inventory of the smart refrigerator, a recommended cuisine list is generated, with each cuisine having a recommendation rate of at least 85%.

[0112] Furthermore, in this embodiment of the invention, user similarity is calculated by combining a time decay factor, that is, an improved cosine similarity is used to address differences in rating scales: If the user selected "Sichuan cuisine" 3 days ago and λ = 0.9, then the original rating R u,川菜 =1.2R u,川菜 The attenuation is 1.2 × 0.9 3 ≈0.874.

[0113] Next, a nearest neighbor selection method (Top-K similar user selection) and clustering noise reduction are employed to reduce computational complexity while maintaining recommendation accuracy. By calculating the improved cosine similarity (including time decay factor) between the target user and all users, valid users with a similarity ≥ 0.3 are retained. Users are sorted in descending order of similarity, and the Top-K (e.g., K = 20) users are selected to form the nearest neighbor set N(u), eliminating low-similarity noise interference. K-means is used to perform coarse-grained clustering on all users, and the similarity of the target user is calculated only with users within the same cluster. Top-K selection is then performed within each cluster.

[0114] The method for generating the candidate set of recommended dishes is as follows:

[0115] Step 1: Predict user interest in cuisines they haven't tried:

[0116] N(u) is the Top-K nearest neighbor set of user u.

[0117] Step 2: Cuisine Ranking and Filtering: Limit the number of recommended dishes from the same cuisine, aiming to recommend no more than two Sichuan dishes. Incorporate a surprise factor and implement inventory-based filtering to remove dishes lacking key ingredients (e.g., if "beef" is unavailable, filter "beef stew with potatoes," etc.): S c = User u's acceptance level / Cuisine c's unpopularity level, unpopularity level = 1 - popularity level / max(popularity level).

[0118] Step 3: Output a real-time recommendation list and user feedback for online learning and updates: sorted in descending order, and output Top-N recommendations based on diversity rules. At the same time, according to the user feedback mechanism, the weight of user feedback for online learning and updates is applied, including explicit feedback such as collection and rating, as well as implicit feedback such as cooking time and remaining ingredients. The calculation can be referred to as follows: Implicit score = learning degree factor × (1 - remaining ingredients / total amount required for the recipe).

[0119] Finally, the top 3 recommended cuisines are output, along with a list of required seasonings and hints for missing ingredients.

[0120] Based on the real-time inventory of food in the smart refrigerator, the recommended menu is filtered to remove dishes that are missing key ingredients. Then, through a multi-objective optimization model, user preferences and refrigerator inventory are comprehensively considered to rank and generate the final recommended menu, ensuring that the feasibility of each recommended menu reaches more than 95%.

[0121] The aforementioned approach, by establishing user-specific cuisine mapping relationships and applying similarity algorithms, enhances the personalization and accuracy of cuisine recommendations. The integration with the smart refrigerator ensures that recommended dishes not only match the user's taste but are also immediately actionable, preventing the awkward situation of users being unable to cook recommended dishes due to insufficient ingredients.

[0122] Figure 4 This is a flowchart of ingredient status recognition and intelligent cuisine recommendation according to an embodiment of the present invention, such as... Figure 4 As shown, after the user puts in the ingredients, multimodal data is collected and preprocessed; state label recognition is performed based on the multimodal fusion recognition and detection model; then, candidate recipes are matched with the knowledge graph; and the user selects a recipe by combining collaboration, inventory filtering, and multi-objective optimization ranking; after the recipe is selected, the automatic cutting device cuts the vegetables; and after the vegetables are deemed to be cut to standard, the cooking process begins.

[0123] According to the above embodiments of the present invention, performing a cooking operation on a target ingredient according to a recommended cuisine includes: performing a cutting operation on the target ingredient according to the recommended cuisine to obtain a cut target ingredient; obtaining the taste information of family members; determining the seasoning information required for the cooking operation based on the taste information; and adding seasonings to the cut target ingredient according to the seasoning information to perform a cooking operation on the target ingredient.

[0124] In this embodiment, the cutting form can be switched (e.g., rotating blade for shredding, flat blade for dicing) using a robotic arm and a multi-blade module; cutting parameters can be automatically matched based on the cuisine requirements (e.g., potato shreds 0.2mm thick, potato chunks 1cm thick). 3Specifically: A high frame rate camera (200fps) + LED ring light source monitors the position and shape of ingredients in real time; based on the cuisine selected by the user, cutting parameters are extracted from the database (threshold dynamically adjusted range), and the parameter library supports fuzzy matching (e.g., "potato chunks" default to 1cm). 3 However, the "braised potatoes" setting automatically adjusted to 1.5cm. 3 The system uses Mask R-CNN to segment the main food region, calculates 3D point clouds, and employs grid path planning to achieve adaptive parameter adjustment. For soft ingredients, a vibration-based algorithm reduces blade adhesion, and real-time cutting is performed. If the user reports a cutting deviation, the recipe parameter database is updated based on the user's feedback score, allowing the system to learn from user habits. Here, the cutting parameters of the vegetable cutter are automatically adjusted based on recommended cuisine and ingredient conditions; for example, the thickness of potato shreds is set to 0.2mm, and the size of potato chunks is set to 1.5cm. 3 Ensure that the ingredients are cut in accordance with the requirements of the cuisine.

[0125] In addition, information about family members' tastes can be obtained in the following ways:

[0126] 1) Family member taste profile modeling: 2) Personalized preference collection: Explicit input: User sets salty / sweet / spicy preference level (1-5 points); Implicit learning: Infer preferences by the amount of seasoning added in historical cooking records (e.g., the user often adds 10% salt).

[0127] The creation of family member taste profiles can be achieved as follows: Users can set taste preference levels through the app interface or other display methods, which are then converted into a baseline weight vector. For example, if a user inputs "salty 4 points," "sweet 3 points," and "spicy 5 points," then... base = [0.8, 0.6, 1.0]; In addition, the implicit preference learning algorithm extracts the ratio of the actual amount of seasoning used to the standard amount of the recipe from historical cooking records, and uses the exponential moving average (EMA) to update the implicit weights.

[0128] According to the above embodiments of the present invention, determining the seasoning information required for cooking operations based on taste information includes: obtaining physiological indicator data of family members; determining the health risk score of family members based on the physiological indicator data; determining the upper limit value of seasoning based on the health risk score; and determining the seasoning information based on the upper limit value of seasoning and taste information.

[0129] In this embodiment, the system can access the smart bracelet / health record API to obtain indicators such as the user's blood pressure and blood sugar, and dynamically limit the amount of high salt / high sugar: calculate the health risk coefficient score, calculated as: risk score = σ(0.3*blood sugar + 0.5*blood pressure - b), and trigger an alarm when the risk score > 0.7 to dynamically limit the upper limit of seasonings.

[0130] By collecting family members' taste preferences through the user interface and combining them with health data (blood pressure, blood sugar, etc.), a multi-objective optimization model is used to automatically generate seasoning recipes, ensuring that the amount of seasoning is within the healthy range while meeting the taste needs of family members. The seasoning adjustment range is set to ±20% of the standard amount in the recipe.

[0131] The adaptive cutting operation and personalized seasoning configuration here enable automation and personalization of the cooking process, improving cooking efficiency and dish quality.

[0132] According to the above embodiments of the present invention, determining the seasoning information required for cooking operations based on taste information includes: obtaining environmental parameters of the room where family members are located; and determining the seasoning information based on the environmental parameters and taste information.

[0133] In this embodiment, multi-objective seasoning optimization can be achieved: (1) Weight allocation algorithm: based on the number of family members dining on the day and their weights (e.g., children's taste priority coefficient 0.6, elderly health restriction coefficient 0.8); (2) Environmental factor compensation: call the weather API to obtain temperature and humidity data, and dynamically adjust the flavor concentration (e.g., when humidity > 80%, the umami flavor is increased by 15%). The multi-objective optimization formula is as follows: α i It represents the member weights, where T represents dimensions such as salty / sweet / spicy, and β represents the health penalty.

[0134] Furthermore, in the multi-objective seasoning optimization, the weight allocation for family members is based on priority coefficients assigned according to their dining roles, calculated using the following formula: Joint weighting of health roles: r i τ represents the inherent priority of the role, exp represents the health risk inhibition term, and τ represents the moderating sensitivity to health impacts.

[0135] The above-mentioned parameters of home environment temperature and humidity are obtained and compensated accordingly. Compensation is given for both the cooking degree of the ingredients and the user's taste. The multi-objective loss function is as follows:

[0136] Finally, the user feedback closed-loop system uses both a visible user correction deviation amount and an implicit dual learning mechanism to adjust and update parameters, ultimately providing health dynamic adjustment risk score parameters.

[0137] The adjustment of the above environmental parameters ensures that dishes can still be cooked to the expected taste under different environmental conditions, thus enhancing the environmental adaptability and user satisfaction of intelligent cooking equipment.

[0138] According to the above embodiments of the present invention, the automatic cooking method further includes: detecting the current flavor value during the cooking operation of the target ingredients according to the recommended cuisine; and adjusting the seasoning addition method according to the current flavor value.

[0139] During the cooking process, the system needs to dynamically adjust the amount of seasonings added to ensure that the real-time ion concentration of the broth (such as saltiness, sweetness, and umami) is as close as possible to the preset target curve. The target curve is generated by a multi-objective optimization model, which comprehensively considers the taste preferences of family members, health constraints, and environmental compensation factors.

[0140] Specifically, (1) Electronic tongue dynamic feedback: detect the ion concentration of soup in stages during the cooking process (e.g., stewing for 10 min / 20 min); PID controller adjustment: compare with the target taste curve and dynamically add seasoning (e.g., add 0.5g of salt at 15 min).

[0141] For example, precise control can be achieved through the coordinated operation of the three components of a PID controller: the proportional component responds quickly to the current concentration deviation, and immediately adds the seasoning if the current flavor value is lower than the target value; the integral component eliminates historical accumulated deviations, and if a certain flavor consistently occurs, it can pause integration based on the accumulated error to prevent over-addition; the integral component predicts the concentration change trend and suppresses oscillations to ensure a balanced flavor.

[0142] The cooking process is controlled in stages, divided into initial and later stages. In the initial stage, high gain is used to quickly approach the target, while in the later stage, low gain is used for fine-tuning. Real-time intervention is implemented under healthy conditions; for example, if a user consumes excessive amounts of a certain seasoning, intervention will force a reduction in the amount of seasoning.

[0143] During the cooking process, the electronic tongue sensor monitors the flavor value in the broth in real time. When the detected flavor value deviates from the target flavor value by more than 5%, the system automatically starts the adjustment mechanism, dynamically adding or reducing seasonings through the PID controller until the flavor deviation is reduced to within 3%.

[0144] By implementing real-time closed-loop seasoning control, the seasonings are dynamically adjusted during the cooking process, ensuring that the final taste of the dish matches the user's expectations, thus improving the accuracy of cooking and the user experience.

[0145] Figure 5 This is a flowchart of the seasoning processing according to an embodiment of the present invention, such as... Figure 5 As shown, after selecting a cuisine, the system uses family members' taste profiles and environmental data to perform multi-objective optimization calculations to obtain an initial seasoning scheme; then, PID control deviation analysis is performed to dynamically add seasonings until cooking is complete.

[0146] Figure 6 This is a flowchart of an optional seasoning process according to an embodiment of the present invention, such as... Figure 6As shown, the process begins by displaying user input and acquiring historical cooking data. A baseline preference vector is determined based on the user input, and implicit weighted EMA updates are performed using the historical cooking data. Next, a comprehensive preference vector is determined. Dynamic constraints on seasonings and temperature and humidity compensation data are then determined based on health APIs and environmental sensors, and these are integrated into a multi-objective optimization model to obtain the final seasoning recommendation. The system then determines whether the user needs to adjust the settings; if so, feedback learning is performed; otherwise, compliance is recorded.

[0147] The technical solution provided by the above embodiments of the present invention establishes a food identification model through deep learning image recognition and detection technology to initially identify food ingredients and recommend cuisines to users. After the user confirms the cuisine of the cooking ingredients, the automatic food cutting device cuts the ingredients, such as potato chunks or shredded potatoes, without requiring the user to manually input the ingredient name. Simultaneously, after identifying the ingredients, the device links with the refrigerator's contents to recommend cuisines to the user. Upon confirmation, the device automatically cuts the ingredients according to the cuisine, significantly improving the intelligence level of the cooking equipment and freeing up the hands of users who are not skilled at cutting vegetables. Furthermore, a personalized seasoning matching control strategy model is established based on the taste preferences of different family members. Unlike fixed cuisines with pre-mixed seasoning ratios, this model considers the overall taste preferences of all family members, especially in large families. It establishes a family cooking seasoning control strategy that also takes into account external factors such as weather and season, as well as user health factors, comprehensively adjusting the seasoning to meet the different taste preferences of family members. This intelligent seasoning matching achieves intelligent and humanized service to users.

[0148] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0149] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0150] Example 2

[0151] According to an embodiment of the present invention, an automatic cooking apparatus for implementing the above-described automatic cooking method is also provided. Figure 7 This is a schematic diagram of an automatic cooking device according to an embodiment of the present invention, such as... Figure 7 As shown, the automatic cooking device includes: an acquisition unit 701, an identification unit 703, a determination unit 705, and a cooking unit 707. The automatic cooking device will be described below.

[0152] The acquisition unit 701 is used to acquire the ingredient feature data of the target ingredient, wherein the ingredient feature data includes: ingredient visual data, ingredient freshness data and ingredient weight data.

[0153] The identification unit 703 is used to identify the target ingredient based on the ingredient feature data through the ingredient status identification model, and obtain the ingredient identification result of the target ingredient. The ingredient status identification model is a model trained by machine learning using multiple sets of training data. The ingredient identification result includes: ingredient name, ingredient status and cuisine label.

[0154] The determination unit 705 is used to determine the recommended cuisine based on the food identification results, historical cooking records, and optional food stored in the smart refrigerator.

[0155] The cooking unit 707 is used to perform cooking operations on the target ingredients according to the recommended cuisine.

[0156] It should be noted that the above-mentioned acquisition unit 701, identification unit 703, determination unit 705 and cooking unit 707 correspond to steps S202 to S208 in the above embodiments. The four units and the corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in the above embodiments.

[0157] As can be seen from the above, in the solution described in the above embodiments of the present invention, the acquisition unit can acquire the ingredient feature data of the target ingredient, wherein the ingredient feature data includes: ingredient visual data, ingredient freshness data, and ingredient weight data; the identification unit uses an ingredient state identification model to identify the target ingredient based on the ingredient feature data, thereby obtaining the ingredient identification result of the target ingredient, wherein the ingredient state identification model is a model trained by machine learning using multiple sets of training data, and the ingredient identification result includes: ingredient name, ingredient state, and cuisine label; the determination unit is used to determine the recommended cuisine based on the ingredient identification result, historical cooking records, and optional ingredients stored in the smart refrigerator; and the cooking unit performs cooking operations on the target ingredient according to the recommended cuisine, thereby realizing the purpose of identifying the ingredient to obtain the ingredient type, state, and cuisine label, and determining the recommended cuisine based on the identification result combined with historical cooking records and optional ingredients stored in the smart refrigerator. This achieves the effect that the cooking equipment can actively identify and recommend suitable cuisines, and can also make adjustments according to the ingredient reserves in the refrigerator, thereby improving the intelligence level of the cooking equipment, providing more personalized and comprehensive cuisine recommendations, and improving the user experience.

[0158] Therefore, the technical solution provided by the embodiments of the present invention solves the technical problem that the intelligent cooking equipment in the related art has a low level of intelligence and cannot fully meet the personalized, intelligent and healthy needs of home cooking.

[0159] Optionally, the acquisition unit includes: a capture module for capturing visual data of the target ingredient through an image acquisition device, wherein the visual data includes the following data of the target ingredient: morphological data, color data, and texture data; a first acquisition module for acquiring the ingredient weight data of the target ingredient through a pressure sensor; and a second acquisition module for acquiring the ingredient freshness data of the target ingredient through an electronic nose sensor.

[0160] Optionally, the acquisition unit includes: an encoding module for performing spatiotemporal feature encoding on multiple frames of images of the target ingredient to obtain ingredient encoding data; an extraction module for extracting geometric topological change information during the deformation process of the target ingredient based on the ingredient encoding data; an enhancement module for enhancing the deformation region of the target ingredient based on the geometric topological change information to obtain an enhancement result; an alignment module for performing spatial alignment processing on multiple frames of images to obtain a spatial alignment processing result; and a third acquisition module for obtaining visual data of the ingredient based on the enhancement processing result and the spatial alignment processing result.

[0161] Optionally, the determining unit includes: a first determining module, used to determine the user's cuisine mapping relationship based on historical cooking records, wherein the user's cuisine mapping relationship is used to record the number of times each user selects each cuisine; a second determining module, used to determine the associated cuisine of the target ingredient based on the ingredient status and the indicated ingredient name in the ingredient identification result, wherein the associated cuisine is the dish corresponding to the target ingredient; a third determining module, used to determine the optional ingredients in the smart refrigerator; and a fourth determining module, used to determine the recommended cuisine based on the user's cuisine mapping relationship, the associated cuisine, and the optional ingredients.

[0162] Optionally, the first determining module includes: a first obtaining submodule, used to obtain raw data of historical cooking records, wherein the raw data includes: user identifier and cuisine; and a matrix processing submodule, used to perform matrix processing on the raw data to obtain user cuisine mapping relationship, wherein the rows of the matrix in the matrix processing are user identifiers, and the columns of the matrix are cuisines.

[0163] Optionally, the fourth determining module includes: a first determining submodule, used to determine the similarity between the target and each user in each historical cooking record; a selection submodule, used to select similar users whose similarity is higher than the similarity threshold; a second determining submodule, used to determine the initial recommended cuisine based on the user cuisine mapping relationship and associated cuisines corresponding to similar users; and a filtering submodule, used to filter the initial recommended cuisine based on the available ingredients to obtain the recommended cuisine.

[0164] Optionally, the cooking unit includes: a cutting module for performing a cutting operation on the target ingredient according to a recommended cuisine to obtain the cut target ingredient; a third acquisition module for acquiring the taste information of family members; a fifth determination module for determining the seasoning information required for the cooking operation based on the taste information; and an adding module for adding seasonings to the cut target ingredient according to the seasoning information to perform the cooking operation on the target ingredient.

[0165] Optionally, the fifth determining module includes: a second acquiring submodule for acquiring physiological indicator data of family members; a third determining submodule for determining the health risk score of family members based on the physiological indicator data; a fourth determining submodule for determining the upper limit value of seasonings based on the health risk score; and a fifth determining submodule for determining seasoning information based on the upper limit value of seasonings and taste information.

[0166] Optionally, the fifth determining module includes: a third obtaining submodule for obtaining environmental parameters of the room where the family member is located; and a sixth determining submodule for determining seasoning information based on the environmental parameters and taste information.

[0167] Optionally, the automatic cooking device further includes: a detection unit for detecting the current flavor value during the cooking operation of the target ingredients according to the recommended cuisine; and an adjustment unit for adjusting the seasoning addition method according to the current flavor value.

[0168] According to another aspect of the present invention, an intelligent cooking device is also provided, which uses any of the above-described automatic cooking methods.

[0169] According to another aspect of the present invention, a processor is also provided, which is used to run a program, wherein the program executes any of the above-described automatic cooking methods during runtime.

[0170] According to another aspect of the present invention, a computer program product is also provided, including computer instructions that, when executed by a processor, perform any of the above-described automatic cooking methods.

[0171] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein any one of the program executions is an automatic cooking method.

[0172] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any communication device in a group of communication devices.

[0173] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: acquiring ingredient feature data of the target ingredient, wherein the ingredient feature data includes: ingredient visual data, ingredient freshness data, and ingredient weight data; identifying the target ingredient based on the ingredient feature data using an ingredient state recognition model to obtain the ingredient recognition result of the target ingredient, wherein the ingredient state recognition model is a model trained by machine learning using multiple sets of training data, and the ingredient recognition result includes: ingredient name, ingredient state, and cuisine label; determining a recommended cuisine based on the ingredient recognition result, historical cooking records, and optional ingredients stored in the smart refrigerator; and performing cooking operations on the target ingredient according to the recommended cuisine.

[0174] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: capturing visual data of the target ingredient through an image acquisition device, wherein the visual data includes the following data of the target ingredient: morphological data, color data, and texture data; obtaining the ingredient weight data of the target ingredient through a pressure sensor; and obtaining the ingredient freshness data of the target ingredient through an electronic nose sensor.

[0175] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: performing spatiotemporal feature encoding on multiple frames of images of the target food ingredient to obtain food ingredient encoding data; extracting geometric topological change information during the deformation process of the target food ingredient based on the food ingredient encoding data; performing enhancement processing on the deformation region of the target food ingredient based on the geometric topological change information to obtain enhancement processing results; performing spatial alignment processing on multiple frames of images to obtain spatial alignment processing results; and obtaining food ingredient visual data based on the enhancement processing results and the spatial alignment processing results.

[0176] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: determining a user cuisine mapping relationship based on historical cooking records, wherein the user cuisine mapping relationship is used to record the number of times each user selects each cuisine; determining the associated cuisine of the target ingredient based on the ingredient status and the indicated ingredient name in the ingredient identification result, wherein the associated cuisine is the dish corresponding to the target ingredient; determining the optional ingredients in the smart refrigerator; and determining a recommended cuisine based on the user cuisine mapping relationship, the associated cuisine, and the optional ingredients.

[0177] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining raw data of historical cooking records, wherein the raw data includes: user identifier and cuisine; performing matrix processing on the raw data to obtain user cuisine mapping relationship, wherein in the matrix processing, the rows of the matrix are user identifiers, and the columns of the matrix are cuisines.

[0178] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: determining the similarity between the target and each user in each historical cooking record; selecting similar users whose similarity is higher than a similarity threshold; determining an initial recommended cuisine based on the user cuisine mapping relationship and associated cuisines corresponding to the similar users; and filtering the initial recommended cuisine based on available ingredients to obtain a recommended cuisine.

[0179] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: performing a cutting operation on the target ingredient according to a recommended cuisine to obtain the cut target ingredient; obtaining the taste information of family members; determining the seasoning information required for the cooking operation according to the taste information; and adding seasonings to the cut target ingredient according to the seasoning information to perform the cooking operation on the target ingredient.

[0180] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: acquiring physiological indicator data of family members; determining health risk scores of family members based on the physiological indicator data; determining upper limits for seasonings based on the health risk scores; and determining seasoning information based on the upper limits for seasonings and taste information.

[0181] Optionally, in this embodiment, the computer-readable storage medium is configured to store program code for performing the following steps: obtaining environmental parameters of the room where the family member is located; and determining seasoning information based on the environmental parameters and taste information.

[0182] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0183] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0184] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0185] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0186] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0188] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0189] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An automatic cooking method, characterized in that, include: Obtain the ingredient characteristic data of the target ingredient, wherein the ingredient characteristic data includes: ingredient visual data, ingredient freshness data, and ingredient weight data; The target ingredient is identified by the ingredient status recognition model based on the ingredient feature data, and the ingredient recognition result of the target ingredient is obtained. The ingredient status recognition model is a model trained by machine learning using multiple sets of training data. The ingredient recognition result includes: ingredient name, ingredient status and cuisine label. The recommended cuisine is determined based on the ingredient identification results, historical cooking records, and available ingredients stored in the smart refrigerator. The target ingredients are cooked according to the recommended cuisine.

2. The automatic cooking method according to claim 1, characterized in that, Obtain the ingredient characteristic data of the target ingredient, including: The visual data of the target food ingredient is captured by an image acquisition device, wherein the visual data includes the following data of the target food ingredient: morphological data, color data, and texture data; The weight data of the target ingredient is obtained through a pressure sensor; The freshness data of the target ingredient is obtained through an electronic nose sensor.

3. The automatic cooking method according to claim 1, characterized in that, Obtaining the visual data of the target ingredient includes: Spatiotemporal feature encoding is performed on multiple frames of images of the target food ingredient to obtain food ingredient encoded data; Extract the geometric and topological change information of the target food during the deformation process based on the food coding data; The deformation region of the target food ingredient is enhanced based on the geometric topological change information to obtain the enhancement result; Spatial alignment processing is performed on the multi-frame images to obtain the spatial alignment processing result; The visual data of the food ingredients is obtained based on the enhancement processing result and the spatial alignment processing result.

4. The automatic cooking method according to claim 1, characterized in that, Based on the ingredient identification results, historical cooking records, and available ingredients stored in the smart refrigerator, recommended cuisines are determined, including: The user cuisine mapping relationship is determined based on the historical cooking records, wherein the user cuisine mapping relationship is used to record the number of times each user selects each cuisine; Based on the food status and the food name shown in the food identification result, the associated cuisine of the target food is determined, wherein the associated cuisine is the dish corresponding to the target food; The optional ingredients in the smart refrigerator are determined; The recommended cuisine is determined based on the user's cuisine mapping relationship, the associated cuisines, and the available ingredients.

5. The automatic cooking method according to claim 4, characterized in that, Determining the user's cuisine mapping relationship based on the historical cooking records includes: Obtain the raw data of the historical cooking records, wherein the raw data includes: user identifier and cuisine; The original data is matrix-processed to obtain the user's cuisine mapping relationship, wherein the rows of the matrix in the matrix-processing are the user identifiers, and the columns of the matrix are the cuisines.

6. The automatic cooking method according to claim 4, characterized in that, The recommended cuisine is determined based on the user's cuisine mapping relationship, the associated cuisines, and the available ingredients, including: The target is determined to be similar to each user in each of the aforementioned historical cooking records; Select similar users whose similarity score is higher than the similarity threshold; The initial recommended cuisine is determined based on the user cuisine mapping relationship corresponding to the similar users and the associated cuisines; The initial recommended cuisines are filtered based on the available ingredients to obtain the recommended cuisines.

7. The automatic cooking method according to claim 1, characterized in that, Perform cooking operations on the target ingredients according to the recommended cuisine, including: The target ingredient is cut according to the recommended cuisine to obtain the cut target ingredient; Obtain information on family members' tastes; The seasoning information required for the cooking operation is determined based on the flavor information; Add seasonings to the cut target ingredient according to the seasoning information to perform the cooking operation on the target ingredient.

8. The automatic cooking method according to claim 7, characterized in that, Determining the seasoning information required for the cooking operation based on the flavor information includes: Obtain the physiological indicator data of the family members; The health risk score of the family member is determined based on the physiological indicator data; The upper limit value of the seasoning is determined based on the aforementioned health risk score; The seasoning information is determined based on the upper limit value of the seasoning and the flavor information.

9. The automatic cooking method according to claim 7, characterized in that, Determining the seasoning information required for the cooking operation based on the flavor information includes: Obtain the environmental parameters of the room where the family member is located; The seasoning information is determined based on the environmental parameters and the flavor information.

10. The automatic cooking method according to claim 1, characterized in that, Also includes: During the cooking process of the target ingredient according to the recommended cuisine, the current flavor value is detected; Adjust the seasoning addition method according to the current taste value.

11. An automatic cooking device, characterized in that, include: The acquisition unit is used to acquire the ingredient feature data of the target ingredient, wherein the ingredient feature data includes: ingredient visual data, ingredient freshness data, and ingredient weight data; The identification unit is used to identify the target ingredient based on the ingredient feature data using an ingredient status identification model, and to obtain the ingredient identification result of the target ingredient. The ingredient status identification model is a model trained by machine learning using multiple sets of training data. The ingredient identification result includes: ingredient name, ingredient status, and cuisine label. The determining unit is used to determine the recommended cuisine based on the food identification results, historical cooking records, and optional food stored in the smart refrigerator; A cooking unit is used to perform cooking operations on the target ingredients according to the recommended cuisine.

12. A smart cooking device, characterized in that, The intelligent cooking device uses the automatic cooking method described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program performs the automatic cooking method according to any one of claims 1 to 10.

Images