Multi-kitchen-ware linkage control method and system for smart kitchen

Abstract

The invention discloses a multi-kitchen-ware linkage control method and system for a smart kitchen, and the method comprises the steps: carrying out the multi-modal data fusion of food materials and user modal data, generating a recommended recipe list matched with a current scene through a personalized recommendation algorithm, and adjusting the recipe parameters; according to the to-be-executed recipe, the virtual interaction system is synchronized with the multiple kitchenware to realize linkage logic so as to control the kitchenware to execute a preparation task, and a cutting and matching guide is output to a user; according to the to-be-executed recipe and the matching feedback information, taking at least one of time and energy consumption as an optimization target, and generating and carrying out time sequence optimization on a multi-kitchen-ware control instruction set so as to control multiple kitchen wares to execute cooking tasks; cooking parameters of various kitchen wares are dynamically adjusted through sensing data obtained in real time. The recommended recipe is dynamically matched with the actual scene, the virtual interaction system and the kitchen ware state are deeply bound to improve the guiding effect and the kitchen ware cooperation ability of the cutting and matching stage, and cross-kitchen ware time sequence optimization under multiple targets in the cooking stage is achieved.

Description

Technical Field

[0001] This invention relates to the field of electronic information technology, and in particular to a method and system for multi-kitchen appliance linkage control in a smart kitchen. Background Technology

[0002] A smart kitchen refers to a modern kitchen system that uses technologies such as the Internet of Things (IoT), artificial intelligence (AI), and big data to intelligently manage kitchen equipment, ingredients, and cooking processes, achieving automated cooking, precise control, and personalized experiences. It's not just about making individual appliances smart (like a smart rice cooker or smart oven), but about the interconnectedness of the entire kitchen ecosystem, allowing devices to work collaboratively and even adjust cooking plans based on user habits and real-time data.

[0003] In current smart kitchens, individual appliances (such as smart ovens, air fryers, and rice cookers) can independently complete preset tasks, such as timed cooking and temperature adjustment. Some high-end systems can achieve basic collaboration between devices; for example, after the steamer finishes steaming fish, the oven automatically starts baking side dishes. The system provides recipes for users to choose from and identifies and manages ingredients through QR codes or RFID technology.

[0004] In the process of implementing the technical solution of this invention, at least the following technical problems were found in the prior art: 1. Weak inter-device coordination: Most traditional smart kitchens still automate individual devices or achieve simple linkage functions, lacking true intelligent scheduling. For example, when the rice cooker finishes cooking rice, the oven may just start roasting meat, resulting in the food not being served at the same time.

[0005] 2. Food identification relies on manual labeling. Similar foods (such as pork belly and ribs) need to be distinguished by QR codes or RFID tags, which increases the barrier to entry.

[0006] 3. The system's built-in recipes are rigid and lack scene linkage functions. They can only be executed according to preset recipes and cannot recommend matching recipes based on existing ingredients. The recipe parameters are fixed and require users to filter or change recipes, causing users to spend a lot of energy to match or improve recipes. Moreover, the amount of ingredients in the recipes often does not match the actual situation, resulting in poor cooking results.

[0007] 4. The cooking process lacks flexibility and can only be executed according to a preset procedure. It cannot be adjusted in real time according to the actual situation (such as the state of the ingredients), which easily leads to poor cooking results.

[0008] In conclusion, traditional smart kitchens cannot meet actual usage needs. Summary of the Invention

[0009] This invention provides a method and system for multi-kitchen appliance linkage control in a smart kitchen, which solves the problem that traditional smart kitchens cannot meet actual usage needs.

[0010] In a first aspect, the present invention provides a method for multi-kitchen appliance linkage control in a smart kitchen, comprising: Step 101: After fusing the current food modal data acquired by the sensor and the user modal data input by the user, a recommended recipe list matching the current food category, proportion, weight, and user health and dietary preferences is generated through a personalized recommendation algorithm, and the recipe parameters are adjusted. Step 102: In response to the user's selected recipe, the virtual interactive system synchronizes with multiple kitchen utensils to achieve linkage logic and control the multiple kitchen utensils to perform preparation tasks. It outputs cutting and preparation guidelines to the user in real time according to the status of multiple kitchen utensils and receives the user's cutting and preparation feedback information. Step 103: Based on the recipe to be executed and the cutting and preparation feedback information, generate a multi-kitchen utensil control instruction set. With time and energy consumption as optimization objectives, optimize the timing of the multi-kitchen utensil control instruction set and control multiple kitchen utensils to perform cooking tasks. Step 104: Dynamically adjust the cooking parameters of various kitchen utensils using real-time sensor data.

[0011] Optionally, the step of acquiring the current food modal data specifically includes: Based on the user's current accurate information on various ingredients, the system acquires RGB images, depth images, and actual weight data for each ingredient collected by sensors. An image segmentation model based on multi-task learning is used to output the food category, part, and volume corresponding to each image; The theoretical weight of each ingredient is calculated by querying a preset density library, and the actual weight data is used as an auxiliary label for image segmentation to correct the image segmentation confidence. Based on the corrected recognition results, the current food modal data is obtained.

[0012] Optionally, the method further includes: If the current ingredient modal data or the user modal data is updated, a new list of recommended recipes will be generated again through a personalized recommendation algorithm after multimodal data fusion is performed again. Determine if the type of the newly acquired recipe to be executed has changed. If it has changed, repeat steps 102 to 104. Otherwise, jump to step 104 to adjust the cooking parameters of the multi-cooker.

[0013] Optionally, the method further includes: When abnormal status parameters of kitchen utensils are detected, the kitchen utensils fault recovery process is initiated; If the status parameters of the kitchen appliance are still abnormal after the kitchen appliance failure recovery process is completed, a new kitchen appliance that can replace the faulty kitchen appliance will be started to continue the cooking task.

[0014] Optionally, the virtual interaction system in step 102 may specifically be an AR interaction system, a video interaction system, or a voice interaction system.

[0015] Optionally, step 103 specifically includes: Based on the kitchen utensils, cooking time, and ingredient quantities used in the recipe to be executed, generate start instructions, continuous running time setting instructions, cooking parameter setting instructions, and shutdown instructions for each type of kitchen utensil. When there are missing cutting steps in the cutting feedback information, a cooking parameter change instruction is generated for the corresponding kitchen utensils; When it is detected that the kitchen utensils used in the recipe to be executed include those linked to the range hood, a delayed start command for the range hood is generated; When it is detected that the kitchen utensils used in the recipe to be executed include high-power kitchen utensils used in parallel, a peak-shifting power consumption instruction or a parallel start instruction is generated based on the current maximum energy consumption allowable value of the kitchen. The generated instructions are combined to form the Duoduo Kitchenware Control Instruction Set; With time and energy consumption as optimization objectives, the instructions in the multi-kitchen utensil control instruction set are time-optimized, and then the multi-kitchen utensil control instruction set is executed according to the optimized timing to control multiple kitchen utensils to perform cooking tasks.

[0016] Optionally, the optimization objective is at least one of time and energy consumption, specifically: The optimization goal is to ensure that all kitchen utensils have the same cooking completion time; or The optimization objective is to minimize the overall time; or The optimization objective is to minimize energy consumption; or The optimization objective is to simultaneously optimize both time and energy consumption.

[0017] Optionally, the sensing data in step 104 specifically includes: food burnt information, food firmness information, food temperature information, and kitchen temperature and humidity information.

[0018] Optionally, the cooking parameters in step 104 may specifically include: heat, cooking time for different cooking stages, and pressure.

[0019] Secondly, the present invention also provides a multi-kitchen appliance linkage control system for a smart kitchen, the system comprising: The recommendation unit is used to fuse the current food modal data acquired by the sensor and the user modal data input by the user into multimodal data, and then generate a list of recommended recipes that match the current food category, proportion, weight, and user health and dietary preferences through a personalized recommendation algorithm, and adjust the recipe parameters. The interactive unit is used to respond to the recipe selected by the user. The virtual interactive system synchronizes with multiple kitchen utensils to realize linkage logic and control multiple kitchen utensils to perform preparation tasks. It outputs cutting and preparation guidelines to the user in real time according to the status of multiple kitchen utensils and receives cutting and preparation feedback information from the user. The control unit is used to generate a multi-kitchen utensil control instruction set based on the recipe to be executed and the cutting and preparation feedback information. After timing optimization of the multi-kitchen utensil control instruction set with at least one of time and energy consumption as optimization objectives, it controls multiple kitchen utensils to perform cooking tasks. The adjustment unit is used to dynamically adjust the cooking parameters of various kitchen utensils based on real-time sensor data.

[0020] One or more technical solutions provided by this invention have at least the following technical effects or advantages: This invention identifies food ingredients by acquiring current food ingredient modal data through sensors, without relying on QR codes or RFID tags or manual marking, thus lowering the barrier to entry and improving the level of intelligence.

[0021] This invention utilizes a personalized recommendation algorithm to generate a recommended recipe list based on multimodal fusion data. It dynamically adjusts recipe types according to the category, proportion, and weight of existing ingredients, as well as the user's health and dietary preferences, automatically adapting to the user's situation. For example, it automatically filters high-sugar recipes for diabetic users. It also automatically adapts to the actual ingredients and quantities; for example, when cucumber constitutes a large proportion, it recommends cucumber salad as the main dish, and when cucumber constitutes a small proportion, it recommends shredded vegetables salad as a side dish. Furthermore, this invention can dynamically adjust recipe parameters based on the category, proportion, and weight of existing ingredients, as well as the user's health and dietary preferences, automatically adjusting to the optimal parameters based on the ingredients. For example, when the actual weight of vegetables exceeds the original recipe weight, it automatically increases the amount of seasoning. It also automatically matches user health data (such as low-salt requirements, low-sugar requirements) and dietary preferences (such as a preference for spicy or sweet foods), for example, adjusting the amount of white sugar from 5 grams to 2 grams for users managing their weight. In summary, the recipes recommended in this invention not only feature scene-linking capabilities but also real-time adjustment capabilities. This solves the technical problem of traditional smart kitchens where fixed recipes require users to spend considerable effort creating or modifying recipes, and avoids poor cooking results due to insufficient or excessive ingredients. It is highly practical and improves the user experience. Furthermore, when the type, proportion, weight of ingredients, or data related to the user's health and dietary preferences change, it supports the generation of new recommended recipe lists, dynamically matching actual scenarios and demonstrating strong adaptability.

[0022] In the cutting and preparation stage, based on the recipe to be executed selected by the user, the present invention not only synchronizes with multiple kitchen utensils through a virtual interaction system to implement linkage logic and then controls multiple kitchen utensils to perform preparation tasks. For example, the virtual interaction system controls the preheating of the oven and prompts the user to replenish water when the water level in the steam box is insufficient. The present invention can also guide the user on how to preprocess and process ingredients in real time according to the states of multiple kitchen utensils. For example, it judges whether the pressure cooker is boiling through the temperature curve and then controls the real-time projection of AR marks to prompt the blanching of ribs. When the preheating task of the oven is completed, it prompts the user to put the pork belly to be roasted in the oven by voice. Compared with the traditional virtual interaction system that can only guide the user, the virtual interaction system is deeply bound to the states of the kitchen utensils, realizing the joint completion of cutting and preparation by the virtual interaction system, the kitchen utensils, and the user. It not only has a good guiding effect, is closer to the actual usage needs of the user, but also can improve the coordination ability of multiple kitchen utensils in the cutting and preparation stage.

[0023] The multi-kitchen utensil control instruction set of the present invention is not only related to the recipe to be executed, but also related to the cutting and preparation feedback information of the user. It can establish the linkage between various kitchen utensils in the cutting and preparation stage and the cooking stage, and can also take at least one of time and energy consumption as the optimization goal, reasonably plan the execution timing of control instructions for each kitchen utensil, and make various kitchen utensils cooperate deeply. For example, when the rice cooker finishes cooking the rice, the oven also finishes roasting the pork belly, making the meal and the dishes served at the same time; another example is that the 1Kw steam box is preferentially powered, and the 2Kw oven pauses heating to avoid tripping that may be caused by high-power operation at the same time, realizing peak-shifting power consumption; another example is that the oven and the stove run synchronously to reduce the overall cooking time; the range hood starts with a delay to reduce ineffective operation and realizes an energy-saving strategy; in practical applications, cooking time and energy consumption are often a contradiction. To achieve the overall optimum, time and energy consumption can be used as optimization goals at the same time, taking into account both the cooking time and the power consumption of the equipment. Generally speaking, the present invention solves the technical problem of weak coordination ability among traditional smart kitchen devices, realizes the cross-kitchen utensil timing optimization of multiple goals, and improves the intelligent scheduling level among kitchen utensils.

[0024] The present invention can dynamically adjust the cooking parameters of kitchen utensils according to the sensed data obtained in real time, ensuring that the current cooking parameters fit the actual situation to achieve a better cooking effect.

[0025] Preferably, the present invention sets a kitchen utensil fault recovery process. For example, when the oven temperature is abnormal, it automatically turns off the oven and notifies the user, and automatically starts a new kitchen utensil when the fault cannot be recovered. For example, it turns on the air fryer to replace the oven to bake food to continue the cooking task, taking into account both cooking safety and task completion efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of a multi-kitchen utensil linkage control method for a smart kitchen of the present invention; Figure 2 It is an architecture diagram of a multi-kitchen utensil linkage control system for a smart kitchen of the present invention. Detailed Implementation

[0027] This invention provides a method and system for multi-kitchen appliance linkage control in a smart kitchen, which solves the problem that traditional smart kitchens cannot meet actual usage needs.

[0028] To better understand, a detailed description will be provided below with reference to the accompanying drawings and specific embodiments. Obviously, the embodiments described in this invention are only a part of the embodiments of this invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] Please refer to the following. Figure 1 The present invention provides a detailed description of a multi-kitchen appliance linkage control method for a smart kitchen in an embodiment of the present invention.

[0030] Step 101: After multimodal data fusion of the current food modal data acquired by the sensor and the user modal data input by the user, a recommended recipe list matching the current food category, proportion, weight, and the user's health and dietary preferences is generated through a personalized recommendation algorithm, and the recipe parameters are adjusted.

[0031] In practical implementation, for example, users accurately select ingredients such as rice, vegetables, pork belly, eggs, corn, and ribs. A camera captures images of the ingredients, a smart scale transmits the weight in real time, and the user submits their health records and dietary preferences via an app. Visual modal data of the current ingredients, including volume estimation and category recognition, is obtained through image segmentation algorithms based on CNN and Transformer. Weight modal data is obtained through dynamic calibration algorithms, and user modal data is obtained through data preprocessing and extraction algorithms. A cross-modal feature fusion architecture combines features from visual modal data, weight modal data, and user modal data. A knowledge graph is used to query a recipe library that matches the current ingredient category, proportion, weight, and the user's health and dietary preferences. A personalized recommendation algorithm generates a recommended recipe list; for example, when cucumber constitutes a large proportion, it is recommended as a main dish (cucumber salad), and when cucumber constitutes a small proportion, it is recommended as a side dish (shredded vegetables salad). Additionally, high-sugar recipes are automatically filtered out when a diabetic user is detected. After recommending the recipes, the system adjusts the recipe parameters as needed. For example, if a diabetic user is detected, the white sugar in the recipe is replaced with erythritol; if a user is managing their weight, the amount of white sugar is reduced from 5 grams to 2 grams; if a user with high blood pressure is detected, the salt content is automatically reduced; if a user likes spicy food, the amount of chili peppers is increased; if a large proportion of corn is detected in a corn and pork rib soup, the cooking time is increased. Finally, based on the user's rating of the recommended recipes, the recommendation strategy is updated using a reinforcement learning algorithm.

[0032] The step of obtaining the current food modal data in step 101 specifically includes: Step 1011: Based on the user's current accurate selection of various ingredients, acquire the RGB image, depth image, and actual weight data of each ingredient collected by the sensor. The RGB image represents the ingredient's color, texture, etc., the depth image represents the ingredient's 3D shape, and the actual weight data is used to correct the image recognition results.

[0033] Step 1012: An image segmentation model based on multi-task learning is adopted. CNN is used to extract local features of food ingredients, and Transformer is used to capture global context. At the same time, the food category (such as green vegetables and eggs), food part (ribs and pork belly) and food volume corresponding to each image are output.

[0034] Step 1013: Query a preset density database to calculate the theoretical weight of each ingredient, and use the actual weight data as auxiliary labels for image segmentation to correct the image segmentation confidence. For example, the average density of corn kernels is 0.72. The image segmentation confidence is corrected using the actual weight data of corn. The reasonableness of the visual recognition result is inferred from the weight. When the actual weight of corn is much lower than the expected weight (the product of the predicted volume and the average density output by the model), the corn recognition confidence output by the image segmentation model is forcibly corrected in order to calibrate the image segmentation result.

[0035] Step 1014: Based on the corrected recognition results, obtain the current food modal data. The current food modal data includes the visual modal data and weight modal data of the current food.

[0036] Step 102: In response to the user's selected recipe, the virtual interactive system synchronizes with multiple kitchen utensils to achieve linkage logic and control the multiple kitchen utensils to perform preparation tasks. It outputs cutting and preparation guidelines to the user in real time based on the status of multiple kitchen utensils and receives the user's cutting and preparation feedback information.

[0037] Specifically, the virtual interaction system in step 102 can be an AR interaction system, a video interaction system, or a voice interaction system. In practical applications, the type of interaction system is selected according to the specific circumstances, and this invention does not impose any limitations.

[0038] In the specific implementation of step 102, for example, after the user selects the desired recipe from the recommended recipe list generated by the system, the virtual interactive system controls various kitchen utensils in the cutting and preparation stage based on the user's selected recipe. Firstly, at the device communication and protocol conversion layer, a multi-mode communication chipset supporting Zigbee / Bluetooth / WiFi / PLC is used. The protocol conversion engine dynamically translates the private protocols of different kitchen utensils, unifying the device control interface. The virtual interactive system uses a real-time status synchronization engine to monitor various kitchen utensils in real time. Based on the recipe to be executed, and according to the linkage logic, it not only controls various kitchen utensils to perform preparation tasks, such as controlling the oven to preheat at 100℃, and prompting the user to add water when the steamer water level is insufficient, but also guides the user on how to pre-process and process ingredients based on the status of various kitchen utensils. For example, when ingredients are placed on the smart cutting board, an AR chef is projected to guide the user in cutting the ingredients; the pressure cooker temperature curve is used to determine whether boiling has occurred, and AR markers are used to project real-time prompts for blanching pork ribs; and when the oven preheating task is completed, a voice prompt prompts the user to put in the pork belly to be roasted.

[0039] Step 103: Based on the recipe to be executed and the cutting and preparation feedback information, generate a multi-kitchen utensil control instruction set. With time and energy consumption as optimization objectives, optimize the timing of the multi-kitchen utensil control instruction set and control multiple kitchen utensils to perform cooking tasks.

[0040] Step 103 specifically includes: Step 1031: Based on the kitchen utensils, cooking time, and ingredient quantities used in the recipe to be executed, generate start instructions, continuous running time setting instructions, cooking parameter setting instructions, and shutdown instructions for each type of kitchen utensil.

[0041] Step 1032: When there are missing cutting steps in the cutting feedback information, generate cooking parameter change instructions for the corresponding kitchen utensils. For example, if the user does not complete the blanching operation as prompted by the AR interactive system, automatically adjust the subsequent pressure cooker parameters, and achieve deodorization through multi-stage pressure regulation, so that the temperature inside the pressure cooker can reach 110℃-121℃, which can more efficiently decompose some volatile amine fishy substances. It can also provide supplementary guidance on adding deodorizing seasonings such as ginger slices and cooking wine to reduce the fishy smell.

[0042] Step 1033: When it is detected that the kitchen utensils used in the recipe to be executed include those linked to the range hood, a delayed start command for the range hood is generated. For example: during the stage of heating the pan on the stove, no cooking oil is put in the pan, and no fumes are produced. The range hood is set to start 20 seconds after the stove is heated to reduce unnecessary operation.

[0043] Step 1034: When it is detected that the kitchen appliances used in the recipe to be executed include high-power kitchen appliances used in parallel, a peak-shifting power supply instruction or a parallel start instruction is generated based on the current maximum allowable energy consumption value of the kitchen. For example: when it is detected that the kitchen appliances used in the recipe to be executed include a 1kW steam oven and a 2kW oven used in parallel, if the current maximum allowable energy consumption value of the kitchen is less than the theoretical energy consumption value of parallel use, a peak-shifting power supply instruction is generated, prioritizing power supply to the steam oven, and suspending heating of the 2kW oven to avoid tripping the circuit breaker due to simultaneous high-power operation; if the current maximum allowable energy consumption value of the kitchen is greater than the theoretical energy consumption value of parallel use, a parallel start instruction is generated, and the steam oven and oven run simultaneously to shorten the overall cooking time.

[0044] Step 1035: Combine the generated instructions together to form the Duoduo Kitchenware Control Instruction Set.

[0045] Step 1036: Taking at least one of time and energy consumption as optimization objectives, optimize the timing of the instructions in the multi-kitchen utensil control instruction set, and execute the multi-kitchen utensil control instruction set according to the optimized timing to control multiple kitchen utensils to perform cooking tasks.

[0046] In the specific implementation of step 1036, for example, a multi-objective optimization scheme for time and energy consumption is set using a collaborative optimization framework of mixed integer programming and reinforcement learning. The decision variables are defined as kitchen equipment operation variables. The instantaneous power of the kitchen equipment, the task completion time, and the target weight are set as objective functions. At least one of time and energy consumption is used as the optimization objective. The time-optimal solution set and the energy-optimal solution set are solved using a non-dominated sorting genetic algorithm. Then, energy consumption-time coupling control is performed to dynamically adjust the equipment power. The kitchen equipment control command sequence that meets the design requirements is calculated through high-frequency data sampling and energy consumption analysis algorithms. The multi-kitchen equipment control command set is executed according to the optimized timing sequence to control multiple kitchen equipment to perform cooking tasks.

[0047] Among them, at least one of time and energy consumption is the optimization objective. Specifically, the optimization objective can be to make the cooking completion time of each kitchen appliance the same. For example, when the rice cooker finishes cooking rice, the oven also finishes baking pork belly, so that the food can be served at the same time.

[0048] The optimization objective is to minimize the overall cooking time, taking at least one of time or energy consumption as the objective. In practice, provided that electricity supply allows, multiple kitchen appliances can run in parallel. The start time of the earliest cooking appliance is taken as the starting point, and the end time of the latest cooking appliance is taken as the ending point. The time between the starting point and the ending point is defined as the overall time, and the optimization objective is to minimize the overall cooking time.

[0049] The optimization objective can be either time or energy consumption. Specifically, the optimization objective can be to minimize energy consumption. For example, among several kitchen appliances with similar functions, the appliance with the lowest energy consumption can be selected to complete the cooking process, thereby reducing energy consumption.

[0050] The optimization objective can be either time or energy consumption. Specifically, it can be to optimize both time and energy consumption simultaneously. In practical applications, cooking time and energy consumption are often contradictory. To achieve overall optimization, both time and energy consumption can be optimized simultaneously, taking into account both cooking time and the power consumption of the equipment.

[0051] Using the completion of a standard four dishes and one soup as the test scenario, various optimization strategies (time optimization only, energy consumption optimization only, and time-energy consumption synchronous optimization) under the multi-kitchenware linkage control method of the present invention were used to verify the traditional sequential execution strategy. The verification results are shown in Table 1 below.

[0052] Table 1. Performance Verification Comparison under Different Optimization Strategies

[0053] As shown in Table 1, the shortest total time for the time-only optimization strategy provided by this invention is 52 minutes. The minimum total energy consumption for the energy-only optimization strategy is 2.6 kWh, and the minimum peak power is 3.5 kW. The total time, total energy consumption, and peak power of the time-energy simultaneous optimization strategy all fall between the first two optimization strategies, with a total time of 58 minutes, a total energy consumption of 2.9 kWh, and a peak power of 4.3 kW. Based on the above verification results, it can be considered that the three optimization strategies can achieve the optimization objectives.

[0054] Step 104: Dynamically adjust the cooking parameters of various kitchen utensils using real-time sensor data.

[0055] Specifically, the sensor data in step 104 includes: food charring information, food firmness information, food temperature information, and kitchen temperature and humidity information. The cooking parameters in step 104 include: heat level, cooking time for different cooking stages, and pressure. The linkage between the sensor data and cooking parameters is as follows: when the food charring exceeds the preset charring level, shorten the oven baking time or reduce the stove heat; when the food temperature exceeds the preset temperature value, reduce the oven baking heat; on hot days, when the kitchen temperature is high, reduce the oven preheating time; when the food firmness is below the preset value, increase the pressure inside the pressure cooker.

[0056] In the specific implementation process, for example: separate physical property databases are established for different ingredients; then, an infrared spectral sensor combined with an OpenCV edge detection and recognition algorithm is used, and a support vector machine classifier is used to determine the burntness of the food; a PT1000 platinum resistance temperature sensor is used in conjunction with a non-contact infrared temperature measurement method to detect the food temperature; a BME680 environmental sensor is used to monitor the temperature and humidity of the kitchen; and a strain gauge sensor built into the pressure cooker is used in conjunction with ultrasonic echo analysis to detect the hardness of the food. Furthermore, a composite control algorithm based on fuzzy logic is used to control the actuators. Specifically, in the oven, a PWM control solid-state relay is used to adjust the heating element power, a stepper motor controls the damper opening, an IGBT module is used in the stove to adjust the heat, and a proportional valve is used in the pressure cooker to control the electromagnetic pressure relief device.

[0057] To dynamically match the actual scenario and update the recommended recipe list, thus enhancing adaptability, this invention further includes the following steps: If the current ingredient modal data and user modal data in step 101 are updated, multimodal data fusion is performed again, and a new recommended recipe list is generated again using a personalized recommendation algorithm; it is determined whether the type of the newly acquired recipe to be executed has changed. If it has changed, steps 102 to 104 are executed again; otherwise, step 104 is skipped to adjust the cooking parameters of multiple kitchen utensils. For example, if only the amount of the original ingredients increases, an instruction to extend the heating time is generated.

[0058] To balance cooking safety and task completion efficiency, the present invention further includes the following steps: when abnormal status parameters of the cookware are detected, such as when abnormal status of the cookware is detected by temperature probe and current sensor, a cookware fault recovery process is initiated, such as automatically shutting down the faulty cookware and notifying the user; after the cookware fault recovery process is completed, if the status parameters of the cookware are still abnormal, a new cookware that can replace the faulty cookware is activated, such as using an air fryer to replace the oven when it malfunctions, in order to continue the cooking task.

[0059] like Figure 2 As shown, the present invention also provides a multi-kitchen appliance linkage control system for a smart kitchen, the system comprising: The recommendation unit is used to fuse the current food modal data acquired by the sensor and the user modal data input by the user into multimodal data, and then generate a list of recommended recipes that match the current food category, proportion, weight, and user health and dietary preferences through a personalized recommendation algorithm, and adjust the recipe parameters. The interactive unit is used to respond to the recipe selected by the user. The virtual interactive system synchronizes with multiple kitchen utensils to realize linkage logic and control multiple kitchen utensils to perform preparation tasks. It outputs cutting and preparation guidelines to the user in real time according to the status of multiple kitchen utensils and receives cutting and preparation feedback information from the user. The control unit is used to generate a set of multiple kitchen utensils control instructions based on the recipe to be executed and the cutting and preparation feedback information. With time and energy consumption as optimization objectives, the control unit optimizes the timing of the set of multiple kitchen utensils control instructions and controls multiple kitchen utensils to perform cooking tasks. The adjustment unit is used to dynamically adjust the cooking parameters of various kitchen utensils based on real-time sensor data.

[0060] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for multi-utensil linkage control in a smart kitchen, characterized in that, include: Step 101: After fusing the current food modal data acquired by the sensor and the user modal data input by the user, a recommended recipe list matching the current food category, proportion, weight, and user health and dietary preferences is generated through a personalized recommendation algorithm, and the recipe parameters are adjusted. Step 102: In response to the user's selected recipe, the virtual interactive system synchronizes with multiple kitchen utensils to achieve linkage logic and control the multiple kitchen utensils to perform preparation tasks. It outputs cutting and preparation guidelines to the user in real time according to the status of multiple kitchen utensils and receives the user's cutting and preparation feedback information. Step 103: Based on the recipe to be executed and the cutting and preparation feedback information, generate a multi-kitchen utensil control instruction set. With time and energy consumption as optimization objectives, optimize the timing of the multi-kitchen utensil control instruction set and control multiple kitchen utensils to perform cooking tasks. Step 104: Dynamically adjust the cooking parameters of various kitchen utensils using real-time sensor data.

2. The method as described in claim 1, characterized in that, The steps for acquiring the current food modal data specifically include: Based on the user's current accurate information on various ingredients, the system acquires RGB images, depth images, and actual weight data for each ingredient collected by sensors. An image segmentation model based on multi-task learning is used to output the food category, part, and volume corresponding to each image; The theoretical weight of each ingredient is calculated by querying a preset density library, and the actual weight data is used as an auxiliary label for image segmentation to correct the image segmentation confidence. Based on the corrected recognition results, the current food modal data is obtained.

3. The method as described in claim 1, characterized in that, The method further includes: If the current ingredient modal data or the user modal data is updated, a new list of recommended recipes will be generated again through a personalized recommendation algorithm after multimodal data fusion is performed again. Determine if the type of the newly acquired recipe to be executed has changed. If it has changed, repeat steps 102 to 104. Otherwise, jump to step 104 to adjust the cooking parameters of the multi-cooker.

4. The method as described in claim 1, characterized in that, The method further includes: When abnormal status parameters of kitchen utensils are detected, the kitchen utensils fault recovery process is initiated; If the status parameters of the kitchen appliance are still abnormal after the kitchen appliance failure recovery process is completed, a new kitchen appliance that can replace the faulty kitchen appliance will be started to continue the cooking task.

5. The method as described in claim 1, characterized in that, The virtual interaction system in step 102 specifically includes an AR interaction system, a video interaction system, and a voice interaction system.

6. The method as described in claim 1, characterized in that, Step 103 specifically includes: Based on the kitchen utensils, cooking time, and ingredient quantities used in the recipe to be executed, generate start instructions, continuous running time setting instructions, cooking parameter setting instructions, and shutdown instructions for each type of kitchen utensil. When there are missing cutting steps in the cutting feedback information, a cooking parameter change instruction is generated for the corresponding kitchen utensils; When it is detected that the kitchen utensils used in the recipe to be executed include those linked to the range hood, a delayed start command for the range hood is generated; When it is detected that the kitchen utensils used in the recipe to be executed include high-power kitchen utensils used in parallel, a peak-shifting power consumption instruction or a parallel start instruction is generated based on the current maximum energy consumption allowable value of the kitchen. The generated instructions are combined to form the Duoduo Kitchenware Control Instruction Set; With time and energy consumption as optimization objectives, the instructions in the multi-kitchen utensil control instruction set are time-optimized, and then the multi-kitchen utensil control instruction set is executed according to the optimized timing to control multiple kitchen utensils to perform cooking tasks.

7. The method as described in claim 1, characterized in that, The optimization objective is at least one of time and energy consumption, specifically: The optimization goal is to ensure that all kitchen utensils have the same cooking completion time; or The optimization objective is to minimize the overall time; or The optimization objective is to minimize energy consumption; or The optimization objective is to simultaneously optimize both time and energy consumption.

8. The method as described in claim 1, characterized in that, The sensing data in step 104 specifically includes: food burnt information, food firmness information, food temperature information, and kitchen temperature and humidity information.

9. The method as described in claim 1, characterized in that, The cooking parameters in step 104 are specifically: heat, cooking time for different cooking stages, and pressure.

10. A multi-kitchen appliance linkage control system for a smart kitchen, characterized in that, The system includes: The recommendation unit is used to fuse the current food modal data acquired by the sensor and the user modal data input by the user into multimodal data, and then generate a list of recommended recipes that match the current food category, proportion, weight, and user health and dietary preferences through a personalized recommendation algorithm, and adjust the recipe parameters. The interactive unit is used to respond to the recipe selected by the user. The virtual interactive system synchronizes with multiple kitchen utensils to realize linkage logic and control multiple kitchen utensils to perform preparation tasks. It outputs cutting and preparation guidelines to the user in real time according to the status of multiple kitchen utensils and receives cutting and preparation feedback information from the user. The control unit is used to generate a multi-kitchen utensil control instruction set based on the recipe to be executed and the cutting and preparation feedback information. After timing optimization of the multi-kitchen utensil control instruction set with at least one of time and energy consumption as optimization objectives, it controls multiple kitchen utensils to perform cooking tasks. The adjustment unit is used to dynamically adjust the cooking parameters of various kitchen utensils based on real-time sensor data.

Images