Method for automatic recipe recognition and recipe execution
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-01-27
- Publication Date
- 2026-08-13
Smart Images
Figure EP2026051998_13082026_PF_FP_ABST
Abstract
Description
[0001] 202401244
[0002] 1
[0003] Method for automatic recipe recognition and execution
[0004] The invention relates to a method for automatic recipe recognition according to claim 1, a computer program product according to claim 10 and a computer-readable storage medium according to claim 11.
[0005] A recipe collection is already known in the prior art, which comprises a limited number of manually entered recipes and associated settings for a household appliance for executing the predefined recipe, wherein the operator can select one of the limited number of predefined recipes for cooking by means of an operator input. The operator is strictly limited to the limited number of predefined recipes in the recipe collection when making a recipe selection.
[0006] The object of the invention is, in particular but not limited to, providing a generic device with improved user-friendliness and flexibility. This object is achieved according to the invention by the features of claim 1, while advantageous embodiments and further developments of the invention can be found in the dependent claims.
[0007] A method for automatic recipe recognition and execution is proposed, wherein at least one cooking parameter of an image, in particular a video, of a cooking process stored in a database is recognized and compared with a digital catalog, wherein, based on the comparison, at least one setting of at least one household appliance is selected from the catalog for the execution of at least one recipe step of a recipe corresponding to the cooking process.
[0008] Such a design can particularly increase user-friendliness by allowing the operator to easily replicate the cooking process depicted in the database. Specifically, the analysis and determination of the depicted cooking process can be automated, thereby [202401244]
[0009] 2
[0010] Advantageously, the operator can simplify or avoid the necessary analysis of the cooking process. Furthermore, a recipe step that corresponds particularly precisely to the cooking process, especially to a specific stage within the cooking process, can be selected, thereby improving the cooking result.In particular, an optimized setting of the household appliance with regard to the cooking process can be selected automatically, whereby the operator does not need to have any prior knowledge of the digital catalog and the predefined recipes offered therein and / or any prior knowledge of conversion factors with regard to at least one cooking parameter, for example a set heating level of a cooking appliance shown in the illustration, to a selection of the setting, in particular a heating level of the household appliance corresponding to the heating level of the cooking appliance shown with regard to heat output, whereby the household appliance and the cooking appliance shown in the illustration differ in particular and may therefore, for example, have different dimensions and / or increments of adjustable power levels.Furthermore, flexibility can be increased, particularly by allowing users to flexibly select settings for individual recipe steps from the digital catalog to execute the recipe corresponding to the cooking process, instead of having to choose from a predefined recipe in the digital catalog. Specifically, the need to store settings for predefined recipes in the digital catalog can be reduced, and / or a particularly large number of flexibly configurable recipes can be made available by selecting and combining individual settings for each predefined recipe step. Advantageously, the operator is not limited to the predefined recipes in the digital catalog when choosing a recipe.The recipe corresponding to the cooking process can, in particular in the procedure, be transmitted to a backend of the digital catalog, thereby expanding the digital catalog, in particular by creating awareness of new recipes and / or in a particularly efficient way, by adding a recipe corresponding to the cooking process.
[0011] The household appliance can be any household appliance that a professional would deem suitable for carrying out the recipe step, particularly in a partially automated manner, and which has at least a 202401244
[0012] 3
[0013] The appliance in question is a household appliance that has a setting and is designed, in particular, to be switchable on and off. Preferably, the household appliance has at least one switching-on control, for example, a button and / or a switch, by means of which the household appliance can be switched on and, in particular, off by an operator. The household appliance is designed, in particular, as a kitchen appliance. For example, the household appliance could be a cooking appliance, such as a hob, in particular an induction hob, or an oven, or a microwave, and / or a grill, and / or a steamer, or an air fryer, or a rice cooker, or the like, and / or another type of processing appliance, such as a food processor, and / or a mixer, or the like.The household appliance is preferably designed as an electric appliance and is specifically intended for a cooking function powered by electricity to carry out the recipe step. Alternatively, the household appliance may be designed differently from an electric appliance, for example, gas-powered, such as a gas cooktop or the like. The household appliance is specifically designed as a user-operated appliance, which is available to the operator for carrying out the recipe step.
[0014] The illustration includes, in particular, at least one image, especially in an image file format known to those skilled in the art, of the cooking process. Preferably, the illustration is designed as at least one video, especially in a video file format known to those skilled in the art, of the cooking process. The phrase "the recipe corresponds to the cooking process" means, in particular, that the recipe imitates and / or reproduces the cooking process, especially with the greatest possible similarity based on the information available in the illustration and / or the means available to the operator, especially the at least one household appliance and / or ingredients. The cooking process includes, in particular, at least one cooking step, preferably more than one cooking step, each of which corresponds to a step in the recipe.The cooking process is preferably comprised of at least one cooking process, wherein the at least one recipe step of the recipe corresponds in particular to one cooking process of the cooking process. At least one cooking process of the cooking process corresponding to the recipe step is preferably represented in the figure by means of the at least one 202401244.
[0015] 4
[0016] The household appliance and the model household appliance are designed to be similar, particularly with regard to providing at least one type of cooking function, such as a cooking function, a mixing function, or the like, for carrying out the corresponding recipe step and the cooking process itself. They are preferably designed to be compatible with each other. The household appliance and the model household appliance may be identical or at least partially different, but are designed to be compatible with each other, particularly with regard to their suitability for carrying out the corresponding recipe step and the cooking process itself, especially with regard to providing at least one corresponding cooking function.Alternatively or additionally, it is conceivable that, based on the comparison, at least one setting of the at least one household appliance is selected for the execution of at least one recipe step, which corresponds to a cooking process carried out manually in the illustration, whereby the subsequent recipe step can be automated, particularly compared to the corresponding manually executed cooking process. The cooking process is understood to be, in particular, a food preparation process that preferably includes at least one cooking process carried out by means of the at least one household appliance shown, in which food is prepared by the application of heat, but can also be free of a cooking process, such as mixing and / or puréeing and / or kneading and / or pressing and / or filtering and / or the like using the at least one household appliance shown.Preferably, more than one cooking parameter is recognized with respect to the cooking process, particularly with respect to at least one cooking process of the cooking process, and in particular, the comparison is performed based on the more than one cooking parameter. Preferably, at least one, and more preferably than one, cooking parameter of each cooking process of the depicted cooking process is recognized. Advantageously, the similarity of the recipe to the cooking process can be improved. The cooking parameters preferably provide at least information regarding a sequence corresponding to a sequence of cooking processes depicted in the figure and / or a sequence of cooking parameters within a cooking process. Preferably, the cooking parameter is designed as a machine-readable parameter. The cooking parameter preferably has at least one character, in particular at least one Unicode 202401244 character.
[0017] 5
[0018] The cooking parameter consists of characters, preferably at least one letter and / or at least one number and / or at least one special character, making it particularly efficient for machine reading. Alternatively or additionally, the cooking parameter may, for example, include at least a section of the image containing relevant information regarding the cooking process, and / or other data types extracted during recognition that appear useful to a person skilled in the art, particularly types of metadata.
[0019] The digital catalog is preferably designed as a manufacturer's database belonging to the manufacturer of the household appliance and / or a manufacturer of a household appliance application, particularly one mentioned below. The digital catalog preferably contains predefined recipes for dishes, and for each predefined recipe, the settings associated with executing the predefined recipe, such as power level, temperature, cooking time, vacuum and / or steam production, and / or the like, for at least one household appliance.
[0020] Preferably, the digital catalog includes, or is part of, at least one "BSH Backend CMS Dish service" and / or a "CMS ReCipe Editor". The digital catalog is preferably customizable and / or expandable by the manufacturer and / or, for example, by the manufacturer's external partners. The digital catalog is preferably stored in a cloud unit. Alternatively or additionally, it would be conceivable for the digital catalog to be stored at least partially in an operator device, particularly a smart device, and / or in a household appliance application, and / or in at least one household appliance. The cloud unit preferably includes at least one cloud, which is provided, in particular, by the manufacturer of the household appliance and / or by another host. The cloud unit is based on a cloud computing model.The digital catalog is preferably designed as at least a recipe collection and preferably includes assignments between predefined recipe parameters, in particular individual predefined recipe steps, and at least one setting of a household appliance, in particular at least the manufacturer's setting, and / or at least one control signal for setting the household appliance to achieve and / or execute a parameter corresponding to the predefined recipe parameter. In the comparison, the determined cooking parameter is preferably compared with the predefined cooking parameters of the digital catalog and in particular with at least one, in particular 202401244.
[0021] 6
[0022] The settings are assigned to predefined recipe parameters that are as similar as possible. At least one setting of the household appliance, which is assigned in the digital catalog to the predefined recipe parameter (which is specifically assigned to at least one recognized cooking parameter), is preferably selected for the execution of at least one recipe step.
[0023] The method is executed, in particular, based on commands from a computer program product, which is preferably at least partially designed as a household appliance connection application. The household appliance connection application is preferably designed as an application for at least one operator terminal. The operator terminal can be a computer and / or advantageously a smart device, such as a smartphone and / or a tablet and / or a smart home assistant and / or a smart speaker, and / or in another configuration of the operator terminal that appears useful to a person skilled in the art. The operator terminal preferably has a user interface with a visual user interface and / or with at least one control element and / or with at least one acoustic user interface.Preferably, the operator terminal is configured to output the image to the operator and / or access the image. The household appliance connection application is preferably designed as an application for controlling connectable household appliances in an operator's household, particularly via the operator terminal that at least partially stores the household appliance connection application. Advantageously, a particularly high level of user-friendliness can be achieved. The household appliance connection application can, for example, be designed as a Home Connect app adapted with respect to the method according to the invention or as another comparable application for controlling household appliances.The computer program product can comprise the entire household appliance connection application or only a part thereof, and / or be designed as an update to an existing household appliance connection application according to the invention. The computer program product is preferably stored at least partially in the operator terminal and / or in the cloud unit and / or in the at least one household appliance. Preferably, the method is at least partially implemented by the at least one operator terminal and / or by the cloud unit and / or by 202401244.
[0024] 7
[0025] the household appliance, in particular by means of at least one computing and / or storage unit thereof.
[0026] The method preferably comprises at least one process step, in particular an initialization step, in which the operator initializes the method by means of at least one operator input at an operator interface, in particular of the operator terminal device and / or the household appliance, preferably by means of a function of the household appliance application. The operator selects the at least one image from the database by means of the operator input, in particular in the initialization step. The method preferably comprises a further process step, in particular a transmission step, in which at least one piece of information regarding the operator input and / or regarding the image and / or the image itself is transmitted, in particular to the cloud unit and / or, for example, to the household appliance and / or the operator terminal device, for the execution of the recognition of the at least one cooking parameter.The information relating to the image is preferably configured as an identification parameter, for example as a URL or the like, by means of which the image can be accessed unambiguously. Preferably, the transmission is configured as a data transmission and / or wireless transmission, for example via WLAN and / or LAN and / or a mobile data connection and / or Bluetooth and / or another connection that would appear suitable to a person skilled in the art.In the transmission step, the information regarding the operator input for selecting the image and / or the information regarding the selected image and / or, for example, the image itself, is preferably transmitted from a device and / or unit where the operator input for selecting the image was executed to another device and / or unit, which performs the recognition of at least one cooking parameter. The recognition of the at least one cooking parameter based on the image, particularly in a recognition step of the method, is preferably performed by the cloud unit and / or, for example, at least partially by the operator terminal, wherein the cloud unit and / or the operator terminal, in particular, at least partially stores and / or executes an algorithm for the recognition.Alternatively or additionally, it is conceivable that the recognition, in particular in the recognition step of the procedure, is carried out by the household appliance202401244.
[0027] 8
[0028] The process is carried out by the household appliance, in particular by storing and / or executing at least part of the recognition algorithm. For the execution of the recognition by the user device or by the household appliance, the method can be designed to be free of the transmission step, with both the initialization step and the recognition step being executed on the user device or on the household appliance. When recognizing the cooking parameter from the representation of the cooking process, particularly in the recognition step, a first recognition sub-step is preferably executed in which the individual cooking processes of the cooking process are recognized and the representation is divided, in particular, according to the cooking processes.Preferably, when recognizing the cooking parameter from the representation of the cooking process, particularly in the recognition step, a second recognition sub-step is performed in which the at least one cooking parameter is recognized for each section of the subdivided representation assigned to a respective cooking process. The recognition algorithm is preferably designed to subdivide the representation based on the individual cooking processes of the cooking process recognized by the algorithm, and in particular to recognize the at least one cooking parameter for each cooking process, especially individually. Preferably, the comparison, particularly in a comparison step, is performed at least partially, and preferably completely, by the cloud unit, wherein the cloud unit in particular stores and / or executes at least partially an algorithm for performing the comparison.Alternatively or additionally, it would be conceivable that the comparison is performed at least partially by the user terminal and / or by the household appliance, wherein the user terminal and / or the household appliance, in particular, stores and / or executes at least partially an algorithm for performing the comparison. It is conceivable that the method, particularly in the absence of the transmission step, includes at least one communication step, which takes place in particular between the recognition step and the comparison step, in which the at least one recognized cooking parameter is communicated for comparison, in particular to the cloud unit and / or, for example, to the user terminal and / or to the household appliance. Preferably, the communication of the at least one recognized cooking parameter is implemented electronically and / or wirelessly.
[0029] 9
[0030] The method can, by way of example, include a further process step, in particular a storage step. It is conceivable that the recipe corresponding to the cooking process, and in particular the associated setting for executing the recipe step, is stored, especially non-volatilely, for example as a favorite, particularly in the storage step, whereby the operator can repeatedly retrieve the recipe, in particular without having to re-identify the at least one cooking parameter and compare it again, in particular without having to re-analyze the image. The household appliance application and / or the cloud unit and / or the operator terminal and / or the household appliance and / or, by way of example, the digital catalog, can store the recipe, in particular the at least one setting.It is conceivable that the recipe corresponding to the cooking process is only saved when a corresponding operator input is executed, for example, to favorite, archive, or similar actions. Alternatively or additionally, it is conceivable that the recipe and / or recipe steps are only stored temporarily, in particular only for a single execution of the recipe, preferably for a single output and / or control, wherein the method is executed for each execution of the same recipe based on the at least one image in order to determine the recipe.
[0031] Preferably, in the method, at least one additional cooking parameter of the cooking process representation is recognized, particularly in the recognition step, and compared, for example, with the digital catalog and / or processed using the comparison algorithm. Based on the recognized additional cooking parameter, for example, based on the comparison and / or processing, at least one manual instruction regarding a manually executed recipe step of the recipe corresponding to the cooking process is generated. The additional cooking parameter is specifically designed as a cooking parameter of a cooking process that is carried out without the use of a household appliance, particularly manually, for example, by chopping and / or stirring and / or cutting and / or peeling and / or the like.The manual instruction can, in particular, suggest and / or order a manual cooking operation by the operator, which corresponds specifically to the manually executed cooking process. Alternatively or additionally, the operator instruction can be generated based on a comparison of the cooking parameter with the digital catalog, whereby the algorithm to the202401244.
[0032] 10
[0033] The comparison and / or the digital catalog provides, in particular, a manual operator operation to be performed in relation to the setting of the household appliance, for example, stirring during a cooking process or the like, on which the manual instruction is based. The manual instruction is preferably issued to the operator in a manner identical to and / or corresponding to a note mentioned below.
[0034] The term "intended" should be understood to mean specifically programmed, designed, and / or equipped. The fact that an object is intended for a specific function should be understood to mean that the object fulfills and / or executes this specific function in at least one application and / or operating state.
[0035] Furthermore, it is proposed that at least one household appliance be controlled independently based on the selected setting and / or that the selected setting be displayed as a notification, particularly to the operator. Advantageously, at least one recipe step can be executed at least partially autonomously by the household appliance according to the selected setting, thereby increasing ease of use. In particular, the number of operator inputs on the household appliance can be reduced to a single setting of the selected setting. It is conceivable that the household appliance could be controlled completely autonomously based on the setting, whereby the setting is configured on the appliance, ideally without requiring any operator input, and the recipe step is executed accordingly.Alternatively, it is conceivable that, for the execution of the recipe step associated with the selected setting, at least one operator input is requested and / or executed, for example, at the household appliance and / or at the operator's terminal device, to release the selected setting that was automatically set in the household appliance and / or to manually select the selected setting, particularly based on the instruction. The instruction can be designed as information about the selected setting, based on which the household appliance is controlled, in particular completely, automatically. Alternatively, the instruction, particularly in the absence of automatic control based on the selected setting, can be designed as a suggestion and / or as an instruction based on which the operator can use the selected setting.
[0036] 11
[0037] The settings can be adjusted manually. Displaying this message can improve the operator's overview of the cooking process and / or enhance safety. Specifically, the operator can conveniently lock, stop, or adjust a selected setting, or ignore the suggestion. Advantageously, the requirement for a connection to transmit the selected setting can be reduced, making even non-internet-enabled appliances suitable for executing this recipe step. The message is preferably displayed visually. Alternatively or additionally, it is conceivable that the message could be displayed audibly and / or haptically.Preferably, the notification is issued by the operator terminal device, in particular in the household appliance connection application and / or in a widget integrated into a user interface which is provided for access to the database, in particular integrated into a social network, and / or in a stand-alone application and / or by means of at least one loudspeaker, and / or by the household appliance, in particular at a user interface of the household appliance.
[0038] The method may include a process step, in particular a transmission step, in which the selected setting and / or at least information regarding the setting after comparison, preferably from the cloud unit, is transmitted to the operator terminal and / or to the household appliance for the output of the notification, and / or in which at least one control signal for controlling the household appliance according to the selected setting is transmitted, in particular from the cloud unit to the operator terminal.The method preferably includes a process step, in particular an output step, in which the message is output to the operator at the operator terminal device and / or at the household appliance and / or the control signal for the independent control, in particular directly from the cloud unit and / or via the operator terminal device, is output to the household appliance, in particular to a control unit of the household appliance.
[0039] Furthermore, it is proposed that the database be designed as a publicly accessible video and / or image database, in particular as an online database, preferably at least one online platform, especially preferably a social network and / or, for example, a blog and / or the like. Advantageously, the 202401244
[0040] 12
[0041] The image for recognition can be read and / or analyzed directly from the publicly accessible video and / or image database, thereby reducing storage requirements, particularly by avoiding the need for non-volatile storage of the image in the cloud unit, the user's terminal device, and / or the household appliance. Advantageously, the image can be accessed and shared with the cloud unit in a particularly simple and efficient manner, especially via the identification parameter. In particular, user-friendliness can be increased by reducing the number of user inputs required to initialize the process and / or by allowing the user to select the image from a particularly aesthetically pleasing user interface on the social network, accessible via the user's terminal device and / or the household appliance.The method can be designed, in particular, without requiring non-volatile storage of the image. The image used for recipe recognition is specifically stored non-volatilely in the publicly accessible video and / or image database. Alternatively or additionally, it is proposed that the database be designed as a local video and / or image database on a user terminal, particularly the one mentioned above. Advantageously, this increases flexibility regarding the image's origin, especially by allowing the user to download an image of a cooking process stored on any online database to the local video and / or image database of the user terminal and use it to execute the method. In particular, the user can initiate the method very easily using a self-filmed and / or self-photographed image of a cooking process.Advantageously, the image can be analyzed based on a video and / or image upload created by the operator, particularly to the cloud unit and / or appliance connection application, to identify at least one cooking parameter. The image of the cooking process is stored non-volatilely in the memory of the operator's device. The image of the cooking process is also stored non-volatilely in the local video and / or image database.
[0042] Alternatively or additionally, it would be conceivable that the database is designed as a video and / or image database stored in the cloud unit. For example, the operator could control the image using the operator's terminal device, in particular using the 202401244
[0043] 13
[0044] The appliance connection application uploads the image of the cooking process to the cloud unit for non-volatile storage and retrieves it from there. Preferably, the image stored in the cloud unit is accessible to the operator via the operator's terminal device, in particular by means of the appliance connection application. Preferably, however, the cloud unit is configured without non-volatile storage of the at least one image.
[0045] Furthermore, it is proposed that the at least one cooking parameter contain at least one piece of information regarding a dish name and / or at least one ingredient and / or at least one cooking method and / or at least one result of the cooking process and / or at least one physical characteristic measurable in the cooking process and / or at least one result of the cooking process and / or at least one, in particular the aforementioned, household appliance used in the cooking process. Advantageously, the cooking parameter can describe the cooking process with particular precision and / or efficiency, and in particular, the cooking parameter contains particularly relevant information regarding the cooking process. In addition to information regarding the name of the dish cooked in the cooking process, the cooking parameter can contain at least one piece of information regarding a synonym of the dish name.The at least one piece of information regarding the ingredient may include at least an identification of a type of ingredient and / or information regarding the relevance of the ingredient, for example by designating it as a main ingredient or a minor ingredient, and / or at least information regarding a quantity of the ingredient and / or at least an identification of a state of the ingredient, for example regarding a raw, cut, frozen, soaked state of the ingredient or the like, and / or at least other information regarding the ingredient that appears meaningful to a person skilled in the art.The at least one piece of information regarding the cooking method of the cooking process preferably includes at least one piece of information regarding at least one cooking method, for example, boiling and / or frying and / or grilling and / or steaming and / or simmering and / or brothing and / or deep-frying and / or braising and / or the like, and / or regarding at least one processing method, for example, mixing and / or puréeing and / or kneading and / or pressing and / or filtering and / or the like, of at least one ingredient in at least one cooking process of the cooking process. The physical parameter can, for example, be a duration of the cooking process and / or a cooking process.
[0046] 14
[0047] The information regarding the result of the cooking process can be configured as information regarding a final result and / or an intermediate result of the cooking process, in particular as the final result of a cooking step, and / or is preferably configured as information regarding a state of the result, for example, regarding color and / or consistency and / or shape and / or degree of doneness, for example, medium, rare, well-done, and / or the like.
[0048] Preferably, the cooking parameter, in particular the information regarding the imaginary household appliance, includes at least information regarding a type of imaginary household appliance used in the cooking process and / or a power level set in the cooking process, in particular a heating level and / or speed level and / or a pressure level and / or the like, of the imaginary household appliance used in the cooking process.
[0049] Alternatively or additionally, at least one of the pieces of information mentioned above regarding the cooking parameter can be determined by the algorithm for the execution of the comparison based on the cooking parameter, particularly during the comparison.
[0050] Furthermore, it is proposed that the cooking parameter be recognized using an AI algorithm and / or an OCR algorithm. The abbreviation "AI" stands for "Artificial Intelligence." The abbreviation "OCR" stands for "Optical Character Recognition." Advantageously, the cooking parameter can be recognized in a particularly reliable and intelligent manner. In particular, even a cooking parameter that is not recognizable by a conventional, especially non-AI, algorithm can be recognized using the AI algorithm. Advantageously, a particularly large amount of information, especially a large number of cooking parameters, can be recognized in the image. In particular, the cooking parameter can be recognized using a highly adaptable algorithm that can, for example, continue to learn during the execution of the procedure.The AI algorithm may include an AI algorithm specifically developed and / or compiled for the purpose of recognizing at least one cooking parameter of the cooking process. Alternatively or additionally, it would be conceivable that the AI algorithm includes at least one GPAI202401244.
[0051] 15
[0052] (General Purpose AI), for example ChatGPT or similar, is used for or based on the recognition.
[0053] Furthermore, it is proposed that the comparison be performed using an AI algorithm. Advantageously, the comparison can achieve a particularly reliable result and be performed using a highly adaptable algorithm.
[0054] In particular, the comparison can be based on a complex input, especially on at least one cooking parameter, and a complex similarity measure.
[0055] Advantageously, a predefined recipe parameter can also be assigned to a different, but as similar as possible, cooking parameter. The AI algorithm is preferably designed to assign at least one cooking parameter to at least one entry in the digital catalog that is as similar as possible, in particular to the recipe parameter to which at least one setting is assigned in the digital catalog.
[0056] Furthermore, it is proposed that the comparison should include at least one user-specific instruction regarding a household appliance available to the user. The comparison is particularly advantageous if it is tailored to the user's specific circumstances, thereby increasing ease of use. It is also advantageous to determine a recipe specifically tailored to the user based on the illustration, which is particularly easy for the user to execute.The algorithm for selecting the setting is specifically designed to consider only entries in the digital catalog that are assigned to at least one household appliance available to the user, in particular the type of household appliance available to the user, and / or converts at least one setting recognized in the illustration, in particular regarding a cooking time and / or a power level, of the household appliance shown in the illustration to a setting corresponding to the household appliance. For example, if a gas hob is used in the illustration, a gas heating level is converted to an induction heating level for the selected setting if the user only has an induction hob and no gas hob. The household appliance whose at least one setting is determined for executing the recipe step is specifically designed as the household appliance available to the user.The comparison preferably considers at least one setting of the 202401244.
[0057] 16
[0058] The user selects the household appliance available to them. The user information is preferably stored in the household appliance connection application. For example, the user information can be stored in an user profile. During the comparison, at least one additional piece of user information can be taken into account, for example, regarding other cooking utensils available to the user, and / or the user's cooking skills, and / or the user's preferences, and / or ingredients available to the user, and / or the like.
[0059] Furthermore, it is proposed that the selected setting and / or the recipe be modifiable and / or expandable based on at least one operator input. Advantageously, this can increase the adaptability of the recipe step, particularly to operator needs, and / or the safety during the execution of the recipe step. The operator input preferably takes place at the user interface of the household appliance and / or at the operator's terminal device, especially within the household appliance application. For example, the operator can modify or specify entries such as a time schedule and / or sequence of recipe steps for the execution of the recipe. For example, a setting that the operator perceives as unsafe and / or disadvantageous can be changed.
[0060] Furthermore, a computer program product, in particular the aforementioned one, and in particular a household appliance connection application, comprising commands that, when executed by a computer, cause it to execute the procedure, is proposed. Advantageously, the procedure can be executed by means of the household appliance connection application and, in particular, controlled and / or monitored and / or initiated by the operator.
[0061] Furthermore, a computer-readable storage medium comprising instructions that, when executed by a computer, in particular the one mentioned above, cause it to execute the method, is proposed. The computer-readable storage medium preferably stores the computer program product. The computer-readable storage medium is preferably at least part of the household appliance and / or the operator terminal and / or the cloud unit, in particular a computing and / or storage unit thereof.
[0062] 17
[0063] The method, the computer program product, and the computer-readable storage medium are not limited to the application and embodiment described above. In particular, the method, the computer program product, and the computer-readable storage medium may, to achieve a functionality described herein, have a different number of process steps, individual elements, components, and units than specified herein.
[0064] Further advantages will become apparent from the following description of the drawing. The drawing illustrates an embodiment of the invention. The drawing, the description, and the claims contain numerous features in combination. A person skilled in the art will expediently consider the features individually and combine them into meaningful further combinations.
[0065] It shows:
[0066] Fig. 1 A flowchart of a process for recipe recognition and execution.
[0067] Figure 1 shows a flowchart of a method for recipe recognition and execution, wherein at least one cooking parameter of a cooking process stored in a database 12 is recognized and compared with a digital catalog 16, wherein, based on the comparison, at least one setting of at least one household appliance 20 is selected for the execution of at least one recipe step of a recipe corresponding to the cooking process from the catalog 16.
[0068] The procedure includes an initialization step 100. In initialization step 100, the cooking process diagram 14 is selected from database 12 for recipe recognition by means of at least one operator input from an operator 30. The operator input is executed on an operator terminal 18, in particular a smart device.
[0069] Alternatively, it is conceivable that the operator input for the selection shown in Figure 14 is performed on a different type of operator terminal (not shown) that would be useful to a person skilled in the art and / or on an operator interface 36 of the household appliance 20.
[0070] 18
[0071] Figure 14 contains at least one image. Figure 14 is designed as a video 24. Alternatively, another design of Figure 14, which would appear sensible to a person skilled in the art, would be conceivable.
[0072] Database 12 is configured as a publicly accessible video and / or image database 22. Database 12 is configured as an online database. Database 12 is configured as a database 12 of a social network 32. The operator terminal 18 accesses database 12 at least for the selection shown in Figure 14.
[0073] Alternatively or additionally, the database 12 can be designed as a differently structured database 12 that appears sensible to the expert, in particular as a differently structured online database.
[0074] Alternatively or additionally, the database 12 can be configured as a local video and / or image database (not shown) of the operator terminal 18.
[0075] For example, database 12 may contain the local video and / or image database of the operator terminal 18 and / or the publicly accessible video and / or image database 22.
[0076] The procedure includes a transmission step 102, in which at least one piece of information regarding the operator input for the selection of Figure 14 and / or the selected Figure 14 is transmitted to a cloud unit 28. For example, based on the operator input for the selection of Figure 14, a URL associated with Figure 14 is transmitted to the cloud unit 28.
[0077] Alternatively, it is conceivable that the procedure is designed without transmission step 102, or that the information regarding the operator input for the selection of Figure 14 and / or the selected Figure 14 is transmitted in transmission step 102, in particular from the household appliance 20, to the operator terminal 18.
[0078] The procedure includes a recognition step 104, in which at least one cooking parameter from Figure 14 is recognized. The recognition of the cooking parameter is performed by the cloud unit 28. Alternatively or additionally, 202401244
[0079] 19
[0080] For example, it is conceivable that the operator terminal 18 performs the recognition at least partially.
[0081] The recognition of the cooking parameter is performed by an algorithm. The algorithm first identifies individual cooking processes within the cooking process shown in Figure 14. The algorithm then subdivides Figure 14 according to the individual cooking processes identified by the algorithm. Based on the subdivided Figure 14, the algorithm identifies at least one cooking parameter of each individual cooking process. This at least one cooking parameter contains at least one piece of information regarding a dish name and / or at least one ingredient 34 and / or at least one cooking method and / or at least one result (not shown) of the cooking process and / or at least one measurable physical characteristic of the cooking process and / or at least one household appliance 40 used in the cooking process.
[0082] The information regarding the illustrated household appliance 40 includes at least one piece of information regarding the type of illustrated household appliance 40 used in the cooking process. In this example, the illustrated household appliance 40 is designed as a cooktop.
[0083] The information relating to the illustrated household appliance 40 includes at least information relating to at least one power level set in the cooking process, in particular at least one heating level, of the illustrated household appliance 40 used in the cooking process.
[0084] The cooking parameter is recognized using a computer algorithm and / or an OCR algorithm.
[0085] The procedure includes a comparison step 106, in which the comparison and, based on the comparison, the at least one setting of the household appliance 20 for the execution of the at least one recipe step from the catalog 16 is selected. Comparison step 106 is performed in this case by the cloud unit 28. Alternatively, 202401244
[0086] 20
[0087] or additionally, it would be conceivable, for example, that the operator terminal 18 performs the comparison step 106 at least partially.
[0088] Alternatively, it is conceivable, for example, that the recognition step 104 is carried out by the operator terminal 18 and the comparison step 106 by the cloud unit 28, wherein at least one communication step (not shown) is arranged between the recognition step 104 and the comparison step 106, in which at least one cooking parameter is transmitted to the cloud unit 28.
[0089] The comparison, in particular comparison step 106, is performed by an algorithm, in particular by another algorithm. The comparison is performed using a computer algorithm. The at least one cooking parameter and the digital catalog 16, and in particular at least one further piece of input information, serve as input values for the algorithm.
[0090] The digital catalog 16 contains entries in which predefined recipes are assigned to settings of household appliances.
[0091] The comparison is performed here using an AI algorithm based on a neural network 50 as an example. For the comparison of at least one cooking parameter with the digital catalog 16, the algorithm has at least one input layer 52. Input layer 52 has at least one neuron 54, which has at least one cooking parameter as an input value. Input layer 52 has at least one neuron 56, which has at least one digital catalog 16 as an input value.
[0092] The input layer 52 has at least one neuron 58, which has at least one additional input piece of information besides at least one input value. At least one piece of operator information is taken into account during the comparison. This operator information can be implemented as part of the additional input information.
[0093] The comparison takes into account at least one piece of operator information regarding a household appliance 26 available to the operator 30. The household appliance 20, whose at least one setting for executing the recipe step is determined, is configured as the household appliance 26 available to the operator 30. In the comparison 202401244
[0094] 21
[0095] At least one setting of the household appliance 26 available to the operator 30 is selected.
[0096] The household appliance 20, 26 is designed as a household appliance 20 corresponding to the illustrated household appliance 40, in particular at least with regard to one type of household appliance, which is available to the operator 30. The household appliance 20 is designed in this case as a cooktop.
[0097] During the comparison, the household appliance 20 suitable for carrying out the recipe step is selected, in particular from a list of household appliances 26 available to the operator 30. During the comparison, the algorithm converts at least one setting of the household appliance 40 shown in Figure 14 into a setting corresponding to the selected household appliance 20, 26.
[0098] Alternatively or additionally, at least one further operator information, for example regarding further cooking utensils available to the operator (not shown) and / or cooking skills of the operator and / or preferences of the operator and / or ingredients available to the operator and / or the like, can be taken into account in the comparison.
[0099] For comparison purposes, the algorithm in this case has at least one hidden layer 60. The hidden layer 60 may also contain and / or process further metadata.
[0100] For comparison purposes, the algorithm has at least one output layer 62. Output layer 62 has at least one selected setting.
[0101] The procedure includes a transmission step 108 in which the selected setting and / or at least information regarding the selected setting is transmitted to the operator terminal 18.
[0102] Alternatively, it is conceivable that the selected setting and / or at least information regarding the selected setting, in particular in the form of a control signal, is transmitted directly from the cloud unit 28 to the household appliance 20. The procedure has an output step 110 in which the selected setting and / or at least information, in particular in the form of a 202401244
[0103] 22
[0104] A control signal is output to the household appliance 20 for controlling the household appliance 20 according to the selected setting. At least one household appliance 20 is controlled independently based on the selected setting, particularly in output step 110.
[0105] Alternatively or additionally, it is conceivable that the selected setting is displayed as a message, particularly in output step 110. This message is displayed on the user interface 36 of the household appliance 20 and / or on the operator terminal 18 and / or on another operator terminal (not shown). It is also conceivable that the selected setting can be adjusted on the household appliance 20, for example, solely by means of operator input, particularly on the user interface 36 of the household appliance 20 and / or on the operator terminal 18. Based on the selected setting, the household appliance 20 can be controlled at least partially manually by the operator 30 in output step 110.
[0106] The selected setting and / or recipe can be modified and / or extended based on at least one operator input. The operator input takes place on the device intended for displaying the message, in particular on the operator interface 36 of the household appliance 20 and / or on the operator terminal 18. The operator input is based on the message.
[0107] A computer program product, in particular a household appliance connection application, comprises commands which, when the program is executed by a computer (not shown), cause it to execute at least part of the above-mentioned procedure. The computer may include a computing and / or storage unit (not shown) of the operator terminal 18 and / or the cloud unit 28 and / or the household appliance (20).
[0108] A computer-readable storage medium (not shown), comprising instructions which, when executed by the computer, cause it to execute the procedure at least partially, stores the computer program product. The computer-readable storage medium may include a computing and / or storage unit (not shown) of the operator terminal 18 and / or the cloud unit 28 and / or the household appliance (20).
[0109] 23
[0110] Of objects that appear multiple times in the figure, only one can be labelled with a reference symbol. 202401244
[0111] 24
[0112] Reference sign
[0113] 12 Database
[0114] Figure 14
[0115] 16 digital catalog
[0116] 18 Operator terminal
[0117] 20 household appliances
[0118] 22 Database
[0119] 24 Video
[0120] 26 household appliances
[0121] 28 cloud units
[0122] 30 operators
[0123] 32 social network
[0124] 34 ingredients
[0125] 36 User interface
[0126] 38 ingredients
[0127] 40 Imaging household device 50 Neural network 52 Input layer
[0128] 54 neurons
[0129] 56 neurons
[0130] 58 neurons
[0131] 60 hidden layers
[0132] 62 Output layer
[0133] 100 Initialization step 102 Transmission step 104 Recognition step
[0134] 106 Comparison step
[0135] 108 Transmission step 110 Output step
Claims
202401244 25 Claims 1. Method for automatic recipe recognition and execution, wherein at least one cooking parameter of an image (14), in particular a video (24), of a cooking process stored in a database (12) is recognized and compared with a digital catalog (16), wherein, based on the comparison, at least one setting of at least one household appliance (20) is selected for the execution of at least one recipe step of a recipe corresponding to the cooking process from the catalog (16).
2. Method according to claim 1, characterized in that the at least one household appliance (20) is controlled independently based on the selected setting and / or the selected setting is output as a notification.
3. Method according to one of the preceding claims, characterized in that the database (12) is designed as a publicly accessible video and / or image database (22), in particular of a social network (32).
4. Method according to one of the preceding claims, characterized in that the database (12) is configured as a local video and / or image database of an operator terminal device (18).
5. A method according to one of the preceding claims, characterized in that the at least one cooking parameter comprises at least one piece of information relating to a dish name and / or at least one ingredient (34) and / or at least one cooking method and / or at least one result of the cooking process and / or relating to at least one physical characteristic measurable in the cooking process and / or relating to at least one imaging household appliance (40) used in the cooking process. 26 6. Method according to one of the preceding claims, characterized in that the cooking parameter is recognized by means of a KL algorithm and / or an OCR algorithm.
7. Method according to one of the preceding claims, characterized in that the comparison is performed using a computer algorithm.
8. Method according to one of the preceding claims, characterized in that at least one operator information relating to a household appliance (26) available to the operator (30) is taken into account in the comparison.
9. Method according to one of the preceding claims, characterized in that the selected setting and / or the recipe can be modified and / or extended based on at least one operator input.
10. Computer program product, in particular a household appliance connection application, comprising commands which, when the program is executed by a computer, cause it to execute a method according to one of the preceding claims.
11. Computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to execute a method according to any one of claims 1 to 9.