Unmanned intelligent kitchen visual identification scheduling method and system

Abstract

The invention discloses an unmanned intelligent kitchen visual identification scheduling method and system, and the method comprises the steps: weighing and recognizing food materials and the weight of the food materials, and automatically generating a production work order of the corresponding food materials; matching a production work order, visually identifying food materials of the dishes, and obtaining an automatic production process of the dishes; the priority of the production work orders is calculated, the production work orders are automatically sorted, and cooking equipment is scheduled to complete a cooking link; production work order intelligent scheduling is carried out, a shortest job algorithm is adopted, and tasks with high cooking priorities are scheduled preferentially in combination with production work order priorities. By means of the visual recognition scheduling system, the linkage mechanical arm and the intelligent cooking equipment, full automation of the kitchen cooking link can be achieved, a chef only needs to place food materials of dishes into the basins of the shelf, subsequent automatic cooking can be conducted, tedious manual labor is avoided, meanwhile, it is ensured that the quality and taste of the dishes are stable, and the cooking efficiency is improved. And the cooking difference caused by human factors is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual recognition, and particularly relates to a visual recognition scheduling method and system for an unmanned intelligent kitchen. Background Art

[0002] In large canteens such as factories, institutions, and schools, usually a weekly menu schedule is formulated, covering the dish arrangements for each meal period such as breakfast, lunch, and dinner. The kitchen staff in the back kitchen prepare ingredients according to the daily menu and complete the cooking of the dishes. Traditionally, these large canteens mainly rely on the experience of chefs to arrange the cooking work and the order of serving dishes to ensure that each dish can be completed in a timely manner. However, arranging the cooking order of dishes manually is not only time-consuming and inefficient, but also easily affected by unexpected situations.

[0003] In related technologies, to solve this problem, an intelligent unmanned kitchen is introduced, which can automatically process related dishes and serve meals. However, it is found in actual use that the coordination and cooperation among the robots in the unmanned kitchen still need to be improved and optimized. Summary of the Invention

[0004] The purpose of the present invention is to propose a visual recognition scheduling method and system for an unmanned intelligent kitchen to solve the problems in the prior art.

[0005] To this end, the present invention provides a visual recognition scheduling method and system for an unmanned intelligent kitchen, including:

[0006] A visual recognition scheduling method for an unmanned intelligent kitchen, including:

[0007] S100, weighing and identifying the ingredients and their weights, and automatically generating a production work order for the corresponding ingredients;

[0008] S200, matching the production work order in step S100, and visually identifying the dish ingredients to obtain the automatic production process of the dishes;

[0009] S300, calculating the priority of the production work order;

[0010] S400, intelligent scheduling of the production work order, using the shortest job algorithm, combining the priority of the production work order in step S300, preferentially scheduling the production work order with a shorter cooking duration, and after determining the sorting of the production work order, cooking according to the sorting.

[0011] As a further description of the above technical solution, in step S100, when the ingredients of the dish are weighed by the automatic weighing platform, a production work order for the dish is automatically generated.

[0012] As a further description of the above technical solution, in step S200, the portion basin is located by image recognition technology, and the ingredients in the portion basin are identified.

[0013] As a further description of the above technical solution, in step S300, the calculation of the production order priority includes:

[0014] When the number of production orders is equal to 1, the priority of the production order is the highest, and the scheduling cooking equipment completes the cooking process for the production order;

[0015] When the number of production orders is greater than 1, the production orders are sorted based on the priority obtained from the following priority calculation formula:

[0016]

[0017] where ω 1 , ω 2 , ω 3 , ω 4 are weight coefficients.

[0018] As a further description of the above technical solution, in step S400, the shortest job algorithm includes:

[0019] Combined with the priority calculation formula, increase ω 2 , and preferentially schedule production orders with shorter cooking times.

[0020] A scheduling system for executing the above unmanned intelligent kitchen vision recognition scheduling method includes:

[0021] A dish library and cooking process module for storing the cooking processes of dishes;

[0022] A visual image acquisition and preprocessing module for uniformly processing the collected food ingredient images and storing them in the food ingredient image database;

[0023] A food ingredient weighing module for weighing each food ingredient of a dish, and automatically generating a production order for the current dish after all food ingredients are weighed;

[0024] A food ingredient vision recognition and classification module for identifying the types of food ingredients in each serving basin and searching for matching dishes in the dish library based on the identified food ingredient types;

[0025] A priority calculation module for calculating the priority of production orders to be cooked;

[0026] An intelligent scheduling module that uses the shortest job first algorithm to schedule the cooking order of production orders.

[0027] As a further description of the above technical solution, the food ingredient visual recognition and classification module adopts the MobileNetV3 model. Based on the TensorFlow deep learning framework, the MobileNet V3-large is trained using the labeled food ingredient dataset to learn the food ingredient features. The trained MobileNet V3-large model is used to process the images to identify the specific food ingredient types in each serving basin.

[0028] As a further description of the above technical solution, the food ingredient visual recognition and classification module at least includes a scanner. The scanner is used to identify the image information of the serving basin and obtain the position of each serving basin, the types of food ingredients contained in the serving basin, and the cooking requirements for the corresponding dishes.

[0029] As a further description of the above technical solution, the visual image acquisition and preprocessing module at least includes a high-resolution optical camera;

[0030] Among them, the preprocessing in the visual image acquisition and preprocessing module includes cropping the collected food ingredient images to a unified size, and performing brightness adjustment, contrast adjustment, background noise removal, and normalization processing.

[0031] As a further description of the above technical solution, the cooking processes in the dish library and cooking process module at least include a material using function, a preprocessing function, a cooking function, and a meal serving function.

[0032] Beneficial effects:

[0033] 1. The present invention utilizes a visual recognition scheduling system to link the robotic arm and intelligent cooking equipment, which can realize the full automation of the cooking process in the back kitchen. The chef only needs to place the food ingredients of the dish in the serving basin on the shelf, and then the subsequent automated cooking can be carried out, avoiding cumbersome manual labor. At the same time, it also ensures the stable quality and taste of the dishes and reduces cooking differences caused by human factors.

[0034] 2. The present invention can automatically schedule the production order of the dishes, flexibly respond to peak hours, optimize the production process, and can maximize the working efficiency of the kitchen and improve the utilization efficiency of the overall equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0036] Figure 1Flow schematic diagram of a vision recognition and scheduling method for an intelligent unmanned kitchen provided by the present invention Detailed implementation manners

[0037] The content of the present invention can be more easily understood by referring to the following detailed description of the preferred implementation methods of the present invention and the included embodiments. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. In case of conflict, the definitions in this specification shall prevail.

[0038] The embodiments of the present application provide a vision recognition and scheduling method and system for an intelligent unmanned kitchen, which solve the problem in the prior art that in an intelligent unmanned kitchen, cooking tasks cannot be automatically scheduled according to the actual situation, resulting in chaos during peak hours, inability to serve meals in a timely manner, and affecting the dining experience.

[0039] After the dishes are recognized by vision in the present application, production work orders are automatically generated, and the priorities of the production work orders are calculated through algorithms to achieve intelligent sorting. During the cooking process of multiple production work orders, intelligent scheduling of the production work orders is realized, the cooking order of the production work orders can be automatically adjusted, flexibly corresponding to peak hours, optimizing the production process, maximizing the work efficiency of the kitchen, and improving the utilization efficiency of the overall equipment.

[0040] As Figure 1 shown, a vision recognition and scheduling method for an intelligent unmanned kitchen includes:

[0041] S100, weighing and identifying the ingredients and their weights, automatically generating production work orders corresponding to the ingredients, and performing cooking and processing according to the production work orders during subsequent cooking;

[0042] S200, matching the production work orders in step S100, and visually recognizing the dish ingredients to obtain the automatic production process of the dish. After visually recognizing the ingredients of the dish, the dish can be automatically matched with the production work orders generated in S100, and at the same time, the matching dish and the corresponding automatic production process of the dish are searched in the dish library according to the recognized ingredients, such as the seasonings required for dish cooking, the process of ingredient processing, the automatic production parameters for configuring the dish during cooking, and the meal serving process when serving the meal, etc.;

[0043] S300. Based on the generated production work order, calculate the priority of the production work order. The reference factors for calculating the priority of the production work order include multiple factors such as the urgency of the production work order, the cooking duration of the dishes, the cooking process of the dishes in the work order, and the idle state of the cooking equipment. Among them, the urgency of the production work order will give priority to cooking urgent orders, the cooking duration of the dishes will give priority to cooking dishes with shorter cooking durations and improve the overall throughput of the kitchen, the cooking process of the dishes in the work order will give priority to cooking dishes with the same process, and the idle state of the cooking equipment will give priority to cooking dishes on idle cooking equipment.

[0044] In some embodiments, the following can be referred to:

[0045] When the number of production work orders is equal to 1, the priority of the production work order is the highest, and the cooking equipment is scheduled to complete the cooking process for the production work order;

[0046] When the number of production work orders is greater than 1, the production work orders are sorted based on the priority obtained from the following priority calculation formula:

[0047]

[0048] where ω 1 ,ω 2 ,ω 3 ,ω 4 are weight coefficients, where ω 1 ,ω 2 ,ω 3 ,ω 4 can be adjusted according to the actual situation.

[0049] Through the above technical solutions, the production order, that is, the production sequence of the dishes, can be adjusted in real time, avoiding the problem of untimely meal delivery during peak hours and affecting the meal delivery efficiency.

[0050] S400. Intelligent scheduling of production work orders. The shortest job algorithm is adopted, combined with the priority of the production work orders in step S300, to give priority to scheduling tasks with higher cooking priorities. Specifically, while considering the priority of the production work orders, the work order tasks with shorter durations are processed first. The priority formula is used to approximate the solution of the shortest job algorithm by appropriately increasing the size. When the portion basin filled with ingredients is placed on the shelf at the kitchen entrance, the system automatically determines the optimally matched intelligent cooking equipment. For example, if the current dish is cooked in an oven, the system first detects that the current operating state of the oven is empty. When there are multiple empty ovens, it detects the temperature and humidity requirements of the oven closest to the cooking process of the dish. According to the priority, the robotic arm transports the current shelf to the most suitable cooking equipment for cooking.

[0051] Through the above technical solutions, the cooking efficiency can be further improved, the waiting time required for non-cooking can be minimized as much as possible, and the utilization efficiency of cooking equipment can be increased.

[0052] The present invention also provides a scheduling system for executing the above scheduling method, specifically including:

[0053] A dish library and a cooking process module, which configure dishes and their corresponding ingredients and cooking processes;

[0054] A visual image acquisition and preprocessing module, which is used to uniformly process the collected ingredient images and store them in the ingredient image database;

[0055] An ingredient weighing module, which is used to weigh each ingredient of the dish, and after all ingredients are weighed, automatically generate a production work order for the current dish;

[0056] An ingredient visual recognition and classification module, which is used to identify the types of ingredients in each portion basin, and search for matching dishes in the dish library based on the identified ingredient types;

[0057] A priority calculation module, which calculates the priority according to the urgency of the production work order, the cooking duration, the cooking process of the dishes in the work order, and the idle state of the cooking equipment, and sorts the cooking order of the production work orders according to the calculated priority;

[0058] An intelligent scheduling module, which combines the work order priority and the shortest job first algorithm, preferentially schedules tasks with shorter cooking durations, and reduces the overall waiting time. After determining the production order of the work orders, the robotic arm transports the ingredients to the intelligent cooking equipment for cooking.

[0059] Optionally, the ingredient visual recognition and classification module is equipped with a scanner at the entrance, which is used to identify the image information of each portion basin on the shelf, obtain the position of each portion basin, the types of ingredients contained in the portion basin, and the cooking requirements of the corresponding dishes, such as cooking time, required cooking equipment, cooking procedures, etc. In some embodiments, the visual recognition of ingredients uses the MobileNet V3 model, and the MobileNetV3-large is trained based on the TensorFlow deep learning framework using the labeled ingredient dataset to learn the features of ingredients such as color, shape, and texture. The trained MobileNet V3-large model is used to process the image to identify the specific types of ingredients in each portion basin. Search for matching dishes in the dish library according to the identified ingredients. If the identified ingredients can make multiple dishes, the system will give priority to the matching according to the requirements of the production work order.

[0060] Optionally, the dish ingredient image acquisition and preprocessing module uses a high-resolution optical camera to scan the serving bowls on the shelf to collect images of the ingredients in the serving bowls for each dish. These images contain visual information such as the color, shape, and texture of the ingredients, which can be used to visually identify the ingredients. The collected ingredient images are cropped to a uniform size, and after preprocessing such as brightness adjustment, contrast adjustment, background noise removal, and normalization, they are uniformly stored in the ingredient image database.

[0061] Optionally, the cooking process in the dish library and cooking process module includes four functions: ingredients, preprocessing, cooking and serving. Ingredients are mainly used to configure the ingredients and seasonings needed for cooking dishes, preprocessing is mainly used to configure the preprocessing steps and processes that the ingredients of the dishes need to go through, cooking is mainly used to configure the automatic production parameters of the dishes including ingredients and weight, seasonings and weight, cooking equipment, and cooking programs, and serving is mainly used to configure the serving process of the dishes.

[0062] In some implementations, the chef uses an automatic weighing station to complete the task of weighing the ingredients for the day's cooking, and at the same time generates a production work order; the portion bowls containing the ingredients are placed on the shelf at the entrance, and the camera on the barcode scanner automatically identifies the dishes that need to be made. The system will determine the current number of production work orders. When the number of work orders is greater than 1, it will automatically calculate the priority of each work order, and then combine the shortest job first algorithm to prioritize the tasks with shorter cooking time. After determining the cooking order of the work order, the robot arm links the intelligent cooking equipment to complete the cooking task of the dish.

[0063] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A visual recognition scheduling method for an unmanned intelligent kitchen, characterized in that: include: S100, weighing and identifying the ingredients and their weight, and automatically generating a production work order for the corresponding ingredients; S200, matching the production work order in step S100, visually identifying the ingredients of the dish, and obtaining the automatic production process of the dish; S300, calculate the production work order priority; S400, intelligent scheduling of production work orders, using the shortest job algorithm, combined with the production work order priority in step S300, prioritizes scheduling of production work orders with shorter cooking times, and after determining the order of the production work orders, cooks according to the order.

2. According to claim 1, the visual recognition scheduling method for an unmanned intelligent kitchen is characterized in that: In step S100, after the ingredients of a dish are weighed by the automatic weighing platform, a production work order for the dish is automatically generated.

3. The visual recognition scheduling method for an unmanned intelligent kitchen according to claim 1 is characterized in that: In step S200, the portion bowl is located and the food in the portion bowl is identified by using image recognition technology.

4. The visual recognition scheduling method for an unmanned intelligent kitchen according to claim 1 is characterized in that: In step S300, the calculation of the production work order priority includes: When the number of production work orders is equal to 1, the production work order has the highest priority, and the cooking equipment is scheduled to complete the cooking process for the production work order; When the number of production work orders is greater than 1, the production work orders are sorted based on the priority obtained by the following priority calculation formula: Among them, ω1, ω2, ω3, ω4 are weight coefficients.

5. The visual recognition scheduling method for an unmanned intelligent kitchen according to claim 4 is characterized in that: In step S400, the shortest operation algorithm includes: Combined with the priority calculation formula, ω2 is increased to give priority to scheduling production orders with shorter cooking times.

6. A scheduling system for executing the visual recognition scheduling method of any one of claims 1 to 5, characterized in that: include: The dish library and cooking process module are used to store the cooking process of dishes; The visual image acquisition and preprocessing module is used to uniformly process the acquired food images and store them in the food image database; The ingredient weighing module is used to weigh each ingredient of a dish and automatically generate a production work order for the current dish after all ingredients have been weighed; The food visual recognition and classification module is used to identify the type of food in each serving bowl and search for matching dishes in the dish library based on the identified type of food; A priority calculation module, used to calculate the priority of production work orders to be cooked; The intelligent scheduling module uses the shortest job first algorithm to schedule the cooking sequence of production orders.

7. The dispatching system according to claim 6, characterized in that: The food visual recognition and classification module adopts the MobileNetV3 model. Based on the TensorFlow deep learning framework, MobileNetV3-large is trained with the labeled food dataset to learn food features. The trained MobileNet V3-large model is used to process the image and identify the specific food types in each serving bowl.

8. The visual recognition scheduling method and system for an unmanned intelligent kitchen according to claim 6 is characterized in that: The food visual recognition and classification module includes at least a scanner, which is used to identify image information of portion bowls and obtain the position of each portion bowl, the type of food contained in the portion bowl, and the cooking requirements of the corresponding dish.

9. The dispatching system according to claim 6, characterized in that: The visual image acquisition and preprocessing module at least includes a high-resolution optical camera; The preprocessing in the visual image acquisition and preprocessing module includes cropping the collected food images into a uniform size, and performing brightness adjustment, contrast adjustment, background noise removal, and normalization processing.

10. The unmanned intelligent kitchen visual recognition scheduling method and system according to claim 6, characterized in that: The cooking process in the dish library and cooking process module at least includes material function, pre-processing function, cooking function and serving function.

Images