Visual real-time monitoring system for kitchen

Abstract

The invention discloses a visual real-time monitoring system for a kitchen, belongs to the technical field of monitoring, and solves the problems that the efficiency of kitchen monitoring is relatively poor and the monitoring effect which a restaurant wants to achieve at first cannot be achieved only by the cooperation of manpower. Comprising a monitoring module, a data processing module, a display module and a voice broadcast module, the output end of the monitoring module is coupled with the input end of the data processing module, and the output end of the data processing module is coupled with the input ends of the display module and the voice broadcast module. The monitoring module is used for collecting pictures in a kitchen, generating a monitoring video and transmitting the monitoring video to the data processing module, and the data processing module is used for judging whether food crossing occurs or not according to the monitoring video and feeding back a result to the voice broadcast module. According to the invention, the monitoring and data processing module identifies whether a chef wears a cap or not, if not, voice reminding is carried out, the chef grade is distinguished, the vegetable cutting action of an intermediate chef is monitored, cross contamination of raw meat and vegetables is prevented, the security monitoring burden is reduced, and the monitoring effect is maintained.

Description

Technical Field

[0001] The present invention relates to the field of monitoring technology, and in particular to a kitchen visual real-time monitoring system. Background Art

[0002] The restaurant kitchen is the core area of ​​professional cooking. Compared with the home kitchen, it pays more attention to efficiency and durability. It is equipped with large commercial stoves, ovens, refrigerators, cold storage, dishwashers and other professional equipment to meet the needs of high-intensity and large-volume cooking. The restaurant kitchen design emphasizes the optimization of the workflow, usually adopting a linear or L-shaped layout to ensure that chefs and staff can complete the preparation and cooking of food in the shortest time. The walls and floors are usually made of easy-to-clean materials to maintain hygiene standards. In addition, the restaurant kitchen must also comply with strict food safety regulations. Therefore, from food storage to garbage disposal, every link is carefully designed to ensure food safety. In a busy restaurant kitchen, the chef team needs to work closely to ensure that every dish can be presented to customers in the best condition. This is the factory for creating delicious food and the key to the success of the restaurant.

[0003] At present, there are also visual restaurant kitchens. Visual restaurant kitchens are a kitchen design that makes the cooking process transparent. Customers can intuitively see the chef's cooking operations and the hygiene conditions of the kitchen. This design not only increases the interactivity and viewing of the restaurant, but also enhances customers' trust in food safety. In this open or semi-open layout, chefs work in a bright and clean environment, use professional equipment to cook, and communicate visually with customers. This kitchen design is usually combined with modern decorative style, integrating functionality and aesthetics, so that customers can appreciate the art of cooking while enjoying delicious food.

[0004] However, in the actual working process, customers are unable to monitor the situation in the kitchen in real time. At the same time, customers do not understand the kitchen rules and cannot reasonably monitor the situation in the restaurant kitchen. Under normal circumstances, the restaurant needs to arrange a security personnel to ensure continuous monitoring of the restaurant kitchen after training.

[0005] This not only increases the restaurant's operating costs, but also requires a long period of training for security personnel before each post. There is no guarantee that security personnel can always monitor the surveillance footage to detect problems. Therefore, the efficiency of kitchen monitoring relying solely on manpower is relatively low, and may not achieve the monitoring effect that the restaurant originally wanted to achieve.

[0006] Therefore, a kitchen visualization real-time monitoring system is proposed to solve or alleviate the above problems. Summary of the invention

[0007] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a kitchen visual real-time monitoring system.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] A kitchen visual real-time monitoring system includes a monitoring module, a data processing module, a display module, and a voice broadcast module. The output end of the monitoring module is coupled to the input end of the data processing module, and the output end of the data processing module is coupled to the input ends of the display module and the voice broadcast module. The monitoring module is used to collect images in the kitchen to generate monitoring videos and transmit them to the data processing module. The data processing module is used to determine whether food cross-over occurs based on the monitoring video and feed back the results to the voice broadcast module. The display module is used to display the monitoring video, and the voice broadcast module is used to broadcast voice based on the feedback results of the data processing module.

[0010] Preferably, the monitoring module includes a camera, which is used to capture images in the kitchen to generate monitoring videos and transmit them to the data processing module.

[0011] Preferably, the data processing module includes a processor, and the processor is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module.

[0012] Preferably, the data processing module is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module, including the following steps:

[0013] Obtain surveillance video, and extract each frame from the surveillance video for denoising and normalization processing;

[0014] Using an object detection model to detect the position of each staff member in the surveillance video, and cropping the area of ​​each staff member based on the staff detection frame, the object detection model includes a YOLO model;

[0015] Using a classification model to detect the presence of a hat in the region of each worker, the classification model comprising a convolutional neural network;

[0016] The detection result of each staff member's area is judged as having a hat or not having a hat, and the staff member is marked.

[0017] Preferably, the data processing module is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module, and further includes the following steps:

[0018] The object detection model is used to detect the head and hat of the staff member in the staff area marked with a hat, and the hat classification model is used to identify the head and hat of the staff member in the staff area marked with a hat;

[0019] A hat feature vector is extracted from the hat classification model output, and the hat feature vector is compared with a pre-registered chef hat feature library through cosine similarity to output a chef grade, and mark the chef registration of each staff member. The chef grades include junior chef, intermediate chef, and senior chef.

[0020] Preferably, the data processing module is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module, and further includes the following steps:

[0021] Detect the chefs in the surveillance video according to the target detection model, obtain the bounding box position of each chef, and determine whether the chef is a mid-level chef according to the mark, and filter out the area of ​​chefs with a mid-level chef level;

[0022] Using a posture estimation model to detect the chef's skeleton key points, the skeleton key points include hands, wrists, elbows, and shoulders;

[0023] Identify whether the chef's hand is in the action of holding a knife. If the chef's hand is detected in front of the chopping board and holding the handle of the knife, it is marked as a "holding knife" action;

[0024] Use object detection methods to identify the area of ​​the cutting board and determine its bounding box to determine whether the chef is standing in front of the cutting board;

[0025] By detecting the chef's relative position and direction, it is determined whether he has formed a reasonable interaction with the chopping board, and the output result is that the intermediate chef is cutting vegetables and marked.

[0026] Preferably, the data processing module is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module, and further includes the following steps:

[0027] Intercept the area marked as the middle-level chef cutting vegetables in the surveillance video;

[0028] Use the object detection model to identify whether there is raw meat and vegetables on the cutting board. If both are present, proceed to the next step. If not, repeat the above steps.

[0029] Perform object tracking on detected knives, raw meat, and vegetables, and track the movement and position of each object;

[0030] Using an action recognition model to recognize the chef's vegetable cutting action, and analyzing the contact between the knife and raw meat and vegetables during the vegetable cutting process, to determine whether the chef is processing raw meat or vegetables, and to determine whether the chef transfers the knife from raw meat to vegetables at the same time, or vice versa, the action recognition model comprising a time recurrent neural network;

[0031] If it is determined that the knife is transferred from raw meat to vegetables at the same time, the output result is that there is cross contamination. Otherwise, no result is output.

[0032] Preferably, the display module comprises a display, and the display is used to display the surveillance video.

[0033] Preferably, the voice broadcast module includes a speaker, and the speaker is used to broadcast the voice according to the feedback result of the data processing module.

[0034] The present invention has the following beneficial effects:

[0035] The present invention can identify according to the monitoring video monitored in real time by the monitoring module in conjunction with the data processing module, and then output in the monitoring video whether the chef is wearing the chef hat in compliance with the regulations. If the chef hat is not worn as required, it can be announced through the voice broadcast module. At the same time, when it is determined that everyone wears the chef hat, the level of each chef in the monitoring video can be judged by the different chef hats worn by chefs of different levels, and then the intermediate chef who is responsible for cutting vegetables can be found, and the actions of the intermediate chef can be identified to determine whether he is cutting vegetables. If it is determined that he is cutting vegetables, it is necessary to accurately monitor whether he cross-contaminates raw meat and vegetables and output this wrong behavior through the voice broadcast module, thereby reducing the security personnel's monitoring efforts on the monitoring video in the kitchen and maintaining a decent monitoring effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0037] Figure 1 It is a structural block diagram of the present invention.

[0038] 1. Monitoring module; 2. Data processing module; 3. Display module; 4. Voice broadcast module. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0040] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0042] In the description of the present invention, it should be understood that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the inventive product is conventionally placed when in use, or are the orientations or positional relationships conventionally understood by those skilled in the art. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0043] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0044] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0045] A kitchen visual real-time monitoring system, such as Figure 1As shown, it includes a monitoring module 1, a data processing module 2, a display module 3, and a voice broadcast module 4. The output end of the monitoring module 1 is coupled to the input end of the data processing module 2, and the output end of the data processing module 2 is coupled to the input ends of the display module 3 and the voice broadcast module 4. The monitoring module 1 is used to collect pictures in the kitchen to generate monitoring videos and transmit them to the data processing module 2. The data processing module 2 is used to determine whether food cross-over occurs based on the monitoring video and feed back the results to the voice broadcast module 4. The display module 3 is used to display the monitoring video, and the voice broadcast module 4 is used to broadcast voice based on the feedback results of the data processing module 2.

[0046] The present invention provides an integrated kitchen monitoring system, which realizes automatic supervision of kitchen operations through real-time monitoring and data processing, determines whether there are any violations, and feeds back the results through the display module 3 and the voice broadcast module 4, thereby reducing the workload of security personnel and improving monitoring efficiency.

[0047] Preferably, the monitoring module 1 includes a camera, which is used to capture images in the kitchen to generate monitoring videos and transmit them to the data processing module 2.

[0048] The functions of the monitoring module 1 are implemented by the camera, which can accurately capture the images in the kitchen, provide raw materials for subsequent data processing, ensure that there are no blind spots in monitoring, and enhance the accuracy of monitoring.

[0049] Preferably, the data processing module 2 includes a processor, and the processor is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module 4.

[0050] The processor enhances the system's processing capabilities, enabling rapid analysis of surveillance video, identification of violations such as food cross-contamination, and real-time feedback of results, improving the system's response speed and processing efficiency.

[0051] Preferably, the data processing module 2 is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module 4, including the following steps:

[0052] Obtain surveillance video, and extract each frame from the surveillance video for denoising and normalization processing;

[0053] Use the object detection model to detect the position of each staff member in the surveillance video, and crop the area of ​​each staff member based on the staff detection frame. The object detection model includes the YOLO model.

[0054] Using a classification model including a convolutional neural network to detect the presence of a hat in each worker's area;

[0055] The detection result of each staff member's area is judged as having a hat or not having a hat, and the staff member is marked.

[0056] Through the above method and steps, the recognition accuracy of the system is improved. By preprocessing the video frames, detecting objects, and identifying classification models, the system can accurately identify whether the chef is wearing a hat and judge the chef's level, providing a basis for subsequent judgments on violations.

[0057] Preferably, the data processing module 2 is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module 4, and also includes the following steps:

[0058] The object detection model is used to detect the head and hat of the staff member in the staff area marked with a hat, and the hat classification model is used to identify the head and hat of the staff member in the staff area marked with a hat;

[0059] The hat feature vector is extracted from the hat classification model output, and the hat feature vector is compared with the pre-registered chef hat feature library through cosine similarity to output the chef grade and mark the chef registration of each staff member. The chef grades include junior chef, intermediate chef, and senior chef.

[0060] Through the above method steps, the chef level can be identified and customized monitoring can be provided for chef behaviors at different levels. Mainly by comparing the hat feature vector with the pre-registered hat feature library, the system can distinguish the chef level and mark the level of each staff member, making the monitoring more targeted.

[0061] Preferably, the data processing module 2 is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module 4, and also includes the following steps:

[0062] Detect the chefs in the surveillance video according to the target detection model, obtain the bounding box position of each chef, and determine whether the chef is a mid-level chef according to the mark, and filter out the area of ​​chefs with a mid-level chef level;

[0063] Use the posture estimation model to detect the chef’s skeleton key points, including hands, wrists, elbows, and shoulders;

[0064] Identify whether the chef's hand is in the action of holding a knife. If the chef's hand is detected in front of the chopping board and holding the handle of the knife, it is marked as a "holding knife" action;

[0065] Use object detection methods to identify the area of ​​the cutting board and determine its bounding box to determine whether the chef is standing in front of the cutting board;

[0066] By detecting the chef's relative position and direction, it is determined whether he has formed a reasonable interaction with the chopping board, and the output result is that the intermediate chef is cutting vegetables and marked.

[0067] Through the above method steps, using posture estimation and target detection models, accurate monitoring of the intermediate chef's cutting actions is achieved. The system can determine whether the chef is performing cutting actions, providing technical support for preventing cross contamination.

[0068] Preferably, the data processing module 2 is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module 4, and also includes the following steps:

[0069] Intercept the area marked as the middle-level chef cutting vegetables in the surveillance video;

[0070] Use the object detection model to identify whether there is raw meat and vegetables on the cutting board. If both are present, proceed to the next step. If not, repeat the above steps.

[0071] Perform object tracking on detected knives, raw meat, and vegetables, and track the movement and position of each object;

[0072] Using an action recognition model to recognize the chef's cutting action, and analyzing the contact between the knife and raw meat and vegetables during the cutting process, to determine whether the chef is processing raw meat or vegetables, and to determine whether the chef transfers the knife from raw meat to vegetables at the same time, or vice versa, the action recognition model includes a time recurrent neural network;

[0073] If it is determined that the knife is transferred from raw meat to vegetables at the same time, the output result is that there is cross contamination. Otherwise, no result is output.

[0074] Through the above method and steps, it is possible to accurately monitor and determine whether there is food cross-contamination. By identifying whether the chef handles raw meat and vegetables at the same time, it is possible to determine whether there is a risk of cross-contamination and provide timely feedback.

[0075] Preferably, the display module 3 includes a display, and the display is used to display the monitoring video.

[0076] Preferably, the voice broadcasting module 4 includes a speaker, and the speaker is used to broadcast the voice according to the feedback result of the data processing module 2.

[0077] The present invention realizes intelligent supervision in the kitchen through a real-time monitoring system. The system uses the monitoring video captured by the monitoring module 1 as a video stream, and cooperates with the data processing module 2 to perform intelligent analysis to identify whether the chef wears the chef hat as required. Once it is found that the chef is not wearing the chef hat, the system will issue a warning through the voice broadcast module 4.

[0078] When all chefs wear hats as required, the system can also identify the hats of chefs of different levels, and then identify the intermediate chefs responsible for cutting vegetables. The system further analyzes the actions of the intermediate chefs to determine whether they are cutting vegetables, and monitors whether there is cross contamination of raw meat and vegetables during their operation. Once cross contamination is detected, the system will report it in time through the voice broadcast module 4, effectively reducing the burden of security personnel on kitchen video monitoring while maintaining effective monitoring quality.

[0079] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A kitchen visual real-time monitoring system, characterized in that: The invention comprises a monitoring module (1), a data processing module (2), a display module (3), and a voice broadcast module (4); the output end of the monitoring module (1) is coupled to the input end of the data processing module (2); the output end of the data processing module (2) is coupled to the input ends of the display module (3) and the voice broadcast module (4); the monitoring module (1) is used to collect images in the kitchen to generate monitoring videos and transmit them to the data processing module (2); the data processing module (2) is used to determine whether food crossover occurs based on the monitoring videos and feed back the results to the voice broadcast module (4); the display module (3) is used to display the monitoring videos; and the voice broadcast module (4) is used to broadcast voice based on the results fed back by the data processing module (2).

2. A kitchen visual real-time monitoring system according to claim 1, characterized in that: The monitoring module (1) comprises a camera, which is used to collect images in the kitchen to generate monitoring videos and transmit them to the data processing module (2).

3. A kitchen visual real-time monitoring system according to claim 1, characterized in that: The data processing module (2) comprises a processor, which is used to determine whether food crossover occurs based on the monitoring video and to feed back the result to the voice broadcast module (4).

4. A kitchen visual real-time monitoring system according to claim 1, characterized in that: The data processing module (2) is used to determine whether food crossover occurs based on the monitoring video and to feed back the result to the voice broadcast module (4), and comprises the following steps: Obtain surveillance video, and extract each frame from the surveillance video for denoising and normalization processing; Using an object detection model to detect the position of each staff member in the surveillance video, and cropping the area of ​​each staff member based on the staff detection frame, the object detection model includes a YOLO model; Using a classification model to detect the presence of a hat in the region of each worker, the classification model comprising a convolutional neural network; The detection result of each staff member's area is judged as having a hat or not having a hat, and the staff member is marked.

5. A kitchen visual real-time monitoring system according to claim 4, characterized in that: The data processing module (2) is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module (4), and also includes the following steps: The object detection model is used to detect the head and hat of the staff member in the staff area marked with a hat, and the hat classification model is used to identify the head and hat of the staff member in the staff area marked with a hat; A hat feature vector is extracted from the hat classification model output, and the hat feature vector is compared with a pre-registered chef hat feature library through cosine similarity to output a chef grade, and mark the chef registration of each staff member. The chef grades include junior chef, intermediate chef, and senior chef.

6. A kitchen visual real-time monitoring system according to claim 5, characterized in that: The data processing module (2) is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module (4), and also includes the following steps: Detect the chefs in the surveillance video according to the target detection model, obtain the bounding box position of each chef, and determine whether the chef is a mid-level chef according to the mark, and filter out the area of ​​chefs with a mid-level chef level; Using a posture estimation model to detect the chef's skeleton key points, the skeleton key points include hands, wrists, elbows, and shoulders; Identify whether the chef's hand is in the action of holding a knife. If the chef's hand is detected in front of the chopping board and holding the handle of the knife, it is marked as a "holding knife" action; Use object detection methods to identify the area of ​​the cutting board and determine its bounding box to determine whether the chef is standing in front of the cutting board; By detecting the chef's relative position and direction, it is determined whether he has formed a reasonable interaction with the chopping board, and the output result is that the intermediate chef is cutting vegetables and marked.

7. A kitchen visual real-time monitoring system according to claim 6, characterized in that: The data processing module (2) is used to determine whether food crossover occurs based on the monitoring video and feed back the result to the voice broadcast module (4), and also includes the following steps: Intercept the area marked as the middle-level chef cutting vegetables in the surveillance video; Use the object detection model to identify whether there is raw meat and vegetables on the cutting board. If both are present, proceed to the next step. If not, repeat the above steps. Perform object tracking on detected knives, raw meat, and vegetables, and track the movement and position of each object; Using an action recognition model to recognize the chef's vegetable cutting action, and analyzing the contact between the knife and raw meat and vegetables during the vegetable cutting process, to determine whether the chef is processing raw meat or vegetables, and to determine whether the chef transfers the knife from raw meat to vegetables at the same time, or vice versa, the action recognition model comprising a time recurrent neural network; If it is determined that the knife is transferred from raw meat to vegetables at the same time, the output result is that there is cross contamination. Otherwise, no result is output.

8. The kitchen visual real-time monitoring system according to claim 1, characterized in that: The display module (3) comprises a display, and the display is used to display monitoring video.

9. The kitchen visual real-time monitoring system according to claim 1, characterized in that: The voice broadcast module (4) comprises a speaker, and the speaker is used to broadcast voice according to the feedback result of the data processing module (2).

Images