A machine vision-based kitchen supervision method, system, device and medium

Abstract

A kitchen supervision method, system, device and medium based on machine vision are provided, and relate to the field of kitchen supervision. In the method, a human body small picture is intercepted from a video stream, the human body small picture is input into a preset clothing re-identification model, a first feature vector of a chef's clothing is extracted and a first similarity is calculated, and an alarm is triggered when the first similarity corresponding to a continuous preset number of frames is all less than a first similarity threshold. A face frame is positioned according to face detection and input into a preset mask classification model to detect whether a standard is met, and an alarm is triggered when a continuous preset number of frames do not meet the standard. A head frame is positioned according to head detection and input into a preset hat re-identification model, a second feature vector of a chef's hat is extracted and a second similarity is calculated, and an alarm is triggered when the second similarity corresponding to a continuous preset number of frames is all less than a second similarity threshold. The technical solution provided in the application improves the supervision efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the technical field of kitchen supervision, specifically to a machine vision-based kitchen supervision method, system, equipment, and medium. Background Technology

[0002] Food safety in the catering industry has always been a major public concern, especially in kitchen management. Ensuring that chefs wear uniforms, masks, and hats that meet hygiene standards is crucial for food safety. As the catering industry continues to expand, traditional supervision and management methods are no longer sufficient to meet the growing demands.

[0003] Currently, kitchen supervision typically relies on manual inspections or simple video surveillance devices to check whether staff attire complies with regulations. Manual inspections depend on managers periodically entering the kitchen to observe and record; while traditional video surveillance mainly relies on video playback to review specific moments. However, these methods have significant drawbacks, such as high labor costs, low efficiency, and a tendency to miss inspections.

[0004] Therefore, there is a need for a method that can efficiently detect whether chefs are wearing uniforms, masks, and hats that comply with hygiene regulations. Summary of the Invention

[0005] This application provides a machine vision-based method, system, equipment, and medium for kitchen supervision, which improves the efficiency and accuracy of supervision.

[0006] The first aspect of this application provides a machine vision-based kitchen supervision method, applied to a kitchen supervision platform, the method comprising:

[0007] The video stream is acquired in real time from the target camera, human bodies are detected from the video stream, and a small human body image is cropped with the human body as the center. Face detection and head detection are performed on the small human body image.

[0008] The human body image is input into a preset clothing re-identification model, the first feature vector of the chef's uniform is extracted, and the first feature vector is compared with the feature vector in the first template library to obtain the first similarity. When the first similarity corresponding to a preset number of consecutive frames is less than the first similarity threshold, an alarm is triggered.

[0009] The face detection locates the face frame and inputs the face frame into a preset mask classification model to check whether it is in compliance. An alarm is triggered when a preset number of consecutive frames fail to meet the standard.

[0010] The head frame is located based on the head detection and is input into the preset hat re-identification model. The second feature vector of the chef's hat is extracted and compared with the feature vector in the second template library to obtain the second similarity. An alarm is triggered when the second similarity corresponding to a preset number of consecutive frames is less than the second similarity threshold.

[0011] When an alarm is triggered, a screenshot is saved and the detection result, alarm reason, and timestamp information are sent to the target object.

[0012] Optionally, the method further includes training a preset clothing re-identification model, specifically including:

[0013] The deep neural network architecture is pre-trained using a first human body dataset, enabling the deep neural network architecture to learn the basic features of human body weight recognition.

[0014] The convolutional layer, the first residual layer, the second residual layer, and the third residual layer of the deep neural network architecture are locked. The fourth residual layer, the pooling layer, and the fully connected layer of the deep neural network architecture are adjusted. The deep neural network architecture is trained using a second human body dataset to obtain an intermediate clothing re-identification model. The second human body dataset includes clothing data of chefs under different lighting, angles, postures, and occlusions.

[0015] The intermediate clothing re-identification model is trained using angular metric loss to obtain the preset clothing re-identification model.

[0016] Optionally, training the intermediate clothing re-identification model based on the angular metric loss to obtain the preset clothing re-identification model includes:

[0017] The angle measurement loss is calculated using the following formula:

[0018]

[0019] Where L represents the angle metric loss, y is the correct class label, and θ y θ is the angle between the input sample and the weight vector of the correct class, j is the label of the incorrect class, and θ is the angle between the input sample and the weight vector of the correct class. j is the angle between the input sample and the weight vector of the error class, s is the scale factor (constant), and m is the angular interval (constant).

[0020] Optionally, the method further includes training a preset mask classification model, specifically including:

[0021] Collect a third-party human body dataset, which includes images of multiple chefs in different scenarios, including those not wearing masks, wearing masks improperly, and wearing masks properly. The different scenarios include multiple angles, different lighting conditions, and occlusion situations.

[0022] The third human body dataset is first annotated, the first annotation including the coordinates of the face region, and a face detection model is trained based on the third human body dataset after the first annotation;

[0023] The third human body dataset is labeled a second time, including a category label for mask wearing status. A mask classification model is trained based on the third human body dataset after the second labeling.

[0024] The face detection model and the mask classification model are fused together to obtain the preset mask classification model.

[0025] Optionally, the method further includes training a preset hat re-identification model, specifically including:

[0026] Collect a multimodal dataset of chef hats, which includes brim shape data from top / bottom viewpoints, optical property data of reflective material hat surface, and hat badge feature data under dynamic blur conditions;

[0027] A dual-branch network structure is constructed. The first branch extracts features from the complete chef hat images in the chef hat multimodal dataset through the backbone network to capture global information, including the overall shape, color and texture of the hat. The second branch extracts local texture features under occlusion.

[0028] A contrastive learning method is used to compare the features extracted by the first branch and the second branch. When the comparison result meets the requirements, the preset hat re-identification model is constructed based on the dual-branch network structure.

[0029] Optionally, the method further includes:

[0030] When no alarm is triggered for a consecutive preset time period, new human body images are collected to construct an incremental training set, which includes human body images that have not triggered an alarm within the consecutive preset time period.

[0031] The output of the original model is used as a soft label through knowledge distillation technology. The parameters of the fully connected layer of the original model are adjusted by combining the incremental training set. The original model includes the preset clothing re-identification model, the preset mask classification model, and the preset hat re-identification model.

[0032] The cosine similarity drift detection algorithm is used to monitor the feature vector distribution of the adjusted original model. When the shift in the feature vector distribution exceeds a preset threshold, the adaptive adjustment of the model structure is triggered, and the feature embedding dimension is dynamically expanded.

[0033] Optionally, detecting a human body from the video stream and cropping a small image of the human body centered on the human body includes:

[0034] Thermodynamic features of each pixel in the video stream are extracted. The thermodynamic features include the rate of temperature change and the thermal diffusivity. A segmentation model of moving targets and background is constructed based on the thermodynamic features to distinguish human targets from steam interference areas.

[0035] By calculating the parallax of multiple cameras, depth information of the kitchen scene is obtained, a three-dimensional spatial model of the kitchen scene is constructed based on the depth information, and the three-dimensional spatial model is used to eliminate the detection blind spots caused by occlusion.

[0036] After eliminating areas of steam interference and detection blind spots, a small image of the human body is captured.

[0037] A second aspect of this application provides a machine vision-based kitchen monitoring system, including a video acquisition module, a clothing recognition module, a mask recognition module, a hat recognition module, and an alarm response module, wherein:

[0038] The video acquisition module is configured to acquire video streams from a target camera in real time, detect human bodies from the video stream and capture a small image of the human body centered on the human body, and perform face detection and head detection on the small image of the human body.

[0039] The clothing recognition module is configured to input the small human body image into a preset clothing re-recognition model, extract the first feature vector of the chef's uniform, and compare the first feature vector with the feature vector in the first template library to obtain the first similarity. When the first similarity corresponding to a preset number of consecutive frames is less than the first similarity threshold, an alarm is triggered.

[0040] The mask recognition module is configured to locate the face frame based on the face detection and input the face frame into a preset mask classification model to detect whether it is in compliance with the standard. An alarm is triggered when a preset number of consecutive frames fail to meet the standard.

[0041] The hat recognition module is configured to locate the head frame based on the head detection and input the head frame into a preset hat re-recognition model, extract the second feature vector of the chef's hat, and compare the second feature vector with the feature vector in the second template library to obtain the second similarity. When the second similarity corresponding to a preset number of consecutive frames is less than the second similarity threshold, an alarm is triggered.

[0042] The alarm response module is configured to save a screenshot and send the detection result, alarm reason, and timestamp information to the target object when an alarm is triggered.

[0043] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the foregoing.

[0044] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any of the preceding descriptions.

[0045] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0046] 1. By acquiring video streams from target cameras in real time and detecting human bodies, faces, and heads in the video streams, the system can obtain dynamic information about kitchen staff in real time, promptly identify potential problems, and ensure the real-time and continuous nature of supervision. Utilizing pre-set clothing re-identification models, mask classification models, and hat re-identification models, the system can quickly extract feature vectors from human body thumbnails, face bounding boxes, and head bounding boxes, and compare them with feature vectors in the template library. This allows the system to quickly determine whether the kitchen staff's attire complies with regulations, enabling rapid response.

[0047] 2. By using clothing re-identification models, mask classification models, and hat re-identification models to extract and compare different features, the attire of kitchen staff can be evaluated from multiple perspectives, improving the accuracy and comprehensiveness of the identification. During the training process of the clothing re-identification model and the hat re-identification model, the use of the angle metric loss function enables the model to learn more discriminative feature vectors, further improving the model's recognition accuracy.

[0048] 3. An alarm is triggered when the similarity of a number of consecutive frames is less than the similarity threshold. This continuous frame judgment method can effectively avoid false alarms caused by misjudgment of a single frame and improve the reliability of the alarm. The wearing of clothing, masks and hats are monitored and judged separately. An alarm is triggered when any one of them fails to meet the standard, ensuring comprehensive supervision of the kitchen staff's attire and timely detection and correction of non-standard behaviors.

[0049] 4. When an alarm is triggered, the system will save a screenshot and send the detection result, alarm reason and timestamp information to the target object, providing detailed records and evidence for subsequent investigation and management, facilitating the tracing and analysis of the problem, and helping to take targeted improvement measures. Attached Figure Description

[0050] Figure 1This is a schematic flowchart of the machine vision-based kitchen supervision method disclosed in the embodiments of this application;

[0051] Figure 2 This is another schematic diagram of the machine vision-based kitchen supervision method disclosed in the embodiments of this application;

[0052] Figure 3 This is a schematic diagram of the training preset clothing re-identification model disclosed in the embodiments of this application;

[0053] Figure 4 This is a schematic diagram of the modules of the machine vision-based kitchen supervision system disclosed in the embodiments of this application;

[0054] Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0055] Explanation of reference numerals in the attached diagram: 401, Video acquisition module; 402, Clothing recognition module; 403, Mask recognition module; 404, Hat recognition module; 405, Alarm response module; 501, Processor; 502, Communication bus; 503, User interface; 504, Network interface; 505, Memory. Detailed Implementation

[0056] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0057] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0058] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0059] This embodiment discloses a machine vision-based kitchen supervision method, applied to a kitchen supervision platform. Figure 1 This is a flowchart illustrating the machine vision-based kitchen supervision method disclosed in an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps:

[0060] S101. Real-time acquisition of video stream from target camera, detection of human body from video stream and cropping human body image of human body as center, and performing face detection and head detection on human body image.

[0061] S102. Input the human body image into a preset clothing re-identification model, extract the first feature vector of the chef's uniform, and compare the first feature vector with the feature vector in the first template library to obtain the first similarity. When the first similarity corresponding to a preset number of consecutive frames is less than the first similarity threshold, an alarm is triggered.

[0062] S103. Locate the face frame based on the face detection and input the face frame into the preset mask classification model to detect whether it is in compliance. When a preset number of consecutive frames fail to meet the standard, an alarm is triggered.

[0063] S104. Locate the head frame based on the head detection and input the head frame into the preset hat re-identification model. Extract the second feature vector of the chef's hat and compare the second feature vector with the feature vector in the second template library to obtain the second similarity. When the second similarity corresponding to a preset number of consecutive frames is less than the second similarity threshold, trigger an alarm.

[0064] S105. When an alarm is triggered, save a screenshot and send the detection result, alarm reason and timestamp information to the target object.

[0065] Real-time acquisition of dynamic information about kitchen staff provides foundational data for subsequent uniform compliance checks. Video streams are captured via target cameras (e.g., cameras installed in the kitchen). Human detection algorithms (such as YOLO) are used to detect human bodies from the video stream, and small images of each detected body are cropped, each assigned a unique ID. Then, face and head detection are performed on the cropped images to further obtain facial and head information. The cropped images are input into a pre-defined clothing re-identification model to extract the first feature vector of the chef's uniform. The extracted first feature vector is then compared with feature vectors in a first template library to calculate the first similarity. If the first similarity for a consecutive preset number of frames is less than the first similarity threshold, an alarm is triggered. Based on the face detection results, face bounding boxes are located and input into a pre-defined mask classification model. The mask classification model determines whether the mask is worn correctly or improperly. If the mask is not worn correctly for a consecutive preset number of frames, an alarm is triggered. Based on the head detection results, the head bounding box is located and input into a preset hat re-identification model. The hat re-identification model extracts the second feature vector of the chef's hat and compares it with the feature vectors in the second template library to calculate the second similarity. If the second similarity of a consecutive preset number of frames is less than the second similarity threshold, an alarm is triggered. When an alarm is triggered, the system automatically saves a screenshot of the alarm moment and sends the detection results, alarm reason, and timestamp information to the target object (such as kitchen managers or supervisors).

[0066] By capturing video streams in real time from target cameras and detecting human bodies, faces, and heads within the streams, the system can obtain dynamic information about kitchen staff in real time, promptly identify potential problems, and ensure the real-time and continuous nature of supervision. Using pre-set clothing re-identification, mask classification, and hat re-identification models, feature vectors for human body thumbnails, face bounding boxes, and head bounding boxes are quickly extracted and compared with feature vectors in a template library. This allows for rapid judgment of whether kitchen staff attire conforms to regulations, enabling quick response. By using the clothing re-identification, mask classification, and hat re-identification models to extract and compare different features, the system can evaluate kitchen staff attire from multiple perspectives, improving the accuracy and comprehensiveness of identification. During the training of the clothing and hat re-identification models, an angle metric loss function is used, enabling the models to learn more discriminative feature vectors, further improving the model's recognition accuracy. An alarm is triggered when the similarity of a consecutive preset number of frames is less than a similarity threshold. This continuous frame judgment method effectively avoids false alarms caused by single-frame misjudgments, improving the reliability of the alarm. The system monitors and assesses the wearing of clothing, masks, and hats separately. An alarm is triggered if any item fails to meet standards, ensuring comprehensive supervision of kitchen staff attire and enabling timely detection and correction of non-compliant behavior. When an alarm is triggered, the system saves a screenshot and sends the detection results, alarm reason, and timestamp information to the target, providing detailed records and evidence for subsequent investigations and management. This facilitates tracing and analyzing the problem and helps in implementing targeted improvement measures.

[0067] Figure 2 This is another schematic diagram of the machine vision-based kitchen supervision method disclosed in the embodiments of this application, such as... Figure 2 As shown, the machine vision-based kitchen supervision method includes steps 201, video frame extraction; 202, human detection and tracking to extract human body thumbnails and tracking IDs; 203, extraction of chef uniform feature vectors; 204, outputting the cosine similarity between the vector and all clothing template vectors; 205, determining whether the similarity values ​​of three consecutive frames with the same ID are all greater than the threshold of 0.7; 206, if not, issuing an alarm and recording the information; 207, face detection (including masks) to obtain face thumbnails; 208. The process involves several steps: 209. Categorizing whether masks are worn; 210. Determining if three consecutive frames with the same ID are all of people not wearing masks or wearing masks improperly; 211. If so, triggering an alarm and recording the information; 212. Detecting heads (including hats) to obtain small head images; 213. Extracting the chef's hat feature vector; 214. Outputting the cosine similarity between the output vector and all hat template vectors; 215. Determining if the similarity value of three consecutive frames with the same ID is greater than a threshold of 0.7; 216. If not, triggering an alarm and recording the information. Steps 203, 207, and 211 can be executed simultaneously.

[0068] Optionally, the method further includes training a preset clothing re-identification model, specifically including:

[0069] The deep neural network architecture is pre-trained using a first human body dataset, enabling the deep neural network architecture to learn the basic features of human body weight recognition.

[0070] The convolutional layer, the first residual layer, the second residual layer, and the third residual layer of the deep neural network architecture are locked. The fourth residual layer, the pooling layer, and the fully connected layer of the deep neural network architecture are adjusted. The deep neural network architecture is trained using a second human body dataset to obtain an intermediate clothing re-identification model. The second human body dataset includes clothing data of chefs under different lighting, angles, postures, and occlusions.

[0071] The intermediate clothing re-identification model is trained using angular metric loss to obtain the preset clothing re-identification model.

[0072] Choose a large-scale, diverse first human body dataset that covers various scenes and personnel changes, containing rich visual features. For example, the Market-1501 dataset is a commonly used human body recognition dataset containing a large number of human images in different scenes, and can be used as a first human body dataset. Select ResNet (Residual Network) as the deep neural network architecture. ResNet is a commonly used deep learning architecture that can effectively alleviate the gradient vanishing problem in deep neural networks, improving the training efficiency and performance of the network. In the pre-training stage, use the ResNet architecture to train on the first human body dataset, enabling the model to learn the basic features of human body recognition, such as human contours, poses, and local textures. Through pre-training on the first human body dataset, the ResNet model can learn the basic features of human body recognition in general scenes. For example, the model can recognize the approximate outline of the human body, limb positions, facial contours, and other features, laying the foundation for subsequent clothing re-recognition tasks. Figure 3This is a schematic diagram of the training of the pre-defined clothing re-identification model disclosed in this application embodiment. After pre-training, the first few layers of the ResNet architecture are locked, including convolutional layers, the first residual layer (residual layer 0), the second residual layer (residual layer 1), and the third residual layer (residual layer 2). These layers learn some general features, such as edges and textures, which have strong versatility and do not need to be adjusted in subsequent fine-tuning. The fourth residual layer (residual layer 3), pooling layer, and fully connected layer of the ResNet architecture are adjusted. The fourth residual layer can further extract higher-level features, such as local texture and color features of the human body. The pooling layer is used to reduce the dimensionality of features while retaining important feature information. The fully connected layer is used to map features to specific categories, such as different styles of chef's clothing. Clothing data of chefs under different lighting, angles, poses, and occlusions are collected as a second human body dataset. For example, images of chefs wearing different styles of chef's uniforms are taken in a kitchen environment, including images from different angles such as the front, side, and back, as well as images under different lighting conditions. The adjusted ResNet architecture is trained using a second human body dataset to obtain an intermediate clothing re-identification model. In this stage, the model learns specific features of chefs' clothing, such as color, texture, and style. During training, the model requires a large number of labeled chef clothing images to learn the correct clothing features. Angular metric loss is a commonly used loss function to optimize feature learning in deep learning models. Training the intermediate clothing re-identification model using angular metric loss allows the model to generate more discriminative feature vectors. Angular metric loss makes the feature angles of intra-class samples smaller, enhancing intra-class cohesion, while making the feature angles of inter-class samples larger, improving inter-class separation. This allows the model to better identify differences between different chefs' clothing, improving the accuracy of clothing re-identification.

[0073] When pre-trained using the first human body dataset, the model learns the basic features for human body weight recognition. These basic features have a certain degree of universality and can adapt to human body weight recognition tasks in different scenarios and data distributions. When the fourth residual layer, pooling layer, and fully connected layer of the deep neural network architecture are adjusted using the second human body dataset, the model can further learn specific features of chef's clothing under complex conditions such as different lighting, angles, poses, and occlusions. This gives the model stronger generalization ability, enabling it to better adapt to various complex kitchen environments and accurately recognize chef's clothing in different scenarios in practical applications. Focusing on the convolutional layer, first residual layer, second residual layer, and third residual layer of the deep neural network architecture, these layers are typically responsible for extracting low-level features from images, such as edges, textures, and colors. Adjusting and training the fourth residual layer, pooling layer, and fully connected layer allows the model to focus more on extracting higher-level, more abstract features, such as clothing style, color combinations, and detailed textures, further improving the model's ability to accurately extract features from chef's clothing, thereby improving the accuracy of clothing re-recognition. The use of angular metric loss and the training of the intermediate clothing re-identification model enable the generated feature vectors to have clearer class discrimination; that is, feature vectors of the same chef's clothing are more clustered in the feature space, while feature vectors of different chefs' clothing are more widely separated. Through this training method, the pre-set clothing re-identification model can more accurately distinguish and identify chefs' clothing, thereby reducing the false recognition rate and improving the accuracy of clothing re-identification. Locking some layers and adjusting and training others reduces the number of network parameters and training time.

[0074] Optionally, training the intermediate clothing re-identification model based on the angular metric loss to obtain the preset clothing re-identification model includes:

[0075] The angle measurement loss is calculated using the following formula:

[0076]

[0077] Where L represents the angle metric loss, y is the correct class label, and θ y θ is the angle between the input sample and the weight vector of the correct class, j is the label of the incorrect class, and θ is the angle between the input sample and the weight vector of the correct class. j is the angle between the input sample and the weight vector of the error class, s is the scale factor (constant), and m is the angular interval (constant).

[0078] Optionally, the method further includes training a preset mask classification model, specifically including:

[0079] Collect a third-party human body dataset, which includes images of multiple chefs in different scenarios, including those not wearing masks, wearing masks improperly, and wearing masks properly. The different scenarios include multiple angles, different lighting conditions, and occlusion situations.

[0080] The third human body dataset is first annotated, the first annotation including the coordinates of the face region, and a face detection model is trained based on the third human body dataset after the first annotation;

[0081] The third human body dataset is labeled a second time, including a category label for mask wearing status. A mask classification model is trained based on the third human body dataset after the second labeling.

[0082] The face detection model and the mask classification model are fused together to obtain the preset mask classification model.

[0083] A third-party human dataset was collected, containing images of multiple chefs in various scenarios, including those without masks, those not wearing masks correctly (e.g., masks not fully covering the mouth and nose, masks worn crookedly), and those wearing masks correctly. The dataset was ensured to cover multiple angles (e.g., front, side, oblique side), different lighting conditions (e.g., strong light, weak light, backlight), and occlusion situations (e.g., partially obscured by kitchen utensils or food) to enhance the model's generalization ability and robustness, enabling it to adapt to various complex kitchen environments. First annotation (face region coordinates): The third-party human dataset was first annotated with the coordinates of the face regions. This step was to train the face detection model, enabling it to accurately locate face regions in images. Professional image annotation software, such as LabelImg, could be used. The annotation results are typically saved as XML or JSON files, containing information such as the coordinates of the top-left corner (x1, y1) and bottom-right corner (x2, y2) of the face region. Second annotation (mask wearing status category label): The third human body dataset is annotated a second time, with the annotation content being the category label for mask wearing status. Based on mask wearing status, the category label is divided into three classes: 0 indicates no mask, 1 indicates improper mask wearing, and 2 indicates correct mask wearing. This step is to train the mask classification model, enabling it to classify different mask wearing statuses. The annotation results are also saved as a CSV or JSON file, with each row or object corresponding to an image, including the image path and mask wearing status category label information. The face detection model is trained using the third human body dataset with the first annotation. The model can use mainstream face detection algorithms, such as the YOLO series (e.g., YOLOv5, YOLOv7, etc.). During training, the model learns the features of face regions and can accurately locate the face position in the image. Appropriate hyperparameters need to be set during training, such as learning rate, batch size, and number of training epochs, and the model's performance is evaluated using a validation set. The training strategy is adjusted based on the evaluation results. The mask classification model is then trained using the third human body dataset with the second annotation. The model can employ lightweight deep neural networks, such as MobileNet, to meet the needs of mask classification tasks. During training, the model learns the features of different mask-wearing conditions, enabling it to classify input face images. Appropriate hyperparameters need to be set, and model performance should be evaluated using a validation set to adjust the training strategy accordingly. The trained face detection model and mask classification model are then fused to construct a pre-defined mask classification model. Specifically, the face detection model first detects face regions in the image, generating face bounding boxes; then, these face bounding boxes are used as input to the mask classification model for classification, yielding the classification result based on mask wearing condition. In this way, the fused model can simultaneously perform face detection and mask classification, improving the efficiency and accuracy of detection and classification.During the fusion process, the parameters of the face detection model and the mask classification model can be jointly optimized. Through end-to-end training, the two models can work together better, further improving the overall performance of the model. Specifically, a multi-task learning approach can be adopted, setting corresponding loss functions for each task. For example, the loss function for face detection can be bounding box regression loss and classification loss, while the loss function for mask classification can be cross-entropy loss. Multiple loss functions are then combined into a single total loss function using methods such as weighted summation to guide model training.

[0084] By collecting image datasets containing images of chefs wearing masks in different scenarios—without masks, improperly worn masks, and properly worn masks—and these scenarios encompassing multiple angles, varying lighting conditions, and occlusion, the model can learn the characteristics of mask-wearing in diverse and complex environments. This helps improve the model's generalization ability, enabling it to more accurately identify mask-wearing in different scenarios in practical applications. The dataset is first annotated, including the coordinates of face regions, and a face detection model is trained based on this. The face detection model accurately locates face regions, providing accurate input for the mask classification model. This helps improve the accuracy of the mask classification model because accurate face detection ensures that the mask classification model focuses only on face regions, avoiding interference from other areas. The dataset is then second-annotated, including category labels for mask-wearing, and a mask classification model is trained based on this. The mask classification model can accurately identify mask-wearing, including categories such as not wearing a mask, improperly wearing a mask, and properly wearing a mask. This helps improve the model's classification performance, enabling it to more accurately determine whether chefs are wearing masks correctly. The face detection model and the mask classification model are then fused to obtain a preset mask classification model. The fused model leverages the strengths of both face detection and mask classification to improve overall performance. The face detection model provides accurate face regions, while the mask classification model performs precise classification based on these regions. Working together, the two models can more effectively identify whether masks are being worn.

[0085] Optionally, the method further includes training a preset hat re-identification model, specifically including:

[0086] Collect a multimodal dataset of chef hats, which includes brim shape data from top / bottom viewpoints, optical property data of reflective material hat surface, and hat badge feature data under dynamic blur conditions;

[0087] A dual-branch network structure is constructed. The first branch extracts features from the complete chef hat images in the chef hat multimodal dataset through the backbone network to capture global information, including the overall shape, color and texture of the hat. The second branch extracts local texture features under occlusion.

[0088] A contrastive learning method is used to compare the features extracted by the first branch and the second branch. When the comparison result meets the requirements, the preset hat re-identification model is constructed based on the dual-branch network structure.

[0089] Collect image data of chef's hats under different viewing angles (top and bottom), lighting conditions, materials (reflective materials), and dynamic blur conditions. This data should cover multimodal information such as brim shape, optical properties of the hat surface, and hat badge features. Annotate the collected images, including detailed information on brim shape, optical properties of the hat surface, and hat badge features. The annotated data will be used for model training and validation. The first branch (global feature extraction) extracts features from the complete chef's hat image using a backbone network (such as ResNet-50), capturing global information such as the hat's overall shape, color, and texture. This branch aims to extract the overall features of the hat for recognition under different viewing angles and lighting conditions. The second branch (local feature extraction) extracts local texture features under occlusion. This branch focuses on extracting key features of the hat, such as detailed features of the hat badge, under partial occlusion or dynamic blur conditions. Through local feature extraction, the model can better handle recognition tasks under occlusion and blur conditions. Compare the features extracted by the first and second branches. Through comparative learning, the model can learn the correlation between global and local features, thereby improving the accuracy and robustness of recognition. Contrastive loss functions (such as InfoNCE loss) are used to measure the similarity between features, and the model's discriminative ability is enhanced by optimizing feature representations. Through contrastive learning, the model can learn stable representations of hat features under different conditions, thereby improving the model's generalization ability and adaptability. The two-branch network structures trained through contrastive learning are fused to construct a predefined hat re-identification model. The fused model can utilize both global and local features simultaneously, improving recognition accuracy and robustness. During training, the model's performance is further optimized by adjusting hyperparameters such as the learning rate and optimizer (such as Adam or SGD). Simultaneously, data augmentation techniques (such as random flipping, rotation, and cropping) can be used to increase dataset diversity and improve the model's generalization ability.

[0090] By collecting a multimodal dataset of chef's hats, including brim shape data from top / bottom views, optical property data of reflective material caps, and cap badge feature data under dynamic blur conditions, the model can learn the characteristics of hats under various complex conditions. This helps improve the model's generalization ability, enabling it to more accurately identify chef's hats in different scenarios in practical applications. A dual-branch network structure is constructed. The first branch extracts features from the complete chef's hat image through the backbone network, capturing global information such as the overall shape, color, and texture of the hat; the second branch extracts local texture features under occlusion. This dual-branch structure can simultaneously capture global and local features, improving the model's ability to extract hat features and thus improving the accuracy of hat re-identification. A contrastive learning method is used to compare the features extracted by the first and second branches. By maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs, the model can learn more discriminative feature representations. When the comparison results meet the requirements, a preset hat re-identification model is built based on the dual-branch network structure, which helps to further optimize model performance and improve the accuracy of hat re-identification.

[0091] Optionally, the method further includes:

[0092] When no alarm is triggered for a consecutive preset time period, new human body images are collected to construct an incremental training set, which includes human body images that have not triggered an alarm within the consecutive preset time period.

[0093] The output of the original model is used as a soft label through knowledge distillation technology. The parameters of the fully connected layer of the original model are adjusted by combining the incremental training set. The original model includes the preset clothing re-identification model, the preset mask classification model, and the preset hat re-identification model.

[0094] The cosine similarity drift detection algorithm is used to monitor the feature vector distribution of the adjusted original model. When the shift in the feature vector distribution exceeds a preset threshold, the adaptive adjustment of the model structure is triggered, and the feature embedding dimension is dynamically expanded.

[0095] When no alarm is triggered for a consecutive preset time period, new human body images are collected to construct an incremental training set. These newly collected human body images contain samples that did not trigger alarms within that time period, providing the model with the latest data and helping it learn the latest features and patterns. This helps enhance the model's adaptability to new data, enabling it to better cope with possible changes and new situations in the kitchen environment. Through knowledge distillation technology, the output of the original model is used as a soft label, combined with the incremental training set to adjust the parameters of the fully connected layers of the original model. Knowledge distillation technology can transfer knowledge from complex models to lightweight models, significantly reducing the demand for computing resources while maintaining performance. In this embodiment, the application of knowledge distillation technology helps optimize model parameters and improve the model's training efficiency and practical application effects. Specifically, the student model can learn the predictive ability of the teacher model, thereby improving the model's running efficiency while maintaining high accuracy. A cosine similarity drift detection algorithm is used to monitor the feature vector distribution of the adjusted original model. Cosine similarity evaluates the similarity between two vectors by calculating the cosine of the angle between them and is widely used in deep learning to measure the similarity between high-dimensional vectors. When the shift in feature vector distribution exceeds a preset threshold, an adaptive adjustment of the model structure is triggered. This helps to promptly detect changes in model performance and ensures that the model maintains good performance across different time periods. The adaptive adjustment dynamically expands the feature embedding dimension, enabling the model to better adapt to changes in data distribution and improve its generalization ability. By dynamically expanding the feature embedding dimension, the model can learn richer feature representations, thus maintaining high recognition accuracy even in complex and ever-changing kitchen environments.

[0096] Optionally, detecting a human body from the video stream and cropping a small image of the human body centered on the human body includes:

[0097] Thermodynamic features of each pixel in the video stream are extracted. The thermodynamic features include the rate of temperature change and the thermal diffusivity. A segmentation model of moving targets and background is constructed based on the thermodynamic features to distinguish human targets from steam interference areas.

[0098] By calculating the parallax of multiple cameras, depth information of the kitchen scene is obtained, a three-dimensional spatial model of the kitchen scene is constructed based on the depth information, and the three-dimensional spatial model is used to eliminate the detection blind spots caused by occlusion.

[0099] After eliminating areas of steam interference and detection blind spots, a small image of the human body is captured.

[0100] By calculating the temperature change rate of each pixel in the video stream, the temperature difference between the human body and the background can be captured. The human body typically has a relatively stable temperature, while the background (such as steam) experiences greater temperature variations. Extracting the temperature change rate effectively distinguishes human targets from steam interference areas. Thermal diffusivity reflects the rate at which heat diffuses within an object. The thermal diffusivity of the human body is relatively low, while that of steam is high. Extracting thermal diffusivity further enhances the distinction between the human body and the background. Using the extracted thermodynamic features, a segmentation model is constructed to segment moving targets (human bodies) and the background (steam) in the video stream. This model can be based on deep learning algorithms, such as convolutional neural networks (CNNs), which learn the relationship between thermodynamic features and moving targets through training, thus achieving accurate segmentation. Depth information of the kitchen scene is obtained through multi-camera disparity calculation. Disparity calculation calculates depth information by comparing pixel differences between images of the same scene captured by different cameras. Specifically, stereo vision algorithms, such as block matching algorithms or feature-based matching algorithms, can be used to calculate disparity maps and obtain depth information. Based on the acquired depth information, a 3D spatial model of the kitchen scene is constructed. This model can be represented as point cloud data or 3D mesh data, containing the 3D coordinates of each pixel. The 3D spatial model accurately represents the position and spatial relationships of objects in a kitchen scene. Within the 3D spatial model, the relative positions and occlusion relationships between objects can be determined. By analyzing the 3D spatial model, occluded areas can be identified, and the parameters of the detection algorithm can be adjusted to eliminate detection blind spots caused by occlusion. For example, the depth information in the 3D spatial model can be used to adjust the position and size of the detection window to ensure that the detection algorithm can cover the occluded areas. Using the segmentation model constructed above, the steam interference area in the video stream is segmented from the human target. The segmentation result can be represented as a binary mask, where the human target area is the foreground (value 1), and the steam interference area is the background (value 0). By applying this mask, the steam interference area can be removed from the video stream. Based on the depth information in the 3D spatial model, the parameters of the detection algorithm are adjusted to ensure that the detection algorithm can cover the occluded areas. For example, the position and size of the detection window can be adjusted to cover the occluded human body portion. In this way, detection blind spots can be eliminated, and detection accuracy can be improved. After eliminating steam interference areas and detection blind spots, a small image of the human body is captured, centered on the detected human. The size of the small image can be adjusted according to actual needs to ensure sufficient human feature information is included. The captured small image will be used for subsequent feature extraction and comparison to enable supervision and management of kitchen staff.

[0101] By extracting the thermodynamic features of each pixel in the video stream, including the rate of temperature change and thermal diffusivity, it is possible to effectively distinguish human targets from steam interference areas. This method utilizes the differences in thermodynamic properties between the human body and steam to construct a segmentation model of moving targets and background, improving the accuracy of human detection. In complex kitchen environments, heat sources such as steam may interfere with detection, but the segmentation method based on thermodynamic features can effectively reduce this interference, ensuring accurate detection of human targets. Depth information of the kitchen scene is obtained through multi-camera parallax calculation, and a 3D spatial model of the kitchen scene is constructed. This 3D spatial model provides more comprehensive scene information, helping to eliminate detection blind spots caused by occlusion. In practical applications, objects such as wall cabinets in the kitchen may obscure parts of the human body, leading to detection blind spots. Using the 3D spatial model, the obscured parts of the human body can be accurately identified, thereby improving the completeness and accuracy of detection. After eliminating steam interference areas and detection blind spots, cropping a small image of the human body ensures that the cropped image is more accurate and complete. This is crucial for subsequent feature extraction and analysis, as an accurate small image of the human body provides more reliable feature information, thereby improving the performance of the entire kitchen supervision system. By combining techniques such as thermodynamic feature extraction, moving target segmentation, multi-camera parallax calculation, and 3D spatial model construction, the accuracy and robustness of human detection are significantly improved. In complex kitchen environments, these techniques effectively address various interference factors, ensuring the reliability of human detection. Through these techniques, this method can adapt to different kitchen environments and scene changes, enhancing the system's practicality and adaptability. Whether in steamy cooking areas or complex scenes with obstructions, this method can accurately detect human bodies, providing strong technical support for kitchen supervision.

[0102] This embodiment also discloses a machine vision-based kitchen supervision system. Figure 4 This is a schematic diagram of the modules of the machine vision-based kitchen supervision system disclosed in the embodiments of this application, as shown below. Figure 4 As shown, the system includes a video acquisition module 401, a clothing recognition module 402, a mask recognition module 403, a hat recognition module 404, and an alarm response module 405, wherein:

[0103] The video acquisition module 401 is configured to acquire video streams from a target camera in real time, detect human bodies from the video stream and capture a small human body image centered on the human body, and perform face detection and head detection on the small human body image.

[0104] Clothing recognition module 402 is configured to input the small human body image into a preset clothing re-recognition model, extract the first feature vector of the chef's uniform, and compare the first feature vector with the feature vector in the first template library to obtain the first similarity. When the first similarity corresponding to a preset number of consecutive frames is less than the first similarity threshold, an alarm is triggered.

[0105] The mask recognition module 403 is configured to locate the face frame based on the face detection and input the face frame into a preset mask classification model to detect whether it is in compliance with the standard. When a preset number of consecutive frames fail to meet the standard, an alarm is triggered.

[0106] The hat recognition module 404 is configured to locate the head frame based on the head detection and input the head frame into a preset hat re-recognition model, extract the second feature vector of the chef's hat and compare the second feature vector with the feature vector in the second template library to obtain the second similarity, and trigger an alarm when the second similarity corresponding to a consecutive preset number of frames is less than the second similarity threshold.

[0107] The alarm response module 405 is configured to save a screenshot and send the detection result, alarm reason and timestamp information to the target object when an alarm is triggered.

[0108] Optionally, the system further includes a first training module configured for:

[0109] The deep neural network architecture is pre-trained using a first human body dataset, enabling the deep neural network architecture to learn the basic features of human body weight recognition.

[0110] The convolutional layer, the first residual layer, the second residual layer, and the third residual layer of the deep neural network architecture are locked. The fourth residual layer, the pooling layer, and the fully connected layer of the deep neural network architecture are adjusted. The deep neural network architecture is trained using a second human body dataset to obtain an intermediate clothing re-identification model. The second human body dataset includes clothing data of chefs under different lighting, angles, postures, and occlusions.

[0111] The intermediate clothing re-identification model is trained using angular metric loss to obtain the preset clothing re-identification model.

[0112] Optionally, the first training module is configured to:

[0113] The angle measurement loss is calculated using the following formula:

[0114]

[0115] Where L represents the angle metric loss, y is the correct class label, and θ yθ is the angle between the input sample and the weight vector of the correct class, j is the label of the incorrect class, and θ is the angle between the input sample and the weight vector of the correct class. j is the angle between the input sample and the weight vector of the error class, s is the scale factor (constant), and m is the angular interval (constant).

[0116] Optionally, the system further includes a second training module configured for:

[0117] Collect a third-party human body dataset, which includes images of multiple chefs in different scenarios, including those not wearing masks, wearing masks improperly, and wearing masks properly. The different scenarios include multiple angles, different lighting conditions, and occlusion situations.

[0118] The third human body dataset is first annotated, the first annotation including the coordinates of the face region, and a face detection model is trained based on the third human body dataset after the first annotation;

[0119] The third human body dataset is labeled a second time, including a category label for mask wearing status. A mask classification model is trained based on the third human body dataset after the second labeling.

[0120] The face detection model and the mask classification model are fused together to obtain the preset mask classification model.

[0121] Optionally, the system further includes a third training module configured for:

[0122] Collect a multimodal dataset of chef hats, which includes brim shape data from top / bottom viewpoints, optical property data of reflective material hat surface, and hat badge feature data under dynamic blur conditions;

[0123] A dual-branch network structure is constructed. The first branch extracts features from the complete chef hat images in the chef hat multimodal dataset through the backbone network to capture global information, including the overall shape, color and texture of the hat. The second branch extracts local texture features under occlusion.

[0124] A contrastive learning method is used to compare the features extracted by the first branch and the second branch. When the comparison result meets the requirements, the preset hat re-identification model is constructed based on the dual-branch network structure.

[0125] Optionally, the system further includes an update module configured for:

[0126] When no alarm is triggered for a consecutive preset time period, new human body images are collected to construct an incremental training set, which includes human body images that have not triggered an alarm within the consecutive preset time period.

[0127] The output of the original model is used as a soft label through knowledge distillation technology. The parameters of the fully connected layer of the original model are adjusted by combining the incremental training set. The original model includes the preset clothing re-identification model, the preset mask classification model, and the preset hat re-identification model.

[0128] The cosine similarity drift detection algorithm is used to monitor the feature vector distribution of the adjusted original model. When the shift in the feature vector distribution exceeds a preset threshold, the adaptive adjustment of the model structure is triggered, and the feature embedding dimension is dynamically expanded.

[0129] Optionally, the video acquisition module 401:

[0130] Thermodynamic features of each pixel in the video stream are extracted. The thermodynamic features include the rate of temperature change and the thermal diffusivity. A segmentation model of moving targets and background is constructed based on the thermodynamic features to distinguish human targets from steam interference areas.

[0131] By calculating the parallax of multiple cameras, depth information of the kitchen scene is obtained, a three-dimensional spatial model of the kitchen scene is constructed based on the depth information, and the three-dimensional spatial model is used to eliminate the detection blind spots caused by occlusion.

[0132] After eliminating areas of steam interference and detection blind spots, a small image of the human body is captured.

[0133] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0134] This embodiment also discloses an electronic device, referring to... Figure 5 The electronic device may include: at least one processor 501, at least one communication bus 502, user interface 503, network interface 504, and at least one memory 505.

[0135] The communication bus 502 is used to enable communication between these components.

[0136] The user interface 503 may include a display screen and a camera. Optionally, the user interface 503 may also include a standard wired interface and a wireless interface.

[0137] The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0138] The processor 501 may include one or more processing cores. The processor 501 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 505, and by calling data stored in memory 505. Optionally, the processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 501 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 501 and may be implemented as a separate chip.

[0139] The memory 505 may include random access memory (RAM) or read-only memory. Optionally, the memory 505 may include a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 505 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 505 may also be at least one storage device located remotely from the aforementioned processor 501. Figure 5 As shown, the memory 505, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a machine vision-based kitchen supervision method.

[0140] exist Figure 5 In the electronic device shown, the user interface 503 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 501 can be used to call the application program stored in the memory 505 for the kitchen supervision method based on machine vision. When executed by one or more processors 501, the electronic device performs one or more methods as described in the above embodiments.

[0141] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0142] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0144] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0146] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 505 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory 505 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0147] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will be readily apparent to those skilled in the art upon consideration of the disclosure in this specification. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A machine vision-based kitchen supervision method, characterized by, The method is applied to a kitchen monitoring platform, and comprises the following steps: Real-time video stream is collected from a target camera, a human body is detected from the video stream, and a human body small picture of the human body is intercepted with the human body as the center, and face detection and head detection are performed on the human body small picture; The human body small picture is input into a preset clothing re-identification model, a first feature vector of a chef's uniform is extracted, and the first feature vector is compared with feature vectors in a first template library to obtain a first similarity, and an alarm is triggered when the first similarity corresponding to a continuous preset number of frames is all less than a first similarity threshold; A face frame is positioned according to the face detection, and the face frame is input into a preset mask classification model to detect whether a standard is met, and an alarm is triggered when a continuous preset number of frames do not meet the standard; A head frame is positioned according to the head detection, and the head frame is input into a preset hat re-identification model, a second feature vector of a chef's hat is extracted, and the second feature vector is compared with feature vectors in a second template library to obtain a second similarity, and an alarm is triggered when the second similarity corresponding to a continuous preset number of frames is all less than a second similarity threshold; When the alarm is triggered, a screenshot is saved, and detection results, alarm reasons and timestamp information are sent to a target object, The human body is detected from the video stream, and the human body small picture of the human body is intercepted with the human body as the center, and comprises the following steps: Thermodynamic features of each pixel point in the video stream are extracted, the thermodynamic features include temperature change rate and thermal diffusivity, a segmentation model of a moving target and a background is constructed according to the thermodynamic features, and a human body target and a steam interference area are distinguished; Depth information of a kitchen scene is obtained through multi-camera parallax calculation, a three-dimensional space model of the kitchen scene is constructed according to the depth information, and the three-dimensional space model is used to eliminate detection blind areas caused by occlusion; The human body small picture of the human body is intercepted after the steam interference area and the detection blind area are eliminated, The method further comprises training a preset hat re-identification model, specifically comprising: A chef's hat multi-modal data set is collected, the chef's hat multi-modal data set includes hat brim shape data under a top view / perspective view, optical characteristic data of a reflective material hat surface, and hat badge feature data under a dynamic blur condition; A double-branch network structure is constructed, a first branch extracts features of complete chef's hat images in the chef's hat multi-modal data set through a backbone network to capture global information, the global information includes overall shape, color and texture of the hat, and a second branch extracts local texture features under an occlusion state; A comparison learning method is used to compare the features extracted by the first branch and the second branch, and the preset hat re-identification model is constructed based on the double-branch network structure when the comparison result meets the requirements. 2.The machine vision-based kitchen supervision method according to claim 1, wherein, The method further comprises training a preset clothing re-identification model, specifically comprising: A first human body data set is used to pre-train a deep neural network architecture, so that the deep neural network architecture learns basic features of human body re-identification; locking the convolutional layer, the first residual layer, the second residual layer and the third residual layer of the deep neural network architecture, adjusting the fourth residual layer, the pooling layer and the fully connected layer of the deep neural network architecture, training the deep neural network architecture using a second human body dataset to obtain an intermediate clothing recognition model, the second human body dataset including clothing data of a chef under different light, angle, posture and occlusion; training the intermediate clothing recognition model according to the angle measurement loss to obtain the preset clothing recognition model.

3. The machine vision-based kitchen supervision method according to claim 2, wherein, The training of the intermediate clothing recognition model according to the angle measurement loss to obtain the preset clothing recognition model comprises: The angle measurement loss is calculated by the following formula: ； wherein, L denotes the angle metric loss, y is the correct class label, is the angle between the input sample and the weight vector of the correct class, j is the incorrect class label, is the angle between the input sample and the weight vector of the incorrect class, s is a scale factor, is a constant, and m is an angle interval, is a constant.

4. The machine vision-based kitchen supervision method according to claim 1, wherein, The method further comprises training a preset mask classification model, specifically comprising: collecting a third human body dataset, the third human body dataset including images of a plurality of chefs without wearing masks, wearing non-standard masks and wearing standard masks under different scenes, the different scenes including different angles, different light conditions and occlusion conditions; performing first labeling on the third human body dataset, the first labeling including coordinates of a face region, and training a face detection model according to the third human body dataset after the first labeling; performing second labeling on the third human body dataset, the second labeling including class labels of mask wearing conditions, and training a mask classification model according to the third human body dataset after the second labeling; fusing the face detection model and the mask classification model to obtain the preset mask classification model.

5. The machine vision-based kitchen supervision method according to claim 1, wherein, The method further comprises: when no alarm is triggered for a continuous preset time period, collecting new human small images to construct an incremental training set, the incremental training set including human small images that do not trigger alarms within the continuous preset time period; adjusting the fully connected layer parameters of the original model by using the output of the original model as a soft label through a knowledge distillation technology, in combination with the incremental training set, the original model including the preset clothing recognition model, the preset mask classification model and the preset hat recognition model; using a cosine similarity drift detection algorithm to monitor the feature vector distribution of the adjusted original model, and triggering adaptive adjustment of the model structure when the drift of the feature vector distribution exceeds a preset threshold, dynamically expanding the feature embedding dimension.

6. A machine vision-based back-of-house supervision system, characterized by, The system comprises a video acquisition module, a clothing recognition module, a mask recognition module, a hat recognition module and an alarm response module, wherein: The video acquisition module is configured to acquire a video stream from a target camera in real time, detect a human body from the video stream and intercept a human small image of the human body with the human body as the center, and perform face detection and head detection on the human small image. The clothing recognition module is configured to input the human small image into a preset clothing recognition model, extract a first feature vector of a chef's uniform, and compare the first feature vector with feature vectors in a first template library to obtain a first similarity, and trigger an alarm when the first similarity corresponding to a continuous preset number of frames is less than a first similarity threshold. The mask recognition module is configured to position a face frame according to the face detection and input the face frame into a preset mask classification model to detect whether a mask is standardized, and trigger an alarm when a continuous preset number of frames does not reach the standard. The hat recognition module is configured to position a head frame according to the head detection, input the head frame into a preset hat re-identification model, extract a second feature vector of a chef's hat, and compare the second feature vector with feature vectors in a second template library to obtain a second similarity, and trigger an alarm when the second similarity corresponding to a continuous preset number of frames is less than a second similarity threshold. The alarm response module is configured to save a screenshot and send detection results, alarm reasons and timestamp information to a target object when an alarm is triggered. The method further includes: extracting thermodynamic features of each pixel point in the video stream, the thermodynamic features including temperature change rate and thermal diffusivity, constructing a segmentation model of a moving target and a background according to the thermodynamic features, and distinguishing a human body target from a steam interference area; obtaining depth information of a kitchen scene through multi-camera parallax calculation, constructing a three-dimensional space model of the kitchen scene according to the depth information, and eliminating a detection blind area caused by occlusion by using the three-dimensional space model; taking a human body small picture of the human body after eliminating the steam interference area and the detection blind area, The system further includes a third training module configured to: collect a chef's hat multi-modal data set, the chef's hat multi-modal data set including hat brim shape data under a top view / down view perspective, optical property data of a reflective material hat surface, and hat badge feature data under a dynamic blur condition; construct a double-branch network structure, a first branch extracts features of complete chef's hat images in the chef's hat multi-modal data set through a backbone network to capture global information, the global information including overall shape, color and texture of the hat, and a second branch extracts local texture features under an occlusion state; use a contrast learning method to compare the features extracted by the first branch and the second branch, and construct the preset hat re-identification model based on the double-branch network structure when the comparison result meets the requirements.

7. An electronic device, comprising: The electronic device includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions, when the instructions are executed, the method of any one of claims 1-5 is performed.

Images