Intelligent canteen management system and method based on Internet of Things technology

Abstract

The invention relates to the field of intelligent monitoring, and provides an intelligent canteen management system and method based on the Internet of Things technology, which analyzes employee targets in real-time canteen kitchen monitoring images to judge whether the employees have illegal behaviors that the employees do not wear chef caps, masks and chef clothes or not. And generating an alarm prompt in response to the existence of the illegal behavior. Thus, automatic detection is achieved through the computer vision technology, the time and cost of manual inspection are reduced, illegal behaviors can be found in time, an alarm prompt can be generated, and the canteen management efficiency and food safety can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent monitoring, and more specifically, to a smart canteen management system and method based on Internet of Things technology. Background Art

[0002] In today's society, food safety issues have attracted much attention. As an important place for people's daily diet, the management level of the canteen is directly related to the health of the public. In canteen management, the dress code of employees is crucial. As one of the essential equipment for kitchen staff, the role of the chef's hat cannot be underestimated. It can effectively prevent hair from falling into food and avoid hygiene problems caused by hair mixed into food. Masks also play a key role. They can effectively block droplets and dust, especially during the cooking process, to prevent the chef's mouth and nose droplets from contaminating food, providing a strong guarantee for food safety. Chef uniforms not only help to maintain overall cleanliness, but also show a professional image, allowing diners to feel the standardized management of the canteen.

[0003] However, traditional canteen management methods often rely on manual inspections by canteen managers to supervise employee dress codes. This method not only consumes a lot of time and manpower, but managers need to patrol various areas of the canteen to ensure that every employee complies with the dress code. Moreover, manual inspections are prone to omissions and delays. For example, during peak meal times, managers may not be able to detect individual employees' dress code problems in time; or when managers are busy with other matters, they may relax their supervision of employees' dress code. Once irregular dress codes occur, it will bring potential risks to food hygiene and affect the overall image and reputation of the canteen.

[0004] Therefore, a smart canteen management solution based on Internet of Things technology is needed. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present application provides a smart canteen management system and method based on Internet of Things technology.

[0006] According to one aspect of the present application, a smart canteen management system based on Internet of Things technology is provided, which includes: The canteen kitchen data collection module is used to obtain real-time monitoring images of the canteen kitchen; A canteen kitchen staff target detection module is used to perform human target detection on the real-time monitoring image of the canteen kitchen to obtain multiple canteen staff area of ​​interest images; A canteen kitchen staff wear detection data sorting module is used to perform chef hat detection, mask detection and chef uniform detection on the images of the regions of interest of the multiple canteen staff to obtain a multi-channel wear detection map of the multiple canteen staff; A canteen kitchen staff wear detection data encoding module is used to perform feature encoding on the multiple canteen staff multi-channel wear detection images to obtain a canteen staff wear classification feature vector; The alarm prompt generation module is used to obtain the alarm prompt generation result based on the classification feature vector of the canteen staff wear.

[0007] According to another aspect of the present application, a smart canteen management method based on Internet of Things technology is provided, which includes: Obtain real-time monitoring images of the canteen kitchen; Performing human target detection on the real-time monitoring image of the cafeteria kitchen to obtain images of regions of interest to multiple cafeteria employees; Performing chef hat detection, mask detection and chef uniform detection on the plurality of canteen staff region of interest images respectively to obtain a plurality of canteen staff multi-channel wearing detection images; Performing feature encoding on the plurality of canteen staff multi-channel wear detection images to obtain a canteen staff wear classification feature vector; Based on the canteen staff wear classification feature vector, an alarm prompt generation result is obtained.

[0008] This application has significant technical effects due to the adoption of the above technical solutions: The application provides a smart canteen management system and method based on the Internet of Things technology, which analyzes the employee targets in the real-time monitoring images of the canteen kitchen to determine whether there are any violations such as employees not wearing chef hats, masks, or chef uniforms, and generates an alarm prompt in response to the existence of violations. In this way, automatic detection is achieved through computer vision technology, reducing the time and cost of manual inspections, and being able to detect violations in a timely manner and generate alarm prompts, which is conducive to improving the efficiency of canteen management and food safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0010] Figure 1 This is a flowchart of a smart canteen management method based on Internet of Things technology according to an embodiment of the present application.

[0011] Figure 2In the smart canteen management method based on the Internet of Things technology according to an embodiment of the present application, chef hat detection, mask detection and chef uniform detection are performed on the images of the areas of interest of the multiple canteen employees to obtain a flowchart of multi-channel wearing detection images of multiple canteen employees.

[0012] Figure 3 This is a flowchart of performing feature encoding on the multiple canteen employee multi-channel wear detection images to obtain a canteen employee wear classification feature vector in a smart canteen management method based on the Internet of Things technology according to an embodiment of the present application.

[0013] Figure 4 This is a flowchart of obtaining an alarm prompt generation result based on the classification feature vector of the canteen employees' wear in the smart canteen management method based on the Internet of Things technology according to an embodiment of the present application.

[0014] Figure 5 This is a system block diagram of a smart canteen management system based on Internet of Things technology according to an embodiment of the present application. DETAILED DESCRIPTION

[0015] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0016] As mentioned in the above background technology, in canteen management, employee dress code is very important. Chef hats can prevent hair from falling into food, masks can effectively block droplets and dust to avoid contaminating food, and chef uniforms help maintain an overall clean and professional image. Traditional canteen management methods often rely on manual inspections by canteen managers to supervise employee dress codes, which not only consumes a lot of time and manpower, but is also prone to omissions and untimely situations. Therefore, a smart canteen management solution based on the Internet of Things technology is expected.

[0017] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. In addition, deep learning and neural networks have also demonstrated a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields. The development of deep learning and neural networks provides new solutions and solutions for smart canteen management.

[0018] Figure 1 Flow chart of the smart canteen management method based on Internet of Things technology according to an embodiment of the present application. Figure 1As shown, according to the embodiment of the present application, the smart canteen management method based on the Internet of Things technology includes: S110, obtaining a real-time monitoring image of the canteen kitchen; S120, performing human target detection on the real-time monitoring image of the canteen kitchen to obtain multiple canteen employee area of ​​interest images; S130, performing chef hat detection, mask detection and chef uniform detection on the multiple canteen employee area of ​​interest images respectively to obtain multiple canteen employee multi-channel wear detection images; S140, performing feature encoding on the multiple canteen employee multi-channel wear detection images to obtain a canteen employee wear classification feature vector; S150, based on the canteen employee wear classification feature vector, obtaining an alarm prompt generation result.

[0019] In step S110, a real-time monitoring image of the canteen kitchen is obtained. It should be understood that the real-time monitoring image of the canteen kitchen can clearly show the overall clothing of the employees, including the head, face and body parts. This is the basis for judging whether the employees wear chef hats, masks and chef uniforms. In addition, the monitoring image also shows the overall environmental information of the kitchen, including workbenches, cooking equipment and food storage areas. In general, the real-time monitoring image is the core input data of the smart canteen management, which contains the employee's wearing information and the background of the working environment. By analyzing these images, it is possible to effectively judge whether the employee's dress complies with the regulations, thereby improving the efficiency of canteen management and food safety. In particular, in an implementable embodiment of the present application, a high-resolution network camera can be selected to collect real-time monitoring image data, and it can be installed in key positions of the kitchen, such as entrances, workbenches and cooking areas, to ensure that the camera can cover all employee work areas and avoid blind spots. As an IoT device, the camera can be connected to the backend server in real time through the network, and the collected image data can be analyzed on the server to determine whether the employee has irregular wearing violations, thereby realizing smart canteen management.

[0020] In step S120, human target detection is performed on the real-time monitoring image of the canteen kitchen to obtain multiple images of the area of ​​interest of the canteen employees. Specifically, in an embodiment of the present application, human target detection is performed on the real-time monitoring image of the canteen kitchen to obtain multiple images of the area of ​​interest of the canteen employees, including: passing the real-time monitoring image of the canteen kitchen through a human target detection network to obtain the multiple images of the area of ​​interest of the canteen employees. It should be understood that the real-time monitoring image may contain many irrelevant factors (such as equipment, ingredients, etc.). In order to filter out these interfering information, focus on the wearing status of the employees, and extract the area of ​​interest of each employee, that is, it is necessary to quickly and accurately locate the position of each employee in the real-time monitoring image, so as to better learn and identify the wearing specifications of the model. Therefore, in the technical solution of the present application, the real-time monitoring image of the canteen kitchen is passed through a human target detection network to obtain multiple images of the area of ​​interest of the canteen employees. The human target detection network in the present application is a target detection network based on an anchor-free window. Anchor-free object detection networks (such as CenterNet, DETR, etc.) no longer rely on fixed anchor boxes and can more flexibly adapt to objects of different sizes and shapes, thus better adapting to complex environments such as canteen kitchens. In particular, the anchor-free design reduces the complexity of anchor box matching by directly regressing the center point and bounding box of the target, reducing the probability of false detection and missed detection, thereby improving detection accuracy.

[0021] In step S130, chef hat detection, mask detection and chef uniform detection are performed on the plurality of cafeteria staff region of interest images to obtain a plurality of cafeteria staff multi-channel wear detection images. Specifically, Figure 2 This is a flowchart of performing chef hat detection, mask detection and chef uniform detection on the images of the regions of interest of multiple canteen employees in the smart canteen management method based on the Internet of Things technology according to the embodiment of the present application to obtain a multi-channel wearing detection map of multiple canteen employees. Figure 2As shown, chef hat detection, mask detection and chef uniform detection are performed on the multiple canteen staff region of interest images respectively to obtain multiple canteen staff multi-channel wearing detection images, including: S131, each canteen staff region of interest image in the multiple canteen staff region of interest images is passed through a chef hat target detection network to obtain multiple chef hat region of interest images; S132, each canteen staff region of interest image in the multiple canteen staff region of interest images is passed through a mask target detection network to obtain multiple mask region of interest images; S133, each canteen staff region of interest image in the multiple canteen staff region of interest images is passed through a chef uniform target detection network to obtain multiple chef uniform region of interest images; S134, the chef hat region of interest image, the mask region of interest image, and the chef uniform region of interest image corresponding to the same canteen staff region of interest image in the multiple chef hat region of interest images, the multiple mask region of interest images, and the multiple chef uniform region of interest images are merged to obtain a canteen staff multi-channel wearing detection image, thereby obtaining the multiple canteen staff multi-channel wearing detection images.

[0022] In order to more clearly analyze the wearing of chef hats by each employee and avoid confusion or misjudgment caused by multiple people appearing in the image at the same time, the technical solution of the present application needs to pass each of the multiple canteen employee area of ​​interest images through the chef hat target detection network to obtain multiple chef hat area of ​​interest images. By processing each canteen employee area of ​​interest image separately, more detailed feedback can be provided on the wearing status of each employee. This focus helps to more accurately judge the wearing standards of each employee. The chef hat target detection network in the present application is an anchor window-based target detection network. Anchor window-based detection is achieved by first generating anchor frames and then performing classification and regression within these frames. This design can effectively reduce the amount of calculation and improve the processing speed, which helps to make real-time analysis of wearing standard detection more feasible.

[0023] In particular, considering that the encoding process is the same when the chef hat target detection network performs chef hat detection on multiple cafeteria staff region of interest images, the processing of one cafeteria staff region of interest image is used to illustrate. In an embodiment of the present application, a possible implementation method of passing the cafeteria staff region of interest image through the chef hat target detection network to obtain the chef hat region of interest image may be: using the chef hat target detection network to process the cafeteria staff region of interest image using the following formula to obtain the chef hat region of interest image; wherein the formula is: ;in, is the image of the area of ​​interest of the cafeteria staff, is the anchor frame, is the chef hat region of interest image, Indicates classification, Indicates regression.

[0024] It is not difficult to understand that in the canteen kitchen environment, food safety and hygiene are of vital importance. Employees wearing masks can effectively prevent the spread of bacteria and viruses and protect food safety. In order to monitor in real time whether the canteen kitchen staff wear masks, it is necessary to pass each of the multiple canteen employee area of ​​interest images through the mask target detection network to obtain multiple mask area of ​​interest images. By extracting the mask area of ​​interest image of each employee and processing the mask area of ​​interest image in subsequent steps, it is possible to detect the mask wearing situation of the monitored employees, and then determine whether the employees have wearing specifications. The mask target detection network in this application is a target detection network based on anchor windows. By matching the anchor frame with the actual target, false detection (other items mistakenly identified as masks) and missed detection (failure to identify masks) can be reduced to ensure the reliability of the mask wearing detection results.

[0025] Chef uniforms are the standard attire for canteen staff, which can effectively prevent food contamination and cross-infection. By detecting whether employees are wearing chef uniforms, it can be ensured that the hygiene standards of the canteen are complied with, thereby ensuring food safety. Therefore, in the technical solution of the present application, it is necessary to pass each of the multiple canteen employee area of ​​interest images through the chef uniform target detection network to obtain multiple chef uniform area of ​​interest images. By processing the extracted chef uniform area of ​​interest images of each employee, the employee's dress can be identified, thereby realizing the detection of employee wearing specifications. The chef uniform target detection network in the present application is an anchor window-based target detection network. The anchor window-based target detection design can use contextual information (such as employee posture) to improve the accuracy of chef uniform target detection. This contextual association helps to more accurately judge whether employees are wearing chef uniforms and reduce the possibility of misjudgment.

[0026] The wearing specifications of each employee involve multiple aspects, including chef hats, masks and chef uniforms. Detecting each part separately may not fully reflect the overall wearing situation of the employees. In order to form a comprehensive view, it is convenient to comprehensively evaluate whether the employees' wearing meets the specifications, it is necessary to merge the chef hat region of interest images, mask region of interest images, and chef uniform region of interest images corresponding to the same canteen employee region of interest image in the multiple chef hat region of interest images, the multiple mask region of interest images, and the multiple chef uniform region of interest images to obtain a canteen employee multi-channel wearing detection map, and then obtain multiple canteen employee multi-channel wearing detection maps. By merging images, the status of multiple wearing elements can be considered at the same time, thereby reducing the information loss caused by separate detection. For example, an employee may wear a mask but not a chef hat. Separate detection may lead to misjudgment, while the merged multi-channel image can more clearly reflect this situation. That is, the merged image can provide a rich data foundation to support subsequent data analysis.

[0027] In step S140, chef hat detection, mask detection and chef uniform detection are performed on the plurality of cafeteria staff region of interest images to obtain a plurality of cafeteria staff multi-channel wear detection images. Specifically, Figure 3 This is a flow chart of performing feature encoding on the multiple canteen staff multi-channel wear detection images to obtain the canteen staff wear classification feature vector in the smart canteen management method based on the Internet of Things technology according to the embodiment of the present application. Figure 3 As shown, the plurality of canteen employee multi-channel wear detection images are feature encoded to obtain a canteen employee wear classification feature vector, including: S141, the plurality of canteen employee multi-channel wear detection images are respectively passed through a wear detector based on a channel attention mechanism to obtain a plurality of canteen employee wear detection feature vectors; S142, the plurality of canteen employee wear detection feature vectors are cascaded to obtain an employee wear detection feature vector; S143, the plurality of canteen employee wear detection feature vectors are two-dimensionally arranged to obtain a canteen employee wear detection feature matrix; S144, the canteen employee wear detection feature vectors are cascaded to obtain an employee wear detection feature matrix; The matrix is ​​passed through a wear feature emphasizer based on a spatial attention mechanism to obtain a cafeteria employee wear detection emphasized feature matrix; S145, the cafeteria employee wear detection emphasized feature matrix is ​​expanded to obtain an employee wear detection emphasized feature vector; S146, based on the employee wear detection emphasized feature vector, the employee wear detection feature vector is corrected for fine-grained inconsistency based on multi-level fusion to obtain an optimized employee wear detection feature vector; S147, the optimized employee wear detection feature vector and the employee wear detection emphasized feature vector are fused to obtain the cafeteria employee wear classification feature vector.

[0028] In order to extract the features of each channel (such as chef hat, mask, chef uniform) image from the multi-channel wear detection map of canteen employees, these features can reflect the status and characteristics of each wear element, it is necessary to pass the multiple canteen employee multi-channel wear detection maps through the wear detector based on the channel attention mechanism to obtain multiple canteen employee wear detection feature vectors. The wear detector based on the channel attention mechanism in this application is a convolutional neural network model using the channel attention mechanism. It should be known to those of ordinary skill in the art that the convolutional neural network model (CNN) performs well in image processing and feature extraction, and can effectively capture spatial features in images. For wear detection tasks, CNN can extract complex image features to help identify various wear elements. The channel attention mechanism can dynamically adjust the weight of each channel, emphasizing the most important features for judging wear specifications. For example, if the wearing of a mask is critical to food safety, the model will automatically increase the weight of features related to the mask, thereby improving the sensitivity and accuracy of the detection. And the channel attention mechanism can combine the contextual information between different channels to identify the relationship between wear elements, so as to more comprehensively judge the wear specifications of employees.

[0029] Considering that the wear detection feature vector of each cafeteria employee may contain different feature information, in order to achieve comprehensive integration of feature information, improve the feature expression ability and classification accuracy of subsequent wear specifications, it is necessary to cascade the multiple cafeteria employee wear detection feature vectors to obtain the employee wear detection feature vector. By integrating multiple feature vectors into a unified feature vector, the model training and reasoning process can be simplified, making data processing more efficient, and better resisting the impact of individual employees' irregular wear, enhancing the robustness of the overall detection.

[0030] Next, the multiple cafeteria employee wear detection feature vectors are arranged in two dimensions to obtain a cafeteria employee wear detection feature matrix. It should be understood that by arranging multiple feature vectors into a matrix, the wearing status of multiple employees can be processed in batches to improve the calculation efficiency. By analyzing the features of key areas in the feature matrix, employees with irregular wear can be better identified.

[0031] In order to emphasize the regional features most relevant to the wearing specifications in the feature matrix formed by arrangement, in the technical solution of the present application, the canteen staff wearing detection feature matrix is ​​passed through a wearing feature emphasizer based on a spatial attention mechanism to obtain a canteen staff wearing detection emphasized feature matrix. Focusing on the features of these key areas can improve the model's sensitivity to wearing specifications, thereby more accurately judging whether there are violations of wearing specifications. The wearing feature emphasizer based on the spatial attention mechanism in the present application is a convolutional neural network model using a spatial attention mechanism. The spatial attention mechanism can adaptively learn the importance of features and automatically adjust the degree of attention to different areas. By emphasizing the features of key areas and suppressing irrelevant information, a high accuracy can be maintained when detecting wearing specifications.

[0032] The canteen staff wear detection emphasized feature matrix is ​​usually high-dimensional. In order to reduce the dimension of feature data and make subsequent analysis simpler, it is necessary to expand the canteen staff wear detection emphasized feature matrix to obtain the staff wear detection emphasized feature vector. One-dimensional feature vectors are usually more efficient than high-dimensional matrices in calculation and can accelerate the execution of algorithms.

[0033] In particular, it is considered that the employee wear detection feature vector is obtained by cascading multiple canteen employee wear detection feature vectors. In this process, each canteen employee wear detection feature vector is obtained by processing the canteen employee multi-channel wear detection map by a wear detector based on a channel attention mechanism. The employee wear detection emphasis feature vector is obtained by processing the wear feature emphasizer based on the spatial attention mechanism. Different networks and mechanisms may use different algorithms and parameters when processing images and extracting features, which will lead to differences in feature distribution. There is a fine-grained inconsistency in feature distribution between the employee wear detection feature vector and the employee wear detection emphasis feature vector, which will cause the fused feature vector to have abnormal stacking on the boundary of the feature manifold. Based on this, in the technical solution of the present application, before fusing the employee wear detection feature vector and the employee wear detection emphasis feature vector to obtain the canteen employee wear classification feature vector, the employee wear detection feature vector is corrected for fine-grained inconsistency based on the employee wear detection emphasis feature vector based on the multi-level fusion.

[0034] Specifically, in an embodiment of the present application, based on the employee wear detection emphasis feature vector, the employee wear detection feature vector is corrected for fine-grained inconsistency based on multi-level fusion to obtain an optimized employee wear detection feature vector, including: calculating the position difference between the employee wear detection feature vector and the employee wear detection emphasis feature vector to obtain an employee wear detection shift position difference representation vector; after cascading the employee wear detection feature vector and the employee wear detection emphasis feature vector into an employee wear detection joint feature vector, the employee wear detection joint feature vector is input into a one-dimensional convolution layer to obtain an employee wear detection shift convolution coding representation vector; after linear embedding modulation encoding is performed on the employee wear detection feature vector using a first embedding matrix into an employee wear detection feature linear modulation vector, the employee wear detection feature vector is calculated. The linear modulation vector and the employee wear detection emphasized feature vector are differentially represented by position to obtain the employee wear detection shift modulation differential representation vector; the employee wear detection shifted differential representation vector is used as the value vector, the employee wear detection shift convolutional coding representation vector and the employee wear detection shift modulation differential representation vector are used as the query vector and the key vector, and the employee wear detection shifted differential representation vector, the employee wear detection shift convolutional coding representation vector and the employee wear detection shift modulation differential representation vector are input into the shift corrector based on the simulated converter structure to obtain the employee wear detection multi-scale hierarchical fusion shift modulation weight vector; the position point multiplication between the employee wear detection multi-scale hierarchical fusion shift modulation weight vector and the employee wear detection feature vector is calculated to obtain the optimized employee wear detection feature vector.

[0035] In an embodiment of the present application, another expression of performing fine-grained inconsistency correction based on multi-level fusion on the employee wear detection feature vector based on the employee wear detection emphasis feature vector to obtain an optimized employee wear detection feature vector may be: processing the employee wear detection feature vector by the employee wear detection emphasis feature vector according to the following formula to obtain the optimized employee wear detection feature vector; wherein the formula is: ; ; ; ; ;in, represents the employee wear detection feature vector, represents the employee wear detection emphasized feature vector, It means to subtract by position point. It represents the vector representing the employee's wear detection displacement according to the position difference, Indicates cascade operation, represents one-dimensional convolutional coding, represents the employee wear detection shift convolutional coding representation vector, represents the first embedding matrix, represents the first modulation bias vector, represents the employee wear detection shift modulation differential representation vector, represents vector multiplication, represents the normalized exponential function, represents the transposed vector of the eigenvector, represents the length of the feature vector, represents the activation function, represents the multi-scale hierarchical fusion shift modulation weight vector of employee wear detection, It means point multiplication by position. Represents the optimized employee wear detection feature vector.

[0036] It should be understood that in the technical solution of the present application, in the process of fusing the employee wear detection feature vector and the employee wear detection emphasis feature vector, a fine-grained inconsistency correction method based on multi-level fusion is proposed to address the problem of fine-grained inconsistency in feature distribution that arises in the process of fusing the employee wear detection feature vector and the employee wear detection emphasis feature vector, aiming to reduce the abnormal stacking phenomenon of the fused feature vector on the boundary of the feature manifold, so as to improve the robustness and classification performance of the model.

[0037] Specifically, the method first calculates the element-by-element difference between the employee wear detection feature vector and the employee wear detection emphasis feature vector to obtain the employee wear detection shift position differential representation vector, thereby capturing the difference between the two feature vectors. Then, the employee wear detection feature vector and the employee wear detection emphasis feature vector are cascaded into an employee wear detection joint feature vector and input into a one-dimensional convolution layer to generate an employee wear detection shift convolution coding representation vector. That is, the deep-level shift implicit information between the employee wear detection feature vector and the employee wear detection emphasis feature vector is captured by one-dimensional convolution coding. At the same time, the employee wear detection feature vector is linearly embedded and modulated using the first embedding matrix to be encoded into an employee wear detection feature linear modulation vector, and the element-by-element difference between the employee wear detection feature linear modulation vector and the employee wear detection emphasis feature vector is calculated to obtain an employee wear detection shift modulation differential representation vector. The employee wear detection shift modulation differential representation vector is used to represent the position expression of the shift information of the employee wear detection feature vector and the employee wear detection emphasis feature vector in the linear space. That is, in the technical solution of the present application, the employee wear detection shift position differential representation vector, the employee wear detection shift convolutional coding representation vector and the employee wear detection shift modulation differential representation vector respectively represent the explicit expression of the fine-grained distribution difference shift information between the employee wear detection feature vector and the employee wear detection emphasis feature vector in different feature spaces.

[0038] Next, the employee wear detection shift position differential representation vector, the employee wear detection shift convolutional coding representation vector and the employee wear detection shift modulation differential representation vector are used as the query vector and the key vector respectively, and these vectors are input into the shift corrector based on the simulated transformer structure to generate the employee wear detection multi-scale hierarchical fusion shift modulation weight vector. In this way, the attention mechanism is used to evaluate the importance of features at different scales, that is, the attention mechanism is used to simulate the hierarchical precipitation expression of the fine-grained difference features between the employee wear detection feature vector and the employee wear detection emphasized feature vector in different feature spaces. Finally, the element-by-element product of the employee wear detection multi-scale hierarchical fusion shift modulation weight vector and the employee wear detection feature vector is calculated to obtain the optimized employee wear detection feature vector, that is, the weight is assigned to each element in the employee wear detection feature vector through the weighted operation, which reduces the influence of local disturbances and improves the quality of the feature vector.

[0039] In summary, in the method, the employee wear detection shift position differential representation vector, the employee wear detection shift convolutional coding representation vector and the employee wear detection shift modulation differential representation vector respectively represent the employee wear detection feature vector and the employee wear detection emphasized feature vector, which are explicit expressions of fine-grained distribution difference shift information in different feature spaces. The generation of the employee wear detection multi-scale hierarchical fusion shift modulation weight vector uses the attention mechanism to evaluate the importance of features at different scales and provide weight information for feature shift correction. Ultimately, the generation of the optimized employee wear detection feature vector reduces the impact of disturbances, improves the robustness of the model to local disturbances in the input data, enhances the model's adaptability to noisy or variant data sets, improves the fusion quality of the feature vector, and improves the overall classification performance and training efficiency of the model.

[0040] The optimized employee wear detection feature vector contains the basic wear information features, while the employee wear detection emphasis feature vector emphasizes the key features. In order to integrate the information of these two vectors and form a more comprehensive feature representation, the optimized employee wear detection feature vector and the employee wear detection emphasis feature vector are fused in the technical solution of the present application to obtain the canteen employee wear classification feature vector. The fused feature vector can provide richer information, so that the model can make more accurate classification when judging whether the employee wear is standardized, thereby improving the accuracy. Specifically, when one of the feature vectors is used alone, some key information may be lost. Through fusion, information can be retained to the maximum extent, and information loss in the judgment process can be reduced to improve resistance to noise and interference.

[0041] In step S150, based on the canteen staff wear classification feature vector, an alarm prompt generation result is obtained. Specifically, Figure 4 This is a flowchart of obtaining the alarm prompt generation result based on the classification feature vector of the canteen staff wear in the smart canteen management method based on the Internet of Things technology according to the embodiment of the present application. Figure 4 As shown, based on the canteen staff wear classification feature vector, an alarm prompt generation result is obtained, including: S151, the canteen staff wear classification feature vector is passed through a classifier-based violation detector to obtain a classification result, and the classification result is used to indicate whether there is a violation of the canteen staff not wearing the standard at the current time point; S152, in response to the violation of the canteen staff not wearing the standard at the current time point, an alarm prompt is generated.

[0042] Finally, the canteen staff wear classification feature vector is passed through a classifier-based violation detector to obtain a classification result, and the classification result is used to indicate whether there is a violation of the canteen staff not wearing the standard at the current time point, and in response to the violation of the canteen staff not wearing the standard at the current time point, an alarm prompt is generated. It should be understood that the classifier is a machine learning model that is used to assign input data to predefined categories. The classifier learns the mapping relationship between features and categories by training the existing labeled data. In practical applications, after inputting a new canteen staff wear classification feature vector, the classifier will make a judgment based on the learned rules and output the classification result. The classification result directly indicates whether there is a violation of the staff not wearing the standard at the current time point, providing clear information. When the classification result shows that there is a violation, an alarm prompt can be automatically generated to notify the management personnel in time to ensure a quick response. The detection and alarm mechanism based on the classifier can improve the compliance awareness of employees, reduce the occurrence of violations, and improve the overall management level.

[0043] In summary, the smart canteen management method based on the Internet of Things technology according to the embodiment of the present application is explained, which analyzes the employee targets in the real-time monitoring image of the canteen kitchen to determine whether there are any violations such as employees not wearing chef hats, masks, or chef uniforms, and generates an alarm prompt in response to the existence of violations. In this way, automatic detection is achieved through computer vision technology, reducing the time and cost of manual inspections, and being able to detect violations in a timely manner and generate alarm prompts, which is conducive to improving the efficiency of canteen management and food safety.

[0044] Figure 5 : is a system block diagram of a smart canteen management system based on Internet of Things technology according to an embodiment of the present application. Figure 5 As shown, according to the embodiment of the present application, the smart canteen management system 100 based on the Internet of Things technology includes: a canteen kitchen data acquisition module 110, which is used to obtain real-time monitoring images of the canteen kitchen; a canteen kitchen employee target detection module 120, which is used to perform human target detection on the real-time monitoring image of the canteen kitchen to obtain multiple canteen employee area of ​​interest images; a canteen kitchen employee wear detection data sorting module 130, which is used to perform chef hat detection, mask detection and chef uniform detection on the multiple canteen employee area of ​​interest images respectively to obtain multiple canteen employee multi-channel wear detection images; a canteen kitchen employee wear detection data encoding module 140, which is used to feature encode the multiple canteen employee multi-channel wear detection images to obtain a canteen employee wear classification feature vector; an alarm prompt generation module 150, which is used to obtain an alarm prompt generation result based on the canteen employee wear classification feature vector.

[0045] Here, those skilled in the art can understand that the specific functions and operations of each unit and module in the above-mentioned smart canteen management system 100 based on the Internet of Things technology have been referred to above. Figures 1 to 4 The description of the smart canteen management method based on the Internet of Things technology has been introduced in detail, and therefore, its repeated description will be omitted.

[0046] In summary, the smart canteen management system 100 based on the Internet of Things technology according to the embodiment of the present application is explained, which analyzes the employee targets in the real-time monitoring image of the canteen kitchen to determine whether there are any violations such as employees not wearing chef hats, masks, or chef uniforms, and generates an alarm prompt in response to the existence of violations. In this way, automatic detection is achieved through computer vision technology, reducing the time and cost of manual inspections, and being able to detect violations in a timely manner and generate alarm prompts, which is conducive to improving the efficiency of canteen management and food safety.

[0047] As described above, the smart canteen management system 100 based on the Internet of Things technology according to the embodiment of the present application can be implemented in various wireless terminals, such as a server for smart canteen management based on the Internet of Things technology. In one example, the smart canteen management system 100 based on the Internet of Things technology according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the smart canteen management system 100 based on the Internet of Things technology can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the smart canteen management system 100 based on the Internet of Things technology can also be one of the many hardware modules of the wireless terminal.

[0048] Alternatively, in another example, the smart canteen management system 100 based on the Internet of Things technology and the wireless terminal may also be separate devices, and the smart canteen management system 100 based on the Internet of Things technology may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

Claims

1. A smart canteen management method based on Internet of Things technology, It is characterized in that include: Obtain real-time monitoring images of the canteen kitchen; Performing human target detection on the real-time monitoring image of the cafeteria kitchen to obtain images of regions of interest to multiple cafeteria employees; Performing chef hat detection, mask detection and chef uniform detection on the plurality of canteen staff region of interest images respectively to obtain a plurality of canteen staff multi-channel wearing detection images; Performing feature encoding on the plurality of canteen staff multi-channel wear detection images to obtain a canteen staff wear classification feature vector; Based on the canteen staff wear classification feature vector, an alarm prompt generation result is obtained.

2. According to claim 1, the smart canteen management method based on Internet of Things technology, It is characterized in that Performing human target detection on the real-time monitoring image of the canteen kitchen to obtain multiple images of the area of ​​interest of the canteen employees, including: passing the real-time monitoring image of the canteen kitchen through a human target detection network to obtain the multiple images of the area of ​​interest of the canteen employees.

3. According to claim 2, the smart canteen management method based on Internet of Things technology, It is characterized in that The chef hat detection, mask detection and chef uniform detection are respectively performed on the images of the regions of interest of the multiple cafeteria employees to obtain a multi-channel wearing detection map of multiple cafeteria employees, including: Passing each of the plurality of cafeteria staff region of interest images through a chef hat object detection network to obtain a plurality of chef hat region of interest images; Passing each of the plurality of canteen staff region of interest images through a mask target detection network to obtain a plurality of mask region of interest images; Passing each of the plurality of canteen staff region of interest images through a chef uniform object detection network to obtain a plurality of chef uniform region of interest images; The chef hat region of interest image, the mask region of interest image, and the chef uniform region of interest image corresponding to the same canteen staff region of interest image among the multiple chef hat region of interest images, the multiple mask region of interest images, and the multiple chef uniform region of interest images are merged to obtain a multi-channel wearing detection image of canteen staff, and then the multiple canteen staff multi-channel wearing detection images are obtained.

4. According to claim 3, the smart canteen management method based on Internet of Things technology, It is characterized in that The plurality of canteen staff multi-channel wear detection images are feature encoded to obtain a canteen staff wear classification feature vector, including: The plurality of canteen staff multi-channel wear detection images are respectively passed through a wear detector based on a channel attention mechanism to obtain a plurality of canteen staff wear detection feature vectors; Cascading the multiple canteen staff wear detection feature vectors to obtain a staff wear detection feature vector; Arranging the plurality of cafeteria staff wear detection feature vectors in two dimensions to obtain a cafeteria staff wear detection feature matrix; The cafeteria staff wear detection feature matrix is ​​passed through a wear feature emphasizer based on a spatial attention mechanism to obtain a cafeteria staff wear detection emphasized feature matrix; Expanding the cafeteria staff wear detection emphasis feature matrix to obtain the staff wear detection emphasis feature vector; Based on the employee wear detection emphasis feature vector, performing fine-grained inconsistency correction based on multi-level fusion on the employee wear detection feature vector to obtain an optimized employee wear detection feature vector; The optimized employee wear detection feature vector and the employee wear detection emphasis feature vector are integrated to obtain the cafeteria employee wear classification feature vector.

5. According to claim 4, the smart canteen management method based on Internet of Things technology, It is characterized in that Based on the employee wear detection emphasis feature vector, the employee wear detection feature vector is corrected for fine-grained inconsistency based on multi-level fusion to obtain an optimized employee wear detection feature vector, including: Calculating the position difference between the employee wear detection feature vector and the employee wear detection emphasis feature vector to obtain an employee wear detection shift position difference representation vector; After concatenating the employee wear detection feature vector and the employee wear detection emphasis feature vector into an employee wear detection joint feature vector, the employee wear detection joint feature vector is input into a one-dimensional convolution layer to obtain an employee wear detection shift convolution coding representation vector; After performing linear embedding modulation encoding on the employee wear detection feature vector using the first embedding matrix to obtain an employee wear detection feature linear modulation vector, a position difference between the employee wear detection feature linear modulation vector and the employee wear detection emphasis feature vector is calculated to obtain an employee wear detection shift modulation difference representation vector; Taking the employee wear detection shifted position differential representation vector as a value vector, the employee wear detection shifted convolutional coding representation vector and the employee wear detection shifted modulation differential representation vector as a query vector and a key vector, the employee wear detection shifted position differential representation vector, the employee wear detection shifted convolutional coding representation vector and the employee wear detection shifted modulation differential representation vector are input into a shift corrector based on an imitation transformer structure to obtain a multi-scale hierarchical fusion shift modulation weight vector for employee wear detection; The optimized employee wear detection feature vector is obtained by calculating the point-by-point multiplication between the employee wear detection multi-scale hierarchical fusion shift modulation weight vector and the employee wear detection feature vector.

6. According to claim 5, the smart canteen management method based on Internet of Things technology, It is characterized in that Based on the canteen staff wear classification feature vector, the alarm prompt generation result is obtained, including: The canteen staff wear classification feature vector is passed through a classifier-based violation detector to obtain a classification result, and the classification result is used to indicate whether there is a violation of the canteen staff not wearing standard clothes at the current time point; In response to the violation of canteen staff not wearing standard clothing at the current time point, an alarm prompt is generated.

7. According to the smart canteen management method based on Internet of Things technology described in claim 6, It is characterized in that The human object detection network is an object detection network based on no anchor window, and the chef hat object detection network, the mask object detection network and the chef uniform object detection network are all object detection networks based on anchor windows.

8. According to claim 7, the smart canteen management method based on Internet of Things technology, It is characterized in that The wearable detector based on the channel attention mechanism is a convolutional neural network model using the channel attention mechanism, and the wearable feature emphasizer based on the spatial attention mechanism is a convolutional neural network model using the spatial attention mechanism.

9. A smart canteen management system based on Internet of Things technology, It is characterized in that include: The canteen kitchen data collection module is used to obtain real-time monitoring images of the canteen kitchen; A canteen kitchen staff target detection module is used to perform human target detection on the real-time monitoring image of the canteen kitchen to obtain multiple canteen staff area of ​​interest images; A canteen kitchen staff wear detection data sorting module is used to perform chef hat detection, mask detection and chef uniform detection on the images of the regions of interest of the multiple canteen staff to obtain a multi-channel wear detection map of the multiple canteen staff; A canteen kitchen staff wear detection data encoding module is used to perform feature encoding on the multiple canteen staff multi-channel wear detection images to obtain a canteen staff wear classification feature vector; The alarm prompt generation module is used to obtain the alarm prompt generation result based on the classification feature vector of the canteen staff wear.

10. According to claim 9, the smart canteen management system based on Internet of Things technology, It is characterized in that The cafeteria kitchen staff target detection module is used to: obtain the plurality of cafeteria staff area of ​​interest images by passing the real-time monitoring image of the cafeteria kitchen through a human target detection network.

Images