A Smart Monitoring and Analysis System for Standardized Operation Processes of Restaurant Chain Stores

Abstract

This invention discloses an intelligent monitoring and analysis system for standardized operation processes in restaurant chains, belonging to the field of data processing technology. It includes: a multi-dimensional data perception module that collects video streams, order data, and material weight data; a spatiotemporal semantic association module that synchronizes the time deviation of each device to within 10ms based on the IEEE 1588 protocol and extracts corresponding video slices according to order timestamps; an operation behavior recognition module that extracts human skeleton points and identifies atomic action sequences such as feeding and stirring; a material flow tracking module that triggers pressure sensors in response to feeding actions to record weight changes at a sampling frequency of no less than 100Hz, calculating the measured feeding amount when the standard deviation is less than 0.3g; and a process compliance evaluation module that compares the atomic action sequences and measured feeding amounts with standard operating procedures to generate a quantitative score. This invention solves the pain points of logical disconnect between video and order data and difficulty in tracing material losses, achieving refined closed-loop monitoring of kitchen operations.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to an intelligent monitoring and analysis system for the standardized operation process of a restaurant chain store. Background Technology

[0002] In the restaurant chain industry, the degree of standardization in store operations directly determines a brand's market competitiveness and operational stability. As the scale of a chain expands, efficient supervision of its widely distributed stores has become a cornerstone of corporate management. This type of supervision encompasses multiple dimensions, including kitchen operating procedures, food safety and quality, and material cost control. Its core objective is to reduce management errors, improve overall operational efficiency, and ensure a consistent customer experience through unified process standards.

[0003] Real-time monitoring and intelligent analysis of kitchen operations and material consumption are key to achieving refined store operations. This technological approach aims to utilize computer vision, the Internet of Things, and big data analytics to track the entire supply chain of raw materials and assess the compliance of kitchen staff's actions. By establishing a mapping between order requirements and actual operations, the system can automatically identify abnormal deviations in the operational process, thereby providing quantitative support for management decisions.

[0004] Current technologies still face significant challenges in the field of food service supervision. Traditional standardized supervision relies primarily on periodic manual on-site inventory checks and random inspections of surveillance videos. This model suffers from severe lag, only able to detect total material shortages through final inventory discrepancies, but unable to accurately pinpoint whether losses stem from excessive feeding during food preparation, food damage, or improper operation. Ordinary video surveillance systems lack the ability to understand the semantics of kitchen operations, resulting in a logical disconnect between the monitored footage and specific order material consumption. Because of the inability to semantically correlate order data with real-time video, management struggles to achieve refined and real-time digital mapping of operational processes. These deficiencies lead to overly coarse-grained supervision, failing to fundamentally address core pain points such as the high degree of arbitrariness in kitchen operations and the difficulty in tracing material losses.

[0005] Therefore, there is an urgent need for an optimized intelligent monitoring and analysis system for the standardized operation process of restaurant chain stores. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent monitoring and analysis system for the standardized operation process of catering chain stores, so as to solve the problems mentioned in the background art, such as excessively coarse monitoring granularity, difficulty in tracing material loss, and logical disconnect between monitoring images and order data.

[0007] The technical solution of the present invention includes: The multi-dimensional data sensing module is used to collect multi-dimensional raw data related to store operations. The multi-dimensional raw data includes at least video streams collected by network cameras, order data generated by the sales terminal system, and material weight data collected by the pressure sensor array. The spatiotemporal semantic association module is used to synchronize the time deviation of network cameras, sales terminal systems and pressure sensor arrays within 10 milliseconds based on the IEEE 1588 precision time protocol, generate timestamps based on orders in the order data, and retrieve and extract video slices of the corresponding time window from the video stream. The operation behavior recognition module is used to extract human key points from video slices, construct a human skeleton motion model, and identify the atomic action sequence that constitutes the kitchen operation process based on the human skeleton motion model. The atomic actions include at least feeding, stir-frying and plating. The material flow tracking module is used to respond to the feeding action identified by the operation behavior recognition module, trigger the pressure sensor array to record the weight change at a sampling frequency of not less than 100 Hz, and calculate the actual feeding amount based on the weight change; The process compliance evaluation module is used to obtain atomic action sequences and actual delivery volumes, compare them with preset standardized operating procedures, and generate a quantitative score to characterize the compliance of store operation processes.

[0008] Preferably, the spatiotemporal semantic association module is also used for: Based on Zhang's calibration method and PnP algorithm, the network camera is calibrated to obtain its intrinsic and extrinsic parameters in order to establish a mapping from visual space to physical space. Based on the kitchen layout, the physical space is divided into multiple functionally independent processing grids, and each processing grid is associated with a network camera covering the area; When a new order is captured, the system reads a preset time window from the standardized operation process database based on the order's generation timestamp and the corresponding dishes. It then extracts a sequence of consecutive video frames within that time window from the video stream of the network camera associated with the target processing grid, forming a video slice.

[0009] Preferably, the spatiotemporal semantic association module is also used for: To identify the operator executing the order in the video slice, specifically: locate the image-based identification code deployed on the operator using a target detection algorithm, and decode and extract the unique employee number; Using the order number as a unique index, order data, operator numbers, processing grid numbers, and video slice storage paths are integrated into a structured semantic mapping matrix to achieve precise association between orders, personnel, and surveillance images.

[0010] Preferably, the operation behavior recognition module is also used for: A top-down detection approach is adopted. First, the human body region is located through the target detection network, and then the high-resolution network HRNet is used to extract the two-dimensional image coordinates of 25 core skeleton points of the human body. The coordinates, velocities, and accelerations of the 25 core skeleton points in consecutive frames are concatenated into a feature vector, which is then input into an operation classification model that integrates a Long Short-Term Memory Network (LSTM) and a Graph Convolutional Neural Network (GCN) to identify atomic action sequences and their corresponding time intervals. Among them, the Graph Convolutional Neural Network (GCN) uses a graph structure constructed with 25 core skeleton points as nodes and an adjacency matrix set according to the physical connections of the human body to model the spatial correlation of joints.

[0011] Preferably, the material flow tracking module is used to calculate the actual delivery quantity, specifically including: Before the feeding action begins, multiple sampling points of the pressure sensor array are continuously collected. When the standard deviation is less than 0.3 grams, the stable value is recorded as the weight before feeding. After the feeding action is completed, the stable weight after feeding is recorded in the same way. The absolute value of the difference between the stable weight before feeding and the stable weight after feeding is taken as the actual feeding amount.

[0012] Preferably, the material flow tracking module is also used to handle scenarios involving the mixed delivery of multiple materials, specifically including: When the feeding action is detected, the profit area is dynamically delineated in the video frame by combining the key points of the chef's hand with the detection frame of the feeding container. The deep learning semantic segmentation model with embedded spatial attention module is invoked to process the interest region and output a probability map of each pixel belonging to different material categories in order to identify the types of materials that are delivered and their coverage areas in the image. A multimodal fusion algorithm is introduced to estimate the visual volume of various materials based on the material coverage area, and to calculate the theoretical mass by combining the material density database. The theoretical mass is then cross-validated with the actual mass change value measured by the pressure sensor array. If the deviation between the two exceeds a preset threshold, an anomaly flag is triggered.

[0013] Preferably, the process compliance evaluation module is used to generate quantitative scores, specifically including: The minimum cumulative distance between the measured action sequence and the standard action sequence is calculated using the dynamic time warping algorithm and then normalized into an action similarity score. Calculate the material error rate between the measured dosage and the standard dosage; Risk weight coefficients are assigned to different processes, and the similarity scores of actions, material error rates, process completeness, and operation time deviations are combined and weighted to obtain the score for each process. The scores are then adjusted according to the risk weight coefficients, and finally aggregated to generate the process compliance score for the entire order.

[0014] Preferably, it also includes an anomaly tracing and analysis module, used for: When the process compliance score is lower than the preset qualified threshold, the spatiotemporal semantic association module is used to retrieve the store identifier, processing grid, operation timestamp and operator number of the abnormality from the corresponding order number; The system sends an alert command to the store terminal via the MQTT protocol, triggering a red full-screen pop-up window and a voice correction broadcast. It automatically extracts high-definition video clips before and after the time of the anomaly, queries the material weight curve for the corresponding time period, and generates an anomaly cause analysis report.

[0015] Preferably, it also includes an inventory closed-loop verification module, used for: At the end of each operating cycle, calculate the logical inventory, which is defined as the initial inventory minus the total amount of standard materials corresponding to the sales order; The material flow tracking module summarizes the total measured delivery volume and calculates the measured remaining inventory, which is defined as the initial inventory minus the total measured delivery volume. The measured remaining inventory is compared with the logical inventory. If the absolute value of the difference between the two exceeds the preset loss tolerance limit, the anomaly analysis process is triggered. Cluster analysis is performed on the operation time, dish category and employee number of all low-scoring orders in the period to identify the core cause of the loss anomaly.

[0016] Preferably, it also includes a standardized process self-learning module, used for: Regularly retrieve high-scoring orders from each store from the cloud database as excellent operational samples; Generative adversarial networks are used to extract features from the action rhythm, feeding sequence, and heat control patterns in excellent operation samples; After the extracted feature vectors are reviewed by experts, they are synchronized to the standardized operation process database to dynamically update the standardized operating procedures.

[0017] In summary, this application includes at least one of the following beneficial technical effects: 1. This invention synchronizes the time deviation between network cameras, point-of-sale (POS) systems, and pressure sensor arrays to within 10 milliseconds using the IEEE 1588 protocol. Based on order timestamps, it precisely extracts corresponding operation video slices from the video stream, constructing a semantic mapping matrix that includes orders, personnel, processing grids, and video paths. This deeply binds previously isolated business data with operational images from the physical world, enabling a refined mapping of each order to its kitchen processing steps, thus refining the granularity of supervision from "store / time period" to "order / process."

[0018] 2. This invention extracts 25 core skeletal points of the human body using HRNet, and combines LSTM and GCN fusion networks to identify atomic action sequences such as feeding and stirring. Simultaneously, a pressure sensor is triggered in response to the feeding action to record weight changes at a sampling frequency of 100Hz. The measured feeding amount is calculated when the standard deviation is less than 0.3 grams. A multimodal fusion algorithm is used to cross-validate the visual recognition results with the weight data, increasing the confidence level of single-time material consumption to the gram level. Furthermore, a dynamic time warping algorithm is employed to compare the measured action sequence with standard operating procedures, generating a quantitative process compliance score. This achieves a comprehensive evaluation of the standardization level of the kitchen from two dimensions: operational behavior and material consumption.

[0019] 3. When a low-scoring order is detected, the present invention automatically locates the responsible personnel, extracts video clips, and generates a rectification work order through the anomaly tracing and analysis module, realizing real-time correction during the process and evidence preservation afterward; at the same time, it uses the ARIMA model to predict the trend of historical scoring data, identifies the high-incidence period of violations in advance, and generates early warning plans, transforming passive response into proactive prevention. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the intelligent monitoring and analysis system for standardized operation processes of restaurant chain stores, as described in this invention. Figure 2 This is a flowchart illustrating the spatiotemporal semantic association and mapping of multidimensional data in this invention; Figure 3 This is a flowchart illustrating the main stages of operational behavior recognition and quantitative tracking of material flow in this invention. Figure 4 This is a flowchart outlining the main stages of process compliance evaluation and automatic anomaly tracing in this invention. Figure 5 This is a schematic diagram of the multi-level interaction relationship and data flow between edge computing and cloud platform collaboration in this invention. Detailed Implementation

[0021] The following will be combined with the appendix Figures 1 to 5The technical solution of the present invention is clearly and completely described below. Obviously, the following embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0022] This embodiment discloses an intelligent monitoring and analysis system for standardized operation processes in restaurant chains. Built upon modern IoT sensing technology and deep learning algorithms, it aims to achieve precise monitoring of kitchen operations in restaurant stores through comprehensive digital monitoring. In terms of overall architecture, the system adopts a distributed layout combining edge computing and cloud collaboration. This ensures both real-time data processing at the lower levels and in-depth global analysis at the headquarters level. Simultaneously, edge computing nodes are deployed at the store level to handle local data in real time.

[0023] This system mainly includes a multi-dimensional data perception module, a spatiotemporal semantic association module, an operation behavior recognition module, a material flow tracking module, a process compliance evaluation module, an anomaly traceability analysis module, an inventory closed-loop verification module, and a standardized process self-learning module.

[0024] The first module to be described is the multidimensional data perception module. As the bottom data acquisition layer of the entire system, it is responsible for comprehensively collecting multidimensional raw data such as the physical environment, personnel operations, and business orders within the store. The multidimensional data perception module is internally divided into an image acquisition submodule, an order integration submodule, a weight perception submodule, and an environmental parameter sensor group.

[0025] The image acquisition submodule deploys high-resolution network cameras in the kitchen's preliminary processing area, cooking area, food preparation area, and washing and disinfection area. Each camera has a resolution of 3840×2160 pixels and a frame rate of 25 frames per second. The video stream is compressed in real time using a high-efficiency video encoding standard.

[0026] The camera connects to the edge computing node via a wired or wireless network and pushes video streams using the RTSP protocol. Each camera is assigned a globally unique identifier, which is bound to the IP address and installation location in the edge computing node configuration file and used to locate the processing grid to which it belongs in the system topology.

[0027] The order integration submodule connects to the sales terminal system via RESTful API or gRPC to monitor its database changes or message queues in real time.

[0028] After a customer completes an order, the order integration submodule instantly captures the order details by polling the database or subscribing to the message queue. This includes the order number, dish details, the standard bill of materials for each dish, the millisecond-level timestamp of the order, and any special notes or requirements from the customer. This provides an accurate business data benchmark for subsequent material tracking and operational evaluation.

[0029] The weight sensing submodule relies on an array of pressure sensors embedded under the mixing station, refrigerator, and key processing stations to achieve real-time monitoring of material weight.

[0030] The sensor adopts a high-precision strain gauge structure with a static resolution of 0.1 grams and a sampling frequency of 100 Hz. The analog signal from the sensor is converted from analog to digital and then sent to the microcontroller to run a digital filtering algorithm. It integrates a second-order Butterworth low-pass filter with a cutoff frequency of 5 Hz, which effectively filters out high-frequency interference above 5 Hz, such as vibration of the freezer compressor and movement of people.

[0031] The filtered weight data, along with a timestamp, is sent to the edge computing node via RS485 or CAN fieldbus.

[0032] The environmental parameter sensor group includes temperature and humidity sensors and oil fume concentration sensors distributed at key nodes in the kitchen. The sensors collect environmental status parameters at 10-second intervals and report them to edge computing nodes via wireless network or wired connection. The collected temperature, humidity and oil fume concentration values ​​are used to provide environmental context correction basis for subsequent compliance evaluation, such as to compensate for cooking time or material characteristics under high temperature and high humidity conditions.

[0033] In summary, the multi-dimensional data perception module constructs a complete, synchronous, and high-precision data foundation for store operations through the collaborative collection of data from four dimensions: images, orders, weight, and environment. This data foundation provides reliable data support for subsequent spatiotemporal semantic association, operational behavior recognition, and material flow tracking.

[0034] This system maintains a globally unified standardized operating process database on the cloud platform. This database stores the standard operating procedures for each dish, including data such as the standard amount of materials to be added, the standard time range for each process, and the sequence of processes, providing a benchmark for subsequent process compliance evaluation.

[0035] Next, we will introduce the spatiotemporal semantic association module. This module is responsible for connecting the surveillance images in the physical world with the order data in the logical world. It is the core link to achieve deep binding of orders, personnel and operation processes.

[0036] In terms of clock synchronization, the spatiotemporal semantic association module first executes a strict distributed clock synchronization process, using the IEEE 1588 precision time protocol to force the time deviation of all network cameras, sales terminal systems, pressure sensor nodes and edge computing nodes deployed in the store to be synchronized within 10 milliseconds, ensuring that all subsequent cross-device data retrieval and association based on timestamps have a unified time series reference.

[0037] In terms of spatial calibration and mesh generation, the spatiotemporal semantic association module realizes the mapping from visual space to physical space based on a preset coordinate calibration algorithm. The coordinate calibration algorithm uses Zhang's calibration method to perform offline calibration of each camera, obtain its intrinsic parameter matrix and distortion coefficient, and then arranges several calibration points with known world coordinates in the kitchen. The calibration points adopt a checkerboard or circular target, and their world coordinates are obtained by measuring with a laser rangefinder or tape measure. Then, the position and attitude of each camera relative to the virtual three-dimensional space coordinate system, i.e., the extrinsic parameters, are calculated by the PnP algorithm.

[0038] After calibration, the spatiotemporal semantic association module divides the three-dimensional space into several functionally independent processing grids based on the actual layout of the kitchen. The boundary coordinates of the grids are pre-configured in the system based on kitchen drawings or on-site measurements. For example, grid No. 1 corresponds to the stir-frying area, grid No. 2 corresponds to the material preparation area, and each processing grid is associated with one or more cameras covering that area.

[0039] When the order integration submodule captures a new order containing a specific dish, the spatiotemporal semantic association module immediately starts a video slice retrieval mechanism based on a sliding window. This mechanism takes the order generation timestamp as its center and reads a preset time window from the standardized operation process database according to the standard process duration corresponding to the dish. For example, the time span is 40 seconds, from 10 seconds before the order is placed to 30 seconds after the order is placed.

[0040] The spatiotemporal semantic association module finds which processing grid a dish should be processed in based on the dish information in the order. Then, it extracts a sequence of consecutive video frames within the corresponding time window from the video stream of the camera associated with that grid to form a candidate operation video slice for the order.

[0041] After acquiring the video slices, the spatiotemporal semantic association module further identifies the identity of the operator who executed the order by using an image-based identification code deployed on the chef's hat or epaulettes. This identification code can be a QR code containing the employee number or a mark with a specific color pattern. The employee number template corresponding to these marks is collected from the image and registered in the system when the employee joins the company.

[0042] This system performs target detection on each frame in a video slice, locates the area where the identification code is located, and then extracts the unique employee number through QR code decoding technology or template recognition algorithm based on feature matching.

[0043] Finally, the spatiotemporal semantic association module uses the order number as the unique index to integrate all the above-mentioned association information into a structured semantic mapping matrix. Each record in this matrix contains the order number, operator number, processing grid number, the start and end time range of the video slice, and the storage path or URL pointing to the corresponding multi-angle video stream.

[0044] By using a semantic mapping matrix, previously isolated order data, personnel information, and surveillance images are tightly coupled together, providing a precise data association foundation for subsequent operational behavior recognition and material flow tracking.

[0045] The operation behavior recognition module is responsible for performing deep action semantic analysis on the video slices locked by the spatiotemporal semantic association module to extract the specific behavior sequence of the operator.

[0046] The operation behavior recognition module first calls the deep convolutional neural network model preloaded in the edge computing node to extract human key points for each frame of the video. The deep convolutional neural network model adopts a top-down detection approach, that is, it first uses a target detection network such as YOLOv5 or Faster R-CNN to locate the human body region in the image, and then uses a high-resolution network HRNet to extract the fine skeleton point coordinates for each region.

[0047] This system can identify and track 25 core skeletal points of the human body, including the top of the head, neck, shoulders, elbows, wrists, spinal nodes, hips, knees, and ankles. The two-dimensional image coordinates of these key points in consecutive frames constitute the raw data stream of human motion.

[0048] After obtaining the key point coordinates of each frame, the operation behavior recognition module constructs a three-dimensional skeleton motion model of the operator through coordinate transformation between consecutive frames. This system uses the coordinate difference of the same key point in adjacent frames to calculate the displacement vector, and further obtains the acceleration features by differentiating the displacement vector with respect to time. These kinematic features can effectively reflect the force and speed of the action, providing rich input information for subsequent action classification.

[0049] Considering the numerous dynamic disturbances in the kitchen environment, such as billowing fumes, spraying steam, and light and shadow changes caused by fluctuating stove flames, the operation behavior recognition module specifically introduces environmental interference removal logic. This logic first constructs a Gaussian mixture background model for each pixel, modeling the background pixels as a mixture of multiple Gaussian distributions, and continuously updates the model parameters to distinguish the foreground human body from dynamic disturbances such as background smoke and steam. Subsequently, before extracting the skeleton motion trajectory, this system uses a temporal difference filter to perform pixel-by-pixel noise reduction preprocessing on the original video frames. For example, it uses the adjacent frame difference method to extract the motion region, and then combines morphological opening operations to remove isolated noise points, thereby preserving the real human body motion signal to the maximum extent and eliminating the interference of environmental factors.

[0050] After noise reduction, the skeleton point sequence is organized into a feature vector. The feature vector is constructed by adding the three-dimensional coordinates (i.e., image coordinates) of 25 key points to the depth values ​​obtained by triangulation or monocular depth estimation, as well as the first derivative velocity and second derivative acceleration of these coordinates, and concatenating them into a feature vector at each time step.

[0051] The feature vectors are input into the operational classification model that integrates a Long Short-Term Memory (LSTM) network and a Graph Convolutional Neural Network (GCN). The LSTM is responsible for capturing the temporal dependencies of actions, while the GCN uses the natural graph structure of the human skeleton to model the spatial joint dependencies.

[0052] The graph is constructed by using 25 key points as nodes and setting up an adjacency matrix according to the physical connections of the human body, such as the spine connecting the head and shoulders, and the shoulders, elbows, and wrists. After being trained on a large number of standardized operation samples, this fusion model can accurately identify common atomic actions (i.e., the smallest operational units that constitute a process) in the catering kitchen, such as washing, cutting, adding ingredients, stir-frying, plating, covering, and wiping. Each action is assigned a category label and a corresponding time interval, forming a complete operation log.

[0053] The operation behavior recognition module also includes a fine-grained hand motion recognition unit. When the confidence level of the hand area in the human skeleton model is high, this unit will be triggered. The digital zoom function of the camera will be used to crop and enlarge the hand area, or the hand details will be improved through super-resolution image enhancement technology, thereby capturing micro-movements such as grasping, rotating, and pressing of the fingers.

[0054] The identified hand micro-movement sequences are then fed into a specially trained behavior discrimination model, which dynamically time-warped matches the continuous hand movements with preset standardized handwashing procedure templates and glove changing action templates.

[0055] The method for obtaining the glove-changing action template is to pre-collect videos of multiple employees performing handwashing and glove-changing actions in a standardized manner, extract key hand point sequences as standard templates, and if the matching degree is lower than the set threshold, it will be determined that the employee has not performed the handwashing procedure as required or has not changed gloves between handling raw and cooked food, and the corresponding violation will be recorded.

[0056] Through the above multi-level recognition process, the operation behavior recognition module transforms the original video slices into a structured sequence of operation actions, providing behavioral basis for the subsequent quantitative tracking of material consumption and the automatic evaluation of operation compliance.

[0057] Next, we will introduce the material flow tracking module, which is closely coupled with the operation behavior recognition module and is responsible for quantifying the material consumption behind each operation action.

[0058] When the operation behavior recognition module determines that the feeding action has started, the material flow tracking module immediately sends a trigger signal to the weight sensing submodule, causing it to enter high-frequency monitoring mode. In this mode, the weight sensing submodule continuously records the pressure sensor readings at its highest sampling frequency of 100 Hz until the operation behavior recognition module detects that the feeding action has ended.

[0059] This system determines the stable state by analyzing the recorded weight data. The specific determination method is as follows: select 20 consecutive sampling points before feeding begins, calculate their standard deviation. If the standard deviation is less than 0.3 grams, the weight is considered to be stable, and this stable value is recorded as the weight before feeding. Similarly, after feeding ends, select 20 consecutive sampling points, calculate the standard deviation, and if it is less than 0.3 grams, record the stable weight after feeding.

[0060] The measured amount of feed is defined as the absolute value of the difference between the stable weight before and after feeding.

[0061] In a single-material delivery scenario, the weight difference represents the consumption of that material. However, in a restaurant kitchen, chefs often add multiple materials at the same time, such as chopped green onions and minced ginger into the pot simultaneously. In this case, a simple weight change cannot distinguish the consumption of each material. To cope with such complex scenarios, the material flow tracking module introduces a pixel segmentation method based on a spatial attention mechanism.

[0062] The system first dynamically defines a benefit area in the video frame. This benefit area is automatically generated by combining the key point position of the chef's hand with the detection frame of the feeding container. The union of the rectangles circumscribed by the key points of the hand and the rectangles circumscribed by the detection frame of the container is taken as the benefit area, ensuring that the entire feeding action range is covered.

[0063] Then, a pre-trained deep learning semantic segmentation model is invoked. This model adopts an encoder-decoder structure and embeds a spatial attention module, such as a convolutional block attention module, in the decoding stage to enhance the attention to the texture and color features of different materials in the feeding area. The deep learning semantic segmentation model outputs a probability map of each pixel belonging to different material categories, such as scallion, ginger, and meat.

[0064] By performing threshold segmentation and connected component analysis on the probability graph, this system can identify the types of materials being delivered and the coverage areas of each type of material in the image.

[0065] Meanwhile, this system combines object detection algorithms to identify the specifications of the containers held by chefs. This object detection model, based on YOLOv5 or Faster R-CNN architecture, is pre-trained using a large number of images of common kitchen containers. It can detect container types such as seasoning spoons, side dish plates, and condiment bowls, and further distinguish different specifications; for example, a standard seasoning spoon has a capacity of 5 ml, a large seasoning spoon has a capacity of 15 ml, and a side dish plate has a capacity of 100 grams. After detecting the container type, the system queries the corresponding capacity value from its built-in container specification database.

[0066] To achieve high-confidence material consumption measurement, this system introduces a multimodal fusion algorithm to cross-validate pressure sensor data and visual recognition results. The visual part estimates the visual volume of various materials based on the material coverage area obtained from semantic segmentation, combined with monocular depth estimation or by using known container dimensions as a reference.

[0067] Specifically, this system uses depth estimation algorithms such as MiDaS to obtain the depth value of each pixel within the benefit area, combines camera intrinsic parameters to convert the pixel coordinates into a 3D point cloud, and then performs integration to calculate the volume of point clouds belonging to the same material category; or when the container size is known, the volume is estimated by the ratio between the pixels and the actual size.

[0068] Subsequently, the system queries the typical density of the corresponding material from the material density database based on the visual classification results, calculates the theoretical mass, and compares the theoretical mass with the actual mass change value measured by the pressure sensor. If the deviation of the ratio of the two from 1 is less than a preset threshold, such as 10%, the data is considered valid; otherwise, an anomaly marker is triggered and a review is prompted.

[0069] Through the above multimodal verification, this system can improve the confidence level of single material consumption measurement to the 1-gram level, ensuring the accuracy of subsequent material cost accounting.

[0070] Through the aforementioned mechanism, the material flow tracking module transforms each material feeding action into a quantifiable material consumption record. It not only distinguishes between various mixed materials but also eliminates abnormal data caused by sensor noise or visual misjudgment through cross-validation. This provides accurate material feeding data support for subsequent process compliance evaluation, enabling the system to comprehensively assess the standardization level of store operations from both operational behavior and material consumption dimensions.

[0071] As the core of the system's decision-making, the process compliance evaluation module integrates a standardized operational process database. This database stores detailed standard operating procedures for each dish, including the standard dosage of each material, the standard duration range of each process, and the standard sequence between processes, forming a benchmark for evaluating operational compliance.

[0072] During the evaluation process, the process compliance evaluation module first receives the actual operation log output from the operation behavior recognition module. The log records each atomic action identified in the order processing in chronological order, including the action category label and the start and end time of the action. This log is called the actual action sequence, denoted as sequence A.

[0073] Meanwhile, the process compliance evaluation module retrieves the standard operating procedures for the corresponding dish from the database, parses them into a standard action sequence, denoted as Sequence B. The standard action sequence is composed of the steps included in the dish in sequence, with each step corresponding to one or more atomic actions.

[0074] To quantify the deviation between the actual operation and the standard process, the process compliance evaluation module uses the dynamic time warping algorithm to calculate the minimum cumulative distance between sequence A and sequence B. Dynamic time warping finds a warped path that aligns the two sequences, minimizing the sum of the distances between corresponding elements on the path.

[0075] Sequence elements are action category labels. The distance between two categories is defined as: 0 for the same category and 1 for the different category. The smaller the cumulative distance, the more compliant the operation. The cumulative distance is normalized to a similarity score between 0 and 1. The calculation formula is similarity = 1 / (1 + cumulative distance). The higher the similarity score, the closer it is to the standard.

[0076] Meanwhile, the process compliance evaluation module obtains the actual amount of material fed into the order for each feeding from the material flow tracking module and calculates the material error rate. The material error rate is defined as the absolute value of the difference between the actual feeding amount and the standard feeding amount divided by the standard feeding amount, expressed as a percentage. For processes where the feeding amount is not specified in the standard, this indicator is not included in the calculation.

[0077] Considering that different processes have varying degrees of impact on food safety and product quality, the process compliance evaluation module introduces risk weight coefficients for various processes. For example, for processes that are directly related to food safety, such as oil temperature monitoring and the distinction between raw and cooked food, the risk weight coefficient is set between 0.8 and 1.0; for processes that mainly affect visual appeal, such as the aesthetics of plating, the risk weight coefficient is set at around 0.2.

[0078] The weighting coefficients can be dynamically adjusted according to the brand management strategy and are pre-configured for each process in the database.

[0079] The process compliance evaluation module evaluates each process step separately, and then combines the results into a total score for the order. For a single process step, four dimensions of indicators are considered: First, the dynamic time warping similarity between the measured action sequence of this process and the standard action template is calculated using the same method as above.

[0080] Second, the material delivery error rate involved in this process is the average of all material delivery error rates within this process.

[0081] Third, process completeness is defined as the proportion of the number of atomic actions actually performed in the process to the total number of standard atomic actions. The actual atomic actions are determined by comparing the measured action sequence with the corresponding action segment in the standard action template.

[0082] Fourth, operation time deviation refers to the degree of deviation between the actual time consumed in the operation and the standard time range. The actual time is obtained from the operation log, and the standard time range is read from the database. If the actual time is within the standard range, the deviation is 0; if it exceeds the range, the deviation is calculated by dividing the absolute value of the difference between the actual time and the upper or lower limit of the standard by the corresponding standard limit, and then multiplying by 100% to obtain the percentage.

[0083] Convert the above four indicators into scores from 0 to 100: the similarity score is directly multiplied by 100; the material error rate score is 100 × (1 - material error rate), and is 0 when the material error rate is greater than 100%; the process integrity score is 100 × process integrity; the operation time deviation score is 100 × (1 - operation time deviation), and is 0 when the deviation is greater than 100%.

[0084] Then, the four scores of the process are weighted and summed, with the weights assigned as similarity 0.3, material error rate 0.3, process integrity 0.2, and operation time deviation 0.2 to obtain the preliminary score of the process. The preliminary score is then multiplied by its risk weight coefficient to obtain the final score of the process.

[0085] Finally, the final scores of all processes involved in the order are summed up and then divided by the sum of the risk weight coefficients of these processes to obtain the overall process compliance score of the order. The score ranges from 0 to 100, with a higher score indicating that the operation is more in line with the standards.

[0086] If the score of a single order processing is lower than the system's preset qualified threshold, such as 80 points, an anomaly alarm will be triggered immediately, and the relevant information of the order will be pushed to the anomaly tracing and analysis module for in-depth tracing and handling.

[0087] Through the aforementioned evaluation mechanism, the process compliance evaluation module transforms complex kitchen operations into intuitive quantitative scores, providing managers with accurate compliance monitoring data. Furthermore, the scores and anomaly signals output by this module directly drive subsequent anomaly tracing and early warning processes, ensuring the effective execution of the regulatory closed loop.

[0088] Upon receiving a low-score signal from the process compliance evaluation module, the anomaly tracing and analysis module immediately initiates a deep tracing process. The module first retrieves the detailed information of the order from the semantic mapping matrix stored in the spatiotemporal semantic association module based on the order number corresponding to the low-score order. This information includes the store identifier, processing grid number (i.e., the specific stall area in the kitchen), operation timestamp, operator number, and specific violations, thereby accurately locating the spatiotemporal coordinates of the anomaly and the responsible personnel.

[0089] The anomaly tracing and analysis module includes a feedback execution unit, which is used to send early warning instructions to the store terminal. While completing the location, the feedback execution unit sends a JSON format early warning instruction to the wall-mounted tablet terminal configured in the store's kitchen via the MQTT protocol. After the kitchen application running on the terminal parses the instruction, it displays a red full-screen pop-up window and simultaneously plays a preset voice correction file, prompting the operator to immediately correct the current violation.

[0090] The system backend automatically extracts 5-second high-definition video clips before and after the anomaly from the video storage of the edge computing nodes, based on the timestamp of the anomaly occurrence. Simultaneously, it queries the time-series database of the weight perception submodule for weight data within that time period, generating a material weight curve. The system also retrieves the complete details of the order from the order integration submodule, selects the corresponding template from the report template library based on the violation type, and fills in order information, video clip links, weight curves, and other data to automatically generate a preliminary anomaly cause analysis report.

[0091] All the above materials were encrypted using the AES-256 symmetric encryption algorithm and uploaded to a cloud database via HTTPS for permanent storage, for use in subsequent auditing and accountability.

[0092] The anomaly tracing and analysis module also has a built-in trend prediction function to realize the transformation from passive response to proactive prevention. This function regularly collects all compliance score data of the store in the past 30 days and builds a time series dataset with hourly granularity. The system uses exponential smoothing or autoregressive integrated moving average model ARIMA to model and analyze the time series, and identify the patterns of violations in specific time periods such as weekday lunch peak, Friday evening market or specific stalls.

[0093] Taking exponential smoothing as an example, its prediction formula is:

[0094] in, This represents the predicted value at the next moment. The actual observed value at the current moment. The current predicted value is represented by α, which is a smoothing coefficient ranging from 0 to 1, controlling the weight of recent data on the prediction. The ARIMA model describes the temporal dependencies of the data through a combination of autoregressive terms, differencing order, and moving average terms, with specific parameters obtained by fitting historical data.

[0095] Based on the prediction results, this system can identify high-risk periods or stalls for violations in advance and automatically generate targeted personnel allocation suggestions or simplified operation plans. The plan is generated based on a preset rule base. For example, if it is predicted that the probability of violations will increase during the stir-frying time from 6 pm to 8 pm on Friday, suggestions such as "add skilled chefs during peak hours" or "prepare semi-finished products in advance" will be matched from the rule base, and the plan will be sent to the store manager's management backend before the violation may occur.

[0096] Through this mechanism, the anomaly tracing and analysis module transforms reactive correction after the fact into proactive prevention before the fact, effectively reducing the overall violation rate.

[0097] In summary, the anomaly tracing and analysis module constructs a complete closed-loop anomaly handling process through precise source tracing, real-time early warning, evidence preservation, and trend prediction. This not only ensures the timely correction of individual violations but also optimizes the standardized operation and management of stores from a holistic perspective.

[0098] This system adopts a distributed architecture that combines edge computing and cloud collaboration to achieve real-time processing of underlying data and in-depth global analysis at headquarters. Edge computing nodes are deployed on the store side. These nodes are typically industrial-grade servers or embedded devices that run the Linux operating system and install the core system software modules.

[0099] Edge computing nodes are responsible for receiving and processing raw data streams collected from the multi-dimensional data perception module in real time, including images, weights, environmental parameters, and order data. They also undertake the computationally demanding tasks of the image acquisition sub-module and the operation behavior recognition module. By performing localized human key point extraction and action classification on the video stream, they ensure that the end-to-end response latency from data acquisition to behavior recognition result output is controlled within 100 milliseconds, meeting the needs of real-time monitoring of the kitchen.

[0100] The cloud platform is deployed in a central data center or cloud service provider and is responsible for aggregating the de-identified regulatory data uploaded by chain stores across the country. The de-identification process is completed at edge computing nodes or during the upload process. For example, Gaussian blurring algorithm is used to blur faces and customer facial information in videos, and SHA-256 algorithm is used to hash employee numbers to protect personal privacy.

[0101] The cloud platform's compliance evaluation module for operational processes involves complex algorithms for cross-store big data analysis, such as dynamic time regularization calculations based on massive samples and dynamic adjustment of risk weights. It is also responsible for maintaining a globally unified standardized operational process database, which stores the standard operating procedures for each dish and allows headquarters managers to remotely update standard data through the management interface.

[0102] Based on data aggregated in the cloud, the platform generates a nationwide heat map of process compliance for the headquarters management. This heat map displays the distribution of compliance scores for each region and store in map form. The scores are based on the comprehensive score output by the process compliance evaluation module and are mapped to different colors according to preset score ranges. For example, green is above 90 points, yellow is between 80 and 89 points, and red is below 80 points. The color intensity intuitively reflects the severity of deviations from standardization execution. Headquarters managers can quickly locate high-risk areas through the heat map.

[0103] The regional management terminal has a drill-down function, which allows users to access specific stores through hyperlinks or data interfaces from the heat map to view the distribution of abnormal nodes within the store, such as stalls, time periods, or specific operators with high rates of violations, and retrieve relevant video clips, material curves, and abnormal reports to assist in remote supervision and decision-making.

[0104] By leveraging the collaborative division of labor between the edge and the cloud, we can ensure the rigid requirement of real-time response at the store level while also enabling in-depth global monitoring and data mining at the headquarters level, providing scalable technical support for the standardized operation of chain brands.

[0105] The system also includes an inventory closed-loop verification module, which is responsible for performing periodic comparison analysis between logical inventory and inventory based on actual consumption. At the end of each business day, it automatically calculates the logical inventory for that day, which is defined as the initial inventory for that day minus the total amount of standard materials corresponding to the sales terminal orders.

[0106] At the same time, the inventory closed-loop verification module summarizes the total amount of all measured delivery quantities for the day from the material flow tracking module, and calculates the total process loss, which is the difference between the total amount of measured delivery quantities and the total amount of standard materials.

[0107] Based on the above data, the system further calculates the actual remaining inventory, which is equal to the initial inventory of the day minus the total actual delivery volume of the day. Then, the actual remaining inventory is compared with the logical inventory. If the absolute value of the difference between the two exceeds the system's preset loss tolerance limit, such as exceeding the standard loss rate of 3%, the anomaly analysis process is triggered. At this time, the system calls the anomaly tracing analysis module to perform cluster analysis on all order records in the period whose process compliance score is lower than the preset qualified threshold.

[0108] The anomaly traceability analysis module first obtains the compliance scores and corresponding operation records of all orders within the period from the process compliance evaluation module, filters out orders with scores below the qualified threshold, such as 80 points, and constructs a numerical feature vector for each filtered record. The dimensions include the dish category using one-hot encoding, the operation period taking the number of hours of the order generation time and normalizing it, and the employee number using one-hot encoding. Then, these features are concatenated to form a feature vector.

[0109] Next, K-means or DBSCAN clustering algorithms are used to group these feature vectors. The parameters of the clustering algorithm are determined using conventional methods. For example, the K value of K-means is determined by the silhouette coefficient or elbow rule, and the neighborhood radius and minimum number of samples of DBSCAN are determined by debugging.

[0110] After clustering, statistical analysis is performed on each cluster to calculate indicators such as the average material error rate and average operation time deviation of orders within the cluster, and compare them with the overall average level. If the material error rate of a certain cluster is significantly higher, further analysis is conducted on the dominant dish category, operation time period or employee number in that cluster to identify the main causes of abnormal losses, such as over-dispensing of specific dishes, frequent operational errors during specific time periods or habitual deviations of specific employees.

[0111] In this way, the system can accurately pinpoint the core risk points of cost loss, providing data support for subsequent process optimization and personnel training.

[0112] In summary, the inventory closed-loop verification module uses the above mechanism to compare daily material consumption data with standards in a closed loop, which not only verifies the accuracy of material flow tracking, but also reveals potential operational problems.

[0113] To achieve continuous optimization and self-evolution of the system, this system integrates a standardized process self-learning module. This module regularly retrieves order compliance score data from stores across the country from the cloud database, selects outstanding operation samples with scores in the top 5% across the entire network, and extracts corresponding video clips and operation logs as learning materials.

[0114] The standardized process self-learning module uses a generative adversarial network to extract features from the action rhythm, feeding sequence and heat control rules in the samples. The generative adversarial network includes a generator and a discriminator. Through adversarial training, the generator learns to represent the temporal and spatial patterns of excellent operations.

[0115] After training, feature vectors are extracted from the intermediate layer of the generator. After online review and confirmation by culinary experts at headquarters, they are synchronized to the standardized operation process database with one click. This is used to update the standard operating procedures for the corresponding dishes, ensuring that the level of standardization can be continuously improved as front-line employees accumulate experience.

[0116] In terms of in-depth processing of material flow tracking, when faced with complex scenarios involving mixed feeding of multiple materials, this system adopts a pixel segmentation method based on spatial attention mechanism. It calls a pre-trained semantic segmentation model, which adopts an encoder-decoder structure and embeds a spatial attention module in the decoding stage. It takes the dynamically defined interest region in the video frame as input and outputs a probability map of each pixel belonging to different material categories.

[0117] By using threshold segmentation and connected component analysis to obtain the coverage areas of various materials, and then combining the step-like weight change data recorded by the pressure sensor, the total weight change is allocated according to the pixel area ratio of the coverage areas of various materials, thereby accurately decomposing the consumption of each material.

[0118] The process compliance evaluation module considers environmental adaptability during the scoring process. This system acquires environmental parameters in real time through temperature and humidity sensors. When the ambient temperature exceeds 35 degrees Celsius, the compliance evaluation module automatically adjusts the evaluation criteria according to the preset compensation algorithm.

[0119] The compensation algorithm is based on a temperature-duration correction function fitted from experimental data. For example, the standard duration of the heating process is shortened by 1.5% for every 1 degree Celsius increase in temperature, while the time threshold for temperature control is relaxed.

[0120] This system configures a reference time and temperature compensation coefficient for the heating process of each dish. The difference between the real-time temperature and the reference temperature of 25 degrees Celsius is multiplied by the compensation coefficient to obtain the time adjustment amount, dynamically correcting the standard time range and making the supervision more engineering-rational.

[0121] In terms of robust design for operation behavior recognition, this system adopts multi-view geometric consistency verification to address the common occlusion problem in the kitchen. By fusing data from at least two image acquisition sub-modules deployed at different angles, the system reconstructs the key point coordinates of the occluded parts in three-dimensional space using epipolar constraint relationships.

[0122] The intrinsic and extrinsic parameters of each camera are obtained through camera calibration. If the same human key point is detected in two viewpoints, its three-dimensional coordinates are calculated by direct linear transformation or triangulation. When a viewpoint is missing due to occlusion, the two-dimensional coordinates of the occluded viewpoint are inferred from the reconstructed three-dimensional model to complete the skeleton model and ensure the integrity of motion capture under intensive operation.

[0123] The process compliance evaluation module also introduces a dynamic learning rate feedback mechanism, which can automatically identify the operational maturity of stores. For newly opened stores, the system automatically lowers the judgment threshold of the compliance score during the first 30-day operation period, for example, from 80 points to 70 points, allowing for higher material placement error rates, such as within 20%, without deducting points, and pushes more operation guidance with demonstration videos in the feedback execution unit.

[0124] As operating time increases and employee proficiency improves, the system will increase the threshold by 2 points and decrease the error tolerance by 2% every 30 days according to a linear decreasing strategy, until the standardized red line requirements set by the brand are met.

[0125] When generating the final regulatory report, the anomaly tracing and analysis module will proactively access customer review data for related orders. By connecting with third-party food delivery platforms or the store's own membership system, it will obtain customer ratings for orders in terms of taste, food preparation speed, hygiene, and other dimensions.

[0126] The system pairs the process compliance score sequence of the same employee over a period of time with the corresponding customer taste score sequence and calculates the Pearson correlation coefficient. When the correlation coefficient is close to 0 or negative, it prompts the headquarters management to re-examine whether there are design flaws in the standardized operating procedures, providing full-link data support for business decisions from the execution end to the consumer end.

[0127] This system also supports seamless integration with the cold chain logistics monitoring system. The material flow tracking module reads the electronic tags on the material packaging through RFID readers or QR code scanners during the material warehousing process, obtains the historical temperature control data throughout the transportation process, and stores it in the database as the initial quality attributes of the batch of materials.

[0128] In subsequent processing stages, the process compliance evaluation module dynamically adjusts regulatory standards based on the initial quality status of the materials. For example, if refrigerated meat experiences short-term temperature fluctuations during transportation, the system automatically calculates the extended cooking time based on a preset risk coefficient and prompts the operator to monitor and execute the process, ensuring food safety compliance.

[0129] In terms of data security and access control, this system provides multi-level access control. The headquarters management end has the highest authority and can monitor all data in real time and adjust global standardized process parameters. The regional management end is responsible for anomaly coordination and personnel performance evaluation within its jurisdiction. The store execution end focuses on receiving real-time alerts and implementing rectification.

[0130] All data transmissions are encrypted using the AES-256 symmetric encryption algorithm. The keys are distributed and updated regularly by the headquarters key management system to ensure the security of core operational data.

[0131] This embodiment transforms complex kitchen operations into quantifiable, traceable, and evaluable digital assets through the collaborative work of the aforementioned modules. This improves the management efficiency of chain brands, reduces material waste and food safety risks, and empowers enterprises with the intelligent drive to continuously optimize their operational processes through self-learning and trend prediction functions.

[0132] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0133] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment includes only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A smart monitoring and analysis system for standardized operation processes of restaurant chain stores, characterized in that, include: The multi-dimensional data sensing module is used to collect multi-dimensional raw data related to store operations. The multi-dimensional raw data includes at least video streams collected by network cameras, order data generated by the sales terminal system, and material weight data collected by the pressure sensor array. The spatiotemporal semantic association module is used to synchronize the time deviation of network cameras, sales terminal systems and pressure sensor arrays within 10 milliseconds based on the IEEE 1588 precision time protocol, generate timestamps based on orders in the order data, and retrieve and extract video slices of the corresponding time window from the video stream. The operation behavior recognition module is used to extract human key points from video slices, construct a human skeleton motion model, and identify the atomic action sequence that constitutes the kitchen operation process based on the human skeleton motion model. The atomic actions include at least feeding, stir-frying and plating. The material flow tracking module is used to respond to the feeding action identified by the operation behavior recognition module, trigger the pressure sensor array to record the weight change at a sampling frequency of not less than 100 Hz, and calculate the actual feeding amount based on the weight change; The process compliance evaluation module is used to obtain atomic action sequences and actual delivery volumes, compare them with preset standardized operating procedures, and generate a quantitative score to characterize the compliance of store operation processes.

2. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 1, characterized in that, The spatiotemporal semantic association module is also used for: Based on Zhang's calibration method and PnP algorithm, the network camera is calibrated to obtain its intrinsic and extrinsic parameters in order to establish a mapping from visual space to physical space. Based on the kitchen layout, the physical space is divided into multiple functionally independent processing grids, and each processing grid is associated with a network camera covering the area; When a new order is captured, the system reads a preset time window from the standardized operation process database based on the order's generation timestamp and the corresponding dishes. It then extracts a sequence of consecutive video frames within that time window from the video stream of the network camera associated with the target processing grid, forming a video slice.

3. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 2, characterized in that, The spatiotemporal semantic association module is also used for: To identify the operator executing the order in the video slice, specifically: locate the image-based identification code deployed on the operator using a target detection algorithm, and decode and extract the unique employee number; Using the order number as a unique index, order data, operator numbers, processing grid numbers, and video slice storage paths are integrated into a structured semantic mapping matrix to achieve precise association between orders, personnel, and surveillance images.

4. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 1, characterized in that, The operation behavior recognition module is also used for: A top-down detection approach is adopted. First, the human body region is located through the target detection network, and then the high-resolution network HRNet is used to extract the two-dimensional image coordinates of 25 core skeleton points of the human body. The coordinates, velocities, and accelerations of the 25 core skeleton points in consecutive frames are concatenated into a feature vector, which is then input into an operation classification model that integrates a Long Short-Term Memory Network (LSTM) and a Graph Convolutional Neural Network (GCN) to identify atomic action sequences and their corresponding time intervals. Among them, the Graph Convolutional Neural Network (GCN) uses a graph structure constructed with 25 core skeleton points as nodes and an adjacency matrix set according to the physical connections of the human body to model the spatial correlation of joints.

5. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 1, characterized in that, The material flow tracking module is used to calculate the actual delivery quantity, specifically including: Before the feeding action begins, multiple sampling points of the pressure sensor array are continuously collected. When the standard deviation is less than 0.3 grams, the stable value is recorded as the weight before feeding. After the feeding action is completed, the stable weight after feeding is recorded in the same way. The absolute value of the difference between the stable weight before feeding and the stable weight after feeding is taken as the actual feeding amount.

6. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 5, characterized in that, The material flow tracking module is also used to handle scenarios involving the mixed delivery of multiple materials, specifically including: When the feeding action is detected, the profit area is dynamically delineated in the video frame by combining the key points of the chef's hand with the detection frame of the feeding container. The deep learning semantic segmentation model with embedded spatial attention module is invoked to process the interest region and output a probability map of each pixel belonging to different material categories in order to identify the types of materials that are delivered and their coverage areas in the image. A multimodal fusion algorithm is introduced to estimate the visual volume of various materials based on the material coverage area, and to calculate the theoretical mass by combining the material density database. The theoretical mass is then cross-validated with the actual mass change value measured by the pressure sensor array. If the deviation between the two exceeds a preset threshold, an anomaly flag is triggered.

7. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 1, characterized in that, The process compliance evaluation module is used to generate quantitative scores, specifically including: The minimum cumulative distance between the measured action sequence and the standard action sequence is calculated using the dynamic time warping algorithm and then normalized into an action similarity score. Calculate the material error rate between the measured dosage and the standard dosage; Risk weight coefficients are assigned to different processes, and the similarity scores of actions, material error rates, process completeness, and operation time deviations are combined and weighted to obtain the score for each process. The scores are then adjusted according to the risk weight coefficients, and finally aggregated to generate the process compliance score for the entire order.

8. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 1, characterized in that, It also includes an anomaly tracing and analysis module, used for: When the process compliance score is lower than the preset qualified threshold, the spatiotemporal semantic association module is used to retrieve the store identifier, processing grid, operation timestamp and operator number of the abnormality from the corresponding order number; The system sends an alert command to the store terminal via the MQTT protocol, triggering a red full-screen pop-up window and a voice correction broadcast. It automatically extracts high-definition video clips before and after the time of the anomaly, queries the material weight curve for the corresponding time period, and generates an anomaly cause analysis report.

9. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 1, characterized in that, It also includes an inventory closed-loop verification module, used for: At the end of each operating cycle, calculate the logical inventory, which is defined as the initial inventory minus the total amount of standard materials corresponding to the sales order; The material flow tracking module summarizes the total measured delivery volume and calculates the measured remaining inventory, which is defined as the initial inventory minus the total measured delivery volume. The measured remaining inventory is compared with the logical inventory. If the absolute value of the difference between the two exceeds the preset loss tolerance limit, the anomaly analysis process is triggered. Cluster analysis is performed on the operation time, dish category and employee number of all low-scoring orders in the period to identify the core cause of the loss anomaly.

10. The intelligent monitoring and analysis system for standardized operation processes of catering chain stores according to claim 1, characterized in that, It also includes a standardized process self-learning module, used for: Regularly retrieve high-scoring orders from each store from the cloud database as excellent operational samples; Generative adversarial networks are used to extract features from the action rhythm, feeding sequence and heat control patterns in excellent operation samples; After the extracted feature vectors are reviewed by experts, they are synchronized to the standardized operation process database to dynamically update the standardized operating procedures.

Images