Computer vision-based kitchen hygiene safety intelligent monitoring system and method

Abstract

This invention relates to the field of computer vision and intelligent monitoring technology, and particularly to a computer vision-based intelligent monitoring system and method for kitchen hygiene and safety. The system adopts an architecture of "image acquisition module - server intelligent analysis subsystem - management and alarm terminal." The server intelligent analysis subsystem includes modules for image preprocessing, static basis update, target candidate region extraction, multi-path detection splitting, parallel deep learning recognition, time-series decision-making and alarm, and data storage and management. The method includes image acquisition and preprocessing, static basis update and target candidate region extraction, multi-path detection splitting, parallel deep learning fine-grained recognition, time-series decision-making and alarm, and data storage and management. This invention is adapted to the special scenarios of kitchens, offering high detection efficiency, strong anti-interference capabilities, and comprehensive functions. It can achieve accurate detection of harmful organisms and personnel wearing prohibited clothing, real-time alarms, and data traceability, forming a complete management closed loop, and is applicable to safety management in various kitchen scenarios.

Description

Technical Field

[0001] This invention relates to the field of computer vision and intelligent monitoring technology, and in particular to a computer vision-based intelligent monitoring system and method for kitchen hygiene and safety. Background Technology

[0002] As the core area for food processing and storage, the kitchen's hygiene, safety, and operational standards are directly related to food safety and are a crucial link in protecting public health. Wearing chef's hats and masks is a key physical barrier to prevent human contaminants (such as hair, dander, and droplets) from entering food, playing an irreplaceable role in preventing cross-contamination, ensuring food hygiene and safety, and complying with hygiene regulations. Currently, monitoring of kitchen hygiene and personnel operational standards mainly relies on manual inspections, which suffers from many problems such as low efficiency, high missed detection rates, poor real-time performance, high labor costs, and the inability to achieve data traceability, making it difficult to meet the management needs of large-scale, standardized kitchens.

[0003] With the development of computer vision technology, various intelligent monitoring systems are gradually being applied to kitchen scenarios, but existing technologies still have the following core defects.

[0004] 1. Poor scene adaptability: Most existing monitoring models are designed for general scenarios and have not been optimized for the special environment of the kitchen (high temperature, steam, oil fumes, large changes in light, and many clutters). They are easily affected by environmental interference, resulting in high false alarm and false alarm rates (such as misjudging steam or moving tableware as harmful organisms, or missing small harmful organisms due to oil fumes). 2. Low detection efficiency: Most systems use a single detection model to process all scene targets without classifying and sorting the targets. They still perform complex calculations for irrelevant interference objects, which consumes a lot of server resources, resulting in high detection latency and failing to meet the needs of real-time kitchen monitoring. 3. Limited functionality: Existing systems often only perform a single function (such as detecting only harmful organisms or only detecting personnel clothing), which cannot simultaneously cover the two core monitoring needs of kitchen hygiene and safety and personnel operation standards. This requires the deployment of multiple systems, increasing deployment costs and maintenance difficulties. 4. Weak anti-interference capability: There are many static interference objects (such as food, kitchen utensils, and cleaning tools) and dynamic interference objects (such as moving plates, fluttering rags, and steam flow) in the kitchen. The existing system lacks an effective interference filtering mechanism, which easily triggers invalid alarms and affects the work efficiency of management personnel. 5. Low data value utilization: It only realizes the real-time alarm function, without structured storage and analysis of the detection data, and cannot provide data support for kitchen hygiene management, personnel training and hazard rectification, making it difficult to form a complete management closed loop of "monitoring-alarm-analysis-rectification"; 6. Insufficient substrate adaptability: The kitchen environment is characterized by slow changes in lighting and slight movement of static objects. The static substrate reference images of existing systems are mostly fixed and unchanging, which can easily lead to a large number of false foregrounds in foreground segmentation, further exacerbating the false alarm problem.

[0005] In view of the shortcomings of the existing technologies, there is an urgent need for an intelligent monitoring system and method that is adapted to the special scenario of the kitchen, has high detection efficiency, strong anti-interference ability, comprehensive functions, and can realize data traceability and closed-loop management, so as to solve the pain points of monitoring kitchen hygiene and safety and personnel operation standards. Summary of the Invention

[0006] To address the problems of poor scenario adaptability, low detection efficiency, limited functionality, weak anti-interference ability, low data value utilization, and insufficient substrate adaptability in existing kitchen monitoring technologies, this invention provides a computer vision-based intelligent monitoring system and method for kitchen hygiene and safety. This system enables real-time and accurate detection of harmful organism intrusion and personnel wearing inappropriate clothing, reducing false alarm and missed alarm rates, improving monitoring efficiency, and simultaneously achieving structured storage and analysis of detection data. This provides data support for kitchen management, forming a complete management loop and adapting to the dynamic changes of various kitchen scenarios.

[0007] To achieve the objectives of this invention, the technical solution adopted is: a computer vision-based intelligent monitoring method for kitchen hygiene and safety, comprising the following steps.

[0008] S1. Acquire high-quality images of the kitchen and preprocess them to remove image blur and noise caused by kitchen fumes and steam, and output the preprocessed static base reference image of the kitchen environment.

[0009] S2. Based on the moving average model, the static base reference image of the kitchen environment is updated online adaptively, and the pixels of the background area are updated to adapt to the dynamic changes of the kitchen environment and avoid the false foreground problem caused by the fixed base.

[0010] S3. Extract candidate regions of dynamic targets from the image processed in step S2 through background subtraction, local adaptive threshold binarization, morphological optimization, and connected component analysis, and eliminate invalid regions to reduce the computational load of subsequent models.

[0011] S4. Extract the four-dimensional core features of the candidate region, including area ratio, aspect ratio, motion speed and trajectory irregularity, and guide the candidate target to S5 through a preset diversion rule function.

[0012] S5. Two production lines, the pest identification line and the personnel wearing identification line, are executed in parallel to identify the types of pests and the wearing status of personnel's work hats and masks, respectively.

[0013] S6. Perform time-series analysis and confidence accumulation on the recognition results, filter out single-frame misjudgments, and trigger real-time alarms on multiple terminals after determining valid targets to ensure the accuracy and reliability of alarms.

[0014] As an optimized solution of the present invention, in step S2, during the initialization, N consecutive frames of kitchen scene images are acquired during non-operational periods in the kitchen, the median pixel value of corresponding pixels in each frame is calculated, and an initial static base reference image is generated. B 0, the formula is: .

[0015] Where (x,y) are the image pixel coordinates. I 1~ I N This refers to N consecutive frames of images acquired during the initialization phase, where median represents the median value operation.

[0016] During normal operation, each frame of the preprocessed image... I t Determine if pixel (x, y) is a background pixel. If it is, update the static basis according to the preset learning rate. The formula is: .

[0017] in: B t+1 (x,y) represents the base pixel value of the next frame. B t (x,y) represents the base pixel values ​​of the current frame, and α is the learning rate. I t (x,y) represents the pixel values ​​of the currently acquired image. D t (x,y) is the identifier for the foreground segmentation result.

[0018] As an optimized solution of the present invention, in step S3, candidate regions of dynamic targets are extracted through background subtraction, local adaptive threshold binarization, morphological optimization, and connected component analysis, specifically including: S31. Calculate image difference: For and The difference between corresponding pixels is calculated to obtain the difference image. The formula is: This reflects the pixel grayscale difference between the current image and the base image.

[0019] S32. Local Adaptive Threshold Binarization: A local adaptive thresholding algorithm is used. A 5×5 pixel window centered at (x,y) is used to calculate the mean and standard deviation of the difference image within the window, determining the adaptive threshold. This segments the difference image into foreground and background, obtaining the foreground mask. Mt The formula is: .

[0020] in, The mean pixel value of a local window in the difference image. Let k be the standard deviation of pixels in the local window of the difference image, and k be the threshold adjustment coefficient. The pixel grayscale difference value at coordinates (x, y).

[0021] S33, Foreground Mask Optimization: [This section appears to be incomplete and requires further context.] M t Morphological processing is performed, first opening (erosion followed by dilation) to eliminate small noise points; then closing (dilation followed by erosion) to fill holes in the target area, resulting in an optimized foreground mask. .

[0022] S34. Connectivity Analysis: [This section discusses] Connectivity analysis is performed to identify continuous foreground pixel regions in the image. A bounding rectangle is defined for each connected region, which serves as a candidate target region, denoted as . R i = ( x i ,y i ,w i ,h i ), i For candidate target number, ( x i ,y i ) No. i The top-left pixel coordinates of the bounding rectangle of the candidate target region w i ,h i For the first i The width and height of the bounding rectangle of each candidate target region.

[0023] As an optimized solution of the present invention, in step S4, the first... i Candidate target areas R i = ( x i ,y i ,w i ,h i Extract the four-dimensional core features that distinguish target types in a kitchen scene, and construct the unique feature vector F corresponding to the candidate region. iThe eigenvector is defined as: Based on candidate target regions R i Corresponding feature vector F i Construct a branching rule function for multi-conditional branching judgments. By applying thresholds to the four-dimensional features, precise segmentation of candidate targets is achieved. The formula is as follows: .

[0024] in: For the target area percentage, For the target aspect ratio, For the target speed of motion, For the irregularity of the target trajectory, T a1 This represents the lower limit of the area to be tested by personnel. T a2 This represents the upper limit of the area to be tested by personnel. T ar1 The lower limit of the aspect ratio for personnel inspection. T ar2 The upper limit of the aspect ratio for personnel inspection. T s The threshold for motion speed, T b1 This represents the lower limit of the area to be monitored for harmful organisms. T b2 This represents the upper limit of the area to be monitored for harmful organisms. T irr This is the threshold for trajectory irregularity.

[0025] As an optimized embodiment of the present invention, in step S5, the pest precise identification pipeline includes the following.

[0026] S5a-1. Perform image preprocessing on the candidate regions that are determined to be "harmful organism detection" after the diversion.

[0027] S5a-2. Input the preprocessed candidate region image into the lightweight YOLOv5n optimization model. The model outputs the target bounding box, category confidence, and target confidence.

[0028] S5a-3, Confidence Filtering: Set target confidence threshold and category confidence threshold to filter prediction results with confidence levels below the threshold and reduce misjudgments.

[0029] S5a-4, Non-maximum suppression: The NMS algorithm is used to remove prediction boxes with excessive overlap and retain the best prediction results.

[0030] S5a-5, Output identification results: Record the type of pest, target coordinates, and confidence level. If no valid pest is identified, output "No pest".

[0031] As an optimization scheme of the present invention, the total loss function of the lightweight YOLOv5n optimization model is as follows.

[0032] ;in: This represents the overall loss value of the pest detection model. The weights are the bounding box loss. For bounding box loss, The weights for the target confidence loss. For target confidence loss, The weights for the class loss, This is the category loss.

[0033] As an optimized solution of the present invention, in step S5, the personnel wearing identification includes the following.

[0034] S5b-1. Perform image preprocessing on the candidate regions that are determined to be "personnel detection" after the splitting.

[0035] S5b-2: Input a cropped version of the YOLOv5s model, output the bounding box of the person and the confidence score of the target. Set the confidence score threshold, filter invalid predictions, and remove overlapping boxes using the NMS algorithm to obtain the target area of ​​the person.

[0036] S5b-3: Extract the head region from the personnel target area, input it into a lightweight CNN model, and output the work hat wearing status, mask wearing status, and corresponding confidence scores.

[0037] S5b-4. Violation Judgment: If a work hat is not worn, a mask is not worn, or is worn improperly, it is judged as a violation of the wearing regulations.

[0038] S5b-5, Output recognition results: Record the personnel location, work hat wearing status, mask wearing status, and confidence level; if no valid personnel are identified, output "No personnel".

[0039] As an optimization scheme of the present invention, the total loss function of the lightweight CNN model is: ;in: The overall loss value of the personnel wearing status recognition model. The weighting of the loss due to wearing a work hat. Loss of wearing a work hat Weighting of the loss of consciousness due to mask-wearing status. Loss due to mask-wearing status.

[0040] As an optimized solution of the present invention, step S6 includes: S6-1, temporal caching: caching the recognition results of 10 consecutive frames, establishing a temporal queue, and recording the target type, location, confidence level and timestamp of each frame.

[0041] S6-2. The rules for determining effective targets include: 1. Determining effective harmful organisms: If the same type of harmful organism is identified in 3 or more consecutive frames, and the target location overlap (IOU) is ≥0.5 and the average confidence level is ≥0.6, it is determined to be an effective harmful organism invasion.

[0042] 2. Valid determination of personnel wearing violations: If the same person is detected wearing violations in 2 or more consecutive frames, and the position overlap (IOU) of the personnel is ≥0.6 and the mean confidence level is ≥0.7, it is determined to be a valid violation.

[0043] 3. False detection filtering: If a target is detected in a single frame but the above consecutive frame condition is not met, it is judged as a single frame false detection, no alarm is triggered, and only the log is recorded.

[0044] 4. Alarm Trigger: When a valid harmful organism intrusion or a valid person wearing prohibited clothing is detected, an alarm mechanism is triggered to generate alarm information. The alarm information includes: alarm type, alarm time, alarm location, target details, and on-site image frames.

[0045] 5. Multi-terminal alarm push: Web management backend: pop-up window prompts alarm information, synchronously displays on-site images and target details, and marks the alarm status as unprocessed.

[0046] 6. Mobile terminals: Alarm notifications are pushed via mini-programs / apps, accompanied by on-site images, allowing managers to click to view details and confirm processing.

[0047] 7. Alarm Handling Tracking: After an administrator confirms the handling of an alarm on any terminal, the system records the handler, handling time, and handling result, and updates the alarm status to "handled", forming a closed loop for alarm handling.

[0048] 8. Output results: valid alarm information and alarm processing records. Identification results that do not meet the valid judgment conditions are only logged and do not trigger alarms.

[0049] To achieve the objectives of this invention, the technical solution adopted is: a computer vision-based intelligent monitoring system for kitchen hygiene and safety, comprising an image acquisition unit, a server intelligent analysis subsystem, and a management alarm terminal. The server intelligent analysis subsystem includes an image preprocessing module, a static basis update module, a target candidate region extraction module, a multi-path detection and splitting module, a parallel deep learning recognition module, a time-series decision-making and alarm module, and a data storage and management module. The image acquisition unit acquires real-time images of the kitchen, providing high-quality image data for subsequent intelligent analysis. It supports the deployment of multiple cameras, achieving comprehensive coverage of the kitchen without blind spots, and adapting to the layout requirements of kitchens of different sizes. The management alarm terminal receives alarm information pushed by the server intelligent analysis subsystem, displays detection results and on-site images, and enables alarm confirmation, processing, and tracing. It also supports parameter configuration and data statistical analysis, providing a convenient management entry point.

[0050] Image preprocessing module: preprocesses the high-quality images of the acquired kitchen scene, removes image blur and noise caused by kitchen fumes and steam, and outputs a static base reference image of the preprocessed kitchen environment.

[0051] Static Basis Update Module: Based on the moving average model, the static basis reference image of the kitchen environment is updated online adaptively, and the pixels in the background area are updated to adapt to the dynamic changes of the kitchen environment and avoid the false foreground problem caused by the fixed basis.

[0052] The target candidate region extraction module extracts candidate regions of dynamic targets from the image processed by the static basis update module through background subtraction, local adaptive threshold binarization, morphological optimization, and connected component analysis, eliminating invalid regions and reducing the computational load of subsequent models.

[0053] Multi-path detection and diversion module: Extracts four-dimensional core features of candidate regions, including area ratio, aspect ratio, motion speed and trajectory irregularity, and guides candidate targets to parallel deep learning recognition module through preset diversion rule functions.

[0054] Parallel deep learning recognition module: including a pest accurate recognition pipeline and a personnel wear recognition pipeline. The two pipelines run in parallel, respectively realizing the recognition of pest species and the recognition of personnel wearing work hats and masks.

[0055] The timing decision and alarm module performs timing analysis and confidence accumulation on the identification results, filters out single-frame misjudgments, and triggers real-time alarms on multiple terminals after determining a valid target, ensuring the accuracy and reliability of the alarms.

[0056] Data storage and management module: It stores image data, detection results, alarm information, system configuration parameters, etc. in a structured manner, supports data query, statistics, and traceability, provides data support for kitchen management, and also supports online adjustment of system parameters.

[0057] This invention has positive technical effects.

[0058] 1) Strong scene adaptability: All deep learning models are customized and optimized for special scenarios in kitchens with high temperature, steam, oil fumes, large changes in lighting, and many small targets. At the same time, a static basis adaptive update mechanism is adopted to adapt to the dynamic changes in the kitchen environment, which greatly reduces the false alarm and false negative rates and solves the problem of poor scene adaptability of existing technologies.

[0059] 2) High detection efficiency: The "candidate region extraction + multi-path detection diversion" mechanism is adopted to guide candidate targets to a dedicated detection pipeline, avoiding invalid calculations of general models. At the same time, a lightweight model architecture is adopted to achieve parallel inference, with a detection latency of ≤0.5 seconds, which meets the real-time monitoring needs of the kitchen and solves the problem of low detection efficiency in existing technologies.

[0060] 3) Comprehensive functions: It simultaneously realizes two core functions: pest intrusion detection and personnel wearing compliance identification. It eliminates the need to deploy multiple systems, reducing deployment costs and maintenance difficulties, and solving the problem of single function in existing technologies.

[0061] 4) Strong anti-interference capability: Through multiple mechanisms such as image preprocessing, static base adaptive update, whitelist interference filtering, and time-series decision misjudgment filtering, it effectively filters static and dynamic interference in the kitchen, avoids invalid alarms, improves the work efficiency of managers, and solves the problem of weak anti-interference capability of existing technologies.

[0062] 5) High data value utilization rate: It realizes the structured storage and statistical analysis of detection data, alarm data, and image data, providing data support for kitchen hygiene management, personnel training, and hazard rectification, forming a complete management closed loop of "monitoring-alarm-analysis-rectification", and solving the problem of low data value utilization rate of existing technologies.

[0063] 6) Flexible deployment and convenient operation and maintenance: Supports multi-camera deployment to adapt to kitchens of different sizes (restaurants, canteens, central kitchens); supports local / private cloud / public cloud deployment, small kitchens can use lightweight servers, and large kitchens can be flexibly expanded and shrunk; supports multi-terminal management, and managers can handle alarms and adjust parameters anytime and anywhere, with low operation and maintenance costs.

[0064] 7) High recognition accuracy: Model optimization and data augmentation are performed for small targets in the kitchen (such as cockroaches) and obstructed scenarios (such as oil fume obstruction), with a pest recognition accuracy of ≥95% and a personnel wearing violation recognition accuracy of ≥98%, meeting the high accuracy requirements of kitchen safety management. Attached Figure Description

[0065] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0066] Figure 1 This is an overall architecture diagram of the intelligent monitoring system for kitchen hygiene and safety based on computer vision, which is based on the present invention.

[0067] Figure 2 This is a block diagram of the server intelligent analysis subsystem of the present invention.

[0068] Figure 3 This is a flowchart of the intelligent monitoring method for kitchen hygiene and safety based on computer vision, according to the present invention.

[0069] Figure 4 This is a schematic diagram of the static basis adaptive update process of the present invention.

[0070] Figure 5 This is a logical schematic diagram of the multi-path detection and diversion method of the present invention.

[0071] Figure 6 This is a schematic diagram of the parallel deep learning recognition pipeline of the present invention.

[0072] Figure 7 This is a schematic diagram of the deployment of the central kitchen monitoring system in an embodiment of the present invention.

[0073] Figure 8 This is a bar chart comparing the kitchen target detection accuracy of the present invention with that of existing technologies.

[0074] Figure 9 This is a line graph comparing the detection delay of the kitchen monitoring system of the present invention with that of the prior art. Detailed Implementation

[0075] like Figure 1 and 2 As shown, this invention discloses a computer vision-based intelligent monitoring system for kitchen hygiene and safety. It adopts a three-layer architecture of "image acquisition unit - server intelligent analysis subsystem - management alarm terminal". Each module works in concert to realize the fully automated processing from image acquisition, intelligent analysis, accurate recognition to real-time alarm and data storage. The structure is simple and flexible in deployment. The specific structure is as follows.

[0076] 1.1 Image acquisition unit.

[0077] (1) Core functions: Real-time acquisition of kitchen images to provide high-quality image data for subsequent intelligent analysis; support for multi-camera deployment to achieve no blind spots in the kitchen and adapt to the layout requirements of kitchens of different sizes.

[0078] (2) Hardware composition: It consists of several high-definition network cameras. The cameras use wide-angle lenses (viewing angle ≥ 120°), support H.265 encoding, and have the ability to resist oil fumes, steam and low light (minimum light ≤ 0.01 lux). They can adapt to special scenarios with large changes in kitchen lighting and a lot of oil fumes and steam. They support remote parameter configuration (such as acquisition frame rate, resolution and exposure) without the need for on-site debugging.

[0079] (3) Deployment requirements: Based on the kitchen layout, deploy cameras in key areas such as the food storage area, processing area, stove area, dishwashing room, and entrance / exit. The installation height is 2.5-3.5 meters to ensure coverage of all monitoring areas without blind spots. The cameras are directly connected to the server via network cable, supporting real-time image transmission without intermediate edge processing equipment, reducing data transmission delay and hardware overhead.

[0080] (4) Acquisition parameters: The default acquisition resolution is 1920×1080. The acquisition frame rate can be adjusted according to actual needs (1-3 seconds / frame) to balance real-time performance and server processing pressure. The acquired image format is JPG, which supports 80% compression rate transmission to reduce bandwidth usage and ensure smooth data transmission.

[0081] 1.2 Server Intelligent Analysis Subsystem.

[0082] (1) Core function: Receives on-site images transmitted by the image acquisition module, and completes image preprocessing, static basis update, target candidate region extraction, multi-path detection and diversion, fine-grained recognition, time-series decision-making and alarm triggering through a series of intelligent algorithms and deep learning models. At the same time, it realizes the structured storage of data and adaptive optimization of the model. It is the core processing unit of the system.

[0083] (2) Hardware configuration: Equipped with a high-performance CPU (Intel Xeon E5 and above), GPU (NVIDIA Tesla T4 and above, for accelerating model inference), 16GB or more of memory, and 1TB or more of SSD solid-state drive (for data storage and model loading).

[0084] (3) Software composition: (3.1) Image preprocessing module: Denoising, enhancement and normalization are performed on the acquired images to remove image blur and noise caused by kitchen fumes and steam, improve image quality and provide support for subsequent target detection; Specifically, Gaussian filtering is used for noise reduction and histogram equalization enhancement to ensure that image details are clear and distinguishable. (3.2) Static base update module: Based on the moving average model, the static base reference image of the kitchen environment is updated online adaptively. Only the pixels in the background area are updated, completely avoiding the interference of foreground targets (people, pests) and ensuring that the base image always matches the current kitchen environment, thus solving the problem of false foreground caused by changes in lighting and movement of static objects.

[0085] (3.3) Target candidate region extraction module: Through algorithms such as background subtraction, local adaptive threshold binarization, morphological optimization, and connected component analysis, candidate regions of dynamic targets are extracted from the preprocessed image, invalid regions are eliminated, the amount of subsequent model computation is reduced, and the detection efficiency is improved.

[0086] (3.4) Multi-path detection and diversion module: Extract the four-dimensional core features of the candidate region (area ratio, aspect ratio, movement speed, trajectory irregularity), and guide the candidate target to the corresponding dedicated detection pipeline (personnel detection / harmful organism detection) through the preset diversion rule function, or determine it as a static interference / ignore it, avoid the invalid calculation of the general model, and further improve the detection efficiency.

[0087] (3.5) Parallel deep learning recognition module: It includes a pipeline for accurate identification of harmful organisms and a pipeline for identification of personnel wearing clothing. The two pipelines are executed in parallel to identify the types of harmful organisms and the wearing status of personnel's work hats and masks, respectively. All models are customized and optimized for kitchen scenarios to improve recognition accuracy.

[0088] (3.6) Timing decision and alarm module: Performs timing analysis and confidence accumulation on the recognition results of continuous frames, filters out misjudgments in a single frame (such as misjudgments caused by steam or momentary obstruction), and triggers real-time alarms on multiple terminals after determining a valid target, ensuring the accuracy and reliability of the alarm.

[0089] (3.7) Data storage and management module: It stores image data, detection results, alarm information, system configuration parameters, etc. in a structured manner, supports data query, statistics and traceability, provides data support for kitchen management, and supports online adjustment of system parameters without restarting the system.

[0090] 1.3 Managing alarm terminals.

[0091] (1) Core functions: Receive alarm information pushed by the server intelligent analysis subsystem, display detection results and on-site images, realize alarm confirmation, processing and tracing, and support system parameter configuration and data statistical analysis, providing a convenient management entry for managers.

[0092] (2) Terminal type: Supports multi-terminal collaboration, including Web management backend and mobile terminal (phone / tablet) to meet management needs in different scenarios. The specific functions are as follows.

[0093] (2.1) Web management backend: Deployed on the computer, it supports functions such as viewing alarm information, querying historical data, generating statistical reports (statistics on pest activity and personnel violations), configuring system parameters, and managing camera status, providing managers with a comprehensive management portal.

[0094] (2.2) Mobile terminal: Implemented through DingTalk, WeChat mini program or dedicated APP, supporting real-time push of alarm information (SMS, push notification), on-site image viewing, alarm confirmation and processing, so that managers can receive and process alarms anytime and anywhere, and improve response efficiency.

[0095] 2. A computer vision-based intelligent monitoring method for kitchen hygiene and safety.

[0096] Based on the above system, such as Figure 3 As shown, this invention provides a computer vision-based intelligent monitoring method for kitchen hygiene and safety, including the following steps, to achieve fully automated processing from image acquisition to alarm output. The steps are clear and practical, as detailed below.

[0097] 2.1 Image acquisition and preprocessing stage.

[0098] (1) Core objective: To acquire high-quality images of the kitchen scene, remove environmental interference (oil fumes, steam, noise) through preprocessing, lay the foundation for subsequent target detection and recognition, and ensure detection accuracy.

[0099] (2) Execution process.

[0100] 1. Each camera in the image acquisition module acquires real-time images of the kitchen at a preset frame rate (1-3 seconds / frame). The acquired images have a resolution of 1920×1080 and are in JPG format.

[0101] 2. The camera transmits the captured images to the server's intelligent analysis subsystem in real time via network cable. During transmission, the images are compressed (80% compression rate) to reduce bandwidth usage, ensure smooth data transmission, and reduce transmission latency to ≤0.1 seconds.

[0102] (3) The image preprocessing module of the server intelligent analysis subsystem performs three steps of processing on the received image.

[0103] 1. Denoising: Gaussian filtering algorithm is used to remove Gaussian noise and salt-and-pepper noise (caused by oil fumes and steam) from the image, smooth the image texture, and avoid noise interference with target detection.

[0104] 2. Enhancement Processing: Histogram equalization algorithm is used to improve image contrast and solve the problems of dark images and blurred details caused by uneven lighting in the kitchen, ensuring that target features are clearly distinguishable.

[0105] 3. Normalization processing: Normalize the image pixel values ​​to the [0,1] interval to eliminate the model calculation deviation caused by pixel value differences, improve the model recognition accuracy, and facilitate unified model processing.

[0106] (4) Output result: The clear image after preprocessing, denoted as I t (t is the collection timestamp).

[0107] 2.2 Static basis update and target candidate region extraction stage.

[0108] (1) Core objective: Adaptively update the static base reference image of the kitchen environment, accurately extract dynamic target candidate regions, eliminate static interference and noise, reduce the computational load of subsequent models, and improve detection efficiency.

[0109] (2) Execution process: The server intelligent analysis subsystem executes static basis update, background difference, foreground segmentation, morphological optimization and connected component analysis in sequence, as follows.

[0110] 2.3 Adaptive update of static base reference image.

[0111] like Figure 4 As shown, a moving average model is used to achieve online adaptive updating of the static base reference image, adapting to dynamic changes in the kitchen environment (such as slow changes in lighting, adjustments to food placement, and movement of kitchen utensils), and avoiding the false foreground problem caused by a fixed base. The specific formulas and rules are as follows.

[0112] Initial Basis Generation: During system initialization, during non-operational periods in the kitchen (e.g., 2:00-2:05 AM), N consecutive frames (N≥150) of kitchen scene images are acquired. The median pixel value of corresponding pixels in each frame is calculated to generate an initial static basis reference image. B 0， The formula is: .

[0113] Where (x,y) are the image pixel coordinates. I 1~ I N The initialization phase consists of N consecutive frames of images, where median represents the median value; this ensures that the initial substrate accurately reflects the static environmental characteristics of the kitchen.

[0114] Online adaptive update: During normal system operation, the system updates the pre-processed image for each frame. I t Determine whether pixel (x, y) is a background pixel (determined by subsequent foreground segmentation results). If it is a background pixel, update the static basis according to the preset learning rate, using the following formula: .

[0115] B t+1 The physical meaning of (x,y): the base pixel value of the next frame; value / explanation: the pixel value at pixel (x,y) of the updated static base reference image, which is the new base reference. B t The physical meaning of (x,y): The base pixel value of the current frame; Value / Explanation: The pixel value of the static base reference image at pixel (x,y) before the update. The physical meaning of α: The learning rate; Value / Explanation: 0.01 (experimental calibration), adjustable range 0.005~0.02; The smaller the value, the smoother the base update and the stronger the anti-interference ability; The larger the value, the faster the update speed and the better it adapts to rapid changes in the environment. I t The physical meaning of (x,y): the pixel value of the currently acquired image; the first... t The pixel value at pixel (x,y) of the kitchen environment image in the frame. D t The physical meaning of (x,y): Foreground segmentation result identifier; related to the foreground mask. M t (x, y) takes completely identical values, 0 = background region, 1 = foreground region; only when (x, y) are the values ​​completely identical ... D t Update when (x,y)=0 to avoid foreground targets interfering with the basis.

[0116] Update principles: 1. Pixel-level update: Each pixel in the image is judged and updated independently to adapt to local environmental changes (such as local lighting); 2. Slow update: The learning rate α is minimized to ensure the stability of the base image; 3. Foreground masking: Only background pixels are updated to completely eliminate the influence of moving targets such as people and harmful organisms on the base.

[0117] 2.4 Background difference and foreground segmentation.

[0118] Preprocessed image I t Compared with the current static base reference image B t Perform difference operations to extract dynamic target regions. Specific steps include: calculating image differences: ... and The difference between corresponding pixels is calculated to obtain the difference image. The formula is: This reflects the pixel grayscale difference between the current image and the base image.

[0119] Local adaptive thresholding binarization: A local adaptive thresholding algorithm is used. A 5×5 pixel window centered at (x,y) is used to calculate the mean and standard deviation of the difference image within the window, determining the adaptive threshold. This segments the difference image into foreground (dynamic target) and background (static environment), yielding the foreground mask. M t The core formula is: .

[0120] in, The mean pixel value of a local window in the difference image. Δt(x,y) represents the standard deviation of pixels in the local window of the difference image, k is the threshold adjustment coefficient (default value is 2.0, and 2.0~2.5 is used for complex lighting scenes), and Δt(x,y) is the pixel gray-level difference value at coordinates (x,y).

[0121] Foreground mask optimization: M t Morphological processing is performed (first opening operation: erosion → dilation, to eliminate small noise points; then closing operation: dilation → erosion, to fill the voids inside the target area) to obtain the optimized foreground mask. .

[0122] Connected component analysis: Connectivity analysis is performed to identify continuous foreground pixel regions in the image. A bounding rectangle is defined for each connected region, which serves as a candidate target region, denoted as . R i = ( x i ,y i ,w i ,h i )( i (Candidate target number).

[0123] Detailed parameter description: x i ,y i The physical meaning of ) is: i The top-left pixel coordinates of the bounding rectangle of the candidate target region; Note: The top-left corner of the image is the origin of the coordinate system, with x as the horizontal axis and y as the vertical axis. w i ,h i The physical meaning of: the first i The width and height of the bounding rectangle of each candidate target area; Note: This reflects the overall size of the candidate target.

[0124] 3. Multipath detection and diversion stage based on regional features.

[0125] (1) Core objective: Extract the core features of the candidate target area, and guide the candidate target to the corresponding dedicated detection pipeline (personnel detection / harmful organism detection) through preset diversion rules, or determine it as a static interference / ignore it, so as to avoid the invalid calculation of the general model and improve the detection efficiency and accuracy.

[0126] (2) Execution process: such as Figure 5 As shown, the server considers each candidate target region. R i Extract four-dimensional core features and construct corresponding feature vectors F i Then, the task is split according to the splitting rule function, as follows.

[0127] 3.1 Calculation of feature vectors for candidate regions.

[0128] For the first i Candidate target areas R i = ( x i ,y i ,w i ,h i Extract the four-dimensional core features that distinguish target types in a kitchen setting, and construct a unique feature vector corresponding to the candidate region. F i The eigenvector is defined as: All four-dimensional features undergo dimensionless quantization to facilitate rule adaptation across cameras and scenes; each feature is based on the candidate target region. R i The geometric and kinematic properties are calculated, and the calculation method and physical meaning are explained in detail below.

[0129] Feature name: Area percentage; Calculation method: Physical meaning: Candidate target region R i The overall size of the image is the proportion of the total size; the value range is [0,1]; the distinction is based on the fact that the area of ​​the personnel target is significantly larger than that of the pests.

[0130] Feature name: Target aspect ratio; Calculation method: ( w i Candidate region R i width, h i Candidate regionR i (Height); Physical meaning: Candidate target area R i Morphological characteristics; Value range: [0, +∞]; Distinguishing criteria: The aspect ratio of personnel targets is relatively fixed, while the aspect ratio of harmful organisms varies greatly.

[0131] Feature name: Target motion velocity; Calculation method: Based on candidate target regions from 3 consecutive frames of images. R i Center coordinate calculation: Where: T is the single-frame acquisition interval (1-3 seconds). , Candidate region for frame t R i The center coordinates; physical meaning: candidate target area. R i The corresponding target movement speed; the distinction is based on the fact that harmful organisms (rats, cockroaches) move significantly faster than humans.

[0132] Feature name: Target trajectory irregularity; Calculation method: Calculate candidate target region R i The deviation rate between the motion trajectory coordinates and the fitted straight line for 5 consecutive frames: Candidate target area R i The smoothness of the target movement trajectory; value range: [0,1]; basis: the larger the value, the more irregular the trajectory; the irregularity of the trajectory of harmful organisms is much higher than that of personnel (personnel mostly move in straight lines / smoothly).

[0133] 3.2 Flow splitting rule function.

[0134] Based on candidate target regions R i Corresponding feature vector F i Construct a branching rule function for multi-conditional branching judgments. By applying thresholds to the four-dimensional features, precise segmentation of candidate targets is achieved. The core formula is: .

[0135] ∧ represents AND logic, meaning that all conditions must be met simultaneously before a candidate target region can be included. R i It is determined to be of the corresponding type and directed to the corresponding testing pipeline.

[0136] Otherwise indicates that if none of the above conditions are met, the candidate target region is directly ignored. Ri This reduces subsequent invalid calculations.

[0137] All decision thresholds are based on the candidate target region. R i eigenvectors F i The settings are strongly correlated with regional characteristics to ensure the accuracy of triage determination.

[0138] Detailed explanation of the diversion threshold parameters: All thresholds are empirical values ​​calibrated through extensive experiments in typical kitchen scenarios (restaurants, canteens, central kitchens), and can be fine-tuned according to the actual kitchen scenario (area, layout, lighting), as detailed below.

[0139] T a1 Physical meaning: Lower limit of the area ratio for personnel detection; Default value: 0.02; Adjustable range: 0.01~0.03; Calibration basis: Candidate area corresponding to personnel at the conventional deployment height of kitchen cameras (2.5-3.5 meters). R i The minimum area percentage.

[0140] T a2 Physical meaning: The upper limit of the area ratio for personnel detection; Default value: 0.30; Adjustable range: 0.20~0.40; Calibration basis: Candidate area corresponding to personnel at the standard deployment height of kitchen cameras. R i The maximum area ratio (avoiding excessively large area ratios due to proximity to the target).

[0141] T ar1 Physical meaning: Lower limit of aspect ratio for personnel detection; Default value: 0.40; Adjustable range: 0.30~0.50; Calibration basis: Candidate area corresponding to the body of kitchen staff when standing / operating. R i The minimum aspect ratio.

[0142] T ar2 Physical meaning: Upper limit of aspect ratio for personnel detection; Default value: 0.80; Adjustable range: 0.70~0.90; Calibration basis: Candidate area corresponding to the body of kitchen staff when standing / operating. R i The maximum aspect ratio.

[0143] T sPhysical meaning: Movement speed threshold (distinguishing between people and pests); Default value: 3 pixels / second; Adjustable range: 2~5 pixels / second; Calibration basis: Candidate area corresponding to normal human operation in a kitchen scene. R i Maximum movement speed, candidate regions corresponding to pests R i The minimum critical value of motion speed.

[0144] T b1 Physical meaning: Lower limit of the area percentage for pest detection; Default value: 0.001; Adjustable range: 0.0005~0.002; Calibration basis: Candidate areas corresponding to pests such as rats / cockroaches under kitchen camera. R i The minimum area ratio (small target detection threshold).

[0145] T b2 Physical meaning: Upper limit of the area ratio for pest detection; Default value: 0.02; Adjustable range: 0.01~0.03; Calibration basis: Candidate areas corresponding to pests and personnel. R i The area ratio threshold is used to avoid overlapping of target type determinations.

[0146] T irr Physical meaning: Trajectory irregularity threshold; Default value: 0.60; Adjustable range: 0.50~0.70; Calibration basis: Candidate areas corresponding to harmful organisms and personnel in a kitchen setting. R i The trajectory irregularity threshold is defined as the value of the threshold, and the trajectory irregularity of harmful organisms is greater than this value.

[0147] Supplementary rule explanation: f speed ≈ 0: This means the target's movement speed is less than 0.3 pixels / second, and it is judged as a static target, excluding interference from moving targets.

[0148] Whitelist features: Includes a feature library of common static distractors in the kitchen (such as fallen kitchen utensils, placed food, moving plates, and cleaning tools). The server learns offline and updates online. Offline training is based on images of common kitchen items, and online supplementation of new distractor features is based on actual detection results to reduce the false alarm rate.

[0149] 4. Parallel deep learning fine-grained recognition stage.

[0150] Core objectives: such as Figure 6As shown, based on the multi-path detection and diversion results, the candidate region images are sent to the pest precise identification pipeline and the personnel detection and wearable analysis pipeline, respectively. The two pipelines are executed in parallel, enabling fine-grained and accurate identification of targets, improving detection accuracy and processing efficiency, and adapting to kitchen scenarios.

[0151] 4.1 Pest identification pipeline.

[0152] Model architecture: A lightweight convolutional neural network model based on the YOLOv5n architecture with pruning and retraining; the lightweight design ensures fast inference on the server side, prunes redundant convolutional layers, retains the core detection layers, and reduces the number of model parameters; retraining is based on a kitchen pest labeled dataset to improve the recognition accuracy of small targets and fast-moving targets, and adapts to kitchen fumes and steam interference scenarios.

[0153] Core formula: The model's total loss function measures the deviation between the model's predictions and the true target. The smaller the loss value, the higher the recognition accuracy. The formula is: .

[0154] formula With candidate target region R i eigenvectors F i This involves deep integration of traffic splitting logic to achieve a closed-loop process of "traffic splitting and filtering - precise identification." (Formula) The calculation results (loss value) can be used to verify the rationality of the diversion threshold, forming a closed loop of "diversion-identification-feedback optimization".

[0155] Detailed explanation of the loss function and parameters.

[0156] Type: Total Loss; Physical Meaning: Overall loss value of the pest detection model; Value Range / Explanation: ≥0, when the loss value approaches 0, the model has the highest recognition accuracy; after training convergence, the loss value stabilizes between 0.05 and 0.15.

[0157] Type: Weight coefficient; Physical meaning: Weight of bounding box loss, used to adjust the importance of bounding box prediction; Value / Calculation method: 5.0 (experimental calibration); Value range / Explanation: Adjustable range 3.0~7.0; There are many small targets in the kitchen, so it is necessary to increase the weight of bounding box prediction to reduce positioning error.

[0158] Type: Bounding box loss; Physical meaning: Measures the deviation of the model's predicted bounding box from the ground truth bounding box in terms of position, size, and shape; Value / Calculation method: Calculated based on the center distance, aspect ratio, and intersection-union ratio of the predicted and ground truth bounding boxes; Value range / Explanation: [0,2], the smaller the value, the more accurate the bounding box localization.

[0159] Type: Weighting coefficient; Physical meaning: Weight of target confidence loss, adjusting the importance of target existence prediction; Value / Calculation method: 1.0 (experimental calibration); Value range / Explanation: Adjustable range 0.5~1.5; Balances the weight of target existence with location and classification to avoid missed detections.

[0160] Type: Confidence loss; Physical meaning: Measures the deviation between the model's confidence in the existence of the target and the true label; Value / Calculation method: Calculated using the cross-entropy loss function; Value range / Explanation: [0, +∞), the smaller the value, the more accurate the judgment of the existence of the target.

[0161] Type: Weighting coefficient; Physical meaning: Weight of category loss, adjusting the importance of pest species identification; Value / Calculation method: 2.0 (experimental calibration); Value range / Explanation: Adjustable range 1.0~3.0; It is necessary to accurately distinguish pests such as rats and cockroaches to improve the category identification weight.

[0162] Type: Class Loss; Physical Meaning: Measures the deviation between the model's predicted pest species and the actual species; Value / Calculation Method: Calculated using the Focal Loss function to alleviate the class imbalance problem; Value Range / Explanation: [0, +∞), the smaller the value, the higher the species identification accuracy.

[0163] Model training details: Dataset: Images of harmful organisms (mainly rats and cockroaches) in different scenarios such as restaurants, canteens, and central kitchens were collected, totaling 10,000+ frames, with target bounding boxes and species labels annotated; images of interfering scenes such as oil fumes, steam, and changes in lighting were added to enhance the model's anti-interference ability; the dataset was divided into training set, validation set, and test set in an 8:1:1 ratio.

[0164] Training parameters: initial learning rate 0.001, learning rate decayed using cosine annealing; batch size 16, training 100 epochs; early stopping strategy, training stops when the validation set loss does not decrease for 10 consecutive epochs to avoid overfitting.

[0165] Optimization strategies: For small targets (such as small cockroaches), data augmentation techniques such as image scaling, flipping, and cropping are used to improve the recognition accuracy of small targets; for blurry scenes with kitchen fumes, blurry image enhancement training is added to allow the model to adapt to different degrees of image blur.

[0166] Identification process: 1) Identify candidate areas for "pest detection" after triage. Perform image preprocessing (normalize the size to 640×640 pixels, keep the aspect ratio unchanged, and fill the edges with black pixels).

[0167] 2) Input the preprocessed candidate region image into the lightweight YOLOv5n optimization model. The model outputs the target bounding box, category confidence, and target confidence.

[0168] 3) Confidence screening: Set the target confidence threshold to 0.5 and the category confidence threshold to 0.6 to filter prediction results with confidence levels below the thresholds and reduce misjudgments.

[0169] 4) Non-maximum suppression (NMS): The NMS algorithm (IOU threshold 0.3) is used to remove prediction boxes with excessive overlap and retain the best prediction results.

[0170] 5) Output the identification results: Record the pest species type (e.g., rat, cockroach), target coordinates (x, y, w, h), and confidence score (conf), and label it as Result. pest = (type,x,y,w,h,conf).

[0171] Output results: pest species, target location and confidence level. If no valid pest is identified, output "No pest".

[0172] 4.2 Personnel detection and wearable analysis pipeline.

[0173] Model Architecture: A lightweight two-stage model of "personnel detection + wearing status recognition" is adopted. The first stage uses a cropped version of YOLOv5s to achieve fast personnel detection, and the second stage uses a lightweight CNN model to achieve fine-grained recognition of the wearing status of work hats and masks. Both stages of the model are retrained for kitchen scenarios (oil fume obstruction, diverse personnel postures, and changes in lighting) to adapt to the operating scenarios of kitchen staff.

[0174] Core formula: The loss function of the wearable state recognition model, used to measure the accuracy of wearable state recognition, is as follows: .

[0175] formula (Including only losses related to work caps and masks), actually related to the candidate target area. R i eigenvectorsF i This is deeply integrated with the triage logic, forming a closed-loop process of "triage screening - personnel identification - wearable detection". The calculation result (loss value) of the formula can be used to verify the personnel triage threshold in Section 3.2. T a1 , T a2 , T ar1 , T ar2 , T s This ensures the rationality of the process and optimizes the feature selection logic in Section 3.1, forming a closed loop of "triage-wearable recognition-feedback optimization". For example, if a batch of triage results in... R i After inputting the model, The loss values ​​remained consistently high, and analysis revealed that this was primarily due to losses in some smaller individuals. < T a1 If an organism is misjudged as "ignored" or "harmful," the parameters in section 3.2 can be fine-tuned based on the loss value feedback. T a1 (Lower limit of personnel detection area ratio), expand the screening range of personnel candidate areas; if it is found that some personnel are moving rapidly ( ≈ T s If mistakenly diverted to a pest control line, it can be finely adjusted. T s Thresholds improve the accuracy of personnel diversion, thereby reducing... Loss value, improving wearable analytics.

[0176] Detailed explanation of loss function and parameters: Type: Total loss; Physical meaning: Overall loss value of the personnel wearing status recognition model; Value range: ≥0, stabilizes between 0.03 and 0.10 after training convergence, the smaller the loss value, the higher the recognition accuracy.

[0177] Type: Weighting coefficient; Physical meaning: Weight of work cap wearing loss; Value / calculation method: 1.0 (experimental calibration); Value range / description: Adjustable range 0.8~1.2, balanced with mask wearing loss weight, and adapted to kitchen operation specifications.

[0178] Type: Cap loss; Physical meaning: Measures the bias in recognizing cap wearing status (wearing / not wearing); Value / Calculation method: Calculated using cross-entropy loss function; Value range / Explanation: [0, +∞), the smaller the value, the more accurate the cap wearing recognition.

[0179] Type: Weighting coefficient; Physical meaning: Weight of mask-wearing loss; Value / Calculation method: 1.0 (experimental calibration); Value range / Explanation: Adjustable range 0.8~1.2, balanced with the weight of work cap wearing loss, in compliance with kitchen hygiene standards.

[0180] Type: Mask loss; Physical meaning: Measures the deviation in identifying mask wearing status (wearing / not wearing / not wearing properly); Value / Calculation method: Calculated using cross-entropy loss function; Value range / Explanation: [0, +∞), the smaller the value, the more accurate the mask wearing identification.

[0181] Model training details: (1) Dataset: Collected 8000+ frames of images of kitchen staff postures (standing, bending over) and different occlusion scenarios (oil fume occlusion, hand occlusion), labeled with personnel bounding boxes, work hat wearing status (wearing / not wearing), and mask wearing status (wearing / not wearing / not wearing properly); added images of kitchen lighting changes and steam interference to improve model adaptability; divided into training set, validation set, and test set in an 8:1:1 ratio.

[0182] (2) Training parameters: initial learning rate 0.001, gradient descent optimization algorithm; batch size 32, training rounds 80; early stopping strategy, stop when the validation set loss does not decrease for 8 consecutive rounds to avoid overfitting.

[0183] (3) Optimization strategy: For the scenario where kitchen fumes cover the face, focus on training the facial feature extraction layer to improve the accuracy of wearing status recognition under occlusion; classify and label work hats and masks of different colors and styles to ensure that the model is adapted to the dress code of different kitchens.

[0184] Identification process: 1. Identify candidate areas for "personnel detection" after traffic splitting. Perform image preprocessing (normalize the size to 416×416 pixels, maintain the aspect ratio, and fill the edges).

[0185] 2. First stage (person detection): Input the YOLOv5s cropped model, output the person bounding box and target confidence score, set the confidence score threshold to 0.6, filter invalid predictions, and remove overlapping boxes using the NMS algorithm (IOU threshold 0.4) to obtain the person target area.

[0186] 3. Second stage (wearing status recognition): Extract the head region from the target area of ​​the person (based on human key point detection, locate the head coordinates), input it into a lightweight CNN model, and output the wearing status of the work hat, the wearing status of the mask and the corresponding confidence score (confidence score threshold 0.7).

[0187] 4. Violation judgment: Not wearing a work hat or mask or wearing them improperly is judged as a violation of the wearing rules.

[0188] 5. Output recognition results: Record the personnel's location, work hat wearing status, mask wearing status, and confidence level, and label them as Result. wear =(x,y,w,h,cap_state,cap_state,conf).

[0189] Output results: personnel location, wearing status of work hat and mask (including violation judgment). If no valid personnel are identified, output "No personnel".

[0190] 4.3 Timing-based decision-making and alarm phase.

[0191] Core objective: To perform time-series analysis on the parallel identification results of the S4 stage, filter out single-frame misjudgments (such as misjudgments caused by steam or momentary obstruction), trigger real-time alarms on multiple terminals after confirming valid targets, ensure the accuracy and reliability of alarms, and avoid invalid alarms interfering with the work of management personnel.

[0192] Execution process: The timing decision and alarm module of the server intelligent analysis subsystem performs the following operations.

[0193] (1) Temporal caching: Cache the recognition results of 10 consecutive frames (collection interval 1-3 seconds, caching time 10-30 seconds), establish a temporal queue, and record the target type, location, confidence level and timestamp of each frame.

[0194] (2) Valid target determination rules: Valid pest determination: If the same type of pest is identified in 3 or more consecutive frames, and the target location overlap (IOU) is ≥0.5 and the average confidence level is ≥0.6, it is determined to be a valid pest invasion.

[0195] Valid determination of personnel wearing violations: If the same person is found to be wearing a violation (not wearing a work hat / not wearing a mask / not wearing it properly) in 2 or more consecutive frames, and the personnel position overlap (IOU) is ≥0.6 and the mean confidence level is ≥0.7, it is determined to be a valid violation.

[0196] (3) False detection filtering: If a target is detected in a single frame but the above continuous frame condition is not met, it is judged as a single frame false detection, no alarm is triggered, and only the log is recorded.

[0197] (4) Alarm Trigger: When it is determined that there is a valid invasion of harmful organisms or a valid violation of personnel wearing protective clothing, the alarm mechanism is triggered and alarm information is generated. The alarm information includes: alarm type (invasion of harmful organisms / violation of personnel wearing protective clothing), alarm time, alarm location (camera number + area), target details (type of harmful organism / type of violation), and on-site image frame.

[0198] (5) Multi-terminal alarm push: Web management backend: pop-up window prompts alarm information, synchronously displays on-site images and target details, and marks the alarm status as "unprocessed".

[0199] (6) Mobile terminal: Push alarm notifications via mini-program / APP, with attached on-site images, allowing managers to click to view details and confirm processing; "Rat intrusion detected, please take action" "Staff not wearing masks, please rectify immediately").

[0200] (7) Alarm handling tracking: After the administrator confirms the alarm handling on any terminal, the system records the handler, handling time, and handling result (such as "harmful organisms have been removed" or "the violator has been rectified"), and updates the alarm status to "handled", forming an alarm handling closed loop.

[0201] (8) Output results: valid alarm information (pushed to each management terminal), alarm processing records. Identification results that do not meet the valid judgment conditions are only recorded in the log and do not trigger alarms.

[0202] 4.4 Data storage and management stage.

[0203] Core objective: To structure and classify all data during system operation, support data query, statistics, and traceability, provide data support for kitchen hygiene management, personnel training, and hazard rectification, and achieve a complete management closed loop of "monitoring-alarm-analysis-rectification".

[0204] Execution process: The data storage and management module of the server intelligent analysis subsystem performs the following operations.

[0205] (1) Data classification and storage: The data is divided into 4 categories and stored in SSD solid-state drives using a structured storage method. Data is retained for 90 days (configurable), and expired data is automatically backed up and deleted.

[0206] (2) Image data: Preprocessed images and on-site image frames at the time of alarm are named in the format of "camera number-date-timestamp" for easy traceability.

[0207] (3) Detection data: the identification results (harmful organisms / personal clothing), target coordinates, confidence level, timestamp of each frame, and associated camera number and area information.

[0208] (4) Alarm data: alarm information, alarm type, alarm location, processing record (processor, processing time, processing result), associated with the corresponding on-site image.

[0209] (5) System data: system configuration parameters (camera parameters, model thresholds, alarm rules), device operating status (camera online / offline, server load), model update log.

[0210] (6) Data query function: It supports querying data by time range, alarm type, region, camera number and other conditions. It can view historical detection results, alarm records and on-site images to realize data traceability.

[0211] (7) Data statistical analysis: Automatically generate statistical reports.

[0212] (8) Pest statistics: Statistics on the number of pest invasions and high-incidence areas by species, region and time period, to provide a basis for pest control.

[0213] (9) Personnel violation statistics: The number of violations is counted by type of violation (not wearing a work hat / mask), personnel, and time period to provide support for personnel training and assessment.

[0214] (10) Equipment operation statistics: Statistics on camera online rate, server load, and alarm response time provide a reference for system operation and maintenance.

[0215] (11) Data export and backup: Supports exporting statistical reports, historical data and image data to Excel and JPG formats, and supports manual / automatic backup (automatic backup every morning) to prevent data loss.

[0216] (12) System parameter management: Supports administrators to adjust system parameters (such as acquisition frame rate, model threshold, alarm rules) online through the Web management backend and mobile terminal. The parameter adjustment takes effect in real time without restarting the system.

[0217] (13) Output results: various types of structured data, statistical reports, and data backup files, which can be queried, exported, and traced.

[0218] Implementation Case: Deployment of a Monitoring System for a Small Restaurant Kitchen.

[0219] Kitchen scene: A small restaurant kitchen, about 30 square meters in size, including four areas: food storage area, processing area, stove area, and dishwashing area. The lighting varies greatly, and there is a lot of oil fumes and steam. There are 3-5 staff members.

[0220] System deployment plan 1.

[0221] (1) Image acquisition module: Two high-definition network cameras are deployed and installed in two key areas respectively. The installation height is 2.5 meters, the viewing angle is 180°, and the H.265 encoding is supported. The minimum illumination is 0.01 lux, and it is resistant to oil fumes and steam. The acquisition resolution is 1920×1080, the frame rate is 1 second / frame, the image format is JPG, and the compression rate is 80%.

[0222] (2) Server intelligent analysis subsystem: a local lightweight server is used, equipped with Intel Xeon E5 CPU, NVIDIA Tesla T4 GPU, 16GB memory and 1TB SSD; all software modules of the system are deployed, and the optimized harmful biometric identification model and personnel wearable identification model are loaded; the diversion threshold is set to the default value, the learning rate α=0.01, and the alarm confidence threshold is configured by default.

[0223] (3) Management alarm terminal: Deploy Web management backend (1 computer), mobile terminal (manager's mobile phone, via enterprise WeChat mini program), and on-site voice broadcaster (1 unit, installed at the kitchen entrance).

[0224] (4) Operational effect: After the system is in operation, it can achieve monitoring of the area without blind spots; it can accurately identify harmful organisms such as rats and cockroaches, with a false alarm rate of ≤3% and a false alarm rate of ≤2%; it can accurately identify the wearing status of staff's work hats and masks, with a violation identification accuracy of ≥98%; the alarm response time is ≤0.3 seconds, and the on-site voice broadcast promptly reminds the rectification of violations. The alarm information and processing records can be viewed in real time on the Web terminal and mobile terminal; the data is stored for 90 days, and it supports querying historical data by date and region, generating monthly statistical reports to support kitchen hygiene management.

[0225] like Figure 7 As shown, the kitchen scene is a large central kitchen with an area of ​​approximately 200 square meters. It includes six areas: food storage area, preliminary processing area, fine processing area, stove area, dishwashing area, and delivery area. It has 20-30 staff members, operates 24 hours a day, has uniform lighting but a large amount of oil fumes, and has high requirements for pest control. It also needs to meet the requirements of large-scale and standardized management.

[0226] System deployment plan 2.

[0227] (1) Image acquisition module: 12 high-definition network cameras are deployed to cover all 6 areas (2 in the food storage area, 2 in the preliminary processing area, 2 in the fine processing area, 2 in the stove area, 2 in the dishwashing area, and 2 in the delivery area). The installation height is 3.0 meters, the viewing angle is 180°, and remote parameter configuration is supported. The acquisition resolution is 1920×1080, the frame rate is 2 seconds / frame, and it is suitable for 24-hour operation. It can still clearly acquire images in low-light environments (such as the night storage area). The lens has an anti-oil fume coating to reduce the impact of oil fume adhesion on the acquisition effect.

[0228] (2) Server Intelligent Analysis Subsystem: Deployed on a private cloud, configured with 2 high-performance servers (load balanced), each server equipped with an Intel Xeon E5 CPU, NVIDIA Tesla T4 GPU, 32GB of memory, and 2TB SSD; realizes parallel inference of models, improves processing efficiency, and can process real-time images from 12 cameras simultaneously; adjust the traffic splitting threshold: T s =4 pixels / second, T irr =0.55, suitable for scenarios with dense personnel and complex objectives in central kitchens; static basis update learning rate α=0.015, suitable for 24-hour light changes.

[0229] (3) Management alarm terminal: Deploy Web management backend (3 computers, respectively for hygiene management, personnel management and equipment management), mobile terminal (all management personnel's mobile phones, accessed through enterprise WeChat mini program), and on-site voice broadcaster (3 units, respectively installed in processing area, stove area and delivery area); support alarm hierarchical push, major alarms (such as a large number of harmful organisms invading, multiple people wearing the same clothes at the same time) are pushed to all management personnel, and general alarms (such as a single person violating the rules, a single harmful organism appearing) are pushed to the person in charge of the corresponding area to ensure that alarms are accurately reached and handled in a timely manner.

[0230] (4) Operational effect: The system achieves 24-hour real-time monitoring with no blind spots; the accuracy of pest identification is ≥96%, the false alarm rate is ≤2%, and it can accurately identify small targets such as small cockroaches (area ratio ≥0.001); the accuracy of personnel wearing violation identification is ≥99%, and it can distinguish the situation of improper mask wearing (such as not covering the mouth and nose) and work hat wearing crookedly; the alarm response time is ≤0.5 seconds, and the hierarchical push ensures that alarms are handled in a timely manner; the data supports statistics by region, personnel and pest type, and generates daily, weekly and monthly reports, providing data support for the large-scale and standardized management of central kitchens; the system can be flexibly expanded and reduced to adapt to changes in the number of staff and the needs of regional adjustment, and cameras can be added and detection functions (such as food storage standard detection) can be expanded in the future.

[0231] like Figure 8 As shown in the bar chart, the accuracy of identifying harmful organisms and identifying violations of personnel wearing protective clothing are intuitively compared between the present invention and three existing mainstream kitchen monitoring technologies. This quantitatively demonstrates the technical advantages of the present invention in terms of detection accuracy and provides data support for the practicality of the solution.

[0232] like Figure 9 As shown in the figure, the detection delay of the present invention and three existing mainstream kitchen monitoring technologies are compared in the form of a line graph under different numbers of cameras. This quantitatively demonstrates the advantages of the present invention in detection efficiency and system scalability, highlighting the technical value of "multi-path splitting and parallel inference".

[0233] This invention is adapted to the special scenario of the kitchen, with high detection efficiency, strong anti-interference ability and comprehensive functions. It can achieve accurate detection of harmful organisms and personnel wearing violations, real-time alarm and data traceability, forming a complete management closed loop, and is suitable for safety management in various kitchen scenarios.

[0234] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A computer vision-based intelligent monitoring method for kitchen hygiene and safety, characterized in that: Includes the following steps: S1. Acquire high-quality images of the kitchen and preprocess them to remove image blur and noise caused by kitchen fumes and steam, and output the preprocessed static base reference image of the kitchen environment. S2. Based on the moving average model, the static base reference image of the kitchen environment is updated online adaptively, and the pixels in the background area are updated to adapt to the dynamic changes of the kitchen environment and avoid the false foreground problem caused by the fixed base. S3. Extract candidate regions of dynamic targets from the image processed in step S2 through background subtraction, local adaptive threshold binarization, morphological optimization, and connected component analysis, and eliminate invalid regions to reduce the computational load of subsequent models. S4. Extract the four-dimensional core features of the candidate region, including area ratio, aspect ratio, motion speed and trajectory irregularity, and guide the candidate target to S5 through a preset diversion rule function; S5. Two production lines, the pest identification line and the personnel wear identification line, are executed in parallel to identify the types of pests and the wearing status of personnel's work hats and masks, respectively. S6. Perform time-series analysis and confidence accumulation on the recognition results, filter out single-frame misjudgments, and trigger real-time alarms on multiple terminals after determining valid targets to ensure the accuracy and reliability of alarms.

2. The intelligent monitoring method for kitchen hygiene and safety based on computer vision according to claim 1, characterized in that: In step S2, during the initialization process, N consecutive frames of kitchen scene images are acquired during non-operational periods in the kitchen. The median pixel value of each corresponding pixel in each frame is calculated to generate an initial static base reference image. B 0, the formula is: Where (x,y) are the image pixel coordinates. I 1~ I N For N consecutive frames of images acquired during the initialization phase, median represents the median value operation; During normal operation, each frame of the preprocessed image... I t Determine if pixel (x, y) is a background pixel. If it is, update the static basis according to the preset learning rate. The formula is: ;in: B t+1 (x,y) represents the base pixel value of the next frame. B t (x,y) represents the base pixel values ​​of the current frame, and α is the learning rate. I t (x,y) represents the pixel values ​​of the currently acquired image. D t (x,y) is the identifier for the foreground segmentation result.

3. The intelligent monitoring method for kitchen hygiene and safety based on computer vision according to claim 2, characterized in that: In step S3, candidate regions for dynamic targets are extracted through background subtraction, local adaptive threshold binarization, morphological optimization, and connected component analysis, specifically including: S31. Calculate image difference: For and The difference between corresponding pixels is calculated to obtain the difference image. The formula is: This reflects the pixel grayscale difference between the current image and the base image; S32. Local Adaptive Threshold Binarization: A local adaptive thresholding algorithm is used. A 5×5 pixel window centered at (x,y) is used to calculate the mean and standard deviation of the difference image within the window, determining the adaptive threshold. This segments the difference image into foreground and background, obtaining the foreground mask. M t The formula is: ;in, The mean pixel value of a local window in the difference image. Let k be the standard deviation of pixels in the local window of the difference image, and k be the threshold adjustment coefficient. The pixel grayscale difference value at coordinates (x, y); S33, Foreground Mask Optimization: [This section appears to be incomplete and requires further context.] M t Morphological processing is performed, first opening (erosion followed by dilation) to eliminate small noise points; then closing (dilation followed by erosion) to fill holes in the target area, resulting in an optimized foreground mask. ; S34. Connectivity Analysis: [This section discusses] Connectivity analysis is performed to identify continuous foreground pixel regions in the image. A bounding rectangle is defined for each connected region, which serves as a candidate target region, denoted as . R i = ( x i ,y i ,w i ,h i ), i For candidate target number, ( x i , y i ) No. i The top-left pixel coordinates of the bounding rectangle of the candidate target region w i ,h i For the first i The width and height of the bounding rectangle of each candidate target region.

4. The intelligent monitoring method for kitchen hygiene and safety based on computer vision according to claim 3, characterized in that: In step S4, for the first i Candidate target areas R i = ( x i ,y i ,w i ,h i Extract the four-dimensional core features that distinguish target types in a kitchen scene, and construct the unique feature vector F corresponding to the candidate region. i The eigenvector is defined as: Based on candidate target regions R i Corresponding feature vector F i Construct a branching rule function for multi-conditional branching judgments. By applying thresholds to the four-dimensional features, precise segmentation of candidate targets is achieved. The formula is as follows: ;in: For the target area percentage, For the target aspect ratio, For the target speed of motion, For the irregularity of the target trajectory, T a1 This represents the lower limit of the area to be tested by personnel. T a2 This represents the upper limit of the area to be tested by personnel. T ar1 The lower limit of the aspect ratio for personnel inspection. T ar2 The upper limit of the aspect ratio for personnel inspection. T s The threshold for motion speed, T b1 This represents the lower limit of the area to be monitored for harmful organisms. T b2 This represents the upper limit of the area to be monitored for harmful organisms. T irr This is the threshold for trajectory irregularity.

5. The intelligent monitoring method for kitchen hygiene and safety based on computer vision according to claim 4, characterized in that: In step S5, the pest precise identification pipeline includes: S5a-1. Perform image preprocessing on the candidate regions that are determined to be "harmful organism detection" after the splitting; S5a-2: Input the preprocessed candidate region image into the lightweight YOLOv5n optimization model. The model outputs the target bounding box, category confidence, and target confidence. S5a-3, Confidence Filtering: Set target confidence threshold and category confidence threshold to filter prediction results with confidence levels below the threshold and reduce misjudgments; S5a-4, Non-maximum suppression: The NMS algorithm is used to remove prediction boxes with excessive overlap and retain the best prediction results; S5a-5, Output identification results: Record the type of pest, target coordinates, and confidence level. If no valid pest is identified, output "No pest".

6. The intelligent monitoring method for kitchen hygiene and safety based on computer vision according to claim 5, characterized in that: The total loss function of the lightweight YOLOv5n optimization model is: ;in: This represents the overall loss value of the pest detection model. The weights are the bounding box loss. For bounding box loss, The weights for the target confidence loss. For target confidence loss, The weights for the class loss, This is the category loss.

7. The intelligent monitoring method for kitchen hygiene and safety based on computer vision according to claim 6, characterized in that: In step S5, personnel wear identification includes: S5b-1. Perform image preprocessing on the candidate regions that are determined to be "personnel detection" after splitting; S5b-2: Input the YOLOv5s cropped model, output the personnel bounding box and target confidence score, set the confidence score threshold, filter invalid predictions, remove overlapping boxes using the NMS algorithm, and obtain the personnel target area; S5b-3: Extract the head region from the personnel target area, input it into a lightweight CNN model, and output the work hat wearing status, mask wearing status, and corresponding confidence scores; S5b-4. Violation Judgment: If a work cap is not worn, a mask is not worn, or is worn improperly, it is considered a violation of the dress code. S5b-5, Output recognition results: Record the personnel location, work hat wearing status, mask wearing status, and confidence level; if no valid personnel are identified, output "No personnel".

8. The intelligent monitoring method for kitchen hygiene and safety based on computer vision according to claim 7, characterized in that: The total loss function of the lightweight CNN model is: ;in: The overall loss value of the personnel wearing status recognition model. The weighting of the loss due to wearing a work hat. Loss of wearing a work hat Weighting of the loss of consciousness due to mask-wearing status. Loss due to mask-wearing status.

9. The intelligent monitoring method for kitchen hygiene and safety based on computer vision according to claim 8, characterized in that: Step S6 includes: S6-1, Temporal Cache: Cache the recognition results of 10 consecutive frames, establish a temporal queue, and record the target type, location, confidence level and timestamp of each frame; S6-2, Rules for determining valid targets include: Valid pest identification: If the same type of pest is identified in 3 or more consecutive frames, and the target location overlap (IOU) is ≥0.5 and the mean confidence level is ≥0.6, it is determined to be a valid pest invasion; Valid determination of personnel wearing violation: If the same person is detected wearing violation in 2 or more consecutive frames, and the position overlap (IOU) of the personnel is ≥0.6 and the mean confidence level is ≥0.7, it is determined to be a valid violation; False detection filtering: If a target is detected in a single frame but the above consecutive frame condition is not met, it is judged as a single frame false detection, no alarm is triggered, only log is recorded; Alarm Trigger: When a valid pest intrusion or valid personnel wearing prohibited clothing is detected, an alarm mechanism is triggered to generate alarm information, which includes: alarm type, alarm time, alarm location, target details, and on-site image frames; Multi-terminal alarm push: Web management backend: pop-up window prompts alarm information, synchronously displays on-site images and target details, and marks the alarm status as unprocessed; Mobile terminals: Alarm notifications are pushed via mini-programs / apps, accompanied by on-site images, allowing managers to click to view details and confirm processing; Alarm handling tracking: After an administrator confirms the handling of an alarm on any terminal, the system records the handler, the handling time, and the handling result, and updates the alarm status to "handled", forming a closed loop of alarm handling; Output results: valid alarm information and alarm processing records. Identification results that do not meet the valid judgment conditions are only logged and do not trigger alarms.

10. A computer vision-based intelligent monitoring system for kitchen hygiene and safety, executing the computer vision-based intelligent monitoring method for kitchen hygiene and safety as described in claim 1, characterized in that: The system includes an image acquisition unit, a server-side intelligent analysis subsystem, and a management alarm terminal. The server-side intelligent analysis subsystem comprises an image preprocessing module, a static basis update module, a target candidate region extraction module, a multi-path detection and splitting module, a parallel deep learning recognition module, a time-series decision-making and alarm module, and a data storage and management module. The image acquisition unit acquires real-time images of the kitchen, providing high-quality image data for subsequent intelligent analysis. It supports multi-camera deployment, achieving blind-spot-free coverage of the kitchen and adapting to the layout requirements of kitchens of different sizes. The management alarm terminal receives alarm information pushed by the server-side intelligent analysis subsystem, displays detection results and on-site images, and enables alarm confirmation, processing, and tracing. It also supports parameter configuration and data statistical analysis and provides a convenient management interface. Image preprocessing module: preprocesses the high-quality images of the acquired kitchen scene, removes image blur and noise caused by kitchen fumes and steam, and outputs a static base reference image of the preprocessed kitchen environment; Static Basis Update Module: Based on the moving average model, the static basis reference image of the kitchen environment is updated online adaptively, and the pixels in the background area are updated to adapt to the dynamic changes of the kitchen environment and avoid the false foreground problem caused by the fixed basis. Target candidate region extraction module: Extracts candidate regions of dynamic targets from the image processed by the static basis update module through background subtraction, local adaptive threshold binarization, morphological optimization, and connected component analysis, and eliminates invalid regions to reduce the computational load of subsequent models; Multi-path detection and diversion module: Extracts four-dimensional core features of candidate regions, including area ratio, aspect ratio, motion speed and trajectory irregularity, and guides candidate targets to parallel deep learning recognition module through preset diversion rule function; Parallel deep learning recognition module: including a pest accurate recognition pipeline and a personnel wear recognition pipeline. The two pipelines execute in parallel, respectively realizing the recognition of pest species and the recognition of personnel wearing work hats and masks; Timing-based decision and alarm module: Performs timing-based analysis and confidence accumulation on the identification results, filters out single-frame misjudgments, and triggers real-time alarms on multiple terminals after determining a valid target, ensuring the accuracy and reliability of the alarms; Data storage and management module: It stores image data, detection results, alarm information, system configuration parameters, etc. in a structured manner, supports data query, statistics, and traceability, provides data support for kitchen management, and also supports online adjustment of system parameters.

Images