A food safety intelligent assessment method, system, medium and equipment

Abstract

An intelligent food safety assessment method, system, medium and equipment relate to the technical field of food safety management. The method comprises: obtaining a surveillance video of a back kitchen area and environmental parameters collected by a sensor network; extracting a key frame sequence of the surveillance video, and identifying at least one key area corresponding to each key frame image in the key frame sequence; performing abnormality detection on the environmental parameters and each key area to obtain abnormal events; calculating a total food safety score of the back kitchen area based on each abnormal event, and generating a food safety grade of the back kitchen area according to the total food safety score. Implementing the technical solution provided by the present application can improve the accuracy and efficiency of food safety assessment in the back kitchen, and effectively improve the food safety management effect of the back kitchen.

Description

Technical Field

[0001] The present application relates to the technical field of food safety management, and specifically to an intelligent food safety assessment method, system, medium and equipment. Background Art

[0002] With the improvement of people's living standards and the enhancement of food safety awareness, the food safety management of catering enterprises has received more and more attention. At present, most catering enterprises have installed video monitoring systems in the kitchen and supervised the food safety status of the kitchen manually.

[0003] The existing back-kitchen supervision method mainly relies on supervisors to regularly watch the surveillance video footage. When an abnormal situation is found, the supervisor needs to judge the type and severity of the abnormality based on experience and decide whether to take corresponding treatment measures. This manual experience assessment method lacks comprehensive evaluation and quantification of back-kitchen food safety management, resulting in low accuracy of back-kitchen food safety assessment, which affects the food safety management of the back-kitchen. Summary of the invention

[0004] The present application provides an intelligent food safety assessment method, which improves the accuracy and efficiency of food safety assessment in the back kitchen and effectively improves the food safety management effect in the back kitchen.

[0005] In a first aspect, the present application provides an intelligent food safety assessment method, the method comprising:

[0006] Obtain surveillance video of the kitchen area and environmental parameters collected by the sensor network;

[0007] Extracting a key frame sequence of the surveillance video, and identifying at least one key area corresponding to each key frame image in the key frame sequence;

[0008] Performing abnormality detection on the environmental parameters and each of the key areas to obtain abnormal events;

[0009] The total food safety score of the back kitchen area is calculated based on each of the abnormal events, and the food safety grade of the back kitchen area is generated according to the total food safety score.

[0010] By adopting the above technical solution, the surveillance video of the back kitchen area and the environmental parameters collected by the sensor network are obtained as basic data, the key frame sequence of the surveillance video is extracted and the key areas are identified, thereby realizing efficient processing of the video data; at the same time, anomaly detection is performed on the environmental parameters and key areas, so that abnormal events in the back kitchen area can be discovered in time; further, the total food safety score is calculated based on the detected abnormal events and the food safety grade is generated, making the food safety assessment of the back kitchen area more objective and quantitative, avoiding the problem of incomplete judgment standards of different supervisors in the traditional manual experience assessment method, improving the accuracy and efficiency of the food safety assessment in the back kitchen, and effectively improving the food safety management effect in the back kitchen.

[0011] Optionally, the acquisition of the surveillance video of the kitchen area and the environmental parameters collected by the sensor network includes:

[0012] Obtain the current light intensity of the kitchen area;

[0013] Determine the video acquisition parameters of the camera based on the current light intensity, the video acquisition parameters including sampling frame rate, resolution and exposure parameters;

[0014] Determine the sampling frequency of each sensor in the sensor network according to the historical environmental parameters of the back kitchen area;

[0015] The monitoring video acquired by the camera according to the video acquisition parameters and the environmental parameters acquired by the sensor network according to the sampling frequency are acquired.

[0016] By adopting the above technical solution, the current light intensity of the kitchen area is obtained and the video acquisition parameters are dynamically adjusted accordingly, which optimizes the video acquisition quality under different lighting conditions; at the same time, the sampling frequency of the sensor is determined based on the historical data of the environmental parameters, which can reduce the system resource occupation while ensuring the timeliness of data acquisition; further, by using the optimized acquisition parameters to perform data acquisition, the acquisition quality of the monitoring video and environmental parameters is ensured. Through this adaptive data acquisition method, not only the availability and accuracy of the collected data are improved, but also the reasonable scheduling of system resources is achieved.

[0017] Optionally, extracting the key frame sequence of the surveillance video includes:

[0018] Determining a target activity area in the surveillance video;

[0019] Determining candidate key frames based on the target activity area;

[0020] Calculating the similarity between the candidate key frame and the previous key frame;

[0021] When the similarity is less than a similarity threshold, determining the candidate key frame as a key frame;

[0022] The key frames are arranged in time sequence to form a key frame sequence.

[0023] By adopting the above technical scheme, the target activity area in the monitoring video is calculated and the candidate key frames are determined accordingly, so as to realize the preliminary screening of the key information of the video; at the same time, by calculating the similarity between the candidate key frame and the previous key frame and setting the threshold judgment, the redundant information can be effectively removed; the screened key frames are further organized in chronological order to form a key frame sequence with timing characteristics, which reduces the data processing amount while ensuring the integrity of the key information of the video, and improves the processing efficiency of subsequent anomaly detection.

[0024] Optionally, determining a candidate key frame based on the target activity area includes:

[0025] When the area of ​​the target activity region is greater than a first threshold, or the area of ​​the target activity region is less than the first threshold and greater than the second threshold and the movement speed is greater than the speed threshold, or the target activity region meets a preset shape feature, the corresponding video frame is marked as a candidate key frame.

[0026] By adopting the above technical solution, by setting a graded threshold for the target activity area, combining the motion speed threshold and the preset shape feature for multi-condition judgment, flexible identification of different types of active targets is achieved; at the same time, by comprehensively considering the size, speed and shape characteristics of the target, the key moment when abnormal behavior occurs can be captured more accurately. This multi-dimensional candidate key frame determination method improves the accuracy of key frame selection, avoids the omission of important information, and provides a more reliable data basis for subsequent abnormal event detection.

[0027] Optionally, the identifying at least one key area corresponding to each key frame image in the key frame sequence includes:

[0028] Calculating pixel differences between adjacent key frame images in the key frame sequence to obtain a pixel change region;

[0029] Counting the motion vectors in each of the key frame images to obtain a motion area;

[0030] Merging the pixel change area and the motion area to obtain a target area;

[0031] The target area is framed based on a minimum circumscribed rectangle to obtain at least one key area corresponding to each key frame image.

[0032] By adopting the above technical solution, by calculating the pixel difference and motion vector of adjacent key frames, it is possible to identify the target area from both static and dynamic dimensions; at the same time, by merging the pixel change area and the motion area, the spatial range of the target activity can be fully captured; further, the target area is selected using the minimum enclosing rectangle, and a regular key area is obtained for subsequent processing, which improves the accuracy and completeness of the key area division and provides a more precise region of interest for the detection of abnormal events.

[0033] Optionally, the performing abnormality detection on the environmental parameters and each of the key areas to obtain abnormal events includes:

[0034] Marking parameters of the environmental parameters that exceed the corresponding preset range as environmental abnormal events, wherein the environmental parameters at least include temperature, humidity and concentration of harmful gases;

[0035] Extract key features of the target person in each of the key areas, and mark the person whose key features do not meet the preset kitchen entry specifications as a person abnormal event, wherein the key features at least include facial information features and person clothing features;

[0036] The trajectory features of the moving targets in the key area are extracted, and the moving targets that meet the preset feature templates are marked as abnormal insect and rodent events.

[0037] By adopting the above technical solution, by setting a preset range of environmental parameters and detecting the exceeding of the range, real-time monitoring of the back kitchen environment safety is achieved; at the same time, by analyzing the facial and clothing features of the target personnel and comparing them with the preset specifications, abnormal situations such as unauthorized personnel and illegal clothing can be discovered in time; further, by extracting the trajectory features of the moving targets and matching them with the insect and rodent feature templates, intelligent detection of insect and rodent invasion is achieved. This multi-dimensional anomaly detection method covers the three key aspects of environmental safety, personnel management and pest control, making the detection of abnormal events in the back kitchen area more comprehensive and accurate.

[0038] Optionally, the calculating the total food safety score of the back kitchen area based on each of the abnormal events, and generating the food safety grade of the back kitchen area according to the total food safety score, includes:

[0039] Count the number of occurrences of each abnormal event within a preset time period, as well as the duration of each abnormal event;

[0040] Calculate the deduction value of each abnormal event according to the number of occurrences, duration and corresponding weight coefficient of each abnormal event;

[0041] The deduction points of each abnormal event are deducted from the initial score to obtain the total food safety score;

[0042] Based on a preset scoring range, a food safety grade corresponding to the total food safety score is determined.

[0043] By adopting the above technical solution, a quantitative assessment of the severity of abnormal situations is achieved by counting the number of occurrences and duration of abnormal events. At the same time, by introducing weight coefficients to calculate the deduction values ​​for various types of abnormal events, the degree of impact of different types of abnormal events can be reflected. The deduction values ​​are further deducted from the initial scores and the safety level is determined based on the scoring range, thus achieving hierarchical management of food safety conditions. This scoring method based on quantitative indicators makes food safety assessment results more objective and standardized, and provides managers with a clear basis for judging safety conditions.

[0044] In a second aspect of the present application, a food safety intelligent assessment system is provided, the system comprising:

[0045] The data acquisition module is used to obtain the monitoring video of the kitchen area and the environmental parameters collected by the sensor network;

[0046] A video processing module, used to extract a key frame sequence of the surveillance video and identify at least one key area corresponding to each key frame image in the key frame sequence;

[0047] An anomaly detection module, used to perform anomaly detection on the environmental parameters and each of the key areas to obtain an abnormal event;

[0048] The grade assessment module is used to calculate the total food safety score of the back kitchen area based on each of the abnormal events, and generate the food safety grade of the back kitchen area according to the total food safety score.

[0049] In a third aspect of the present application, a computer storage medium is provided, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above method steps.

[0050] In a fourth aspect of the present application, an electronic device is provided, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0051] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0052] This application obtains the surveillance video of the back kitchen area and the environmental parameters collected by the sensor network as basic data, extracts the key frame sequence of the surveillance video and identifies the key areas, thereby realizing efficient processing of the video data; at the same time, anomaly detection is performed on the environmental parameters and key areas, so that abnormal events in the back kitchen area can be discovered in a timely manner; further, based on the detected abnormal events, the total food safety score is calculated and the food safety grade is generated, making the food safety assessment of the back kitchen area more objective and quantitative, avoiding the problem of inconsistent judgment standards of different supervisors in the traditional manual experience assessment method, improving the accuracy and efficiency of the food safety assessment in the back kitchen, and effectively improving the food safety management effect in the back kitchen. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of an intelligent food safety assessment method provided in an embodiment of the present application;

[0054] Figure 2 This is a module diagram of an intelligent food safety assessment system provided in an embodiment of the present application;

[0055] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0056] Description of reference numerals: 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0057] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0058] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0059] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0060] The following will provide a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0061] Please refer to Figure 1 , a flowchart of a method for intelligent assessment of food safety is proposed. The method can be implemented by a computer program, a single-chip microcomputer, or run on an intelligent assessment system for food safety. The computer program can be integrated into a computer device or run as an independent tool application. Specifically, the method includes steps 10 to 40, which are as follows:

[0062] Step 10: Obtain the surveillance video of the kitchen area and the environmental parameters collected by the sensor network.

[0063] In the embodiment of the present application, the back kitchen area refers to the area used by a catering enterprise for food processing, storage and cooking, including but not limited to the food preparation area, cutting area, cooking area, cleaning area and food storage area.

[0064] In the embodiment of the present application, the surveillance video refers to the video data collected by the camera installed in the back kitchen area, which can record the operating behavior of the personnel and the food processing process in the back kitchen area.

[0065] In the embodiment of the present application, the sensor network refers to a network system composed of multiple sensor devices deployed in the back kitchen area, including but not limited to temperature sensors, humidity sensors, smoke sensors, and gas sensors.

[0066] In the embodiment of the present application, environmental parameters refer to various environmental indicators of the back kitchen area collected by the sensor network, including but not limited to values ​​such as temperature, humidity, smoke concentration and harmful gas concentration.

[0067] Specifically, the surveillance video collected by the camera in the back kitchen area is obtained in real time; at the same time, a sensor network including temperature sensors, humidity sensors, smoke sensors and gas sensors is deployed in these back kitchen areas to continuously collect environmental parameters such as temperature, humidity, smoke concentration and harmful gas concentration. Among them, the surveillance video collected by the camera can record the staff's clothing and faces; the environmental parameters collected by the sensor network can reflect the environmental conditions of the back kitchen area, which is convenient for monitoring whether there is an abnormal environment that affects food safety. Specifically, the system receives the video data stream transmitted by each camera in real time through the network interface, and regularly reads the values ​​of each sensor in the sensor network through the data acquisition module, and stores the collected surveillance video and environmental parameters in the database, providing a data basis for subsequent abnormal detection and safety assessment.

[0068] Based on the above embodiment, as an optional embodiment, the step of obtaining the monitoring video of the kitchen area and the environmental parameters collected by the sensor network may further include the following steps:

[0069] Step 101: Obtain the current light intensity of the kitchen area.

[0070] Specifically, considering that the lighting conditions in the kitchen area vary at different times, in order to ensure the image quality of the surveillance video, it is necessary to obtain the current light intensity through a light sensor installed in the kitchen area.

[0071] Step 102: Determine the video acquisition parameters of the camera based on the current light intensity. The video acquisition parameters include sampling frame rate, resolution, and exposure parameters.

[0072] Specifically, the lighting environment of the kitchen area is divided into strong lighting environment (greater than 800), standard lighting environment (300-800) and weak lighting environment (less than 300) according to the acquired light intensity. In a strong lighting environment, a lower sampling frame rate (15 frames / second) and a higher resolution (1920×1080 pixels) are set, and the exposure parameters are reduced to avoid overexposure; in a standard lighting environment, a medium sampling frame rate (20 frames / second) and resolution (1280×720 pixels) are used, and standard exposure parameters are used to maintain picture clarity; in a weak lighting environment, the sampling frame rate (30 frames / second) is increased and the resolution (640×480 pixels) is reduced, and the exposure parameters are increased to improve the picture brightness. By adaptively adjusting the video acquisition parameters based on light intensity, clear monitoring images can be obtained in different lighting environments, which not only ensures the accuracy of abnormal behavior detection, but also avoids the waste of storage space.

[0073] Step 103: Determine the sampling frequency of each sensor in the sensor network according to the historical environmental parameters of the back kitchen area.

[0074] Specifically, considering that the environmental parameters in the back kitchen area fluctuate differently at different times, in order to improve the efficiency of data collection, the system determines the sampling frequency of the corresponding sensor based on the historical data analysis of the changing rules of each environmental parameter. The historical data of environmental parameters in the past month can be obtained to calculate the fluctuation range of each parameter at different times. For parameters with large fluctuations, such as the temperature and smoke concentration in the cooking area, the sampling frequency of the corresponding sensor is set to a higher frequency (such as once every 10 seconds); for parameters with moderate fluctuations, such as the temperature and humidity in the cutting area, a medium sampling frequency (such as once every 30 seconds) is used; for parameters with small fluctuations, such as the temperature and humidity in the food storage area, a lower sampling frequency (such as once every 5 minutes) is used.

[0075] Step 104: Obtain the monitoring video collected by the camera according to the video collection parameters, and the environmental parameters collected by the sensor network according to the sampling frequency.

[0076] Specifically, the environmental parameters collected by the sensor network according to the set sampling frequency are obtained, and the monitoring video is obtained from the camera according to the video acquisition parameters determined by the aforementioned light intensity. This method of dynamically adjusting the sampling frequency based on historical data not only ensures the timely monitoring of important parameters, but also avoids the waste of data acquisition resources and improves the efficiency of system operation. By storing video data and environmental parameter data in time synchronization, complete and multi-dimensional data support is provided for subsequent anomaly detection.

[0077] Step 20: extract a key frame sequence of the surveillance video, and identify at least one key area corresponding to each key frame image in the key frame sequence.

[0078] In the embodiment of the present application, the key frame sequence refers to an image sequence containing important visual information extracted from the surveillance video, wherein each key frame represents a moment when the scene or content in the video changes significantly, and is used to record key operational behaviors and scene changes in the back kitchen area, such as the important moment when people or insects and rodents enter a certain area.

[0079] In the embodiment of the present application, the key area refers to the area in the key frame image that is selected by the target detection algorithm and contains operators, insects and rodents, or pixel changes. For example, the key area contains the complete morphological features of the person, which is used for subsequent detection of the person's identity, work clothes, protective equipment, etc.

[0080] Specifically, in order to improve the processing efficiency of surveillance videos, the system first needs to extract key frame sequences containing important information from a large amount of video data. Specifically, a scene change detection algorithm based on deep learning is used to calculate the image pixel differences between adjacent video frames. When the pixel change amplitude of a certain area exceeds a preset threshold, the frame is saved as a key frame, thereby obtaining a key frame sequence reflecting the important scene changes in the back kitchen area. After obtaining the key frame sequence, the system performs pixel change area analysis on each key frame image, and selects the continuous pixel area with significant difference as the key area by calculating the pixel difference matrix between the current key frame and the previous key frame. These key areas may contain scene change information such as personnel activities, object movement, and equipment status changes. For each key frame image, the system can identify multiple key areas to cope with the situation where multiple changes occur simultaneously in the picture. This key frame and key area extraction method based on pixel changes can fully capture the dynamic change information in the back kitchen area, which not only avoids repeated processing of static scenes, but also provides accurate analysis objects for subsequent anomaly detection, improving the processing efficiency and detection accuracy of the system.

[0081] Based on the above embodiment, as an optional embodiment, the step of extracting the key frame sequence of the surveillance video may further include the following steps:

[0082] Step 2011: Determine the target activity area in the surveillance video.

[0083] Specifically, in order to accurately identify effective scene changes in surveillance videos, it is necessary to first calculate the target activity area. By calculating the inter-frame differences of adjacent video frames within the time window, the area where the pixel value changes exceed the preset threshold is obtained as the pixel value change area; the dense optical flow method based on the Lucas-Kanade algorithm is used to calculate the motion vector field of the target area, and the motion displacement and direction information are extracted to obtain the motion speed characteristics; the contour extraction algorithm is used to perform morphological processing on the target area, and the geometric parameters such as the perimeter, area, and aspect ratio of the target are obtained as shape features. These three features are weighted and fused to construct a feature description of the target activity area, thereby effectively filtering out interference information caused by environmental factors such as lighting changes and camera jitter while retaining effective activity information. This target activity area calculation method based on multi-feature fusion not only improves the robustness of target detection, but also provides reliable feature support for subsequent key frame extraction and abnormal behavior recognition.

[0084] Step 2012: Determine candidate key frames based on the target activity area.

[0085] Specifically, in order to filter out video frames with real information value from the surveillance video, the system sets up a multi-level judgment mechanism based on the characteristics of the target activity area to determine the candidate key frames. First, the area of ​​the target activity area is calculated. When the area is greater than the first threshold (for example, 20% of the screen), it indicates that a large-scale scene change has occurred, and the frame is marked as a candidate key frame; when the area of ​​the target activity area is between the first threshold and the second threshold (for example, 5%-20%), the motion speed characteristics of the area are further judged. If the motion speed exceeds the speed threshold (for example, 2 meters / second), it indicates that there is a small target such as a mouse that moves quickly, and the frame is also marked as a candidate key frame; in addition, even if the area and speed of the target activity area are small, if its shape characteristics meet the pre-defined feature template (such as the outline of insects and mice, etc.), the frame will also be marked as a candidate key frame. This multi-level judgment mechanism not only takes into account the characteristics of different types of scene changes, but also effectively avoids missing important information, thereby improving the selection quality of candidate key frames.

[0086] Step 2013: Calculate the similarity between the candidate key frame and the previous key frame.

[0087] Step 2014: When the similarity is less than the similarity threshold, the candidate key frame is determined as a key frame.

[0088] Specifically, in order to avoid repeated or similar scenes from being repeatedly extracted, the candidate key frames need to be further screened. The image similarity calculation method can be used to calculate the similarity between the current candidate key frame and the previous key frame that has been determined. The similarity is obtained by extracting the color histogram features and texture features of the two frames of images and calculating the cosine distance of these features. When the calculated similarity is less than the preset similarity threshold (for example, 0.85), it indicates that the current candidate key frame contains scene information that is significantly different from the previous key frame, and the candidate key frame is determined as a key frame.

[0089] Step 2015: Arrange the key frames in chronological order to form a key frame sequence.

[0090] Specifically, after all candidate key frames are screened, the remaining key frames are sorted by timestamp to form a key frame sequence that reflects scene changes. For example, when a worker enters the screen (forming the first key frame), then performs cutting and mixing operations at the operating table (forming the second key frame), then turns around to take and put seasonings (forming the third key frame), and finally leaves the screen (forming the fourth key frame), the system will extract the images of these four key moments in turn and form a key frame sequence that reflects the operation process in chronological order.

[0091] Based on the above embodiment, as an optional embodiment, the step of identifying at least one key area corresponding to each key frame image in the key frame sequence may further include the following steps:

[0092] Step 2021: Calculate the pixel difference between adjacent key frame images in the key frame sequence to obtain the pixel change area.

[0093] Specifically, in order to accurately locate the scene change position and target motion in the key frame sequence, it is necessary to perform pixel-level and motion feature analysis on the key frames. First, the pixel difference matrix between adjacent key frames in the key frame sequence is calculated, and the matrix is ​​binarized by setting a difference threshold. The areas with significant differences are marked as pixel change areas, which usually correspond to the parts of the scene that have changed.

[0094] Step 2022: Count the motion vectors in each key frame image to obtain the motion area.

[0095] Specifically, the optical flow estimation algorithm is used to count the pixel motion vector field in each key frame image. By performing cluster analysis on the direction and amplitude of the motion vector, continuous areas with similar motion characteristics are obtained as motion areas. These areas reflect the position and motion trend of the moving target in the picture.

[0096] Step 2023: Merge the pixel change area and the motion area to obtain the target area.

[0097] Specifically, in order to accurately define the areas that need to be focused on in the key frame, it is necessary to integrate and analyze the pixel change areas and motion areas obtained above. The region merging algorithm is used to merge the pixel change areas and motion areas that have spatial overlap or a distance less than the preset distance threshold to obtain a complete target area. The merging process eliminates the small gaps between regions through the expansion and corrosion operations in morphological processing, connects adjacent areas into a whole, and obtains a more complete target area outline. In addition, by clustering the direction and amplitude of the motion vector, continuous areas with similar motion characteristics are obtained as motion areas. These areas reflect the position and motion trend of the moving target in the picture. The system associates the pixel change area with the motion area for analysis, which can more accurately determine whether the scene change is caused by target motion or other factors (such as lighting changes).

[0098] Step 2024: Select the target area based on the minimum bounding rectangle to obtain at least one key area corresponding to each key frame image.

[0099] Specifically, the target area is selected by the minimum enclosing rectangle, that is, the minimum enclosing rectangle of each target area is calculated, that is, the minimum rectangular frame that can completely surround the target area, and the rectangular frame is defined as the key area. For each key frame image, it may contain multiple target areas, so multiple key areas are generated accordingly. For example, when multiple staff members operate at different positions of the operating table at the same time, a key area is generated at each operating position, thereby realizing simultaneous tracking of multiple activity areas. This key area extraction method based on region merging and minimum enclosing rectangle not only ensures the integrity of the target area, but also provides a regular rectangular boundary, which is convenient for subsequent feature extraction and behavior analysis.

[0100] Step 30: Perform anomaly detection on environmental parameters and key areas to obtain abnormal events.

[0101] Specifically, in order to comprehensively monitor the safety of the back kitchen area, the embodiment of the present application performs abnormality detection from three dimensions: environmental safety, personnel norms, and pest control. Environmental parameters, including temperature, humidity, and harmful gas concentration, are collected in real time through sensors. When a parameter value continues to exceed its preset safety range, such as temperature exceeding 40°C or humidity exceeding 85% or harmful gas concentration exceeding the standard, it is marked as an environmental abnormality event. At the same time, the personnel in the key area are analyzed for characteristics, facial information features are extracted for identity recognition, and personnel clothing features are extracted through image segmentation algorithms, including the wearing of protective equipment such as work hats, masks, and gloves. When unauthorized personnel or those who are not dressed in accordance with the regulations are detected, they are marked as personnel abnormal events. In addition, the trajectory features of moving targets in key areas can be extracted through target tracking algorithms, including movement speed, direction, and path, and matched with pre-established pest and mouse motion feature templates. When a moving target that meets the characteristics of pests and mice is found, it is marked as an pest and mouse abnormal event. This multi-dimensional abnormality detection method realizes all-round supervision of environmental safety, personnel management, and pest control in the back kitchen area, and improves the level of food safety protection.

[0102] Based on the above embodiment, as an optional embodiment, the step of performing abnormal detection on environmental parameters and key areas to obtain abnormal events may also include the following steps:

[0103] Step 301: Marking parameters in the environmental parameters that exceed the corresponding preset range as environmental abnormality events, where the environmental parameters at least include temperature, humidity and concentration of harmful gases.

[0104] Specifically, through the sensor network distributed in the back kitchen area, environmental parameters including temperature, humidity and harmful gas concentration are collected in real time. Based on food safety specifications and equipment operation requirements, the system sets a safety threshold range for each environmental parameter, such as the safety range of temperature is 0-35°C, the safety range of humidity is 30%-75%, and the safety range of harmful gas concentration (such as carbon monoxide) is 0-50ppm. When it is detected that a parameter value exceeds its preset range for multiple consecutive sampling cycles, the corresponding environmental abnormality event will be triggered. For example, when it is detected that the temperature in the frying area continues to exceed 35°C and is on an upward trend, the system will mark it as a temperature abnormality event; when it is detected that the humidity in the cleaning area continues to exceed 75%, the system will mark it as a humidity abnormality event; when it is detected that the concentration of harmful gases in the gas area exceeds the safety threshold, the system will mark it as a gas leakage abnormality event.

[0105] Step 302: extract key features of the target person in each key area, and mark the person whose key features do not meet the preset kitchen entry specifications as a person abnormal event, where the key features at least include facial information features and person clothing features.

[0106] Specifically, the key features of the target person are extracted from the key area image through the deep learning model, including facial information features and clothing features. For facial information features, the system uses a face recognition algorithm to extract facial feature vectors, matches them with the pre-established authorized personnel facial feature library, and determines whether the person is an authorized kitchen person. For clothing features, the image segmentation and target detection algorithms are used to identify the clothing of the person, including but not limited to the color of the work clothes, the style of the chef's hat, the wearing of masks and other features, and compare them with the preset kitchen personnel specifications. When the system detects that the facial information features or the clothing features of the person in the key features do not meet the specifications, a personnel abnormal event will be triggered. For example, one day the system detected a staff member in the back kitchen area wearing a gray work suit instead of the prescribed white work suit, and wearing an ordinary hat instead of a white chef's hat, then the situation was marked as a personnel abnormal event of dress violation; for another example, when the system detects that a stranger appears in the back kitchen area, it cannot match the corresponding facial information in the authorized personnel library, so it is marked as an abnormal event of unauthorized intrusion. This personnel anomaly detection method based on multi-dimensional features realizes the intelligent management of the identity and dress code of kitchen staff, and effectively prevents security risks such as unauthorized intrusion and illegal operations.

[0107] Step 303: extracting the trajectory features of the moving targets in the key area, and marking the moving targets that meet the preset feature template as abnormal insect and rodent events.

[0108] Specifically, the target tracking algorithm is used to extract the trajectory features of the moving target from the key area image sequence, including characteristic parameters such as movement speed, movement direction, activity range and movement continuity. The system has pre-established a feature template library for insect and rodent targets, which contains typical movement features of common insects and rodents such as mice and cockroaches, such as fast and irregular movement patterns, movement trajectories close to corners or the ground, and frequent activities at night. When the matching degree of the trajectory features of the moving target detected by the system with the preset insect and rodent feature template exceeds the threshold, the target is marked as an abnormal insect and rodent event. For example, at night, the system detected a small moving target in the back kitchen storage area. The target showed the characteristics of rapid movement close to the ground, and the movement trajectory showed an irregular broken line shape, and it stopped at the corner of the wall many times. After matching its trajectory features with the insect and rodent feature template, the system found that the similarity was as high as 95%, and then marked the situation as an abnormal insect and rodent event suspected of mouse invasion.

[0109] Step 40: Calculate the total food safety score of the back kitchen area based on each abnormal event, and generate the food safety grade of the back kitchen area according to the total food safety score.

[0110] Specifically, different deduction weights are set for different types of abnormal events. Environmental abnormal events are deducted according to the degree of parameter exceeding the standard, including the product of the exceeding value of three types of parameters, temperature exceeding the standard, humidity exceeding the standard and harmful gas exceeding the standard and the corresponding deduction coefficient; personnel abnormal events are deducted according to the degree of violation, including the product of the number of abnormal behaviors of unauthorized intrusion and failure to dress in accordance with the standard and the corresponding deduction coefficient; insect and rodent abnormal events are deducted according to the number of times they are found, multiplying the number of detections by the corresponding deduction coefficient. The system takes the full score of 100 as the basis, and dynamically calculates the deduction points according to the abnormal events that occur and their duration during the detection cycle to obtain the total food safety score of the back kitchen area. Then, the safety level is set according to the total score, and each score interval corresponds to a safety level, so that the total food safety score can be mapped to the food safety level. This scoring and grading method based on abnormal events can not only quantitatively reflect the food safety status of the back kitchen area, but also provide managers with clear rectification direction and priority guidance, improving the scientificity and efficiency of food safety management.

[0111] Based on the above embodiment, as an optional embodiment, the step of calculating the total food safety score of the back kitchen area based on each abnormal event and generating the food safety level of the back kitchen area according to the total food safety score may further include the following steps:

[0112] Step 401: Count the number of occurrences of each abnormal event within a preset time period, and the duration of each abnormal event.

[0113] Specifically, within a preset time period (such as 12 hours), the number of occurrences and durations of environmental abnormal events, personnel abnormal events, and pest and rodent abnormal events are counted separately. For environmental abnormal events, the system records the number of times that parameters such as temperature, humidity, and harmful gas concentration exceed the standard, as well as the cumulative duration of each parameter value being in an excessive state; for personnel abnormal events, the system records the number of violations such as unauthorized intrusion and improper dress of personnel, as well as the duration of the violation state; for pest and rodent abnormal events, the system records the number of times pests and rodents are detected, as well as the duration of their activities in the monitored area.

[0114] Step 402: Calculate the deduction value of each abnormal event according to the number of occurrences, duration and corresponding weight coefficient of each abnormal event.

[0115] Specifically, according to the pre-set weight coefficient, the number of occurrences and duration of abnormal events are multiplied by the corresponding weight coefficient to calculate the deduction value of each type of abnormal event. For example, within a 12-hour detection period, the system detects that the temperature parameter exceeds the standard twice, and the cumulative duration is 30 minutes. If the weight of the number of temperature exceeding the standard is 2 points / time, and the weight of the duration is 0.1 points / minute, then the deduction value of the abnormal temperature event is 2×2+30×0.1=7 points. This abnormal event scoring method based on multi-dimensional indicators not only takes into account the frequency of abnormal events, but also pays attention to the duration of abnormal conditions, which can more comprehensively and objectively reflect the degree of food safety risk in the back kitchen area.

[0116] Step 403: deduct the deduction value of each abnormal event from the initial score to obtain the total food safety score.

[0117] Step 404: Based on a preset scoring range, determine the food safety level corresponding to the total food safety score.

[0118] Specifically, the embodiment of the present application uses 100 points as the initial score, and deducts the calculated deduction values ​​for environmental abnormal events, personnel abnormal events, and insect and rodent abnormal events from the initial score in turn, to obtain a total food safety score that reflects the current safety status of the back kitchen area. The total score is then compared with the pre-set score range to determine the corresponding food safety level, where 90 points and above correspond to level A, indicating that the back kitchen area is in excellent operating condition; 80 to 89 points correspond to level B, indicating that the operating condition is good; 70 to 79 points correspond to level C, indicating that there are certain safety hazards that need to be improved in a timely manner; and 70 points or less correspond to level D, indicating that there are serious safety hazards that need to be rectified immediately. This method of determining food safety levels based on quantitative scoring can intuitively reflect the safety management level of the back kitchen area, facilitate management personnel to quickly understand the safety status and take corresponding management measures, and improve the accuracy and efficiency of food safety management.

[0119] See also Figure 2 , is a module diagram of a food safety intelligent assessment system provided in an embodiment of the present application, the food safety intelligent assessment system may include: a data acquisition module, a video processing module, an anomaly detection module and a grade assessment module, wherein:

[0120] The data acquisition module is used to obtain the monitoring video of the kitchen area and the environmental parameters collected by the sensor network;

[0121] A video processing module, used to extract a key frame sequence of the surveillance video and identify at least one key area corresponding to each key frame image in the key frame sequence;

[0122] An anomaly detection module, used to perform anomaly detection on the environmental parameters and each of the key areas to obtain an abnormal event;

[0123] The grade assessment module is used to calculate the total food safety score of the back kitchen area based on each of the abnormal events, and generate the food safety grade of the back kitchen area according to the total food safety score.

[0124] Optionally, the data acquisition module is further used to obtain the current light intensity of the back kitchen area;

[0125] Determine the video acquisition parameters of the camera based on the current light intensity, the video acquisition parameters including sampling frame rate, resolution and exposure parameters;

[0126] Determine the sampling frequency of each sensor in the sensor network according to the historical environmental parameters of the back kitchen area;

[0127] The monitoring video acquired by the camera according to the video acquisition parameters and the environmental parameters acquired by the sensor network according to the sampling frequency are acquired.

[0128] Optionally, the video processing module is further used to determine the target activity area in the surveillance video;

[0129] Determining candidate key frames based on the target activity area;

[0130] Calculating the similarity between the candidate key frame and the previous key frame;

[0131] When the similarity is less than a similarity threshold, determining the candidate key frame as a key frame;

[0132] The key frames are arranged in time sequence to form a key frame sequence.

[0133] Optionally, the video processing module is also used to mark the corresponding video frame as a candidate key frame when the area of ​​the target activity area is greater than a first threshold, or the area of ​​the target activity area is less than the first threshold and greater than the second threshold and the movement speed is greater than a speed threshold, or the target activity area meets a preset shape feature.

[0134] Optionally, the video processing module is further used to calculate the pixel difference between adjacent key frame images in the key frame sequence to obtain a pixel change area;

[0135] Counting the motion vectors in each of the key frame images to obtain a motion area;

[0136] Merging the pixel change area and the motion area to obtain a target area;

[0137] The target area is framed based on a minimum circumscribed rectangle to obtain at least one key area corresponding to each key frame image.

[0138] Optionally, the anomaly detection module is further used to mark parameters of the environmental parameters that exceed the corresponding preset range as environmental anomaly events, and the environmental parameters at least include temperature, humidity and concentration of harmful gases;

[0139] Extract key features of the target person in each of the key areas, and mark the person whose key features do not meet the preset kitchen entry specifications as a person abnormal event, wherein the key features at least include facial information features and person clothing features;

[0140] The trajectory features of the moving targets in the key area are extracted, and the moving targets that meet the preset feature templates are marked as abnormal insect and rodent events.

[0141] Optionally, the rating module is further used to count the number of occurrences of each abnormal event within a preset time period, and the duration of each abnormal event;

[0142] Calculate the deduction value of each abnormal event according to the number of occurrences, duration and corresponding weight coefficient of each abnormal event;

[0143] The deduction points of each abnormal event are deducted from the initial score to obtain the total food safety score;

[0144] Based on a preset scoring range, a food safety grade corresponding to the total food safety score is determined.

[0145] It should be noted that: when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0146] An embodiment of the present application also provides a computer storage medium, which can store multiple instructions. The instructions are suitable for being loaded by a processor and executing an intelligent food safety assessment method of the above embodiment. The specific execution process can be found in the specific description of the above embodiment, which will not be repeated here.

[0147] Please refer to Figure 3 The application also discloses an electronic device. Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0148] The communication bus 302 is used to realize the connection and communication between these components.

[0149] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0150] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0151] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0152] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally also be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3 , the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of an intelligent food safety assessment method.

[0153] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program storing a method for intelligent assessment of food safety in the memory 305, and when executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0154] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0157] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or can exist separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional unit.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0159] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure.

[0160] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An intelligent food safety assessment method, characterized in that: The method comprises: Obtain surveillance video of the kitchen area and environmental parameters collected by the sensor network; Determining a target activity area in the surveillance video; When the area of ​​the target activity region is greater than a first threshold, or the area of ​​the target activity region is less than the first threshold and greater than the second threshold and the motion speed is greater than the speed threshold, or the target activity region meets a preset shape feature, marking the corresponding video frame as a candidate key frame; Calculating the similarity between the candidate key frame and the previous key frame; When the similarity is less than a similarity threshold, determining the candidate key frame as a key frame; Arranging the key frames in time sequence to form a key frame sequence; Calculating pixel differences between adjacent key frame images in the key frame sequence to obtain a pixel change region; Counting the motion vectors in each of the key frame images, and performing cluster analysis on the directions and amplitudes of the motion vectors to obtain continuous regions with similar motion characteristics as motion regions; Merging the pixel change area and the motion area to obtain a target area; Select the target area based on the minimum circumscribed rectangle, and obtain at least one key area corresponding to each key frame image, wherein the key area refers to an area containing an operator or an insect or a rodent or a pixel change, which is selected by a target detection algorithm in the key frame image; Performing abnormality detection on the environmental parameters and each of the key areas to obtain abnormal events; The total food safety score of the back kitchen area is calculated based on each of the abnormal events, and the food safety grade of the back kitchen area is generated according to the total food safety score.

2. The intelligent food safety assessment method according to claim 1, characterized in that: The acquisition of the monitoring video of the kitchen area and the environmental parameters collected by the sensor network includes: Obtain the current light intensity of the kitchen area; Determine the video acquisition parameters of the camera based on the current light intensity, the video acquisition parameters including sampling frame rate, resolution and exposure parameters; Determine the sampling frequency of each sensor in the sensor network according to the historical environmental parameters of the back kitchen area; The monitoring video acquired by the camera according to the video acquisition parameters and the environmental parameters acquired by the sensor network according to the sampling frequency are acquired.

3. The intelligent food safety assessment method according to claim 1, characterized in that: The abnormality detection of the environmental parameters and the key areas to obtain abnormal events includes: Marking parameters of the environmental parameters that exceed the corresponding preset range as environmental abnormal events, wherein the environmental parameters at least include temperature, humidity and concentration of harmful gases; Extract key features of the target person in each of the key areas, and mark the person whose key features do not meet the preset kitchen entry standards as a person abnormal event, wherein the key features at least include facial information features and person clothing features; The trajectory features of the moving targets in the key area are extracted, and the moving targets that meet the preset feature templates are marked as abnormal insect and rodent events.

4. The intelligent food safety assessment method according to claim 1, characterized in that: The calculating the total food safety score of the back kitchen area based on each of the abnormal events, and generating the food safety grade of the back kitchen area according to the total food safety score, comprises: Count the number of occurrences of each abnormal event within a preset time period, as well as the duration of each abnormal event; Calculate the deduction value of each abnormal event according to the number of occurrences, duration and corresponding weight coefficient of each abnormal event; The deduction points of each abnormal event are deducted from the initial score to obtain the total food safety score; Based on a preset scoring range, a food safety grade corresponding to the total food safety score is determined.

5. An intelligent food safety assessment system, characterized in that: The system comprises: The data acquisition module is used to obtain the monitoring video of the kitchen area and the environmental parameters collected by the sensor network; A video processing module is used to determine the target activity area in the monitoring video; when the area of ​​the target activity area is greater than a first threshold, or the area of ​​the target activity area is less than the first threshold and greater than the second threshold and the motion speed is greater than the speed threshold, or the target activity area meets the preset shape feature, the corresponding video frame is marked as a candidate key frame; the similarity between the candidate key frame and the previous key frame is calculated; when the similarity is less than the similarity threshold, the candidate key frame is determined as a key frame; the key frames are arranged in chronological order to form a key frame sequence; the pixel difference between adjacent key frame images in the key frame sequence is calculated to obtain a pixel change area; the motion vectors in each of the key frame images are counted, and a continuous area with similar motion characteristics is obtained as a motion area by clustering analysis of the direction and amplitude of the motion vector; the pixel change area and the motion area are merged to obtain a target area; the target area is framed based on the minimum circumscribed rectangle to obtain at least one key area corresponding to each key frame image, and the key area refers to the area containing operators or insects and rats or pixel changes framed by the target detection algorithm in the key frame image; An anomaly detection module, used to perform anomaly detection on the environmental parameters and each of the key areas to obtain an abnormal event; The grade assessment module is used to calculate the total food safety score of the back kitchen area based on each of the abnormal events, and generate the food safety grade of the back kitchen area according to the total food safety score.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 4.

7. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-4.

Images