Multi-area live-fire leaving detection method and system based on target recognition and medium

Abstract

The invention discloses a target recognition-based multi-region fire-moving and person-leaving detection method and system and a medium. The method comprises the following steps of: detecting features and coordinates of persons and flames on real-time acquired images based on a trained target recognition model; and connecting lines on the acquired images to form a live area, detecting whether the live area is in a to-be-detected area by combining a ray crossing method, if so, judging whether the live area has flame characteristics and has no operating personnel based on the characteristics of the personnel and / or flame output by the target identification model, and if so, triggering an early warning mechanism. An advanced computer vision technology and an image processing algorithm are utilized, real-time monitoring is carried out in multiple operation areas, whether a chef is on duty or not is effectively detected, and potential safety hazards caused by the fact that fire leaves people are prevented. And the specific position where the abnormity occurs can be quickly positioned through multi-area monitoring, so that a manager can take measures in time, and the safety management level of the kitchen is remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of kitchen target detection, and in particular relates to a multi-region fire and person detection method, system and medium based on target recognition. Background Art

[0002] Open flames and no one is allowed to cook refers to the situation where the chef leaves the open flame area and fails to extinguish the flame in time when cooking with open flames in the kitchen. This situation may lead to serious safety accidents. Once the chef leaves the open flame area and the flame is not controlled in time, it may cause a fire. With the development of the catering industry, kitchen safety issues have received more and more attention, especially in commercial kitchens, where open flames are often used during cooking, and the risk of fire is high. Traditional kitchen safety management methods mainly include passive measures such as manual inspections and installation of smoke alarms. However, these methods have many shortcomings:

[0003] ① First, manual inspection is not only inefficient, but also easily affected by human factors, resulting in unstable monitoring results;

[0004] ②Secondly, although installing equipment such as smoke alarms can sound an alarm at the early stage of a fire, they are passive response measures and cannot effectively prevent a fire before it occurs;

[0005] ③In addition, there are many problems with manual video surveillance. For example, long-term monitoring can easily cause fatigue to the monitoring personnel, which leads to a decrease in attention and inability to detect the chef's departure in time.

[0006] In recent years, with the development of computer technology, it has become possible to monitor the activities in the kitchen in real time through camera monitoring and image processing technology. Target recognition technology can identify the location and behavior patterns of personnel, thereby realizing effective monitoring of whether the chef is on duty. Compared with traditional means, this method has higher real-time and accuracy, can detect abnormal situations and issue timely warnings at the first time, and effectively reduce the risk of fire. However, the existing monitoring system based on target recognition still has certain limitations, such as insufficient adaptability to complex environments and high false alarm rates. Especially for large kitchens or designated hot work areas, the existing single-area monitoring method is difficult to fully cover various key areas, and it is also impossible to control the time when the operating personnel leave the hot work area when the hot work is started. Therefore, the present invention designs a multi-area hot work departure real-time detection scheme based on target recognition, which is of great significance to improving the safety management level of the kitchen. Summary of the invention

[0007] The purpose of the present invention is to provide a multi-zone fire and person detection method, system and medium based on target recognition, aiming to solve the above-mentioned problems.

[0008] The present invention is mainly achieved through the following technical solutions:

[0009] The multi-region moving fire and leaving person detection method based on target recognition includes the following steps:

[0010] Step S1: Video acquisition: obtain surveillance video in a fixed area and convert it into multiple continuous frames of images;

[0011] Step S2: Target recognition: Input the image collected in step S1 into the trained target recognition model, and output the features and coordinates of the person and the flame;

[0012] Step S3: Area delineation and detection: Using the clear image of the fixed area as the background, delineate multiple areas to be detected on the captured image; based on the coordinates output in step S2, connect the lines on the captured image to form a hot work area; combine the ray crossing method to detect whether the hot work area is within the area to be detected, if so, proceed to step S4;

[0013] Step S4: Detection of personnel absence during fire operation: Based on the characteristics of personnel and / or flames output in step S2, determine whether there are flame characteristics and no operators in the fire operation area. If so, trigger the early warning mechanism.

[0014] In order to better implement the present invention, further, in step S2, training the target recognition model includes the following steps:

[0015] Step S21: Use the YOLOv8 model to build a target recognition model;

[0016] Step S22: Collect pictures of flames and people in the kitchen scene, and use a labeling tool to label the target borders and categories in each picture to form a training set, a verification set, and a test set;

[0017] Step S23: Building a training environment, using the training set to train the target recognition model, and then using the validation set and the test set to correct the target recognition model in turn, and finally obtaining the trained target recognition model.

[0018] In order to better implement the present invention, further, step S3 includes the following steps:

[0019] Step S31: Obtain the four vertex coordinates P = {(x1, y1), (x2, y1), (x2, y2), (x1, y2)} and the center coordinates of the rectangular coordinates of the person and / or flame output by the target recognition model

[0020] Step S32: Assume that the vertices of the region to be detected are T = {(x1, y1), (x2, y2), ..., (x i ,y i),…,(x n ,y n )} and the test point is (P x ,P y );

[0021] Step S33: Substitute the four vertex coordinates P and the center coordinate C of the rectangular coordinates into the following formulas to determine whether the hot work area is within the area to be detected:

[0022]

[0023] If there exists t ≥ 0 and x i ≤P x +t≤x i+1 or x i+1 ≤P x +t≤x i , it is considered that there is an intersection between the hot fire area and the area to be detected, and the personnel and / or flames are in the area to be detected.

[0024] In order to better implement the present invention, further, in the step S4, the early warning mechanism is: triggering the fire-related absence counter and starting a countdown. If the operator returns to the fire-related area before the countdown ends, the fire-related absence counter is reset; if the operator does not return to the fire-related area before the countdown ends, the presence of fire-related absence is detected and early warning measures are triggered.

[0025] In order to better implement the present invention, further, step S4 includes the following steps:

[0026] Step S41: First, if it is detected that the hot work area is within the designated area to be detected, then based on the output of the target recognition model, it is determined whether there is an operator in the hot work area. If not, proceed to step S42, otherwise end;

[0027] Step S42: Based on the output of the target recognition model, determine whether there is a fire feature. If yes, proceed to step S43, otherwise end;

[0028] Step S43: trigger the hot work and no-personnel counter of the area to start timing. If an operator is detected before the timing ends, proceed to step S41, otherwise it is determined that there is a hot work and no-personnel operation in the area.

[0029] The present invention is mainly achieved through the following technical solutions:

[0030] A multi-region active fire and absence detection system based on target recognition is based on the above method and includes a model training module, a target recognition module, a region division module and an active fire and absence detection module; the model training module is used to build a target recognition model, and in combination with a python script in a training environment, uses a training set to train the target recognition model, and outputs the trained target recognition model; the target recognition module is used to use the trained target recognition model to recognize the features and coordinates of people and flames on an input image; the region division module is used to delineate multiple areas to be detected on the image to be detected, and detects whether the active fire area formed based on the coordinates is within the area to be detected based on the ray intersection method; the active fire and absence detection module is used to detect whether there is a flame feature and no operating personnel in the active fire area within the area to be detected, and trigger an early warning mechanism.

[0031] A computer-readable storage medium stores a computer program, which implements the above method when executed by a processor.

[0032] The beneficial effects of the present invention are as follows:

[0033] The present invention utilizes advanced computer vision technology and image processing algorithms to monitor in real time in multiple operating areas, effectively detect whether the chef is on duty, prevent safety hazards caused by "being away from the chef when the fire is on", and effectively improve the safety management level of the kitchen. The present invention divides the kitchen into multiple independent operating areas to ensure that each key area can be effectively monitored. This method overcomes the limitations of single-area monitoring and improves the coverage of overall monitoring. The multi-area division can be flexibly adjusted according to the actual kitchen layout and is suitable for kitchen environments of different sizes and structures, ensuring that each key area can be effectively monitored, ensuring the comprehensiveness and pertinence of the monitoring. Through this multi-area monitoring, the present invention can quickly locate the specific location where the abnormality occurs, making it convenient for managers to take timely measures, thereby significantly improving the safety management level of the kitchen.

[0034] The present invention allows users to set different time thresholds for fire-free and unattended according to actual conditions, so as to more flexibly adapt to the needs of different scenarios and ensure the accuracy and flexibility of the system. For example, a shorter time threshold can be set during busy hours, and it can be appropriately extended during non-peak hours. Combining fire-free and unattended detection and customized time thresholds, the fire-free and unattended behavior can be quickly judged. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the overall principle of the multi-region active fire and absent person detection system for target identification of the present invention;

[0036] Figure 2 It is a schematic diagram of the principle of regional division of the present invention;

[0037] Figure 3 The present invention is a flow chart of the multi-region moving fire and leaving person detection method for target recognition. DETAILED DESCRIPTION

[0038] Embodiment 1:

[0039] A multi-region fire and person detection method based on target recognition comprises the following steps:

[0040] Step 1: Video Capture:

[0041] Use a digital camera to shoot digital video in a fixed area and convert it into multiple frames of continuous images.

[0042] Step 2: Target Identification:

[0043] Using YOLOv8 as the target recognition model, combined with pyqt5 (graphical interface library) and python scripts, we finally trained a target recognition model that can output the features and coordinates of people and flames based on images.

[0044] Step 3: Area demarcation and detection:

[0045] like Figure 2 As shown, the first clear frame image of a fixed area is used as the background to divide multiple areas to be detected; on the real-time acquired image, based on the coordinates output by the target recognition model, multi-point lines are drawn to form a closed polygonal area as the active fire area; at the same time, combined with the ray intersection method, the personnel and / or flame features and coordinates in the active fire area are traversed to determine whether the active fire area is in the area to be detected. If so, proceed to step 4.

[0046] Step 4: Fire and human behavior detection:

[0047] When flame characteristics are identified in the hot work area and there are no workers, the hot work absence counter is triggered and the countdown begins according to the configured allowed hot work absence time. If the worker returns to the hot work area before the countdown ends, the counter will be reset. If the worker has not returned to the hot work area before the countdown ends, the detection result is that the worker has left the hot work area, and the corresponding early warning measures can be triggered later.

[0048] Preferably, in step 2, the YOLOv8 (deep learning framework) target recognition model is used to train and identify personnel and flame features, and pictures of flames and personnel in the kitchen scene are collected, and the target borders and categories in each picture are annotated using annotation tools. A total of 976 pictures are included, of which the training set includes 875 pictures, the verification set includes 46 pictures, and the test set includes 54 pictures. A model training environment is built and combined with a python script, and finally a target recognition model that can output the characteristics and coordinates of personnel and flames according to the image is trained.

[0049] To build a training environment, you need to install the Python environment and related training support libraries, and finally call related library functions through Python scripts for training.

[0050] For example: train.py script

[0051] from ultralytics import YOLO

[0052] model=YOLO(myYOLO.yaml').load('. / wt / yolov8n.pt')

[0053] model.train(data='. / Mydata.yaml',epochs=200,imgsz=320)

[0054] like Figure 1 As shown, the relevant libraries include:

[0055] torchvision-GPU library: A GPU-accelerated version of torchvision, used for deep learning of image processing, accelerating image data loading, preprocessing, and model training.

[0056] torch-GPU library: GPU version of the PyTorch framework, providing APIs to simplify the deployment of models and data on GPUs and accelerate deep learning tasks.

[0057] ultralytics library: an open source computer vision and deep learning project focusing on tasks such as object detection, providing models such as YOLOv5 and easy-to-use tools.

[0058] Graphical interface library PyQt5: PyQt5 is a Python GUI toolkit based on the Qt library, used to create cross-platform desktop applications, with rich UI components and flexible development methods.

[0059] Preferably, in step 3, the region division can use the first frame clear image of the fixed area as the background. On the image, multiple points are connected to form a closed polygonal area, which is a plurality of areas to be detected. A single area can be represented by multiple points as follows: S = {(x i ,y i )|i=1,2,…,n},

[0060] Region detection can be based on ray intersection detection:

[0061] First, use the target recognition model to identify the four vertex coordinates and center coordinates of the person's rectangular coordinates:

[0062] 4 vertex coordinates set P = {(x1,y1),(x2,y1),(x2,y2),(x1,y2)}

[0063] Center coordinates

[0064] Then, suppose there are region vertices T = {(x1,y1),(x2,y2),…,(x i ,y i ),…,(x n ,y n )}, and the test point (P x ,P y ), substitute the upper coordinate set P and the center point C into the following equation to determine whether the target is within the area.

[0065] make

[0066] If t ≥ 0 and P x +t is within the range of the edge, that is, x i ≤P x +t≤x i+1 or x i+1 ≤P x +t≤x i , then it is considered that there is an intersection, that is, it can be determined that the target is in the area.

[0067] Embodiment 2:

[0068] A multi-region fire and person detection method based on target recognition, such as Figure 3 As shown, the following steps are included:

[0069] 1. Start: The process starts from the start node.

[0070] 2. Area detection: First, based on the ray intersection method, detect whether the hot fire area identified by the target recognition model is within the designated area to be detected.

[0071] If the hot work area is within the designated range, proceed to the next step.

[0072] If the hot work area is not within the designated range, proceed to step 7 and the process ends.

[0073] 3. Determine whether there are any operating personnel: In the hot work area, detect whether there are any operating personnel based on the target recognition model.

[0074] If there are no operators, proceed to the next step.

[0075] If there are operators, proceed to step 7 and the process ends.

[0076] 4. Determine whether there are signs of active fire: In the absence of operating personnel, detect whether there are signs of active fire based on the target recognition model.

[0077] If hot work features are present, proceed to the next step.

[0078] If there is no hot work feature, proceed to step 7 and the process ends.

[0079] 5. Determine whether there is a fire and absence counter in the area:

[0080] If there is a hot and unoccupied counter, proceed to the next step.

[0081] If there is no fire and absence counter, add a new regional fire and absence counter and start countdown, then go to step 1.

[0082] 6. Determine whether the countdown has ended:

[0083] If the countdown has not ended, if the operator returns to the hot work area before the countdown ends, the hot work absence counter is reset, and the process returns to step 1 to continue the cycle detection.

[0084] If the countdown ends, and if the operator has not returned to the hot work area before the countdown ends, it is determined that the operator in the area has left the area to do hot work, and proceed to step 7.

[0085] 7. End: The process ends when the end condition is met.

[0086] Embodiment 3:

[0087] A multi-region fire and person detection system based on target recognition is performed based on the method of embodiment 1 or embodiment 2, such as Figure 1 As shown, it includes a model training module, a target recognition module, a region division module and a moving fire and human detection module;

[0088] The model training module is used to build a model training environment and combine with a python script, use a training set to train a target recognition model, and output the trained target recognition model; specifically, the model training module includes a YOLO target algorithm unit, a graphical interface library pyqt5, a torch-GPU library, a torchvision-GPU library, and an ultralytics library. The YOLO target algorithm unit is used to build a target recognition model. The YOLO (You Only Look Once) target recognition algorithm is a fast and accurate target recognition algorithm that detects multiple targets at a time by dividing the image into grids and predicting the target bounding box and category probability within each grid.

[0089] The target recognition module is used to recognize the features and coordinates of people and flames on the input image using the trained target recognition model;

[0090] The area division module is used to delineate multiple areas to be detected on the image to be detected, and detect whether the hot fire area formed based on the coordinates is within the area to be detected based on the ray intersection method; specifically, the area division module includes a drawing area unit and an area data unit, the drawing area unit is used to delineate multiple areas to be detected, and the area data unit is used to store the division parameters of the multiple areas to be detected.

[0091] The fire-propelled person-free detection module is used to detect whether there is a flame feature in the fire-propelled area within the area to be detected and whether there is no operator, and trigger the early warning mechanism. Specifically, it also includes a fire-propelled person-free detection module, which is used to execute the early warning mechanism, and the time threshold can be customized through the internal fire-propelled person-free interval time unit.

[0092] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A multi-region fire and person detection method based on target recognition, characterized in that: The following steps are involved: Step S1: Video acquisition: obtain surveillance video in a fixed area and convert it into multiple continuous frames of images; Step S2: Target recognition: Input the image collected in step S1 into the trained target recognition model, and output the features and coordinates of the person and the flame; Step S3: Area delineation and detection: Using the clear image of the fixed area as the background, delineate multiple areas to be detected on the captured image; based on the coordinates output in step S2, connect the lines on the captured image to form a hot work area; combine the ray crossing method to detect whether the hot work area is within the area to be detected, if so, proceed to step S4; Step S4: Detection of personnel absence during fire operation: Based on the characteristics of personnel and / or flames output in step S2, determine whether there are flame characteristics and no operators in the fire operation area. If so, trigger the early warning mechanism.

2. The multi-region fire and person detection method based on target recognition according to claim 1 is characterized in that: In step S2, training the target recognition model includes the following steps: Step S21: Use the YOLOv8 model to build a target recognition model; Step S22: Collect pictures of flames and people in the kitchen scene, and use a labeling tool to label the target borders and categories in each picture to form a training set, a verification set, and a test set; Step S23: Building a training environment, using the training set to train the target recognition model, and then using the validation set and the test set to correct the target recognition model in turn, and finally obtaining the trained target recognition model.

3. The multi-region fire and person detection method based on target recognition according to claim 1 is characterized in that: The step S3 comprises the following steps: Step S31: Obtain the four vertex coordinates P = {(x1, y1), (x2, y1), (x2, y2), (x1, y2)} and the center coordinates of the rectangular coordinates of the person and / or flame output by the target recognition model Step S32: Assume that the vertices of the region to be detected are T = {(x1, y1), (x2, y2), ..., (x i ,y i ),…,(x n ,y n )} and the test point is (P x ,P y ); Step S33: Substitute the four vertex coordinates P and the center coordinate C of the rectangular coordinates into the following formulas to determine whether the hot work area is within the area to be detected: make If there exists t ≥ 0 and x i ≤P x +t≤x i+1 or x i+1 ≤P x +t≤x i , it is considered that there is an intersection between the hot fire area and the area to be detected, and the personnel and / or flames are in the area to be detected.

4. The multi-region fire and person detection method based on target recognition according to claim 1 is characterized in that: In step S4, the early warning mechanism is: triggering the fire-related absence counter and starting a countdown. If the operator returns to the fire-related area before the countdown ends, the fire-related absence counter is reset; if the operator does not return to the fire-related area before the countdown ends, the fire-related absence behavior is detected and early warning measures are triggered.

5. The multi-region fire and person detection method based on target recognition according to any one of claims 1 to 4, characterized in that: The step S4 comprises the following steps: Step S41: First, if it is detected that the hot work area is within the designated area to be detected, then based on the output of the target recognition model, it is determined whether there is an operator in the hot work area. If not, proceed to step S42, otherwise end; Step S42: Based on the output of the target recognition model, determine whether there is a fire feature. If yes, proceed to step S43, otherwise end; Step S43: trigger the hot work and no-personnel counter of the area to start timing. If an operator is detected before the timing ends, proceed to step S41, otherwise it is determined that there is a hot work and no-personnel operation in the area.

6. A multi-zone fire and person detection system based on target recognition, based on the method described in any one of claims 1 to 5, characterized in that: It includes a model training module, a target recognition module, an area division module and a fire-moving and human-free detection module; the model training module is used to build a target recognition model, and in combination with a python script in a training environment, uses a training set to train the target recognition model, and outputs the trained target recognition model; the target recognition module is used to use the trained target recognition model to recognize the characteristics and coordinates of people and flames on an input image; the area division module is used to delineate multiple areas to be detected on the image to be detected, and detect whether a fire-moving area formed based on coordinates is within the area to be detected based on a ray crossing method; the fire-moving and human-free detection module is used to detect whether there is a flame feature and no operating personnel in the fire-moving area within the area to be detected, and trigger an early warning mechanism.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 5 is implemented.

Images