Fire smoke detection and fire behavior identification method based on knowledge prior

Abstract

The invention relates to the technical field of fire and smoke detection and fire behavior recognition, in particular to a fire and smoke detection and fire behavior recognition method based on knowledge priori, which comprises a system module, system knowledge and a model. The system module is composed of six sub-modules including a data preparation module, a fire and smoke knowledge labeling module, a fire and smoke detection module, a fire knowledge labeling module, a fire recognition module and a fire early warning module. The system is composed of a system module, system knowledge and a model, fire smoke detection and fire behavior recognition capacity in an open environment are completed together, four fire smoke types of fire smoke, fire smoke and smoke, dense smoke and light smoke can be detected, large, medium and small fire behaviors are recognized, a category loss function of target detection is punished, and the target detection accuracy is improved. In other words, the loss punishment of identification errors of the detection model is increased, so that the fire and smoke detection accuracy of the detection model is improved, and false early warning of the system is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of fire smoke detection and fire intensity recognition, and in particular to a fire smoke detection and fire intensity recognition method based on prior knowledge. Background Art

[0002] The development of artificial intelligence technology has promoted the intelligent transformation and upgrading of various industries. With the widespread application of deep learning algorithms in the field of image recognition, the traditional fire warning method has gradually changed from manual intervention to machine intelligence. Taking the fire smoke detection and fire identification scenarios as an example, for fire safety fire monitoring, supervisors use the supervision platform to check or automatically rotate the monitoring point video to see if there are any fire safety hazards. Or use the back-end AI technology to perform real-time frame extraction and intelligent analysis of the monitoring video to detect fire smoke or smog and issue real-time warnings. Therefore, the birth of intelligent recognition technology can replace manpower to complete fire supervision and early warning to a certain extent, helping front-end security inspectors or back-end managers to reduce their burden and empower themselves.

[0003] Due to the complexity of fires and fire spread and the large changes in fire spread, as well as the uncontrollable monitoring scenes and surveillance cameras, AI intelligent recognition technology has posed a great challenge. Traditional image processing or pattern recognition technologies are difficult to solve fire and smoke detection in open scenes. Although the rapid update of deep learning algorithms has promoted the improvement of image recognition technology capabilities, there are still many challenges for complex fire and smoke detection and fire identification in development environments. Existing single-point technical solutions are difficult to have good detection and identification performance in open environments. Therefore, we proposed a fire and smoke detection and fire identification method based on knowledge priors. Summary of the invention

[0004] The purpose of the present invention is to provide a fire smoke detection and fire intensity identification method based on prior knowledge to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A fire smoke detection and fire intensity recognition method based on knowledge prior includes a system module, system knowledge and a model. The system module consists of a data preparation module, a fire smoke knowledge annotation module, a fire smoke detection module, a fire intensity knowledge annotation module, a fire intensity recognition module, and a fire warning module, and six submodules. The system knowledge and models include data knowledge prior, fire and smoke knowledge prior, subject detection model, fire knowledge prior, and fire warning model, a total of 3 knowledge bases and 2 models; The data preparation module includes the following steps: Data demand analysis: clarify demand scenarios and develop data implementation plans, including data crawler sources, different types of fire smoke, different sizes of fires, and their data volumes; Data collection, this task focuses on the crawler parameter settings, including crawler tool selection, such as Python scrapy crawler framework, preparing search keywords, setting the number of crawlers at a time, and setting a total of 12 search keywords for the four types of fire smoke detection and three types of fire intensity recognition implemented by this method, which fully ensures the generalization ability of the data in real scenarios; Data screening: Manual screening is performed on all the data obtained by the crawler in the above steps to eliminate invalid noise image data that cannot be identified manually.

[0006] As a preferred solution of the present invention, the fire smoke knowledge labeling module combines common fire types and fire smoke types to complete fire smoke knowledge labeling. The rich and diverse fire smoke types obtained by the above-mentioned data preparation module are labeled in the fire smoke knowledge base to form fire smoke knowledge priors. The fire smoke knowledge base includes: "fire smoke", "fire smoke and smoke", "thick smoke", and "light smoke".

[0007] As a preferred solution of the present invention, the fire smoke detection module selects the detectable multi-scale target detection algorithm yolov5 based on the anchor-free architecture as the fire smoke detection model, improves yolov5, and proposes yolov5_ploss to penalize the classification loss, reduce the model's false detection of non-fire smoke noise and the ability to detect fire smoke of different sizes, thereby improving the reliability and timeliness of warnings. The yolov5_ploss loss expression is as follows: (1) In the formula, and Respectively Factor and adjustment factor, loss_fcn is CEWithLogitsLoss loss type, represents the target box size factor, and is the minimum height and width of the allowed smoke size, h and w represent the actual height and width of the smoke. If the detection error occurs in a small target smoke, the penalty will increase. The category annotation of the same smoke is divided into one or more attributes, including small smoke, medium smoke and large smoke. Represents the proportion of samples belonging to category c. It can be seen that The larger it is, the greater the penalty for incorrect samples in training.

[0008] As a preferred solution of the present invention, the fire intensity knowledge labeling module is used to manually label the fire size and divide the fire size into three levels, namely "small fire", "medium fire" and "strong fire".

[0009] As a preferred solution of the present invention, the fire size calculation method process is as follows: Import the original image and prepare the original image with fire and smoke; Calculate the original image size and get the original image size height. , width is ; Fire smoke target labeling: manually label the fire smoke in the original image; Calculate the smoke size and obtain the height of the target area after the artificial frame is , width is ; Fire calculation, relative width is , relatively high .

[0010] As a preferred solution of the present invention, the fire identification module includes step 1, step 2, step 3 and step 4; The step 1 is to import the original image and the fire smoke knowledge base; The second step is to send the image to the subject detection model for fire and smoke detection.

[0011] As a preferred solution of the present invention, the step three is to perform target cropping on the target result returned by the model from the original image, and then import it into the fire intensity knowledge base, calculate the fire smoke size and the original image size and compare them with the fire intensity knowledge base to obtain the fire intensity recognition result; The step 4 is to form a fire information document with the fire smoke detection result and the fire intensity identification result.

[0012] As a preferred solution of the present invention, the fire warning module mainly combines the fire information formed in the above steps with business needs to form a timely fire warning solution.

[0013] As a preferred solution of the present invention, the fire warning model includes three models, namely model one, model two and model three. Model one: pushes message information to the business system, and the business system implements the warning.

[0014] As a preferred solution of the present invention, the second model: push the warning information to the camera equipped with the front alarm function, and make a real-time alarm; the third model: push the warning information to the supervision platform, and the platform side will make a secondary warning transformation.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. In the present invention, the system module and the system knowledge and model are composed of two parts, which together complete the fire smoke detection and fire identification capabilities in an open environment. It can detect four types of fire smoke, namely "fire smoke", "fire smoke and smoke", "thick smoke" and "light smoke", and identify three types of fire, namely "large", "medium" and "small". The category loss function of the target detection is penalized, that is, the loss penalty for the detection model's recognition errors is increased, thereby improving the detection accuracy of the detection model for fire smoke, thereby reducing the false alarms of this system. Then we design a fire size calculation formula based on the detection results of the target detection model and the original image size, and form a fire size recognition model based on the prior knowledge of manual annotation.

[0016] 2. In the present invention, the fire smoke detection and fire identification results are encapsulated as structured information, thereby ensuring that the message information output by this method can be used for accurate early warning. At the same time, combined with the fire warning scenario, this patent designs three fire warning models for reference by business systems, front cameras and monitoring platforms. The method designed in this patent can more accurately identify the type of fire smoke and the size of the fire at a reasonable computing resource cost, which helps to provide more detailed information for fire relief. At the same time, this method can greatly reduce false warnings and can be replicated and promoted to the industry. The fire smoke detection and fire identification method designed based on this method has been applied in multiple real scenarios. The recognition results can prove that this method can accurately detect the type of fire smoke and the size of the fire, and the output structured information can be used for disaster warning applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the fire smoke detection and fire intensity identification system based on prior knowledge of the present invention; Figure 2 This is an example diagram of the fire and smoke knowledge annotation of the system of the present invention; Figure 3 This is a schematic diagram of an example structure of the fire knowledge annotation of the system of the present invention; Figure 4 This is a flow chart of fire identification of the present invention; Figure 5 This is a diagram showing the fire and smoke detection and fire identification effects of the system of the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] For examples, see Figure 1-5 , the present invention provides a technical solution: A fire smoke detection and fire intensity recognition method based on knowledge prior includes a system module, system knowledge and a model. The system module consists of a data preparation module, a fire smoke knowledge annotation module, a fire smoke detection module, a fire intensity knowledge annotation module, a fire intensity recognition module, and a fire warning module, and six submodules. The system knowledge and models include data knowledge prior, fire and smoke knowledge prior, subject detection model, fire knowledge prior, and fire warning model, a total of 3 knowledge bases and 2 models; The data preparation module includes the following steps: Data demand analysis: clarify demand scenarios and develop data implementation plans, including data crawler sources, different types of fire smoke, different sizes of fires, and their data volumes; Data collection, this task focuses on the crawler parameter settings, including crawler tool selection, such as Python scrapy crawler framework, preparing search keywords, setting the number of crawlers at a time, and setting a total of 12 search keywords for the four types of fire smoke detection and three types of fire intensity recognition implemented by this method, which fully ensures the generalization ability of the data in real scenarios; Fire smoke and smog Medium fire smoke and fumes Small fire and smoke Fire smoke Medium fire smoke Small fire smoke Small thick smoke Medium smoke Heavy smoke Small light smoke Medium to light cigarettes Big light smoke Data screening: Manual screening is performed on all the data obtained by the crawler in the above steps to eliminate invalid noise image data that cannot be identified manually.

[0020] In this embodiment, Figure 1 , Figure 2 and Figure 3 As shown, the fire smoke knowledge labeling module combines common fire types and fire smoke types to complete fire smoke knowledge labeling. The rich and diverse fire smoke types obtained through the above data preparation module are labeled with the fire smoke knowledge base to form fire smoke knowledge priors. The fire smoke knowledge base includes: "fire smoke", "fire smoke and smoke", "thick smoke", and "light smoke". The fire smoke detection module selects the detectable multi-scale target detection algorithm yolov5 based on the anchor-free architecture as the fire smoke detection model, improves yolov5, and proposes yolov5_ploss to penalize the classification loss, reduce the model's false detection of non-fire smoke noise and the ability to detect fire smoke of different sizes, thereby improving the reliability and timeliness of warnings. The yolov5_ploss loss expression is as follows: (1) In the formula, and Respectively Factor and adjustment factor, loss_fcn is CEWithLogitsLoss loss type, represents the target box size factor, and is the minimum height and width of the allowed smoke size, h and w represent the actual height and width of the smoke. If the detection error occurs in a small target smoke, the penalty will increase. The category annotation of the same smoke is divided into one or more attributes, including small smoke, medium smoke and large smoke. Represents the proportion of samples belonging to category c. It can be seen that The larger it is, the greater the penalty for incorrect samples in training.

[0021] In this embodiment, Figure 4 and Figure 5 As shown, the fire intensity knowledge labeling module is used to manually label the fire size and divide the fire size into three levels, namely "small fire", "medium fire" and "fire". The fire size calculation method is as follows: Import the original image and prepare the original image with fire and smoke; Calculate the original image size and get the original image size height. , width is ; Fire smoke target labeling: manually label the fire smoke in the original image; Calculate the smoke size and obtain the height of the target area after the artificial frame is , width is ; Fire calculation, relative width is , relatively high .

[0022] The fire identification module includes step one, step two, step three and step four; Step 1 is to import the original image and fire smoke knowledge base; Step 2 is to send the image to the subject detection model for fire and smoke detection; Step 3 is to crop the target result returned by the model from the original image, then import it into the fire intensity knowledge base, calculate the fire smoke size and the original image size and compare them with the fire intensity knowledge base to obtain the fire intensity recognition result; Step 4 is to form a fire information file with the smoke detection results and the fire intensity identification results.

[0023] Fire warning module, this module mainly combines the fire information formed in the above steps with business needs to form a timely fire warning plan. The fire warning model includes three models: Model 1, Model 2 and Model 3. Model 1: pushes the message information to the business system, and the business system implements the warning. Model 2: pushes the warning information to the camera equipped with the front alarm function, and makes a real-time alarm. Model 3: pushes the warning information to the supervision platform, and the platform side conducts secondary warning transformation.

[0024] Among them, by encapsulating the fire smoke detection and fire identification results into structured information, it is ensured that the message information output by this method can be used for accurate early warning. At the same time, combined with the fire warning scenario, this patent designs three fire warning models for reference by business systems, front cameras and monitoring platforms. The method designed in this patent can more accurately identify the type of fire smoke and the size of the fire at a reasonable computing resource cost, which helps to provide more detailed information for fire relief. At the same time, this method can greatly reduce false warnings and can be replicated and promoted to the industry. The fire smoke detection and fire identification method designed based on this method has been applied in multiple real scenarios. The recognition results can prove that this method can accurately detect the type of fire smoke and the size of the fire, and the output structured information can be used for disaster warning applications.

[0025] Workflow of the present invention: When the fire smoke detection and fire intensity recognition method based on knowledge prior designed by the present scheme is working, first, we manually annotate the fire smoke type to form knowledge prior, and the annotation types include "fire smoke", "fire smoke and smoke", "thick smoke", and "light smoke". These annotation types ensure that the present method can detect more types of fire smoke. Considering that fire accidents are low-probability or low-frequency events in real scenarios, most of the daily monitoring data are negative samples. In view of this situation, we penalize the category loss function of target detection, that is, increase the loss penalty for the detection model's recognition error, thereby improving the detection accuracy of the fire smoke by the present detection model, thereby reducing the false warnings of the present system. Next, we design a fire size calculation formula based on the detection results of the target detection model and the size of the original image, and form a fire size recognition model in combination with the manually annotated prior knowledge. In addition, we encapsulate the fire smoke detection and fire intensity recognition results in structured information, thereby ensuring that the message information output by the present method can be used for accurate warnings.

[0026] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A fire smoke detection and fire intensity identification method based on prior knowledge, including system modules, system knowledge and models, characterized in that: The system module consists of six submodules: data preparation module, fire smoke knowledge labeling module, fire smoke detection module, fire intensity knowledge labeling module, fire intensity identification module, and fire warning module; The system knowledge and models include data knowledge prior, fire and smoke knowledge prior, subject detection model, fire knowledge prior, and fire warning model, a total of 3 knowledge bases and 2 models; The data preparation module includes the following steps: Data demand analysis: clarify demand scenarios and develop data implementation plans, including data crawler sources, different types of fire smoke, different sizes of fires, and their data volumes; Data collection, this task focuses on the crawler parameter settings, including crawler tool selection, such as Python scrapy crawler framework, preparing search keywords, setting the number of crawlers at a time, and setting a total of 12 search keywords for the four types of fire smoke detection and three types of fire intensity recognition implemented by this method, which fully ensures the generalization ability of the data in real scenarios; Data screening: Manual screening is performed on all the data obtained by the crawler in the above steps to eliminate invalid noise image data that cannot be identified manually.

2. The method for fire smoke detection and fire intensity identification based on prior knowledge according to claim 1, characterized in that: The fire smoke knowledge labeling module combines common fire types and fire smoke types to complete fire smoke knowledge labeling. The rich and diverse fire smoke types obtained by the above data preparation module are labeled in the fire smoke knowledge base to form fire smoke knowledge priors. The fire smoke knowledge base includes: "fire smoke", "fire smoke and smoke", "thick smoke", and "light smoke".

3. The method for fire smoke detection and fire intensity identification based on prior knowledge according to claim 1, characterized in that: The fire smoke detection module selects the detectable multi-scale target detection algorithm yolov5 based on the anchor-free architecture as the fire smoke detection model, improves yolov5, and proposes yolov5_ploss to penalize the classification loss, reduce the model's false detection of non-fire smoke noise and the ability to detect fire smoke of different sizes, thereby improving the reliability and timeliness of warnings. The yolov5_ploss loss expression is as follows: In the formula, f a and f m They represent the α factor and adjustment factor respectively, loss_fcn is the CEWithLogitsLoss loss type, β represents the target box size factor, and h min and w min is the minimum height and width of the allowed smoke size, h and w represent the actual height and width of the smoke. If the detection error occurs in a small target smoke, the penalty will increase. The category annotation of the same smoke is divided into one or more attributes, including small smoke, medium smoke and large smoke. Represents the proportion of samples belonging to category c. It can be seen that The larger it is, the greater the penalty for incorrect samples in training.

4. The method for fire smoke detection and fire intensity identification based on prior knowledge according to claim 1 is characterized in that: The fire intensity knowledge labeling module is used to manually label the fire size and divide the fire size into three levels, namely "small fire", "medium fire" and "fire".

5. The method for fire smoke detection and fire intensity identification based on prior knowledge according to claim 4 is characterized in that: The fire size calculation method process is as follows: Import the original image and prepare the original image with fire and smoke; Calculate the original image size and obtain the original image size height as H Y , width is W Y ; Fire smoke target labeling: manually label the fire smoke in the original image; Calculate the size of the fire and smoke, and obtain the height of the target area after the artificial frame is H X , width is W X ; Fire calculation, relative width is Relatively high 6. The method for fire smoke detection and fire intensity identification based on prior knowledge according to claim 1, characterized in that: The fire identification module includes step 1, step 2, step 3 and step 4; The step 1 is to import the original image and the fire smoke knowledge base; The step 2 is to send the image to the subject detection model for fire and smoke detection; The step three is to crop the target result returned by the model from the original image, then import it into the fire intensity knowledge base, calculate the fire smoke size and the original image size and compare them with the fire intensity knowledge base to obtain the fire intensity recognition result; The step 4 is to form a fire information document with the fire smoke detection result and the fire intensity identification result.

7. The method for fire smoke detection and fire intensity identification based on prior knowledge according to claim 1, characterized in that: The fire warning module mainly combines the fire information formed in the above steps with business needs to form a timely fire warning plan.

8. The method for fire smoke detection and fire intensity identification based on prior knowledge according to claim 1, characterized in that: The fire warning model includes three models: Model 1, Model 2 and Model 3. Model 1: pushes message information to the business system, and the business system implements the warning.

9. The method for fire smoke detection and fire intensity identification based on prior knowledge according to claim 8, characterized in that: Model 2: Pushing warning information to a camera equipped with a front alarm function and making a real-time alarm call; Model 3: Push the warning information to the supervision platform, and the platform will conduct secondary warning transformation.

Images