Fire-fighting risk event identification method, device and equipment

Abstract

The invention provides a fire-fighting risk event identification method, device and equipment, and relates to the technical field of fire-fighting detection, and the method comprises the steps: obtaining a sample image set; the sample image set comprises an open fire smoke image, a smoking behavior image and a fire fighting access occupation image, and extracting a first type of image features, a second type of image features and a third type of image features from the open fire smoke image, the smoking behavior image and the fire fighting access occupation image respectively, and training a comprehensive deep neural network by using the first type of image features, the second type of image features and the third type of image features to obtain a target deep neural network, and inputting the intermediate image features into a sub-network corresponding to the target deep neural network to determine the type of the fire risk event. According to the invention, the fire-fighting risk event identification method is deployed to different fire-fighting risk event scenes through cooperative communication among the modules, the scene adaptability is improved, and real-time monitoring of fire-fighting risk events is realized.

Description

Technical Field

[0001] The present invention relates to the field of fire detection technology, and in particular to a fire risk event identification method, device and equipment. Background Art

[0002] Fire risk event recognition is based on the target detection algorithm of computer vision. The development of target detection can be divided into two periods: the traditional target detection algorithm period (1998-2014) and the target detection algorithm period based on deep learning (2014-present). The traditional target detection algorithm is mainly based on manual feature extraction, and the process is as follows: 1) select the region of interest and select the area that may contain the object; 2) extract features from the area that may contain the object; 3) detect and classify the extracted features. However, the traditional target detection algorithm has the following three main disadvantages: 1) the recognition effect is not good enough and the accuracy is not high; 2) the calculation amount is large and the operation speed is slow; 3) multiple correct recognition results may be generated. It was not until the rise of convolutional neural networks in 2012 that the field of target detection was pushed to a new level. Mature target detection methods are mainly divided into the following two stages: generating 1) region proposals from the image, and 2) generating the final object bounding box from the region proposals. Among them, FasterRCNN is the first end-to-end deep learning detection algorithm that is closest to real-time performance. Although Faster RCNN has higher accuracy, faster speed, and is very close to real-time performance, it still has some computational redundancy in the subsequent detection stage; in addition, if the IOU threshold is set low, it will cause noise detection problems, and if the IOU is set high, it will cause overfitting.

[0003] Fire risk event identification usually consists of target detection tasks in three scenarios: open fire smoke, smoking behavior, and fire passage occupancy. For the identification of open fire smoke, the main problem currently is the lack of correlation analysis between the characteristics of open fire and smoke, and the low accuracy of smoke detection; for the identification of smoking behavior, the main problem currently is that it is easily affected by the surrounding environment, and the recognition accuracy of smoking behavior is not high; for the identification of fire passage occupancy, the main problem currently is the category identification problem of occupied targets, and at the same time, there are high requirements for multi-channel high concurrency and real-time processing of scene recognition and rapid deployment of models. Summary of the invention

[0004] The purpose of the present invention is to solve the current problem that it is difficult to accurately identify fire risk events and cannot meet the requirements of multi-channel high-concurrency real-time processing. The present invention provides a fire risk event identification method, device and equipment.

[0005] The technical solution of the embodiment of the present application is implemented as follows: A first aspect of an embodiment of the present application provides a method for identifying a fire risk event, comprising: Acquire a sample image set; the sample image set includes an open fire smoke image, a smoking behavior image, and a fire passage occupancy image; Extracting first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image, respectively; there are difference features between the first-category image features, the second-category image features, and the third-category image features; Using the first type of image features, the second type of image features, and the third type of image features, a comprehensive deep neural network is trained to obtain a target deep neural network; the target neural network includes a first subnetwork corresponding to the first type of image features, a second subnetwork corresponding to the second type of image features, and a third subnetwork corresponding to the third type of image features; The intermediate image features are obtained from the image to be identified, and the intermediate image features are input into the subnetwork corresponding to the target deep neural network to determine the type of fire risk event.

[0006] Optionally, after acquiring the sample image set, the method further includes: Various types of images in the sample image set are cleaned, labeled and enhanced.

[0007] Optionally, extracting first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image, respectively, includes: The Yolov5 algorithm improved based on the fire risk event scenario is used to extract the first type of image features, the second type of image features and the third type of image features from the open fire smoke image, the smoking behavior image and the fire passage occupancy image respectively.

[0008] Optionally, the using the first type of image features, the second type of image features, and the third type of image features to train a comprehensive deep neural network to obtain a target deep neural network includes: The first subnetwork of the integrated deep neural network is trained using the first type of image features, the second subnetwork of the integrated deep neural network is trained using the second type of image features, and the third subnetwork of the integrated deep neural network is trained using the third type of image features to obtain a target deep neural network.

[0009] Optionally, the using the first type of image features, the second type of image features, and the third type of image features to train a comprehensive deep neural network to obtain a target deep neural network includes: A training set and a test set are created using the first type of image features, the second type of image features, and the third type of image features, and the comprehensive deep neural network is trained in combination with a loss function to obtain a target deep neural network.

[0010] Optionally, acquiring intermediate image features from the image to be identified, and inputting the intermediate image features into a subnetwork corresponding to the target deep neural network to determine the type of the fire risk event includes: Acquire an intermediate image feature from the image to be recognized, and determine a target image feature of the same type as the intermediate image feature; the target image feature is at least one of the first type of image feature, the second type of image feature, and the third type of image feature; The intermediate image features are input into a subnetwork corresponding to the target image features in the target deep neural network to determine the type of fire risk event.

[0011] Optionally, obtaining intermediate image features from the image to be identified, and inputting the intermediate image features into a subnetwork corresponding to the target deep neural network to determine the type of fire risk event, including: Open fire and smoke events in the scene are located and framed, the monitoring time is recorded, and an open fire and smoke warning signal is issued; or smoking behaviors in the scene are located and framed, the monitoring time is recorded, and the image and location information of the smoker is recorded; or objects occupying fire passages in the scene are located and framed, the monitoring time is recorded, and the image and location information of objects occupying fire passages are recorded.

[0012] The second aspect of the embodiment of the present application provides a fire risk event identification device, including: an acquisition module, a feature extraction module, a training module and a determination module; wherein, The acquisition module is configured to acquire a sample image set; the sample image set includes an open fire smoke image, a smoking behavior image, and a fire passage occupancy image; The feature extraction module is configured to extract first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image, respectively; there are difference features between the first-category image features, the second-category image features, and the third-category image features; The training module is configured to train the comprehensive deep neural network using the first type of image features, the second type of image features, and the third type of image features to obtain a target deep neural network; the target neural network includes a first subnetwork corresponding to the first type of image features, a second subnetwork corresponding to the second type of image features, and a third subnetwork corresponding to the third type of image features; The determination module is configured to obtain intermediate image features from the image to be identified, and input the intermediate image features into the subnetwork corresponding to the target deep neural network to determine the type of fire risk event.

[0013] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the fire risk event identification method described in the first aspect.

[0014] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0015] Compared with the prior art, the technical solution provided by this application has the following beneficial effects: The present invention provides a fire risk event recognition method, device and equipment, which obtains a sample image set, extracts first-class image features, second-class image features and third-class image features from the open fire smoke image, smoking behavior image and fire passage occupancy image in the sample image set, respectively, trains the comprehensive deep neural network using the first-class image features, the second-class image features and the third-class image features to obtain the target deep neural network, and inputs the intermediate image features into the subnetwork corresponding to the target deep neural network to determine the type of fire risk event. The fire risk event recognition method is deployed to different fire risk event scenarios through collaborative communication between modules, which improves the scenario adaptability, realizes real-time monitoring of fire risk events, and maintains the original recognition efficiency while improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of a fire risk event identification method provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of a fire risk event identification device provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] Below, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present application.

[0018] The terms used herein are only for describing specific embodiments and are not intended to limit the present application. The terms "include", "comprising", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components. All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0019] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0020] In some embodiments, see Figure 1 , Figure 1 A flowchart of a fire risk event identification method provided in an embodiment of the present application; the fire risk event identification method provided in an embodiment of the present application includes: S110, obtaining a sample image set; the sample image set includes an open fire smoke image, a smoking behavior image, and a fire passage occupancy image.

[0021] In this embodiment, the historical monitoring data may be classified and sorted to obtain a sample image set.

[0022] In some embodiments, after acquiring the sample image set at S110, the method further includes: Clean, label and enhance various types of images in the sample image set.

[0023] Here, the various types of images in the sample image set are cleaned. The main purpose is to clean out the images that do not contain targets and are easy to predict, and retain the image data that does not contain targets and contains targets that are easy to interfere with the prediction based on deep learning convolutional neural network. Image annotation refers to adding labels or bounding boxes to objects or regions in the image so that subsequent models can learn to recognize these objects. Image enhancement is to improve the generalization ability of the model by increasing the diversity of data.

[0024] In one example, all types of images in the sample image set are required to be above 720p and have clear imaging. The location of the risk target is marked in the image, and the corresponding risk is marked; specifically, the location and category of the open fire and smoke are marked in the open fire and smoke image; the location of the smoker is marked in the smoking behavior image; the location and category of the occupied objects are marked in the fire passage occupancy image; in order for the deep neural network to better fit the target information, the labeling process needs to be standardized and the target subject should be framed as much as possible. In addition, the enhancement processing includes common data enhancement methods such as random cropping, scaling, rotation, random flipping, color transformation, etc. At the same time, for target detection, mosaic enhancement is introduced to randomly crop and splice multiple fire risk event images to produce new risk images for training. The benefit of this is that the targets are more diverse and the background information is richer, which is helpful for the learning of deep neural networks.

[0025] S120, extracting first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image, respectively; there are difference features among the first-category image features, the second-category image features, and the third-category image features.

[0026] Here, the first category of image features, the second category of image features, and the third category of image features may be typical features contained in open fire and smoke images, smoking behavior images, and fire escape occupancy images. For example, the first category of image features may include flame shape, flame size, and smoke density; the second category of image features may include smoke density and human portraits; and the third category of image features may include the outline and volume of occupied objects. These features may have common features as well as different features to distinguish between open fire and smoke events, smoking behavior events, and fire escape occupancy events.

[0027] In some embodiments, S120, extracting first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image, respectively, includes: The improved Yolov5 algorithm based on fire risk event scenarios is used to extract the first-category image features, the second-category image features and the third-category image features from the open fire smoke images, the smoking behavior images and the fire passage occupancy images respectively.

[0028] In this embodiment, the overall network structure corresponding to the improved Yolov5 algorithm can be a one-stage structure, which is divided into four parts: input end, Backbone, Neck, and Prediction, wherein the functions of each part are as follows: Input end: fill or scale the fire risk event image input after the fire risk scene preprocessing to make its size 3*640*640, and normalize the image data and convert it into a floating point number between 0 and 1. Backbone: extract the image features of the fire risk event image. Neck: between Backbone and Head, its function is to further extract and fuse the fire risk event image feature information output by Backbone. Prediction: classify the fire risk event feature information output by Neck (whether it is open fire smoke, smoking behavior, and fire passage occupation), locate (mark the location of open fire smoke, smoking behavior, and fire passage occupation targets), and output the classification probability, confidence, box and other information of detecting open fire smoke, smoking behavior, and fire passage occupation targets.

[0029] For the input end, Mosaic data enhancement is used, that is, random scaling, random cropping, and random arrangement are used for splicing to form new data; at the same time, the image is adaptively scaled and the least black border is added, so that the amount of calculation will be reduced during reasoning, that is, the target detection speed will be improved. For Backbone, the Focus structure mainly slices the input [3, 640, 640] data at an interval of 2 in the second and third dimensions, and then splices the sliced ​​data in the first dimension, which is equivalent to downsampling each channel image by 2 times, and then splicing the downsampled images, and finally inputting the spliced ​​data into CBL for processing; the CBL component consists of 1 convolution layer, 1 Batchnormalize layer, and 1 LeakyRelu activation function layer; the CSP structure consists of 2n CBL components ResUnit_n, especially the ResUnit_n module in CSP1_n superimposes the input signal to the output, while the ResUnit_n module in CSP2_n does not iterate the input signal to the output. For Neck, the essence of the SPP module is to use pooling windows of different sizes to perform pooling operations on the data respectively, and then splice the pooling results of different pooling windows. The main function is to solve the image distortion problem caused by cropping and scaling the input image and pooling and fusion of different scales, which is beneficial to target detection when the size of the detected targets varies greatly. For the FPN+PAN structure, FPN is top-down, and the high-level feature information is transferred and fused by upsampling; PAN is bottom-up, and the bottom-level positioning features are transferred and fused by downsampling. For Prediction, the output dimensions include the category probability dimension of the object, the confidence dimension of the bounding box, and the bounding box location information dimension.

[0030] S130, using the first type of image features, the second type of image features and the third type of image features to train the comprehensive deep neural network to obtain a target deep neural network; the target neural network includes a first sub-network corresponding to the first type of image features, a second sub-network corresponding to the second type of image features and a third sub-network corresponding to the third type of image features.

[0031] In some embodiments, S130, training the integrated deep neural network using the first type of image features, the second type of image features, and the third type of image features to obtain a target deep neural network includes: The first subnetwork of the comprehensive deep neural network is trained using the first type of image features, the second subnetwork of the comprehensive deep neural network is trained using the second type of image features, and the third subnetwork of the comprehensive deep neural network is trained using the third type of image features to obtain a target deep neural network.

[0032] Here, the trained first sub-network, second sub-network and third sub-network can be encapsulated into a recognition model, namely the target deep neural network. The target deep neural network provides the recognition function of open fire smoke, smoking behavior, and fire passage occupancy scenes. According to the recognition scene, the three functions can be arbitrarily combined to meet the recognition requirements of specific scenes.

[0033] In some embodiments, S130, training the integrated deep neural network using the first type of image features, the second type of image features, and the third type of image features to obtain a target deep neural network includes: The first, second and third image features are used to create training sets and test sets, and the comprehensive deep neural network is trained in combination with the loss function to obtain the target deep neural network.

[0034] Here, according to different fire risk scenarios, the preprocessed images can be divided into training sets and test sets in a ratio of 8:2. On this basis, the comprehensive deep neural network is trained in combination with the loss function until the loss function reaches the preset value to obtain the target deep neural network.

[0035] S140, obtaining intermediate image features from the image to be identified, and inputting the intermediate image features into the sub-network corresponding to the target deep neural network to determine the type of fire risk event.

[0036] In some embodiments, S140, obtaining intermediate image features from the image to be identified, and inputting the intermediate image features into a subnetwork corresponding to the target deep neural network to determine the type of fire risk event, including: Acquire an intermediate image feature from the image to be recognized, and determine a target image feature of the same type as the intermediate image feature; the target image feature is at least one of the first type of image feature, the second type of image feature, and the third type of image feature; The intermediate image features are input into the subnetwork corresponding to the target image features in the target deep neural network to determine the type of fire risk event.

[0037] In this embodiment, if the target image feature is a first-category image feature, the intermediate image feature is input into the first subnetwork; if the target image feature is a second-category image feature, the intermediate image feature is input into the second subnetwork; if the target image feature is a third-category image feature, the intermediate image feature is input into the third subnetwork.

[0038] In some embodiments, S140, obtaining intermediate image features from the image to be identified, and inputting the intermediate image features into a subnetwork corresponding to the target deep neural network to determine the type of fire risk event, including: Open fire and smoke events in the scene are located and framed, the monitoring time is recorded, and an open fire and smoke warning signal is issued; or smoking behaviors in the scene are located and framed, the monitoring time is recorded, and the image and location information of the smoker is recorded; or objects occupying fire passages in the scene are located and framed, the monitoring time is recorded, and the image and location information of objects occupying fire passages are recorded.

[0039] The embodiment of the present invention obtains a sample image set, extracts the first type of image features, the second type of image features and the third type of image features from the open fire smoke images, smoking behavior images and fire passage occupancy images in the sample image set, respectively, trains the comprehensive deep neural network using the first type of image features, the second type of image features and the third type of image features to obtain a target deep neural network, and inputs the intermediate image features into the subnetwork corresponding to the target deep neural network to determine the type of fire risk event. The fire risk event identification method is deployed to different fire risk event scenarios through collaborative communication between modules, which improves the scenario adaptability, realizes real-time monitoring of fire risk events, and maintains the original recognition efficiency while improving the recognition accuracy.

[0040] In some embodiments, see Figure 2 , Figure 2 A schematic diagram of the structure of a fire risk event identification device provided in an embodiment of the present application; The present application embodiment provides a fire risk event identification device 200, including: an acquisition module 210, a feature extraction module 220, a training module 230 and a determination module 240; wherein, The acquisition module 210 is configured to acquire a sample image set; the sample image set includes an open fire smoke image, a smoking behavior image, and a fire passage occupancy image; The feature extraction module 220 is configured to extract first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image, respectively; there are difference features among the first-category image features, the second-category image features, and the third-category image features; The training module 230 is configured to train the comprehensive deep neural network using the first type of image features, the second type of image features, and the third type of image features to obtain a target deep neural network; the target neural network includes a first sub-network corresponding to the first type of image features, a second sub-network corresponding to the second type of image features, and a third sub-network corresponding to the third type of image features; The determination module 240 is configured to obtain intermediate image features from the image to be identified, and input the intermediate image features into the sub-network corresponding to the target deep neural network to determine the type of fire risk event.

[0041] In some embodiments, the fire risk event identification device 200 further includes a preprocessing module; the preprocessing module is configured as follows: Clean, label and enhance various types of images in the sample image set.

[0042] In some embodiments, the feature extraction module 220 is specifically configured as follows: The improved Yolov5 algorithm based on fire risk event scenarios is used to extract the first-category image features, the second-category image features and the third-category image features from the open fire smoke images, the smoking behavior images and the fire passage occupancy images respectively.

[0043] In some embodiments, the training module 230 is specifically configured as follows: The first subnetwork of the comprehensive deep neural network is trained using the first type of image features, the second subnetwork of the comprehensive deep neural network is trained using the second type of image features, and the third subnetwork of the comprehensive deep neural network is trained using the third type of image features to obtain a target deep neural network.

[0044] In some embodiments, the training module 230 is specifically configured as follows: The first, second and third image features are used to create training sets and test sets, and the comprehensive deep neural network is trained in combination with the loss function to obtain the target deep neural network.

[0045] In some embodiments, the determination module 240 is specifically configured as follows: Acquire an intermediate image feature from the image to be recognized, and determine a target image feature of the same type as the intermediate image feature; the target image feature is at least one of the first type of image feature, the second type of image feature, and the third type of image feature; The intermediate image features are input into the subnetwork corresponding to the target image features in the target deep neural network to determine the type of fire risk event.

[0046] In some embodiments, the determination module 240 is specifically configured as follows: Open fire and smoke events in the scene are located and framed, the monitoring time is recorded, and an open fire and smoke warning signal is issued; or smoking behaviors in the scene are located and framed, the monitoring time is recorded, and the image and location information of the smoker is recorded; or objects occupying fire passages in the scene are located and framed, the monitoring time is recorded, and the image and location information of objects occupying fire passages are recorded.

[0047] The fire risk event identification device provided in the embodiment of the present application can implement each process in the embodiment corresponding to the above-mentioned fire risk event identification method. To avoid repetition, it will not be described here.

[0048] It should be noted that the fire risk event identification device provided in the embodiment of the present application and the fire risk event identification method provided in the embodiment of the present application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned fire risk event identification method, and the repeated parts will not be repeated.

[0049] In some embodiments, see Figure 3 , Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. An electronic device 300 provided in an embodiment of the present application includes a processor 310 and a memory 320; the memory 320 stores a computer program, wherein the computer program implements the above-mentioned fire risk event identification method when executed by the processor.

[0050] Specifically, the processor 310 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 310 may also include an onboard memory for cache purposes. The processor 310 may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiment of the present application.

[0051] The memory 320 may be any medium capable of containing, storing, conveying, propagating or transmitting instructions. For example, the memory 320 may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device, component or propagation medium. Specific examples of the memory 320 include: a magnetic storage device, such as a magnetic tape or a hard disk (HDD); an optical storage device, such as a compact disk (CD-ROM); a random access memory (RAM) or flash memory; and / or a wired / wireless communication link.

[0052] The present application also provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned fire risk event identification method is implemented. The computer-readable medium may be included in the device / apparatus / system described in the above-mentioned embodiment; or it may exist independently without being assembled into the device / apparatus / system. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed, the method according to the embodiment of the present application is implemented.

[0053] According to an embodiment of the present application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical cable, radio frequency signal, etc., or any suitable combination of the above.

[0054] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present application may be combined in a variety of ways and / or combinations, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined in a variety of ways and / or combinations. All of these combinations and / or combinations fall within the scope of the present application. Therefore, the scope of the present application should not be limited to the above-described embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A method for identifying fire risk events, characterized in that: include: Get a sample image set; The sample image set includes open fire smoke images, smoking behavior images, and fire passage occupancy images; Extracting first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image, respectively; there are difference features between the first-category image features, the second-category image features, and the third-category image features; Using the first type of image features, the second type of image features, and the third type of image features, a comprehensive deep neural network is trained to obtain a target deep neural network; the target neural network includes a first subnetwork corresponding to the first type of image features, a second subnetwork corresponding to the second type of image features, and a third subnetwork corresponding to the third type of image features; The intermediate image features are obtained from the image to be identified, and the intermediate image features are input into the subnetwork corresponding to the target deep neural network to determine the type of fire risk event.

2. The fire risk event identification method according to claim 1, characterized in that: After acquiring the sample image set, the method further includes: Various types of images in the sample image set are cleaned, labeled and enhanced.

3. The fire risk event identification method according to claim 1, characterized in that: The extracting of first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image respectively includes: The Yolov5 algorithm improved based on the fire risk event scenario is used to extract the first type of image features, the second type of image features and the third type of image features from the open fire smoke image, the smoking behavior image and the fire passage occupancy image respectively.

4. The fire risk event identification method according to claim 1, characterized in that: The method of training a comprehensive deep neural network using the first type of image features, the second type of image features, and the third type of image features to obtain a target deep neural network includes: The first subnetwork of the integrated deep neural network is trained using the first type of image features, the second subnetwork of the integrated deep neural network is trained using the second type of image features, and the third subnetwork of the integrated deep neural network is trained using the third type of image features to obtain a target deep neural network.

5. The fire risk event identification method according to claim 1, characterized in that: The method of training a comprehensive deep neural network using the first type of image features, the second type of image features, and the third type of image features to obtain a target deep neural network includes: A training set and a test set are created using the first type of image features, the second type of image features, and the third type of image features, and the comprehensive deep neural network is trained in combination with a loss function to obtain a target deep neural network.

6. The fire risk event identification method according to claim 1, characterized in that: The step of obtaining intermediate image features from the image to be identified and inputting the intermediate image features into a subnetwork corresponding to the target deep neural network to determine the type of fire risk event includes: Acquire an intermediate image feature from the image to be recognized, and determine a target image feature of the same type as the intermediate image feature; the target image feature is at least one of the first type of image feature, the second type of image feature, and the third type of image feature; The intermediate image features are input into a subnetwork corresponding to the target image features in the target deep neural network to determine the type of fire risk event.

7. The fire risk event identification method according to claim 1, characterized in that: Acquiring intermediate image features from the image to be identified, and inputting the intermediate image features into a subnetwork corresponding to the target deep neural network to determine the type of fire risk event, including: Open fire and smoke events in the scene are located and framed, the monitoring time is recorded, and an open fire and smoke warning signal is issued; or smoking behaviors in the scene are located and framed, the monitoring time is recorded, and the image and location information of the smoker is recorded; or objects occupying fire passages in the scene are located and framed, the monitoring time is recorded, and the image and location information of objects occupying fire passages are recorded.

8. A fire risk event identification device, characterized in that: include: Acquisition module, feature extraction module, training module and determination module; wherein, The acquisition module is configured to acquire a sample image set; the sample image set includes an open fire smoke image, a smoking behavior image, and a fire passage occupancy image; The feature extraction module is configured to extract first-category image features, second-category image features, and third-category image features from the open fire smoke image, the smoking behavior image, and the fire passage occupancy image, respectively; there are difference features between the first-category image features, the second-category image features, and the third-category image features; The training module is configured to train the comprehensive deep neural network using the first type of image features, the second type of image features, and the third type of image features to obtain a target deep neural network; the target neural network includes a first subnetwork corresponding to the first type of image features, a second subnetwork corresponding to the second type of image features, and a third subnetwork corresponding to the third type of image features; The determination module is configured to obtain intermediate image features from the image to be identified, and input the intermediate image features into the subnetwork corresponding to the target deep neural network to determine the type of fire risk event.

9. An electronic device comprising a processor and a memory; the memory stores a computer program, wherein: When the computer program is executed by the processor, the computer program implements the fire risk event identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images