Method, apparatus, device, and medium for determining fire level

Abstract

The present invention discloses a method, device, equipment and medium for determining the fire severity level. The method detects the flame density in an image frame based on a flame density detection model to determine a flame density map corresponding to the image frame; wherein, the image frame is obtained by a monitoring device monitoring the flame in a region to be measured; according to the flame density map, the fire severity level of the flame in the image frame is determined; wherein, the flame density map is used to represent the density data of the marked points on each flame. The technical solution of the present invention can monitor the flame density in the monitoring area in real time, accurately and quickly classify the fire severity level in the monitoring area, which is beneficial to the reasonable dispatch of fire fighting resources, saves human resources, and improves the early warning and prevention of fires.

Description

Technical Field

[0001] The present invention relates to the technical field of fire monitoring, and in particular to a method, device, equipment and medium for determining the fire level. Background Art

[0002] With the rapid development of Internet technology and information technology, it has driven the popularization of big data, artificial intelligence, Internet of Things and 5G network, and at the same time promoted the development of image processing technology and video processing technology. Among them, the security monitoring system based on image processing technology or video processing technology has been widely used and popularized in different fields and occasions because of its rich image information, high pixel resolution, advanced technology and intuitive convenience.

[0003] At present, the security monitoring systems for fire monitoring all monitor whether there is a fire scene in the monitored area, such as the presence or absence of flames. However, it is impossible to judge the size of the fire level in the monitored area, and thus it is impossible to directly and accurately feedback the fire warning information for the monitoring personnel to reasonably dispatch fire resources. The monitoring personnel need to judge based on the monitoring video by themselves, which is likely to cause the monitoring personnel to respond fatigued and ignore the fire situations with great harm.

[0004] Therefore, how to provide a technical solution for accurately dividing the fire level is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] The present invention provides a method, device, equipment and medium for determining the fire level, which can monitor the flame density in the monitored area in real time, so as to accurately and quickly divide the fire level in the monitored area, which is beneficial to reasonably dispatch fire resources, save human resources and improve the early warning and prevention of fires.

[0006] According to one aspect of the present invention, a method for determining the fire level is provided, and the method includes:

[0007] Detect the flame density in the image frame based on the flame density detection model to determine the flame density map corresponding to the image frame; wherein, the image frame is obtained by a monitoring device monitoring the flame in the area to be measured;

[0008] Determine the fire level of the flame in the image frame according to the flame density map; wherein, the flame density map is used to represent the density data of the marked points on each flame.

[0009] According to another aspect of the present invention, a device for determining the fire level is provided, and the device includes:

[0010] A flame density map determination module, configured to detect the flame density in an image frame based on a flame density detection model, and determine a flame density map corresponding to the image frame; wherein, the image frame is obtained by a monitoring device monitoring the flame in a region to be measured;

[0011] A fire severity level determination module, configured to determine the fire severity level of the flame in the image frame according to the flame density map; wherein, the flame density map is used to represent the density data of the marked points on each flame.

[0012] According to another aspect of the present invention, there is provided an electronic device, which includes:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the method for determining the fire severity level according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions, and when the computer instructions are executed by a processor, the method for determining the fire severity level according to any embodiment of the present invention is implemented.

[0017] The technical solution of the embodiment of the present invention detects the flame density in an image frame based on a flame density detection model, and determines a flame density map corresponding to the image frame; wherein, the image frame is obtained by a monitoring device monitoring the flame in a region to be measured; according to the flame density map, the fire severity level of the flame in the image frame is determined; wherein, the flame density map is used to represent the density data of the marked points on each flame. This technical solution can monitor the flame density in the monitoring area in real time, accurately and quickly divide the fire severity level in the monitoring area, which is beneficial to the reasonable dispatch of fire fighting resources, saves human resources, and improves the early warning and prevention of fires.

[0018] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0020] Figure 1 It is a flowchart of a method for determining the fire severity level provided in the first embodiment of the present invention;

[0021] Figure 2 It is a flowchart of a method for determining the fire severity level provided in the second embodiment of the present invention;

[0022] Figure 3 It is a schematic diagram of a preprocessing convolutional neural network provided in the second embodiment of the present invention;

[0023] Figure 4 It is a schematic diagram of a first convolutional neural network provided in the second embodiment of the present invention;

[0024] Figure 5 It is a schematic diagram of a third convolutional neural network provided in the second embodiment of the present invention;

[0025] Figure 6 It is a schematic diagram of a fourth convolutional neural network provided in the second embodiment of the present invention

[0026] Figure 7 It is a schematic diagram of the structure of a device for determining the fire severity level provided in the third embodiment of the present invention;

[0027] Figure 8 It is a schematic diagram of the structure of an electronic device for implementing the embodiments of the present invention. Detailed implementation manners

[0028] To enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1 As shown in the flowchart of a method for determining the fire severity level provided in Embodiment 1 of the present invention, this embodiment is applicable to the situation of monitoring and warning indoor or outdoor areas prone to fire. This method can be executed by a device for determining the fire severity level, and the device for determining the fire severity level can be implemented in the form of hardware and / or software. As Figure 1 shown, the method includes:

[0032] S110. Detect the flame density in the image frame based on the flame density detection model, and determine the flame density map corresponding to the image frame; wherein, the image frame is obtained by a monitoring device monitoring the flame in the area to be measured.

[0033] In an indoor scenario, a smoke detector is usually used to monitor the concentration of indoor smoke to determine whether a fire has occurred indoors; while in an outdoor scenario, due to the large monitoring area and strong air circulation outdoors, a monitoring terminal display is usually used to monitor whether there is a fire in the monitoring area. Among them, monitoring fires through surveillance videos consumes a large amount of human resources and there are a large number of false alarms, making it easy for monitoring personnel to feel fatigued.

[0034] In view of the above problems, a flame density detection model is used to identify the image frames in the surveillance video to determine whether a fire has occurred in the monitoring area and the severity level of the fire. Among them, the flame density detection model can be used to perform density detection on the image frames extracted from the surveillance video. The flame density map can be characterized by the proportion of the pixel points where the flame is located in the image frame, or by the number of flames in the image frame, or by the marked features of the flames in the image frame.

[0035] Specifically, the obtained image frame can be input into a pre-trained flame density detection model, and the flame density map corresponding to the image frame is output.

[0036] Among them, the monitoring device can be a device with image acquisition and transmission functions such as a monitoring camera, an infrared camera, etc.

[0037] As an optional but non-limiting implementation manner, the method for determining the image frame may include, but is not limited to, the processes of steps A1 to A2 as follows:

[0038] Step A1: Obtain the video data obtained by the monitoring device monitoring the flame in the area to be measured.

[0039] In order to comprehensively detect the fire level in the monitoring area, at least one monitoring device may be arranged at the locations where fires are likely to occur.

[0040] Among them, the video data may be the video data containing the flame in the video picture collected by the monitoring device. For example, it can be determined whether there is a flame in the video data through a flame recognition model. The acquisition of the video data can be achieved by the monitoring device collecting the video in the monitoring area, and then using the Video Capture video capture method in the traditional Open CV (Open Source Computer Vision Library) to load the RTSP (Real Time Streaming Protocol) address of the monitoring device in each operation center to realize the access to the video stream collected by the monitoring device.

[0041] Step A2: Intercept the video data at a preset period to obtain an image frame.

[0042] Generally, the frame rate of the obtained video data is usually about 25 frames per second. However, in an actual fire scene, the change rate of the flame density in the same monitoring area is not too large. Therefore, the interception period of the image frame does not have to be selected according to the video frame rate. For example, the preset period can be set to intercept the video data at one frame per second, that is, intercept an image frame from the video data every 0.5 seconds.

[0043] It can be understood that the image frames collected by the monitoring device are usually color images. To reduce the computational complexity when the flame density detection model detects the image frames, optionally, after obtaining the image frames, the image frames can be preprocessed and converted into grayscale image frames. That is, the image frame composed of three RGB channels is converted into a single-channel image, so that each pixel point in the image has only 256 values from 0 to 255. The advantage of such a setting is that it can reduce the amount of calculation on the basis of ensuring the accuracy of the subsequent calculation results. Specifically, it can be directly implemented by using the grayscale processing method in Open CV, that is, when reading the image frame, the reading mode is set to 0.

[0044] Further, the converted grayscale image frame can be converted into an image matrix. The advantage of this setting is that it can further reduce the computational complexity when the flame density detection model detects the image frame. Specifically, each pixel point in the grayscale image frame is represented by only one value (i.e., the grayscale value). Therefore, the pixel data of the grayscale image frame can be regarded as an image matrix. The rows of the image matrix correspond to the height of the grayscale image frame (in pixels), the columns of the image matrix correspond to the width of the grayscale image frame (in pixels), the elements of the image matrix correspond to the pixel data of the grayscale image frame, and the value of the matrix element is the grayscale value of each pixel point.

[0045] S120. Determine the fire intensity level of the flame in the image frame according to the flame density map; wherein, the flame density map is used to represent the density data of the marked points on each flame.

[0046] Among them, the flame density map includes the density data of the marked points of each flame in the image frame. It can be understood that the greater the density data, the greater the fire intensity of the flame. Among them, the fire intensity level can be the danger level of the flame in the image frame in the current scene and can be determined according to the specific scene.

[0047] In an alternative embodiment, to determine the fire intensity level of the flame in the image frame according to the flame density map, a fire intensity level determination model can be established. By pre-training a large number of flame density maps with marked fire intensity levels, the fire intensity level determination model is obtained. In the embodiment of the present invention, the flame density map can be input into the fire intensity level determination model to output the fire intensity level of the flame.

[0048] In another alternative embodiment, to determine the fire intensity level of the flame in the image frame according to the flame density map, it can include but is not limited to the processes of steps B1 to B2 as follows:

[0049] Step B1. Determine the flame density data according to the flame density map.

[0050] It should be noted that there is at least one flame in the image frame, and the density data of different flames are different. Therefore, it is necessary to determine the flame density data of the image frame according to the density data of each flame in the flame density map.

[0051] In an embodiment of the present invention, the flame density data can be determined in the following ways. For example, the maximum density data among the density data of each flame in the flame density map is used as the flame density data of the image frame; for another example, the fire intensity level coefficients of each flame are determined respectively according to the density data of each flame in the flame density map, and weighted summation is performed to determine the flame density data of the image frame; for another example, the average value of the density data of each flame in the flame density map is used as the flame density data of the image frame. It should be noted that the embodiment of the present invention does not limit the determination method of the flame density data of the image frame.

[0052] Step B2: Determine the fire intensity level of the flame in the image frame according to the comparison result between the flame density data and a preset threshold.

[0053] Among them, the preset threshold can be set according to the specific scenario where the monitoring area is located and the shooting distance of the monitoring device. For example, if the current scenario is a wild hilltop, the flame density threshold can be set to 5 / 10 / 15, indicating the fire intensity levels of small / medium / large respectively; if the current scenario is indoors, the flame density threshold can be set to 10 / 20 / 30, indicating the fire intensity levels of small / medium / large respectively. It should be noted that the embodiment of the present invention does not limit the division of the fire intensity levels.

[0054] The embodiment of the present invention provides a method for determining the fire intensity level. This method detects the flame density in the image frame based on a flame density detection model to determine a flame density map corresponding to the image frame; among them, the image frame is obtained by the monitoring device monitoring the flame in the area to be measured; according to the flame density map, the fire intensity level of the flame in the image frame is determined; among them, the flame density map is used to represent the density data of the marked points on each flame. This technical solution can monitor the flame density in the monitoring area in real time to accurately and quickly divide the fire intensity level in the monitoring area, which is beneficial to the reasonable dispatch of fire protection resources, saves human resources, and improves the early warning and prevention of fires.

[0055] Embodiment 2

[0056] Figure 2 It is a flowchart of a method for determining the fire intensity level provided by Embodiment 2 of the present invention. This embodiment is optimized on the basis of the above embodiment. As Figure 2 shown, this method includes:

[0057] S210: Preprocess the image frame based on a preprocessing submodel to determine a first feature map.

[0058] Among them, the preprocessing sub-model can be used to preliminarily extract the flame features in the image frame. The establishment process of the preprocessing sub-model can be to adjust the convolution kernel parameters of each convolution layer in the pre-constructed preprocessing convolutional neural network based on the obtained parameters to obtain the preprocessing sub-model.

[0059] Specifically, Figure 3 is a schematic diagram of a preprocessing convolutional neural network provided in the second embodiment of the present invention. As Figure 3 shown, the preprocessing convolutional neural network can include two convolutional layers and two activation function layers. The first convolutional layer contains 16 convolutional kernels each with a size of 9×9, and the second convolutional layer contains 32 convolutional kernels each with a size of 7×7. Among them, the two activation function layers are respectively located after the two convolutional layers, and the activation function layer can be PReLU (Parametric Rectified Linear Unit). The advantage of setting the activation function layer is to increase the non-linearity of the preprocessing sub-model, so that the preprocessing sub-model can approximate a non-linear function.

[0060] S220. Classify the number of flames in the first feature map based on the flame number classification sub-model to determine the second feature map.

[0061] Among them, the flame number classification sub-model can be used to extract and classify the flame number features in the image frame to obtain the number of flames in the image frame. The second feature map is used to represent the number of flames in the image frame. For example, there are three clusters of flames in the image frame. In the embodiment of the present invention, the first feature map is input into the pre-trained flame number classification sub-model, and the second feature map containing the flame number features is output.

[0062] As an optional but non-limiting implementation manner, the construction method of the flame number classification sub-model can include but is not limited to the processes of the following steps C1 to C2:

[0063] Step C1. Obtain a set of sample flame images, and label the number classification information according to the number of flames in the sample flame images to obtain a set of labeled flame images.

[0064] Among them, the set of sample flame images can collect sample flame images according to the number of flames to meet the accurate classification of the sample flame images with different numbers of flames by the flame number determination sub-model. Exemplarily, 1000 sample flame images with the number of flames from 1 to 10 are each selected as the set of sample flame images. Another example is to select sample flame images with the number of flames from 1 to 5 according to a certain ratio, and the number ratio of the sample flame images with the number of flames from 1 to 5 is 4:4:3:2:3. It should be noted that the embodiment of the present invention does not limit the determination method of the set of sample flame images.

[0065] Step C2: Use the set of labeled flame images as a training set to train the first convolutional neural network to obtain a flame number classification sub-model.

[0066] In the embodiments of the present invention, when using the set of labeled flame images as a training set and inputting it into the first convolutional neural network for training, a loss function can be set to correct the training process to obtain a flame number classification sub-model. Among them, the cross-entropy loss can be used as the loss function to represent the probability distribution difference between the output of the flame number classification sub-model and the observation result.

[0067] Figure 4 It is a schematic diagram of a first convolutional neural network provided in Embodiment 2 of the present invention. As Figure 4 shown, the first convolutional neural network may include four convolutional layers, seven activation function layers, two max-pooling layers, three fully connected layers, and one sigmoid function layer. Among them, the stride of both max-pooling layers is 2; the number of neurons included in the three fully connected layers is 512, 256, and 10 in sequence. Among them, the four activation function layers are respectively located after the four convolutional layers, the two max-pooling layers are respectively located after the first two activation function layers, the three fully connected layers are located after the last activation function layer, the three activation function layers are respectively located after the three fully connected layers, and the sigmoid function layer is located after the last activation function layer.

[0068] It should be noted that the sigmoid function is an activation function used for the output of hidden layer neurons, and its value range is (0, 1). It can map a real number to the interval (0, 1) for binary classification. The advantage of this setting is that it is easy to classify image frames with relatively complex or small differences in features.

[0069] Optionally, the first convolutional neural network may further include an SPP-Net function layer. The advantage of setting the SPP-Net function layer is that when used to train the flame number classification sub-model, it can support the input of image frames of any size.

[0070] S230: Based on the flame density detection sub-model, perform flame density detection on the first feature map and the second feature map to determine a flame density map corresponding to the image frame.

[0071] It should be noted that in order to accurately classify the number of flames in the first feature map through the flame number classification sub-model, using the max-pooling layer can reduce the estimation mean shift caused by the parameter error of the convolutional layer, but it will cause partial information loss of the image frame. Therefore, in the embodiments of the present invention, inputting the first feature map and the second feature map into the flame density detection sub-model together can recover the detailed features lost due to the max-pooling layer and improve the resolution of the output flame density map.

[0072] In an embodiment of the present invention, based on the flame density detection sub-model, the first feature map and the second feature map are subjected to flame density detection to determine a flame density map corresponding to the image frame. The detailed features lost in the second feature map can be restored according to the first feature map, and the flame density is marked according to the number of flames marked in the second feature map, so as to obtain a flame density map corresponding to the image frame. It should be noted that the number of flames marked in the second feature map will affect the granularity of feature extraction by the convolutional layer in the flame density detection sub-model, that is, the more the number of flames marked in the second feature map, the more features extracted by the convolutional layer in the flame density detection sub-model.

[0073] As an optional but non-limiting implementation manner, the construction method of the flame number classification sub-model may include, but is not limited to, the processes of steps D1 to D2 as follows:

[0074] Step D1: Mark marker points on the flames of the sample flame images in the sample flame image set, so as to represent the flame intensity by the marker point density data.

[0075] In an embodiment of the present invention, marker points are marked on the flames of the sample flame images in the sample flame image set. For example, according to the number classification result of the flames in the sample flame image, the corresponding positions in each cluster of flames are set to 1.

[0076] Step D2: Use the sample flame image set marked with marker points as the training set to train the second convolutional neural network to obtain a flame density detection sub-model.

[0077] In an embodiment of the present invention, the sample flame image set marked with marker points is used as the training set and input into the second convolutional neural network for training. A loss function can be set to correct the training process to obtain a flame density detection sub-model. Among them, the loss function can adopt the standard pixel-wise Euclidean distance loss.

[0078] Optionally, the flame density detection sub-model may include a preliminary processing sub-model and an output sub-model. Correspondingly, the second convolutional neural network may include a third convolutional neural network and a fourth convolutional neural network.

[0079] Figure 5 It is a schematic diagram of a third convolutional neural network provided in the second embodiment of the present invention. As Figure 5As shown in the figure, the third convolutional neural network may include four convolutional layers, four activation function layers, and two max pooling layers. Among them, the first convolutional layer contains 20 convolutional kernels each with a size of 7×7, the second convolutional layer contains 40 convolutional kernels each with a size of 5×5, the third convolutional layer contains 20 convolutional kernels each with a size of 5×5, and the fourth convolutional layer contains 10 convolutional kernels each with a size of 5×5. Among them, the four activation function layers are respectively located after the four convolutional layers, and the two max pooling layers are respectively located after the first two activation function layers.

[0080] Figure 6 It is a schematic diagram of a fourth convolutional neural network provided in the second embodiment of the present invention. As Figure 6 shown, the fourth convolutional neural network may include two convolutional layers and two deconvolutional layers. Among them, the first convolutional layer contains 24 convolutional kernels each with a size of 3×3, the second convolutional layer contains 32 convolutional kernels each with a size of 3×3, and the number of features of the two deconvolutional layers are 16 and 18 respectively. Among them, the two deconvolutional layers are located after the two convolutional layers. The advantage of such a setting is that the image output by the flame density detection sub-model can be restored to the original image size, further repairing the detail loss caused by the max pooling layer.

[0081] As an optional but non-limiting implementation, using the set of sample flame images marked with marker points as the training set to train the second convolutional neural network to obtain the flame density detection sub-model may include, but is not limited to, the processes of steps E1 to E3 as follows:

[0082] Step E1: Process the set of sample flame images based on the preprocessing sub-model to obtain sample feature maps.

[0083] Among them, the sample feature maps may be images obtained by preliminarily extracting the features in the sample flame images. Processing the set of sample flame images based on the preprocessing sub-model can retain more detail features in the sample flame images and avoid losing image detail features during subsequent further processing.

[0084] Step E2: Process the sample feature maps based on the flame number classification sub-model to obtain classification feature maps.

[0085] Among them, the classification feature maps are used to represent the flame number classification results in the sample feature maps.

[0086] Step E3: Use the sample feature maps, the set of sample flame images marked with marker points, and the classification feature maps as the training set to train the second convolutional neural network to obtain the flame density detection sub-model.

[0087] In an embodiment of the present invention, the sample feature map, the set of sample flame images marked with marker points, and the classification feature map are used as a training set and simultaneously input into a second convolutional neural network for training. A loss function is set to correct the training results. When the training results meet the preset requirements, a flame density detection sub-model is determined.

[0088] S240. Determine the fire intensity level of the flame in the image frame according to the flame density map; wherein, the flame density map is used to represent the density data of the marker points on each flame.

[0089] An embodiment of the present invention provides a method for determining the fire intensity level. The method preprocesses the image frame based on a preprocessing sub-model to determine a first feature map, classifies the number of flames in the first feature map based on a flame number classification sub-model to determine a second feature map, performs flame density detection on the first feature map and the second feature map based on a flame density detection sub-model to determine a flame density map corresponding to the image frame, and determines the fire intensity level of the flame in the image frame according to the flame density map; wherein, the flame density map is used to represent the density data of the marker points on each flame. In this technical solution, first, shared features of the image frame are extracted and advanced prior processing of the number of flames in the image frame is performed, and then the flame density is estimated based on the extracted shared features and the flame number classification result, which can improve the resolution of the flame density map, and further improve the accurate classification of the flame level, facilitate the reasonable dispatch of fire protection resources, save human resources, and improve the early warning and prevention of fires.

[0090] Embodiment III

[0091] Figure 7 It is a schematic structural diagram of a device for determining the fire intensity level provided in Embodiment III of the present invention. As Figure 7 shown, the device includes:

[0092] A flame density map determination module 710, configured to detect the flame density in the image frame based on a flame density detection model, and determine a flame density map corresponding to the image frame; wherein, the image frame is obtained by a monitoring device monitoring the flame in the area to be measured;

[0093] A fire intensity level determination module 720, configured to determine the fire intensity level of the flame in the image frame according to the flame density map; wherein, the flame density map is used to represent the density data of the marker points on each flame.

[0094] An embodiment of the present invention provides a device for determining the fire level. This method detects the flame density in the image frame based on a flame density detection model to determine a flame density map corresponding to the image frame. Among them, the image frame is obtained by a monitoring device monitoring the flame in the area to be measured. According to the flame density map, the fire level of the flame in the image frame is determined. Among them, the flame density map is used to represent the density data of the marked points on each flame. This technical solution can monitor the flame density in the monitoring area in real time to accurately and quickly classify the fire level in the monitoring area, which is beneficial to the reasonable dispatch of fire resources, saves human resources, and improves the early warning and prevention of fires.

[0095] Further, the flame density map determination module 710 includes:

[0096] A first feature map determination unit, configured to preprocess the image frame based on a preprocessing sub-model to determine a first feature map;

[0097] A second feature map determination unit, configured to classify the number of flames in the first feature map based on a flame number classification sub-model to determine a second feature map;

[0098] A flame density map determination unit, configured to perform flame density detection on the first feature map and the second feature map based on a flame density detection sub-model to determine a flame density map corresponding to the image frame.

[0099] Further, the flame number classification sub-model is constructed in the following manner:

[0100] Obtain a sample flame image set, and label the number classification information according to the number of flames in the sample flame image to obtain a labeled flame image set;

[0101] Use the labeled flame image set as a training set to train a first convolutional neural network to obtain a flame number classification sub-model.

[0102] Further, the flame density detection sub-model is constructed in the following manner:

[0103] Mark marked points on the flames of the sample flame images in the sample flame image set to represent the fire situation of the flames through the density data of the marked points;

[0104] Use the sample flame image set marked with marked points as a training set to train a second convolutional neural network to obtain a flame density detection sub-model.

[0105] Further, using the sample flame image set marked with marked points as a training set to train a second convolutional neural network to obtain a flame density detection sub-model includes:

[0106] Process the sample flame image set based on the preprocessing submodel to obtain a sample feature map;

[0107] Process the sample feature map based on the flame quantity classification submodel to obtain a classification feature map;

[0108] Use the sample feature map, the sample flame image set marked with marker points, and the classification feature map as a training set to train a second convolutional neural network to obtain a flame density detection submodel.

[0109] Further, the fire severity determination module 720 includes:

[0110] A flame density data determination unit for determining flame density data according to the flame density map;

[0111] A fire severity determination unit for determining the fire severity of the flame in the image frame according to the comparison result between the flame density data and a preset threshold.

[0112] Further, the method for determining an image frame includes:

[0113] Obtain video data obtained by a monitoring device monitoring the flame in the area to be measured;

[0114] Intercept the video data at a preset period to obtain an image frame.

[0115] The device for determining the fire severity provided by the embodiments of the present invention can execute the method for determining the fire severity provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0116] Embodiment 4

[0117] Figure 8 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described herein and / or claimed.

[0118] As Figure 8As shown, the electronic device 10 includes at least one processor 11 and a memory communicatively connected to the at least one processor 11, such as read-only memory (ROM) 12, random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12, and RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0119] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0120] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for determining the fire severity level.

[0121] In some embodiments, the method for determining the fire severity level can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for determining the fire severity level described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the method for determining the fire severity level by any other appropriate means (e.g., by means of firmware).

[0122] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0123] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0124] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0127] A computing system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The relationship between the client and the server is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0128] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0129] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for determining the fire level, characterized in that, Including: Detecting the flame density in the image frame based on the flame density detection model to determine a flame density map corresponding to the image frame; wherein, the image frame is obtained by a monitoring device monitoring the flame in the area to be measured; Determining the fire severity level of the flame in the image frame according to the flame density map; wherein, the flame density map is used to represent the density data of the marked points on each flame; The detecting the flame density in the image frame based on the flame density detection model to determine a flame density map corresponding to the image frame includes: Preprocessing the image frame based on a preprocessing sub-model to determine a first feature map; Classifying the number of flames in the first feature map based on a flame number classification sub-model to determine a second feature map; Detecting the flame density of the first feature map and the second feature map based on a flame density detection sub-model to determine a flame density map corresponding to the image frame.

2. The method according to claim 1, characterized in that, The flame number classification sub-model is constructed in the following manner: Obtaining a set of sample flame images, and labeling quantity classification information according to the number of flames in the sample flame images to obtain a set of labeled flame images; Using the set of labeled flame images as a training set to train a first convolutional neural network to obtain a flame number classification sub-model.

3. The method according to claim 2, wherein The flame density detection sub-model is constructed in the following manner: Marking marked points on the flames of the sample flame images in the set of sample flame images to represent the fire severity by the density data of the marked points; Using the set of sample flame images marked with marked points as a training set to train a second convolutional neural network to obtain a flame density detection sub-model.

4. The method according to claim 3, characterized in that Using the set of sample flame images marked with marked points as a training set to train a second convolutional neural network to obtain a flame density detection sub-model includes: Processing the set of sample flame images based on a preprocessing sub-model to obtain a sample feature map; Processing the sample feature map based on a flame number classification sub-model to obtain a classification feature map; Using the sample feature map, the set of sample flame images marked with marked points, and the classification feature map as a training set to train a second convolutional neural network to obtain a flame density detection sub-model.

5. The method according to claim 1, wherein Determining the fire severity level of the flame in the image frame according to the flame density map includes: Determining flame density data according to the flame density map; Determining the fire severity level of the flame in the image frame according to the comparison result between the flame density data and a preset threshold.

6. The method according to claim 1, wherein The method for determining the image frame includes: Obtaining video data obtained by a monitoring device monitoring the flame in the area to be measured; Intercepting the video data at a preset period to obtain an image frame.

7. A device for determining the fire intensity level, characterized in that, Including: A flame density map determination module, configured to detect the flame density in the image frame based on a flame density detection model to determine a flame density map corresponding to the image frame; wherein, the image frame is obtained by a monitoring device monitoring the flame in the area to be measured; A fire severity level determination module, configured to determine the fire severity level of the flame in the image frame according to the flame density map; wherein, the flame density map is used to represent the density data of the marked points on each flame; The flame density map determination module includes: A first feature map determination unit, configured to preprocess the image frame based on a preprocessing sub-model to determine a first feature map; A second feature map determination unit, configured to classify the number of flames in the first feature map based on a flame number classification sub-model to determine a second feature map; A flame density map determination unit, configured to perform flame density detection on the first feature map and the second feature map based on a flame density detection sub-model to determine a flame density map corresponding to the image frame.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the method for determining the fire severity level according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for implementing the method for determining the fire severity level according to any one of claims 1-6 when executed by a processor.

Images