Fire identification methods, devices and storage media

Abstract

This application provides a fire identification method, apparatus, and storage medium, relating to the field of image processing technology, for improving the efficiency of fire identification. The method includes: inputting an image to be identified into a convolutional neural network (CNN) model to output a set of feature maps; inputting the set of feature maps into a coordinated attention (CA) model to output attention feature data corresponding to the feature map set; inputting the attention feature data set into a fully connected layer and a softmax layer to output a fire identification result, the fire identification result indicating whether a fire has occurred in the location area corresponding to the image to be identified.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and storage medium for fire identification. Background Technology

[0002] Fire has always been a major threat to public safety and property security, and can even endanger people's lives. With the rapid development of internet technology, fire detection technology is also constantly improving to help people detect fires more quickly.

[0003] Among the more advanced technologies currently available, machine learning models identify fires by monitoring real-time footage. Compared to traditional identification methods, machine learning models have more powerful identification capabilities and can more accurately determine the occurrence of a fire, thereby reducing the possibility of false alarms.

[0004] However, machine learning models also face some challenges in application. Existing machine learning models require multiple images or other large amounts of data to identify fires, resulting in long identification times, low efficiency, and the potential for delayed fire detection. Therefore, improving the efficiency of fire identification is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a fire identification method, apparatus, and storage medium to improve fire identification efficiency.

[0006] In a first aspect, a method for fire identification is provided, comprising: acquiring an image to be identified; inputting the image to be identified into a convolutional neural network (CNN) model to output a set of feature maps; inputting the set of feature maps into a coordinated attention (CA) model to output attention feature data corresponding to the set of feature maps; inputting the set of attention feature data into a fully connected layer and a softmax layer to output a fire identification result, wherein the fire identification result is used to indicate whether a fire has occurred in the location area corresponding to the image to be identified.

[0007] The technical solution provided in this application brings at least the following beneficial effects: After acquiring the image to be identified, the feature data in the image to be identified is quickly extracted by using a convolutional neural network (CNN) model to improve the processing speed of the image to be identified, thereby improving the fire identification efficiency. The feature data is output in the form of a feature map set. Furthermore, the feature data in the image to be identified is strengthened by a coordinated attention (CA) model, which ensures the accuracy of fire identification. The feature data is output in the form of attention feature data. Finally, the fully connected layer and the softmax layer determine whether a fire has occurred in the corresponding location area of ​​the image to be identified based on the attention feature data. This can improve the fire identification efficiency while ensuring the accuracy of fire identification, so as to enable timely identification of the occurrence of fire.

[0008] As one possible implementation, the Convolutional Neural Network (CNN) model includes a first feature processing layer, a second feature processing layer, a third feature processing layer, and a fourth feature processing layer connected in sequence. The first feature processing layer includes a first convolutional layer, a first pooling layer, a first ReLU activation function, and a first Dropout layer. The first convolutional layer includes 16 convolutional kernels, each 3×3 in size, and the first pooling layer includes one convolutional kernel of 2×2 size. The second feature processing layer includes a second convolutional layer, a second pooling layer, a second ReLU activation function, and a second Dropout layer. The second convolutional layer contains... The first layer consists of 32 convolutional kernels, with each kernel in the second convolutional layer being 3×3 in size. The second pooling layer consists of one convolutional kernel, with each kernel being 2×2 in size. The third feature processing layer consists of a third convolutional layer, a third pooling layer, a third ReLU activation function, and a third Dropout layer. The third convolutional layer consists of 32 convolutional kernels, with each kernel being 3×3 in size. The third pooling layer consists of one convolutional kernel, with each kernel being 2×2 in size. The fourth feature processing layer consists of a fourth convolutional layer and a fourth ReLU activation function. The fourth convolutional layer consists of 128 convolutional kernels, with each kernel being 6×6 in size.

[0009] As one possible implementation, the CA model includes a channel attention module and a spatial attention module; the channel attention module is used to perform channel attention processing on the feature map set; and the spatial attention module is used to perform spatial attention processing on the feature map set.

[0010] As one possible implementation, the above-mentioned acquisition of the image to be identified includes: acquiring a video to be encoded; encoding the video to be encoded to obtain an initial image; and preprocessing the initial image to obtain the image to be identified, wherein the preprocessing includes image enhancement processing, smoothing filtering processing, and image sharpening processing.

[0011] As one possible implementation, the above method further includes: issuing an alarm message when the fire identification result indicates that a fire has occurred in the location area corresponding to the image to be identified, and the alarm message is used to instruct the location area corresponding to the image to be identified to be maintained.

[0012] Secondly, a fire identification device is provided, comprising: an acquisition module for acquiring an image to be identified; a processing module for inputting the image to be identified into a convolutional neural network (CNN) model and outputting a set of feature maps; the processing module is further configured to input the set of feature maps into a coordinated attention (CA) model and output attention feature data corresponding to the set of feature maps; the processing module is further configured to input the set of attention feature data into a fully connected layer and a softmax layer and output a fire identification result, wherein the fire identification result is used to indicate whether a fire has occurred in the location area corresponding to the image to be identified.

[0013] As one possible implementation, the Convolutional Neural Network (CNN) model includes a first feature processing layer, a second feature processing layer, a third feature processing layer, and a fourth feature processing layer connected in sequence. The first feature processing layer includes a first convolutional layer, a first pooling layer, a first ReLU activation function, and a first Dropout layer. The first convolutional layer includes 16 convolutional kernels, each 3×3 in size, and the first pooling layer includes one convolutional kernel of 2×2 size. The second feature processing layer includes a second convolutional layer, a second pooling layer, a second ReLU activation function, and a second Dropout layer. The second convolutional layer contains... The first layer consists of 32 convolutional kernels, with each kernel in the second convolutional layer being 3×3 in size. The second pooling layer consists of one convolutional kernel, with each kernel being 2×2 in size. The third feature processing layer consists of a third convolutional layer, a third pooling layer, a third ReLU activation function, and a third Dropout layer. The third convolutional layer consists of 32 convolutional kernels, with each kernel being 3×3 in size. The third pooling layer consists of one convolutional kernel, with each kernel being 2×2 in size. The fourth feature processing layer consists of a fourth convolutional layer and a fourth ReLU activation function. The fourth convolutional layer consists of 128 convolutional kernels, with each kernel being 6×6 in size.

[0014] As one possible implementation, the CA model includes a channel attention module and a spatial attention module; the channel attention module is used to perform channel attention processing on the feature map set; and the spatial attention module is used to perform spatial attention processing on the feature map set.

[0015] As one possible implementation, the acquisition module described above is specifically used for: acquiring the video to be encoded; encoding the video to be encoded to obtain an initial image; and preprocessing the initial image to obtain an image to be recognized. The preprocessing includes image enhancement, smoothing filtering, and image sharpening.

[0016] As one possible implementation, the above processing module is also used to: issue an alarm message when the fire identification result indicates that a fire has occurred in the location area corresponding to the image to be identified, and the alarm message is used to instruct the location area corresponding to the image to be identified to be maintained.

[0017] Thirdly, a fire identification device is provided, including a processor that, when executing a computer program, implements the fire identification method as described in the first aspect.

[0018] Fourthly, a computer-readable storage medium is provided, comprising computer instructions; wherein, when the computer instructions are executed, the fire identification method as described in the first aspect is implemented.

[0019] The beneficial effects described in the second to fourth aspects of this invention can be referred to the analysis of the beneficial effects of the first aspect, and will not be repeated here. Attached Figure Description

[0020] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0021] Figure 1 A schematic diagram of the structure of a fire identification system provided in an embodiment of this application;

[0022] Figure 2 A schematic flowchart illustrating a fire identification method provided in an embodiment of this application;

[0023] Figure 3 A flowchart illustrating yet another fire identification method provided in this application embodiment;

[0024] Figure 4 A flowchart illustrating the workflow of a deep flow toolkit provided in this application embodiment;

[0025] Figure 5 A schematic diagram of the architecture of a network model provided in an embodiment of this application;

[0026] Figure 6 A schematic diagram of the structure of a fire detection device provided in an embodiment of this application;

[0027] Figure 7This is a schematic diagram of the structure of another fire identification device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0029] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The words "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences. It should be noted that in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations.

[0030] Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. In embodiments of this application, "instruction" can include direct instruction and indirect instruction. For example, taking the first control information hereinafter as an example, the first control information can directly carry information A itself or its index to achieve the purpose of directly instructing information A. Alternatively, the first control information can also carry information B that is associated with information A, thereby achieving the purpose of indirectly instructing information A while instructing information B.

[0031] Currently, fire detection technologies can be mainly divided into the following four categories:

[0032] 1. Smoke detection technology involves monitoring changes in smoke concentration within an area using smoke detectors to identify smoke characteristics and determine if a fire has occurred. This technology has relatively low investment costs, but its accuracy is low, and smoke detectors are susceptible to false alarms due to interference from other gases.

[0033] 2. Optical identification technology, which uses infrared and light sensors to monitor changes in light within an area to determine if a fire has occurred. This technology is susceptible to light obstruction, leading to missed detections.

[0034] 3. Thermal imaging technology involves monitoring temperature changes within an area and then using an infrared camera to acquire temperature images. This not only determines whether a fire has occurred but also identifies the location and scale of the fire source based on the temperature images. However, this technology is costly, and its accuracy is relatively low due to the different thermal properties of various materials.

[0035] 4. Machine learning model recognition technology, such as background technology, is a relatively advanced recognition technology among related technologies. Although the recognition accuracy can meet normal requirements, it requires a large amount of data to determine whether a fire has occurred in the area, resulting in a long processing time, low recognition efficiency, and inability to identify whether a fire has occurred in the area in a timely manner.

[0036] Based on this, this application provides a fire identification method. The method involves: after acquiring the image to be identified, using a Convolutional Neural Network (CNN) model to quickly extract feature data from the image, thereby improving the processing speed and fire identification efficiency, and outputting the feature data as a set of feature maps. Furthermore, a Coordinated Attention (CA) model is used to enhance the feature data in the image, ensuring the accuracy of fire identification, and outputting the feature data as attention feature data. Finally, a fully connected layer and a Softmax layer determine whether a fire has occurred in the corresponding location area of ​​the image based on the attention feature data. This method improves fire identification efficiency while ensuring accuracy, enabling timely identification of fires.

[0037] The embodiments of this application will now be described in conjunction with the accompanying drawings.

[0038] Figure 1 This is a schematic diagram of a fire identification system provided in an embodiment of this application. The fire identification system includes a fire identification device 10 and a camera 20. The fire identification device 10 and the camera 20 can be connected via wired or wireless means.

[0039] The imaging device 20 can be located near the monitored area. For example, taking a digital park as the monitored area, the imaging device 20 can be installed on the roof of a building in the area where the circuits and communication lines are located, or in a location where images of the area where the communication lines and circuits are located can be captured. This application does not limit the specific installation method or location of the imaging device 20.

[0040] The imaging device 20 can be used to capture images of the area to be identified within the monitored region.

[0041] In some embodiments, the capturing device 20 may employ a color camera to capture color images.

[0042] For example, a color camera can be an RGB camera. An RGB camera uses the RGB color mode, obtaining various colors through variations in the three color channels: red (R), green (G), and blue (B) and their superposition. Typically, an RGB camera uses three different cables to provide the three basic color components, and three independent charge-coupled device (CCD) sensors to acquire the three color signals.

[0043] In some embodiments, the imaging device 20 may employ a depth camera to capture depth images.

[0044] For example, a depth camera can be a time-of-flight (TOF) camera. TOF cameras use TOF technology, and their imaging principle is as follows: A modulated pulsed infrared light emitted by a laser source is reflected upon encountering an object. A light source detector receives the reflected light and calculates the time difference or phase difference between the emission and reflection to determine the distance between the TOF camera and the object being photographed. Based on this distance, the depth value of each point in the scene is obtained.

[0045] The fire identification device 10 is used to acquire the image to be identified captured by the imaging device 20, and based on the image to be identified captured by the imaging device 20, determine whether a fire has occurred in the location area corresponding to the image to be identified.

[0046] In some embodiments, the fire detection device 10 may be an independent server, a server cluster or distributed system composed of multiple servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data service networks.

[0047] In some embodiments, the fire detection device 10 may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc.

[0048] In some embodiments, the fire detection device 10 can communicate with other terminal devices, such as staff terminal devices, via wired or wireless means to send alarm information to the staff terminal devices.

[0049] It should be understood that Figure 1 This is an exemplary schematic diagram. Figure 1 The number of devices included in the fire detection system shown is unlimited; for example, the number of cameras is unlimited. Furthermore, except... Figure 1 In addition to the equipment shown, Figure 1 The fire identification system may also include other equipment, and there are no limitations on this.

[0050] The fire identification method provided in this application embodiment is executed by a fire identification device. Optionally, the fire identification device may be the fire identification device 10 described above; or, the fire identification device may be the processor in the fire identification device 10; or, the fire identification device may be an application (APP) installed in the fire identification device 10 for executing the fire identification method; or, the fire identification device may be a functional module with image processing capabilities in the fire identification device 10. This application embodiment does not limit this.

[0051] Next, as Figure 2 The diagram shown is a flowchart illustrating a fire identification method provided in an embodiment of this application. The method includes the following steps:

[0052] S101. Obtain the image to be recognized.

[0053] Among them, the image to be identified is Figure 1 The image is obtained by the imaging device 20 capturing images of the monitored area. The monitored area is the area where fire needs to be monitored, such as the area containing communication lines and circuits in a digital park.

[0054] Optional, such as Figure 3 As shown, obtaining the image to be recognized can be specifically achieved through the following steps:

[0055] S1011. Acquire the video to be encoded.

[0056] Optionally, this application uses Figure 1 The camera device 20 in the middle takes pictures of the supervised area to obtain the video to be encoded corresponding to the supervised area.

[0057] The video to be encoded is a compressed and encoded video file. For example, the video to be encoded is in video stream format, consisting of a series of time-ordered video frames, each of which is an image.

[0058] S1012. Encode the video to be encoded to obtain the initial image.

[0059] Optionally, this application uses the pre-defined DeepStream SDK to encode the video to be encoded, which can restore the video to be encoded into multiple supervision images corresponding to different times. Each supervision image can be used as the initial image corresponding to the supervision region.

[0060] S1013. Preprocess the initial image to obtain the image to be recognized.

[0061] Optionally, this application preprocesses the initial image using a pre-defined DeepStream SDK to obtain a processed initial image, i.e., the image to be recognized. For example, the image to be recognized can be 64*64*3 pixels in size, or it can be in other sizes or formats; this application does not limit this.

[0062] The preprocessing includes image enhancement, smoothing filtering, and image sharpening.

[0063] like Figure 4 The diagram shown illustrates the workflow of a deep-stream toolkit provided in this application. First, the acquired video to be encoded is encoded to obtain an initial image. Next, the initial image is preprocessed to obtain an image to be recognized.

[0064] In this way, by preprocessing the initial image, the image quality can be improved, making the image smoother and more continuous, and making the image more suitable for analysis and recognition, thereby improving the accuracy and efficiency of fire identification.

[0065] S102. Input the image to be recognized into the convolutional neural network (CNN) model and output a set of feature maps.

[0066] The convolutional neural network (CNN) model includes a first feature processing layer, a second feature processing layer, a third feature processing layer, and a fourth feature processing layer connected in sequence. For example, the feature processing layer can also be referred to as a convolutional block, such as the first convolutional block, the second convolutional block, and the third convolutional block.

[0067] The first feature processing layer includes a first convolutional layer, a first pooling layer, a first activation ReLU function, and a first random deactivation Dropout layer; the first convolutional layer includes 16 convolutional kernels, and the size of each convolutional kernel in the first convolutional layer is 3×3; the first pooling layer includes 1 convolutional kernel, and the size of each convolutional kernel in the first pooling layer is 2×2.

[0068] The second feature processing layer includes a second convolutional layer, a second pooling layer, a second activation ReLU function, and a second random deactivation Dropout layer; the second convolutional layer includes 32 convolutional kernels, and the size of each convolutional kernel in the second convolutional layer is 3×3; the second pooling layer includes 1 convolutional kernel, and the size of each convolutional kernel in the second pooling layer is 2×2.

[0069] The third feature processing layer includes a third convolutional layer, a third pooling layer, a third activation ReLU function, and a third random deactivation Dropout layer; the third convolutional layer includes 32 convolutional kernels, and the size of each convolutional kernel in the third convolutional layer is 3×3; the third pooling layer includes 1 convolutional kernel, and the size of each convolutional kernel in the third pooling layer is 2×2.

[0070] The fourth feature processing layer includes a fourth convolutional layer and a fourth activation ReLU function; the fourth convolutional layer includes 128 convolutional kernels, and the size of each convolutional kernel in the fourth convolutional layer is 6×6.

[0071] It should be noted that CNN models are used to extract data features from the image to be identified and output these features as a set of feature maps. To achieve the required accuracy, the CNN models used in related technologies may include ten or even hundreds of feature processing layers. Even when processing only a small number of images, this requires a long processing time, resulting in low efficiency in fire identification.

[0072] The CNN model provided in this application includes only three feature processing layers and one convolutional layer. Compared with the CNN models in the prior art, the CNN model provided in this application is lightweight. Regarding the processing speed of the images to be identified, the CNN model provided in this application has a significant improvement over the CNN models in the prior art, thereby improving the processing efficiency of the images to be identified and thus improving the identification efficiency of fires.

[0073] For example, taking the CNN model provided in this application, the workflow of the CNN model is described below. When the image to be recognized enters the first feature processing layer, the first convolutional layer performs convolution processing on the image to be recognized, obtaining a first initial feature map set, which includes 16 first initial feature maps. Next, the first activation ReLU function activates the first initial feature map set, and the first pooling layer performs average pooling on the activated first initial feature map set. Finally, the first initial feature map set after the above processing is input to the first random deactivation Dropout layer to obtain the first feature map set, thus avoiding the overfitting problem.

[0074] When the first feature map set enters the second feature processing layer, the second convolutional layer performs convolution processing on the first feature map set to obtain a second initial feature map set, which includes 32 initial feature maps. Next, the second activation ReLU function activates the second initial feature map set, and the second pooling layer performs average pooling on the activated second initial feature map set. Finally, the processed second initial feature map set is input into the second random deactivation Dropout layer to obtain the second feature map set.

[0075] When the second feature map set enters the third feature processing layer, the third convolutional layer performs convolution on the second feature map set to obtain the third initial feature map set, which includes 64 third initial feature maps. Next, the third activation ReLU function activates the third initial feature map set, and the third pooling layer performs average pooling on the activated third initial feature map set. Finally, the processed third initial feature map set is input into the third random deactivation Dropout layer to obtain the third feature map set.

[0076] When the second feature map set enters the fourth feature processing layer, the fourth convolutional layer performs convolution processing on the second feature map set to obtain the fourth initial feature map set, which includes 128 fourth initial feature maps. Following this, the fourth activation ReLU function activates the fourth initial feature map set to obtain the final feature map set.

[0077] It is understandable that the feature strength in the feature map set is greater than that in the third feature map set, the feature strength in the third feature map set is greater than that in the second feature map set, and the feature strength in the second feature map set is greater than that in the first feature map set. The greater the feature strength of the feature map set, the more suitable it is for analysis and recognition, and the higher its recognition accuracy.

[0078] S103. Input the feature map set into the Coordinated Attention (CA) model and output the attention feature data corresponding to the feature map set.

[0079] The CA model includes a channel attention module and a spatial attention module; the channel attention module is used to perform channel attention processing on the feature map set; and the spatial attention module is used to perform spatial attention processing on the feature map set.

[0080] It should be noted that the CA model is used to extract and enhance the feature data in the feature map set. After performing channel attention processing and spatial attention processing on the feature map set, the feature intensity of the attention feature data obtained is higher than that of the feature map set, which can improve the accuracy and efficiency of fire identification.

[0081] S104. Input the attention feature data set into the fully connected layer and the softmax layer, and output the fire identification result.

[0082] The fully connected layer can transform the received attention feature data into high-level features, which in turn enables the softmax layer to make judgments based on the attention feature data, obtaining the probability that a fire has occurred and the probability that no fire has occurred in the location area corresponding to the image to be identified. The higher probability event result is used as the fire identification result input, which is used to indicate whether a fire has occurred in the location area corresponding to the image to be identified.

[0083] like Figure 5 The diagram shown illustrates the architecture of a network model provided in this embodiment. Taking an image of size 64*64*3 as an example, after inputting the image to be recognized into the first feature processing layer, the image is first convolved (Conv), then activated using the ReLU function, followed by averaging (Av-pooling), and finally random deactivation (Dropout). The processing flow of the second to fourth processing layers can be referred to the processing flow of the image to be processed in S102, and will not be repeated here. After the fourth processing layer is completed, the resulting feature map set is input into the Coordinated Attention (CA) model to obtain attention feature data, which is then input into a fully connected layer (FC), followed by a softmax layer, and finally outputs the fire recognition result.

[0084] In this way, after acquiring the image to be identified, a Convolutional Neural Network (CNN) model is used to quickly extract feature data from the image, thereby improving the processing speed and fire identification efficiency. This data is then output as a set of feature maps. Furthermore, a Coordinated Attention (CA) model is used to enhance the feature data in the image, ensuring the accuracy of fire identification. This data is then output as attention feature data. Finally, the fully connected layer and the Softmax layer determine whether a fire has occurred in the corresponding location area of ​​the image based on the attention feature data. This approach improves fire identification efficiency while maintaining accuracy, enabling timely detection of fires.

[0085] based on Figure 2 In the embodiment shown, after S104, the method further includes issuing an alarm message when the fire identification result indicates that a fire has occurred in the location area corresponding to the image to be identified.

[0086] The alarm information is used to instruct the maintenance of the location area corresponding to the image to be identified.

[0087] For example, alarm messages can be in text format and can be sent directly to the staff's terminal device. Alarm messages can also be in audio format and can be played directly on the connected audio device.

[0088] In this way, if a fire occurs in the area corresponding to the image to be identified, staff can be promptly alerted to carry out maintenance to reduce the damage caused by the fire.

[0089] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0090] This application embodiment can divide the fire identification device into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0091] Figure 6 This is a schematic diagram of the structure of a fire detection device provided in an embodiment of this application, as shown below. Figure 6 As shown, the fire identification device 60 includes an acquisition module 601 and a processing module 602.

[0092] Acquisition module 601 is used to acquire the image to be recognized;

[0093] Processing module 602 is used to input the image to be recognized into a convolutional neural network (CNN) model and output a set of feature maps;

[0094] The processing module 602 is also used to input the feature map set into the Coordinated Attention (CA) model and output the attention feature data corresponding to the feature map set;

[0095] The processing module 602 is also used to input the attention feature data set into the fully connected layer and the Softmax layer, and output the fire identification result. The fire identification result is used to indicate whether a fire has occurred in the location area corresponding to the image to be identified.

[0096] In some embodiments, the convolutional neural network (CNN) model includes a first feature processing layer, a second feature processing layer, a third feature processing layer, and a fourth feature processing layer connected in sequence; the first feature processing layer includes a first convolutional layer, a first pooling layer, a first activation ReLU function, and a first random deactivation Dropout layer; the first convolutional layer includes 16 convolutional kernels, each of which is 3×3 in size; the first pooling layer includes one convolutional kernel, each of which is 2×2 in size; the second feature processing layer includes a second convolutional layer, a second pooling layer, a second activation ReLU function, and a second random deactivation Dropout layer; the second convolutional layer includes 3×3 convolutional kernels, each of which is 3×3 in size; the second pooling layer includes 16 convolutional kernels, each of which is 3×3 in size; the second pooling layer includes 16 convolutional kernels, each of which is 3×3 in size; the second pooling layer includes 16 convolutional kernels, each of which is 3×3 in size; the second feature processing ... The first layer has two convolutional kernels, each 3×3 in size. The second pooling layer has one convolutional kernel, each 2×2 in size. The third feature processing layer includes a third convolutional layer, a third pooling layer, a third ReLU activation function, and a third Dropout layer. The third convolutional layer has 32 convolutional kernels, each 3×3 in size. The third pooling layer has one convolutional kernel, each 2×2 in size. The fourth feature processing layer includes a fourth convolutional layer and a fourth ReLU activation function. The fourth convolutional layer has 128 convolutional kernels, each 6×6 in size.

[0097] In some embodiments, the CA model includes a channel attention module and a spatial attention module; the channel attention module is used to perform channel attention processing on the feature map set; and the spatial attention module is used to perform spatial attention processing on the feature map set.

[0098] In some embodiments, the acquisition module 601 is specifically used for: acquiring a video to be encoded; encoding the video to be encoded to obtain an initial image; and preprocessing the initial image to obtain an image to be recognized, wherein the preprocessing includes image enhancement processing, smoothing filtering processing, and image sharpening processing.

[0099] In some embodiments, the above-described processing module is further configured to: issue an alarm message when the fire identification result indicates that a fire has occurred in the location area corresponding to the image to be identified, the alarm message being used to instruct the location area corresponding to the image to be identified to be maintained.

[0100] In the case of implementing the functions of the integrated modules described above in hardware, the embodiments of this application provide... Figure 7 The fire detection device shown. (For example...) Figure 7 As shown, the fire detection device 70 includes a processor 702 and a bus 704. Optionally, the fire detection device 70 may also include a memory 701; alternatively, the fire detection device 70 may also include a communication interface 703.

[0101] Processor 702 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 702 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 702 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0102] The communication interface 703 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0103] The memory 701 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0104] As one possible implementation, the memory 701 can exist independently of the processor 702. The memory 701 can be connected to the processor 702 via a bus 704 and is used to store instructions or program code. When the processor 702 calls and executes the instructions or program code stored in the memory 701, it can implement the fire identification method provided in this application embodiment.

[0105] In another possible implementation, the memory 701 can also be integrated with the processor 702.

[0106] The 704 bus can be an extended industry standard architecture (EISA) bus, etc. The 704 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0107] This application also provides a computer-readable storage medium, which includes computer-executable instructions that, when executed on a computer, cause the computer to perform the method provided in the above embodiments.

[0108] This application also provides a computer program product that can be directly loaded into a memory and contains software code. After being loaded and executed by a computer, the computer program product can implement the methods provided in the above embodiments.

[0109] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0110] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of identifying a fire, characterized by, The method comprises: collecting a video to be encoded; encoding processing the video to be encoded to obtain an initial image; the initial image is an image obtained by encoding processing the video to be encoded using DeepStreamSDK; preprocessing the initial image to obtain an image to be recognized, the preprocessing including image enhancement processing, smoothing filtering processing and image sharpening processing; the image to be recognized is an image obtained by preprocessing the initial image using DeepStream SDK; inputting the image to be recognized into a convolutional neural network CNN model to output a feature map set; the first feature processing layer, the second feature processing layer and the third feature processing layer in the convolutional neural network CNN model are all composed of a convolutional layer, a pooling layer, an activation ReLU function and a random inactivation Dropout layer, and the fourth feature processing layer is composed of a convolutional layer and an activation ReLU function; inputting the feature map set into a coordinated attention CA model to output attention feature data corresponding to the feature map set; the CA model includes a channel attention module and a spatial attention module; the channel attention module is used for channel attention processing on the feature map set; and the spatial attention module is used for spatial attention processing on the feature map set; inputting the attention feature data set into a fully connected layer and a Softmax layer to output a fire identification result, the fire identification result being used for indicating whether a fire occurs in a position area corresponding to the image to be recognized.

2. The method of claim 1, wherein, The convolutional neural network CNN model comprises a first feature processing layer, a second feature processing layer, a third feature processing layer and a fourth feature processing layer connected in sequence; The first feature processing layer comprises a first convolutional layer, a first pooling layer, a first activation ReLU function and a first random inactivation Dropout layer; the first convolutional layer comprises 16 convolutional kernels, the size of the convolutional kernels in the first convolutional layer is all 3x3, the first pooling layer comprises 1 convolutional kernel, and the size of the convolutional kernel in the first pooling layer is 2x2; The second feature processing layer comprises a second convolutional layer, a second pooling layer, a second activation ReLU function and a second random inactivation Dropout layer; the second convolutional layer comprises 32 convolutional kernels, the size of the convolutional kernels in the second convolutional layer is all 3x3, the second pooling layer comprises 1 convolutional kernel, and the size of the convolutional kernel in the second pooling layer is 2x2; The third feature processing layer comprises a third convolutional layer, a third pooling layer, a third activation ReLU function and a third random inactivation Dropout layer; the third convolutional layer comprises 32 convolutional kernels, the size of the convolutional kernels in the third convolutional layer is all 3x3, the third pooling layer comprises 1 convolutional kernel, and the size of the convolutional kernel in the third pooling layer is 2x2; The fourth feature processing layer comprises a fourth convolutional layer and a fourth activation ReLU function; the fourth convolutional layer comprises 128 convolutional kernels, and the size of the convolutional kernels in the fourth convolutional layer is all 6x6.

3. The method of claim 1, wherein, The CA model comprises a channel attention module and a spatial attention module; the channel attention module is configured to perform channel attention processing on the feature map set; and the spatial attention module is configured to perform spatial attention processing on the feature map set.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: In a case where the fire identification result indicates that a fire occurs in a location area corresponding to the to-be-identified image, an alarm information is sent, the alarm information being configured to instruct maintenance of the location area corresponding to the to-be-identified image.

5. A fire identification device, characterized by The device comprises: An acquisition module configured to collect a to-be-encoded video; An encoding processing is performed on the to-be-encoded video to obtain an initial image; the initial image is an image obtained by performing the encoding processing on the to-be-encoded video by using a DeepStream SDK; A preprocessing is performed on the initial image to obtain a to-be-identified image; the preprocessing comprises image enhancement processing, smoothing filtering processing and image sharpening processing; the to-be-identified image is an image obtained by performing the preprocessing on the initial image by using the DeepStream SDK; A processing module configured to input the to-be-identified image into a convolutional neural network (CNN) model to output a feature map set; each of a first feature processing layer, a second feature processing layer and a third feature processing layer in the CNN model is composed of a convolutional layer, a pooling layer, an activation ReLU function and a random deactivation Dropout layer; and a fourth feature processing layer is composed of a convolutional layer and an activation ReLU function; The processing module is further configured to input the feature map set into a coordinated attention (CA) model to output attention feature data corresponding to the feature map set; the CA model comprises a channel attention module and a spatial attention module; the channel attention module is configured to perform channel attention processing on the feature map set; and the spatial attention module is configured to perform spatial attention processing on the feature map set; The processing module is further configured to input the attention feature data set into a fully connected layer and a Softmax layer to output a fire identification result, the fire identification result being configured to indicate whether a fire occurs in a location area corresponding to the to-be-identified image.

6. The apparatus of claim 5, wherein, The CNN model comprises a first feature processing layer, a second feature processing layer, a third feature processing layer and a fourth feature processing layer connected in sequence; The first feature processing layer comprises a first convolutional layer, a first pooling layer, a first activation ReLU function and a first random deactivation Dropout layer; the first convolutional layer comprises 16 convolutional kernels, the size of each of the convolutional kernels in the first convolutional layer is 3×3, the first pooling layer comprises 1 convolutional kernel, and the size of the convolutional kernel in the first pooling layer is 2×2; The second feature processing layer comprises a second convolutional layer, a second pooling layer, a second activation ReLU function and a second random deactivation Dropout layer; the second convolutional layer comprises 32 convolutional kernels, the size of each of the convolutional kernels in the second convolutional layer is 3×3, the second pooling layer comprises 1 convolutional kernel, and the size of the convolutional kernel in the second pooling layer is 2×2; The third feature processing layer includes a third convolutional layer, a third pooling layer, a third activation ReLU function and a third random inactivation Dropout layer; the third convolutional layer includes 32 convolutional kernels, the size of the convolutional kernels in the third convolutional layer is all 3*3, the third pooling layer includes 1 convolutional kernel, and the size of the convolutional kernel in the third pooling layer is 2*2; The fourth feature processing layer includes a fourth convolutional layer and a fourth activation ReLU function; the fourth convolutional layer includes 128 convolutional kernels, and the size of the convolutional kernels in the fourth convolutional layer is all 6*6.

7. The apparatus of claim 5, wherein, The CA model includes a channel attention module and a spatial attention module; the channel attention module is used for performing channel attention processing on the feature map set; and the spatial attention module is used for performing spatial attention processing on the feature map set.

8. The device of any one of claims 5-7, wherein, The processing module is further used for: In a case where the fire identification result indicates that a fire occurs in the position region corresponding to the to-be-identified image, alarm information is sent, and the alarm information is used for indicating that the position region corresponding to the to-be-identified image is maintained.

9. A fire identification device, characterized by The computer readable storage medium includes computer instructions; when the computer instructions are executed, the fire identification method in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium includes computer instructions; when the computer instructions are executed, the fire identification method in any one of claims 1 to 4 is implemented.

Images