Intelligent fire identification method and device, cloud equipment and computer device

Abstract

The invention relates to the field of intelligent fire recognition, in particular to an intelligent fire recognition method and device, cloud equipment and a computer device, and improves the fire recognition accuracy. The intelligent fire identification method comprises the steps that a multi-level separation convolutional neural network is constructed, the multi-level separation convolutional neural network is mainly composed of a lightweight multi-view decoupling convolution module, and the lightweight multi-view decoupling convolution module is formed by cascading at least two lightweight multi-view decoupling convolution sub-modules in different convolution directions; the output of the previous sub-module is used as the input of the next sub-module, and the input of the lightweight multi-view decoupling convolution module is connected to the output of the next sub-module through a residual path to form the final output of the lightweight multi-view decoupling convolution module; the multi-level separation convolutional neural network is trained through the training data set, and parameters of the multi-level separation convolutional neural network are optimized; and identifying the fire hazard through the trained multi-level separation convolutional neural network. The method is suitable for fire identification.

Description

Technical Field

[0001] The present invention relates to the field of intelligent fire identification, and in particular to a method, device, cloud equipment and computer device for intelligent fire identification. Background Art

[0002] Traditional fire monitoring systems are usually based on sensor technologies such as smoke detectors, temperature sensors, flame sensors, etc. Although these sensors are feasible in certain situations, they are often affected by environmental factors such as lighting conditions, atmospheric conditions, and visibility issues.

[0003] In the field of fire image recognition, traditional computer vision methods are often limited by issues such as lighting changes, smoke, response time, and image quality. Traditional methods have difficulty capturing key features in fire images, resulting in unstable recognition performance and low recognition accuracy. Therefore, in recent years, deep learning technology, especially convolutional neural networks (CNNs), has become a powerful tool to solve this problem. CNNs can automatically learn image features, thereby improving the accuracy and robustness of fire image recognition.

[0004] Although CNN has made significant progress in the field of fire image recognition, it still has some limitations. Existing CNNs usually only use traditional convolution kernels, such as 3×3 convolution kernels, which cannot fully capture features of different scales. In addition, they lack adaptability to multi-scale features, making it challenging to timely recognize fire images in different environments. The limitations of existing technologies indicate the need for a new method that can better handle multi-scale features, improve the recognition performance and response speed of fire images, and overcome the problems of illumination changes, smoke, and perspective. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a fire intelligent identification method, device, cloud device and computer device, which decouples traditional convolution through a multi-level separation convolutional neural network, and uses multiple convolutional layers to extract features of images at different scales and levels, which not only improves the model efficiency and better captures the key features in the fire image, but also improves the accuracy of fire identification.

[0006] The present invention adopts the following technical solutions to achieve the above-mentioned purpose. In a first aspect, the present invention provides a fire intelligent identification method, comprising: S1. Construct a multi-level separation convolutional neural network, which is used to perform convolution processing, feature extraction, data dimension reduction and classification on the input fire image. The multi-level separation convolutional neural network is mainly composed of a lightweight multi-view decoupling convolution module, which is composed of at least two lightweight multi-view decoupling convolution submodules with different convolution directions connected in cascade, the output of the previous submodule is used as the input of the next submodule, and the input of the lightweight multi-view decoupling convolution module is connected to the output of the next submodule through a residual path to form the final output of the lightweight multi-view decoupling convolution module; S2. Train the multi-layer separation convolutional neural network through the training data set and optimize the parameters of the multi-layer separation convolutional neural network; S3. Fire is identified through the trained multi-layer separation convolutional neural network.

[0007] Furthermore, the multi-level separation convolutional neural network captures features of different sizes through different convolution kernel scales.

[0008] Furthermore, the lightweight multi-view decoupled convolution submodule divides the input into four groups on average to form four branches running in serial, decouples the traditional 3×3 convolution kernel into 1×3 and 3×1 heterogeneous convolution kernels to capture information in different directions of the horizontal and vertical axes of the image, and all four branches running in serial use 1×3 or 3×1 to encode the two orthogonal view information of the two-dimensional image, and add the result of the convolution of the previous branch as a residual connection to the next branch to capture multi-level information of the image, and finally merge the feature maps of the four branches, and use 1×1 convolution to fuse the features from different branches to obtain the multi-dimensional context information of the image as the output feature of the submodule.

[0009] Furthermore, S2 specifically includes: S201, performing data enhancement on the training image, including random horizontal flipping, random vertical flipping, random horizontal translation, random vertical translation, random brightness change, random color change and random Gaussian noise; S202, performing data normalization on the training image, normalizing the pixel values ​​of the image data to a range of 0 to 1; S203, constructing a weighted cross entropy loss function, measuring the output loss of the training sample through the loss function, calculating the error between the true label and the prediction result of the data sample, back-propagating the error to guide the learning of network model parameters, and iteratively optimizing the model parameters using the gradient descent method; The weighted loss function is: ; is the weight of each category in the prediction probability map, which is used to weight the contribution of the category with a small proportion in the prediction map to the loss function; It is Class The true value of the image, It is Class The predicted value of the image.

[0010] In a second aspect, the present invention provides a fire intelligent identification device, which is used to implement the fire intelligent identification method as described above, comprising: A model building module is used to build a multi-level separation convolutional neural network, which is used to perform convolution processing, feature extraction, data dimension reduction and classification on the input fire image. The multi-level separation convolutional neural network is mainly composed of a lightweight multi-view decoupling convolution module, which is composed of at least two lightweight multi-view decoupling convolution submodules with different convolution directions in cascade, the output of the previous submodule is used as the input of the next submodule, and the input of the lightweight multi-view decoupling convolution module is connected to the output of the next submodule through a residual path to form the final output of the lightweight multi-view decoupling convolution module; the multi-level separation convolutional neural network captures features of different sizes through different convolution kernel scales; A training optimization module is used to train a multi-layer separation convolutional neural network through a training data set and optimize the parameters of the multi-layer separation convolutional neural network; The recognition module recognizes fires through a trained multi-layer separation convolutional neural network.

[0011] Furthermore, the lightweight multi-view decoupled convolution submodule divides the input into multiple groups on average to form multiple branches running in serial, decouples the traditional 3×3 convolution kernel into 1×3 and 3×1 heterogeneous convolution kernels to capture information in different directions of the horizontal and vertical axes of the image, and all the multiple branches running in serial use 1×3 or 3×1 to encode the two orthogonal view information of the two-dimensional image, and add the result of the convolution of the previous branch as a residual connection to the next branch to capture multi-level information of the image; finally, the feature maps of multiple branches are merged, and the features from different branches are fused using 1×1 convolution to obtain the multi-dimensional context information of the image as the output feature of the submodule.

[0012] Furthermore, the training optimization module is specifically used to: Perform data augmentation on training images, including random horizontal flipping, random vertical flipping, random horizontal translation, random vertical translation, random brightness change, random color change, and random Gaussian noise; Perform data normalization on the training images, normalizing the pixel values ​​of the image data to the range of 0 to 1; Construct a weighted cross entropy loss function to measure the output loss of the training sample. By calculating the error between the true label and the predicted result of the data sample, the error is back-propagated to guide the learning of network model parameters, and the gradient descent method is used to iteratively optimize the model parameters. The weighted loss function is: ; is the weight of each category in the prediction probability map, which is used to weight the contribution of the category with a small proportion in the prediction map to the loss function; It is Class The true value of the image, It is Class The predicted value of the image.

[0013] In a third aspect, the present invention provides a cloud device, which includes a server, and the fire intelligent identification device as described above is installed on the server.

[0014] In a fourth aspect, the present invention provides a computer device, including a memory, wherein the memory stores program instructions, and when the program instructions are run, the fire intelligent identification method as described above is executed.

[0015] The beneficial effects of the present invention are: The present invention decouples traditional convolution through a multi-level separation convolutional neural network and uses multiple convolutional layers to extract features of images at different scales and levels, thereby improving model efficiency and better capturing key features in fire images and improving fire identification accuracy.

[0016] The present invention trains a multi-layer separation convolutional neural network through a training data set, optimizes the parameters of the multi-layer separation convolutional neural network, and can adapt to fire images of different types and situations. This enables the present invention to perform real-time and efficient fire identification and to operate stably in different environments.

[0017] The present invention can perform highly accurate fire identification, can adapt to various environments and images, and can further provide the capability of instant alarm. The present invention has wide adaptability in the fields of fire monitoring and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a fire intelligent identification method provided by an embodiment of the present invention; Figure 2 Schematic diagram of a multi-layer separation convolutional neural network provided by an embodiment of the present invention; Figure 3is a schematic diagram of a lightweight multi-view decoupled convolution module provided by an embodiment of the present invention; Figure 4 It is a schematic diagram of a lightweight multi-view decoupled convolution submodule provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0020] The present invention provides a fire intelligent identification method, such as Figure 1 As shown, the specific steps include: Construct a multi-level separation convolutional neural network. The multi-level separation convolutional neural network is used to perform convolution processing, feature extraction, data dimension reduction and classification on the input fire image. The structure of the multi-level separation convolutional neural network is as follows: Figure 2 As shown in the figure, the network consists of a feature extractor and a classifier. The feature extractor consists of multiple convolutional layers and pooling layers. The maximum pooling layer is used between the convolutional layers to downsample and reduce the size of the feature map. The convolution of the convolutional layer is replaced by a lightweight multi-view decoupled convolutional module. The multi-level separation convolutional neural network uses a lightweight multi-view decoupled convolutional module as the basic convolutional module.

[0021] The lightweight multi-view decoupled convolution module structure is as follows Figure 3 As shown in the figure, the lightweight multi-view decoupled convolution module is composed of at least two cascaded lightweight multi-view decoupled convolution sub-modules with different convolution directions. The output of the previous sub-module is used as the input of the next sub-module. The input of the lightweight multi-view decoupled convolution module is connected to the output of the next sub-module through a residual path to form the final output of the lightweight multi-view decoupled convolution module.

[0022] Lightweight multi-view decoupled convolution submodule such as Figure 4 As shown in the figure, the lightweight multi-view decoupled convolution submodule divides the input into 4 groups on average to form 4 branches running in serial mode, decouples the traditional 3×3 convolution kernel into 1×3 and 3×1 heterogeneous convolution kernels to capture the information in different directions of the horizontal and vertical axes of the image, and all the 4 branches running in serial mode use 1×3 or 3×1 to encode the two orthogonal view information of the two-dimensional image, and add the result of the convolution of the previous branch as a residual connection to the next branch to capture the multi-level information of the image; finally, the feature maps of the 4 branches are merged, and the features from different branches are fused using 1×1 convolution to obtain the multi-dimensional context information of the image as the output feature of the submodule.

[0023] The present invention uses a fire image dataset as a specific embodiment, which contains 225 color image data. This fire image dataset consists of two parts, one of which is 119 color images manually labeled as fire images, and the other is 106 color images manually labeled as non-fire images. The fire image dataset is preprocessed, the image is loaded, and the image matrix represented by digital is obtained; the image pixel value is normalized to the range of [0,1], and the normalized expression is:

[0024] x and y are the pixel values ​​before and after the conversion, respectively, and MaxValue and MinValue represent the maximum and minimum values ​​of the sample pixel. Through this linear conversion, the pixel value can be mapped to a range of [0,1].

[0025] Data augmentation: The input image is randomly flipped horizontally with a probability of 50%, randomly flipped vertically with a probability of 50%, randomly shifted horizontally by 10 pixels with a probability of 20%, randomly shifted vertically by 10 pixels with a probability of 20%, randomly changed the brightness in the range of positive 10 to negative 10 with a probability of 20%, randomly changed the color channel with a probability of 20%, and randomly added Gaussian noise with a probability of 10%.

[0026] Parameter setting: The training process of this example is iterated 100 times in total, the basic learning rate is 0.0001, the learning rate is multiplied by 0.9 every 20 iterations, and the model with the best validation set result is saved as the optimal model. The Adam optimizer is used in the training process, the momentum is set to 0.9, and the other parameters are the default parameters of the function.

[0027] Model training: This example uses a weighted cross entropy loss function based on log-likelihood estimation. By calculating the error between the predicted result and the true label of the image, the chain rule is used to back-propagate the error gradient to guide the network model to update the network parameter value. The weighted cross entropy loss function used in this example is as follows: ; is the weight of each category in the prediction probability map, which is used to weight the contribution of the category with a small proportion in the prediction map to the loss function; It is Class The true value of the image, It is Class The predicted value of the image.

[0028] In this example, the images are classified into two categories, namely, fire images and non-fire images, so the parameter m is 2; since fire images are in the minority and non-fire images are in the majority, the w weight of fire images in training is set to 2, and the w weight of non-fire images in training is set to 0.5.

[0029] Image prediction: Call the saved optimal model parameters, perform image preprocessing on the image to be predicted and input it into the model. The model will give a prediction on whether the current scene is a fire.

[0030] Training results: The evaluation results of the fire image intelligent recognition method based on a multi-layer separation convolutional neural network proposed in this application are compared with the classic image classification method ResNet50. The results are shown in Table 1: Table 1 Fire identification comparison chart method Accuracy (%) Parameter quantity (M) Multi-layer Separation Convolutional Neural Network 97.5 1.7 ResNet50 90.2 26.0 The accuracy is used to measure the classification effect of the method. The closer it is to 100%, the better the classification effect. The parameter quantity (M) represents the quantity of all parameters of the network model. M is the abbreviation of million. The smaller the parameter, the simpler the network model is, the easier it is to calculate, and the faster it runs.

[0031] The above results show that compared with the prior art, the present invention has the following beneficial effects: Multi-layer separation convolutional neural network can better capture multi-scale features and improve the recognition accuracy of fire images.

[0032] This method can adapt to fire images in different environments, including illumination changes, smoke and perspective issues.

[0033] This method can process and identify the impact of fire scenes in real time, improving the response speed and performance of the system.

[0034] The specific implementation of the method can be applied in many fields, including but not limited to the following: fire monitoring of buildings and industrial facilities, fire monitoring of cities and forests, emergency response of fire monitoring centers and other applications requiring fire image recognition.

[0035] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A fire intelligent identification method, characterized in that: include: S1. Construct a multi-level separation convolutional neural network, which is used to perform convolution processing, feature extraction, data dimension reduction and classification on the input fire image. The multi-level separation convolutional neural network is composed of a lightweight multi-view decoupling convolution module, which is composed of at least two lightweight multi-view decoupling convolution submodules with different convolution directions connected in cascade, the output of the previous submodule is used as the input of the next submodule, and the input of the lightweight multi-view decoupling convolution module is connected to the output of the next submodule through a residual path to form the final output of the lightweight multi-view decoupling convolution module; S2. Train the multi-layer separation convolutional neural network through the training data set and optimize the parameters of the multi-layer separation convolutional neural network; S3. Fire is identified through the trained multi-layer separation convolutional neural network.

2. The fire intelligent identification method according to claim 1, characterized in that: The multi-layer separation convolutional neural network captures features of different sizes through different convolution kernel scales.

3. The fire intelligent identification method according to claim 1, characterized in that: The lightweight multi-view decoupled convolution submodule divides the input into multiple groups on average to form multiple branches running in serial, decouples the traditional 3×3 convolution kernel into 1×3 and 3×1 heterogeneous convolution kernels to capture information in different directions of the horizontal and vertical axes of the image, and all the multiple branches running in serial use 1×3 or 3×1 to encode the two orthogonal view information of the two-dimensional image, and add the result of the convolution of the previous branch as a residual connection to the next branch to capture multi-level information of the image. Finally, the feature maps of multiple branches are merged, and the features from different branches are fused using 1×1 convolution to obtain the multi-dimensional context information of the image as the output feature of the submodule.

4. The fire intelligent identification method according to claim 1, characterized in that: S2 specifically includes: S201, performing data enhancement on the training image, including random horizontal flipping, random vertical flipping, random horizontal translation, random vertical translation, random brightness change, random color change and random Gaussian noise; S202, performing data normalization on the training image, normalizing the pixel values ​​of the image data to a range of 0 to 1; S203, constructing a weighted cross entropy loss function, measuring the output loss of the training sample through the loss function, calculating the error between the true label and the prediction result of the data sample, back-propagating the error to guide the learning of network model parameters, and iteratively optimizing the model parameters using the gradient descent method; The weighted loss function is: ； is the weight of each category in the prediction probability map, which is used to weight the contribution of the category with a small proportion in the prediction map to the loss function; It is Class The true value of the image, It is Class The predicted value of the image.

5. A fire intelligent identification device, used to implement the fire intelligent identification method according to any one of claims 1 to 4, characterized in that: include: A model building module, used to build a multi-level separation convolutional neural network, the multi-level separation convolutional neural network is used to perform convolution processing, feature extraction, data dimension reduction and classification on the input fire image, the multi-level separation convolutional neural network is composed of a lightweight multi-view decoupling convolution module, the lightweight multi-view decoupling convolution module is composed of at least two lightweight multi-view decoupling convolution submodules with different convolution directions cascaded, the output of the previous submodule is used as the input of the next submodule, the input of the lightweight multi-view decoupling convolution module is connected to the output of the next submodule through a residual path to form the final output of the lightweight multi-view decoupling convolution module; A training optimization module is used to train a multi-layer separation convolutional neural network through a training data set and optimize the parameters of the multi-layer separation convolutional neural network; The recognition module recognizes fires through a trained multi-layer separation convolutional neural network.

6. The fire intelligent identification device according to claim 5, characterized in that: The multi-layer separation convolutional neural network captures features of different sizes through different convolution kernel scales.

7. The fire intelligent identification device according to claim 5, characterized in that: The lightweight multi-view decoupled convolution submodule divides the input into multiple groups on average to form multiple branches running in serial, decouples the traditional 3×3 convolution kernel into 1×3 and 3×1 heterogeneous convolution kernels to capture information in different directions of the horizontal and vertical axes of the image, and all the multiple branches running in serial use 1×3 or 3×1 to encode the two orthogonal view information of the two-dimensional image, and add the result of the convolution of the previous branch as a residual connection to the next branch to capture multi-level information of the image. Finally, the feature maps of multiple branches are merged, and the features from different branches are fused using 1×1 convolution to obtain the multi-dimensional context information of the image as the output feature of the submodule.

8. The fire intelligent identification device according to claim 5, characterized in that: The training optimization module is specifically used to: Perform data augmentation on training images, including random horizontal flipping, random vertical flipping, random horizontal translation, random vertical translation, random brightness change, random color change, and random Gaussian noise; Perform data normalization on the training images, normalizing the pixel values ​​of the image data to the range of 0 to 1; Construct a weighted cross entropy loss function to measure the output loss of the training sample. By calculating the error between the true label and the predicted result of the data sample, the error is back-propagated to guide the learning of network model parameters, and the gradient descent method is used to iteratively optimize the model parameters. The weighted loss function is: ； is the weight of each category in the prediction probability map, which is used to weight the contribution of the category with a small proportion in the prediction map to the loss function; It is Class The true value of the image, It is Class The predicted value of the image.

9. A cloud device, comprising a server, characterized in that: The intelligent fire identification device as described in any one of claims 5 to 8 is installed on the server.

10. A computer device comprising a memory, wherein the memory stores program instructions, characterized in that: When the program instructions are executed, the fire intelligent identification method according to any one of claims 1 to 4 is executed.

Images