Smoke and flame detection method, device, equipment and storage medium

Abstract

The present invention relates to the technical field of computer vision, and more particularly to a smoke and flame detection method, apparatus, device, and storage medium. The smoke and flame detection method comprises: preprocessing labeled image data of a labeled target object, and using the preprocessed labeled image data to establish a training dataset, wherein the target object includes smoke and flame; training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism using the training dataset to obtain a target neural network; obtaining image data to be inspected of an environment to be inspected, inputting the image data to the target neural network model, and obtaining a detection result output by the target neural network model. The present invention improves the accuracy of smoke and flame detection.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a smoke and flame detection method, device, equipment and computer-readable storage medium. Background Art

[0002] With the rapid development of deep learning in the field of computer vision, deep learning-based smoke and flame detection algorithms have become the mainstream smoke and flame detection method. Deep learning-based smoke and flame detection methods are categorized into one-stage and two-stage approaches based on network structure. Two-stage approaches divide the detection problem into two stages: first, obtaining candidate regions for position refinement and classification. Representative networks include R-CNN, Faster R-CNN, and Mask-RCNN. One-stage approaches bypass the candidate region selection stage and directly generate object category probabilities and position coordinates. Compared to two-stage approaches, they are faster. Representative networks include the YOLO series of algorithms, SSD, and CenterNet.

[0003] However, due to the huge deformation of the target in the image data of the detection area, the unclear target in the image data when the detection area is too far away from the image sensor, and the lack of feature information in the image data of the detection area, the current smoke and flame detection algorithms are prone to missed detections, false detections, etc., resulting in inaccurate smoke and flame detection results. Summary of the Invention

[0004] The main purpose of the present invention is to provide a smoke and flame detection method, device, equipment and computer-readable storage medium, aiming to improve the accuracy of smoke and flame detection.

[0005] To achieve the above object, the present invention provides a smoke and flame detection method, which comprises the following steps:

[0006] Preprocessing labeled image data of labeled targets, and using the preprocessed labeled image data to establish a training data set, wherein the targets include smoke and flames;

[0007] An initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism is trained using the training data set to obtain a target neural network model;

[0008] Obtain image data to be inspected of the environment to be inspected, input the image data to be inspected into the target neural network model, and obtain a detection result output by the target neural network model.

[0009] Optionally, the initial neural network model includes a feature extraction network and a feedforward prediction network, wherein the feature extraction network includes a plurality of feature extraction blocks, and each of the feature extraction blocks includes a convolution block and an attention mechanism block;

[0010] The step of training the initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism using the training data set to obtain the target neural network model includes:

[0011] Inputting the training data set into the feature extraction network, and sequentially performing feature extraction through the plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of the previous feature extraction block in the plurality of feature extraction blocks serves as the input data for the next feature extraction block;

[0012] Fusing a plurality of the training features to obtain a learning feature;

[0013] According to the training results, the network parameters in the initial neural network model are adjusted to obtain the target neural network model.

[0014] Optionally, the step of performing feature extraction by any target extraction block in each of the feature extraction blocks to obtain the training feature includes:

[0015] Performing feature extraction through the convolution block of the target extraction block to obtain feature data;

[0016] The feature data extracted by the convolution block is subjected to local feature enhancement by the attention mechanism block of the target extraction block to obtain the training feature.

[0017] Optionally, the convolution block includes a first convolution layer and a pooling layer.

[0018] Optionally, the feedforward prediction network includes a second convolutional layer, a hidden layer and a linear prediction layer; wherein,

[0019] The second convolutional layer is used to extract the target features through a 3*3 convolution kernel;

[0020] The hidden layer includes three fully connected layers, and the first fully connected layer and the second fully connected layer are each connected to an activation function.

[0021] Optionally, the step of preprocessing the labeled image data includes:

[0022] Performing data enhancement on the labeled image data through geometric transformation;

[0023] performing mixed processing on the labeled image data of different targets;

[0024] The labeled image data of different image backgrounds are mixed.

[0025] Optionally, the detection result includes the target category, the target existence probability and the target coordinates.

[0026] In addition, to achieve the above-mentioned object, the present invention further provides a smoke and flame detection device, which includes:

[0027] a preprocessing module, configured to preprocess labeled image data of labeled targets, and establish a training data set using the preprocessed labeled image data, wherein the targets include smoke and flames;

[0028] A training module is used to train an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism using the training data set to obtain a target neural network model;

[0029] The detection module is used to obtain image data to be detected of the environment to be detected, input the image data to be detected into the target neural network model, and obtain the detection result output by the target neural network model.

[0030] In addition, to achieve the above-mentioned objectives, the present invention also provides a smoke and flame detection device, which includes a memory, a processor, and a smoke and flame detection program stored in the memory and executable on the processor. When the smoke and flame detection program is executed by the processor, the steps of the above-mentioned smoke and flame detection method are implemented.

[0031] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium, on which a smoke and flame detection program is stored. When the smoke and flame detection program is executed by a processor, the steps of the above-mentioned smoke and flame detection method are implemented.

[0032] In the present invention, the labeled image data of the labeled target object is preprocessed, and a training data set is established using the preprocessed labeled image data, wherein the target object includes smoke and flame. The initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism is trained using the training data set to obtain a target neural network, and the image data to be inspected of the environment to be inspected is obtained. The image data to be inspected is input into the target neural network model to obtain the detection result output by the target neural network model.

[0033] In the present invention, a convolutional neural network is combined with a multi-head attention mechanism to construct an initial neural network model. The convolution block can learn local representation information. The architecture of the convolution block combined with the multi-head attention mechanism has a strong ability to capture low-frequency signals, so that the trained target neural network model can perceive the global information of the image data to be inspected, avoiding the occurrence of missed detection and false detection due to incomplete feature information. At the same time, the initial neural network model constructed based on the convolutional neural network combined with the multi-head attention mechanism can pay attention to smoke and flames of different sizes and shapes, and can also avoid missed detection and false detection, thereby improving the accuracy of smoke and flame detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a first embodiment of the smoke and flame detection method of the present invention;

[0035] Figure 2 This is a flow chart of an embodiment of a smoke and flame detection method according to the present invention;

[0036] Figure 3 This is a structural diagram of an embodiment of a smoke and flame detection method of the present invention;

[0037] Figure 4 This is a structural diagram of an embodiment of a smoke and flame detection method of the present invention;

[0038] Figure 5 This is a schematic diagram of the functional modules of the smoke and flame detection device involved in the embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of the structure of a smoke and flame detection device according to an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram of the structure of a computer-readable storage medium involved in an embodiment of the present invention.

[0041] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0042] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] The embodiment of the present invention provides a smoke and flame detection method, referring to Figure 1 As shown, Figure 1 1 is a flow chart of the first embodiment of the smoke and flame detection method of the present invention.

[0044] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this application.

[0045] In this embodiment, the device that executes the smoke and flame detection method of the embodiment of the present invention can be a detection device at the scene of the environment to be detected, such as a camera, or a device that establishes a communication connection with the detection device, such as a smart phone, PC (Personal Computer), tablet computer, portable computer, etc. The following description of the execution subject is omitted for convenience. The smoke and flame detection method of this embodiment includes:

[0046] Step S10: preprocessing the labeled image data of the labeled target object, and using the preprocessed labeled image data to establish a training data set, wherein the target object includes smoke and flame;

[0047] In this embodiment, flames and smoke are referred to as targets, and image data of the marked targets (hereinafter referred to as marked data for distinction) is acquired.

[0048] Preprocessing the labeled image data. In a specific embodiment, the process of preprocessing the labeled image data can be to eliminate information in the labeled image data that is not related to the target object, or to enhance the labeled image data. There is no specific limitation here and it can be set according to actual needs.

[0049] In this embodiment, after preprocessing the labeled image data, a training data set is established using the preprocessed labeled image data.

[0050] Step S20: training the initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism using the training data set to obtain a target neural network model;

[0051] In this embodiment, an initial neural network model is constructed based on a convolutional neural network and a multi-head attention mechanism. Specifically, in this embodiment, the backbone structure of the initial neural network model may include a feature extraction network and a feedforward prediction network.

[0052] The initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism is trained using the training data set to obtain a target neural network model. In a specific embodiment, the training of the initial neural network model can be based on a loss function.

[0053] Step S30: Acquire the image data to be inspected of the environment to be inspected, input the image data to be inspected into the target neural network model, and obtain the detection result output by the target neural network model.

[0054] In this embodiment, image data to be inspected of the environment to be inspected is obtained, and the image data to be inspected is input into a target neural network model to obtain a detection result output by the target neural network model.

[0055] In a specific embodiment, the image data to be inspected may be acquired by an image sensor disposed in the area to be inspected, for example, a camera.

[0056] In a specific embodiment, the detection result may include the category of the target object, such as smoke or flame, the probability of the target object appearing, and the coordinates of the target object.

[0057] Furthermore, in one embodiment, when the detection result indicates that a target object exists in the image data to be inspected, an alarm message may be issued so that the user can perform safety processing in a timely manner.

[0058] Further, a feasible embodiment is proposed, referring to Figure 2 , Figure 2 This is a flow chart of an embodiment of the smoke and flame detection method of the present invention. In this embodiment, the labeled image data of the labeled target object is preprocessed, and the preprocessed labeled image data is used to establish a training data set. The initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism is trained using the training data set to obtain the target neural network model (i.e. Figure 2 After the data preprocessing shown in , CNN is combined with a multi-head attention mechanism to detect and recognize the target).

[0059] Obtain the image data to be inspected of the environment to be inspected, input the image data to be inspected into the target neural network model, and obtain the detection result output by the target neural network model (that is, Figure 2 (as shown in the determination of whether there is smoke and flame).

[0060] In this embodiment, if smoke and flames exist in the image data to be inspected, an alarm is triggered in the detection system; if smoke and flames do not exist in the image data to be inspected, an alarm is not triggered in the detection system, and the detection of the image data to be inspected is terminated.

[0061] Furthermore, in some feasible embodiments, in the above step S10: pre-processing the marked image data includes:

[0062] Step S101: performing data enhancement on the labeled image data through geometric transformation;

[0063] In this embodiment, data enhancement is performed on the labeled image data. Specifically, in this embodiment, single-sample data enhancement is performed through geometric transformation. In a specific embodiment, the geometric transformation includes processing such as cropping, rotating, or scaling the labeled image data, which will not be described in detail here.

[0064] Step S102: performing mixing processing on the labeled image data of different targets;

[0065] In this embodiment, the labeled image data is also processed using multi-sample data enhancement. Specifically, in this embodiment, the labeled image data of different targets are mixed to expand the number of training data sets and improve the generalization ability of the target neural network model and the robustness of the test.

[0066] Specifically, in this embodiment, a Mixup algorithm may be used for mixing processing, wherein the processing process is to combine two labeled image data with different target object labels with a certain probability. The calculation formula is: mixed =λ*x1+(1-λ)*x2, where λ is the mixing coefficient calculated by Beta distribution and can be set according to actual needs. x1 and x2 are labeled image data.

[0067] Step S103: performing mixing processing on the labeled image data of different image backgrounds.

[0068] In this embodiment, the labeled image data of different image backgrounds are mixed, thereby expanding the number of training data sets and improving the generalization ability of the target neural network model and the robustness of the test. The specific processing process can be referred to step S102 and will not be repeated here.

[0069] It should be noted that, in this embodiment, by performing single-sample data enhancement on the labeled image data, performing mixed processing on the labeled image data of different targets, and performing mixed processing on the labeled image data of different image backgrounds, the number of training data sets can be expanded, and the generalization ability of the target neural network model and the robustness of the test can be improved.

[0070] Furthermore, in some feasible embodiments, in this embodiment, the detection result includes the target category, the probability of the target appearing, and the coordinates of the target. In a specific embodiment, the coordinates of the target are the coordinates of the center point of the target.

[0071] In a specific embodiment, the coordinates in the detection results can be the coordinates of the target object in the feature data extracted from the target neural network model, or the coordinates of the target object in the feature data can be mapped to the coordinates in the image data to be inspected, or the coordinates of the target object in the feature data can be mapped to the coordinates in the environment to be inspected. There is no specific limitation here and it can be set according to actual needs.

[0072] Furthermore, in a specific embodiment, the detection result may also include the width and height of the target object, and the area of ​​the target object can be obtained based on the width and height of the target object, thereby determining the range of smoke and flame occurrence, making the smoke and flame detection results more practical.

[0073] It should be noted that the detection results in this embodiment include the target category, the probability of target occurrence and the coordinates of the target. Compared with only detecting the probability of target occurrence, this embodiment improves the detail of the detection results, making the detection results more practical.

[0074] In this embodiment, the labeled image data of the labeled target object is preprocessed, and a training data set is established through the preprocessed labeled image data, wherein the target object includes smoke and flames. The initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism is trained through the training data set, and the target neural network model obtains the image data to be inspected of the environment to be inspected, and the image data to be inspected is input into the target neural network model to obtain the detection result output by the target neural network.

[0075] In this embodiment, a convolutional neural network is combined with a multi-head attention mechanism to construct an initial neural network model. The convolution block can learn local representation information. The architecture of the convolution block combined with the multi-head attention mechanism has a strong ability to capture low-frequency signals, so that the initial neural network model can learn global information, so that the trained target neural network model can perceive the global information of the image data to be inspected, avoiding the occurrence of missed detection and false detection due to incomplete feature information. At the same time, the target neural network model constructed based on the convolutional neural network combined with the multi-head attention mechanism can pay attention to smoke and flames of different sizes and shapes, and can also avoid missed detection and false detection, thereby improving the accuracy of smoke and flame detection.

[0076] Furthermore, based on the above-mentioned first embodiment, a second embodiment of the smoke and flame detection method of the present invention is proposed. In this embodiment, the initial neural network model includes a feature extraction network and a feedforward prediction network. The feature extraction network includes multiple feature extraction blocks, and each feature extraction block includes a convolution block and an attention mechanism block.

[0077] In this embodiment, the above step S20: training the initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism using the training data set to obtain the target neural network model includes:

[0078] Step S201: inputting the training data set into the feature extraction network, and sequentially performing feature extraction through the plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of the previous feature extraction block among the plurality of feature extraction blocks serves as the input data of the next feature extraction block;

[0079] In this embodiment, the training data set is input into the feature extraction network, and the multiple feature extraction blocks in the feature extraction network perform feature extraction in sequence to obtain training features.

[0080] Specifically, in this embodiment, each feature extraction block can be connected in series, and the process of performing feature extraction in sequence through multiple feature extraction blocks in the feature extraction network can be: the training features extracted by any target extraction block in each feature extraction block are input into the next feature extraction block connected to the target extraction block, that is, the output result of the previous feature extraction block in the multiple feature extraction blocks is the input data of the next feature extraction block.

[0081] For example, in one embodiment, when two feature extraction blocks are set in the feature extraction network, the two feature extraction blocks can be connected in series. Specifically, the first feature extraction block performs feature extraction on the training data set, and the training features extracted by the first feature extraction block are used as input data of the second feature extraction block for the second feature extraction block to perform feature extraction.

[0082] Step S202: fusing multiple training features to obtain learning features;

[0083] In this embodiment, feature extraction is performed on the training data set through a feature extraction network to obtain training features corresponding to each feature extraction block, and then multiple training features are fused to obtain learning features.

[0084] In a specific implementation, the feature fusion method may be cascade feature fusion, series feature fusion, parallel strategy fusion, etc., which will not be described in detail here.

[0085] Step S203: According to the training result, the network parameters in the initial neural network model are adjusted to obtain the target neural network model.

[0086] In this embodiment, the learning features are input into a feedforward prediction network for learning. Based on the training results, the network parameters of the initial neural network model are adjusted to obtain a target neural network model. Specifically, in one embodiment, the network parameters of the initial neural network model can be adjusted based on a loss function. In this embodiment, when the loss function begins to converge, the training of the initial neural network model is confirmed to be complete, and the target neural network model is obtained. In a specific embodiment, the feedforward prediction network may include a convolutional layer, a hidden layer, and a linear prediction layer.

[0087] It should be noted that, in this embodiment, a convolutional neural network is combined with a multi-head attention mechanism to construct an initial neural network model. The convolution block can learn local representation information. The architecture of the convolution block combined with the multi-head attention mechanism has a strong ability to capture low-frequency signals, so that the trained target neural network model can perceive the global information of the image data to be inspected, avoiding the occurrence of missed detection and false detection due to incomplete feature information. At the same time, the initial neural network model constructed based on the convolutional neural network combined with the multi-head attention mechanism can pay attention to smoke and flames of different sizes and shapes, and can also avoid missed detection and false detection, thereby improving the accuracy of smoke and flame detection.

[0088] Furthermore, in some feasible embodiments, in the above step S201, the step of extracting features by using any target extraction block in each of the feature extraction blocks to obtain the training features includes:

[0089] Step S2011: performing feature extraction through the convolution block of the target extraction block to obtain feature data;

[0090] In this embodiment, each target extraction block in each feature extraction block includes a convolution block and an attention mechanism block. Feature extraction is performed using the convolution block of the target extraction block to obtain feature data. In specific implementations, there are no restrictions on the step size and convolution kernel of the convolution block and they can be set according to actual needs.

[0091] Step S2012: performing local feature enhancement on the feature data extracted by the convolution block through the attention mechanism block of the target extraction block to obtain the training feature.

[0092] In this embodiment, the attention mechanism block of the target extraction block performs local feature enhancement on the feature data extracted by the convolution block to obtain training features.

[0093] Specifically, in this embodiment, the multi-head attention mechanism is used to enable the initial neural network model to focus on the feature information of the subspace in the feature data obtained by convolution on different feature extraction blocks, thereby capturing richer feature information.

[0094] In a specific implementation, the feature data obtained by convolution is straightened and the features are spliced ​​to generate the three matrices required for calculating self-attention to calculate the attention weight on the target block. The formula for calculating self-attention can be expressed as:

[0095]

[0096] Where Q represents the feature matrix, K is the key of the information, V is the value of the information, and S is the softmax (normalization) operation. In this embodiment, by comparing the similarity between Q and K, the V value corresponding to the K key is returned. According to the above formula, the greater the similarity between Q and K, the greater the attention weight.

[0097] Furthermore, in some feasible embodiments, the convolution block includes a first convolution layer and a pooling layer.

[0098] In this embodiment, the convolution block in the feature extraction block includes a first convolutional layer and a pooling layer. The first convolutional layer performs convolution to extract feature information, while the pooling layer performs feature downsampling. This reduces the computational complexity of the multi-head attention mechanism, significantly shortening the feature sequence length while capturing rich multi-layer features.

[0099] Specifically, refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of an embodiment of the smoke and flame detection method of the present invention. This embodiment designs four extraction blocks, each of which includes a convolution block and an attention mechanism block. The convolution block includes a first convolution layer and a pooling layer. In this embodiment, the feature data of each feature extraction block gradually decreases, and the feature data of each layer within the feature extraction block also gradually decreases, which is shown as follows: Figure 3 A pyramid-type structure is shown in .

[0100] Furthermore, in some feasible embodiments, the feedforward prediction network includes a second convolutional layer, a hidden layer, and a linear prediction layer; wherein,

[0101] The second convolutional layer is used to extract the target features through a 3*3 convolution kernel;

[0102] The hidden layer includes three fully connected layers, and the first fully connected layer and the second fully connected layer are each connected to an activation function.

[0103] In this embodiment, the feedforward prediction network includes a second convolutional layer, a hidden layer, and a linear prediction layer.

[0104] Specifically, in this embodiment, the second convolutional layer is used to extract target features through a 3*3 convolution kernel, which can further improve the feature representation capability of the target neural network model.

[0105] In this embodiment, the hidden layer in the feedforward prediction network includes three fully connected layers, and the first fully connected layer and the second fully connected layer are each connected to an activation function.

[0106] The fully connected layer is responsible for converting the feature data output by the convolution into vectors, thereby achieving an end-to-end learning process. Each node in the fully connected layer is connected to all nodes in the previous layer. Fully connected layers can reduce the influence of feature position on detection results and improve the robustness of the entire target neural network model. In specific implementations, the dimensions of the hidden layer can be set according to actual user needs and are not limited here.

[0107] The activation function determines the content to be transmitted to the next fully connected layer, introducing nonlinear characteristics into the neural network. In this embodiment, the activation function can be a ReLU function.

[0108] Furthermore, in a feasible embodiment, the initial neural network model includes a feature extraction network and a feedforward prediction network, wherein the feature extraction network includes multiple feature extraction blocks, each of which includes a convolution block and an attention mechanism block. In this embodiment, the convolution block includes a first convolution layer and a pooling layer. The feedforward prediction network includes a second convolution layer, a hidden layer, and a linear prediction layer, wherein the second convolution layer is used to extract target features using a 3*3 convolution kernel, and the hidden layer includes three fully connected layers, with the first fully connected layer and the second fully connected layer each being connected to a ReLU activation function.

[0109] Further, in a feasible embodiment, referring to Figure 4 , Figure 4 This is a flow chart of an embodiment of the smoke and flame detection method of the present invention. In this embodiment, the process of detecting the image data to be detected can be: inputting the image data to be detected into the feature extraction module in the target neural network model, wherein the feature extraction module is constructed based on a convolutional neural network and a multi-head attention mechanism, and inputting the extracted features to be detected into a feedforward prediction network to obtain a detection result.

[0110] In this embodiment, the initial neural network model includes a feature extraction network and a feedforward prediction network, wherein the feature extraction network includes multiple feature extraction blocks, each of which includes a convolution block and an attention mechanism block. In this embodiment, a training data set is input into the feature extraction network, and feature extraction is performed sequentially through multiple feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of the previous feature extraction block in the multiple feature extraction blocks serves as the input data of the next feature extraction block, and the multiple training features are fused to obtain learning features. Based on the training results, the network parameters in the initial neural network model are adjusted to obtain a target neural network model.

[0111] In this embodiment, a convolutional neural network is combined with a multi-head attention mechanism to construct an initial neural network model. The convolution block can learn local representation information. The architecture of the convolution block combined with the multi-head attention mechanism has a strong ability to capture low-frequency signals, so that the initial neural network model can learn global information, and the trained target neural network model can perceive the global information of the image data to be inspected, avoiding the occurrence of missed detection and false detection due to incomplete feature information. At the same time, the initial neural network model constructed based on the convolutional neural network combined with the multi-head attention mechanism can pay attention to smoke and flames of different sizes and shapes, and can also avoid missed detection and false detection, thereby improving the accuracy of smoke and flame detection.

[0112] In addition, the present invention also provides a smoke and flame detection device, referring to Figure 5 , Figure 5 Schematic diagram of the functional modules of the smoke and flame detection device according to the embodiment of the present invention. The smoke and flame detection device of the present invention comprises:

[0113] a preprocessing module 10 for preprocessing labeled image data of labeled targets, and using the preprocessed labeled image data to establish a training data set, wherein the targets include smoke and flames;

[0114] A training module 20 is configured to train an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism using the training data set to obtain a target neural network model;

[0115] The detection module 30 is used to obtain image data to be detected of the environment to be detected, input the image data to be detected into the target neural network model, and obtain the detection result output by the target neural network model.

[0116] Furthermore, the fourth calling module 40 is further configured to:

[0117] Furthermore, the initial neural network model includes a feature extraction network and a feedforward prediction network, wherein the feature extraction network includes a plurality of feature extraction blocks, and each of the feature extraction blocks includes a convolution block and an attention mechanism block;

[0118] The training module 20 is further used to:

[0119] Inputting the training data set into the feature extraction network, and sequentially performing feature extraction through the plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of the previous feature extraction block in the plurality of feature extraction blocks serves as the input data for the next feature extraction block;

[0120] Fusing a plurality of the training features to obtain a learning feature;

[0121] According to the training results, the network parameters in the initial neural network model are adjusted to obtain the target neural network model.

[0122] Furthermore, the training module 20 is also used to:

[0123] Performing feature extraction through the convolution block of the target extraction block to obtain feature data;

[0124] The feature data extracted by the convolution block is subjected to local feature enhancement by the attention mechanism block of the target extraction block to obtain the training feature.

[0125] Furthermore, the convolution block includes a first convolution layer and a pooling layer.

[0126] Furthermore, the feedforward prediction network includes a second convolutional layer, a hidden layer and a linear prediction layer; wherein,

[0127] The second convolutional layer is used to extract the target features through a 3*3 convolution kernel;

[0128] The hidden layer includes three fully connected layers, and the first fully connected layer and the second fully connected layer are each connected to an activation function.

[0129] Furthermore, the pre-processing module 10 is further configured to:

[0130] Performing data enhancement on the labeled image data through geometric transformation;

[0131] performing mixed processing on the labeled image data of different targets;

[0132] The labeled image data of different image backgrounds are mixed.

[0133] Furthermore, the detection result includes the target category, the target existence probability and the target coordinates.

[0134] Wherein, each functional module of the smoke and flame detection device implements the steps of the above-mentioned smoke and flame detection method when it is running.

[0135] In addition, the present invention also provides a smoke and flame detection device. Figure 6 , Figure 6 FIG1 is a schematic diagram of the structure of a smoke and flame detection device according to an embodiment of the present invention. The smoke and flame detection device according to the embodiment of the present invention may be a device for locally operating a smoke and flame detection system.

[0136] like Figure 6As shown, the smoke and flame detection device according to an embodiment of the present invention may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to implement communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0137] Memory 1005 is provided on the main body of the smoke and flame detection device. Memory 1005 stores a program that, when executed by processor 1001, implements corresponding operations. Memory 1005 is also used to store parameters used by the smoke and flame detection device. Memory 1005 can be high-speed RAM or non-volatile memory, such as disk storage. Memory 1005 can also optionally be a storage device independent of processor 1001.

[0138] Those skilled in the art will understand that Figure 6 The structure of the smoke and flame detection device shown in the figure does not constitute a limitation to the smoke and flame detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0139] like Figure 6 As shown, the memory 1005 as a storage medium may include an operating system, a network processing module, a user interface module, and a smoke and flame detection program.

[0140] exist Figure 6 In the smoke and flame detection device shown, the processor 1001 can be used to call the smoke and flame detection program stored in the memory 1005 and perform the following operations:

[0141] Preprocessing labeled image data of labeled targets, and using the preprocessed labeled image data to establish a training data set, wherein the targets include smoke and flames;

[0142] An initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism is trained using the training data set to obtain a target neural network model;

[0143] Obtain image data to be inspected of the environment to be inspected, input the image data to be inspected into the target neural network model, and obtain a detection result output by the target neural network model.

[0144] Furthermore, the initial neural network model includes a feature extraction network and a feedforward prediction network, wherein the feature extraction network includes multiple feature extraction blocks. The processor 1001 can also be used to call the smoke and flame detection program stored in the memory 1005 and perform the following operations:

[0145] Inputting the training data set into the feature extraction network, and sequentially performing feature extraction through the plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of the previous feature extraction block in the plurality of feature extraction blocks serves as the input data for the next feature extraction block;

[0146] Fusing a plurality of the training features to obtain a learning feature;

[0147] According to the training results, the network parameters in the initial neural network model are adjusted to obtain the target neural network model.

[0148] Furthermore, the processor 1001 may also be configured to call a smoke and flame detection program stored in the memory 1005 and perform the following operations:

[0149] Performing feature extraction through the convolution block of the target extraction block to obtain feature data;

[0150] The feature data extracted by the convolution block is subjected to local feature enhancement by the attention mechanism block of the target extraction block to obtain the training feature.

[0151] Furthermore, the convolution block includes a first convolution layer and a pooling layer.

[0152] Furthermore, the feedforward prediction network includes a second convolutional layer, a hidden layer and a linear prediction layer; wherein,

[0153] The second convolutional layer is used to extract the target features through a 3*3 convolution kernel;

[0154] The hidden layer includes three fully connected layers, and the first fully connected layer and the second fully connected layer are each connected to an activation function.

[0155] Furthermore, the processor 1001 may also be configured to call a smoke and flame detection program stored in the memory 1005 and perform the following operations:

[0156] Performing data enhancement on the labeled image data through geometric transformation;

[0157] performing mixed processing on the labeled image data of different targets;

[0158] The labeled image data of different image backgrounds are mixed.

[0159] Furthermore, the detection result includes the target category, the target existence probability and the target coordinates.

[0160] In addition, the present invention also provides a computer-readable storage medium. Figure 7 , Figure 7 The computer-readable storage medium stores a smoke and flame detection program, which, when executed by a processor, implements the steps of the above-mentioned smoke and flame detection method.

[0161] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0162] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0163] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a smoke and flame detection device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0164] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A smoke and flame detection method, characterized in that: The smoke and flame detection method comprises the following steps: Preprocessing labeled image data of labeled targets, and using the preprocessed labeled image data to establish a training data set, wherein the targets include smoke and flames; The initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism is trained using the training data set to obtain a target neural network model; wherein the initial neural network model includes a feature extraction network and a feedforward prediction network, and the feature extraction network includes a plurality of feature extraction blocks connected in series, each of which includes a convolution block and an attention mechanism block; in the process of training the initial neural network model, the feature data extracted by the convolution block is locally enhanced by the attention mechanism and the feature information of the subspace is captured by the self-attention weight to obtain the output result of the feature extraction block; wherein the step of training the initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism using the training data set to obtain the target neural network model includes: inputting the training data set into the feature extraction network, sequentially performing feature extraction through the plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of the previous feature extraction block in the plurality of feature extraction blocks is the input data of the next feature extraction block; fusing the plurality of training features to obtain learning features; inputting the learning features into the feedforward prediction network to obtain a training result; and adjusting the network parameters in the initial neural network model according to the training result to obtain the target neural network model; Acquire image data to be inspected of the environment to be inspected, input the image data to be inspected into the target neural network model, and obtain a detection result output by the target neural network model; The step of preprocessing the labeled image data includes: Performing data enhancement on the labeled image data through geometric transformation; Mixing the labeled image data with different target objects according to a preset mixing coefficient; The labeled image data of different image backgrounds are mixed.

2. The smoke and flame detection method according to claim 1, wherein: The step of sequentially extracting features through the plurality of feature extraction blocks in the feature extraction network to obtain training features includes: Performing feature extraction through a convolution block in the feature extraction block in the feature extraction network to obtain feature data; The feature data extracted by the convolution block is subjected to local feature enhancement by the attention mechanism block in the feature extraction block in the feature extraction network to obtain the training feature.

3. The smoke and flame detection method according to claim 2, wherein: The convolution block includes a first convolution layer and a pooling layer.

4. The smoke and flame detection method according to claim 1, wherein: The feedforward prediction network includes a second convolutional layer, a hidden layer and a linear prediction layer; wherein, The second convolutional layer is used to pass 3 The convolution kernel of 3 performs feature extraction on the learning features; The hidden layer includes three fully connected layers, and the first fully connected layer and the second fully connected layer are each connected to a layer of activation function.

5. The smoke and flame detection method according to any one of claims 1 to 4, characterized in that: The detection result includes the target category, the probability of the target existing and the target coordinates.

6. A smoke and flame detection device, characterized in that: The smoke and flame detection device comprises: a preprocessing module, configured to preprocess labeled image data of labeled targets, and establish a training data set using the preprocessed labeled image data, wherein the targets include smoke and flames; A training module is used to train the initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism through the training data set to obtain a target neural network model; wherein, the initial neural network model includes a feature extraction network and a feedforward prediction network, and the feature extraction network includes a plurality of feature extraction blocks connected in series, and each of the feature extraction blocks includes a convolution block and an attention mechanism block; in the process of training the initial neural network model, the feature data extracted by the convolution block is locally enhanced by the attention mechanism and the feature information of the subspace is captured by the self-attention weight to obtain the output result of the feature extraction block; wherein, the training data is used to train the initial neural network model based on the convolutional neural network and the multi-head attention mechanism to obtain a target neural network model ... The method comprises the following steps: inputting the training data set into the feature extraction network, sequentially performing feature extraction through a plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of a previous feature extraction block in the plurality of feature extraction blocks serves as input data for the next feature extraction block; fusing a plurality of the training features to obtain learning features; inputting the learning features into the feedforward prediction network to obtain training results; and adjusting the network parameters in the initial neural network model according to the training results to obtain the target neural network model. A detection module, configured to obtain image data to be detected of an environment to be detected, input the image data to be detected into the target neural network model, and obtain a detection result output by the target neural network model; Among them, the preprocessing module is also used to: perform data enhancement on the labeled image data through geometric transformation; mix the labeled image data with different targets according to a preset mixing coefficient; and mix the labeled image data with different image backgrounds.

7. A smoke and flame detection device, characterized in that, The smoke and flame detection device comprises: a memory, a processor, and a smoke and flame detection program stored in the memory and executable on the processor, wherein the smoke and flame detection program is configured to implement the steps of the smoke and flame detection method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a smoke and flame detection program, which, when executed by a processor, implements the steps of the smoke and flame detection method according to any one of claims 1 to 5.

Images