Smoke flame alarm method and device, electronic equipment and storage medium

Abstract

The application provides a smoke flame alarm method and device, electronic equipment and storage medium, and relates to the technical field of security and protection. The method comprises the following steps: acquiring at least one frame of image to be detected from a monitoring video of a target area; using a target smoke flame prediction model to analyze and process the image to be detected, so as to predict whether smoke and flame occur in the target area and obtain a prediction result; wherein the target smoke flame prediction model is a pre-trained improved YOLOV7 network; the pre-trained improved YOLOV7 network comprises a trained input module, a trained backbone network and a trained Head network connected in sequence; the backbone network comprises a plurality of grouped convolution modules; the Head network comprises an SE Block module representing an attention mechanism; when the prediction result is a prediction result of existing smoke and / or flame, corresponding alarm information is generated. The above method improves the accuracy of smoke flame alarm.

Description

TECHNICAL FIELD

[0001] The present application relates to the security technology field, and in particular to a smoke and flame alarm method and device, an electronic device, and a storage medium. BACKGROUND

[0002] Fire is one of the main disasters that most commonly endangers public safety and equipment safety, often causing huge resource and property losses. Especially in the field of power production units, smoke and flame alarm and prevention are crucial to the safety of important power equipment such as photovoltaic and fan box transformers, so adding a detection algorithm to the video monitoring system to locate and identify smoke and flame can ensure the safe operation and maintenance of power production, while also reducing operation and maintenance costs.

[0003] Traditional smoke and flame detection mainly uses infrared camera devices and smoke sensors to achieve flame and smoke alarm functions, but this approach has limitations such as misjudging sunset as flame and dust as smoke. In addition, it has good smoke and flame prediction results for indoor scenes, but cannot be applied on a large scale to outdoor scenes in various complex environments.

[0004] Therefore, the prior art has the technical problem of low alarm accuracy due to low smoke and flame recognition accuracy. SUMMARY

[0005] The present application provides a smoke and flame alarm method, device, electronic device, and storage medium to solve the technical problem of low smoke and flame alarm accuracy in the prior art.

[0006] According to a first aspect of the present application, a smoke and flame alarm method is provided, comprising:

[0007] obtaining at least one frame of an image to be detected from a monitoring video of a target area; wherein the monitoring video is obtained by a user terminal through a monitoring device when monitoring;

[0008] using a target smoke and flame prediction model to analyze and process the image to be detected to predict whether smoke and flame occur in the target area and obtain a prediction result; wherein the target smoke and flame prediction model is a pre-trained improved YOLOV7 network; the pre-trained improved YOLOV7 network includes a trained input module, a trained backbone network, and a trained Head network connected in sequence; the backbone network includes a plurality of grouped convolution modules; and the Head network includes an SE Block module representing an attention mechanism;

[0009] when the prediction result is a prediction result of the presence of smoke and / or flame, generating corresponding alarm information.

[0010] Optionally, the target smoke and flame prediction model is used to analyze and process the image to be detected to predict whether smoke and flame occur in the target region, and a prediction result is obtained, comprising:

[0011] The input module is trained, and the image to be detected is preprocessed to obtain a preprocessed image;

[0012] The backbone network is trained, and the preprocessed image is extracted to obtain target features;

[0013] According to the target features, the Head network is trained, and the prediction result of whether smoke and flame occur in the target region is obtained.

[0014] Optionally, the backbone network specifically comprises: a first grouped convolution module and a second grouped convolution module connected in sequence;

[0015] The first grouped convolution module is composed of a first type of CBS module, a second type of CBS module and a first ELAN module; the convolution kernel size of the second type of CBS module is different from that of the first type of CBS module;

[0016] The second grouped convolution module is composed of an MP module, a second ELAN module and an FPN module.

[0017] Optionally, the number of the second grouped convolution modules is multiple, the multiple second grouped convolution modules are connected in sequence, and the output of each grouped convolution module is used as the input of the Head network.

[0018] Optionally, the training process of the improved YOLOV7 network comprises:

[0019] Obtain a sample data set for training the improved YOLOV7 network; the sample data set comprises picture samples and labels of the picture samples;

[0020] According to a preset ratio, the sample data set is divided into a training set, a test set and a validation set;

[0021] According to the training set, the test set and the validation set, the improved YOLOV7 network composed of an input module, a backbone network and a Head network is trained, and after the training is completed, trained model parameters are obtained, and the improved YOLOV7 network corresponding to the trained model parameters is used as the target smoke and flame prediction model.

[0022] Optionally, the sample data set for training the improved YOLOV7 network comprises:

[0023] extracting a video clip from an existing monitoring video in which a target region and / or other regions except the target region generate smoke and flame;

[0024] extracting a smoke and flame picture from each frame of the extracted video clip;

[0025] eliminating smoke and flame pictures with the same smoke and flame contour from all smoke and flame pictures to obtain a smoke and flame picture set, and pre-processing smoke and flame pictures in the smoke and flame picture set to obtain picture samples;

[0026] labeling the picture samples for smoke region positions and flame region positions, and taking the labeling information as labels of the picture samples.

[0027] Optionally, the pre-processing of the smoke and flame pictures in the smoke and flame picture set to obtain picture samples comprises:

[0028] performing noise reduction processing on the smoke and flame pictures in the smoke and flame picture set by using a target filtering algorithm to obtain noise-reduced smoke and flame pictures;

[0029] performing size adjustment and pixel value normalization processing on the noise-reduced smoke and flame pictures to obtain the picture samples.

[0030] According to a second aspect of the present application, a smoke and flame alarm device is provided, comprising:

[0031] an acquisition module configured to acquire at least one image to be detected from a monitoring video of a target region, wherein the monitoring video is acquired by a user terminal through a monitoring device during monitoring;

[0032] an analysis and prediction module configured to analyze and process the image to be detected by using a target smoke and flame prediction model to predict whether smoke and flame occur in the target region and obtain a prediction result, wherein the target smoke and flame prediction model is a pre-trained improved YOLOV7 network; the pre-trained improved YOLOV7 network comprises a trained input module, a trained backbone network and a trained Head network connected in sequence; the backbone network comprises a plurality of grouped convolution modules; and the Head network comprises an SE Block module representing an attention mechanism;

[0033] a generation module configured to generate corresponding alarm information when the prediction result is a prediction result of the presence of smoke and / or flame.

[0034] According to a third aspect of the present application, an electronic device is provided, comprising at least one processor and a memory;

[0035] The memory stores computer-executable instructions;

[0036] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the smoke flame alarm method as described in the first aspect above.

[0037] According to a fourth aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium stores computer-executable instructions, the computer-executable instructions are executed by a processor to implement the smoke flame alarm method as described in the first aspect above.

[0038] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, the computer program is executed by a processor to implement the smoke flame alarm method of the first aspect.

[0039] The smoke flame alarm method provided by the present application comprises: obtaining at least one frame of image to be detected from a monitoring video of a target area; wherein the monitoring video is obtained by a user terminal through a monitoring device during monitoring; using a target smoke flame prediction model to analyze and process the image to be detected, to predict whether smoke and flame occur in the target area, and obtain a prediction result; wherein the target smoke flame prediction model is a pre-trained improved YOLOV7 network; the pre-trained improved YOLOV7 network comprises a trained input module, a trained backbone network and a trained Head network connected in sequence; the backbone network comprises a plurality of grouped convolution modules; the Head network comprises an SEBlock module representing an attention mechanism; when the prediction result is a prediction result of existing smoke and / or flame, corresponding alarm information is generated.

[0040] The target smoke flame prediction model in the present application is a pre-trained improved YOLOV7 network, in which the backbone network uses a plurality of grouped convolution modules to automatically extract diversified features, having the advantage of small model parameter calculation amount; and the Head network comprises an SEBlock module representing an attention mechanism, having the advantage of scale self-adaptation, i.e. the Head network can enable the improved YOLOV7 network to adaptively fuse features of different scales according to semantic importance, improve the detection accuracy of irregularly shaped targets such as smoke and flame, and reduce misjudgment, so that the present application can accurately identify smoke and flame in the image to be detected through the target smoke flame prediction model, and improve the accuracy of smoke flame alarm.

[0041] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor to limit the scope of the present application. Other features of the present application will become apparent from the following description. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] Figure 1 A schematic flowchart illustrating a smoke and flame alarm method provided in an embodiment of this application;

[0044] Figure 2 A schematic diagram of the network structure of a target smoke and flame prediction model provided in an embodiment of this application;

[0045] Figure 3 A schematic diagram of the network structure of another target smoke and flame prediction model provided in this application embodiment;

[0046] Figure 4 A flowchart illustrating an improved YOLOv7 network training method provided in this application embodiment;

[0047] Figure 5 A schematic diagram illustrating the training process of the improved YOLOv7 network provided in this application embodiment;

[0048] Figure 6 A schematic diagram of smoke and flame images in a surveillance video;

[0049] Figure 7 A schematic diagram of smoke and flame images in manually annotated surveillance video;

[0050] Figure 8 This is a schematic diagram of the processed smoke and flame image;

[0051] Figure 9 This is a schematic diagram of the output results after prediction using the improved YOLOv7 network in this embodiment;

[0052] Figure 10 A flowchart illustrating the improved YOLOv7 network application provided in this application embodiment;

[0053] Figure 11 A schematic diagram illustrating the output results provided in the embodiments of this application;

[0054] Figure 12 This is a schematic diagram of the structure of a smoke and flame alarm device provided in an embodiment of this application;

[0055] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0056] The specific embodiments of the application have been shown by the above drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the inventive concept in any way, but to explain the inventive concept to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0057] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same or similar components are denoted by the same or similar reference numerals throughout the drawings and the written description, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not meant to represent all embodiments consistent with the application.

[0058] Traditional smoke and flame detection mainly realizes the alarm function of flame and smoke through infrared camera devices and smoke sensors, etc. However, this kind of way has the limitation of misjudging sunset as flame and dust as smoke, etc. In addition, it has good smoke and flame prediction effect for indoor scenes, but cannot be applied to outdoor scenes in various complex environments on a large scale.

[0059] However, the prior art has the technical problem of low alarm accuracy caused by low smoke and flame recognition accuracy.

[0060] To solve the above technical problems, the overall inventive concept of the present application is how to provide a method for improving the accuracy of smoke and flame alarm in the security field.

[0061] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0062] Embodiment 1:

[0063] Figure 1 A flowchart of a smoke and flame alarm method provided by an embodiment of the present application is shown. As shown in the figure, the method of the present embodiment comprises: Figure 1

[0064] S10, obtaining at least one frame of image to be detected from the monitoring video of the target area; wherein the monitoring video is obtained by the user terminal when monitoring by the monitoring device.

[0065] It should be understood that when the smoke and flame alarm method is applied to a power production unit, the target area can be a designated area indoors / outdoors of the power production unit. When the smoke and flame alarm method is applied to other scenes, the target area can be adaptively adjusted according to scene information. ​

[0066] S20, using a target smoke flame prediction model to analyze and process the image to be detected to predict whether smoke and flame occur in the target area, and obtain a prediction result; wherein the target smoke flame prediction model is a pre-trained improved YOLOV7 network; the pre-trained improved YOLOV7 network includes a trained input module, a trained backbone network and a trained Head network connected in turn; the backbone network includes a plurality of grouped convolution modules; the Head network includes an SE Block module representing an attention mechanism.

[0067] It should be understood that the YOLOV7 algorithm is an efficient object detection algorithm based on deep neural networks for object recognition and positioning, which can be widely used in real-time detection systems due to its high accuracy and fast running speed.

[0068] In the embodiments of the present application, the improved YOLOV7 network or improved YOLOV7 object detection algorithm is improved. Based on the image to be detected under the monitoring video, the improved YOLOV7 object detection algorithm can realize real-time detection and recognition of the monitoring picture, and output the prediction result to generate the corresponding alarm information.

[0069] The above prediction result is the result of predicting the position of the smoke area and the position of the flame area, judging the category and the probability, for example: the prediction result is one smoke area position and one flame area position, the smoke area position is in the upper left part of the image to be detected, and the flame area position is in the lower left part of the image to be detected, for the smoke area position, the category is judged as smoke, and the probability is 0.7; for the flame area position, the category is judged as fire, and the probability is 0.9.

[0070] In the embodiments of the present application, the specific structure of the target smoke flame prediction model is shown in the following embodiment 2, which will not be described here. The embodiments of the present application extract effective features through the backbone network including a plurality of grouped convolution modules, and then improve the detection accuracy of the target smoke flame prediction model by introducing the SE Block module in the Head network.

[0071] S30, when the prediction result is a prediction result that there is smoke and / or flame, generating corresponding alarm information.

[0072] In the embodiments of the present application, the alarm information includes but is not limited to the prediction result, the blur level of the smoke, the blur level of the flame, the processing strategy corresponding to the blur level of the smoke, the processing strategy corresponding to the blur level of the flame, etc.; wherein various corresponding processing strategies include but are not limited to the number of personnel dispatched for fire extinguishing, dialing a fire telephone, etc.

[0073] The target smoke flame prediction model in the embodiments of the present application is a pre-trained improved YOLOV7 network. In the improved YOLOV7 network, the backbone network adopts a plurality of grouped convolution modules to automatically extract diversified features, and has the advantage of small model parameter calculation amount. The Head network contains an SE Block module representing an attention mechanism, and has the advantage of scale self-adaptation, that is, the Head network can enable the improved YOLOV7 network to adaptively fuse features of different scales according to semantic importance, improve the detection accuracy of irregular targets such as smoke and flame, and reduce misjudgment. Therefore, the target smoke flame prediction model can accurately identify smoke and flame in the image to be detected, and improve the accuracy of smoke and flame alarm.

[0074] In summary, the pre-trained improved YOLOV7 network is added to the video monitoring system, which can solve the technical problems of high false alarm rate and slow running speed in the traditional algorithm, realize fast and effective positioning and identification of smoke and flame, and ensure the safe operation and maintenance of power production, while reducing the operation and maintenance cost. That is, the improved YOLOV7 target detection algorithm is introduced in the embodiments, and related software is written, which can achieve the purpose of real-time detection, alarm and historical information storage in combination with the video monitoring system. On the basis of the above embodiments, the technical solutions of the present application will be described in more detail in combination with several specific embodiments.

[0075] Embodiment 2:

[0076] Figure 2 A schematic diagram of the network structure of a target smoke flame prediction model provided in the embodiments of the present application is shown in Figure 1 On the basis of the embodiments shown in Figure 2 The network structure of the target smoke flame prediction model includes a trained input module 1, a trained backbone network 2 and a trained Head network 3 connected in sequence.

[0077] Then, in step S20, the target smoke flame prediction model is used to analyze and process the image to be detected to predict whether smoke and flame occur in the target area, and obtain a prediction result, including:

[0078] In step S201, the trained input module is used to pre-process the image to be detected to obtain a pre-processed image.

[0079] In step S202, the trained backbone network is used to extract and process the pre-processed image to obtain target features.

[0080] Step S203, according to the target feature, whether smoke and flame occur in the target region is predicted by using the trained Head network, and a prediction result is obtained.

[0081] The target smoke and flame prediction model in the embodiment of the application has the advantage of small model parameter calculation amount of the trained backbone network; and the trained Head network contains the SE Block module representing the attention mechanism and has the advantage of scale self-adaptation, so that the target smoke and flame prediction model can accurately identify the smoke and flame in the image to be detected and improve the accuracy of smoke and flame alarm.

[0082] In a possible implementation manner, as shown in Figure 2 The backbone network 2 specifically includes a first grouped convolution module 21 and a second grouped convolution module 22 connected in sequence.

[0083] The first grouped convolution module is composed of a first type of CBS module, a second type of CBS module and a first ELAN module, and the convolution kernel size of the second type of CBS module is different from that of the first type of CBS module.

[0084] As shown in Figure 3 The first type of CBS module is also called CBS1, and the second type of CBS module is also called CBS2.

[0085] The second grouped convolution module is composed of an MP module, a second ELAN module and an FPN module.

[0086] In the embodiment of the application, the backbone network adopts a plurality of grouped convolution modules, and the second grouped convolution module adopts an FPN module, which can automatically extract diversified features and has the advantage of small model parameter calculation amount.

[0087] In a possible implementation manner, as shown in Figure 2 The number of the second grouped convolution modules 22 is multiple, the multiple second grouped convolution modules 22 are connected in sequence, and the output of each grouped convolution module 22 is used as the input of the Head network.

[0088] As shown in Figure 3 The input module 1, the backbone network 2 and the Head network 3 are connected in sequence, and in the backbone network 2, the first grouped convolution module 21 includes a first CBS1, a first CBS2, a second CBS1, a second CBS2 and an ELAN connected in sequence. The structure of the CBS1 and the structure of the CBS2 are both composed of three modules of Conv, BN and SiLU, and the ELAN is composed of a CBS1, a CBS3 and a Cat module, and the connection relationship is as shown in Figure 3As shown, and the structure of CBS3 is consistent with the structure of CBS1, which is not described here.

[0089] The second group convolution module 22 includes an MP module, an ELAN module and an FPN module connected in sequence, wherein the structure of the MPN is composed of a Maxpool, a CBS2, a CBS3 and a Cat module, and the connection relationship is as shown in the figure Figure 3 , which is not described here.

[0090] As shown Figure 3 , the Head network 3 introduces an SE Block module, wherein the ELAN-H module in the Head network 3 is composed of a CBS1, a CBS3 and a Cat module, and the connection relationship is as shown in the figure Figure 3 , which is not described here.

[0091] In the embodiment of the present application, the trained Head network contains an SE Block module representing an attention mechanism, which has the advantage of scale adaptation, i.e. the trained Head network can enable the improved YOLOV7 network to adaptively fuse features of different scales according to semantic importance, improve the detection accuracy of irregularly shaped targets such as smoke and flame, and reduce misjudgment. Therefore, the target smoke and flame prediction model of the present application can accurately identify smoke and flame in the image to be detected, and improve the accuracy of smoke and flame alarm.

[0092] Embodiment 3:

[0093] Figure 4 A flowchart of a training method of an improved YOLOV7 network provided in the embodiment of the present application. Based on the above Figure 1 mentioned embodiment, the training process of the improved YOLOV7 network is described in more detail in this embodiment.

[0094] As shown Figure 4 , the training process of the improved YOLOV7 network includes:

[0095] S41, obtaining a sample data set for training the improved YOLOV7 network; the sample data set includes picture samples and labels of the picture samples.

[0096] It should be understood that in the case of smoke and flame appearing in the existing monitoring video, the video clip can be collected from the existing monitoring video, and the collected video clip can be processed by frame extraction to obtain a frame-by-frame smoke and flame picture, which can be used as a sample data set for training the improved YOLOV7 network.

[0097] In the embodiment of the present application, the embodiment can add labels to the smoke and flame pictures in the video clip, and the labels are used to mark the smoke area and the flame area, and specifically mark the smoke area position and the flame area position. The embodiment can also perform noise reduction processing and size adjustment on the smoke and flame pictures in the video clip, so that the image size of the adjusted picture sample is consistent with the specified input size of the improved YOLOV7 network. The specific description of step S41 is as follows: steps S411-S414, which will not be repeated here.

[0098] S42, according to the preset proportion, the sample data set is divided into training set, test set and validation set.

[0099] In the embodiment of the present application, the preset proportion can be 6:2:2, or other proportions, which are not limited in the embodiment of the present application.

[0100] In addition, the embodiment of the present application can replace the division with an extraction operation, that is, according to the preset proportion, sample extraction is performed from the sample data set to obtain the training set, the test set and the validation set.

[0101] S43, according to the training set, the test set and the validation set, the improved YOLOV7 network composed of the input module, the backbone network and the Head network is trained, and after the training is completed, the trained model parameters are obtained, and the improved YOLOV7 network corresponding to the trained model parameters is used as the target smoke and flame prediction model.

[0102] In the embodiment of the present application, the purpose of step S43 is to input the training set, the test set and the validation set into the improved YOLOV7 network for training and repeating several rounds.

[0103] It should be understood that the data processed by step S42 is input into the improved YOLOV7 network, and epochs, batch_size and image_size are set for the improved YOLOV7 network to perform model training, and the best weight parameter (i.e. normalization parameter) of the improved YOLOV7 network is obtained after the training is completed. The improved YOLOV7 network corresponding to the best weight parameter is recorded as the target smoke and flame prediction model, which is used for real-time smoke and flame detection of video monitoring of power production units.

[0104] Wherein, epochs is defined as a single training iteration of all batches in forward and backward propagation, which means that one cycle is a single forward and backward pass of the entire input data. batch_size represents the total number of samples in a batch, and the number of samples input to the improved YOLOV7 network at a time. image_size represents the size of the picture sample.

[0105] The embodiment trains the improved YOLOV7 network through the steps S41-S43, can ensure the effectiveness of the improved YOLOV7 network training, and further obtains the target smoke flame prediction model.

[0106] In a possible implementation manner, the step S41 of acquiring the sample data set for training the improved YOLOV7 network comprises the following steps.

[0107] S411, video clips are intercepted from an existing monitoring video in which a target area and / or other areas except the target area have smoke and flame.

[0108] In the embodiment of the application, the existing monitoring video is a smoke and flame monitoring video of a certain area in a historical time period, the existing monitoring video can be collected from the network, and the collection manner is not limited in the embodiment of the application. In addition, the video clip is also referred to as a smoke and flame clip.

[0109] Optionally, the smoke and flame clip in the existing monitoring video can be collected by using an artificial observation manner.

[0110] In order to improve the richness of the sample, the existing monitoring video in the embodiment of the application can be multiple, that is, the embodiment can intercept video clips in multiple existing monitoring videos.

[0111] S412, smoke and flame pictures are acquired from the intercepted video clips.

[0112] In the embodiment of the application, the smoke and flame picture is also referred to as a smoke and flame image or a smoke and flame picture sample. In order to improve the accuracy of the improved YOLOV7 network, different smoke and flame pictures of smoke shapes and flame shapes are collected in the network training process, the picture can have only smoke, only flame, or both smoke and flame.

[0113] In addition, the number of smokes and the number of flames in the smoke and flame picture are not limited in the embodiment of the application, for example, smoke and flame picture A has two smoke areas and one flame area, and smoke and flame picture B has one smoke area and one flame area.

[0114] Optionally, the smoke and flame picture sample can be intercepted and screened by using a computer program to set a frame rate (Frame Per Second, FPS) interval.

[0115] S413, smoke and flame pictures with the same smoke and flame contour are removed from all smoke and flame pictures, a smoke and flame picture set is obtained, and the smoke and flame pictures in the smoke and flame picture set are preprocessed to obtain picture samples.

[0116] In the embodiment of the present application, the contours are the same, that is, the shapes are the same. In order to reduce data redundancy, the embodiment of the present application eliminates the smoke flame picture samples of the same shape.

[0117] The preprocessing described above includes but is not limited to noise reduction processing, image size adjustment, pixel value normalization processing, etc.

[0118] S414, marking the smoke area position and the flame area position of the picture sample, and taking the marking information as the label of the picture sample.

[0119] In the embodiment of the present application, the marking is also called labeling, and the marking information is also called labeling data. In step S414, the embodiment of the present application can use a labeling software (i.e., a labeling tool) to label the smoke area position and the flame area position in the picture sample in the form of a rectangular frame, and save the labeling data.

[0120] In the embodiment of the present application, the execution order between step S413 and step S414 can be interchanged, that is, the embodiment can first execute step S413 and then execute step S414, or first execute step S414 and then execute step S413, and the embodiment of the present application does not make specific limitation on this.

[0121] The picture sample and the label of the picture sample obtained by the above steps S411 to S414 are taken as a sample data set for training the improved YOLOV7 network, which can ensure the effectiveness of the training of the improved YOLOV7 network, and further obtain a target smoke flame prediction model.

[0122] In one possible implementation, in step S413, the smoke flame pictures in the smoke flame picture set are preprocessed to obtain picture samples, including:

[0123] S1, using a target filtering algorithm to perform noise reduction processing on the smoke flame pictures in the smoke flame picture set to obtain the smoke flame pictures after noise reduction.

[0124] It should be understood that the target filtering algorithm is also called an image filtering algorithm. Since the target filtering algorithm can be a Gaussian filtering algorithm, the above noise reduction processing is also called Gaussian filtering processing.

[0125] S2, performing size adjustment and pixel value normalization processing on the smoke flame pictures after noise reduction to obtain picture samples.

[0126] The embodiments of the present application can use a bilinear interpolation algorithm for size adjustment, or other algorithms for size adjustment, and the technical means used for size adjustment in the embodiments of the present application are not specifically limited. The size adjustment in the present embodiment can make the image size of the adjusted picture sample consistent with the specified input size of the improved YOLOV7 network.

[0127] The embodiments of the present application reduce noise in the picture sample through noise reduction processing, while minimizing the loss of original features and improving the signal-to-noise ratio. The size adjustment in the embodiments of the present application can make the size of the picture sample match the parameters of the input module of the improved YOLOV7 network, so as to improve the accuracy of the smoke and flame alarm.

[0128] Embodiment 4:

[0129] Based on embodiments 1-3, the present embodiment provides a training process of the improved YOLOV7 network and an application process of the improved YOLOV7 network after training.

[0130] Figure 5 The present embodiment provides a flowchart of the training process of the improved YOLOV7 network. As shown in Figure 5 , the training process includes the following steps:

[0131] Step S51, search for short fire videos in the existing monitoring videos on the network, and collect video clips from the short fire videos. Save each frame of smoke and flame image captured from the video clip and delete duplicate smoke and flame images.

[0132] For example, 50 video clips are divided into frames at a time interval of 1s using a program, obtaining 4000 smoke and flame images. In addition, 1000 smoke and flame images provided by non-monitoring videos are added to expand the recognition range of the improved YOLOV7 network, i.e. 5000 smoke and flame images are used as sample data sets for training the improved YOLOV7 network. As shown in Figure 6 , the smoke and flame image includes a smoke area and a flame area.

[0133] Step S52, manually mark the smoke area and the flame area in the smoke and flame image respectively and save the label data.

[0134] The present embodiment can use LabelImg software to mark the smoke area and the flame area in the smoke and flame image respectively and save the corresponding label data.

[0135] For example, after marking the smoke and flame images provided by Figure 6 , the smoke and flame image obtained is as shown in Figure 7The smoke flame image in the manually labeled monitoring video is shown, wherein the smoke area is in the dashed rectangular frame, and the flame area is in the solid rectangular frame.

[0136] Step S53, the smoke flame image is denoised, the size is adjusted, and data is normalized by using a Gaussian filtering algorithm, to obtain a picture sample.

[0137] It should be understood that the picture sample is also referred to as a processed smoke flame image.

[0138] For example, the smoke flame image shown in FIG. 1 is denoised, the original image size is adjusted to the input size specified for the improved YOLOV7 network (such as 640*640), and the pixel data is divided by 255 to be normalized to the range of [0, 1], to obtain the processed smoke flame image shown in FIG. 2 (wherein the two circles are filtered out). Figure 7 Figure 8 The difference between FIG. 2 and FIG. 1 is that the two circles are filtered out, to ensure that the improved YOLOV7 network converges quickly. Figure 8 Figure 6

[0139] Step S54, the picture sample and the label data are used as sample data sets for training the improved YOLOV7 network.

[0140] Step S55, the sample data sets are randomly sampled according to a ratio of 6:2:2, to be divided into a training set, a validation set, and a test set.

[0141] For example, 5000 smoke flame images are randomly sampled according to a ratio of 6:2:2, to be divided into a training set (3000 images), a validation set (1000 images), and a test set (1000 images), and the manually labeled labels are also divided.

[0142] Step S56, the improved YOLOV7 network composed of an input module, a backbone network, and a Head network is trained according to the training set and the validation set, to obtain a trained smoke flame prediction model.

[0143] Step S57, the performance of the smoke flame prediction model is tested according to the test set, if the performance meets a preset requirement, step S58 is performed; if the performance does not meet the preset requirement, steps S55-S57 are continuously performed, until the performance meets the preset requirement.

[0144] ​​​The training set, the validation set and the test set and the corresponding label data are input into the improved YOLOV7 network for training, where epochs is set to 200, batch_size is set to 32, image_size is set to 640*640, device is set to GPU, and the average precision and recall of the last training are 98.7% and 98.9% respectively, and the precision of the embodiment can reach 99.0% on the test set.

[0145] The schematic diagram of the output result after prediction by the improved YOLOV7 network in the embodiment is shown in Figure 9 , which includes the type of discrimination, probability and region.

[0146] Step S58, save the best improved YOLOV7 network weight parameter of training, and output the target smoke and flame prediction model.

[0147] After the execution of steps S51-S58, the training process is completed. When the target smoke and flame prediction model is applied, as shown in Figure 10 , the following steps are included.

[0148] Step S101, input the power production unit monitoring video data into the smoke and flame detection software to obtain the power production unit monitoring video data.

[0149] Step S102, collect each frame of image in the monitoring video.

[0150] Step S103, filter the image and adjust the image size.

[0151] In step S103, the image size is adjusted to 640*640.

[0152] Step S104, data normalization according to the input requirements of the improved YOLOV7 network.

[0153] Step S105, use the trained improved YOLOV7 network to detect whether there is smoke and flame in each frame of image in the monitoring video.

[0154] Step S106, if there is smoke and flame, output the name of smoke or flame, prediction probability and draw the prediction position. If there is no smoke and flame, output empty.

[0155] If there is smoke and flame, the output result is shown in Figure 11 .

[0156] In order to effectively identify all possible smoke and fire in the monitoring video in the embodiment, it is suggested to update the smoke and fire samples under the monitoring video for a long time and retrain the correction model to improve the application effect of the target smoke and fire prediction model.

[0157] Embodiment 5:

[0158] Figure 12 A structure diagram of a smoke and fire alarm device provided by the embodiment of the application is provided. The device of the embodiment can be in the form of software and / or hardware. As shown in the figure, the smoke and fire alarm device provided by the embodiment includes an acquisition module 121, an analysis and prediction module 122, and a generation module 123. Figure 12

[0159] Among them:

[0160] The acquisition module 121 is configured to acquire at least one frame of image to be detected from a monitoring video of a target area, wherein the monitoring video is acquired by a user terminal through a monitoring device when monitoring.

[0161] The analysis and prediction module 122 is configured to use a target smoke and fire prediction model to analyze and process the image to be detected, to predict whether smoke and fire occur in the target area, and to obtain a prediction result; wherein the target smoke and fire prediction model is a pre-trained improved YOLOV7 network; the pre-trained improved YOLOV7 network includes a trained input module, a trained backbone network and a trained Head network connected in sequence; the backbone network includes a plurality of grouped convolution modules; the Head network includes an SE Block module representing an attention mechanism.

[0162] The generation module 123 is configured to generate corresponding alarm information when the prediction result is a prediction result of the presence of smoke and / or fire.

[0163] In one possible implementation, the network structure of the target smoke and fire prediction model includes a trained input module, a trained backbone network and a trained Head network connected in sequence.

[0164] The analysis and prediction module 122 is further configured to:

[0165] using the trained input module to pre-process the image to be detected to obtain a pre-processed image.

[0166] using the trained backbone network to extract and process the pre-processed image to obtain a target feature.

[0167] According to the target feature, the trained Head network is used to predict whether smoke and fire occur in the target area and obtain a prediction result. ​

[0168] In a possible implementation, the backbone network specifically comprises: a first group convolution module and a second group convolution module connected in sequence.

[0169] The first group convolution module is composed of a first type of CBS module, a second type of CBS module, and a first ELAN module; the convolution kernel size of the second type of CBS module is different from the convolution kernel size of the first type of CBS module.

[0170] The second group convolution module is composed of an MP module, a second ELAN module, and an FPN module.

[0171] In a possible implementation, the number of the second group convolution modules is multiple, the multiple second group convolution modules are connected in sequence, and the output of each group convolution module is used as the input of the Head network.

[0172] In a possible implementation, the smoke flame alarm device is further configured to:

[0173] obtain a sample data set for training the improved YOLOV7 network; the sample data set comprises picture samples and labels of the picture samples.

[0174] divide the sample data set into a training set, a test set, and a verification set according to a preset ratio.

[0175] train the improved YOLOV7 network composed of the input module, the backbone network, and the Head network according to the training set, the test set, and the verification set, obtain trained model parameters after the training ends, and use the improved YOLOV7 network corresponding to the trained model parameters as the target smoke flame prediction model.

[0176] In a possible implementation, the smoke flame alarm device is further configured to:

[0177] cut video clips from existing monitoring videos in which smoke flames occur in the target area and / or other areas except the target area.

[0178] obtain each frame of smoke flame pictures from the cut video clips.

[0179] remove smoke flame pictures with the same smoke flame contour from all the smoke flame pictures to obtain a set of smoke flame pictures, and preprocess the smoke flame pictures in the set of smoke flame pictures to obtain picture samples.

[0180] label the positions of smoke areas and flame areas in the picture samples, and use the label information as labels of the picture samples.

[0181] In a possible implementation, the smoke flame alarm device is further configured to:

[0182] The target filtering algorithm is used to perform noise reduction processing on the smoke flame pictures in the smoke flame picture set, to obtain the noise-reduced smoke flame pictures.

[0183] The noise-reduced smoke flame pictures are subjected to size adjustment and pixel value normalization processing, to obtain picture samples.

[0184] The smoke flame alarm device provided in the embodiment can be used to execute the smoke flame alarm method provided in any of the method embodiments, and has similar implementation principles and technical effects, which will not be described herein.

[0185] It should be noted that the user information and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions, and provide corresponding operation entrances for user selection authorization or refusal.

[0186] That is, in the technical solutions of the present application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0187] According to the embodiments of the present application, the present application further provides an electronic device and a readable storage medium.

[0188] Figure 13 A structural schematic diagram of an electronic device provided in the embodiment of the present application. The electronic device includes a receiver 130, a transmitter 131, at least one processor 132 and a memory 133. The electronic device composed of the above components can be used to implement the above several specific embodiments of the present application, and will not be described herein.

[0189] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the processor executes the computer execution instructions, each step in the method in the above embodiment is implemented.

[0190] The embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by the processor, each step in the method in the above embodiment is implemented.

[0191] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip systems (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0192] Program code for carrying out methods of the present application can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, causes the functions / acts specified in the flowcharts and / or block diagrams to be implemented. The program code can execute entirely on a machine, partly on a machine, partly on a remote machine or entirely on a remote machine or electronic device.

[0193] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can be a machine-readable signal medium or a machine-readable storage medium. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0194] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0195] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0196] It should be understood that various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the spirit of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in different orders, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

[0197] The specific embodiments described above have been disclosed by way of example and that other embodiments, individually or in combination, are also possible. It is intended that the application protect all changes and modifications that come within the scope of the application as defined by the following claims and their equivalents.

Claims

1. A method of smoke flame detection, characterized in that The method comprises the following steps: obtaining at least one frame of image to be detected from a monitoring video of a target area; wherein the monitoring video is obtained by a user terminal through a monitoring device during monitoring; using a target smoke and flame prediction model to analyze and process the image to be detected, so as to predict whether smoke and flame occur in the target area and obtain a prediction result; wherein the target smoke and flame prediction model is a pre-trained improved YOLOV7 network; the pre-trained improved YOLOV7 network comprises a trained input module, a trained backbone network and a trained Head network connected in sequence; the backbone network comprises a plurality of grouped convolution modules; and the Head network comprises an SE Block module representing an attention mechanism; when the prediction result is a prediction result of existing smoke and / or flame, corresponding alarm information is generated; the step of using a target smoke and flame prediction model to analyze and process the image to be detected, so as to predict whether smoke and flame occur in the target area and obtain a prediction result, comprises the following steps: using the trained input module to pre-process the image to be detected to obtain a pre-processed image; using the trained backbone network to extract and process the pre-processed image to obtain target features; using the trained Head network to predict whether smoke and flame occur in the target area according to the target features and obtain a prediction result; the backbone network specifically comprises a first grouped convolution module and a second grouped convolution module connected in sequence; wherein the first grouped convolution module is composed of a first type of CBS module, a second type of CBS module and a first ELAN module; the convolution kernel size of the second type of CBS module is different from that of the first type of CBS module; the second grouped convolution module is composed of an MP module, a second ELAN module and an FPN module.

2. The method of claim 1, wherein, The number of the second grouped convolution modules is multiple, the second grouped convolution modules are connected in sequence, and the output of each grouped convolution module is used as the input of the Head network.

3. The method of claim 1, wherein, The training process of the improved YOLOV7 network comprises the following steps: obtaining a sample data set for training the improved YOLOV7 network; the sample data set comprises picture samples and labels of the picture samples; dividing the sample data set into a training set, a test set and a validation set according to a preset ratio; training the improved YOLOV7 network composed of an input module, a backbone network and a Head network according to the training set, the test set and the validation set; after the training is completed, trained model parameters are obtained, and the improved YOLOV7 network corresponding to the trained model parameters is used as the target smoke and flame prediction model.

4. The method of claim 3, wherein, the step of obtaining a sample data set for training the improved YOLOV7 network comprises the following steps: cutting video clips from existing monitoring videos in which smoke and flame occur in a target area and / or other areas except the target area; acquiring each frame of smoke flame picture from the intercepted video clip; eliminating smoke flame pictures with the same smoke flame contour from all smoke flame pictures to obtain a set of smoke flame pictures, and pre-processing smoke flame pictures in the set of smoke flame pictures to obtain picture samples; labeling smoke area positions and flame area positions of the picture samples, and taking the labeling information as labels of the picture samples.

5. The method of claim 4, wherein, The pre-processing of the smoke flame pictures in the set of smoke flame pictures to obtain picture samples comprises: performing noise reduction processing on the smoke flame pictures in the set of smoke flame pictures by using a target filtering algorithm to obtain noise-reduced smoke flame pictures; performing size adjustment and pixel value normalization processing on the noise-reduced smoke flame pictures to obtain the picture samples.

6. A smoke flame alarm device, characterized in that comprise: an acquisition module, configured to acquire at least one image to be detected from a monitoring video of a target area, wherein the monitoring video is acquired by a user terminal when monitoring by a monitoring device; an analysis and prediction module, configured to analyze and process the image to be detected by using a target smoke flame prediction model to predict whether smoke and flame occur in the target area and obtain a prediction result, wherein the target smoke flame prediction model is a pre-trained improved YOLOV7 network; the pre-trained improved YOLOV7 network comprises a trained input module, a trained backbone network and a trained Head network connected in sequence; the backbone network comprises a plurality of grouped convolution modules; and the Head network comprises an SE Block module representing an attention mechanism; a generation module, configured to generate corresponding alarm information when the prediction result is a prediction result of existing smoke and / or flame; The analysis and prediction module is specifically configured to: perform preprocessing on the image to be detected by using the trained input module to obtain a preprocessed image; perform extraction processing on the preprocessed image by using the trained backbone network to obtain target features; predict whether smoke and flame occur in the target area according to the target features by using the trained Head network to obtain a prediction result. The backbone network specifically comprises a first grouped convolution module and a second grouped convolution module connected in sequence; The first grouped convolution module is composed of a first type of CBS module, a second type of CBS module and a first ELAN module; and the convolution kernel size of the second type of CBS module is different from that of the first type of CBS module. The second grouped convolution module is composed of an MP module, a second ELAN module and an FPN module.

7. An electronic device, comprising: comprise: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the smoke flame alarm method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are used for realizing the smoke flame alarm method in any one of claims 1 to 5 when executed by the processor.

Images