Fire behavior identification method, device and equipment and storage medium

Abstract

The invention discloses a fire behavior identification method and device, equipment and a storage medium. The method comprises the following steps: acquiring a to-be-identified video frame, and determining a motion foreground area and a contour of a motion foreground in the to-be-identified video frame; processing the contour of the moving foreground by using a preset target detection model, and outputting a fire suspicion area; and determining a fire behavior identification result according to the overlapping area between the fire behavior doubt area and the motion foreground area. According to the technical scheme provided by the embodiment of the invention, the real area with the smoke and fire can be locked in a complex environment, the authenticity of the fire can be accurately identified, the false alarm rate of the smoke and fire is reduced, and rapid and accurate fire monitoring is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of fire alarm technology, and in particular to a fire identification method, device, equipment and storage medium. Background Art

[0002] With the continuous development of intelligent monitoring technology, smoke and fire detection algorithms have gained widespread application in the security field. Traditional fire detection methods primarily rely on temperature sensors and smoke alarms, which have limitations in response speed and accuracy. Especially in vast outdoor areas or complex indoor environments, traditional detection methods often fail to detect fires in a timely manner, delaying firefighting and causing serious damage. To address these issues, smoke and fire recognition algorithms based on image processing and machine learning have emerged. They can effectively monitor and warn of fires in their early stages, significantly improving fire safety.

[0003] The basic principle of fire and smoke detection algorithms is to automatically identify the characteristics of smoke and flames by analyzing image information in surveillance videos and generate alarm signals. However, current common fire and smoke detection algorithms have poor generalization performance and are easily misled by similar characteristics of fire and smoke, resulting in false detections. Summary of the Invention

[0004] The present invention provides a fire identification method, device, equipment and storage medium to solve the problem of prone to false detection in fire identification.

[0005] In a first aspect, the present invention provides a fire identification method, comprising:

[0006] Acquire a video frame to be identified, and determine a motion foreground area and a contour of the motion foreground in the video frame to be identified;

[0007] Using a preset target detection model to process the outline of the moving foreground, and output the suspected fire area;

[0008] The fire identification result is determined based on the overlapping area between the suspected fire area and the motion foreground area.

[0009] In a second aspect, the present invention provides a fire identification device, comprising:

[0010] A foreground recognition module is used to obtain a video frame to be recognized and determine a moving foreground area and a contour of the moving foreground in the video frame to be recognized;

[0011] a suspected fire area determination module, configured to process the outline of the motion foreground using a preset target detection model and output the suspected fire area;

[0012] The fire identification result determination module is used to determine the fire identification result according to the overlapping area between the suspected fire area and the motion foreground area.

[0013] In a third aspect, the present invention provides an electronic device, comprising:

[0014] at least one processor;

[0015] and a memory communicatively coupled to the at least one processor;

[0016] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the fire identification method of the first aspect mentioned above.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions, which are used to enable a processor to implement the fire identification method of the first aspect when executed.

[0018] The fire identification solution provided by the present invention obtains a video frame to be identified, and determines the motion foreground area and the outline of the motion foreground in the video frame to be identified, uses a preset target detection model to process the outline of the motion foreground, outputs a suspected fire area, and determines the fire identification result based on the overlapping area between the suspected fire area and the motion foreground area. By adopting the above technical solution, based on the motion (foreground) area detection of the video frame, non-moving objects are automatically filtered out, and based on the overlapping area between the motion area and the area where suspected fireworks and fire are output by the model, the consistency of the two areas can be evaluated, the area where fireworks and fire actually exist can be locked, and false alarms can be reduced, so that the authenticity of the fire can be accurately identified in a complex environment, the false alarm rate of fireworks and fire can be reduced, and fast and accurate fire monitoring can be achieved, providing important technical support for the security field.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a flow chart of a fire identification method provided according to the first embodiment of the present invention;

[0022] Figure 2This is a flow chart of a fire identification method provided according to the second embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of a model to be trained provided in Example 2 of the present invention;

[0024] Figure 4 This is a schematic diagram of a consistency check calculation provided according to the second embodiment of the present invention;

[0025] Figure 5 This is a structural diagram of a fire identification device provided according to a third embodiment of the present invention;

[0026] Figure 6 It is a structural diagram of an electronic device provided according to the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," and the like in the specification and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein. In the description of the present invention, unless otherwise specified, "plurality" refers to two or more. "And / or" describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0029] Example 1

[0030] Figure 1A flowchart of a fire identification method is provided for the first embodiment of the present invention. This embodiment is applicable to situations where a fire is identified. The method can be executed by a fire identification device. The fire identification device can be implemented in the form of hardware and / or software. The fire identification device can be configured in an electronic device. The electronic device can be composed of two or more physical entities or one physical entity.

[0031] like Figure 1 As shown, the fire identification method provided in the first embodiment of the present invention specifically includes the following steps:

[0032] S101: Acquire a video frame to be identified, and determine a motion foreground area and a contour of the motion foreground in the video frame to be identified.

[0033] In this embodiment, electronic devices such as high-definition cameras can be used to automatically collect and record data of the monitored environment to obtain video frames to be identified. Then, the motion foreground area and the contour of the motion foreground are extracted from the video frames to be identified.

[0034] For example, a background subtractor can be created using a mixed Gaussian model algorithm to separate the dynamic foreground from the static background in the video frame to be identified. After acquiring each frame of the video image to be identified, a background subtractor is used to generate a motion foreground mask, and the pixel values belonging to the motion foreground area are marked as white, while the background area is marked as black. Then, the outermost contour of the motion foreground is determined in the foreground mask, and the contour information is stored. In some examples, for the horizontal and vertical lines of the outermost contour, only vertex information is retained, and for the diagonal lines of the outermost contour, only corner point information is retained. By using vertex information and corner point information instead of lines, the number of points stored is reduced, thereby reducing storage requirements.

[0035] S102: Use a preset target detection model to process the outline of the moving foreground and output a suspected fire area.

[0036] In this embodiment, the outline of the fire area has certain characteristic patterns. A preset target detection model, such as a fire detection model, can be used to process the contour image of the moving foreground. The model can identify the fire area and output a suspected fire area, such as an area suspected of containing flames and / or smoke.

[0037] S103: Determine a fire identification result based on an overlapping area between the suspected fire area and the motion foreground area.

[0038] In this embodiment, the motion foreground area can be used to verify the suspected fire area. Since fireworks have the characteristic of motion, the fire identification result can be determined based on the overlapping area between the suspected fire area and the motion foreground area. For example, if the overlapping area between the suspected fire area and the motion foreground area is greater than the preset area value, the fire identification result can be determined to be the presence of a fire, and a fireworks alarm can be triggered to instruct the monitoring personnel to conduct a secondary review. If it is determined that a fire does exist, firefighting is immediately carried out. In this way, a surveillance camera or drone equipped with the fire identification method described in the embodiment of the present invention can be unmanned 24 hours a day, and automatically alarm in the event of a disaster, greatly ensuring the safety of the environment.

[0039] The technical solution of the embodiment of the present invention is based on the motion (foreground) area detection of the video frame, automatically filters out non-moving objects, and evaluates the consistency of the motion area and the area where fireworks are suspected to exist output by the model based on the overlapping area of the two areas, locks the area where fireworks actually exist, and reduces false alarms. Therefore, it can accurately identify the authenticity of the fire in a complex environment, reduce the false alarm rate of fireworks, realize fast and accurate fire monitoring, and provide important technical support for the security field.

[0040] Example 2

[0041] Figure 2 This is a flow chart of a fire identification method provided in the second embodiment of the present invention. The technical solution of the embodiment of the present invention is further optimized on the basis of the above optional technical solutions, and provides a specific method for identifying fire conditions.

[0042] In some examples, the fire identification result is determined based on the overlapping area between the suspected fire area and the motion foreground area, including: for each suspected fire area, determining the overlapping area of the current suspected fire area and the motion foreground area that intersects with the current suspected fire area, and determining the ratio of the overlapping area to the area of the current suspected fire area as the overlap ratio; if the overlap ratio is greater than or equal to a preset threshold, the fire identification result is determined to be a fire. The advantage of this setting is that by determining the overlap ratio and determining the relationship between the overlap ratio and the preset threshold, consistency verification is achieved between the video motion area and the area where fireworks and fire exist, further ensuring the actual existence of a disaster and reducing the false detection rate of fireworks and fire.

[0043] In some examples, before processing the foreground motion contour using a preset target detection model and outputting the suspected fire area, the method further includes removing the foreground motion contours whose contour area is smaller than a preset value to obtain an updated foreground motion contour. This configuration has the advantage of reducing interference caused by abnormal noise and changes in a small number of pixels in consecutive video frames by removing the smaller foreground motion contours, while also reducing the computational complexity of subsequent verification.

[0044] like Figure 2 As shown, a fire identification method provided by the second embodiment of the present invention specifically includes the following steps:

[0045] S201: Acquire a video frame to be identified, and determine a motion foreground area and a motion foreground outline in the video frame to be identified.

[0046] In some examples, to remove possible noise points and enhance the quality of the moving foreground (mask), morphological operations may be used to perform noise reduction on the moving foreground in the video frame to be identified before determining the moving foreground region and the outline of the moving foreground in the video frame to be identified. For example, an opening operation may be performed using a cross-shaped structuring element. The opening operation includes erosion and dilation, which can effectively eliminate small white noise points in the moving foreground.

[0047] S202: Eliminate the contours of the moving foreground whose contour areas are smaller than a preset value to obtain an updated contour of the moving foreground.

[0048] Specifically, to reduce the interference caused by abnormal noise and changes in a few pixels in consecutive video frames, and to reduce the computational complexity of the subsequent fireworks area verification (step S204), smaller contour areas can be filtered out. For example, contour areas with a pixel area of less than 300 can be removed from the moving foreground.

[0049] S203: Use a preset target detection model to process the outline of the moving foreground and output a suspected fire area.

[0050] In some examples, the method for determining the preset target detection model includes: obtaining target training samples with labels, wherein the categories of the labels include fire-related target labels and fire-confused target labels, and the fire-confused targets are targets similar to fire-related targets; and using the target training samples to train the model to be trained to obtain the preset target detection model.

[0051] Specifically, we first obtain a fireworks dataset, which includes multiple fire scene samples. These fire scene samples are then labeled with high quality to obtain the target training samples. In addition to labeling the actual smoke and flames, this labeling process also uniformly labels objects that could be easily confused with actual smoke and flames. For example, the label corresponding to the sun and sunset glow with flame characteristics is the "flame confusing target" label, and the label corresponding to the fog and clouds with flame smoke characteristics is the "smoke confusing target" label. Among them, the "flame confusing target" label and the "smoke confusing target" label both belong to the "fire confusing target" category. Correspondingly, the sun and sunset glow with flame characteristics, and the fog and clouds with flame smoke characteristics are fire confusing targets. The label corresponding to the flame in the fire scene sample is the "flame target" label, and the label corresponding to the flame smoke is the "flame smoke target" label. Both the flame target and the flame smoke target are fire-related targets. The "flame target" label and the "flame smoke target" label both belong to the "fire confusing target" label. Among them, fire-related targets represent targets associated with fire. The advantage of this setting is that by labeling fire-related targets and fire confusing targets, it is additionally distinguished from categories that are easily confused with fireworks, so that the preset target detection model can automatically perceive easily confused objects.

[0052] In some examples, after obtaining the target training samples with labels, the target training samples can also be enhanced. The enhancement methods include: at least one of: random cropping, Mosaic enhancement, and copy-paste enhancement. Among them, Mosaic enhancement is a data enhancement technology, which is mainly used in target detection tasks. It stitches multiple different images into a mosaic image, generates new labels during stitching, and applies data enhancement to further transform the image and label, thereby improving the performance of the model. Copy-paste enhancement is a data enhancement technology that copies and pastes the target object in the source image into the target image to increase the diversity of the data set, thereby improving the generalization ability of the model.

[0053] In some examples, the model to be trained includes a backbone network Backbone, a neck network Neck, and a detection head network Head connected in series, wherein:

[0054] The neck network Neck is used to perform feature fusion and enhancement on the outputs of multiple preprocessing residual networks PRes of the backbone network Backbone, wherein the preprocessing residual network PRes includes a bottleneck network Bottleneck and a main bottleneck network PBottleneck connected in series, and the main bottleneck network PBottleneck includes a first convolution batch normalization activation function module, a partial convolution PConv and a second convolution batch normalization activation function module connected in series.

[0055] Specifically, due to the huge number of neural network parameters, there will be many redundant features. Figure 3 is a structural diagram of a model to be trained. Figure 3 As shown in the figure, after the image data is input into the model to be trained, it will be processed by the CBS (Conv-BatchNorm-SiLU, convolution batch normalization activation function) module, the maximum pooling layer MaxPool, and the 4-layer preprocessing residual network PRes in the Backbone. Bottleneck includes 3 CBS, which contains the convolution layer Conv, the batch normalization layer BN, and the activation function SiLU. The PConv of the model to be trained includes the Split function, the convolution layer Conv, the splicing layer Concat, the batch normalization layer BN, and the activation function SiLU. The PRes of the Backbone of the model to be trained includes the bottleneck network Bottleneck and N main bottleneck networks PBottleneck. When the data is input into PBottleneck for processing, it will be processed by CBS, PConv, and CBS in sequence. In the residual network, compared with the 3×3 ordinary convolution layer in the ordinary residual block, the use of partial convolution PConv can reduce the number of channels. For example, the number of channels is reduced to 1 / 2 of the original, and the remaining half of the channels are mapped with identity. Without additional calculation, the feature extraction capability of the network can be efficiently utilized. With this improvement, the overall network can be reduced (for example, by a quarter) in size without a significant decrease in model accuracy. The preset target detection model obtained after training can automatically perceive easily confused categories, learn the detailed features and subtle differences between fireworks and non-fireworks, reduce false detections, and quickly adapt to new application scenarios. Among them, Figure 3 N in represents a first preset number. S3, S4, and S5 represent extracted feature maps, which have different resolutions and semantic information, and F5 represents the highest-level feature.

[0056] In some examples, the backbone network Backbone includes: a first preprocessing residual network Pres, a second preprocessing residual network Pres, and a third preprocessing residual network Pres connected in series in sequence; wherein: the first Pres output of the first preprocessing residual network Pres is output to the neck network Neck and the second preprocessing residual network Pres respectively; the second Pres output of the second preprocessing residual network Pres is output to the neck network Neck and the third preprocessing residual network Pres respectively; the third Pres output of the third preprocessing residual network Pres is output to the neck network Neck.

[0057] In some examples, the output of the backbone network Backbone includes a first Pres output, a second Pres output, and a third Pres output; in the neck network Neck: the input of the third convolution batch normalization activation function module is the first Pres output, and the output is output to the first cross-stage partial connection module; the output of the first cross-stage partial connection module is output to the concatenation layer Concat and the first downsampling layer Down respectively; the output of the first downsampling layer Down is output to the first fusion layer Fusion; the output of the first fusion layer Fusion is output to the concatenation layer Concat and the second downsampling layer Down respectively; the output of the second downsampling layer Down is output to the second fusion layer Fusion; the output of the second fusion layer Fusion is output to the concatenation layer Concat t; the input of the fourth convolution batch normalization activation function module is the second Pres output, and the output is output to the second cross-stage partial connection module; the output of the second cross-stage partial connection module is output to the fifth convolution batch normalization activation function module; the output of the fifth convolution batch normalization activation function module is output to the first fusion layer Fusion and the first upsampling layer UP respectively; the output of the first upsampling layer UP is output to the first cross-stage partial connection module; the input of the attention-based intra-scale feature interaction layer AIFI is the third Pres output, and the output is output to the sixth convolution batch normalization activation function module; the output of the sixth convolution batch normalization activation function module is output to the second fusion layer Fusion and the second upsampling layer UP respectively; the output of the second upsampling layer UP is output to the second cross-stage partial connection module.

[0058] Specifically, the feature extraction and fusion in the Neck network of the model to be trained can be achieved using the Cross Stage Partial (CSP) module. Figure 3 As shown in the figure, when data is input to CSP for processing, it will go through Concat, CBS, Split function, M Bot layers, Concat and CBS. The Bot layer includes 2 CBS and 1 Concat. Figure 3 The M in the formula represents the second preset number. The larger M and N are, the more complex feature representations can be extracted. The smaller M and N are, the less computing resources are consumed and the faster the training speed.

[0059] like Figure 3 As shown:

[0060] The output of the second PRes layer (from top to bottom) in the Backbone of the model to be trained is not only output to the third PRes layer, but also processed by CBS1 (the third convolutional batch normalization activation function module) in the Neck network. The output of the third PRes layer in the Backbone is not only output to the fourth PRes layer, but also processed by CBS2 (the fourth convolutional batch normalization activation function module) in the Neck network. The output of the fourth PRes layer in the Backbone is processed by the AIFI (Attention-based Intra-scale Feature Interaction) layer in the Neck network.

[0061] The output of CBS1 is output to CSP1 (the first cross-stage partial connection module). The output of CSP1 is output to the concatenation layer Concat and the first downsampling layer Down1. The output of the first downsampling layer Down1 is output to the first fusion layer Fusion1. The output of the first fusion layer Fusion1 is output to the concatenation layer Concat and the second downsampling layer Down2. The output of the second downsampling layer Down2 is output to the second fusion layer Fusion2. The output of the second fusion layer Fusion2 is output to the concatenation layer Concat.

[0062] The output of CBS2 is output to CSP2 (i.e., the second cross-stage partial connection module). The output of CSP2 is output to CBS3 (i.e., the fifth convolutional batch normalization activation function module). The output of CBS3 is output to the first fusion layer Fusion1 and the first upsampling layer UP1 respectively. The output of the first upsampling layer UP1 is output to CSP1.

[0063] The output of AIFI is output to CBS4 (i.e., the sixth convolutional batch normalization activation function module). The output of CBS4 is output to the second fusion layer Fusion2 and the second upsampling layer UP2 respectively; the output of the second upsampling layer UP2 is output to CSP2.

[0064] The result of the Concat output in the above Neck network will be sent to the Decoder in the Head layer of the model to be trained for processing to obtain the output of the model to be trained.

[0065] By utilizing cross-stage partial connection modules, we can add more skip links, fully integrating the features of the network layers, accelerating network convergence, and improving network detection accuracy. The advantage of this setting is that it improves inference speed while maintaining detection accuracy.

[0066] In some examples, the target training samples are generated by obtaining a segmentation mask and a scene image of a fireworks image; generating a fire scene sample by pasting the image corresponding to the segmentation mask to the scene image, and generating a labeled target training sample using the fire scene sample.

[0067] Specifically, we first obtain open-source fireworks images and use a camera to capture an image of the environment (i.e., a scene image). We then paste the image corresponding to the segmentation mask of the fireworks image into the scene image to generate a new fireworks image, creating a new fire scene image. This image is the fire scene sample. By labeling the fire scene sample, we can generate labeled target training samples.

[0068] S204. For each of the suspected fire areas, determine the overlapping area between the current suspected fire area and the motion foreground area that intersects with the current suspected fire area, and determine the ratio of the overlapping area to the area of the current suspected fire area as the overlapping ratio.

[0069] Specifically, the suspected fire areas output by the model can be traversed, and for each suspected fire area, the overlapping area between the current suspected fire area and the motion foreground area that intersects with the current suspected fire area can be determined, and the ratio of the overlapping area to the area of the current suspected fire area can be determined as the overlapping ratio.

[0070] For example, Figure 4 This is a schematic diagram of consistency check calculation. If the area of the motion foreground region that intersects with the current suspected fire area is A, and the area of the current suspected fire area is B. Figure 4 As shown, overlap ratio = (overlap area of A and B) / B.

[0071] S205: If the overlap ratio is greater than or equal to a preset threshold, the fire identification result is determined to be a fire occurrence.

[0072] Exemplarily, if the overlap ratio is greater than or equal to 0.3, the fire identification result is determined to be a fire occurrence.

[0073] In some examples, if the fire identification result of a preset number of consecutive video frames is that a fire has occurred, or if the fire identification result of a preset time period is that a fire has occurred, a fire alarm is triggered to instruct the monitoring personnel to conduct a secondary review. If it is determined that a fire does exist, firefighting is immediately carried out.

[0074] The fire identification method provided by the embodiment of the present invention eliminates the outline of the moving foreground with a smaller area, reduces abnormal noise and the interference caused by the changes of a few pixels in continuous frames of the video, and at the same time reduces the amount of calculation during subsequent verification. Finally, by determining the overlap ratio and determining the relationship between the overlap ratio and the preset threshold, consistency verification is achieved between the moving area of the video and the area where fireworks and fire exist, further ensuring the actual existence of the disaster and reducing the false detection rate of fireworks and fire.

[0075] Example 3

[0076] Figure 5 This is a structural diagram of a fire identification device provided by the third embodiment of the present invention. Figure 5 As shown, the device includes: a foreground recognition module 301, a fire suspected area determination module 302 and a fire identification result determination module 303, wherein:

[0077] A foreground recognition module is used to obtain a video frame to be recognized and determine a moving foreground area and a contour of the moving foreground in the video frame to be recognized;

[0078] a suspected fire area determination module, configured to process the outline of the motion foreground using a preset target detection model and output the suspected fire area;

[0079] The fire identification result determination module is used to determine the fire identification result according to the overlapping area between the suspected fire area and the motion foreground area.

[0080] The fire identification device provided by the embodiment of the present invention automatically filters out non-moving objects based on motion (foreground) area detection in video frames, locks on areas where fireworks actually exist, reduces false alarms, and evaluates the consistency of the motion area and the area where fireworks are suspected to exist output by the model based on the overlapping area of the two areas. This can accurately identify the authenticity of the fire in various complex environments, further reduce the false alarm rate of fireworks, achieve fast and accurate fire monitoring, and provide important technical support for the security field.

[0081] Optionally, the method for determining the preset target detection model includes: obtaining target training samples with labels, wherein the categories of the labels include fire-related target labels and fire-confused target labels, and the fire-confused targets are targets similar to fire-related targets; and using the target training samples to train the model to be trained to obtain the preset target detection model.

[0082] Optionally, the model to be trained includes a backbone network Backbone, a neck network Neck and a detection head network Head connected in series, wherein: the neck network Neck is used to perform feature fusion and enhancement on the outputs of multiple preprocessing residual networks PRes of the backbone network Backbone, wherein the preprocessing residual network PRes includes a bottleneck network Bottleneck and a main bottleneck network PBottleneck connected in series, and the main bottleneck network PBottleneck includes a first convolution batch normalization activation function module, a partial convolution PConv and a second convolution batch normalization activation function module connected in series.

[0083] Optionally, the backbone network Backbone includes: a first preprocessing residual network Pres, a second preprocessing residual network Pres, and a third preprocessing residual network Pres connected in series in sequence; wherein: the first Pres output of the first preprocessing residual network Pres is output to the neck network Neck and the second preprocessing residual network Pres respectively; the second Pres output of the second preprocessing residual network Pres is output to the neck network Neck and the third preprocessing residual network Pres respectively; the third Pres output of the third preprocessing residual network Pres is output to the neck network Neck.

[0084] Optionally, the output of the backbone network Backbone includes a first Pres output, a second Pres output, and a third Pres output; in the neck network Neck: the input of the third convolution batch normalization activation function module is the first Pres output, and the output is output to the first cross-stage partial connection module; the output of the first cross-stage partial connection module is output to the concatenation layer Concat and the first downsampling layer Down respectively; the output of the first downsampling layer Down is output to the first fusion layer Fusion; the output of the first fusion layer Fusion is output to the concatenation layer Concat and the second downsampling layer Down respectively; the output of the second downsampling layer Down is output to the second fusion layer Fusion; the output of the second fusion layer Fusion is output to the concatenation layer Concat ; The input of the fourth convolution batch normalization activation function module is the second Pres output, and the output is output to the second cross-stage partial connection module; the output of the second cross-stage partial connection module is output to the fifth convolution batch normalization activation function module; the output of the fifth convolution batch normalization activation function module is output to the first fusion layer Fusion and the first upsampling layer UP respectively; the output of the first upsampling layer UP is output to the first cross-stage partial connection module; the input of the attention-based intra-scale feature interaction layer AIFI is the third Pres output, and the output is output to the sixth convolution batch normalization activation function module; the output of the sixth convolution batch normalization activation function module is output to the second fusion layer Fusion and the second upsampling layer UP respectively; the output of the second upsampling layer UP is output to the second cross-stage partial connection module.

[0085] Optionally, the target training sample is generated by obtaining a segmentation mask and a scene image of a fireworks image; generating a fire scene sample by pasting the image corresponding to the segmentation mask to the scene image, and generating a labeled target training sample using the fire scene sample.

[0086] Optionally, the fire identification result determination module includes:

[0087] an overlap ratio determining unit, configured to determine, for each of the suspected fire areas, an overlap area between the current suspected fire area and a motion foreground area intersecting with the current suspected fire area, and determine a ratio of the overlap area to the area of the current suspected fire area as the overlap ratio;

[0088] The fire identification result determination unit is used to determine that the fire identification result is that a fire exists if the overlap ratio is greater than or equal to a preset threshold.

[0089] Optionally, the device further includes:

[0090] The screening module is used to remove the contours of the motion foreground whose contour area is smaller than a preset value before processing the contours of the motion foreground using the preset target detection model and outputting the suspected fire area, so as to obtain the updated contours of the motion foreground.

[0091] The fire identification device provided in the embodiment of the present invention can execute the fire identification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0092] Example 4

[0093] Figure 6 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0094] like Figure 6 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., which is communicatively connected to the at least one processor 41. The memory stores a computer program that can be executed by the at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 into the random access memory (RAM) 43. Various programs and data required for the operation of the electronic device 40 can also be stored in the RAM 43. The processor 41, ROM 42, and RAM 43 are connected to each other via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0095] Multiple components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a magnetic disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0096] Processor 41 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 41 executes the various methods and processes described above, such as the fire identification method.

[0097] In some embodiments, the fire identification method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the fire identification method described above can be performed. Alternatively, in other embodiments, processor 41 can be configured to execute the fire identification method in any other suitable manner (e.g., via firmware).

[0098] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] The computer device provided above can be used to execute the fire identification method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0101] Example 5

[0102] In the context of the present invention, a computer-readable storage medium may be a tangible medium, wherein the computer-executable instructions, when executed by a computer processor, are used to perform a fire identification method, the method comprising:

[0103] Acquire a video frame to be identified, and determine a motion foreground area and a contour of the motion foreground in the video frame to be identified;

[0104] Using a preset target detection model to process the outline of the moving foreground, and output the suspected fire area;

[0105] The fire identification result is determined based on the overlapping area between the suspected fire area and the motion foreground area.

[0106] In the context of the present invention, computer-readable storage medium can be a tangible medium that can contain or store a computer program for use with an instruction execution system, device or equipment or used in conjunction with an instruction execution system, device or equipment. Computer-readable storage medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] The computer device provided above can be used to execute the fire identification method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0108] It is worth noting that in the embodiment of the above-mentioned fire identification device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0109] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A fire identification method, characterized in that: include: Acquire a video frame to be identified, and determine a motion foreground area and a contour of the motion foreground in the video frame to be identified; Using a preset target detection model to process the outline of the moving foreground, and output the suspected fire area; The fire identification result is determined based on the overlapping area between the suspected fire area and the motion foreground area.

2. The method according to claim 1, characterized in that The method for determining the preset target detection model includes: Obtaining target training samples with labels, wherein the categories of the labels include fire-related target labels and fire-confused target labels, and the fire-confused target is a target similar to the fire-related target; The target training samples are used to train the model to be trained to obtain a preset target detection model.

3. The method according to claim 2, characterized in that The model to be trained includes a backbone network Backbone, a neck network Neck, and a detection head network Head connected in series, wherein: The neck network Neck is used to perform feature fusion and enhancement on the outputs of multiple preprocessing residual networks PRes of the backbone network Backbone, wherein the preprocessing residual network PRes includes a bottleneck network Bottleneck and a main bottleneck network PBottleneck connected in series, and the main bottleneck network PBottleneck includes a first convolution batch normalization activation function module, a partial convolution PConv and a second convolution batch normalization activation function module connected in series.

4. The method according to claim 3, characterized in that The backbone network Backbone includes: a first preprocessing residual network Pres, a second preprocessing residual network Pres, and a third preprocessing residual network Pres connected in series; in: The first Pres output of the first preprocessing residual network Pres is output to the neck network Neck and the second preprocessing residual network Pres respectively; The second Pres output of the second preprocessing residual network Pres is output to the neck network Neck and the third preprocessing residual network Pres respectively; The third Pres output of the third preprocessing residual network Pres is output to the neck network Neck.

5. The method according to claim 3 or 4, characterized in that The output of the backbone network Backbone includes a first Pres output, a second Pres output and a third Pres output; In the neck network: The input of the third convolution batch normalization activation function module is the output of the first Pres, and the output is output to the first cross-stage partial connection module; the output of the first cross-stage partial connection module is output to the concatenation layer Concat and the first downsampling layer Down respectively; the output of the first downsampling layer Down is output to the first fusion layer Fusion; the output of the first fusion layer Fusion is output to the concatenation layer Concat and the second downsampling layer Down respectively; the output of the second downsampling layer Down is output to the second fusion layer Fusion; the output of the second fusion layer Fusion is output to the concatenation layer Concat; The input of the fourth convolution batch normalization activation function module is the output of the second Pres, and the output is output to the second cross-stage partial connection module; the output of the second cross-stage partial connection module is output to the fifth convolution batch normalization activation function module; the output of the fifth convolution batch normalization activation function module is output to the first fusion layer Fusion and the first upsampling layer UP respectively; the output of the first upsampling layer UP is output to the first cross-stage partial connection module; The input of the attention-based intra-scale feature interaction layer AIFI is the output of the third Pres, and the output is output to the sixth convolutional batch normalization activation function module; the output of the sixth convolutional batch normalization activation function module is output to the second fusion layer Fusion and the second upsampling layer UP respectively; the output of the second upsampling layer UP is output to the second cross-stage partial connection module.

6. The method according to claim 2, characterized in that The target training sample is generated by: Get the segmentation mask and scene image of the fireworks image; A fire scene sample is generated by pasting the image corresponding to the segmentation mask to the scene image, and a target training sample with a label is generated using the fire scene sample.

7. The method according to claim 1, characterized in that The determining of the fire identification result based on the overlapping area between the suspected fire area and the motion foreground area includes: For each of the suspected fire areas, determining the overlapping area between the current suspected fire area and the motion foreground area intersecting with the current suspected fire area, and determining the ratio of the overlapping area to the area of the current suspected fire area as the overlapping ratio; If the overlap ratio is greater than or equal to a preset threshold, the fire identification result is determined to be the presence of a fire.

8. A fire identification device, characterized in that: include: A foreground recognition module is used to obtain a video frame to be recognized and determine a moving foreground area and a contour of the moving foreground in the video frame to be recognized; a suspected fire area determination module, configured to process the outline of the motion foreground using a preset target detection model and output the suspected fire area; The fire identification result determination module is used to determine the fire identification result according to the overlapping area between the suspected fire area and the motion foreground area.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to execute the fire situation identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the fire identification method according to any one of claims 1 to 7 when executed.

Images