A method, system and storage medium for detecting a smoke fire

Abstract

The application relates to the technical field of intelligent image recognition, and particularly discloses a firework detection method and system and a storage medium. The method comprises the following steps: acquiring a to-be-detected video, and extracting a group of image sequence frames from the to-be-detected video; based on the image sequence frames, a group of corresponding deformation parameters are set according to a time frame sequence; the image sequence frames and the corresponding deformation parameters are taken as inputs, a preset network model is used to acquire detection results of each frame of image; and whether the to-be-detected video has fireworks is determined by combining the detection results of each frame of image. According to the application, a group of image sequence frames are used for firework detection, different proportion prior boxes are set for the to-be-detected image sequence according to a time frame sequence, and the different proportion prior boxes are taken as target prediction regions, so that the deformation state of the target can be better detected, and the determination of the firework recognition is facilitated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and particularly relates to a fireworks detection method and system and a storage medium. BACKGROUND

[0002] As an image recognition technology, the fireworks detection plays a good protective role in industrial production areas, public spaces and many natural environments.

[0003] The existing fireworks detection method mainly uses machine learning or deep learning method to process and analyze the collected video images, extracts the fireworks features in the image frames to realize the fireworks detection. However, when the existing target detection algorithm is used to detect the fireworks in the image frames, the candidate region or the prior box is usually obtained first, and then the image in the candidate region or the prior box is matched with the real fireworks target to realize the detection and recognition of the fireworks. However, since the fireworks are dynamic targets, not simply moving, but the shape of the fireworks is always in a state of change, so when the fixed candidate region box is used for detection, the possibility of missing detection will be very large. SUMMARY

[0004] The purpose of the present application is to provide a fireworks detection method, system and storage medium, which detects the fireworks through a group of image sequence frames, and sets different candidate regions for different frame images, and records the deformation state of the detection target to help determine whether the target is fireworks.

[0005] In a first aspect, the present application provides a fireworks detection method, which adopts the following technical solution:

[0006] Obtain a to-be-detected video, and extract a group of image sequence frames from the to-be-detected video;

[0007] Based on the image sequence frames, set a group of corresponding deformation parameters according to the time frame sequence;

[0008] Take the image sequence frames and the corresponding deformation parameters as inputs, obtain the detection results of each frame image through a preset network model, and determine whether there is fireworks by combining the detection results of each frame image.

[0009] Through the above technical solution, different scales of prediction regions can be assigned to different frame images to fit the dynamic change process of the fireworks target, and then a group of image sequence frames are used to comprehensively evaluate whether there is fireworks, so as to improve the robustness of the entire fireworks detection method.

[0010] Optionally, the intelligent device information includes device purpose, and the extracting a group of image sequence frames from the to-be-detected video comprises:

[0011] Converting a video to be detected into an image sequence;

[0012] Selecting a group of images from the image sequence according to a preset time sequence and image number;

[0013] Data preprocessing is performed on the selected group of images, and the image sequence frame is recorded.

[0014] Optionally, a set of corresponding deformation parameters is set according to the time sequence based on the image sequence frame, including:

[0015] According to the time sequence of the image sequence frame, the time interval between each image frame is obtained;

[0016] According to the time interval, the reference deformation variable of the next image frame relative to the previous image frame is calculated and obtained through a preset calculation method;

[0017] Based on all the reference deformation variables, the corresponding deformation parameters are calculated and obtained.

[0018] Optionally, the image sequence frame and the corresponding deformation parameter are input into a preset network model to obtain the detection result of each frame image, including:

[0019] Feature extraction is performed on each frame image in the image sequence frame to obtain the corresponding feature map;

[0020] According to the deformation parameters and the corresponding feature maps of the image sequence frame, different scale prior boxes are obtained;

[0021] According to the prior box of each frame image, the detection result is obtained through a preset network model.

[0022] Optionally, the different scale prior boxes are obtained according to the deformation parameters and the corresponding feature maps of the image sequence frame, including:

[0023] According to the feature map corresponding to each frame image, a linear change rule is set for the scale of the prior box;

[0024] The linear change rule is adjusted by the deformation parameters corresponding to each frame image;

[0025] According to the feature map corresponding to each frame image and the adjusted linear change rule, the prior box corresponding to different scales is obtained.

[0026] Optionally, the detection result includes target position information, detection score and firework category, and the detection results of each frame image are combined to determine whether there is a firework, including:

[0027] According to the time sequence of the image, the detection results of each frame image are sorted;

[0028] According to the target position information in the detection result, it is judged whether there is target position information indicating the same detection target;

[0029] If yes, the position information belonging to the same detection target is added to the data set;

[0030] According to the target position information in the data set, the morphological change information of the target is calculated and obtained;

[0031] According to the morphological change information of the target and the corresponding detection result, a comprehensive evaluation is given to whether there is fireworks, and evaluation information is output.

[0032] Optionally, according to the target position information in the detection result, it is judged whether there is target position information indicating the same detection target, comprising:

[0033] Taking the detection result of the first frame image as a reference object, the detection results of the subsequent frame images are sequentially taken as comparison objects;

[0034] According to the target position information of the reference object and the comparison object, the position coincidence degree of the two is calculated and obtained;

[0035] It is judged whether the coincidence degree of the two is greater than a preset threshold,

[0036] If yes, the current comparison object is marked as the same detection target as the current reference object.

[0037] Optionally, after the comprehensive evaluation of whether there is fireworks according to the morphological change information of the target and the corresponding detection result, it further comprises:

[0038] According to the detection result, the belonging firework category is confirmed;

[0039] According to the morphological change information of the target, the trend type of the change is obtained by trend analysis method;

[0040] According to the belonging category and the trend type, the fire level is confirmed through the preset fire reference information table.

[0041] In a second aspect, the application provides a firework detection system, comprising:

[0042] A data acquisition module (101) is used to acquire a video to be detected, and extract a group of image sequence frames from the video to be detected;

[0043] A deformation parameter setting module (102) is used to set a group of corresponding deformation parameters according to time frame sequence based on the image sequence frames; a target detection module (103) is used to take the image sequence frames and the corresponding deformation parameters as inputs, and obtain the detection results of each frame image through a preset network model;

[0044] A result determination module (104) is configured to determine whether the detection results of the images have fireworks.

[0045] In a third aspect, the present application provides a computer readable storage medium storing a computer program capable of being loaded by a processor and executing the above-mentioned method for detecting fireworks.

[0046] In summary, the present application detects the fireworks target by using multiple image frames, and assigns different prediction regions of different scales to different image frames to fit the dynamic change process of the fireworks target, so as to better capture the change state of the deformed fireworks target. Then, the detection results of the image frames are used to comprehensively determine the detected target, so as to improve the robustness of the whole method for detecting fireworks. In addition, the change trend of the fireworks is predicted in combination with the position of the target, and the firework situation can also be hierarchically evaluated, so as to facilitate timely response measures. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flowchart of a method for detecting fireworks provided by an embodiment of the present application;

[0048] Figure 2 is a flowchart of setting a group of corresponding deformation parameters according to a time frame sequence provided by an embodiment of the present application;

[0049] Figure 3 is a flowchart of obtaining detection results of image frames by using a preset network model provided by an embodiment of the present application;

[0050] Figure 4 is a flowchart of determining whether the detection results of the images have fireworks provided by an embodiment of the present application;

[0051] Figure 5 is a schematic diagram of a system for detecting fireworks provided by an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be described in detail below with reference to the accompanying drawings. Figure 1 - the accompanying drawings Figure 5 The present application will be further described in detail.

[0053] The present application provides a method for detecting fireworks, referring to Figure 1 , comprising the following steps:

[0054] S100, obtaining a video to be detected, and extracting a group of image sequence frames from the video to be detected.

[0055] The video data of the to-be-detected video representation that needs to be detected whether the fireworks appear can be a video suspected to appear the fireworks by artificial shooting, or a video that needs to be checked for the fire source from the related monitoring device because the related fireworks alarm device is triggered. The specific source of the video is not limited in the application.

[0056] The image sequence frame represents an image sequence obtained after the video is converted into images at a set frame rate.

[0057] It is worth noting that the to-be-detected video here is a to-be-detected video that is intercepted with a corresponding frame image as a reference after a suspected fireworks target is preliminarily detected by a corresponding detector, for example, from the video area before the suspected fireworks target appears.

[0058] Because there are many disturbances for the fireworks recognition, such as light, cloud, chimney, etc., it is easy to be misdetected by only using a single picture frame to detect and recognize.

[0059] In addition, the fireworks detection is different from the general target detection because the fireworks detection belongs to a dynamic target, and the shape is always in a changing state, such as the fire and the smoke gradually becoming larger, so usually multiple frames of images need to be extracted from the video image for comprehensive evaluation. It is also because the shape of the fireworks is always in a changing state, in order to better capture the shape change of the fireworks target, the changing state of the fireworks target needs to be included in the detection model, so different detection range areas are determined according to the time change for different image frames, so as to better fit the change trend of the fireworks target, which can also help to realize more accurate fireworks detection.

[0060] Therefore, in the embodiment of the application, after the to-be-detected video is obtained, image frames are extracted as input images to be detected.

[0061] Specifically, a group of image sequence frames are extracted from the to-be-detected video, including the following steps:

[0062] S110, converting the to-be-detected video into an image sequence.

[0063] S120, selecting a group of images from the image sequence according to a preset time frame sequence and the number of images.

[0064] S130, performing data preprocessing on the selected group of images, and recording as image sequence frames.

[0065] In the embodiments of the present application, first, the video to be detected is converted into an image sequence at a set frame rate, and then a group of images is extracted from the image sequence in a manner similar to frame skipping. Specifically, a group of time intervals can be set according to the time frame sequence and the number of images to be extracted, and starting from the first frame image, images are sequentially extracted from the image sequence at the set time intervals. The finally obtained group of images is denoted as image sequence frames. It should be noted that the number of images extracted and the setting of the time interval can be set according to the actual size of the video, the size of the detection target, etc., and the present application is not limited in this regard.

[0066] After obtaining the image sequence frames, corresponding image preprocessing is also performed, such as filtering, illumination normalization, etc., to improve the distinguishability of the images to be detected.

[0067] S200, based on the image sequence frames, a group of corresponding deformation parameters is set according to the time frame sequence.

[0068] Since the present application aims to use a group of images to comprehensively determine whether there is fireworks, in addition to the conventional target category detection, the morphological changes of the target are also taken into account. Because fireworks are dynamic targets, there are usually changes in the form, such as the fire from big to small, the smoke becoming thick, the fire spreading, the smoke diffusing, etc. The capture of these changes can help determine whether the target is fireworks.

[0069] Because when using deep learning to detect targets, a candidate box or a prior box is usually selected as the detection region, such as the classic target detection algorithms FAST-RCNN, SSD, etc.

[0070] After the detection region is set, the size of the target detection is also determined. Therefore, for cases where the target difference is large, it is difficult to take into account both large and small targets without missing small targets. For example, FAST-RCNN has difficulty in simultaneously taking into account large and small targets in the selection of candidate regions. Although SSD sets prior boxes based on multi-scale features and sets prior boxes of different scales according to feature maps of different scales to achieve dense sampling, it better takes into account large and small targets.

[0071] Although SSD can better fit targets of different sizes by dense sampling in the selection of detection regions, it is difficult to comprehensively detect targets whose form changes. For example, when a target is detected in a frame image, a region box is generated according to the position of the target, and a score belonging to the category of fireworks is assigned. The region box obtained here is obtained from the prior box, i.e., the detection region, set previously.

[0072] But to the next frame or after a few frames of another frame image, the target is still there, but the target form has changed, is no longer the original detection area size, due to the prior box size limit, this time may detect the target area may be the same or similar to the previous detection area, but not the full picture of the target, so as to see the change of the target from the detection result.

[0073] Therefore, in the embodiment of the application, in order to better capture the changed target, the prior box will be adjusted accordingly, that is, the scale of the prior box will change accordingly with the deformation of the target. That is, for the input image sequence frame, a set of corresponding deformation parameters is set according to the time frame sequence, and the deformation parameters are used to guide the setting of the prior box of each frame image.

[0074] Specifically, based on the image sequence frame, a set of corresponding deformation parameters is set according to the time frame sequence, referring to Figure 2 , including the following steps:

[0075] S210, according to the time frame sequence of the image sequence frame, confirming the time interval between each image frame.

[0076] S220, according to the time interval, calculating and obtaining the reference deformation variable of the next frame image relative to the previous frame image through a preset calculation method.

[0077] S230, based on all the reference deformation variables, calculating and obtaining the corresponding deformation parameters.

[0078] Since the deformation of fireworks is not easy to estimate, different scenes will be different. Therefore, the time difference between the extracted image frames is used as a reference rule.

[0079] First, according to the time frame sequence of the image sequence frame, the so-called time frame sequence is the time point after the video is converted into images, and multiple images at the same time point are sequentially numbered according to the conversion order. It is worth noting that the image sequence frame belongs to different time points. The specific time interval between two frames can be calculated, and it is mentioned above that the image sequence frame is extracted by setting the corresponding time interval, that is, the time interval between each image frame can be directly obtained according to the image sequence frame.

[0080] After obtaining the time interval between each image in the image sequence frame, the reference deformation variable of the next frame image relative to the previous frame image is calculated in turn through a preset calculation method, taking the first frame image as a reference standard.

[0081] Let the deformation variable of the i-th frame image relative to the i-1-th frame image be f i , and the time interval between the i-th frame image and the i-1-th frame image be ti , set the deformation coefficient as σ, then the deformation variable f i can be expressed as:

[0082]

[0083] Wherein i∈[2,n], n is the number of images in the selected image sequence frame, f1=0, denoted as the reference initial deformation variable.

[0084] Taking the first frame image as the reference, the deformation parameters θ i , θ i of the i-th frame image can be sequentially calculated through the deformation variable of each frame image relative to the previous frame image.

[0085] w i = w i-1 + f i

[0086]

[0087]

[0088] Wherein w i represents the size of the i-th frame image prior box, and the initial prior box size w1 can be obtained through the initial configuration parameters of the network. There are many prior boxes, and according to the network structure and the linear change rule of the set different scale feature maps and prior boxes, many different scale prior boxes can be obtained, and here w i refers to all prior boxes.

[0089] It is worth mentioning that the deformation of the fireworks target is not necessarily a linear change state, and it can be irregular, and it is related to different fireworks states and the scene and even the weather, so the deformation coefficient set above will consider more factors in combination with the actual situation, and the present application is not limited.

[0090] S300, taking the image sequence frame and the corresponding deformation parameter as input, obtaining the detection result of each frame image through the preset network model.

[0091] Wherein, the preset network model, the present application uses the fireworks detection model obtained by training the SSD network architecture.

[0092] After obtaining the image sequence frame and the corresponding deformation parameter, the trained fireworks detection model can be used for detection, and the detection result of each frame image can be obtained.

[0093] Specifically, taking the image sequence frame and the corresponding deformation parameter as input, obtaining the detection result of each frame image through the preset network model, referring to Figure 3comprising the following steps:

[0094] S310, feature extraction is performed on each frame image in the image sequence frame to obtain a corresponding feature map.

[0095] S320, different scale prior boxes are obtained according to the deformation parameters corresponding to the image sequence frame and the corresponding feature map.

[0096] S330, a detection result is obtained by a preset network model according to the prior box of each frame image.

[0097] In the embodiments of the present application, first, feature extraction is performed on the image to be detected. Because different network levels or depths are considered, the generated features have different descriptions of the characteristics of the image. For example, generally, the feature receptive field of the shallow level is small, which is suitable for processing small targets, and the deep layer resolution information is insufficient and is not suitable for small targets, and is more suitable for processing large targets. SSD is to extract different scale feature maps for detection according to the size difference of various targets, and uses large scale feature maps (relatively earlier feature maps) to detect small objects and uses small scale feature maps (relatively later feature maps) to detect large objects.

[0098] In order to detect targets of different sizes and shapes in the image, different scale feature maps generated by feature extraction are set to different size prior boxes. Because it is considered that there is a certain time difference and the detected target will be found to be deformed, the setting of the prior box will be adjusted according to the deformation parameters corresponding to each frame. It is worth noting here that only the case of the morphological enlargement of the firework detection target is considered, because under normal circumstances, if a firework appears, the normal condition is that the firework will spread and diffuse, and it is just because of this that it is necessary to stop the firework before it grows up.

[0099] Specifically, different scale prior boxes are obtained according to the deformation parameters corresponding to the image sequence frame and the corresponding feature map, comprising the following steps:

[0100] S321, a linear change rule is set for the scale of the prior box according to the feature map corresponding to each frame image.

[0101] S322, the linear change rule is adjusted by the deformation parameters corresponding to each frame image.

[0102] S323, the corresponding different scale prior boxes are obtained according to the feature map corresponding to each frame image and the adjusted linear change rule.

[0103] Firstly, the setting of the prior box includes two aspects of scale (size) and aspect ratio. For the scale of the prior box, a linear increasing rule is set: as the size of the feature map decreases, the scale of the prior box increases linearly.

[0104] The scale of the prior box represents the proportion of the prior box size relative to the picture, and the scale of the kth feature map is denoted as s k , s k may be expressed as:

[0105]

[0106] wherein, m represents the number of different scale feature maps, and s min and s max respectively represent the minimum value and the maximum value of the proportion of the prior box size relative to the picture, and a set of aspect ratios is set, so that a set of prior boxes with the same scale but different aspect ratios can be set for each unit of the feature map according to the scale of the prior box of the current feature map. In this way, prior boxes with different proportions and sizes can be sequentially set for different scale feature maps.

[0107] In the embodiment of the present application, for the first frame image in the image sequence frame to be detected, the prior box is set according to the above rules, but for the subsequent frame images, the first frame image is taken as a reference, and the prior box set for the first frame image is adjusted through the set deformation parameters.

[0108] The generation rule of the prior box of each frame image is adjusted through the introduction of the deformation parameters, and then the corresponding feature maps of each frame image and the adjusted linear change rule are used to obtain prior boxes with different proportions. In this way, the entire detection model can better adapt to the dynamic changes of the fireworks target, help to detect the target deformation process, and thus further confirm whether the target is fireworks.

[0109] S400, the detection results of the combined frame images are determined to have fireworks.

[0110] The detection result includes target position information, a detection score and a belonging category, the target position information is equivalent to a target detection region box, which is generated by a top-left corner coordinate and a length and width of the region box.

[0111] Since the network model takes fireworks detection as the target, the categories set in the training process of the network all belong to the fireworks category. The categories here can be divided according to the actual scene, which can be one category or can be subdivided into multiple categories, such as the open fire category, the smoke category, etc. Therefore, the final presented result is only the detected target region and the detection score of the belonging fireworks category, and the detection score is equivalent to the confidence, and the higher the score is, the higher the probability of belonging to the category of fireworks is.

[0112] Since the present application aims to detect the fireworks target through multiple frames of images and hopes to capture the dynamic change process of the target through the detection result, the generated target position information is recorded, and whether the fireworks exists is comprehensively determined through the change of the position information of the same detection target.

[0113] Specifically, the detection results of the respective frames of images are combined to determine whether the fireworks exists, referring to Figure 4 , comprising the following steps: S410, sorting the detection results of the respective frames of images according to the time frame sequence of the images.

[0114] S420, determining whether there is target position information indicating the same detection target according to the target position information in the detection results.

[0115] S430, if yes, adding the position information belonging to the same detection target to the data set.

[0116] S440, calculating and obtaining the morphological change information of the target according to the target position information in the data set.

[0117] S450, comprehensively evaluating whether the fireworks exists according to the morphological change information of the target and the corresponding detection result, and outputting the evaluation information.

[0118] In the embodiment of the present application, the detection results of the respective frames of images are combined to determine whether the fireworks exists, first, the detection results of the respective frames of images are sorted according to the frame sequence of the images, because the change process of the target is related to time, then whether it is the same target is determined according to the target position information in the detection results.

[0119] Because the fireworks target will not shift or move randomly, but more likely to spread or diffuse on the basis of the related fire source, if the fireworks target is detected in a frame of image, under the condition that the whole model has a certain accuracy, if the fireworks is not effectively controlled or automatically extinguished, the fireworks target should still be detected in the subsequent, and the detected position information has a great possibility of being related to the original position, of course, some special cases cannot be ruled out, for example, the fire source is blown to another place due to the wind or is artificially controlled.

[0120] Therefore, the position information of the detection target can be used to preliminarily determine whether it is the same target.

[0121] Specifically, whether there is target position information indicating the same detection target is determined according to the target position information in the detection results, comprising the following steps:

[0122] S421, taking the detection result of the first frame of image as the reference object, and sequentially taking the detection results of the subsequent frames of images as the comparison objects.

[0123] S422、According to the target position information of the reference object and the comparison object, the position coincidence degree of the two is calculated and obtained.

[0124] S423, judge whether the coincidence degree of the two is greater than a preset threshold.

[0125] S424, if yes, the current comparison object is marked as the same detection target as the current reference object.

[0126] Wherein, the position coincidence degree is equivalent to the intersection over union IOU, that is, the ratio of the intersection and the union of the two bounding boxes.

[0127] By taking the first frame image as a control, taking the target position information in the detection result of the first frame image as a reference object, and taking the target position information in the detection result of the subsequent frame image as a comparison object, the comparison object and the reference object are sequentially calculated according to the intersection over union IOU in the order of image frame sequence, and then it is judged whether the intersection over union IOU is greater than a preset threshold. If yes, it means that it is the same target, and a mark is given, and the comparison object and the reference object are added to the set data collection together. Subsequently, the comparison object with the reference object indicating the same target is also added to the data collection.

[0128] Then, by the target position information in the data collection, the morphological change information of the target can be calculated. Since the target position information in the detection result is associated with the set prior box, that is, the size of the final target detection box is fixed, only such a target detection box is enough, and the final detection result is also the presentation result after filtering, and the detection score is greater than a certain threshold.

[0129] Since the present application gives different prior box setting rules to different frames of images to be detected. If the fireworks target is detected in the first frame image, at this time, only the detection score can be used as the judgment basis. But if the fireworks target is also detected in the subsequent frame image, and it is judged that it belongs to the same target, and the detection box corresponding to the target of each subsequent frame has a certain expansion compared with the detection box of the previous frame, which meets the adjustment of the prior box according to the deformation parameter. It means that the target is deformed, so it can be considered that the probability of the target being fireworks is larger, combined with the corresponding detection score, and then the final result is given. Or generate prompt information for the corresponding detection information, so as to inform the relevant personnel.

[0130] It is worth mentioning that the adjustment of the prior frame by the deformation parameter is not to indicate that the deformation process of the fireworks target cannot be detected without adjustment, but to expand the detection area of the captured target accordingly so as to better adapt to the deformation condition of the target. For example, if the detected fireworks target area has reached the maximum size of the set prior frame, it is difficult to detect the changed target or capture the changed state of the target. By increasing the deformation parameter to adjust the size of the prior frame, the condition can be better fitted.

[0131] In addition, the analysis of the deformation process of the target can not only assist in determining whether it is a fireworks target, but also help to identify the fire situation. For example, if the change trend of the target is large, it indicates that the fire development is rapid, and the corresponding fire situation level is higher.

[0132] Therefore, in the embodiment of the present application, after obtaining the detection result of the image sequence to be detected and obtaining the morphological change information of the detected target, the level of obtaining is evaluated after confirming the presence of the fireworks target.

[0133] Specifically, after giving a comprehensive evaluation of whether there is a fireworks according to the morphological change information of the target and the corresponding detection result, the following steps are further included:

[0134] S510, confirming the classification of the fireworks according to the detection result.

[0135] S520, obtaining the change trend type by trend analysis method according to the morphological change information of the target.

[0136] S530, confirming the fire situation level by the preset fire situation reference information table according to the classification and the trend type.

[0137] The trend analysis method is a method of prediction analysis, and the trend type is usually divided into linear trend line, logarithmic trend line, exponential trend, power trend line, etc.

[0138] The fire situation reference information table is a fire situation level reference table set according to the possible fireworks categories and the growth environment of the fireworks in the actual scene.

[0139] In the embodiments of the present application, after the comprehensive determination of the presence or absence of the firework according to the detection result, the fire level is also evaluated accordingly. First, according to the detection result, the classification of the firework is confirmed. Then, according to the morphological change information of the target, the trend analysis method is used for corresponding calculation to obtain the trend type of the morphological change. Finally, according to the classification and the trend type, the fire level is confirmed by referring to the preset fire reference information table. The confirmation of the fire level is generally after the determination that the current fire target exists, and the fire level information is sent or prompted in another form so as to take corresponding measures.

[0140] The embodiments of the present application also provide a firework detection system, which refers to Figure 5 The system comprises a data acquisition module 101, a deformation parameter setting module 102, a target detection module 103, and a result determination module 104.

[0141] The data acquisition module 101 is configured to acquire a video to be detected and extract a group of image sequence frames from the video to be detected.

[0142] The deformation parameter setting module 102 is configured to set a group of corresponding deformation parameters according to a time frame sequence based on the image sequence frames.

[0143] The target detection module 103 is configured to take the image sequence frames and the corresponding deformation parameters as inputs, and obtain the detection results of each frame of image through a preset network model.

[0144] The result determination module 104 is configured to determine whether there is a firework by combining the detection results of each frame of image.

[0145] In the embodiments of the present application, the data acquisition module 101 is specifically configured to acquire the video to be detected, convert the video to be detected into an image sequence, extract a group of image sequence frames from the image sequence, and pre-process the extracted images.

[0146] The deformation parameter setting module 102 is specifically configured to set a group of corresponding deformation parameters according to a time frame sequence based on the image sequence frames obtained by the data acquisition module 101.

[0147] The target detection module 103 is specifically configured to take the image sequence frames obtained by the data acquisition module 101 and the deformation parameters generated by the deformation parameter setting module 102 as inputs, and obtain the detection results of each frame of image through a preset network model.

[0148] The result determination module 104 is specifically configured to determine whether there is a firework by combining the detection results of each frame of image, and also perform corresponding fire evaluation on the existing fire target.

[0149] The embodiment of the application further provides a computer readable storage medium, which stores a computer program capable of being loaded by a processor and executing any of the smoke detection methods.

[0150] The embodiments of the specific implementation are the preferred embodiments of the application, and do not limit the protection scope of the application, so that: any equivalent changes made according to the principles of the application should be covered within the protection scope of the application.

Claims

1. A method for detecting smoke and fire, characterized in that, include: S100. Acquire the video to be detected and extract a set of image sequence frames from the video to be detected; Step S100 includes S110-S130: S110, convert the video to be detected into an image sequence; S120, select a group of images from the image sequence according to a preset time frame order and number of images, and extract the images in a frame skipping manner; S130, perform data preprocessing on the selected group of images and record them as image sequence frames. S200: Based on the image sequence frames, set a set of corresponding deformation parameters according to the time frame order; wherein step S200 includes S210-S230: S210. Obtain the time interval between each image frame according to the time frame order of the image sequence frames; S220. Based on the time interval, calculate and obtain the reference deformation of the next frame image relative to the previous frame image using a preset calculation method; wherein, the deformation f i The calculation formula is: Among them, t i It is the time interval, i∈[2,n], where n is the number of images in the selected image sequence frame, f1=0, which is the reference initial deformation; f i f is the deformation of the i-th frame relative to the (i-1)-th frame. i-1 Let σ be the deformation of the (i-1)th frame of the image, and σ be the deformation coefficient. S230. Based on all reference deformations, calculate and obtain the corresponding deformation parameters; Among them, the deformation parameter θ i The calculation formula is: Where, θ i W is the deformation parameter. i-1 The size of the prior bounding box in the (i-1)th frame is given by the initial prior bounding box W1, which is obtained through the initial configuration parameters of the network; the size of the prior bounding box W1 in the i-th frame is given by the initial prior bounding box W1. i The prior bounding box W of the (i-1)th frame image i-1 Add the image shape variable f of the i-th frame i sum; S300: Using the image sequence frames and corresponding deformation parameters as input, the detection results of each frame are obtained through a preset network model; wherein step S300 includes S310-S330: S310: Extracting features from each frame in the image sequence to obtain the corresponding feature map; S320: Obtaining prior boxes of different proportions based on the deformation parameters and corresponding feature maps of the image sequence frames; S330: Obtaining the detection results based on the prior boxes of each frame through a preset network model; the preset network model is a fireworks detection model obtained by training an SSD network architecture; Among them, the prior box scale S i k The proportional adjustment formula is: Among them, the prior box S i k Let S be the scale of the prior bounding box of the k-th feature map in the i-th frame image. 1 k Let θ be the scale of the prior bounding box of the k-th feature map in the first frame image, and let θ be the deformation parameter. i The introduction of this method adjusts the prior box generation rules for each frame of the image accordingly. Then, by using the feature maps corresponding to each frame of the image and the adjusted linear change rules, prior boxes of different proportions are obtained so that the entire detection model can adapt to the dynamic changes of the fireworks target. S400: Combine the detection results of each frame image to determine whether there is smoke or fire; the detection results include target location information, detection score, and category, wherein step S400 includes S410-S450: S410. Sort the detection results of each frame of the image according to the time frame order; S420. Based on the target location information in the detection results, determine whether there is target location information indicating the same detection target; S430. If so, add the location information belonging to the same detection target to the dataset; S440. Calculate and obtain the target's morphological change information based on the target location information in the dataset; S450: Based on the target's morphological change information and the corresponding detection results, a comprehensive assessment is made as to whether smoke or fire exists, and the assessment information is output.

2. The smoke detection method according to claim 1, characterized in that, Step S320 includes: S321. Based on the feature maps corresponding to each frame of the image, set a linear variation rule for the scale of the prior bounding box; S322. Adjust the linear change rule using the deformation parameters corresponding to each frame image; S323. Based on the feature maps corresponding to each frame of the image and the adjusted linear change rules, obtain the prior boxes corresponding to different proportions.

3. The smoke detection method according to claim 1, characterized in that, Step S420 includes: S421. Using the detection result of the first frame image as a reference, the detection results of subsequent frame images are used as comparison objects in turn. S422. Calculate and obtain the position overlap between the reference object and the comparison object based on their target position information; S423. Determine whether the overlap between the two is greater than a preset threshold; where the overlap is equivalent to the Intersection over Union (IOU), which is the ratio of the intersection to the union of the two borders. S424. If so, mark the current comparison object as the same detection target as the current reference object.

4. The smoke detection method according to claim 3, characterized in that, Following step S450, the following is also included: S510. Based on the test results, confirm the category of fireworks. S520. Based on the target's morphological change information, obtain the type of change trend through trend analysis. S530. Based on the classification and trend type, confirm the fire level using the preset fire reference information table.

5. A smoke and fire detection system, characterized in that, For performing a fireworks detection method as described in any one of claims 1 to 4, the fireworks detection system comprises: The data acquisition module (101) is used to acquire the video to be detected and extract a set of image sequence frames from the video to be detected; The deformation parameter setting module (102) is used to set a set of corresponding deformation parameters based on the image sequence frames and the time frame order. The target detection module (103) is used to take the image sequence frames and the corresponding deformation parameters as input, and obtain the detection results of each frame image through a preset network model; The result determination module (104) is used to determine whether there is smoke by combining the detection results of each frame image.

6. A computer-readable storage medium storing a computer program capable of being loaded by a processor and executing a fireworks detection method as described in any one of claims 1 to 4.

Images