Multi-scale flame image detection method, device and equipment and storage medium

Abstract

The application relates to the technical field of fire detection, and discloses a multi-scale flame image detection method, device, equipment and storage medium, which comprises the following steps: performing convolution calculation on an initial flame feature image through a grouping multi-scale convolution network model to obtain a target flame feature image; iteratively extracting the target flame feature image according to a target circular convolution structure to obtain feature images of different processes; performing feature bidirectional fusion on the feature images of different processes to obtain a target feature image; and detecting the target feature image through a target image detection layer to obtain a multi-scale flame image detection result; compared with the prior art of detecting a flame image through flame static characteristics, the application can effectively improve the accuracy of detecting a flame image, because the application performs convolution calculation on an initial flame feature image, then iteratively extracts a target flame feature image, then performs feature bidirectional fusion on feature images of different processes, and finally detects a target feature image based on a target image detection layer.

Description

Technical Field

[0001] This invention relates to the field of fire detection technology, and in particular to multi-scale flame image detection methods, apparatus, equipment and storage media. Background Technology

[0002] Among various disasters, fire poses the most common and widespread threat to public safety and social development. It frequently occurs in people's daily lives and work, such as the sudden ignition of flammable and explosive materials, and the spontaneous combustion of electrical appliances, all of which can cause varying degrees of damage to people's safety and property. Therefore, rapid and accurate fire detection and early warning are of paramount importance. Currently, commonly used fire detection and early warning methods rely on characteristic signals of a fire, such as temperature, combustion gases, smoke particles, aerosols, and flame light. However, these characteristic signals gradually weaken during propagation, which limits the detection area of ​​contact detectors due to the height and size of the detection zone. In other words, the core of currently used fire detection and early warning methods is the study of characteristics, including static and dynamic features. Static features include color, shape, number of sharp corners, and roundness, while dynamic features include flashing frequency and the rate of change of flame area. However, static feature processing involves complex image preprocessing operations, and manual feature extraction is easily affected by factors such as shadows, brightness, and energy, leading to a high false alarm rate and ultimately resulting in low accuracy of flame image detection.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a multi-scale flame image detection method, apparatus, device, and storage medium, aiming to solve the technical problem of low accuracy in existing flame image detection technologies.

[0005] To achieve the above objectives, the present invention provides a multi-scale flame image detection method, which includes the following steps:

[0006] The target flame feature image is obtained by performing convolution calculation on the initial flame feature image using a grouped multi-scale convolutional network model.

[0007] The target flame feature image is iteratively extracted based on the target recurrent convolution structure to obtain feature images of different processes;

[0008] The feature images of the different processes are fused bidirectionally to obtain the target feature image;

[0009] The target feature image is detected by the target image detection layer to obtain multi-scale flame image detection results.

[0010] Optionally, before obtaining the target flame feature image by performing convolution calculations on the initial flame feature image using a grouped multi-size convolutional network model, the method further includes:

[0011] Acquire video images of the flames in the target area;

[0012] The initial flame feature image is obtained by extracting features from the flame video image of the target area using an initial convolutional network model.

[0013] Optionally, the step of performing convolution calculations on the initial flame feature image using a grouped multi-scale convolutional network model to obtain the target flame feature image includes:

[0014] Based on the grouped multi-scale convolution model, target-size convolution and target parallel structure convolution are obtained;

[0015] The initial flame feature image is processed by multi-scale calculation through the target size convolution to obtain a multi-scale flame feature image;

[0016] The target flame feature image is obtained by performing convolution calculation on the multi-scale flame feature image through the target parallel structure convolution.

[0017] Optionally, the step of performing convolution calculation on the multi-scale flame feature image through the target parallel structure convolution to obtain the target flame feature image includes:

[0018] Based on the target parallel structure convolution, convolution kernels of different sizes and the number of convolution calculations are obtained;

[0019] By selecting convolution kernels of different scales through convolution calculation strategies, a set of convolution kernels and a target scale convolution kernel are obtained.

[0020] The multi-scale flame feature image is convolved sequentially using the set of convolution kernels according to the number of convolutions to obtain the current flame feature image.

[0021] The target flame feature image is obtained by performing convolution calculation on the current flame feature image using the target scale convolution kernel and dilated convolution calculation strategy.

[0022] Optionally, the step of iteratively extracting the feature image of the target flame based on the target recurrent convolutional structure to obtain feature images of different processes includes:

[0023] The target flame features are initially extracted based on the target recurrent convolutional structure to obtain the current process feature image and extraction order.

[0024] According to the extraction order, the target flame features are iteratively extracted based on the target recurrent convolution structure to obtain the target process feature image;

[0025] By adjusting the current process feature image using the target process feature image, feature images of different processes can be obtained.

[0026] Optionally, the step of bidirectionally fusing the feature images of the different processes to obtain the target feature image includes:

[0027] Obtain bottom-up and top-down feature fusion paths;

[0028] The feature images of the different processes are fused according to a bottom-up feature fusion path to obtain a target-scale feature image;

[0029] The feature images of the different processes are sampled twice according to the top-down feature fusion path to obtain the corresponding sample feature images;

[0030] The target scale feature image is obtained by fusing the target feature image with the sample feature image.

[0031] Optionally, the step of detecting the target feature image through the target image detection layer to obtain multi-scale flame image detection results includes:

[0032] The target feature image is input into the target image detection layer so that the target image detection layer can detect the target feature image and obtain and feed back a set of flame region prediction boxes;

[0033] The prediction boxes of the flame area prediction box set are sorted by score;

[0034] The flame region prediction box ranking results are filtered according to the non-maximum suppression strategy to obtain multi-scale flame image detection results.

[0035] Furthermore, to achieve the above objectives, the present invention also proposes a multi-scale flame image detection device, the multi-scale flame image detection device comprising:

[0036] The computation module is used to perform convolution calculations on the initial flame feature image through a grouped multi-scale convolutional network model to obtain the target flame feature image;

[0037] The extraction module is used to iteratively extract the feature image of the target flame based on the target recurrent convolution structure to obtain feature images of different processes;

[0038] The feature fusion module is used to perform bidirectional feature fusion on the feature images of the different processes to obtain the target feature image;

[0039] The detection module is used to detect the target feature image through the target image detection layer to obtain multi-scale flame image detection results.

[0040] Furthermore, to achieve the above objectives, the present invention also proposes a multi-scale flame image detection device, which includes: a memory, a processor, and a multi-scale flame image detection program stored in the memory and executable on the processor. The multi-scale flame image detection program is configured to implement the multi-scale flame image detection method described above.

[0041] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a multi-scale flame image detection program, which, when executed by a processor, implements the multi-scale flame image detection method as described above.

[0042] The multi-scale flame image detection method proposed in this invention performs convolution calculations on an initial flame feature image using a grouped multi-scale convolutional network model to obtain a target flame feature image. Iterative extraction is then performed on the target flame feature image based on the target recurrent convolutional structure to obtain feature images at different stages. These feature images at different stages are then bidirectionally fused to obtain the target feature image. Finally, the target feature image is detected by a target image detection layer to obtain the multi-scale flame image detection result. Because this invention calculates the initial flame feature image through convolution, then iteratively extracts the target flame feature image, then bidirectionally fuses the feature images at different stages, and finally detects the target feature image based on the target image detection layer, it effectively improves the accuracy of flame image detection compared to existing technologies that detect flame images based on static flame characteristics. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the structure of a multi-scale flame image detection device in the hardware operating environment involved in the embodiments of the present invention;

[0044] Figure 2 This is a flowchart illustrating the first embodiment of the multi-scale flame image detection method of the present invention;

[0045] Figure 3 This is a flowchart illustrating the second embodiment of the multi-scale flame image detection method of the present invention;

[0046] Figure 4 This is a flowchart illustrating the third embodiment of the multi-scale flame image detection method of the present invention;

[0047] Figure 5 This is a schematic diagram of the functional modules of the first embodiment of the multi-scale flame image detection device of the present invention.

[0048] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0049] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0050] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a multi-scale flame image detection device in the hardware operating environment of an embodiment of the present invention.

[0051] like Figure 1 As shown, the multi-scale flame image detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the multi-scale flame image detection device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0053] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a multi-scale flame image detection program.

[0054] exist Figure 1In the multi-scale flame image detection device shown, the network interface 1004 is mainly used for data communication with the network integrated platform workstation; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the multi-scale flame image detection device of the present invention can be set in the multi-scale flame image detection device, and the multi-scale flame image detection device calls the multi-scale flame image detection program stored in the memory 1005 through the processor 1001 and executes the multi-scale flame image detection method provided in the embodiment of the present invention.

[0055] Based on the above hardware structure, an embodiment of the multi-scale flame image detection method of the present invention is proposed.

[0056] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the multi-scale flame image detection method of the present invention.

[0057] In the first embodiment, the multi-scale flame image detection method includes the following steps:

[0058] Step S10: The initial flame feature image is convolved using a grouped multi-scale convolutional network model to obtain the target flame feature image.

[0059] It should be noted that the execution subject of this embodiment is a multi-scale flame image detection device, but it can also be other devices that can achieve the same or similar functions, such as flame image detectors. This embodiment does not limit this, and in this embodiment, a flame image detector is used as an example for explanation.

[0060] It should be understood that the grouped multi-scale convolutional network model refers to a network model obtained by improving the backbone network of the YOLOv3 model. This grouped multi-scale convolutional network model uses the LeakyReLU activation function. Specifically, it adds a multi-scale grouped convolutional structure to the backbone network of the YOLOv3 model. Since the multi-scale grouped convolutional structure has parallel convolutional kernels with different scales, the grouped multi-scale convolutional network model can perform multi-scale convolution calculations on the initial flame feature image without missing the key information of small target flames, and pass the target flame feature image to the feature extraction stage.

[0061] It is understandable that the target flame feature image refers to the feature image after the initial flame feature image is calculated by convolution. The initial flame feature image is the feature image of the flame video image that is initially extracted by the initial convolutional network model, which is composed of ordinary convolutional structures.

[0062] Furthermore, before step S10, the method further includes: acquiring a flame video image of the target area; and extracting features from the flame video image of the target area using an initial convolutional network model to obtain an initial flame feature image.

[0063] It should be understood that the flame video image refers to a video image of a suspected fire in the target area. The flame video image can be obtained by real-time monitoring and shooting by a camera device, which can be a front-end high-definition network camera or other network camera devices. This embodiment does not limit this.

[0064] It is understandable that after obtaining the flame video image of the target area, the initial convolutional network model is used to initially extract the features in the flame video image of the target area to obtain the initial flame feature image. The initial convolutional network model includes convolution calculation and activation function calculation in the process of extracting the flame video image.

[0065] Step S20: Iteratively extract the feature image of the target flame according to the target recurrent convolution structure to obtain feature images of different processes.

[0066] It is understandable that the target recurrent convolutional structure refers to the convolutional structure of the feature image through multiple iterations. This target recurrent convolutional structure is set in the grouped multi-scale convolutional network model. By iteratively extracting the target flame feature image once through the target recurrent convolutional structure, a feature image of one process will be generated. Therefore, by extracting through multiple loop iterations, feature images of different processes can be obtained.

[0067] Further, step S20 includes: initially extracting the target flame features based on the target recurrent convolutional structure to obtain the current process feature image and extraction order; iteratively extracting the target flame features according to the extraction order based on the target recurrent convolutional structure to obtain the target process feature image; and adjusting the current process feature image using the target process feature image to obtain feature images for different processes.

[0068] It should be understood that the current process feature image refers to the process feature image extracted for the first time through the target recurrent convolutional structure, and the extraction order refers to the order in which the target flame features are extracted. Since errors may occur during the first feature extraction process, resulting in low accuracy of the feature images obtained for different processes, it is necessary to iteratively extract the target flame features through the target recurrent convolutional structure according to the extraction order to obtain the target process feature image. Then, compare whether the target process feature image is consistent with the current process feature image. If they are consistent, the current process feature image is directly used as the feature image for different processes. If not, the current process feature image is adjusted through the target process feature image to obtain the feature images for different processes.

[0069] Step S30: Perform bidirectional feature fusion on the feature images of the different processes to obtain the target feature image.

[0070] It is understandable that the target feature image refers to the feature image obtained by fusing different feature images. Specifically, it adopts a two-way feature fusion path, namely a bottom-up feature fusion path and a bottom-up feature fusion path. The two-way feature fusion path can enrich high-level semantic information and help detect small flames.

[0071] Further, step S30 includes: obtaining a bottom-up feature fusion path and a top-down feature fusion path; performing feature fusion on the feature images of the different processes according to the bottom-up feature fusion path to obtain a target scale feature image; performing double sampling on the feature images of the different processes according to the top-down feature fusion path to obtain corresponding sample feature images; and performing feature fusion on the target scale feature image and the sample feature image to obtain a target feature image.

[0072] It should be understood that the bottom-up feature fusion path refers to the path of fusing feature images from different processes from bottom to top. Similarly, the top-down feature fusion path refers to the path of fusing feature images from different processes from top to bottom. After fusing feature images from different processes according to the bottom-up feature fusion path, the output target scale feature image has a scale of 52×52. The sample feature image refers to the feature image obtained by sampling feature images from different processes at twice the size. The scale of the sample feature image includes two categories, namely 13×13 and 26×26.

[0073] It is understandable that after obtaining the target scale feature image and the sample feature image, the fusion layer of the target scale feature image and the sample feature image is extracted respectively. Then, the extracted fusion layer is used to fuse features to obtain the target feature image, which can effectively increase the network depth and avoid the problems of gradient vanishing or gradient exploding.

[0074] Step S40: The target feature image is detected by the target image detection layer to obtain the multi-scale flame image detection result.

[0075] It should be understood that the multi-scale flame image detection result refers to the result of detecting the target feature image in a multi-scale manner. This multi-scale flame image detection result is obtained by detecting the target feature image through a target image detection layer, which can be a YOLO layer.

[0076] Extensive experimental data shows that, under the same conditions, the improved YOLOv3 model in this embodiment ranks first in terms of accuracy, recall, false alarm rate, and precision. Compared with the unimproved YOLOv3 model, the improved YOLOv3 model has a significantly higher detection accuracy and a significantly lower false alarm rate. Therefore, the improved YOLOv3 model in this embodiment can effectively improve the performance of flame detection.

[0077] Understandably, referring to Table 1, which contains experimental data on fire image detection using different methods, the details are as follows:

[0078]

[0079] This embodiment uses a grouped multi-scale convolutional network model to perform convolutional calculations on the initial flame feature image to obtain the target flame feature image; iteratively extracts the target flame feature image according to the target recurrent convolutional structure to obtain feature images at different stages; the feature images at different stages are bidirectionally fused to obtain the target feature image; and the target feature image is detected by a target image detection layer to obtain a multi-scale flame image detection result. Because this embodiment calculates the initial flame feature image through convolution, then iteratively extracts the target flame feature image, then bidirectionally fuses the feature images at different stages, and finally detects the target feature image based on the target image detection layer, it can effectively improve the accuracy of flame image detection compared to existing technologies that detect flame images based on static flame characteristics.

[0080] In one embodiment, such as Figure 3 The second embodiment of the multi-scale flame image detection method of the present invention, based on the first embodiment, includes step S10, which includes:

[0081] Step S101: Obtain the target size convolution and the target parallel structure convolution based on the grouped multi-scale convolution model.

[0082] It should be understood that target size convolution refers to the convolution of the initial flame feature image calculated at multiple scales. The target size convolution can be 3×3 with a stride of 2. Target parallel structure convolution refers to the structure convolution of convolution kernels at different scales. The target parallel structure convolution includes four groups, namely 1×1, 3×3, 1×7 and 7×1.

[0083] Step S102: Perform multi-scale calculations on the initial flame feature image using the target size convolution to obtain a multi-scale flame feature image.

[0084] It is understandable that a multi-scale flame feature image refers to a feature image calculated at multiple scales from an initial flame feature image. After obtaining the multi-scale flame feature image, it will be temporarily saved.

[0085] Step S103: Perform convolution calculation on the multi-scale flame feature image through the target parallel structure convolution to obtain the target flame feature image.

[0086] It should be understood that the target flame feature image refers to the feature image calculated by convolution of multi-scale flame feature images. Specifically, the multi-scale flame feature images are subjected to two, three, and two sets of four convolution calculations respectively. That is, the first layer uses convolution calculation, while the last layer uses dilated convolution calculation to obtain the target flame feature image.

[0087] Further, step S103 includes: obtaining convolution kernels of different scales and the number of convolution calculations based on the target parallel structure convolution; selecting the convolution kernels of different scales through a convolution calculation strategy to obtain a set of convolution kernels and a target scale convolution kernel; performing convolution calculations on the multi-scale flame feature image sequentially through the set of convolution kernels according to the number of convolutions to obtain the current flame feature image; and performing convolution calculations on the current flame feature image through the target scale convolution kernel and dilated convolution calculation strategy to obtain the target flame feature image.

[0088] It is understandable that convolution kernels of different scales can be 1×1, 3×3, 1×7, 7×1, and other scales. The number of convolution calculations refers to the number of times the multi-scale flame feature image is calculated. The set of convolution kernels refers to the set of convolution kernels of scales of 1×1, 3×3, 1×7, and 7×1. Then, the multi-scale flame feature image is convolved by the number of convolutions. Finally, the current flame feature image is calculated using a dilated convolution calculation strategy in the last layer. This can effectively broaden the computational scope and extract small flame image information that is easily overlooked in flame video images.

[0089] This embodiment obtains target-size convolution and target parallel structure convolution based on a grouped multi-scale convolution model; it then performs multi-scale calculations on the initial flame feature image using the target-size convolution to obtain a multi-scale flame feature image; finally, it performs convolution calculations on the multi-scale flame feature image using the target parallel structure convolution to obtain the target flame feature image. Because this embodiment obtains target-size convolution and target parallel structure convolution using a grouped multi-scale convolution model, then uses the target-size convolution to calculate the initial flame feature image at multiple scales, and then uses the target parallel structure convolution to calculate the multi-scale flame feature image, it can effectively improve the accuracy of obtaining the target flame feature image and avoid ignoring small flame image information in the flame video image.

[0090] In one embodiment, such as Figure 4The third embodiment of the multi-scale flame image detection method of the present invention, based on the first embodiment, includes step S40, which includes:

[0091] Step S401: Input the target feature image into the target image detection layer so that the target image detection layer can detect the target feature image and obtain and feed back a set of flame region prediction boxes.

[0092] It is understandable that the set of predicted flame regions refers to the set of boxes that predict the specific areas of the flame. This set of predicted flame regions is obtained by the target image detection layer. Since the target feature image has incorporated the feature information of the backbone network at different depths, the target image detection layer can quickly and accurately detect the predicted flame regions, thereby feeding back the corresponding set of predicted flame regions.

[0093] Step S402: Sort the prediction boxes in the flame area prediction box set by score.

[0094] It should be understood that after obtaining the set of predicted fire area boxes, the score of each predicted box is determined, and the predicted boxes are sorted in descending order of score to obtain the corresponding fire area predicted box sorting result.

[0095] Step S403: The flame region prediction box sorting results are filtered according to the non-maximum suppression strategy to obtain the multi-scale flame image detection results.

[0096] It is understandable that the non-maximum suppression strategy refers to the strategy of suppressing elements corresponding to non-maximum values. In other words, the essence of the non-maximum suppression strategy is to perform a local maximum search on the target element set. By using the non-maximum suppression strategy, the non-maximum prediction boxes in the flame region prediction box ranking results are suppressed to obtain multi-scale flame image detection results.

[0097] This embodiment inputs the target feature image into a target image detection layer, enabling the target image detection layer to detect the target feature image and obtain and return a set of predicted flame regions. The predicted flame regions are then ranked by score. The ranked flame regions are then filtered using a non-maximum suppression strategy to obtain multi-scale flame image detection results. Because this embodiment detects the target feature image through the target image detection layer, ranks the predicted flame regions by score, and then uses a non-maximum suppression strategy to filter the ranked flame regions, it effectively improves the accuracy of flame image detection.

[0098] Furthermore, this embodiment of the invention also proposes a storage medium storing a multi-scale flame image detection program, which, when executed by a processor, implements the steps of the multi-scale flame image detection method described above.

[0099] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0100] In addition, refer to Figure 5 This invention also proposes a multi-scale flame image detection device, which includes:

[0101] The calculation module 10 is used to perform convolution calculations on the initial flame feature image through a grouped multi-scale convolutional network model to obtain the target flame feature image.

[0102] The extraction module 20 is used to iteratively extract the feature image of the target flame according to the target recurrent convolution structure to obtain feature images of different processes.

[0103] The feature fusion module 30 is used to perform bidirectional feature fusion on the feature images of the different processes to obtain the target feature image.

[0104] The detection module 40 is used to detect the target feature image through the target image detection layer to obtain multi-scale flame image detection results.

[0105] This embodiment uses a grouped multi-scale convolutional network model to perform convolutional calculations on the initial flame feature image to obtain the target flame feature image; iteratively extracts the target flame feature image according to the target recurrent convolutional structure to obtain feature images at different stages; the feature images at different stages are bidirectionally fused to obtain the target feature image; and the target feature image is detected by a target image detection layer to obtain a multi-scale flame image detection result. Because this embodiment calculates the initial flame feature image through convolution, then iteratively extracts the target flame feature image, then bidirectionally fuses the feature images at different stages, and finally detects the target feature image based on the target image detection layer, it can effectively improve the accuracy of flame image detection compared to existing technologies that detect flame images based on static flame characteristics.

[0106] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0107] In addition, for technical details not described in detail in this embodiment, please refer to the multi-scale flame image detection method provided in any embodiment of the present invention, which will not be repeated here.

[0108] In one embodiment, the calculation module 10 is further configured to acquire a flame video image of the target area; and to extract features from the flame video image of the target area using an initial convolutional network model to obtain an initial flame feature image.

[0109] In one embodiment, the calculation module 10 is further configured to obtain a target size convolution and a target parallel structure convolution based on a grouped multi-scale convolution model; perform multi-scale calculations on the initial flame feature image using the target size convolution to obtain a multi-scale flame feature image; and perform convolution calculations on the multi-scale flame feature image using the target parallel structure convolution to obtain a target flame feature image.

[0110] In one embodiment, the calculation module 10 is further configured to obtain convolution kernels of different scales and the number of convolution calculations based on the target parallel structure convolution; select the convolution kernels of different scales through a convolution calculation strategy to obtain a set of convolution kernels and a target scale convolution kernel; perform convolution calculations on the multi-scale flame feature image sequentially through the set of convolution kernels according to the number of convolutions to obtain the current flame feature image; and perform convolution calculations on the current flame feature image through the target scale convolution kernel and dilated convolution calculation strategy to obtain the target flame feature image.

[0111] In one embodiment, the extraction module 20 is further configured to perform initial extraction of the target flame features based on the target recurrent convolutional structure to obtain a current process feature image and extraction order; iteratively extract the target flame features according to the extraction order based on the target recurrent convolutional structure to obtain a target process feature image; and adjust the current process feature image using the target process feature image to obtain feature images of different processes.

[0112] In one embodiment, the feature fusion module 30 is further configured to acquire a bottom-up feature fusion path and a top-down feature fusion path; perform feature fusion on the feature images of the different processes according to the bottom-up feature fusion path to obtain a target scale feature image; perform double sampling on the feature images of the different processes according to the top-down feature fusion path to obtain corresponding sample feature images; and perform feature fusion on the target scale feature image and the sample feature image to obtain a target feature image.

[0113] In one embodiment, the detection module 40 is further configured to input the target feature image into the target image detection layer, so that the target image detection layer detects the target feature image, obtains and feeds back a set of flame region prediction boxes; sorts the prediction boxes in the set of flame region prediction boxes by score; and filters the sorting results of the flame region prediction boxes according to a non-maximum suppression strategy to obtain multi-scale flame image detection results.

[0114] Other embodiments or implementation methods of the multi-scale flame image detection device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0115] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0116] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, all-in-one platform workstation, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0118] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A multi-scale flame image detection method, characterized in that, The multi-scale flame image detection method includes the following steps: The target flame feature image is obtained by performing convolution calculation on the initial flame feature image using a grouped multi-scale convolutional network model. The target flame feature image is iteratively extracted based on the target recurrent convolution structure to obtain feature images of different processes; The feature images of the different processes are fused bidirectionally to obtain the target feature image; The target feature image is detected by the target image detection layer to obtain multi-scale flame image detection results; The step of performing convolution calculations on the initial flame feature image using a grouped multi-scale convolutional network model to obtain the target flame feature image includes: Based on the grouped multi-scale convolution model, target-size convolution and target parallel structure convolution are obtained; The initial flame feature image is processed by multi-scale calculation through the target size convolution to obtain a multi-scale flame feature image; The target flame feature image is obtained by performing convolution calculation on the multi-scale flame feature image through the target parallel structure convolution; The step of iteratively extracting the feature image of the target flame based on the target recurrent convolutional structure to obtain feature images of different processes includes: The target flame features are initially extracted based on the target recurrent convolutional structure to obtain the current process feature image and extraction order. According to the extraction order, the target flame features are iteratively extracted based on the target recurrent convolution structure to obtain the target process feature image; By adjusting the current process feature image using the target process feature image, feature images of different processes can be obtained.

2. The multi-scale flame image detection method as described in claim 1, characterized in that, Before obtaining the target flame feature image by performing convolution calculations on the initial flame feature image using a grouped multi-size convolutional network model, the process also includes: Acquire video images of the flames in the target area; The initial flame feature image is obtained by extracting features from the flame video image of the target area using an initial convolutional network model.

3. The multi-scale flame image detection method as described in claim 1, characterized in that, The step of performing convolution calculations on the multi-scale flame feature image through the target parallel structure convolution to obtain the target flame feature image includes: Based on the target parallel structure convolution, convolution kernels of different sizes and the number of convolution calculations are obtained; By selecting convolution kernels of different scales through convolution calculation strategies, a set of convolution kernels and a target scale convolution kernel are obtained. The multi-scale flame feature image is convolved sequentially using the set of convolution kernels according to the number of convolutions to obtain the current flame feature image. The target flame feature image is obtained by performing convolution calculation on the current flame feature image using the target scale convolution kernel and dilated convolution calculation strategy.

4. The multi-scale flame image detection method as described in claim 1, characterized in that, The step of performing bidirectional feature fusion on the feature images of the different processes to obtain the target feature image includes: Obtain bottom-up and top-down feature fusion paths; The feature images of the different processes are fused according to a bottom-up feature fusion path to obtain a target-scale feature image; The feature images of the different processes are sampled twice according to the top-down feature fusion path to obtain the corresponding sample feature images; The target scale feature image is obtained by fusing the target feature image with the sample feature image.

5. The multi-scale flame image detection method according to any one of claims 1 to 4, characterized in that, The step of detecting the target feature image through the target image detection layer to obtain multi-scale flame image detection results includes: The target feature image is input into the target image detection layer so that the target image detection layer can detect the target feature image and obtain and feed back a set of flame region prediction boxes; The prediction boxes of the flame area prediction box set are sorted by score; The flame region prediction box ranking results are filtered according to the non-maximum suppression strategy to obtain multi-scale flame image detection results.

6. A multi-scale flame image detection device, characterized in that, The multi-scale flame image detection device includes: The computation module is used to perform convolution calculations on the initial flame feature image through a grouped multi-scale convolutional network model to obtain the target flame feature image; The extraction module is used to iteratively extract the feature image of the target flame based on the target recurrent convolution structure to obtain feature images of different processes; The feature fusion module is used to perform bidirectional feature fusion on the feature images of the different processes to obtain the target feature image; The detection module is used to detect the target feature image through the target image detection layer to obtain multi-scale flame image detection results; The step of performing convolution calculations on the initial flame feature image using a grouped multi-scale convolutional network model to obtain the target flame feature image includes: Based on the grouped multi-scale convolution model, target-size convolution and target parallel structure convolution are obtained; The initial flame feature image is processed by multi-scale calculation through the target size convolution to obtain a multi-scale flame feature image; The target flame feature image is obtained by performing convolution calculation on the multi-scale flame feature image through the target parallel structure convolution; The step of iteratively extracting the feature image of the target flame based on the target recurrent convolutional structure to obtain feature images of different processes includes: The target flame features are initially extracted based on the target recurrent convolutional structure to obtain the current process feature image and extraction order. According to the extraction order, the target flame features are iteratively extracted based on the target recurrent convolution structure to obtain the target process feature image; By adjusting the current process feature image using the target process feature image, feature images of different processes can be obtained.

7. A multi-scale flame image detection device, characterized in that, The multi-scale flame image detection device includes: a memory, a processor, and a multi-scale flame image detection program stored in the memory and executable on the processor, wherein the multi-scale flame image detection program is configured to implement the multi-scale flame image detection method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores a multi-scale flame image detection program, which, when executed by a processor, implements the multi-scale flame image detection method as described in any one of claims 1 to 5.

Images