Fire detection method, electronic device and storage medium

Abstract

This application proposes a method, electronic device, and storage medium for detecting fireworks, which belong to the field of image processing. The method includes: inputting an image to be tested into a personalized detection model to obtain a segmented image output by the personalized detection model, where the personalized detection model is obtained by training a pre-trained fireworks detection model using user-specific data; performing contour detection and instance separation on the segmented areas of the segmented image to obtain valid instances; and, if a valid instance exists, issuing a fireworks alarm based on the valid instance categorized as smoke and / or fire. In this way, a personalized detection model is customized for the user to match the user-specific data for the desired scenario, and fireworks are identified using the personalized detection model based on the user's desired data, greatly improving detection precision and accuracy in different scenarios. Furthermore, fireworks detection can be performed in real time based on images captured by a drone, greatly improving response speed.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for detecting fireworks, an electronic device, and a storage medium. Background Art

[0002] With the acceleration of urbanization and the continuous improvement of the ecological environment, the risk of fire accidents is also increasing. Therefore, smoke and fire detection for fire prevention and safety management is extremely important in safety management in modern society.

[0003] Common methods for detecting fireworks include: (1) identifying fireworks through temperature measurement; (2) using drones equipped with sensors to capture images and detect fireworks. Method (1) requires the temperature sensor to be relatively close to the fire area, which to some extent affects its response speed and makes it difficult to respond in a timely manner. The detection accuracy of method (2) is difficult to meet the needs of different users. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide a smoke and fire detection method, electronic device and storage medium, which not only improve the response speed of smoke and fire detection, but also improve the detection accuracy of smoke and fire detection to meet the needs of different users.

[0005] In order to achieve the above objectives, the technical solutions adopted in this application are as follows:

[0006] In a first aspect, the present application provides a method for detecting fireworks, the method comprising:

[0007] Inputting the image to be tested into the personalized detection model to obtain a segmented image output by the personalized detection model; wherein the segmented image includes at least one segmented region and a category of the segmented region, and the personalized detection model is obtained by training a pre-trained fireworks detection model using user-specific data;

[0008] Performing contour detection and instance separation on the segmented areas of the segmented image to obtain valid instances;

[0009] In the case that a valid instance exists, a fire alarm is issued according to the valid instance.

[0010] In an optional embodiment, the step of performing contour detection and instance separation on the segmented areas of the segmented image to obtain valid instances includes:

[0011] Smoothing the segmented areas of the segmented image that are classified as smoke or fire to obtain a smoothed image, wherein the smoothed image includes at least one smooth connected area; filtering each of the smooth connected areas in the smoothed image to obtain a corrected image, wherein the corrected image includes at least one valid area;

[0012] For each of the valid areas, contour detection and instance separation are performed on the valid area to obtain a valid instance.

[0013] In an optional embodiment, the step of filtering each of the smooth connected areas in the smooth image to obtain a corrected image includes:

[0014] For each of the smooth connected areas, obtaining an area ratio according to a ratio of an area of ​​the smooth connected area to a total area of ​​the smooth image;

[0015] The smooth connected areas in the smooth image whose area ratio is less than a ratio threshold are filtered out, and the remaining smooth connected areas are used as valid areas to obtain a corrected image.

[0016] In an optional embodiment, the step of smoothing the segmented area of ​​the segmented image that is classified as smoke or fire to obtain a smoothed image includes:

[0017] The segmented areas in the segmented image that belong to the same target category are grouped as a same category; wherein the target category includes smoke and fire;

[0018] An opening operation is performed on each segmented area of ​​the same category group in the segmented image, and then a kernel function of an elliptical kernel is used to process the segmented area to obtain a smooth connected area.

[0019] In an optional embodiment, the step of obtaining the personalized detection model includes:

[0020] Use various common samples to iteratively train the initial model to obtain a fireworks detection model;

[0021] Acquire a user-specific data set; wherein the user-specific data includes a plurality of user-labeled special samples;

[0022] The fireworks detection model is adjusted and trained using the multiple dedicated samples to obtain a personalized detection model.

[0023] In an optional embodiment, the step of using the multiple dedicated samples to adjust and train the fireworks detection model to obtain a personalized detection model includes:

[0024] According to user indicators, the number of layers of the segmentation head of the fireworks detection model is adjusted to obtain a preliminary adjustment detection model;

[0025] In a case where the feature extraction layer of the preliminary adjustment detection model is frozen, the preliminary adjustment detection model is iteratively trained using the dedicated samples to obtain an early result model;

[0026] After unfreezing the feature extraction layer of the previous result model and lowering the learning rate, the previous result model is iteratively trained using the dedicated samples to obtain a personalized detection model.

[0027] In an optional embodiment, the step of iteratively training the initial model using various common samples to obtain a smoke and fire detection model includes:

[0028] Segment and label the fireworks for each general image in the general dataset to obtain general samples;

[0029] Randomly cropping and resizing each of the general samples to obtain a general sample set;

[0030] An SPD module and a pkinet module are added to the YOLOv11 neural network to obtain an initial model; wherein the SPD module is used to convert the spatial dimension of the input feature map into the depth dimension, and the pkinet module is used to perform multi-scale convolution and depth-wise separable convolution on the input feature map;

[0031] The initial model is iteratively trained using the universal sample set to obtain a fireworks detection model.

[0032] In an optional embodiment, the step of issuing a fire alarm according to the effective example includes:

[0033] Obtaining the location range of the valid instance according to the coverage area corresponding to the image to be tested;

[0034] An alarm message is issued based on the position range.

[0035] In a second aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the smoke and fire detection method as described in any one of the aforementioned embodiments.

[0036] In a third aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the smoke and fire detection method as described in any one of the aforementioned embodiments.

[0037] In a fourth aspect, the present application provides a smoke and fire detection device, comprising a detection module, a processing module, and an alarm module;

[0038] The detection module is configured to input a test image into a personalized detection model to obtain a segmented image output by the personalized detection model; wherein the segmented image includes at least one segmented region and a segmented region category; and the personalized detection model is obtained by training a pre-trained fireworks detection model using user-specific data;

[0039] The processing module is used to perform contour detection and instance separation on the segmented areas of the segmented image to obtain valid instances;

[0040] The alarm module is used to issue a fire alarm according to a valid instance when there is one.

[0041] The present application provides a fireworks and fire detection method, electronic device, and storage medium. The method comprises: inputting an image to be tested into a personalized detection model to obtain a segmented image output by the personalized detection model, wherein the segmented image includes at least one segmented region and its category. The personalized detection model is obtained by training a pre-trained fireworks and fire detection model using user-specific data; performing contour detection and instance separation on the segmented regions of the segmented image to obtain valid instances; and, if a valid instance exists, issuing a fireworks and fire alarm based on the valid instance being classified as smoke and / or fire. In this way, a personalized detection model is customized for the user to match the user-specific data for the desired scenario. Fireworks and fire identification is performed using the personalized detection model based on the user's desired data, significantly improving detection precision and accuracy in different scenarios. Furthermore, fireworks and fire detection can be performed in real time based on images captured by a drone, significantly improving response speed.

[0042] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A schematic diagram of the system architecture of a smoke and fire detection system provided in an embodiment of the present application is shown.

[0045] Figure 2 A schematic diagram of the module architecture of an electronic device provided in an embodiment of the present application is shown.

[0046] Figure 3 One of the flow charts of the smoke and fire detection method provided in the embodiment of the present application is shown.

[0047] Figure 4 The second flowchart of the smoke and fire detection method provided in the embodiment of the present application is shown.

[0048] Figure 5 Shown Figure 4 Flow chart of some sub-steps of step 21.

[0049] Figure 6 Shown Figure 4 Flowchart of some sub-steps of step 25.

[0050] Figure 7 Shown Figure 3 Flowchart of some sub-steps of step 13 in FIG.

[0051] Figure 8 Shown Figure 7 Schematic diagram of the flow of some sub-steps of step 131.

[0052] Figure 9 Shown Figure 7 Flow chart of some sub-steps of step 133.

[0053] Figure 10 Shown Figure 3 Flowchart of some sub-steps of step 15.

[0054] Icons: 10-firework detection system; 110-training equipment; 120-client; 130-drone; 140-detection equipment; 150-early warning equipment; 20-electronic equipment; 210-memory; 220-processor; 230-communication module. DETAILED DESCRIPTION

[0055] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Generally, the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present application.

[0057] It should be noted that relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0058] The smoke and fire detection method provided in the embodiment of the present application can be applied to Figure 1 In the smoke and fire detection system 10 shown, referring to Figure 1 The fire and smoke detection system 10 includes a training device 110, a client 120, a drone 130, a detection device 140 and an early warning device 150. The detection device 140 can be communicated with the drone 130 and the early warning device 150 respectively through a network, wireless connection, etc. The training device 110 can be communicated with the detection device 140 through wired or wireless means.

[0059] The user inputs user-specific data into the training device 110 via the client 120 .

[0060] The training device 110 is used to train a fire and smoke detection model using general data, and then adjust and train the fire and smoke detection model using user-specific data to obtain a personalized detection model, and then download and deploy the personalized detection model to the drone 130.

[0061] The drone 130 is used to collect images of the target area, obtain images to be tested, and transmit the images to be tested to the detection device 140 in real time.

[0062] Detection device 140 is configured to implement the smoke and fire detection method provided in an embodiment of the present application, including: inputting a test image into a personalized detection model to obtain a segmented image output by the personalized detection model, the segmented image including at least one segmented region and its category. The personalized detection model is obtained by training a pre-trained smoke and fire detection model using user-specific data; performing contour detection and instance separation on the segmented regions of the segmented image to obtain valid instances; and, if a valid instance exists, issuing a smoke and fire alarm based on the valid instance being classified as smoke and / or fire. The smoke and fire alarm can also send a warning message to warning device 150.

[0063] The early warning device 150 is used to issue a fire alarm upon receiving early warning information.

[0064] The detection device 140 may be, but is not limited to, a cloud platform, a standalone server, a server cluster, a personal computer, a laptop computer, etc. The warning device 150 may be, but is not limited to, a personal computer, a laptop computer, a mobile phone, a mobile terminal, an audible and visual alarm, or any other warning device 150 that can receive information and issue an alarm.

[0065] Please refer to Figure 2 , is a block diagram of an electronic device 20, which may be Figure 1 The detection device 140 in the smoke and fire detection system 10 is shown. The electronic device 20 includes a memory 210, a processor 220, and a communication module 230. The memory 210, processor 220, and communication module 230 are electrically connected to each other, directly or indirectly, to enable data transmission or interaction. For example, these components can be electrically connected via one or more communication buses or signal lines.

[0066] The memory 210 is used to store programs or data and can be, but is not limited to, a random access memory, a read-only memory, a programmable read-only memory, an erasable read-only memory, an electrically erasable read-only memory, etc.

[0067] The processor 220 is used to read / write data or programs stored in the memory 210 and execute corresponding functions. For example, Figure 1 In the illustrated smoke and fire detection system 10 , the processor 220 of the detection device 140 executes the computer program stored in the memory 210 to implement the smoke and fire detection method provided in the embodiment of the present application.

[0068] The communication module 230 is used to establish a communication connection between the electronic device 20 and other communication terminals through the network, and to send and receive data through the network. Figure 1 In the smoke and fire detection system 10 shown, the classmate module of the persistence device sends and receives data to and from the drone 130 and the early warning device 150 through the network.

[0069] It should be understood that Figure 2 The structure shown is only a schematic diagram of the server structure, and the electronic device 20 may also include Figure 2 More or fewer components than shown, or with Figure 2 Different configurations shown. Figure 2 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0070] In order to solve the problems of slow response speed and difficulty in meeting the needs of different users in common fire and smoke detection, the present invention provides a fire and smoke detection method. Figure 3, including steps 11 to 15. And, Figure 1 The detection device 140 in the smoke and fire detection system 10 shown can Figure 2 The structure shown implements the execution of steps 11 to 15 when the processor 220 reads the computer program stored in the memory 210 .

[0071] Step 11: Input the image to be tested into the personalized detection model to obtain a segmented image output by the personalized detection model.

[0072] The segmented image includes at least one segmented area and a category of the segmented area, and the personalized detection model is obtained by training a pre-trained fireworks detection model using user-specific data.

[0073] Step 13: Perform contour detection and instance separation on the segmented areas of the segmented image to obtain valid instances.

[0074] Step 15: If a valid instance exists, a fire alarm is issued according to the valid instance.

[0075] For example, in combination Figure 1 In the illustrated fireworks detection system 10, a target user inputs user-specific data into a training device 110 via a client 120. The training device 110 uses the user-specific data to adjust and train a pre-trained fireworks detection model, generating a personalized detection model for the target user. The training device 110 then downloads and deploys the personalized detection model to a detection device 140. A drone 130 regularly inspects the target area specified by the target user, capturing images in real time according to the inspection cycle specified by the target user. The drone then transmits the images to the detection device 140 in real time.

[0076] Detection device 140 invokes the target user's personalized detection model and inputs the image to be tested into it, generating a segmented image. Detection device 140 then performs contour detection and instance separation on the segmented regions of the segmented image to obtain valid instances. If a valid instance exists, it sends a fire and smoke warning message to warning device 150 based on the valid instance. Upon receiving the fire and smoke warning message, warning device 150 can perform any of the following actions: display the warning message, initiate an audible or visual alarm, or perform other alarming actions.

[0077] In steps 11 through 15 of the aforementioned fireworks detection method, a personalized detection model is customized to match the user's specific data for the desired scenario. This personalized detection model, representing the user's desired data, is used to identify fireworks, significantly improving detection precision and accuracy in various scenarios. Furthermore, fireworks detection can be performed in real time based on drone-captured images, significantly improving response speed.

[0078] Among them, there are many ways to implement the personalized detection model used in obtaining step 11. For example, the model can be trained directly using the user's dedicated data, or the pre-trained model can be fine-tuned using the user's dedicated data. The implementation method is not limited.

[0079] In one example, in order to provide a model that reflects the user's expected detection precision and accuracy on the user's data while reducing the time spent on model training, the fireworks detection method provided in the embodiment of the present application introduces the concept of first training a general fireworks detection model using general data, and then fine-tuning the fireworks detection model based on the dedicated data of each user to obtain a personalized detection model.

[0080] In this example, refer to Figure 4 , the embodiment of the present application obtains a personalized detection model through steps 21 to 25.

[0081] Step 21: Use the common samples to iteratively train the initial model to obtain a fireworks detection model.

[0082] Step 23: Obtain a user-specific data set.

[0083] The user-specific data includes multiple special samples marked by the user.

[0084] In step 25 , the fireworks detection model is adjusted and trained using multiple dedicated samples to obtain a personalized detection model.

[0085] There are various ways to obtain the general samples used in step 21 above. For example, fireworks images can be captured from online data, segmented and labeled based on the smoke and fire areas, and thus obtain general samples. Alternatively, fireworks data on existing platforms and the internet can be filtered, selecting fireworks images taken from a drone's perspective. These images can then be screened based on image clarity and other metrics. Furthermore, each of these screened fireworks images can be segmented and labeled based on the smoke and fire areas in the images to obtain the required number of general samples. The implementation method is not limited.

[0086] Furthermore, there are various ways to iteratively train the initial model to obtain a fireworks detection model. For example, a fireworks detection model can be obtained by iteratively training any YOLO-based neural network using a common sample. Alternatively, a fireworks detection model can be obtained by iteratively training a Transformer-based model using a common sample. The above methods are merely examples, and the implementation is not limited.

[0087] In one example, in order to obtain a fire and smoke detection model with good recognition effect in more scenes, the SPD module and the pkinet module are added to the YOLOv11 neural network in step 11 as the initial model, and after random cropping and resizing of common samples, the initial model is iteratively trained to obtain the concept of the fire and smoke detection model. Figure 5 In step 21, the following sub-steps 211 to 217 are used to iteratively train the initial model using various general samples to obtain a fireworks detection model.

[0088] Step 211 : Segment and label fireworks on each general image in the general data set to obtain general samples.

[0089] Step 213 : randomly crop and resize each universal sample to obtain a universal sample set.

[0090] Step 215: Add the SPD module and the pkinet module to the YOLOv11 neural network to obtain an initial model.

[0091] Step 217 : Use the general sample set to iteratively train the initial model to obtain a fireworks detection model.

[0092] The SPD module converts the spatial dimensions of the input feature map into the depth dimension, enhancing the network's learning of low-resolution images and small objects. The pkinet module performs multi-scale convolution and depthwise separable convolution on the input feature map to enhance multi-scale changes at the network level, enabling the YOLOv11 neural network to capture and integrate features at multiple scales.

[0093] In step 213, a universal sample is extracted from each universal sample using any preset random extraction method, and is cropped and / or resized to obtain a new universal sample. The new universal sample and the previous universal sample are both put into the universal sample set.

[0094] In step 215, the SPD module and the pkinet module can be inserted into any position of the feature extraction layer of the YOLOv11 neural network, and the number of SPD modules and pkinet modules can be 1, 2, 2 and 3 respectively, and their specific positions are not limited. In the iterative training process in step 217, the model can be optimized using a gradient descent method, an accelerated gradient method, a Newton method, or any other optimization algorithm.

[0095] The above steps 211 to 217 obtain a universal fireworks detection model that is adaptable to data of different scales and has good learning and reasoning capabilities for high- and low-resolution images and small object targets, thereby ensuring the recognition effect of the fireworks detection model in various scenarios.

[0096] In other embodiments, the YOLOv11 neural network used in steps 211 to 217 may be replaced by any other convolutional neural network.

[0097] After obtaining the fireworks detection model, there are multiple ways to adjust and train the fireworks detection model using multiple dedicated samples in step 25 to obtain a personalized detection model.

[0098] In one example, a plurality of dedicated samples and any optimization method are used to perform overall iterative training on the fire and smoke detection model to obtain a personalized detection model.

[0099] In another example, in order to meet the detection accuracy and speed required by users and improve the convergence speed of the model, in step 25, the idea of ​​adjusting only the number of layers of the segmentation head of the smoke and fire detection model according to customer needs is introduced, and the feature extraction layer is frozen in the early stage of training and unfrozen in the later stage of training for joint fine-tuning. Figure 6 In step 25 , a plurality of special samples are used to adjust and train the fireworks detection model. The process of obtaining a personalized detection model includes steps 251 to 255 .

[0100] Step 251 : According to user indicators, the number of layers of the segmentation head of the fireworks detection model is adjusted to obtain a preliminary adjustment detection model.

[0101] In step 253 , while freezing the feature extraction layer of the preliminary detection model, the preliminary detection model is iteratively trained using dedicated samples to obtain an early result model.

[0102] In step 255 , after unfreezing the feature extraction layer of the previous result model and lowering the learning rate, the previous result model is iteratively trained using dedicated samples to obtain a personalized detection model.

[0103] User indicators can include any of several model performance indicators, such as detection accuracy and detection speed. In step 251, the user indicators, the fireworks detection model, and its current performance indicators can be sent to the technician's terminal display screen. Based on these indicators, the technician can adjust the number of layers of the segmentation head of the fireworks detection model, such as lowering or increasing them, to obtain a preliminary detection model.

[0104] Alternatively, a pre-set adjustment rule table may be provided, which records the correspondence between the difference between each indicator and the number of layers adjusted. Thus, in step 251, the training device calculates the difference between the user's required detection accuracy and the detection accuracy of the fireworks detection model, queries the pre-set adjustment rule table based on this difference, determines the number of layers to be adjusted, and makes adjustments based on the number of layers adjusted. The same applies to other indicators and will not be further elaborated here.

[0105] In step 253 and step 255, it can also be understood that step 253 is early training and step 255 is late training.

[0106] The conditions for terminating the initial iterative training can be flexibly set. For example, it can be when the number of model iterations reaches a set iteration threshold, when the model loss value reaches a set loss threshold, or when the model loss value reaches stability (i.e., model convergence). The above examples are examples, and the conditions for terminating the initial iterative training are not limited.

[0107] Similar to the conditions for ending early iterative training, the conditions for ending late iterative training can be when the number of model iterations reaches a set iteration threshold, when the model loss reaches a set loss threshold, or when the model loss reaches stability (i.e., model convergence). There are no restrictions on how this can be achieved. Furthermore, during late training, the reduced learning rate is equal to the original learning rate minus the reduced value. The reduced value can be 10 or 20, and there are no restrictions on its value.

[0108] During the early training and the late training, the optimization algorithm used can be gradient descent, method, accelerated gradient method, Newton's method or any other optimization algorithm, which is not limited here.

[0109] Through the above method, the fireworks detection model is fine-tuned into a personalized detection model based on user demand indicators. At the same time, by using user-specific samples to fine-tune the model, the precision, speed and accuracy of the final personalized detection model on user scenario data are improved.

[0110] In step 11, the personalized detection model obtained in the above manner is used to obtain a segmented image of the image to be tested. The personalized detection model segments the image to be tested into a segmented image (also understood as a segmentation mask) comprising multiple segments, and obtains a category for each segmented area, including categories such as smoke, fire, building, and tree.

[0111] Then, in step 13, contour detection and instance separation are performed on the segmented areas of the segmented image, and there are many ways to achieve this, to obtain valid instances.

[0112] In one example, contour detection can be performed directly on each segment to obtain a contour, and then a unique code is assigned to each contour as an instance to obtain a valid example.

[0113] In another example, in order to reduce noise interference and improve the accuracy of valid instances, the idea of ​​smoothing and filtering first and then performing contour detection and instance separation is introduced in step 13. Figure 7In step 13, contour detection and instance separation are performed on the segmented areas of the segmented image to obtain valid instances, including steps 131 to 135.

[0114] Step 131 : Smoothing the segmented areas classified as smoke or fire in the segmented image to obtain a smoothed image.

[0115] The smooth graph includes at least one smooth connected area.

[0116] Step 133: Filter each smooth connected area in the smooth image to obtain a corrected image.

[0117] The corrected image includes at least one valid area.

[0118] Step 135 : For each valid area, perform contour detection and instance separation on the valid area to obtain a valid instance.

[0119] In step 131 , morphological operation, mean filtering, Gaussian filtering or any other smoothing method may be used, and the implementation method thereof is not limited.

[0120] In one example, in order to improve the accuracy of subsequent fire alarms and avoid repeated alarms for the same fire area, the idea of ​​performing opening operations and kernel function processing on multiple segments belonging to the same category is introduced in step 131. Figure 8 Step 131 smoothes the segmented areas classified as smoke or fire in the segmented image, and the process of obtaining the smoothed image includes steps 1311 to 1313.

[0121] In step 1311 , the segmented regions in the segmented image that belong to the same target category are grouped into the same category.

[0122] In step 1313 , an opening operation is performed on each segmented area of ​​the same category group in the segmented image, and then a kernel function of an elliptical kernel is used to process the segmented area to obtain a smooth connected area.

[0123] Target categories include smoke and fire.

[0124] Both opening and kernel functions are processing methods used in morphological algorithms. Opening involves an erosion-then-dilution process, which removes small objects, smooths boundaries, and reduces noise. It also connects adjacent segments (i.e., those with minimal correlation distances and belonging to the same firework) to form a single connected segment. After opening, adjacent segments belonging to the same category group are merged into a new segment. Furthermore, an elliptical kernel function smoothes and directs each segment after opening, generating a corresponding smoothed connected segment. Furthermore, the smoothed connected segment inherits the category of the original segment.

[0125] Through steps 1311 to 1313, the segments belonging to the same smoke or fire area are merged and connected, effectively removing small objects and noise from the segments and smoothing them, making each smoothed connected area more accurate. This helps avoid subsequent repeated alarms for the same smoke or fire area, improving the accuracy of subsequent smoke and fire alarms.

[0126] After obtaining a smoothed image using the above method in step 131, the method for filtering each smoothed connected area in the smoothed image in step 133 to obtain a corrected image can be flexibly set. For example, the smoothed image can be input into a pre-trained smoothing model, and the smoothing model can be used to filter the image to obtain the corrected image. Alternatively, filtering can be performed according to preset rules. The implementation method is not limited.

[0127] In one example, referring to Figure 9 Step 133 filters each smooth connected area in the smooth image, and the process of obtaining the corrected image includes steps 1331 to 1333.

[0128] Step 1331 : For each smooth connected area, obtain an area ratio according to the ratio of the area of ​​the smooth connected area to the total area of ​​the smoothed image.

[0129] In step 1333 , the smooth connected areas in the smoothed image whose area ratio is less than the ratio threshold are filtered out, and the remaining smooth connected areas are used as valid areas to obtain a corrected image.

[0130] In the above process, the total area of ​​the smoothed image can be equal to the total number of pixels in the smoothed image, and the area of ​​the smoothed connected area can be equal to the total number of pixels in the smoothed connected area. The total area of ​​the smoothed image and the area of ​​the smoothed connected area can also be calculated using any area calculation method, and are not limited here.

[0131] The ratio threshold can be 5% or 4%, and its value can vary with the area of ​​the real area corresponding to the image to be tested, without limitation. The ratio threshold can also be understood as a fire safety line or a false connection area. When the area ratio of the smooth connection area is less than the ratio threshold, it means that the smooth connection area is too small, and even smoke or fire will not cause a fire, or it means that it is false or noise.

[0132] In this way, small smoothly connected areas that are below the fireworks safety line or that may be false or noisy are filtered to improve detection precision and accuracy.

[0133] After filtering using the above method, all unfiltered smooth connected areas are considered valid areas. Furthermore, in step 135, for each valid area, contour detection is performed on the cell using Canny edge detection, boundary extraction, polygon fitting, or any other contour detection method to obtain a contour. Each contour is assigned a unique code (such as a color or ID) to obtain a valid instance. That is, a contour assigned a unique code is a valid instance.

[0134] If, after processing steps 11 through 13, a valid instance (classified as smoke or fire) is found, it can be determined that smoke or fire is present in the coverage area of ​​the image being tested, indicating a possible fire. In this case, in step 15, a warning message indicating a fire hazard in the coverage area can be generated and sent to the early warning device for timely alarm.

[0135] In one example, in order to make the alarm more accurate, the concept of making the alarm based on the location range of the valid instance is introduced in step 15. Figure 10 The process of issuing a fire alarm in step 15 includes steps 151 to 153.

[0136] Step 151: Obtain the location range of the valid instance according to the coverage area corresponding to the image to be tested.

[0137] Step 153: Send out an alarm message based on the location range.

[0138] The coordinates of the valid instance are converted between the pixel coordinate system, the camera coordinate system and the world coordinate system to obtain the position range of the valid instance.

[0139] By using the above method, the actual location information of the area where smoke or fire exists is obtained, and an alarm is issued accordingly, thereby improving the accuracy of the alarm.

[0140] Based on the same concept as the above-mentioned smoke and fire detection method, an embodiment of the present application further provides a smoke and fire detection device, including a detection module, a processing module and an alarm module.

[0141] The detection module is configured to input the image to be tested into a personalized detection model and generate a segmented image output by the personalized detection model. The segmented image includes at least one segmented region and its category. The personalized detection model is obtained by training a pre-trained fireworks detection model using user-specific data.

[0142] The processing module is used to perform contour detection and instance separation on the segmented areas of the segmented image to obtain valid instances.

[0143] The alarm module is used to issue a fire alarm according to the valid instance when there is a valid instance.

[0144] The above-mentioned fireworks detection device also includes a model training model, which is used to: use various general samples to iteratively train the initial model to obtain a fireworks detection model; obtain a user-specific data set, the user-specific data includes multiple special samples marked by the user; use multiple special samples to adjust and train the fireworks detection model to obtain a personalized detection model.

[0145] The aforementioned fireworks and fire detection device, through the collaborative work of its detection, processing, and alarm modules, customizes a personalized detection model that matches the user's specific data for the desired scenario. This personalized detection model, based on the user's desired data representation, is used to identify fireworks and fire, significantly improving detection precision and accuracy in various scenarios. Furthermore, the device can perform real-time fireworks and fire detection based on drone-captured images, significantly improving response speed.

[0146] For the specific implementation and effect of the smoke and fire detection device, please refer to the description of the implementation of the smoke and fire detection method above. For example, for the specific implementation and effect of the detection module, please refer to the description of the relevant content of step 11 above. For the specific implementation and effect of the processing module, please refer to the description of the relevant content of step 13 above. For the specific implementation and effect of the alarm module, please refer to the description of the relevant content of step 15 above. No further details will be given here.

[0147] Furthermore, each module of the aforementioned fire and smoke detection device may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of the processor 220 in the electronic device 20 in hardware form, or may be stored in the memory 210 of the electronic device 20 in software form, so that the processor 220 can call and execute the corresponding operations of each module to implement the fire and smoke detection method provided above.

[0148] An embodiment of the present application also provides an electronic device 20, including a processor 220 and a memory 210. The memory 210 stores a computer program executable by the processor 220. The processor 220 can execute the computer program to implement the smoke and fire detection method provided above.

[0149] The embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by the processor 220 , the smoke and fire detection method proposed in the embodiment of the present application is implemented.

[0150] In summary, the smoke and fire detection method, electronic device, and storage medium provided in the embodiments of the present application have at least the following beneficial effects:

[0151] (1) Using a user-customized personalized detection model to detect the presence of fireworks, improving the detection accuracy and speed on user data;

[0152] (2) It can be applied to the image processing stage of adaptive power inspection, which can effectively meet the needs of users and realize autonomous power inspection in the process of training models;

[0153] (3) The personalized detection model incorporates data scale transformation, SPD module, and pkinet module to improve the detection effect of multi-scale and small target objects;

[0154] (4) It can be applied to subsequent fireworks recognition and any related thermal scenarios, solving the problem that users have less actual data but want to obtain better results.

[0155] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0156] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0157] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0158] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for detecting fireworks, characterized in that: The method comprises: Inputting the image to be tested into the personalized detection model to obtain a segmented image output by the personalized detection model; wherein the segmented image includes at least one segmented region and a category of the segmented region, and the personalized detection model is obtained by training a pre-trained fireworks detection model using user-specific data; Performing contour detection and instance separation on the segmented areas of the segmented image to obtain valid instances; In the case where a valid instance exists, issuing a fire alarm according to the valid instance; The step of performing contour detection and instance separation on the segmented areas of the segmented image to obtain valid instances includes: Smoothing the segmented areas of the segmented image that are classified as smoke or fire to obtain a smoothed image, wherein the smoothed image includes at least one smooth connected area; filtering each of the smooth connected areas in the smoothed image to obtain a corrected image, wherein the corrected image includes at least one valid area; performing contour detection and instance separation on each valid area to obtain a valid instance; The step of filtering the smooth connected areas in the smooth image to obtain a corrected image includes: For each of the smooth connected areas, an area ratio is obtained based on a ratio of the area of ​​the smooth connected area to the total area of ​​the smooth image; smooth connected areas in the smooth image having an area ratio less than a ratio threshold are filtered out, and the remaining smooth connected areas are used as valid areas to obtain a corrected image; The step of obtaining the personalized detection model includes: Iteratively train the initial model using the general samples to obtain a fire and smoke detection model; obtain a user-specific data set; wherein the user-specific data includes multiple user-labeled specific samples; and adjust and train the fire and smoke detection model using the multiple specific samples to obtain a personalized detection model. The step of using the multiple dedicated samples to adjust and train the fireworks detection model to obtain a personalized detection model includes: According to user indicators, the number of layers of the segmentation head of the fireworks detection model is adjusted to obtain a preliminary detection model; while freezing the feature extraction layer of the preliminary detection model, the preliminary detection model is iteratively trained using the special samples to obtain a preliminary result model; while unfreezing the feature extraction layer of the preliminary result model and lowering the learning rate, the preliminary result model is iteratively trained using the special samples to obtain a personalized detection model.

2. The method for detecting smoke and fire according to claim 1, wherein: The step of smoothing the segmented area of ​​the segmented image that is classified as smoke or fire to obtain a smoothed image includes: The segmented areas in the segmented image that belong to the same target category are grouped as a same category; wherein the target category includes smoke and fire; An opening operation is performed on each segmented area of ​​the same category group in the segmented image, and then a kernel function of an elliptical kernel is used to process the segmented area to obtain a smooth connected area.

3. The method for detecting smoke and fire according to claim 1, wherein: The step of iteratively training the initial model using the general samples to obtain the fireworks detection model includes: Segment and label the fireworks for each general image in the general dataset to obtain general samples; Randomly cropping and resizing each of the general samples to obtain a general sample set; An SPD module and a pkinet module are added to the YOLOv11 neural network to obtain an initial model; wherein the SPD module is used to convert the spatial dimension of the input feature map into the depth dimension, and the pkinet module is used to perform multi-scale convolution and depth-wise separable convolution on the input feature map; The initial model is iteratively trained using the universal sample set to obtain a fireworks detection model.

4. The method for detecting smoke and fire according to claim 1 or 2, wherein: The step of issuing a fire alarm according to the effective example includes: Obtaining the location range of the valid instance according to the coverage area corresponding to the image to be tested; An alarm message is issued based on the position range.

5. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program executable by the processor, and the processor can execute the computer program to implement the smoke and fire detection method according to any one of claims 1 to 4.

6. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the smoke and fire detection method according to any one of claims 1 to 4 is implemented.

Images