Smoke Fire Recognition System and Its Related Equipment from the Perspective of UAV

Abstract

The present application discloses a smoke fire recognition system and related devices from the perspective of an unmanned aerial vehicle. Among them, the system includes: an acquisition module for acquiring multiple captured images of an object to be recognized; a detection module for inputting the multiple captured images into a preset smoke fire detection model to determine whether there is a fire in the object to be recognized; a feature extraction module for obtaining corresponding smoke fire features according to the multiple captured images when there is a fire in the object to be recognized, and the smoke fire features at least include a smoke fire quantity feature and a smoke fire area feature; and an identification module for identifying the smoke fire level of the object to be recognized according to the smoke fire features. The present application can quickly and accurately detect whether there is a fire in the object to be recognized, and timely and accurately reflect the actual situation of the smoke fire when there is a fire.

Description

Technical Field

[0001] The present application relates to the technical field of fire safety, and more particularly, to a smoke fire recognition system and related equipment from the perspective of an unmanned aerial vehicle (UAV). Background Art

[0002] With the continuous progress and development of society, people's living standards have been continuously improved. Correspondingly, fire and related safety issues have increasingly attracted attention.

[0003] Currently, the early warning of smoke fires is mainly carried out through methods such as temperature, smoke sensors, and manual regular inspections. However, the detection of temperature or smoke changes through sensors has a slow response, a high false alarm rate, and a relatively limited monitoring range, making it difficult to provide timely early warnings and accurately reflect the actual situation of smoke fires.

[0004] Chinese Patent Application No. CN202210013021.1 discloses a vision-based smoke fire detection method, and its implementation steps are as follows: 1) Collect various smoke fire and non-smoke fire images as positive and negative samples; 2) Input the collected images as training data into the model for training and save the model; 3) Input the monitored object pictures into the model for detection to determine whether a smoke fire has occurred. Although this method can improve the detection efficiency and accuracy of whether a smoke fire has occurred in the monitored object, it still cannot timely and accurately reflect the actual situation of the occurring smoke fire. Summary of the Invention

[0005] The objective of the present application is to provide a smoke fire recognition method, system, electronic device, and storage medium from the perspective of an unmanned aerial vehicle, which can quickly and accurately detect whether there is a fire in the object to be recognized, and timely and accurately reflect the actual situation of the smoke fire when there is a fire.

[0006] To achieve the above objective, in a first aspect, the present application provides a smoke fire recognition method from the perspective of an unmanned aerial vehicle, including:

[0007] Obtain multiple captured images of the object to be recognized;

[0008] Input the multiple captured images into a preset smoke fire detection model to determine whether there is a fire in the object to be recognized;

[0009] When there is a fire in the object to be recognized, obtain corresponding smoke fire characteristics based on the multiple captured images, where the smoke fire characteristics at least include smoke fire quantity characteristics and smoke fire area characteristics;

[0010] Identify the smoke fire level of the object to be recognized according to the smoke fire characteristics.

[0011] In a preferred embodiment of the present application, obtaining the corresponding smoke and fire characteristics according to the multiple captured images includes:

[0012] Performing image synthesis on the multiple captured images to obtain a panoramic image of the object to be recognized;

[0013] Obtaining the corresponding smoke and fire characteristics according to the panoramic image.

[0014] In a preferred embodiment of the present application, the smoke and fire characteristics include the quantity characteristics of smoke and fire, the area characteristics of smoke and fire, and the distribution characteristics of smoke and fire.

[0015] In a preferred embodiment of the present application, the quantity characteristics of smoke and fire are the quantity of smoke and fire; the area characteristics of smoke and fire include the maximum area ratio of the local fire in a single captured image among the multiple captured images in the corresponding captured image and the total area ratio of the total fire area in the panoramic image; the distribution characteristics of smoke and fire are the smoke and fire distribution index.

[0016] In a preferred embodiment of the present application, identifying the smoke and fire level of the object to be recognized according to the smoke and fire characteristics includes:

[0017] Calculating the corresponding smoke and fire danger index according to the quantity of smoke and fire, the maximum area ratio of the local fire in the corresponding captured image, the total area ratio of the total fire area in the panoramic image, and the smoke and fire distribution index;

[0018] Identifying the smoke and fire level of the object to be recognized according to the corresponding smoke and fire danger index.

[0019] In a preferred embodiment of the present application, the corresponding smoke and fire danger index is calculated through the following calculation formula:

[0020] ;

[0021] wherein, represents the quantity of smoke and fire, represents the maximum area ratio of the local fire in the corresponding captured image, represents the total area ratio of the total fire area in the panoramic image, represents the smoke and fire distribution index, , , and represent the weight coefficients, represents the weighted correction coefficient, represents the smoke and fire danger index.

[0022] In a preferred embodiment of the present application, obtaining a corresponding smoke and fire distribution index according to the panoramic image includes:

[0023] Extract each channel of the panoramic image to obtain the contour of each fire in the panoramic image;

[0024] Calculate the center point of each fire in the panoramic image according to the contour of each fire in the panoramic image;

[0025] Calculate the corresponding smoke and fire distribution index according to the center point of each fire in the panoramic image and the center point of the panoramic image.

[0026] In a preferred embodiment of the present application, the corresponding smoke and fire distribution index is calculated through the following calculation formula:

[0027] ;

[0028] Wherein, 𝑁 represents the number of smoke and fires, represents the center point of each different fire, represents the center point of the panoramic image, B represents the width of the panoramic image, H represents the height of the panoramic image, and 𝐷 represents the smoke and fire distribution index.

[0029] In a preferred embodiment of the present application, the obtaining the corresponding smoke and fire characteristics according to the multiple captured images further includes:

[0030] Obtain at least one fire captured image from the multiple captured images;

[0031] Calculate the proportion of the local area of the local fire in each fire captured image in the corresponding captured image, and use the proportion of the local area with the largest proportion in the fire captured image as the maximum area proportion of the local fire in the corresponding captured image.

[0032] In a second aspect, the present application provides a smoke and fire recognition system from the perspective of an unmanned aerial vehicle, including:

[0033] An acquisition module for acquiring multiple captured images of an object to be recognized;

[0034] A detection module for inputting the multiple captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be recognized;

[0035] A feature extraction module for obtaining corresponding smoke and fire characteristics according to the multiple captured images when there is a fire in the object to be recognized, and the smoke and fire characteristics at least include a smoke and fire quantity characteristic and a smoke and fire area characteristic;

[0036] An identification module, configured to identify the smoke fire level of the object to be identified according to the smoke fire characteristics.

[0037] In a third aspect, the present application provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned method for identifying smoke fire from the perspective of an unmanned aerial vehicle.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned method for identifying smoke fire from the perspective of an unmanned aerial vehicle is implemented.

[0039] Compared with the prior art, the method, system, electronic device and storage medium for identifying smoke fire from the perspective of an unmanned aerial vehicle in the present application have at least the following beneficial effects:

[0040] In the present application, by inputting multiple acquired captured images into a preset smoke fire detection model, it is possible to quickly and accurately determine whether there is a fire in the object to be identified. Using multiple captured images can effectively reduce the false detection rate. When there is a fire in the object to be identified, based on the multiple captured images, corresponding smoke fire characteristics are obtained. The smoke fire characteristics at least include the smoke fire quantity characteristic and the smoke fire area characteristic. Then, through the smoke fire quantity characteristic and the smoke fire area characteristic, the smoke fire level of the object to be identified is identified. The smoke fire quantity characteristic and the smoke fire area characteristic can more specifically and clearly reflect the situation of the smoke fire, so as to timely and accurately reflect the actual situation of the smoke fire occurring in the object to be identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0042] Figure 1 is a schematic flowchart of the method for identifying smoke fire from the perspective of an unmanned aerial vehicle provided by the embodiment of the present application;

[0043] Figure 2 is a structural block diagram of the system for identifying smoke fire from the perspective of an unmanned aerial vehicle provided by the embodiment of the present application;

[0044] Figure 3 is an internal structural schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The following further describes in detail the specific implementation manners of the present application in conjunction with the accompanying drawings and embodiments. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0046] Traditional early warnings of smoke fire mainly rely on methods such as temperature, smoke sensors, and manual regular inspections. However, the detection of temperature or smoke changes through sensors is slow in response, has a high false alarm rate, and the monitoring range is relatively limited, making it difficult to give early warnings in a timely manner and accurately reflect the actual situation of smoke fire. Existing vision-based smoke fire detection methods can only determine whether a smoke fire occurs in the monitored object, improving the detection efficiency and accuracy, but still cannot reflect the actual situation of the occurring smoke fire in a timely and accurate manner.

[0047] In view of the above problems in the prior art, the embodiments of the present application provide a method, system, electronic device, and storage medium for identifying smoke fire from the perspective of an unmanned aerial vehicle (UAV), which can quickly and accurately detect whether there is a fire in the object to be identified, and timely and accurately reflect the actual situation of the smoke fire when there is a fire.

[0048] See Figure 1 , Figure 1 which is a schematic flowchart of the method for identifying smoke fire from the perspective of an unmanned aerial vehicle provided by the embodiments of the present application.

[0049] In the embodiments of the present application, the following method for identifying smoke fire from the perspective of an unmanned aerial vehicle can be applied to computer devices such as servers.

[0050] The embodiments of the present application provide a method for identifying smoke fire from the perspective of an unmanned aerial vehicle, including the following steps:

[0051] Step S110, obtain multiple captured images of the object to be identified.

[0052] The object to be identified is the monitored object, and the object to be identified can be an object such as a building, a natural landscape, a local area, or a local region.

[0053] Multiple captured images of the object to be identified are obtained by an unmanned aerial vehicle. They can be multiple captured images of the object to be identified from different perspectives. The multiple captured images of the object to be identified can be partial captured images of the object to be identified or panoramic captured images of the object to be identified. In this embodiment, taking the multiple captured images of the object to be identified as partial captured images of the object to be identified as an example to elaborate and illustrate the corresponding content of the embodiment.

[0054] In one embodiment, a drone can be used to take pictures of the object to be recognized, obtaining a captured video of the object to be recognized. Multiple images are extracted from the captured video of the object to be recognized at a fixed frequency of image frames as multiple captured images of the object to be recognized. Optionally, the extracted multiple images can also be filtered to obtain multiple captured images of the object to be recognized. In other embodiments, images obtained by the drone taking pictures of the object to be recognized from different perspectives can also be used as multiple captured images of the object to be recognized.

[0055] Step S120: Input the multiple captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be recognized.

[0056] In one embodiment, the preset smoke and fire detection model is a trained computer vision model. Specifically, various different smoke and fire images and non-smoke and fire images can be used as positive and negative samples for model training to train the computer vision model to obtain the preset smoke and fire detection model. Further, when training the computer vision model, similar smoke and fire images and non-smoke and fire images at similar angles or heights can be added for training to improve the model's discrimination ability and classification ability, improve the accuracy of smoke and fire feature extraction, and thus improve the recognition accuracy of the smoke and fire level of the object to be recognized.

[0057] After the multiple captured images are input into the preset smoke and fire detection model, it can be determined whether there is a fire in the object to be recognized.

[0058] In one embodiment, if it is determined that there is a fire in the object to be recognized, then step S130 is executed; if it is determined that there is no fire in the object to be recognized, then this process ends.

[0059] Step S130: Obtain corresponding smoke and fire features according to the multiple captured images. The smoke and fire features at least include a smoke and fire quantity feature and a smoke and fire area feature.

[0060] In one embodiment, the smoke and fire quantity feature can be the quantity of smoke and fire. The smoke and fire area feature can be the average value of the proportion of the smoke and fire area in each captured image among the multiple captured images, or it can be the displayed area of the smoke and fire shown in the multiple captured images. The displayed area of the smoke and fire can be calculated based on the proportion of the smoke and fire area in each captured image and the shooting position of the drone corresponding to each captured image. In other embodiments, the smoke and fire area feature can also be other area features, which are not listed here. It can be understood that the smoke and fire area feature can also include multiple area features and is not limited to just one.

[0061] In one embodiment, the corresponding smoke and fire features can be directly extracted from the multiple captured images.

[0062] Step S140: Identify the smoke fire level of the object to be identified according to the characteristics of the smoke fire situation.

[0063] In one embodiment, the smoke fire level of the object to be identified can be identified by comparing with the interval thresholds corresponding to the preset smoke fire quantity feature and the smoke fire area feature.

[0064] In other embodiments, the smoke fire level of the object to be identified can also be identified by performing calculations on the smoke fire quantity feature and the smoke fire area feature through a predetermined calculation method to obtain corresponding numerical values, and then identifying the smoke fire level of the object to be identified through the corresponding numerical values.

[0065] In the method for identifying smoke fire situations from the perspective of an unmanned aerial vehicle according to the embodiments of the present application, by inputting multiple acquired captured images into a preset smoke fire detection model, it is possible to quickly and accurately determine whether there is a fire in the object to be identified. Using multiple captured images can effectively reduce the false detection rate; when there is a fire in the object to be identified, based on the multiple captured images, corresponding smoke fire situation features are obtained. The smoke fire situation features at least include the smoke fire quantity feature and the smoke fire area feature. Then, through the smoke fire quantity feature and the smoke fire area feature, the smoke fire level of the object to be identified is identified. The smoke fire quantity feature and the smoke fire area feature can more specifically and clearly reflect the situation of the smoke fire, so as to timely and accurately reflect the actual situation of the smoke fire occurring in the object to be identified.

[0066] In one embodiment, in step S130, obtaining the corresponding smoke fire situation features according to the multiple captured images may include:

[0067] Perform image synthesis on the multiple captured images to obtain a panoramic image of the object to be identified;

[0068] Obtain the corresponding smoke fire situation features according to the panoramic image.

[0069] In this embodiment, it is preferable that the multiple captured images are multiple images extracted from the captured video of the object to be identified at a fixed frequency as the multiple captured images of the object to be identified. Extracting multiple images from the captured video at a fixed frequency is beneficial for the synthesis of the panoramic image of the object to be identified and can ensure the image quality of the synthesized panoramic image of the object to be identified.

[0070] The panoramic image of the object to be identified can more clearly reflect the smoke fire situation features. Through the panoramic image of the object to be identified, the corresponding smoke fire situation features can be obtained more quickly and accurately, improving the recognition accuracy of the smoke fire level of the object to be identified.

[0071] In this embodiment, image synthesis is performed based on multiple captured images to obtain a panoramic image of the object to be recognized, which may include:

[0072] Obtain a time-series data set of the shooting perspectives corresponding to the shooting video of the object to be recognized;

[0073] Extract a data set of the shooting perspectives corresponding to multiple captured images from the time-series data set of the shooting perspectives;

[0074] Obtain a reference image of the object to be recognized and the corresponding reference perspective. Based on the reference image, the corresponding reference perspective, and the data set of the shooting perspectives, transform multiple captured images to obtain multiple transformed images;

[0075] Perform image synthesis based on multiple transformed images to obtain a panoramic image of the object to be recognized.

[0076] It can be understood that the above method for obtaining the panoramic image of the object to be recognized is applicable to the case where the shooting perspective of the drone is not fixed. For the case where the shooting perspective of the drone is fixed, multiple captured images can be directly used for image synthesis to obtain the panoramic image of the object to be recognized.

[0077] When the drone shoots the shooting video of the object to be recognized, the shooting perspective of the drone can be recorded in real time to form a time-series data set of the shooting perspectives; when extracting the data set of the shooting perspectives corresponding to multiple captured images from the time-series data set of the shooting perspectives, the data set of the shooting perspectives corresponding to multiple captured images can be extracted according to the corresponding time series;

[0078] The reference image of the object to be recognized can be selected from multiple captured images. When selecting, the matching or closest captured image can be selected from multiple captured images as the reference image of the object to be recognized according to the predetermined shooting parameters. Correspondingly, the shooting perspective of the selected captured image is used as the reference perspective; alternatively, the reference image of the object to be recognized can also be the reference image of the object to be recognized captured by the drone at the predetermined reference perspective;

[0079] When transforming multiple captured images based on the reference image, the corresponding reference perspective, and the data set of the shooting perspectives, the reference perspective is used as the transformation basis, and the reference image is used as the transformation reference. Combining the data set of the shooting perspectives of multiple captured images, transform multiple captured images to obtain multiple transformed images with the reference perspective.

[0080] Through the above method, a dataset of shooting perspectives corresponding to multiple captured images can be accurately obtained. Then, by combining the reference image and reference perspective of the object to be recognized with the dataset of shooting perspectives, multiple captured images can be transformed, so that the perspectives of the images for image synthesis are unified, and the quality of the transformed images is greatly improved. Furthermore, the panoramic image of the object to be recognized synthesized is more accurate and reasonable, which is conducive to the accurate extraction of smoke fire characteristics.

[0081] In this embodiment, the smoke fire characteristics include the number characteristics of smoke fires, the area characteristics of smoke fires, and the distribution characteristics of smoke fires.

[0082] Among them, the distribution characteristics of smoke fires can reflect the distribution situation between different smoke fires. By combining the distribution characteristics of smoke fires, the connection and correlation between different smoke fires can be strengthened, so that a more detailed judgment can be made when identifying the smoke fire level of the object to be recognized, and the classification and identification of the smoke fire level of the object to be recognized are more specific.

[0083] Furthermore, the number characteristic of smoke fires is the number of smoke fires; the area characteristic of smoke fires includes the maximum area ratio of the local fire in a single captured image among multiple captured images in the corresponding captured image and the total area ratio of the total fire area in the panoramic image; the distribution characteristic of smoke fires is the smoke fire distribution index.

[0084] Among them, the number of smoke fires can directly reflect the severity of the fire. The maximum area ratio of the local fire in the corresponding captured image can reflect the severity of a single fire. The total area ratio of the total fire area in the panoramic image can reflect the overall scope of the fire. The smoke fire distribution index can reflect the dispersion degree of different smoke fires, and the distribution situation between different smoke fires is represented by the dispersion degree of different smoke fires.

[0085] The area characteristic of smoke fires adopts the area ratios of two different smoke fires, which can be measured from different angles, strengthen the identification basis of the smoke fire level of the object to be recognized, enrich the consideration elements of the smoke fire level of the object to be recognized, and make the identification of the smoke fire level of the object to be recognized more accurate; at the same time, the distribution characteristic of smoke fires adopts the smoke fire distribution index, which can effectively strengthen the connection and correlation between different smoke fires, so that the distribution characteristic of smoke fires is more effectively reflected in the identification of the smoke fire level of the object to be recognized.

[0086] In this embodiment, the number of smoke fires and the total area ratio of the total fire area in the panoramic image can be obtained through the panoramic image.

[0087] When obtaining the maximum area ratio of the local fire in the corresponding captured image, it is possible to:

[0088] Obtain at least one fire scene captured image from multiple captured images;

[0089] Calculate the proportion of the local area of the local fire in each fire scene captured image in the corresponding captured image, and use the proportion of the local area of the fire scene captured image with the largest proportion of the local area as the maximum area proportion of the local fire in the corresponding captured image.

[0090] Among them, the fire scene captured image means that the captured image can reflect the existence of smoke fire.

[0091] In this way, the calculation efficiency of the maximum area proportion of the local fire in the corresponding captured image is greatly improved, and the calculation amount is greatly reduced.

[0092] When obtaining the smoke fire distribution index, it is possible to:

[0093] Extract each channel of the panoramic image to obtain the contour of each fire in the panoramic image;

[0094] According to the contour of each fire in the panoramic image, calculate the center point of each fire in the panoramic image;

[0095] According to the center point of each fire in the panoramic image and the center point of the panoramic image, calculate the corresponding smoke fire distribution index.

[0096] The panoramic image of the object to be recognized is usually a color image, and the channels of the panoramic image of the object to be recognized are the red, green, and blue channels of the color image.

[0097] It should be understood that the contour of the fire is the contour of the open fire and does not include smoke; if a certain smoke fire includes an open fire and smoke, the contour of the fire obtained by extraction is the contour of the open fire; if a certain smoke fire does not include an open fire and only has smoke, it means that the generated smoke obscures the open fire, and the contour of the smoke is extracted as the contour of the fire.

[0098] It should be noted that if a certain smoke fire in the panoramic image only has smoke and no open fire, the position of the bottom left corner of the contour of the smoke fire is used as the center point of the fire.

[0099] When extracting each channel of the panoramic image and obtaining the outline of each fire in the panoramic image, each smoke fire in the panoramic image of the object to be recognized can be recognized first, and the category of each smoke fire can be judged. The category of the smoke fire can be smoke or open fire. Understandably, when the category of the smoke fire is smoke, there is no open fire at the location of the smoke fire; when the category of the smoke fire is open fire, there is an open fire at the location of the smoke fire, which may or may not have smoke. Furthermore, combining the category of each smoke fire, the outline of each fire in the panoramic image is obtained. Among them, when the category of the smoke fire is smoke, the outline of the smoke fire is extracted as the outline of the fire; when the category of the smoke fire is open fire, the outline of the open fire of the smoke fire is extracted as the outline of the fire. In this way, the accuracy and comprehensiveness of the outline of each fire in the extracted panoramic image can be improved.

[0100] By extracting each channel of the panoramic image, omission during extraction can be effectively avoided, and moreover, smoke can be accurately excluded in the area containing open fire and smoke, preventing smoke from interfering with the selection of the outline of the fire. As a result, the outline of each fire in the panoramic image can be obtained accurately and relatively quickly, ensuring the accuracy of the center point of each fire in the panoramic image, and further improving the accuracy of the calculated smoke fire distribution index.

[0101] In this embodiment, according to the characteristics of the smoke fire, identifying the level of the smoke fire of the object to be recognized may include:

[0102] Calculating the corresponding smoke fire danger index based on the number of smoke fires, the maximum area ratio of the local fire in the corresponding captured image, the total area ratio of the total fire area in the panoramic image, and the smoke fire distribution index;

[0103] Identifying the level of the smoke fire of the object to be recognized according to the corresponding smoke fire danger index.

[0104] In this embodiment, the level of the smoke fire of the object to be recognized can be quickly identified according to the preset interval threshold of the smoke fire danger index.

[0105] As an optional implementation manner, when calculating the corresponding smoke fire danger index, the corresponding smoke fire danger index can be calculated through the following calculation formula:

[0106] ;

[0107] Among them, represents the number of smoke fires, represents the maximum area ratio of the local fire in the corresponding captured image, represents the total area ratio of the total fire area in the panoramic image, represents the smoke fire distribution index, , , and represents the weight coefficient, represents the weighted correction coefficient, represents the smoke fire danger index.

[0108] Specifically, the smoke fire distribution index 𝐷 can reflect the dispersion degree of different smoke fires. The larger the smoke fire distribution index, the higher the dispersion degree of different smoke fires; the weighted correction coefficient 𝐶 is used to modify the value range. For example, the weighted correction coefficient 𝐶 can be set to 10 to amplify the value so that the value can reach a reasonable range, which is beneficial to the identification of the smoke fire level of the object to be identified.

[0109] In this calculation method, different smoke fire characteristics are given different weight coefficients, which also represent the importance of different smoke fire characteristics; in practical applications, the importance of the smoke fire characteristics of different objects to be identified may be different. The weight coefficients of different smoke fire characteristics can be adjusted according to the type of the object to be identified, so as to more accurately calculate the corresponding smoke fire danger index of the object to be identified.

[0110] Specifically, when calculating the corresponding smoke fire danger index, the weight coefficients of different smoke fire characteristics can be obtained in the following way. Identify the type of the object to be identified through the panoramic image of the object to be identified, and extract the characteristics of the object to be identified. The characteristics of the object to be identified can include the characteristics of the object itself and the environmental characteristics of the object to be identified. Taking a building as an example, the characteristics of the object itself are the characteristics of the building, and the environmental characteristics of the object to be identified can be the characteristics of the natural environment around the building; furthermore, according to the type, characteristics and smoke fire characteristics of the object to be identified, simulate the development of the smoke fire of the object to be identified within a predetermined time, and determine the weight coefficients of different smoke fire characteristics. When determining, determine according to the danger degree of different smoke fire characteristics on the object to be identified, so that the weight coefficients of different smoke fire characteristics adopted are more reasonable and accurate, thereby improving the accuracy of the calculated smoke fire danger index.

[0111] Through this calculation method, the importance of different smoke fire characteristics is fully considered, and the smoke fire danger index of the object to be identified can be obtained more reasonably. Moreover, through the weighted correction coefficient for correction, the value is amplified to reach a reasonable range, and the smoke fire danger index of the object to be identified is corrected twice, so that the calculated smoke fire danger index of the object to be identified has higher accuracy.

[0112] Optionally, when calculating the smoke fire distribution index, the corresponding smoke fire distribution index can be calculated through the following calculation formula:

[0113] ;

[0114] where 𝑁 represents the number of smoke fires, represents the center point of each different fire, represents the center point of the panoramic image, B represents the width of the panoramic image, H represents the height of the panoramic image, and 𝐷 represents the smoke fire distribution index.

[0115] This calculation method effectively and reasonably relates and applies each smoke fire, and combines the width and height of the panoramic image, so that the calculated smoke fire distribution index can more accurately reflect the dispersion degree of different smoke fires, thereby improving the accuracy of the calculated smoke fire danger index, and thus improving the recognition accuracy of the smoke fire level of the object to be recognized.

[0116] In order to execute the method corresponding to the above embodiment to achieve the corresponding functions and technical effects, the following provides a smoke fire recognition system from the perspective of an unmanned aerial vehicle.

[0117] See Figure 2 , Figure 2 which is the structural block diagram of the smoke fire recognition system from the perspective of an unmanned aerial vehicle provided by the embodiment of the present application.

[0118] The smoke fire recognition system from the perspective of an unmanned aerial vehicle provided by the embodiment of the present application includes:

[0119] An acquisition module 210, configured to acquire multiple captured images of the object to be recognized;

[0120] A detection module 220, configured to input the multiple captured images into a preset smoke fire detection model to determine whether there is a fire in the object to be recognized;

[0121] A feature extraction module 230, configured to obtain corresponding smoke fire features according to the multiple captured images when there is a fire in the object to be recognized, and the smoke fire features at least include a smoke fire quantity feature and a smoke fire area feature;

[0122] An identification module 240, configured to identify the smoke fire level of the object to be recognized according to the smoke fire features.

[0123] In the smoke fire recognition system from the perspective of an unmanned aerial vehicle according to an embodiment of the present application, by inputting multiple acquired captured images into a preset smoke fire detection model, it is possible to quickly and accurately determine whether there is a fire in the object to be recognized. Using multiple captured images can effectively reduce the false detection rate. When there is a fire in the object to be recognized, corresponding smoke fire characteristics are obtained based on the multiple captured images. The smoke fire characteristics at least include the smoke fire quantity characteristic and the smoke fire area characteristic. Furthermore, through the smoke fire quantity characteristic and the smoke fire area characteristic, the smoke fire level of the object to be recognized is identified. The smoke fire quantity characteristic and the smoke fire area characteristic can more specifically and clearly reflect the situation of the smoke fire, so as to timely and accurately reflect the actual situation of the smoke fire occurring in the object to be recognized.

[0124] In one embodiment, the feature extraction module 230 may be specifically configured to:

[0125] Perform image synthesis on the multiple captured images to obtain a panoramic image of the object to be recognized;

[0126] Obtain corresponding smoke fire characteristics based on the panoramic image.

[0127] As an alternative implementation manner, when the feature extraction module 230 obtains the corresponding smoke fire distribution index based on the panoramic image, it may be specifically configured to:

[0128] Extract each channel of the panoramic image to extract the contour of each fire in the panoramic image;

[0129] Calculate the center point of each fire in the panoramic image according to the contour of each fire in the panoramic image;

[0130] Calculate the corresponding smoke fire distribution index according to the center point of each fire in the panoramic image and the center point of the panoramic image.

[0131] As an alternative implementation manner, the feature extraction module 230 may also be specifically configured to:

[0132] Obtain at least one fire captured image from the multiple captured images;

[0133] Calculate the proportion of the local area of the local fire in each fire captured image in the corresponding captured image, and use the proportion of the local area with the largest proportion in the fire captured image with the largest local area proportion as the maximum area proportion of the local fire in the corresponding captured image.

[0134] In one embodiment, the recognition module 240 may be specifically configured to:

[0135] According to the number of smoke fire situations, the maximum area ratio of the local fire situation in the corresponding captured image, the total area ratio of the total fire situation in the panoramic image, and the smoke fire distribution index, the corresponding smoke fire danger index is calculated.

[0136] According to the corresponding smoke fire danger index, the smoke fire level of the object to be recognized is identified.

[0137] The above-mentioned smoke fire recognition system from the perspective of the drone can implement the smoke fire recognition method from the perspective of the drone in the above text. For the specific limitations and other content of the above-mentioned embodiments of the smoke fire recognition system from the perspective of the drone, reference can be made to the content of the smoke fire recognition method from the perspective of the drone in the above text, and details will not be repeated in the embodiments.

[0138] An embodiment of the present application further provides an electronic device, including a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned smoke fire recognition method from the perspective of the drone.

[0139] Optionally, the above-mentioned electronic device can be a computer device such as a server.

[0140] In one embodiment, the internal structure of the electronic device of the present application can be as Figure 3 shown.

[0141] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned smoke fire recognition method from the perspective of the drone.

[0142] In several embodiments provided by the present application, it should be understood that the disclosed device / system and method can also be implemented in other ways. The above-described device / system embodiments are merely illustrative. For example, the flowcharts and block diagrams in the drawings show the possible architectures, functions, and operations of the device / system, method, and computer program product according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0143] In addition, each functional module in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

[0144] If the above-mentioned function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0145] The above are only the embodiments of the present application and are not used to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0146] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0147] It should be noted that in this text, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. At the same time, in the description of this application, terms such as "first" and "second" are only used for differential description and cannot be understood as indicating or implying relative importance. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the said element.

Claims

1. A method for identifying smoke fire under the perspective of an unmanned aerial vehicle, characterized in that Including: Obtain multiple captured images of the object to be recognized; Input the multiple captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be recognized; When there is a fire in the object to be recognized, obtain corresponding smoke and fire characteristics according to the multiple captured images, where the smoke and fire characteristics at least include a smoke and fire quantity characteristic and a smoke and fire area characteristic; Identify the smoke and fire level of the object to be recognized according to the smoke and fire characteristics; The obtaining of the corresponding smoke and fire characteristics according to the multiple captured images includes: Perform image synthesis on the multiple captured images to obtain a panoramic image of the object to be recognized; Obtain corresponding smoke and fire characteristics according to the panoramic image; The smoke and fire characteristics include a smoke and fire quantity characteristic, a smoke and fire area characteristic, and a smoke and fire distribution characteristic; The smoke and fire quantity characteristic is the quantity of smoke and fire; the smoke and fire area characteristic includes the maximum area ratio of the local fire in a single captured image among the multiple captured images in the corresponding captured image and the total area ratio of the total fire area in the panoramic image; the smoke and fire distribution characteristic is the smoke and fire distribution index; The identifying of the smoke and fire level of the object to be recognized according to the smoke and fire characteristics includes: Calculate a corresponding smoke and fire danger index according to the quantity of smoke and fire, the maximum area ratio of the local fire in the corresponding captured image, the total area ratio of the total fire area in the panoramic image, and the smoke and fire distribution index; Identify the smoke and fire level of the object to be recognized according to the corresponding smoke and fire danger index; The corresponding smoke and fire danger index is calculated through the following calculation formula: ； Among them, represents the number of smoke fire incidents, represents the maximum area ratio of the local fire in the corresponding captured image, represents the total area ratio of the total fire area in the panoramic image, represents the smoke fire distribution index, , , and represent the weight coefficients, represents the weighted correction coefficient, represents the smoke fire danger index; The obtaining of the corresponding smoke and fire distribution index according to the panoramic image includes: Extract each channel of the panoramic image to extract the contour of each fire in the panoramic image; Calculate the center point of each fire in the panoramic image according to the contour of each fire in the panoramic image; Calculate a corresponding smoke and fire distribution index according to the center point of each fire in the panoramic image and the center point of the panoramic image; The corresponding smoke and fire distribution index is calculated through the following calculation formula: ； Among them, represents the number of smoke fire incidents, represents the center point of each different fire incident, represents the center point of the panoramic image, represents the width of the panoramic image, represents the height of the panoramic image, represents the smoke fire distribution index.

2. The method for identifying smoke fire under the perspective of an unmanned aerial vehicle according to claim 1, wherein The obtaining of the corresponding smoke and fire characteristics according to the multiple captured images further includes: Obtain at least one fire captured image from the multiple captured images; Calculate the local area ratio of the local fire in each fire captured image in the corresponding captured image, and use the local area ratio of the fire captured image with the largest local area ratio as the maximum area ratio of the local fire in the corresponding captured image.

3. A smoke fire recognition system from the perspective of an unmanned aerial vehicle, characterized in that, Including: An obtaining module, configured to obtain multiple captured images of the object to be recognized; A detection module, configured to input the multiple captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be recognized; A feature extraction module, configured to obtain corresponding smoke fire features according to the multiple captured images when there is a fire in the object to be recognized, where the smoke fire features at least include a smoke fire quantity feature and a smoke fire area feature; An identification module, configured to identify the smoke fire level of the object to be recognized according to the smoke fire features; Specifically, the feature extraction module is configured to: Perform image synthesis on the multiple captured images to obtain a panoramic image of the object to be recognized; Obtain corresponding smoke fire features according to the panoramic image; The smoke fire features include a smoke fire quantity feature, a smoke fire area feature, and a smoke fire distribution feature; The smoke fire quantity feature is the quantity of smoke fires; the smoke fire area feature includes the maximum area ratio of the local fire in a single captured image among the multiple captured images in the corresponding captured image and the total area ratio of the total fire area in the panoramic image; the smoke fire distribution feature is the smoke fire distribution index; Specifically, the identification module is configured to: Calculate a corresponding smoke fire danger index according to the quantity of smoke fires, the maximum area ratio of the local fire in the corresponding captured image, the total area ratio of the total fire area in the panoramic image, and the smoke fire distribution index; Identify the smoke fire level of the object to be recognized according to the corresponding smoke fire danger index; The corresponding smoke fire danger index is calculated through the following calculation formula: ； Among them, represents the number of smoke fire incidents, represents the maximum area ratio of the local fire in the corresponding captured image, represents the total area ratio of the total fire area in the panoramic image, represents the smoke fire distribution index, , , and represent the weight coefficients, represents the weighted correction coefficient, represents the smoke fire danger index; When obtaining the corresponding smoke fire distribution index according to the panoramic image, the feature extraction module is specifically configured to: Extract each channel of the panoramic image to obtain the contour of each fire in the panoramic image; Calculate the center point of each fire in the panoramic image according to the contour of each fire in the panoramic image; Calculate the corresponding smoke fire distribution index according to the center point of each fire in the panoramic image and the center point of the panoramic image; The corresponding smoke fire distribution index is calculated through the following calculation formula: ； Among them, represents the number of smoke fire incidents, represents the center point of each different fire incident, represents the center point of the panoramic image, represents the width of the panoramic image, represents the height of the panoramic image, represents the smoke fire distribution index.

4. An electronic device, characterized in that, It includes a memory and a processor. The memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for identifying smoke fires from the perspective of an unmanned aerial vehicle according to any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that, It stores a computer program, and when the computer program is executed by the processor, it implements the method for identifying smoke fires from the perspective of an unmanned aerial vehicle according to any one of claims 1 to 2.

Images