Smoke fire behavior identification system under view angle of unmanned aerial vehicle and related equipment thereof

Abstract

The invention discloses a smog and fire behavior recognition system under the view angle of an unmanned aerial vehicle and related equipment thereof, and the system comprises an obtaining module which is used for obtaining a plurality of shot images of a to-be-recognized object; the detection module is used for inputting the plurality of shot images into a preset smoke fire behavior detection model and judging whether the to-be-identified object has a fire behavior or not; the feature extraction module is used for obtaining corresponding smoke and fire features according to the plurality of shot images when the to-be-identified object has a fire, and the smoke and fire features at least comprise smoke and fire quantity features and smoke and fire area features; and the identification module is used for identifying the smoke fire behavior level of the to-be-identified object according to the smoke fire behavior characteristics. According to the invention, whether the to-be-identified object has the fire behavior can be rapidly and accurately detected, and the actual condition of the smoke fire behavior can be timely and accurately reflected when the fire behavior exists.

Description

Technical Field

[0001] The present application relates to the field of fire safety technology, and more specifically, to a smoke and fire identification system from the perspective of an unmanned aerial vehicle and related equipment. Background Art

[0002] With the continuous progress and development of society, people's living standards are constantly improving. Correspondingly, fire and related safety issues are receiving increasing attention.

[0003] At present, the early warning of smoke fires is mainly carried out through temperature, smoke sensors and manual regular inspections. However, the sensors detect changes in temperature or smoke, which has a slow response, a high false alarm rate and a limited monitoring range, making it difficult to issue early warnings and accurately reflect the actual situation of smoke fires.

[0004] The Chinese patent application with patent application number CN202210013021.1 discloses a vision-based smoke and fire detection method, the implementation steps of which are as follows: 1) Collect various smoke and fire images and non-smoke and fire images as positive and negative samples; 2) Input the collected images as training data into the model for training and save the model; 3) Input the monitored object image into the model for detection to determine whether a smoke and fire has occurred. Although this method can improve the detection efficiency and accuracy of whether a smoke and fire has occurred in the monitored object, it still cannot timely and accurately reflect the actual situation of the smoke and fire that has occurred. Summary of the invention

[0005] The purpose of this application is to provide a method, system, electronic device and storage medium for identifying smoke and fire from the perspective of a drone, which can quickly and accurately detect whether there is a fire on the object to be identified, and timely and accurately reflect the actual situation of the smoke and fire when a fire exists.

[0006] In order to achieve the above objectives, in the first aspect, the present application provides a method for identifying smoke and fire from the perspective of a drone, comprising: Acquire multiple captured images of the object to be identified; Inputting the plurality of captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be identified; When there is a fire in the object to be identified, corresponding smoke fire characteristics are obtained according to the multiple captured images, wherein the smoke fire characteristics at least include a smoke fire quantity characteristic and a smoke fire area characteristic; The smoke and fire level of the object to be identified is identified according to the smoke and fire characteristics.

[0007] In a preferred embodiment of the present application, the step of obtaining corresponding smoke and fire characteristics based on the plurality of captured images includes: Performing image synthesis based on the multiple captured images to obtain a panoramic image of the object to be identified; According to the panoramic image, corresponding smoke and fire characteristics are obtained.

[0008] In a preferred embodiment of the present application, the smoke and fire characteristics include smoke and fire quantity characteristics, smoke and fire area characteristics and smoke and fire distribution characteristics.

[0009] In a preferred embodiment of the present application, the smoke fire quantity feature is the smoke fire quantity; the smoke fire area feature includes the maximum area ratio of the local fire in a single image among multiple images in the corresponding image and the total area ratio of the total fire area in the panoramic image; the smoke fire distribution feature is the smoke fire distribution index.

[0010] In a preferred embodiment of the present application, the step of identifying the smoke and fire level of the object to be identified according to the smoke and fire characteristics includes: Calculate the corresponding smoke and fire hazard index according to the number of smoke and fire, the maximum area ratio of the local fire in the corresponding captured image, the total area ratio of the total fire area in the panoramic image, and the smoke and fire distribution index; The smoke and fire level of the object to be identified is identified according to the corresponding smoke and fire hazard index.

[0011] In a preferred embodiment of the present application, the corresponding smoke and fire hazard index is calculated by the following calculation formula: ; in, Indicates the number of smoke and fire conditions. Indicates the maximum area proportion of the local fire in the corresponding captured image. Indicates the proportion of the total fire area in the panoramic image. represents the smoke and fire distribution index, , , and represents the weight coefficient, represents the weighted correction coefficient, Indicates the smoke and fire hazard index.

[0012] In a preferred embodiment of the present application, the corresponding smoke and fire distribution index is obtained according to the panoramic image, including: Extracting each channel of the panoramic image to obtain the outline of each fire situation in the panoramic image; Calculating the center point of each fire in the panoramic image according to the outline of each fire in the panoramic image; According to the center point of each fire in the panoramic image and the center point of the panoramic image, a corresponding smoke and fire distribution index is calculated.

[0013] In a preferred embodiment of the present application, the corresponding smoke and fire distribution index is calculated by the following calculation formula: ; Among them, 𝑁 represents the number of smoke fires, Indicates the center point of each different fire situation, represents the center point of the panoramic image, B represents the width of the panoramic image, H represents the height of the panoramic image, and 𝐷 represents the smoke and fire distribution index.

[0014] In a preferred embodiment of the present application, the step of obtaining corresponding smoke and fire characteristics according to the plurality of captured images further includes: Acquire at least one fire situation photographed image from the plurality of photographed images; The local area ratio of the local fire in each fire image in the corresponding image is calculated, and the local area ratio of the fire image with the largest local area ratio is taken as the maximum area ratio of the local fire in the corresponding image.

[0015] In the second aspect, the present application provides a smoke and fire identification system from the perspective of a drone, comprising: An acquisition module, used to acquire multiple photographed images of the object to be identified; A detection module, used to input the plurality of captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be identified; A feature extraction module, used for obtaining corresponding smoke fire characteristics according to the plurality of captured images when there is a fire in the object to be identified, wherein the smoke fire characteristics at least include a smoke fire quantity characteristic and a smoke fire area characteristic; The identification module is used to identify the smoke and fire level of the object to be identified according to the smoke and fire characteristics.

[0016] In a third aspect, the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned method for identifying smoke and fire from the perspective of a drone.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for identifying smoke and fire from the perspective of a drone.

[0018] The present application provides a method, system, electronic device and storage medium for identifying smoke and fire from the perspective of a drone. Compared with the prior art, it has at least the following beneficial effects: The present application can quickly and accurately determine whether there is a fire in the object to be identified by inputting the acquired multiple captured images into a preset smoke and fire detection model. The use of multiple captured images can effectively reduce the false detection rate. When there is a fire in the object to be identified, corresponding smoke and fire characteristics are obtained based on the multiple captured images. The smoke and fire characteristics at least include smoke and fire quantity characteristics and smoke and fire area characteristics. Then, the smoke and fire level of the object to be identified is identified through the smoke and fire quantity characteristics and smoke and fire area characteristics. The smoke and fire quantity characteristics and smoke and fire area characteristics can reflect the situation of the smoke and fire more specifically and clearly, so as to timely and accurately reflect the actual situation of the smoke and fire occurring in the object to be identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a smoke and fire identification method from the perspective of a drone provided in an embodiment of the present application; Figure 2 It is a structural block diagram of a smoke and fire identification system from the perspective of a drone provided in an embodiment of the present application; Figure 3 It is a schematic diagram of the internal structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but are not intended to limit the scope of the present application.

[0022] Traditional smoke fire warnings are mainly carried out through temperature and smoke sensors and manual regular inspections. However, the sensors detect changes in temperature or smoke slowly, with a high false alarm rate, and the monitoring range is relatively limited, making it difficult to provide timely warnings and accurately reflect the actual situation of smoke fires. Existing vision-based smoke fire detection methods can only determine whether smoke fires occur in the monitored objects, improve detection efficiency and accuracy, but still cannot timely and accurately reflect the actual situation of smoke fires.

[0023] In response to the above-mentioned problems in the prior art, the embodiments of the present application provide a method, system, electronic device and storage medium for identifying smoke and fire from the perspective of a drone, which can quickly and accurately detect whether there is a fire on the object to be identified, and timely and accurately reflect the actual situation of the smoke and fire when a fire exists.

[0024] See also Figure 1 , Figure 1 It is a flow chart of the smoke and fire identification method from the perspective of a drone provided in an embodiment of the present application.

[0025] The smoke and fire identification method from the perspective of a drone described below in the embodiment of the present application can be applied to computer equipment such as servers.

[0026] The present application embodiment provides a method for identifying smoke and fire from the perspective of a drone, comprising the following steps: Step S110, obtaining a plurality of photographed images of the object to be identified.

[0027] The object to be identified is the monitoring object, and the object to be identified may be a building, a natural landscape, a local area or a local region.

[0028] The multiple images of the object to be identified are obtained by taking pictures of a drone, which may be multiple images of the object to be identified from different perspectives. The multiple images of the object to be identified may be partial images of the object to be identified or panoramic images of the object to be identified. In this embodiment, the multiple images of the object to be identified are taken as partial images of the object to be identified as an example to explain and illustrate the corresponding contents of the embodiment.

[0029] In one embodiment, the object to be identified can be photographed by a drone to obtain a video of the object to be identified, and multiple images can be extracted from the video of the object to be identified at a fixed frequency of image frames as multiple images of the object to be identified; optionally, the extracted multiple images can be filtered to obtain multiple images of the object to be identified. In other embodiments, images obtained by photographing the object to be identified from different viewing angles by a drone can be used as multiple images of the object to be identified.

[0030] Step S120: input the plurality of captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be identified.

[0031] In one embodiment, the preset smoke and fire detection model is a trained computer vision model. Specifically, a variety of different smoke and fire images and non-smoke and fire images can be used as positive and negative samples for model training to train the computer vision model to obtain the preset smoke and fire detection model; further, when training the computer vision model, similar smoke and fire images and non-smoke and fire images at similar angles or heights can be added for training, so as to improve the resolution and classification capabilities of the model, improve the accuracy of smoke and fire feature extraction, and thus improve the recognition accuracy of the smoke and fire level of the object to be identified.

[0032] After multiple captured images are input into the preset smoke and fire detection model, it can be determined whether there is a fire in the object to be identified.

[0033] In one embodiment, if it is determined that the object to be identified is on fire, step S130 is executed; if it is determined that the object to be identified is not on fire, the process ends.

[0034] Step S130, obtaining corresponding smoke and fire characteristics according to the plurality of captured images, wherein the smoke and fire characteristics at least include smoke and fire quantity characteristics and smoke and fire area characteristics.

[0035] In one embodiment, the smoke fire quantity feature may be the smoke fire quantity, the smoke fire area feature may be the average value of the smoke fire area ratio in each of the multiple images, or may be the display area of ​​the smoke fire displayed by the multiple images, and the display area of ​​the smoke fire may be calculated by the smoke fire area ratio in each image and the corresponding shooting position of each image of the drone. In other embodiments, the smoke fire area feature may also be other area features, which are not listed here. It is understandable that the smoke fire area feature may also include multiple area features, not limited to just one.

[0036] In one embodiment, corresponding smoke and fire characteristics may be directly extracted from a plurality of captured images.

[0037] Step S140, identifying the smoke and fire level of the object to be identified according to the smoke and fire characteristics.

[0038] In one embodiment, the smoke and fire level of the object to be identified can be obtained by comparison and identification based on the preset interval thresholds corresponding to the smoke and fire quantity characteristics and the smoke and fire area characteristics.

[0039] In other embodiments, the smoke and fire quantity characteristics and the smoke and fire area characteristics may be calculated in a predetermined manner to obtain corresponding values, and then the smoke and fire level of the object to be identified may be obtained through corresponding numerical identification.

[0040] The method for identifying smoke and fire from the perspective of a drone in the embodiment of the present application can quickly and accurately determine whether there is a fire in the object to be identified by inputting the acquired multiple captured images into a preset smoke and fire detection model. The use of multiple captured images can effectively reduce the false detection rate. When there is a fire in the object to be identified, corresponding smoke and fire characteristics are obtained based on the multiple captured images. The smoke and fire characteristics at least include smoke and fire quantity characteristics and smoke and fire area characteristics. Then, the smoke and fire level of the object to be identified is identified through the smoke and fire quantity characteristics and smoke and fire area characteristics. The smoke and fire quantity characteristics and smoke and fire area characteristics can reflect the situation of the smoke and fire more specifically and clearly, thereby timely and accurately reflecting the actual situation of the smoke and fire occurring in the object to be identified.

[0041] In one embodiment, step S130, obtaining corresponding smoke and fire characteristics according to the plurality of captured images, may include: Performing image synthesis based on multiple captured images to obtain a panoramic image of the object to be identified; According to the panoramic image, the corresponding smoke and fire characteristics are obtained.

[0042] In this embodiment, the multiple captured images are preferably extracted from the captured video of the object to be identified with image frames at a fixed frequency as the multiple captured images of the object to be identified. Extracting multiple images from the captured video with image frames at a fixed frequency can be beneficial to the synthesis of the panoramic image of the object to be identified and ensure the image quality of the synthesized panoramic image of the object to be identified.

[0043] The panoramic image of the object to be identified can more clearly reflect the characteristics of smoke and fire. Through the panoramic image of the object to be identified, the corresponding smoke and fire characteristics can be obtained more quickly and accurately, thereby improving the recognition accuracy of the smoke and fire level of the object to be identified.

[0044] In this embodiment, performing image synthesis based on multiple captured images to obtain a panoramic image of the object to be identified may include: Obtain a time series data set of shooting angles corresponding to the video of the object to be identified; Extracting a data set of shooting angles corresponding to a plurality of shot images from a time series data set of shooting angles; Obtaining a reference image and a corresponding reference viewing angle of the object to be identified, and transforming a plurality of photographed images according to a data set of the reference image, the corresponding reference viewing angle, and the photographing viewing angle to obtain a plurality of transformed images; Image synthesis is performed based on multiple transformed images to obtain a panoramic image of the object to be identified.

[0045] It can be understood that the above method of obtaining a panoramic image of the object to be identified is applicable to the case where the shooting angle of the drone is not fixed. For the case where the shooting angle of the drone is fixed, multiple images can be directly used for image synthesis to obtain a panoramic image of the object to be identified.

[0046] When the drone shoots a video of the object to be identified, the shooting angle of the drone can be recorded in real time to form a time series data set of the shooting angle; when extracting a data set of shooting angles corresponding to a plurality of shooting images from the time series data set of the shooting angle, the data set of shooting angles corresponding to the plurality of shooting images can be extracted according to the corresponding time series; The reference image of the object to be identified can be selected from multiple captured images. When selecting, a matching or closest captured image can be selected from the multiple captured images according to a predetermined shooting parameter as the reference image of the object to be identified. Correspondingly, the shooting angle of the selected captured image is used as the reference angle. Optionally, the reference image of the object to be identified can also be a reference image of the object to be identified obtained by shooting a drone at a predetermined reference angle. When converting multiple captured images based on a benchmark image, a corresponding benchmark viewing angle, and a data set of shooting viewing angles, the benchmark viewing angle is used as a basis for conversion, and the benchmark image is used as a reference for conversion. The multiple captured images are converted in combination with the data sets of shooting viewing angles of the multiple captured images to obtain multiple converted images with the viewing angle being the benchmark viewing angle.

[0047] Through the above method, a data set of shooting angles corresponding to multiple captured images can be accurately obtained, and then the multiple captured images can be transformed using the baseline image and baseline angle of view of the object to be identified, combined with the data set of shooting angles. This can unify the angle of view of the images used for image synthesis and greatly improve the quality of the transformed images, thereby making the synthesized panoramic image of the object to be identified more accurate and reasonable, which is conducive to the accurate extraction of smoke and fire characteristics.

[0048] In this embodiment, the smoke and fire characteristics include smoke and fire quantity characteristics, smoke and fire area characteristics and smoke and fire distribution characteristics.

[0049] Among them, the smoke and fire distribution characteristics can reflect the distribution of different smoke and fire conditions. By combining the smoke and fire distribution characteristics, the connection and correlation between different smoke and fire conditions can be strengthened, so that more detailed judgment can be made when identifying the smoke and fire level of the object to be identified, making the division and identification of the smoke and fire level of the object to be identified more specific.

[0050] Furthermore, the quantity feature of smoke fire is the number of smoke fire; the area feature of smoke fire includes the maximum area ratio of the local fire in a single image among multiple images in the corresponding image and the total area ratio of the total fire area in the panoramic image; the distribution feature of smoke fire is the smoke fire distribution index.

[0051] Among them, the number of smoke fires can directly reflect the severity of the fire, the maximum area ratio of local fires in the corresponding captured images can reflect the severity of a single fire, the total area ratio of the total fire area in the panoramic image can reflect the overall scope of the fire, and the smoke fire distribution index can reflect the degree of discreteness of different smoke fires. The distribution of different smoke fires is represented by the different degrees of discreteness of different smoke fires.

[0052] The smoke fire area feature adopts the area proportions of two different smoke fires, which can be measured from different angles, strengthen the recognition basis of the smoke fire level of the object to be identified, enrich the consideration elements of the smoke fire level of the object to be identified, and make the recognition of the smoke fire level of the object to be identified more accurate; at the same time, the smoke fire distribution feature adopts the smoke fire distribution index, which can effectively strengthen the connection and correlation between different smoke fires, so that the smoke fire distribution feature can be more effectively reflected in the identification of the smoke fire level of the object to be identified.

[0053] In this embodiment, the number of smoke fires and the proportion of the total fire area in the panoramic image can be obtained through the panoramic image.

[0054] When obtaining the maximum area proportion of the local fire in the corresponding captured image, you can: Acquire at least one fire situation image from a plurality of captured images; The local area ratio of the local fire in each fire image in the corresponding image is calculated, and the local area ratio of the fire image with the largest local area ratio is taken as the maximum area ratio of the local fire in the corresponding image.

[0055] The fire image means that the captured image can reflect the existence of smoke and fire.

[0056] In this way, the calculation efficiency of the maximum area ratio of the local fire in the corresponding captured image is greatly improved, and the amount of calculation is greatly reduced.

[0057] When obtaining the smoke and fire distribution index, you can: Extract each channel of the panoramic image to obtain the outline of each fire situation in the panoramic image; According to the outline of each fire situation in the panoramic image, the center point of each fire situation in the panoramic image is calculated; According to the center point of each fire in the panoramic image and the center point of the panoramic image, the corresponding smoke and fire distribution index is calculated.

[0058] The panoramic image of the object to be identified is usually a color image, and the channels of the panoramic image of the object to be identified are the red, green and blue channels of the color image.

[0059] It should be understood that the outline of a fire is the outline of an open flame and does not include smoke. If a smoke fire contains an open flame and smoke, the extracted outline of the fire is the outline of the open flame. If a smoke fire does not contain an open flame but only smoke, it means that the generated smoke has blocked the open flame, and the outline of the smoke is extracted as the outline of the fire.

[0060] It should be noted that if a smoke fire in the panoramic image only has smoke but no open fire, the position of the lower left corner of the outline of the smoke fire is taken as the center point of the fire.

[0061] When extracting each channel of the panoramic image and extracting the outline of each fire in the panoramic image, each smoke fire in the panoramic image of the object to be identified can be identified first, and the category of each smoke fire can be determined. The category of the smoke fire can be smoke or open fire. It can be understood that when the category of the smoke fire is smoke, there is no open fire in the smoke fire; when the category of the smoke fire is open fire, there is open fire in the smoke fire, and there may be smoke or no smoke; then, combined with the category of each smoke fire, the outline of each fire in the panoramic image is extracted, wherein when the category of the smoke fire is smoke, the smoke fire is extracted as the outline of the fire with the outline of the smoke; when the category of the smoke fire is open fire, the smoke fire is extracted as the outline of the fire with the outline of the open fire; in this way, the accuracy and comprehensiveness of the outline of each fire in the extracted panoramic image can be improved.

[0062] By extracting each channel of the panoramic image, omissions during extraction can be effectively avoided, and smoke can be accurately excluded in areas containing open flames and smoke to prevent smoke from interfering with the selection of fire contours. The contour of each fire in the panoramic image can be obtained accurately and relatively quickly, ensuring the accuracy of the center point of each fire in the panoramic image, thereby improving the accuracy of the calculated smoke and fire distribution index.

[0063] In this embodiment, according to the smoke and fire characteristics, identifying the smoke and fire level of the object to be identified may include: The corresponding smoke and fire hazard index is calculated based on the number of smoke and fire conditions, the maximum area ratio of local fire conditions in the corresponding captured image, the total area ratio of the total fire area in the panoramic image, and the smoke and fire distribution index; According to the corresponding smoke and fire hazard index, the smoke and fire level of the object to be identified is identified.

[0064] In this embodiment, the smoke and fire level of the object to be identified can be quickly identified according to the preset interval threshold of the smoke and fire hazard index.

[0065] As an optional implementation, when calculating the corresponding smoke and fire hazard index, the corresponding smoke and fire hazard index can be calculated by the following calculation formula: ; in, Indicates the number of smoke and fire conditions. Indicates the maximum area proportion of the local fire in the corresponding captured image. Indicates the proportion of the total fire area in the panoramic image. represents the smoke and fire distribution index, , , and represents the weight coefficient, represents the weighted correction coefficient, Indicates the smoke and fire hazard index.

[0066] Specifically, the smoke and fire distribution index 𝐷 can reflect the degree of dispersion of different smoke and fire conditions. The larger the smoke and fire distribution index, the higher the degree of dispersion of different smoke and fire conditions. The weighted correction coefficient 𝐶 is used as the range of the modified value. For example, the weighted correction coefficient 𝐶 can be set to 10 to amplify the value so that the value can reach a reasonable range, which is conducive to the identification of the smoke and fire level of the object to be identified.

[0067] In this calculation method, different smoke and fire characteristics are assigned different weight coefficients, which also represent the importance of different smoke and fire characteristics. In practical applications, the importance of smoke and fire characteristics of different objects to be identified may be different. The weight coefficients of different smoke and fire characteristics can be adjusted according to the type of object to be identified, so as to more accurately calculate the smoke and fire hazard index of the corresponding object to be identified.

[0068] Specifically, when calculating the corresponding smoke and fire hazard index, the weight coefficients of different smoke and fire characteristics can be obtained in the following manner: the type of the object to be identified is obtained based on the panoramic image recognition of the object to be identified, and the characteristics of the object to be identified are extracted. The characteristics of the object to be identified may include the inherent characteristics of the object to be identified and the environmental characteristics of the object to be identified. Taking a building as an example, the inherent characteristics of the object to be identified are the characteristics of the building, and the environmental characteristics of the object to be identified may be the characteristics of the natural environment around the building; further, according to the type, characteristics and smoke and fire characteristics of the object to be identified, the development of the smoke and fire of the object to be identified is simulated within a predetermined time, and the weight coefficients of different smoke and fire characteristics are determined. When determining, the weight coefficients are determined based on the degree of danger of different smoke and fire characteristics on the object to be identified, so that the weight coefficients of different smoke and fire characteristics used are more reasonable and accurate, thereby improving the accuracy of the calculated smoke and fire hazard index.

[0069] Through this calculation method, the importance of different smoke and fire characteristics is fully considered, and the smoke and fire hazard index of the object to be identified can be obtained more reasonably. In addition, the weighted correction coefficient is used to make the value amplified to a reasonable range, and the smoke and fire hazard index of the object to be identified is corrected twice, so that the calculated smoke and fire hazard index of the object to be identified has higher accuracy.

[0070] Optionally, when calculating the smoke and fire distribution index, the corresponding smoke and fire distribution index may be calculated by the following calculation formula: ; Among them, 𝑁 represents the number of smoke fires, Indicates the center point of each different fire situation, represents the center point of the panoramic image, B represents the width of the panoramic image, H represents the height of the panoramic image, and 𝐷 represents the smoke and fire distribution index.

[0071] This calculation method effectively and reasonably links and applies various smoke and fire conditions. At the same time, combined with the width and height of the panoramic image, the calculated smoke and fire distribution index can more accurately reflect the discrete degree of different smoke and fire conditions, thereby improving the accuracy of the calculated smoke and fire hazard index, thereby improving the recognition accuracy of the smoke and fire level of the object to be identified.

[0072] In order to execute the methods corresponding to the above embodiments and achieve corresponding functions and technical effects, a smoke and fire identification system from the perspective of a drone is provided below.

[0073] See also Figure 2 , Figure 2 This is a structural block diagram of the smoke and fire identification system from the perspective of a drone provided in an embodiment of the present application.

[0074] The smoke and fire identification system from the perspective of a drone provided in the embodiment of the present application includes: An acquisition module 210 is used to acquire multiple photographed images of the object to be identified; The detection module 220 is used to input the multiple captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be identified; The feature extraction module 230 is used to obtain corresponding smoke fire characteristics according to a plurality of captured images when there is a fire in the object to be identified, and the smoke fire characteristics at least include smoke fire quantity characteristics and smoke fire area characteristics; The identification module 240 is used to identify the smoke and fire level of the object to be identified according to the smoke and fire characteristics.

[0075] The smoke and fire recognition system from the perspective of a drone in the embodiment of the present application can quickly and accurately determine whether there is a fire in the object to be identified by inputting the acquired multiple captured images into a preset smoke and fire detection model. The use of multiple captured images can effectively reduce the false detection rate. When there is a fire in the object to be identified, corresponding smoke and fire characteristics are obtained based on the multiple captured images. The smoke and fire characteristics at least include smoke and fire quantity characteristics and smoke and fire area characteristics. Then, the smoke and fire level of the object to be identified is identified through the smoke and fire quantity characteristics and smoke and fire area characteristics. The smoke and fire quantity characteristics and smoke and fire area characteristics can reflect the situation of the smoke and fire more specifically and clearly, thereby timely and accurately reflecting the actual situation of the smoke and fire occurring in the object to be identified.

[0076] In one embodiment, the feature extraction module 230 may be specifically used for: Performing image synthesis based on multiple captured images to obtain a panoramic image of the object to be identified; According to the panoramic image, the corresponding smoke and fire characteristics are obtained.

[0077] As an optional implementation, when the feature extraction module 230 obtains the corresponding smoke and fire distribution index according to the panoramic image, it can be specifically used to: Extract each channel of the panoramic image to obtain the outline of each fire situation in the panoramic image; According to the outline of each fire situation in the panoramic image, the center point of each fire situation in the panoramic image is calculated; According to the center point of each fire in the panoramic image and the center point of the panoramic image, the corresponding smoke and fire distribution index is calculated.

[0078] As an optional implementation, the feature extraction module 230 may also be specifically used for: Acquire at least one fire situation image from a plurality of captured images; The local area ratio of the local fire in each fire image in the corresponding image is calculated, and the local area ratio of the fire image with the largest local area ratio is taken as the maximum area ratio of the local fire in the corresponding image.

[0079] In one embodiment, the identification module 240 may be specifically configured to: The corresponding smoke and fire hazard index is calculated based on the number of smoke and fire conditions, the maximum area ratio of local fire conditions in the corresponding captured image, the total area ratio of the total fire area in the panoramic image, and the smoke and fire distribution index; According to the corresponding smoke and fire hazard index, the smoke and fire level of the object to be identified is identified.

[0080] The smoke and fire identification system under the perspective of a drone can implement the smoke and fire identification method under the perspective of a drone described above. The specific limitations and other contents of the embodiment of the smoke and fire identification system under the perspective of a drone described above can be found in the contents of the smoke and fire identification method under the perspective of a drone described above, and will not be repeated in the embodiment.

[0081] An embodiment of the present application also provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the above-mentioned smoke and fire identification method from the perspective of a drone.

[0082] Optionally, the electronic device mentioned above may be a computer device such as a server.

[0083] In one embodiment, the internal structure of the electronic device of the present application can be as follows: Figure 3 shown.

[0084] An embodiment of the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned method for identifying smoke and fire from the perspective of a drone.

[0085] In several embodiments provided in the present application, it should be understood that the disclosed device / system and method can also be implemented in other ways. The device / system embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the device / system, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0086] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0087] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0088] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0089] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0090] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations; at the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A smoke and fire identification method from the perspective of a drone, characterized in that: include: Acquire multiple captured images of the object to be identified; Inputting the plurality of captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be identified; When there is a fire in the object to be identified, corresponding smoke fire characteristics are obtained according to the multiple captured images, wherein the smoke fire characteristics at least include a smoke fire quantity characteristic and a smoke fire area characteristic; The smoke and fire level of the object to be identified is identified according to the smoke and fire characteristics.

2. The smoke and fire identification method from the perspective of a drone according to claim 1 is characterized in that: The step of obtaining corresponding smoke and fire characteristics according to the plurality of captured images includes: Performing image synthesis based on the multiple captured images to obtain a panoramic image of the object to be identified; According to the panoramic image, corresponding smoke and fire characteristics are obtained.

3. The smoke and fire identification method from the perspective of a drone according to claim 2 is characterized in that: The smoke and fire characteristics include smoke and fire quantity characteristics, smoke and fire area characteristics and smoke and fire distribution characteristics.

4. The smoke and fire identification method from the perspective of a drone according to claim 3 is characterized in that: The smoke and fire quantity feature is the smoke and fire quantity; the smoke and fire area feature includes the maximum area ratio of the local fire in a single image among multiple images in the corresponding image and the total area ratio of the total fire area in the panoramic image; the smoke and fire distribution feature is the smoke and fire distribution index.

5. The smoke and fire identification method from the perspective of a drone according to claim 4 is characterized in that: The step of identifying the smoke and fire level of the object to be identified according to the smoke and fire characteristics includes: Calculate the corresponding smoke and fire hazard index according to the number of smoke and fire, the maximum area ratio of the local fire in the corresponding captured image, the total area ratio of the total fire area in the panoramic image, and the smoke and fire distribution index; The smoke and fire level of the object to be identified is identified according to the corresponding smoke and fire hazard index.

6. The smoke and fire identification method from the perspective of a drone according to claim 5 is characterized in that: The corresponding smoke and fire hazard index is calculated by the following formula: ； in, Indicates the number of smoke and fire conditions. Indicates the maximum area proportion of the local fire in the corresponding captured image. Indicates the proportion of the total fire area in the panoramic image. represents the smoke and fire distribution index, , , and represents the weight coefficient, represents the weighted correction coefficient, Indicates the smoke and fire hazard index.

7. The smoke and fire identification method from the perspective of a drone according to any one of claims 4 to 6, characterized in that: According to the panoramic image, a corresponding smoke and fire distribution index is obtained, including: Extracting each channel of the panoramic image to obtain the outline of each fire situation in the panoramic image; Calculating the center point of each fire in the panoramic image according to the outline of each fire in the panoramic image; According to the center point of each fire in the panoramic image and the center point of the panoramic image, a corresponding smoke and fire distribution index is calculated.

8. The smoke and fire identification method from the perspective of a drone according to claim 7, characterized in that: The corresponding smoke and fire distribution index is calculated by the following formula: ； Among them, 𝑁 represents the number of smoke fires, Indicates the center point of each different fire situation, represents the center point of the panoramic image, B represents the width of the panoramic image, H represents the height of the panoramic image, and 𝐷 represents the smoke and fire distribution index.

9. The smoke and fire identification method from the perspective of a drone according to claim 4, characterized in that: The step of obtaining corresponding smoke and fire characteristics according to the plurality of captured images further comprises: Acquire at least one fire situation photographed image from the plurality of photographed images; The local area ratio of the local fire in each fire image in the corresponding image is calculated, and the local area ratio of the fire image with the largest local area ratio is taken as the maximum area ratio of the local fire in the corresponding image.

10. A smoke and fire identification system from the perspective of a drone, characterized in that: include: An acquisition module, used to acquire multiple photographed images of the object to be identified; A detection module, used to input the plurality of captured images into a preset smoke and fire detection model to determine whether there is a fire in the object to be identified; A feature extraction module, used for obtaining corresponding smoke fire characteristics according to the plurality of captured images when there is a fire in the object to be identified, wherein the smoke fire characteristics at least include a smoke fire quantity characteristic and a smoke fire area characteristic; The identification module is used to identify the smoke and fire level of the object to be identified according to the smoke and fire characteristics.

11. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for identifying smoke and fire from the perspective of a drone according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed by a processor, implements the method for identifying smoke and fire from the perspective of a drone as described in any one of claims 1 to 9.

Images