Forest smoke and fire detection method and equipment based on inspection view angle of unmanned aerial vehicle, and medium

Abstract

The invention provides a forest smoke and fire detection method and device based on the inspection view angle of an unmanned aerial vehicle and a medium, and belongs to the technical field of smoke and fire detection. Inputting the visual image into a smoke and fire detection model to obtain a smoke and fire detection result; wherein the smoke and fire detection model is obtained by training in a mode of combining a generalized focus loss function and a complete intersection-to-union ratio loss function. According to the forest smoke and fire detection method provided by the embodiment of the invention, the obtained smoke and fire detection model is adopted for detection, early recognition can be performed on forest fire with higher precision and faster speed, fire loss can be reduced, and the ecological environment can be better protected.

Description

Technical Field

[0001] The present invention relates to the technical field of smoke and fire detection, and in particular to a method, device and medium for detecting forest smoke and fire based on the inspection perspective of an unmanned aerial vehicle. Background Art

[0002] Forest fires can cause environmental damage and economic losses. Fires not only lead to large-scale vegetation loss, but also have long-term impacts on biodiversity, ecosystem balance, and regional climate. In addition, the large amount of greenhouse gases such as carbon dioxide produced by forest fires is one of the important driving factors of global warming, so early detection and rapid response to forest fires are of great significance.

[0003] Traditional forest fire monitoring methods usually rely on ground monitoring stations and satellite remote sensing. However, the ground monitoring station method has limitations in coverage and real-time performance, making it difficult to obtain early fire signals comprehensively and quickly. Although satellite remote sensing can provide monitoring over a larger range, it is limited by the imaging cycle and spatial resolution and cannot provide sufficiently accurate fire information in the early stages of a fire. These technical limitations often result in fires being out of control when they are discovered, increasing the difficulty of firefighting.

[0004] With the rapid development of drone technology, the use of drones to detect fires has become a new technology for forest fire monitoring. Drones have the ability to respond quickly, be flexible, have high-altitude viewing angles, and detect multiple spectra. They can accurately identify the characteristics of fireworks and fires in the early stages of a fire and obtain detailed information about the fire in a timely manner. However, due to the complexity of the forest environment and the instability of the drone inspection perspective, there are still many challenges in achieving effective fireworks and fire detection. Summary of the invention

[0005] In view of this, the present invention provides a forest fire detection method, equipment and medium based on the inspection perspective of an unmanned aerial vehicle, which can identify forest fires at an early stage with higher accuracy and faster speed, and is of great significance for reducing fire losses and protecting the ecological environment.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] According to an embodiment of the present invention, a real-time forest fire detection method based on a drone inspection perspective includes:

[0008] Acquire real visual images taken during the drone inspection process; input the visual images into a fireworks detection model to obtain fireworks detection results; wherein the fireworks detection model is trained by combining a generalized focal loss function and a complete intersection-over-union loss function.

[0009] As an embodiment of the first aspect of the present invention, the training method of the smoke and fire detection model includes:

[0010] Construct a fireworks detection dataset from the perspective of drone inspection;

[0011] Amplifying the fireworks detection data set;

[0012] Build a deep object detection model;

[0013] The generalized focal loss function and the complete intersection-over-union loss function are combined, and the deep object detection model is trained based on the fireworks detection dataset to obtain the fireworks detection model.

[0014] As an embodiment of the first aspect of the present invention, the construction of a fireworks detection dataset from a drone inspection perspective includes:

[0015] Acquire images containing trees and fireworks by searching and crawling the Internet, wherein the images of fireworks at least include images containing smoke, images containing fire, and images containing smoke and fire;

[0016] The images are annotated by manual annotation and model-assisted annotation methods to obtain the fireworks detection dataset.

[0017] As an embodiment of the first aspect of the present invention, the amplification of the fireworks detection data set includes at least one method selected from the group consisting of geometric transformation, pixel transformation, mosaic enhancement, mixup enhancement, and cutting and mixing.

[0018] As an embodiment of the first aspect of the present invention, the deep object detection model includes:

[0019] A backbone network, wherein the backbone network is formed by stacking a plurality of convolutional layers and a plurality of dual-feature convolutional modules C2f, and the backbone network is used to extract features of fireworks in the image;

[0020] A feature pyramid fusion network is used to perform multi-scale feature fusion on the features of the fireworks;

[0021] The detection network is used to extract information from the feature pyramid fusion network and determine the category and position information of the image.

[0022] As an embodiment of the first aspect of the present invention, the detection network includes three detection head modules, each of which extracts information and detects different feature layers of the feature pyramid fusion network.

[0023] As an embodiment of the first aspect of the present invention, the training of the deep object detection model based on the smoke and fire detection dataset includes:

[0024] Input the labeled image containing the target fireworks and the corresponding position into the deep object detection model, and train the parameters in the network through the stochastic gradient descent optimizer;

[0025] At least combine the average accuracy, precision, and recall indicators to quantitatively evaluate the training results.

[0026] As an embodiment of the first aspect of the present invention, the training of the deep object detection model based on the smoke and fire detection dataset further includes:

[0027] The deep object detection model is optimized based on the evaluation results, network hyperparameter adjustment and hard negative sample mining.

[0028] In a second aspect, the present invention further discloses an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor for executing the detection method described in the embodiment of the first aspect.

[0029] In a third aspect, the present invention further discloses a computer-readable storage medium, on which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the detection method described in the embodiment of the first aspect.

[0030] The above technical solution of the present invention has at least one of the following beneficial effects:

[0031] According to the forest fire detection method based on the perspective of drone inspection in the embodiment of the present invention, a drone is used to patrol in the forest to capture real visual images for detection, which avoids the problem of low resolution of remote sensing images. The presence of fireworks can be clearly distinguished in the visual image, which helps the model to accurately identify fireworks in the captured image. The method can retain subtle fireworks features in the visual image, which helps to detect forest fires at an early stage, thereby effectively solving the problem that remote sensing images are difficult to capture early signs of fire. In addition, the visual image is input into the fireworks detection model for detection and identification. The fireworks detection model is trained by combining the generalized focal loss function and the complete intersection-over-union loss function, and targeted optimization is performed for the imbalance problem of positive and negative samples and the feature matching of the fireworks area, so that the robustness and performance of fireworks detection can be improved more accurately, as well as the overall efficiency and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of a forest fire detection method based on a drone inspection perspective according to an embodiment of the present invention;

[0033] Figure 2 A flow chart showing a method for training a smoke and fire detection model according to an embodiment of the present invention is shown;

[0034] Figure 3 A flow chart showing a method for training another smoke and fire detection model according to an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of the structure of image stitching enhancement according to an embodiment of the present invention is shown;

[0036] Figure 5 A schematic diagram of the structure of image cutting and mixing according to an embodiment of the present invention is shown;

[0037] Figure 6 A schematic diagram showing the structure of a deep object detection model according to an embodiment of the present invention is shown;

[0038] Figure 7 A block diagram of a SoC system on chip according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0040] The following first describes in detail the forest smoke and fire detection method based on the inspection perspective of a drone according to an embodiment of the present invention with reference to the accompanying drawings.

[0041] Figure 1 FIG. 1 is a flow chart showing a method for detecting forest smoke and fire based on the inspection perspective of an unmanned aerial vehicle according to an embodiment of the present invention. Figure 1 As shown, a forest fire detection method based on a drone inspection perspective according to an embodiment of the present invention may include:

[0042] S110, obtaining real visual images captured during the drone inspection process.

[0043] This type of visual image has a higher resolution, which is more conducive to clearly distinguishing whether there are fireworks. It helps the model accurately identify fireworks in the captured images and can retain subtle fireworks features in the visual images, which helps to detect forest fires at an early stage, thus effectively solving the problem that remote sensing images are difficult to capture early signs of fire.

[0044] In some embodiments, the drone can be equipped with cameras and sensors, such as one or more of a high-resolution visible light camera, a three-dimensional lidar scanning device, a multispectral imager, and a monocular camera.

[0045] In one embodiment of the present invention, it is also necessary to perform standardization processing on the visual image so that the visual image to be detected is more consistent with the data set during the training of the model, which is conducive to improving the accuracy of detection.

[0046] S120, inputting the visual image into a fireworks detection model to obtain a fireworks detection result.

[0047] Among them, the fireworks detection model is trained by combining the generalized focal loss (GFL) and the complete intersection over union loss (CIoU Loss). Since the early fireworks in the image only occupy a small part of the entire image, and most of the image is a non-fireworks background, the detection box output by the network is mostly the background rather than the fireworks, so there is an imbalance between positive and negative samples. In this regard, GFL is introduced. Compared with the traditional focal loss, GFL is more adaptable to diverse and complex detection scenarios, especially in the case of imbalanced categories and unclear target areas. GFL increases the focus on "distinguishing between positive and negative samples", so that the network pays more attention to the features that distinguish positive and negative samples. During training, not only the classification is weighted, but also the position prediction is jointly optimized. At the same time, according to the uncertainty of each sample, higher weights are given to samples with lower confidence. This mechanism helps the model pay more attention to difficult samples and improves the overall accuracy of the model.

[0048] CIoU Loss mainly solves the shortcomings of the original IoU loss function in bounding box regression, especially when dealing with objects with mismatched shapes and large position deviations. On the one hand, the Euclidean distance between the center points of the boxes is introduced as part of the loss. This helps to optimize the center position of the target so that the regression does not only rely on the intersection of the boxes, but also considers the spatial layout and relative position of the boxes. By penalizing the distance between the center points of the bounding boxes, the center position of the predicted box is closer to the true box. On the other hand, an additional term is introduced to measure the shape consistency (such as aspect ratio) between the two bounding boxes. This allows it to better optimize the aspect ratio of the box during the regression process, rather than just the position and size, thereby avoiding the problem of aspect ratio mismatch.

[0049] The forest smoke and fire detection method based on the drone inspection perspective of the embodiment of the present invention can effectively improve the accuracy of forest smoke and fire detection.

[0050] It can be understood that the method of the embodiment of the present invention can be applied to electronic devices, including but not limited to mobile phones, smart TVs, wearable devices, tablet computers, desktop computers, computers, wearable devices (for example, smart watches, smart glasses, helmets), virtual reality (VR) devices, augmented reality (AR) devices, wireless devices in industrial control, wireless devices in self-driving, wireless devices in smart grids, wireless devices in transportation safety, wireless devices in smart cities, wireless devices in smart homes, servers, and the like.

[0051] The training process of the smoke and fire detection model according to the embodiment of the present invention is described in detail below in conjunction with specific embodiments.

[0052] refer to Figure 2 and Figure 3 , Figure 2 A flow chart of a method for training a smoke and fire detection model according to an embodiment of the present invention is shown. Figure 3 FIG. 2 is a flow chart showing another method for training a smoke and fire detection model according to an embodiment of the present invention. Figure 2 As shown, the training method includes:

[0053] S210, construct a fireworks detection dataset from the perspective of drone inspection.

[0054] This step is the data preparation process. Considering that there is no large-scale open source data set in the application scenario of fireworks detection from the perspective of drone inspection, the data is relatively small and cannot meet the training needs. Therefore, in the embodiment of the present invention, a large number of images with fireworks are obtained through a large number of network searches and crawling. Among them, the images of fireworks at least include images containing smoke, images containing fire, and images containing smoke and fire. The crawled fireworks images are labeled by manual labeling and model-assisted labeling methods to construct a larger-scale fireworks detection data set to meet the problem of insufficient existing scene data. This method not only avoids the limitations that may be caused by generating virtual data, but also avoids the forest damage and economic losses that may be caused by shooting real fireworks. This step corresponds to Figure 3 S310 in Data Preparation: Collect fireworks images from the inspection perspective of drones to obtain a dataset.

[0055] Among them, it is also necessary to process the visual image

[0056] In one embodiment of the present invention, the visual image needs to be processed before input: First, the mean and variance statistics of the training data set: according to the RGB channels of the image, the mean and variance of each channel of all training data are counted. Second, data standardization: for training data, verification data and test data, the corresponding mean is subtracted from each channel and divided by the corresponding variance for standardization.

[0057] S220, amplifying the fireworks detection dataset.

[0058] In order to further make up for the lack of data sets, data augmentation is used to expand the data sets so that the model training is more sufficient. By augmenting, the size and diversity of the data sets can be artificially increased, thereby helping the model to better generalize to unseen data. For example, in image recognition tasks, new training samples can be generated by operations such as rotation, flipping, scaling, and cropping. This step corresponds to Figure 3 Corresponding to S320 in , that is, data enhancement, the methods that can be used include geometric change, pixel transformation, mosaic enhancement (Mosaic), mixed enhancement (Mixup) and cutting mixing (CutMix).

[0059] In an embodiment of the present invention, a dataset may be amplified by combining one or more of the following amplification methods.

[0060] The first type is geometric transformation. Random cropping randomly crops some areas from the original image for training; random flipping flips the image horizontally to enhance the model's robustness to changes in the target's direction; random scaling randomly scales the size of the image to allow the model to adapt to targets of different sizes.

[0061] The second is pixel transformation. Adjust the hue, saturation, and brightness (Hue, Saturation, Value, HSV) contrast, saturation, and brightness.

[0062] The third type is Mosaic enhancement.

[0063] like Figure 4 As shown in (a) of the figure, there are four images of the original data, such as image 1, image 2, image 3 and image 4. The four images are stitched together to form a new image, as shown in Figure 4 (b) shows the data after mosaic enhancement. This method can help the model learn the contextual relationship between different targets.

[0064] The fourth type is Mixup.

[0065] Mix two images and their labels in proportion to generate new training samples (datasets).

[0066] The Mixup formula is as follows:

[0067]

[0068]

[0069] λ~Beta(α,α)

[0070] In the formula, x1 and x2 represent the pixel values ​​of the two images, y1 and y2 represent the label values ​​of the two images, is the pixel value of the new image, is the label value of the new image. λ is the value sampled from the Beta distribution, and α is the parameter of the Beta distribution.

[0071] The fifth type is CutMix.

[0072] like Figure 5 As shown in the figure, part of the image 1 is cut out and pasted to the corresponding position of another image 2, and the label is adjusted to reflect the contribution ratio of the two images. This method can help the model learn more robust feature representation and effectively reduce the risk of overfitting.

[0073] The embodiments of the present application increase the diversity and amount of samples through a variety of data augmentation techniques, further enrich the training data, and improve the classification accuracy and adaptability of the model to complex scenarios. The combination of these methods helps the model learn effective features more robustly under diverse training data.

[0074] S230, construct a deep object detection model.

[0075] refer to Figure 6 , Figure 6 FIG. 2 shows a schematic diagram of the structure of a deep object detection model according to an embodiment of the present invention. Figure 6 As shown in the figure, the deep target detection model can be a YOLOv8 network, which includes three parts: the backbone network, the feature pyramid fusion network and the detection network. Among them, the backbone network is composed of multiple convolutional layers and multiple C2f modules stacked together to extract the fireworks features in the image. By constructing the C2f module, it has a smaller parameter amount compared to the traditional ResNet and CSPNet. At the same time, the various modules of the model are optimized. Due to the C2f parameter amount: Conv1: C×C×1×1+C=C 2 +C, Bottleneck: Conv2: C×C×1×1+C=C 2 +C and The amount of calculation is only three corresponding calculations and one total calculation. Compared with the five calculations and one total parameter calculation of CSP, the C2f module in the embodiment of the present invention can reduce the number of parameters and calculations of each module in the model, so that the model can run more efficiently, speed up the detection time, and realize real-time detection.

[0076] The feature pyramid fusion network fuses the multi-scale features of fireworks. After multiple upsampling and downsampling, the detection network outputs feature layers of each scale to determine the category (smoke, fire or background) and location information.

[0077] In some embodiments, the detection network may include three detection head modules (detect), which detect feature layers of three different scales. The detection head module can design two sub-networks to more accurately determine the category (smoke, fire or background) and location information of fireworks.

[0078] S240, model training and optimization.

[0079] The constructed deep object detection model is trained and optimized to adapt to the characteristic distribution of forest fires.

[0080] In an embodiment of the present invention, a generalized focal loss function and a complete intersection-over-union loss function are combined, and the deep object detection model is trained based on the fireworks detection dataset to obtain the fireworks detection model.

[0081] In the embodiments of the present application, as mentioned above, a generalized focal loss Generalized FocalLoss is introduced. Compared with traditional Focal Loss, GFL is more adaptable to diverse and complex detection scenarios, especially in the case of class imbalance and unclear target area. GFL increases the focus on "distinguishing between positive and negative samples", so that the network pays more attention to the features that distinguish positive and negative samples. During training, not only the classification is weighted, but also the position prediction is jointly optimized. At the same time, according to the uncertainty of each sample, higher weights are given to samples with lower confidence. This mechanism helps the model pay more attention to difficult samples and improves the overall accuracy of the model. The formulas of GFL for classification loss and position loss are as follows:

[0082] Classification GFL = -α·(1-p) γ ·log(p)

[0083] Position GFL = |yp|((1-p) γ ·log(p)+(p) γ log(1-p))

[0084] Where α and γ are two hyperparameters that control the weight balance of positive and negative samples and the attention paid to error-prone samples respectively. y is the label of the sample, and p is the probability value predicted by the model.

[0085] CIoU Loss is introduced for the shape optimization of the detection box. As mentioned above, CIoU Loss mainly solves the shortcomings of the original IoU loss function in bounding box regression, especially when dealing with objects with mismatched shapes and large position deviations. It enables it to better optimize the aspect ratio of the box during the regression process, rather than just the position and size, thus avoiding the problem of aspect ratio mismatch.

[0086] The formula of CIoU Loss is as follows:

[0087]

[0088]

[0089] Where IoU represents the IoU value between the predicted box and the true box, that is, the ratio of the overlapping area to the union area. 2 (b,b g ) represents the Euclidean distance between the center point of the predicted box and the real box, b, b g Denote the predicted box and the real box, ν is a parameter to measure the consistency of aspect ratio, and c is the length of the diagonal of the minimum enclosing box containing the predicted box and the real box. Where w and h are the width and height of the predicted box, respectively. g and h g is the width and height of the real box.

[0090] GFL is used to process the characteristics of fireworks targets that account for a small proportion of the image and a large amount of background, so that the model can pay more attention to the important features that distinguish positive and negative samples. Compared with traditional Focal Loss, GFL is more suitable for scenes with unbalanced categories and unclear targets, and gives higher weights to samples with lower confidence, thereby improving the overall accuracy of the model. Secondly, when optimizing the shape of the detection box, CIoU Loss not only considers the intersection of the box and the union of the box, but also introduces the Euclidean distance and aspect ratio consistency of the center point of the box. These optimization items improve the position and shape of the box, so that the predicted box can more accurately match the characteristics of the fireworks area. Combining these optimization measures, the model can more effectively adapt to the complex feature distribution of forest fire detection.

[0091] In the embodiment of the present application, the training process of the model is to first mark the location of fireworks in the fireworks data collected from the inspection perspective of the drone, input the image containing the target fireworks and the corresponding location label into the network, and train the parameters in the network through the stochastic gradient descent optimizer.

[0092] In an embodiment of the present application, during the model training process, the model also needs to be evaluated. For example, the training results can be quantitatively evaluated by combining indicators such as average accuracy, precision, and recall.

[0093] After the training is completed, combined with the evaluation results, the model is optimized through network hyperparameter adjustment, hard negative sample mining, etc., so that a more accurate fireworks detection model can be obtained, which is conducive to the early detection of fireworks.

[0094] S250, output the detection result.

[0095] There are many detection frames output by the deep object detection model, and it is necessary to select the accurate detection frames of fireworks. First, all detection frames are screened by the non-maximum suppression method to obtain the predicted targets with high confidence, and the repeated predicted targets with low confidence are removed. The location and category of the fireworks are marked on the original image, and finally output.

[0096] As shown in Table 1, the model has a good prediction effect on fireworks and can accurately and effectively identify fireworks in the forest. The results of the model prediction evaluation index are as follows:

[0097] Table 1 Model prediction evaluation results

[0098] category Number of images Target quantity Accuracy Recall mAP50 mAP50-95 all 2344 4297 0.872 0.85 0.887 0.666 fire 963 1649 0.853 0.81 0.855 0.599 cigarette 1851 2648 0.89 0.89 0.919 0.732

[0099] It can be seen from the results in Table 1 that the detection accuracy is very high, indicating that the method of the embodiment of the present invention can accurately detect fireworks.

[0100] It should be noted that the training method of the above-mentioned fireworks detection model and the forest fireworks detection method based on the perspective of drone inspection can be run in different electronic devices. For example, the fireworks detection model can be trained in a first electronic device and then sent to a second electronic device. When the second electronic device detects forest fireworks, it can directly use the trained fireworks detection model to detect forest fireworks. In addition, the training method and the detection method can be run in the same device. For example, after the drone inspection, the real visual image taken is obtained, and the image is sent to an electronic device that stores the fireworks detection model. The electronic device runs the fireworks detection model to detect and judge the visual image. The present application does not limit the operation mode of the execution subject.

[0101] The present application also provides an electronic device, comprising:

[0102] a memory for storing instructions to be executed by one or more processors of the device, and

[0103] A processor is used to execute the above embodiment in combination with Figures 1 to 6 The method explained.

[0104] The present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the above-mentioned embodiment. Figures 1 to 6 The method explained.

[0105] The present application also provides a computer program product comprising instructions, which, when executed on an electronic device, enables a processor to execute the above-mentioned embodiments. Figures 1 to 6 The method explained.

[0106] Reference now Figure 7 , which is a block diagram of a SoC (System on Chip) 700 according to an embodiment of the present application. Figure 7 In the FIG, similar components have the same reference numerals. In addition, the dashed boxes are optional features of more advanced SoCs. Figure 7 In the embodiment, SoC 700 includes: an interconnect unit 750, which is coupled to an application processor 710; a system agent unit 780; a bus controller unit 790; an integrated memory controller unit 740; a group or one or more coprocessors 720, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 730; and a direct memory access (DMA) unit 760. In one embodiment, the coprocessor 720 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, a general purpose computing on GPU (GPGPU), a high throughput integrated multi-core architecture (MIC) processor, or an embedded processor.

[0107] The static random access memory (SRAM) unit 730 may include one or more computer-readable media for storing data and / or instructions. The computer-readable storage medium may store instructions, specifically, temporary and permanent copies of the instructions. The instructions may include: when executed by at least one unit in the processor, the Soc 700 executes the processing method according to the above embodiment, which may be specifically referred to in the above embodiment. Figure 1 and Figure 6 The interaction method explained will not be repeated here.

[0108] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0109] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0110] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0111] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or a tangible machine-readable memory for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a machine (e.g., computer) readable form.

[0112] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0113] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0114] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0115] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. A forest fire detection method based on drone inspection perspective, characterized in that: include: Acquire visual images captured during the drone inspection process and perform standardization processing on the visual images; The standardized visual image is input into a fireworks detection model to obtain a fireworks detection result; wherein the fireworks detection model is trained by combining a generalized focal loss function and a complete intersection-over-union loss function.

2. The forest fire detection method based on the inspection perspective of an unmanned aerial vehicle according to claim 1 is characterized in that: The training method of the smoke and fire detection model includes: Construct a fireworks detection dataset from the perspective of drone inspection; Amplifying the fireworks detection data set; Build a deep object detection model; The generalized focal loss function and the complete intersection-over-union loss function are combined, and the deep object detection model is trained based on the fireworks detection dataset to obtain the fireworks detection model.

3. The forest fire detection method based on the inspection perspective of an unmanned aerial vehicle according to claim 2 is characterized in that: The fireworks detection dataset constructed from the perspective of drone inspection includes: Acquire images containing trees and fireworks by searching and crawling the Internet, wherein the images of fireworks at least include images containing smoke, images containing fire, and images containing smoke and fire; The images are annotated by manual annotation and model-assisted annotation methods to obtain the fireworks detection dataset.

4. The forest smoke and fire detection method based on the inspection perspective of an unmanned aerial vehicle according to claim 2 or 3, characterized in that: The amplification of the fireworks detection data set includes at least one method selected from the group consisting of geometric transformation, pixel transformation, splicing enhancement, mixing enhancement and cutting and mixing.

5. The forest smoke and fire detection method based on the inspection perspective of an unmanned aerial vehicle according to claim 2 or 3, characterized in that: The deep object detection model includes: A backbone network, wherein the backbone network is formed by stacking a plurality of convolutional layers and a plurality of dual-feature convolutional modules, and the backbone network is used to extract features of fireworks in the image; A feature pyramid fusion network is used to perform multi-scale feature fusion on the features of the fireworks; The detection network is used to extract information from the feature pyramid fusion network and determine the category and position information of the image.

6. The method for detecting forest smoke and fire based on the inspection perspective of an unmanned aerial vehicle according to claim 5, characterized in that: The detection network includes three detection head modules, each of which extracts information and detects different feature layers of the feature pyramid fusion network.

7. The forest fire detection method based on the inspection perspective of an unmanned aerial vehicle according to claim 3 is characterized in that: The training of the deep object detection model based on the smoke and fire detection dataset includes: Input the labeled image containing the target fireworks and the corresponding position into the deep object detection model, and train the parameters in the network through the stochastic gradient descent optimizer; At least combine the average accuracy, precision, and recall indicators to quantitatively evaluate the training results.

8. The method for detecting forest smoke and fire based on the inspection perspective of an unmanned aerial vehicle according to claim 7, characterized in that: The training of the deep object detection model based on the smoke and fire detection dataset further includes: The deep object detection model is optimized based on the evaluation results, network hyperparameter adjustment and hard negative sample mining.

9. An electronic device, comprising: A memory for storing instructions executed by one or more processors of an electronic device, and a processor for executing the detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the detection method according to any one of claims 1 to 8.

Images