Fire identification method and device, electronic equipment and storage medium

Abstract

The application provides a forest fire identification method and device, an electronic device and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a first target image of a to-be-identified region; performing an image identification operation on the first target image to obtain a first identification result; and determining a forest fire target of the to-be-identified region based on the first identification result. The forest fire identification method and device, the electronic device and the storage medium provided by the application can improve the forest fire identification accuracy.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and more specifically, relates to a forest fire identification method and device, electronic device, and storage medium. Background Technology

[0002] Dense forest vegetation and high combustible material loads make forest fires spread rapidly and easily become difficult to control. Effective forest fire identification enables early detection and response to fires, minimizing damage to forest resources, destruction of habitats, and air pollution, thereby avoiding casualties and significant property losses.

[0003] Most existing forest fire identification methods rely on single visible light images for identification, which are easily affected by factors such as clouds, haze, and changes in lighting, misidentifying non-fire targets as smoke and fire. The identification accuracy is difficult to meet the needs of actual forest fire prevention and monitoring. Summary of the Invention

[0004] The purpose of this application is to provide a forest fire identification method, apparatus, electronic device, and storage medium that can improve the accuracy of forest fire identification. To achieve the above objective, the technical solutions provided by this application are as follows: Firstly, a method for identifying forest fires is provided, including: Acquire the first target image of the region to be identified; An image recognition operation is performed on the first target image to obtain a first recognition result; Based on the first identification result, determine the forest fire targets in the area to be identified; The steps of the image recognition operation include: Multi-scale feature extraction is performed on the first target image to obtain feature maps at multiple scales; Global average pooling is performed on the feature maps at each scale to obtain the scene feature vectors at each scale; After concatenating the scene feature vectors corresponding to multiple scales, the first convolutional layer is used for compression to obtain the scene fusion feature vector. The scene fusion feature vector is sequentially input into the fully connected layer and the normalization layer to obtain the dynamic weights for each scale; Based on the dynamic weights of each scale, feature maps of multiple scales are weighted and fused to obtain a fused feature map; The fused feature map is input into the three detection branches respectively to obtain the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence of each bounding box; The first recognition result is determined based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box.

[0005] Secondly, a forest fire identification device is provided, comprising: The image acquisition module is used to acquire the first target image of the region to be identified; The image recognition module is used to perform image recognition operations on the first target image to obtain a first recognition result; The forest fire identification module is used to determine forest fire targets in the area to be identified based on the first identification result; Specifically, during the execution of the image recognition operation, the image recognition module is used for: Multi-scale feature extraction is performed on the first target image to obtain feature maps at multiple scales; Global average pooling is performed on the feature maps at each scale to obtain the scene feature vectors at each scale; After concatenating the scene feature vectors corresponding to multiple scales, the first convolutional layer is used for compression to obtain the scene fusion feature vector. The scene fusion feature vector is sequentially input into the fully connected layer and the normalization layer to obtain the dynamic weights for each scale; Based on the dynamic weights of each scale, feature maps of multiple scales are weighted and fused to obtain a fused feature map; The fused feature map is input into the three detection branches respectively to obtain the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence of each bounding box; The first recognition result is determined based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box.

[0006] Thirdly, embodiments of this application also provide an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the forest fire identification method provided by any possible implementation of the first aspect.

[0007] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the forest fire identification method provided by any possible implementation of the first aspect.

[0008] The beneficial effects of the technical solution provided in this application are as follows: This application's embodiments utilize multi-scale feature extraction to simultaneously capture features of small-scale, distant fire points, smoke, and large-area fires, avoiding missed detections due to insufficient feature extraction at a single scale. Based on this, global average pooling is performed on the feature maps at each scale, followed by feature concatenation and convolutional compression to obtain a scene fusion feature vector. This effectively integrates scene information at various scales, eliminates redundant interference, and enhances the representational ability of fire features. The scene fusion feature vector is input into a fully connected layer for nonlinear transformation, and then passed through a normalization layer to output dynamic weights corresponding to the feature maps at each scale. Weighted summation of the multi-scale feature maps based on these dynamic weights automatically strengthens scale features more valuable for fire identification and weakens scale features with strong background interference, resulting in a fused feature map that integrates the advantages of multiple scales. Finally, the fused feature map is input into three detection branches, which independently complete target localization, category judgment, and confidence evaluation, reducing localization bias and misclassification.

[0009] Therefore, the method of this embodiment can effectively reduce the problems of false detection and missed detection caused by clouds, fog, changes in lighting, and complex backgrounds, accurately identify open flames and smoke targets, and greatly improve the accuracy of forest fire identification. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.

[0011] Figure 1 A flowchart of the forest fire identification method provided in the embodiments of this application; Figure 2 A schematic diagram illustrating the principle of the dynamic weight determination method provided in this application embodiment; Figure 3 A schematic diagram illustrating the calculation process of the attention module provided in an embodiment of this application; Figure 4 This is a structural block diagram of the forest fire identification device provided in the embodiments of this application; Figure 5 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.

[0013] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.” When describing multiple (two or more) items, if the relationship between the multiple items is not explicitly defined, the multiple items can refer to one, several or all of the multiple items. For example, the description of "parameter A includes A1, A2, A3" can be implemented as parameter A includes A1 or A2 or A3, or it can be implemented as parameter A includes at least two of the three items A1, A2 and A3.

[0014] This application provides a forest fire identification method, which can be executed by an electronic device, such as... Figure 1 As shown, the method may include: S101: Obtain the first target image of the region to be identified.

[0015] In this embodiment, the area to be identified refers to the geographical area where forest fire monitoring and fire identification need to be carried out, typically including key forest areas, forest fire prevention and control zones, nature reserves, forest farms, and surrounding mountainous and woodland areas prone to fire. The first target image can be a visible light image, which can be acquired by visible light imaging devices (such as cameras or webcams) deployed in the area to be identified. Specifically, a drone equipped with a visible light camera can be used to cruise and photograph the area to be identified along a preset flight path to obtain the first target image; alternatively, a fixed monitoring camera or pan-tilt camera can be used in the area to be identified to continuously collect images of the forest area within the monitoring range to obtain the first target image.

[0016] S102: Perform image recognition operation on the first target image to obtain the first recognition result.

[0017] The steps involved in image recognition include: Multi-scale feature extraction is performed on the first target image to obtain feature maps at multiple scales; Global average pooling is performed on the feature maps at each scale to obtain the scene feature vectors at each scale; After concatenating the scene feature vectors corresponding to multiple scales, the first convolutional layer is used for compression to obtain the scene fusion feature vector. The scene fusion feature vector is sequentially input into the fully connected layer and the normalization layer to obtain the dynamic weights for each scale; Based on the dynamic weights of each scale, feature maps of multiple scales are weighted and fused to obtain a fused feature map; The fused feature map is input into the three detection branches respectively to obtain the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence of each bounding box; The first recognition result is determined based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box.

[0018] In this embodiment, by performing image recognition operations on the first target image, the pixel information, texture features, and color features in the first target image can be analyzed to extract key features related to forest fires (such as the high temperature hue of flames and the gray scale distribution of smoke), and then output a first recognition result containing the forest fire bounding box and the bounding box category (such as smoke / open flame).

[0019] Specifically, multi-scale feature extraction can be performed on the first target image to generate feature maps of different scales. The feature maps of different scales correspond to targets of different scales, such as small fire spots, smoke areas, and large fires. Therefore, multi-scale feature extraction can simultaneously take into account local details and global semantic information, and improve adaptability to fires of different scales, such as those at long distances, those with occlusion, and those with different distances.

[0020] Based on this, such as Figure 2 As shown, global average pooling is performed on the feature maps at each scale to compress them into fixed-dimensional scene feature vectors. These vectors can represent the overall scene information of the image at that scale, highlighting key patterns such as forest background, vegetation, and smoke. The scene feature vectors from multiple scales are concatenated and then processed through a first convolutional layer (1x1 convolutional layer) for channel compression and information fusion, removing redundant information to obtain a compact and more representative scene fusion feature vector. Subsequently, the scene fusion feature vector is input into a fully connected layer for nonlinear transformation, and then through a normalization layer (e.g., a softmax normalization layer) to output the dynamic weights w1, w2, w3, and w4 corresponding to the feature maps at each scale.

[0021] By using dynamic weights to perform weighted summation of multi-scale feature maps, the system can automatically enhance scale features more valuable for fire detection based on the current scene content, while weakening scale features with strong background interference, resulting in a fused feature map that integrates the advantages of multi-scale information. Finally, the fused feature map is input into three detection branches to complete target localization, category determination, and confidence assessment, respectively, outputting bounding box coordinates, the category of each bounding box, and the confidence score of each bounding box. These bounding box coordinates, the category of each bounding box, and the confidence score of each bounding box can be used as the first recognition result for subsequent forest fire target identification.

[0022] In this embodiment, all three detection branches can be implemented using convolutional layers. In the target localization detection branch, positional features are extracted from the fused feature map using 1-2 convolutional layers. Based on the regional features of the fire target, the upper-left and lower-right coordinates of the bounding box are predicted, and the precise coordinate range of each bounding box is output, thus completing target localization. In the category judgment detection branch, category features are extracted from the fused feature map using 1-2 convolutional layers to distinguish between open flame, smoke, and background categories, and the target category corresponding to each bounding box is output, thus achieving fire type determination. In the confidence assessment detection branch, the probability of target presence is predicted from the fused feature map using 1-2 convolutional layers, and a confidence score is output indicating that each bounding box contains a real fire target, used to characterize the reliability of the detection results.

[0023] S103: Determine the forest fire targets in the area to be identified based on the first identification result.

[0024] In this embodiment, a confidence threshold (e.g., 0.7) can be preset to filter out bounding boxes with a confidence level higher than the confidence threshold, thereby eliminating false detection results; further, bounding boxes corresponding to open flame and smoke are filtered out; for bounding boxes of the same category, deduplication is performed based on the non-maximum suppression principle, and the final retained valid bounding boxes are the boundary range of the forest fire target, and the category corresponding to each valid bounding box is the category of the corresponding forest fire target.

[0025] As can be seen from the above, this embodiment, through multi-scale feature extraction, can simultaneously capture the features of small-scale distant fire points, smoke, and large-area fires, avoiding missed detections due to insufficient feature extraction at a single scale. Based on this, global average pooling is performed on the feature maps at each scale, followed by feature concatenation and convolutional compression to obtain a scene fusion feature vector. This effectively integrates scene information at various scales, eliminates redundant interference, and enhances the representational ability of fire features. The scene fusion feature vector is input into a fully connected layer for nonlinear transformation, and then passed through a normalization layer to output dynamic weights corresponding to the feature maps at each scale. Weighted summation of the multi-scale feature maps based on these dynamic weights enables automatic enhancement of scale features more valuable for fire identification, while weakening scale features with strong background interference, resulting in a fused feature map that integrates the advantages of multiple scales. Finally, the fused feature map is input into three detection branches, which independently complete target localization, category judgment, and confidence assessment, reducing localization bias and category misjudgment.

[0026] Therefore, the method of this embodiment can effectively reduce the problems of false detection and missed detection caused by clouds, fog, changes in lighting, and complex backgrounds, accurately identify open flames and smoke targets, and greatly improve the accuracy of forest fire identification.

[0027] In one embodiment of this application, the forest fire identification method further includes: Input the feature map of each scale into the semantic segmentation head to obtain the binary segmentation mask of each scale; After upsampling the binary segmentation mask at each scale to the original size of the first target image, a fusion process is performed to obtain the fused mask; The first recognition result is determined based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box, including: The coordinate ranges of multiple bounding boxes are corrected based on the fusion mask to obtain the corrected coordinate ranges of multiple bounding boxes. The corrected coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box are used as the first recognition result.

[0028] In this embodiment, the feature maps at each scale can be input into the semantic segmentation head to obtain a binary segmentation mask for each scale. Specifically, the semantic segmentation head includes 1-2 convolutional layers and a sigmoid activation layer connected in sequence. For the feature map at each scale, firstly, pixel-level fire features are extracted through the convolutional layers, compressing the number of channels in the feature map to a single channel; then, the single-channel feature map is normalized through the sigmoid activation layer to generate a pixel-level probability map with values ​​between 0 and 1; finally, the pixel-level probability map is binarized based on a preset segmentation threshold to obtain a binary segmentation mask for the corresponding scale, where areas with a pixel value of 1 represent fire areas and areas with a pixel value of 0 represent background areas.

[0029] Furthermore, the binary segmentation masks at each scale are upsampled to the original image size and fused to obtain a fused mask. Based on the fused mask, the pixel-level contours of forest fire targets can be accurately distinguished. Therefore, within the image region corresponding to each bounding box, smoke pixels and background pixels can be distinguished according to the fused mask. Smoke pixels are retained while background pixels are removed, thereby eliminating redundant background and interference areas within the box. The resulting corrected bounding boxes better match the actual contours of the fire. Using the coordinate range of the corrected bounding boxes, the category of each bounding box, and the confidence level of each bounding box as the first recognition result can further improve the accuracy of forest fire identification.

[0030] In one embodiment of this application, for each scale, feature extraction is performed on the first target image at that scale to obtain a feature map for that scale, including: The first feature map is obtained by extracting features from the first target image through the second convolutional layer; The first feature map is input into the attention module to obtain the second feature map; The second feature map is downsampled at this scale to obtain the feature map at this scale.

[0031] In this embodiment, when performing multi-scale feature extraction on the first target image, for each scale, features can first be extracted from the first target image through a second convolutional layer (e.g., a Ghost convolutional layer) to obtain a first feature map. The GhostConv convolutional layer can achieve network lightweighting while fully preserving core visual features, such as the texture of smoke, transparency, and the bright outline of open flames. Then, the first feature map is input into an attention module to obtain a second feature map. The attention module automatically focuses on the target areas of smoke and open flames, suppresses background interference, and strengthens the expression of weak features in small-scale and long-distance scenarios, avoiding missed detections and false detections of small targets. Finally, downsampling is performed on the second feature map at that scale. This can reduce the size of the feature map and the amount of computation while preserving core semantic information, forming a compact and highly discriminative feature map at that scale. This provides high-quality single-scale feature input for subsequent multi-scale dynamic weighted fusion, improving the overall recognition accuracy.

[0032] For example, if the size of the first target image is 640×640, feature maps of multiple scales such as 320×320, 160×160, 80×80, and 40×40 can be obtained by extracting features from the first target image.

[0033] In one embodiment of this application, the attention module includes a first attention branch and a second attention branch; the parameters of the first attention branch and the second attention branch are different. The first feature map is input into the attention module to obtain the second feature map, which includes: The first feature map is input into the first attention branch and the second attention branch respectively to obtain the first attention feature map and the second attention feature map; Calculate the average brightness of the first feature map, the gradient variance of each pixel in the first feature map, and the brightness uniformity of the first feature map; The weights corresponding to the first and second attention feature maps are determined based on the average brightness, gradient variance, and brightness uniformity. Based on the weights corresponding to the first and second attention feature maps, the first and second attention feature maps are weighted and summed to obtain the second feature map.

[0034] In this embodiment, corresponding first attention branch and second attention branch are set according to the different characteristics of smoke area and open flame area. By fusing the first attention feature map output by the first attention branch and the second attention feature map output by the second attention branch, the diffusion of smoke and the clarity of open flame can be taken into account, thereby improving the accuracy and reliability of forest fire identification.

[0035] The first attention branch is an attention branch suitable for smoke feature extraction, specifically, as follows: Figure 3 As shown, the first attention branch includes a first channel attention branch and a first spatial attention branch. The first channel attention branch includes a globally average pooling layer, a third convolutional layer (1×1 convolution), and a sigmoid activation layer connected in sequence. The first spatial attention branch includes a fourth convolutional layer, a first feature enhancement layer, and a sigmoid activation layer connected in sequence. The first feature enhancement layer outputs a first feature enhancement map, where the weight of each pixel in the first feature enhancement map is negatively correlated with the gradient variance of each pixel.

[0036] The second attention branch is an attention branch suitable for open flame feature extraction. Specifically, the second attention branch includes a second channel attention branch and a second spatial attention branch. The second channel attention branch consists of a globally average pooling layer, a fifth convolutional layer (1×1 convolution), and an activation layer connected in sequence. The second spatial attention branch consists of a sixth convolutional layer, a second feature enhancement layer, and an activation layer connected in sequence. The second feature enhancement layer outputs a second feature enhancement map, where the weight of each pixel is positively correlated with the gradient variance of each pixel. The kernel size of the fourth convolutional layer is larger than that of the sixth convolutional layer.

[0037] When designing the first attention branch, it was considered that smoke regions typically have blurred edges, smooth textures, small pixel gradient changes, and low gradient variance; while interference regions such as mountains, clouds, and obstructions have complex textures, sharp edges, drastic gradient changes, and high gradient variance. Therefore, in the first spatial attention branch, its first feature enhancement layer adopts a weighting mechanism negatively correlated with gradient variance. High weights are assigned to smoke regions with low gradient variance to enhance effective features; while weights are automatically reduced for interference regions with high gradient variance to suppress background noise. Simultaneously, a fourth convolutional layer with a larger kernel (e.g., a 5×5 kernel) is used to obtain a larger receptive field, adapting to the large-scale, diffuse distribution characteristics of smoke, and achieving overall feature modeling of the smoke region.

[0038] When designing the second attention branch, considering that the open flame region has characteristics such as sharp edges, high brightness, and dramatic gradient changes, its pixel gradient variance is significantly higher than that of the smoke region. Furthermore, open flames often exhibit locally concentrated distribution and well-defined boundaries, clearly distinguishing them from the diffuse nature of smoke. Therefore, in the second spatial attention branch, the second feature enhancement layer adopts a weighting mechanism positively correlated with gradient variance: for open flame regions with high gradient variance (i.e., sharp edges and prominent brightness), higher weights are assigned to emphasize and enhance the feature representation of the open flame; for background regions with low gradient variance, the weights are reduced to effectively suppress irrelevant interference. Simultaneously, a smaller sixth convolutional layer (e.g., a 3×3 convolutional kernel) is used to accurately capture the local details of the open flame, adapting to the characteristics of concentrated distribution and sharp edges of the open flame, achieving accurate localization and feature extraction of the open flame region, and avoiding detail loss due to excessively large convolutional kernels.

[0039] Furthermore, based on the characteristics of low average brightness, small gradient variance, and high brightness uniformity in smoke regions, and high average brightness, large gradient variance, and low brightness uniformity in open flame regions, the average brightness, gradient variance, and brightness uniformity of the first feature map can be calculated. The weights of the first and second attention feature maps can then be adaptively assigned based on these characteristics. When the first feature map more closely resembles the characteristics of smoke (low average brightness, small gradient variance, and high brightness uniformity), the weight of the first attention feature map is automatically increased to strengthen the expression of smoke features and prevent diffuse smoke from being missed. When the region more closely matches the characteristics of open flame (high average brightness, large gradient variance, and low brightness uniformity), the weight of the second attention feature map is automatically increased to highlight the edges and bright areas of the open flame, thereby improving the accuracy of open flame localization.

[0040] For example, if the spatial dimensions of the first feature map are H×W×C, the average brightness, gradient variance, and brightness uniformity of the first feature map can be calculated using the following formula: ; in, Indicates average brightness. This represents the brightness of the c-th channel at the (i,j)-th pixel position; Represents the gradient variance. Indicates brightness gradient, Indicates variance calculation; Indicates brightness uniformity. This represents the maximum brightness of the first feature map. This represents the minimum brightness of the first feature map. This represents a very small positive number, used to avoid a denominator of 0.

[0041] Based on this, smoke scores and open flame scores can be calculated using the average brightness, gradient variance, and brightness uniformity of the first feature map: ; in, The score is indicated by the smoke. This indicates that the score is based on open flame. , and All are weighting coefficients, and In this embodiment, α1=0.3, α2=0.5, and α3=0.2 are set.

[0042] Furthermore, by normalizing the smoke score and the open flame score, the weights corresponding to the first attention feature map and the second attention feature map can be obtained: ; in, The weights of the first attention feature map are represented. This represents the weights of the second attention feature map.

[0043] Recently, based on the weights corresponding to the first and second attention feature maps respectively, a weighted sum of the first and second attention feature maps was performed to obtain the second feature map.

[0044] As can be seen from the above, this embodiment sets up corresponding first attention branches and second attention branches according to the different characteristics of smoke areas and open flame areas, which can specifically enhance effective features and suppress background interference. At the same time, the weights of the first attention feature map and the second attention feature map are dynamically allocated based on the average brightness, gradient variance and brightness uniformity, which can highlight the fire target and thus improve the accuracy of forest fire identification.

[0045] In one embodiment of this application, the first target image is a visible light image or an infrared image; the forest fire identification method further includes: Acquire a second target image of the region to be identified; the second target image is a visible light image or an infrared image, and the second target image is different from the first target image; Perform image recognition operation on the second target image to obtain the second recognition result; Among them, determining the forest fire targets in the area to be identified based on the first identification result includes: The first and second identification results are fused together, and the wildfire targets in the area to be identified are determined based on the fusion result.

[0046] In this embodiment, the first target image can be a visible light image or an infrared image. Taking the first target image as a visible light image as an example, the method of the embodiment can be used to perform an image recognition operation on the first target image to obtain a first recognition result. Correspondingly, an infrared image of the area to be recognized can be obtained as the second target image, and the method of the embodiment can be used to perform an image recognition operation on the second target image to obtain a second recognition result.

[0047] Visible light images can clearly reflect the shape, texture and distribution characteristics of smoke, but in scenarios with insufficient light, smoke obstruction, or long-distance shooting, their ability to perceive weak open flames and hidden fire points is weak, and they are easily affected by light, clouds, fog and vegetation shadows. Infrared images, on the other hand, are based on thermal radiation imaging and are extremely sensitive to high-temperature open flame areas. They can penetrate smoke, obstructions and dark environments to accurately capture thermal anomaly fire points, but their texture details are poor and their ability to characterize smoke and low-temperature smoldering fires is insufficient.

[0048] Therefore, this embodiment obtains a first target image and a second target image, respectively, and then fuses the two for decision-making. This can comprehensively utilize the imaging advantages of different images, weaken the recognition limitations of a single modality in complex environments, and thus improve the reliability and accuracy of forest fire target determination.

[0049] In one embodiment of this application, the first identification result and the second identification result are fused, and the forest fire target in the area to be identified is determined based on the fusion result, including: Calculate the pixel coordinates of the center point of each bounding box in the first and second recognition results; Perform coordinate transformation on the pixel coordinates of the center point to obtain the geographic coordinates of the center point of each bounding box; Each bounding box in the first identification result is taken as a first bounding box, and each bounding box in the second identification result is taken as a second bounding box. For each first bounding box, if there is a second bounding box among multiple second bounding boxes, and the geographical distance between the center point of the second bounding box and the center point of the first bounding box is less than a preset distance threshold, then the midpoint of the line connecting the two center points is taken as the midpoint of the forest fire target, and the intersection of the second bounding box and the first bounding box is taken as the boundary range of the forest fire target.

[0050] In this embodiment, when fusing the first and second recognition results, the center point of each bounding box can first be converted from pixel coordinates to unified geographic coordinates. Specifically, laboratory intrinsic parameter calibration and lens distortion correction can be performed on the visible light camera and infrared camera to obtain the camera intrinsic parameter matrix and distortion coefficients; combined with the relative flight altitude and equivalent focal length of the camera at the time of UAV imaging, the ground sample distance (GSD) of the visible light and infrared images can be calculated respectively; based on the UAV's ground coordinates and attitude data obtained by real-time kinematic (RTK), through a four-step coordinate transformation from image plane coordinate system to body coordinate system to northeast-northeast sky coordinate system to ground coordinate system, combined with the target centroid pixel coordinates and the corresponding GSD, the geographic coordinates of each bounding box can be calculated respectively.

[0051] Based on this, each bounding box in the first identification result can be used as the first bounding box, and each bounding box in the second identification result can be used as the second bounding box. For each first bounding box, if there is a second bounding box and the geographical distance between the center point of the second bounding box and the center point of the first bounding box is less than a preset distance threshold (e.g., 3m), it is determined to be a real fire. The midpoint of the line connecting the two center points is taken as the midpoint of the final forest fire target. The dual-modal fire image, positioning data and imaging timestamp are packaged simultaneously and sent to the ground command platform through the wireless transmission module to trigger a fire warning. Otherwise, it is determined to be a false detection, no warning is triggered, and the data is only retained for subsequent model optimization.

[0052] Based on the same principle as the forest fire identification method provided in the embodiments of this application, the embodiments of this application also provide a forest fire identification device, such as... Figure 4 As shown, the forest fire identification device 20 may specifically include: an image acquisition module 21, an image recognition module 22, and a forest fire identification module 23.

[0053] Among them, the image acquisition module 21 is used to acquire the first target image of the region to be identified; Image recognition module 22 is used to perform image recognition operation on the first target image to obtain a first recognition result; Forest fire identification module 23 is used to determine forest fire targets in the area to be identified based on the first identification result; During the image recognition operation, the image recognition module is specifically used for: Multi-scale feature extraction is performed on the first target image to obtain feature maps at multiple scales; Global average pooling is performed on the feature maps at each scale to obtain the scene feature vectors at each scale; After concatenating the scene feature vectors corresponding to multiple scales, the first convolutional layer is used for compression to obtain the scene fusion feature vector. The scene fusion feature vector is sequentially input into the fully connected layer and the normalization layer to obtain the dynamic weights for each scale; Based on the dynamic weights of each scale, feature maps of multiple scales are weighted and fused to obtain a fused feature map; The fused feature map is input into the three detection branches respectively to obtain the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence of each bounding box; The first recognition result is determined based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box.

[0054] In one embodiment of this application, the image recognition module 22 is specifically used for: Input the feature map of each scale into the semantic segmentation head to obtain the binary segmentation mask of each scale; After upsampling the binary segmentation mask at each scale to the original size of the first target image, a fusion process is performed to obtain the fused mask; The first recognition result is determined based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box, including: The coordinate ranges of multiple bounding boxes are corrected based on the fusion mask to obtain the corrected coordinate ranges of multiple bounding boxes. The corrected coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box are used as the first recognition result.

[0055] In one embodiment of this application, for each scale, the image recognition module 22 is specifically used for: The first feature map is obtained by extracting features from the first target image through the second convolutional layer; The first feature map is input into the attention module to obtain the second feature map; The second feature map is downsampled to obtain the feature map at this scale.

[0056] In one embodiment of this application, the attention module includes a first attention branch and a second attention branch; the parameters of the first attention branch and the second attention branch are different; the image recognition module 22 is further used for: The first feature map is input into the first attention branch and the second attention branch respectively to obtain the first attention feature map and the second attention feature map; Calculate the average brightness of the first feature map, the gradient variance of each pixel in the first feature map, and the brightness uniformity of the first feature map; The weights corresponding to the first and second attention feature maps are determined based on the average brightness, gradient variance, and brightness uniformity. Based on the weights corresponding to the first and second attention feature maps, the first and second attention feature maps are weighted and summed to obtain the second feature map.

[0057] In one embodiment of this application, the first target image is a visible light image or an infrared image; the image recognition module 22 is specifically used for: Acquire a second target image of the region to be identified; the second target image is a visible light image or an infrared image, and the second target image is different from the first target image; Perform image recognition operation on the second target image to obtain the second recognition result; Among them, determining the forest fire targets in the area to be identified based on the first identification result includes: The first and second identification results are fused together, and the wildfire targets in the area to be identified are determined based on the fusion result.

[0058] In one embodiment of this application, the image recognition module 22 is further configured to: Calculate the pixel coordinates of the center point of each bounding box in the first and second recognition results; Perform coordinate transformation on the pixel coordinates of the center point to obtain the geographic coordinates of the center point of each bounding box; Each bounding box in the first identification result is taken as a first bounding box, and each bounding box in the second identification result is taken as a second bounding box. For each first bounding box, if there is a second bounding box among multiple second bounding boxes, and the geographical distance between the center point of the second bounding box and the center point of the first bounding box is less than a preset distance threshold, then the midpoint of the line connecting the two center points is taken as the midpoint of the forest fire target, and the intersection of the second bounding box and the first bounding box is taken as the boundary range of the forest fire target.

[0059] The apparatus in this application embodiment can execute the method provided in this application embodiment, and the implementation principle is similar. The actions performed by each module in the apparatus of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the apparatus, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0060] Figure 5 A schematic diagram of the structure of an electronic device to which this application embodiment applies is shown, such as... Figure 5 As shown, the electronic device can be used to implement the methods provided in any embodiment of this application.

[0061] like Figure 5As shown, the electronic device 300 may primarily include at least one processor 301. Figure 5 The diagram shows components such as a memory 302, a communication module 303, and an input / output interface 304. Optionally, these components can be connected and communicate with each other via a bus 305. It should be noted that... Figure 5 The structure of the electronic device 300 shown is merely illustrative and does not constitute a limitation on the electronic devices to which the methods provided in the embodiments of this application are applicable.

[0062] The memory 302 can be used to store operating systems and applications, etc. The applications can include computer programs that implement the methods shown in the embodiments of this application when invoked by the processor 301, and can also include programs for implementing other functions or services. The memory 302 can be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and computer programs, or it can be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0063] Processor 301 is connected to memory 302 via bus 305 and implements corresponding functions by calling the application programs stored in memory 302. Processor 301 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0064] Electronic device 300 can connect to a network via communication module 303 (which may include, but is not limited to, components such as a network interface) to communicate with other devices (such as user terminals or servers) through the network and achieve data interaction, such as sending data to or receiving data from other devices. Communication module 303 may include wired network interfaces and / or wireless network interfaces, meaning the communication module may include at least one of wired or wireless communication modules.

[0065] The electronic device 300 can connect to necessary input / output devices, such as a keyboard or display device, via the input / output interface 304. The electronic device 300 itself may have a display device, and other display devices can also be connected externally via the interface 304. Optionally, a storage device, such as a hard drive, can also be connected via the interface 304, allowing data from the electronic device 300 to be stored, read, or transferred to the memory 302. It is understood that the input / output interface 304 can be a wired or wireless interface. Depending on the specific application scenario, the device connected to the input / output interface 304 can be an integral part of the electronic device 300 or an external device connected to the electronic device 300 when needed.

[0066] The bus 305 used to connect the components may include a path for transmitting information between the components. The bus 305 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Depending on its function, the bus 305 may be divided into an address bus, a data bus, a control bus, etc.

[0067] Optionally, for the solution provided in the embodiments of this application, the memory 302 can be used to store a computer program that executes the solution of this application, and the processor 301 runs the computer program. When the processor 301 runs the computer program, it implements the operation of the method or apparatus provided in the embodiments of this application.

[0068] Based on the same principle as the method provided in the embodiments of this application, the embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the corresponding content of the aforementioned method embodiments.

[0069] This application also provides a computer program product, which includes a computer program that, when executed by a processor, can implement the corresponding content of the aforementioned method embodiments.

[0070] It should be noted that the terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0071] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0072] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0073] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. A method for identifying forest fires, characterized in that, include: Acquire the first target image of the region to be identified; An image recognition operation is performed on the first target image to obtain a first recognition result; Based on the first identification result, determine the forest fire targets in the area to be identified; The steps of the image recognition operation include: Multi-scale feature extraction is performed on the first target image to obtain feature maps at multiple scales; Global average pooling is performed on the feature maps at each scale to obtain the scene feature vectors at each scale; After concatenating the scene feature vectors corresponding to multiple scales, the first convolutional layer is used for compression to obtain the scene fusion feature vector. The scene fusion feature vector is sequentially input into the fully connected layer and the normalization layer to obtain the dynamic weights for each scale; Based on the dynamic weights of each scale, feature maps of multiple scales are weighted and fused to obtain a fused feature map; The fused feature map is input into the three detection branches respectively to obtain the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence of each bounding box; The first recognition result is determined based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box.

2. The forest fire identification method as described in claim 1, characterized in that, Also includes: Input the feature map of each scale into the semantic segmentation head to obtain the binary segmentation mask of each scale; After upsampling the binary segmentation mask at each scale to the original size of the first target image, a fusion process is performed to obtain the fused mask; The step of determining the first recognition result based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box includes: The coordinate ranges of the multiple bounding boxes are corrected based on the fusion mask to obtain the corrected coordinate ranges of the multiple bounding boxes. The coordinate range of the corrected multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box are used as the first recognition result.

3. The forest fire identification method as described in claim 1, characterized in that, For each scale, feature extraction is performed on the first target image at that scale to obtain a feature map for that scale, including: The first feature map is obtained by extracting features from the first target image through the second convolutional layer; The first feature map is input into the attention module to obtain the second feature map; The second feature map is downsampled to obtain a feature map at this scale.

4. The forest fire identification method as described in claim 3, characterized in that, The attention module includes a first attention branch and a second attention branch; The parameters of the first attention branch and the second attention branch are different; The first feature map is input into the attention module to obtain the second feature map, which includes: The first feature map is input into the first attention branch and the second attention branch respectively to obtain the first attention feature map and the second attention feature map; Calculate the average brightness of the first feature map, the gradient variance of each pixel in the first feature map, and the brightness uniformity of the first feature map; The weights corresponding to the first attention feature map and the second attention feature map are determined based on the average brightness, the gradient variance, and the brightness uniformity. Based on the weights corresponding to the first attention feature map and the second attention feature map, the first attention feature map and the second attention feature map are weighted and summed to obtain the second feature map.

5. The forest fire identification method as described in claim 4, characterized in that, The first attention branch includes a first channel attention branch and a first spatial attention branch; The second attention branch includes a second channel attention branch and a second spatial attention branch; The first channel attention branch includes a global average pooling layer, a third convolutional layer, and an activation layer connected in sequence; the first spatial attention branch includes a fourth convolutional layer, a first feature enhancement layer, and an activation layer connected in sequence; wherein, the first feature enhancement layer is used to output a first feature enhancement map, and the weight of each pixel in the first feature enhancement map is negatively correlated with the gradient variance of each pixel; The second channel attention branch includes a global average pooling layer, a fifth convolutional layer, and an activation layer connected in sequence; the second spatial attention branch includes a sixth convolutional layer, a second feature enhancement layer, and an activation layer connected in sequence; wherein, the second feature enhancement layer is used to output a second feature enhancement map, and the weight of each pixel in the second feature enhancement map is positively correlated with the gradient variance of each pixel; the kernel size of the fourth convolutional layer is larger than the kernel size of the sixth convolutional layer.

6. The forest fire identification method as described in claim 1, characterized in that, The first target image is a visible light image or an infrared image; the forest fire identification method further includes: Acquire a second target image of the region to be identified; the second target image is a visible light image or an infrared image, and the second target image is different from the first target image; Perform an image recognition operation on the second target image to obtain a second recognition result; The process of determining forest fire targets in the area to be identified based on the first identification result includes: The first identification result and the second identification result are fused together, and the forest fire target in the area to be identified is determined based on the fusion result.

7. The forest fire identification method as described in claim 6, characterized in that, The first identification result and the second identification result are fused together, and the forest fire target in the area to be identified is determined based on the fusion result, including: Calculate the pixel coordinates of the center point of each bounding box in the first recognition result and the second recognition result; Perform coordinate transformation on the pixel coordinates of the center point to obtain the geographic coordinates of the center point of each bounding box; Each bounding box in the first identification result is taken as a first bounding box, and each bounding box in the second identification result is taken as a second bounding box. For each first bounding box, if there is a second bounding box among multiple second bounding boxes, and the geographical distance between the center point of the second bounding box and the center point of the first bounding box is less than a preset distance threshold, then the midpoint of the line connecting the two center points is taken as the midpoint of the forest fire target, and the intersection of the second bounding box and the first bounding box is taken as the boundary range of the forest fire target.

8. A forest fire identification device, characterized in that, include: The image acquisition module is used to acquire the first target image of the region to be identified; The image recognition module is used to perform an image recognition operation on the first target image to obtain a first recognition result; The forest fire identification module is used to determine forest fire targets in the area to be identified based on the first identification result; Specifically, during the execution of the image recognition operation, the image recognition module is used for: Multi-scale feature extraction is performed on the first target image to obtain feature maps at multiple scales; Global average pooling is performed on the feature maps at each scale to obtain the scene feature vectors at each scale; After concatenating the scene feature vectors corresponding to multiple scales, the first convolutional layer is used for compression to obtain the scene fusion feature vector. The scene fusion feature vector is sequentially input into the fully connected layer and the normalization layer to obtain the dynamic weights for each scale; Based on the dynamic weights of each scale, feature maps of multiple scales are weighted and fused to obtain a fused feature map; The fused feature map is input into the three detection branches respectively to obtain the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence of each bounding box; The first recognition result is determined based on the coordinate range of multiple bounding boxes, the category of each bounding box, and the confidence level of each bounding box.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the forest fire identification method according to any one of claims 1 to 7 when running the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the forest fire identification method according to any one of claims 1 to 7.

Images