Method for identifying fire point in satellite remote sensing image in real time

Abstract

The invention discloses a method for identifying fire points in a satellite remote sensing image in real time, which comprises the following steps: acquiring a remote sensing fire point data set, and dividing the remote sensing fire point data set into a training set, a test set and a verification set; and constructing an initial fire point identification network model, training the fire point identification network model through the training set and the verification set to obtain a fire point identification network model, and testing the fire point identification network model through the test set to obtain a final fire point identification network model. The final fire point identification network model is obtained through an improved YOLOv8 shallow feature extraction module and a multi-scale feature fusion module; and performing fire point identification on the target area by adopting the final fire point identification network model to obtain an identification result. According to the invention, the learning ability of the model for a multi-scale target and the detection ability of the model are improved.

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition, and in particular relates to a method for real-time recognition of fire points in satellite remote sensing images. Background Art

[0002] Traditional manual monitoring methods, such as manual patrols and setting up watchtowers, require a lot of manpower and cannot completely detect fires immediately; the range of fire monitoring methods using chemical sensors is not large enough and the cost is very high; the range of fire identification using drones is about 10km, and there are problems with drone endurance, which still cannot meet the requirements for real-time detection of early fires. The use of satellite remote sensing data for fire detection has become a hot topic of research due to its large detection range. The algorithm proposed based on multispectral data identifies fires by considering specific conditions related to reflectivity, using brightness temperature with contextual analysis, multi-channel threshold technology and fire index method. However, it depends on the environment in which the fire occurs, the intensity of the fire and other conditions, and there is still room for improvement. In recent studies, image recognition methods based on deep learning have provided new ideas for improving the accurate fire detection capabilities of remote sensing data. Among them, the two-stage image recognition method R-CNN series has shown good results in identifying high-resolution polar satellite images, but the time period for polar satellites to obtain satellite images is often about 1-8 days, which cannot meet the needs of real-time fire detection. In order to realize the real-time detection of fire, it is necessary to use the images of synchronous satellites such as Sunflower 8 satellite, but the resolution of synchronous satellite images is low. At this time, the characteristics of fire in satellite images are small, dense, and unclear texture, and the effect of R-CNN is not the best. In order to improve the recognition ability of fire in synchronous satellite images and realize real-time detection of fire, a method for real-time identification of fire points in satellite remote sensing images is proposed. Summary of the invention

[0003] In order to solve the above technical problems, the present invention proposes a method for real-time identification of fire points in satellite remote sensing images, which improves the model's learning ability for multi-scale targets and the model's detection ability.

[0004] To achieve the above object, the present invention provides a method for real-time identification of fire points in satellite remote sensing images, comprising: obtaining a remote sensing fire point data set, dividing the remote sensing fire point data set into a training set, a test set and a validation set;

[0005] Constructing an initial fire point recognition network model, training the fire point recognition network model through the training set and the validation set to obtain a fire point recognition network model, testing the fire point recognition network model through the test set to obtain a final fire point recognition network model, wherein the final fire point recognition network model is obtained by improving the shallow feature extraction module and the multi-scale feature fusion module of YOLOv8;

[0006] The final fire point recognition network model is used to identify fire points in the target area to obtain recognition results.

[0007] Optionally, the final fire point recognition network model is obtained by improving the shallow feature extraction module and multi-scale feature fusion module of YOLOv8 and includes:

[0008] The RFD downsampling module is introduced into the YOLOv8 backbone network to process a large amount of information and redundancy in the input image in the initial stage of feature extraction, and to process feature maps with rich semantic information in the subsequent stage of feature extraction;

[0009] Use the SSFF module to perform multi-scale feature extraction on the high-resolution feature layer P2, the medium-resolution feature layer P3, and the low-resolution feature layer P4;

[0010] Design a multi-scale encoder to process feature maps of different scales and add the processed feature information element by element;

[0011] The detection head is improved by combining the DCNv2 module to predict and identify fire point targets.

[0012] Optionally, the convolution of the feature maps P3, P4 and P5 with a series of Gaussian kernels with increasing standard deviations in the SSFF module includes:

[0013] F σ (i, j) = ∑ u ∑ v f(iu,iv)×G σ (u, v),

[0014]

[0015] Among them, F σ (i, j) represents the value of the convolution result at the coordinate (i, j), G σ (u, v) represents the value of the Gaussian kernel at (u, v), f(iu, iv)*G σ (u, v) means multiplying the value of the input image at (iu, jv) by the value of the Gaussian kernel at (u, v), and reducing the noise in the image by taking a weighted average of each pixel in the image and its neighborhood; G σ (x, y) is a two-dimensional Gaussian function, σ is the standard deviation, σ 2 is the variance, is the normalization constant, e is the base of the natural logarithm, (x 2 +y 2 ) represents the square of the distance from the point to the origin, and f is a two-dimensional feature map.

[0016] Optionally, the multi-scale encoder is a triple feature encoder, which performs feature fusion on feature maps of different sizes to improve feature information.

[0017] Optionally, the RFD downsampling module includes a shallow RFD unit and a deep RFD unit;

[0018] The shallow RFD unit is used to process a large amount of information and redundancy in the input image at the initial stage of feature extraction;

[0019] The deep RFD unit is used to process feature maps rich in semantic information in the subsequent stage of feature extraction.

[0020] Optionally, the detection head is improved by combining the DCNv2 module and introducing feature modulation including:

[0021]

[0022] Among them, y(p) is the result of the output feature after the modulated deformable convolution operation at position p, and w k is the weight Δm of each sampling point in the convolution kernel k is the coefficient that modulates the input feature at the kth position, x(p+p k +Δp k ) means at position p+p k +Δp k Values ​​sampled from the input feature map.

[0023] Optionally, the calculation accuracy Precision during the testing of the fire point recognition network model by the test set includes:

[0024]

[0025] Among them, Precision is the ratio of correctly identified positive samples to all positive samples detected by the model, TP is the number of positive samples correctly predicted as positive samples by the model, and FP is the number of negative samples incorrectly predicted as positive samples by the model.

[0026] Optionally, calculating the recall rate Recall during the testing of the fire point recognition network model using a test set includes:

[0027]

[0028] Among them, Recall is the ratio of true positive samples correctly detected by the model to all positive samples in the data set, TP is the number of positive samples correctly predicted as positive samples by the model, FP is the number of negative samples incorrectly predicted as positive samples by the model, and FN is the number of positive samples incorrectly predicted as negative samples by the model.

[0029] Optionally, calculating the average accuracy AP during the testing of the fire point recognition network model using a test set includes:

[0030] AP = ∫P(R)dR.

[0031] Among them, AP is the average precision value of all recall levels of a specific category, P is the Precision, R is the Recall rate, and dR is the change in the recall rate.

[0032] Optionally, the amount of GFLOPs of parameters calculated during the testing of the fire point recognition network model using a test set includes:

[0033] GFLOPs = (2C i K 2 -1)HWC0,

[0034] Among them, GFLOPs is an indicator to measure the computational complexity of the model, which is used to indicate the number of floating-point operations required by the model when running; C i is the number of channels of the input data; K is the size of the convolution kernel; HWC0 represents the height, width and number of channels of the output feature map.

[0035] Technical effect of the present invention: The present invention discloses a method for real-time identification of fire points in satellite remote sensing images. The RFD downsampling module is introduced to reconstruct the backbone network of the original model, more fire point information is retained in the feature extraction stage, and more high-resolution p2 feature layer feature information is used, so that the improved model can pay attention to tiny fire points, greatly improving the recall rate of the model, which is crucial for remote sensing fire point tasks. By using the self-made TFE module to fuse the triple scale features of different scales, the model can learn the generalized features of fire points of different scales, and combined with the high-level features extracted by the SSFF module, the model's learning ability for multi-scale targets is improved. Finally, the detection head of the original model is improved, and the detection head is improved using the DCNv2 module, so that the model can notice the diverse texture features around the fire point, further improving the detection ability of the model. It was tested on the Sunflower 8 satellite remote sensing fire point dataset, and the recall rate of key indicators was 2% better than the mainstream model, and the average accuracy was improved by 1%. After conducting fire point detection experiments on full-frame remote sensing images, it was confirmed that RF-Yolo has the ability to identify fire points in the sunflower-8 remote sensing satellite images, providing a new method for the timely detection of fires. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0037] Figure 1 This is a YOLOv8 network structure diagram of an embodiment of the present invention;

[0038] Figure 2 This is a detailed display diagram of the C2f, SPPF, and DETECT modules of an embodiment of the present invention;

[0039] Figure 3 This is a structural diagram of the SSFF module according to an embodiment of the present invention;

[0040] Figure 4 This is a structural diagram of a TFE module according to an embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram of GhostConv according to an embodiment of the present invention;

[0042] Figure 6 This is a structural diagram of Dysample according to an embodiment of the present invention;

[0043] Figure 7 Schematic diagram of the RFD module of an embodiment of the present invention, wherein (a) is a shallow RFD unit, and (b) is a deep RFD unit;

[0044] Figure 8 This is a schematic diagram of the deformable convolution principle of an embodiment of the present invention;

[0045] Fig. 9 The improved RF-YOLO network structure of the embodiment of the present invention;

[0046] Fig.10 This is a comparison chart of the PR curves of the mainstream yolo models in the embodiments of the present invention. DETAILED DESCRIPTION

[0047] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0048] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0049] This embodiment provides a method for real-time identification of fire points in satellite remote sensing images, including: obtaining a remote sensing fire point data set, dividing the remote sensing fire point data set into a training set, a test set, and a validation set;

[0050] Constructing an initial fire point recognition network model, training the fire point recognition network model through the training set and the validation set to obtain a fire point recognition network model, testing the fire point recognition network model through the test set to obtain a final fire point recognition network model, wherein the final fire point recognition network model is obtained by improving the shallow feature extraction module and the multi-scale feature fusion module of YOLOv8;

[0051] The final fire point recognition network model is used to identify fire points in the target area to obtain recognition results.

[0052] Furthermore, the final fire point recognition network model is obtained by improving the shallow feature extraction module and multi-scale feature fusion module of YOLOv8 and includes:

[0053] The RFD downsampling module is introduced into the YOLOv8 backbone network to process a large amount of information and redundancy in the input image in the initial stage of feature extraction, and to process feature maps with rich semantic information in the subsequent stage of feature extraction;

[0054] Use the SSFF module to perform multi-scale feature extraction on the high-resolution feature layer P2, the medium-resolution feature layer P3, and the low-resolution feature layer P4;

[0055] Design a multi-scale encoder to process feature maps of different scales and add the processed feature information element by element;

[0056] The detection head is improved by combining the DCNv2 module to predict and identify fire point targets.

[0057] Specifically, the Yolov8 model is mainly composed of three parts: backbone, neck, and head. The backbone network of Yolov8 is Darknet (CSPDarknet53) with 53 convolutional layers, which is modified from YOLOv4 and designed as the backbone network of YOLOv8, which contains C3 (CSP bottleneck including 3 convolutional layers) and ConvBNSiLU modules. In Yolov8, the C3 module is replaced by the C2f module, and the SPP module is replaced by the more efficient SPPF structure. This is the main difference between YOLOv8 and YOLOv8. The network structure diagram of YOLOv8 is as follows Figure 1 As shown, the details are as follows Figure 2As shown in the figure. In the feature extraction stage of YOLOv8, a CSPDarkNet53 backbone network composed of multiple C2f modules is used, and then the three different scale feature layers P3, P4, and P5 extracted by the backbone network are used as the input of the feature pyramid network (FPN), and then a multi-scale fusion structure is constructed at the neck. In the decoding process of the feature layer, three heads of different sizes corresponding to the effective feature branches of the backbone network are used to predict the bounding box of the object. After upsampling the features of different scales, pixel-by-pixel decoding is performed as the recognition prediction of the object to complete the recognition of the object. In the detection head, three different anchor boxes are output according to the features of the three scales. The mask prototype module is responsible for outputting the prototype mask. The prototype mask is processed to obtain the detection box mask. Each grid also generates a category probability distribution, indicating the probability of belonging to each category in the area. For the prediction of multiple bounding boxes, the NMS algorithm is applied to suppress redundant boxes, retain boxes with high confidence and low overlap, and finally output the final detection result.

[0058] In order to solve the problem that small-sized fire targets in remote sensing data are difficult to identify, Robust Feature Downsampling (RFD) is introduced in the backbone network to improve the feature loss of small targets caused by dimensionality reduction of feature maps, and enhance the model's ability to use high-resolution features, so that the model can pay attention to higher-quality small fire features, thereby improving the recognition ability of small-sized features; in view of the diverse scales of fire targets, a multi-scale encoder Triple feature encoding module (TFE) is designed to add feature information of different scales element by element to increase the model's context perception ability, and combined with Scale sequence feature fusion module (SSFF) to better combine the high-level information of multi-scale feature maps, so that the model can better learn the features of fire targets of different scales; in view of the different texture features around the fire targets, the Convolutional Network V2 (DCNv2) module is used in the detection head to enable the model to learn different texture features around the fire points, thereby improving the accuracy of fire target identification. In summary, I proposed a new fire detection model RF-Yolo for real-time fire detection.

[0059] In the feature extraction stage, the RFD downsampling module is introduced into the backbone network to reduce the problem of small target information loss caused by the convolution process of remote sensing images. The SSFF module is combined and the high-resolution feature map of the p2 detection layer is introduced into the network. The extraction of shallow information is further enhanced. This shallow information is very important for small and dense fire points in the fire point detection task. An encoder (TFE) is proposed in the neck to fuse different features at multiple scales to improve the robustness of the model to fire point targets. Finally, we improved the detection head by combining the DCNv2 module in the training stage. The improved DCN-head can better predict and identify fire point targets.

[0060] Furthermore, the scale sequence feature fusion module is a scale sequence feature fusion module in the network structure of ASF-YOLO proposed based on the YOLOv8 segmentation framework, which can better combine the high-level information of multi-scale feature maps (i.e., detailed information of feature maps of deep feature layers with the same aspect ratio). Further construct a sequential representation of multi-scale feature maps generated from the backbone (i.e., P3, P4, and P5), which capture image content at different levels of detail or scale. The feature maps P3, P4, and P5 in the SSFF module are convolved with a series of Gaussian kernels with increasing standard deviations, including:

[0061] F σ (i,j)=∑ u ∑ v f(iu,iv)×G σ (u, v),

[0062]

[0063] Among them, F σ (i, j) represents the value of the convolution result at the coordinate (i, j), G σ (u, v) represents the value of the Gaussian kernel at (u, v), f(iu, iv)*G σ (u, v) means multiplying the value of the input image at (iu, jv) by the value of the Gaussian kernel at (u, v), and reducing the noise in the image by taking a weighted average of each pixel in the image and its neighborhood; G σ (x, y) is a two-dimensional Gaussian function, σ is the standard deviation, σ 2 is the variance, is the normalization constant, e is the base of the natural logarithm, (x 2 +y 2 ) represents the square of the distance from the point to the origin, f is a two-dimensional feature map, and Fσ is generated by smoothing through a series of convolutions using a two-dimensional Gaussian filter with increasing standard deviation σ.

[0064] These feature maps of different scales are then stacked horizontally, and their scale sequence features are extracted using 3D convolution. Since the output feature maps of the above Gaussian smoothing have different resolutions, the nearest neighbor interpolation method is used to align all feature maps to the same resolution as P3. This is because the high-resolution feature map level P3 contains most of the information that is critical for small object detection and segmentation, and the SSFF module is designed based on the P3 level. The present invention uses more P2 feature maps on the basis of the original P3, P4, and P5 feature maps, so the SSFF module is also used at the P2 level. The SSFF module structure diagram is shown in the figure. Figure 3 As shown in the figure, it includes a Gaussian kernel convolution module. Gaussian kernel convolution can smooth the feature map and reduce the impact of noise. In remote sensing images, the background may be complex and there are various interference factors. Gaussian kernel convolution can help highlight the characteristics of the fire point and suppress irrelevant background information; using normalization methods such as batch normalization to normalize the feature map after Gaussian kernel convolution can accelerate the network training process and make the network converge faster. At the same time, it helps to prevent the problem of gradient disappearance or gradient explosion and make the model more stable. At the feature level, normalization can make features of different scales and amplitudes comparable, which is conducive to subsequent fusion operations; using upsampling methods such as bilinear interpolation to upsample the normalized feature map, adjust the feature maps of different scales to the same or similar size for subsequent fusion; finally, using 3D convolution to convolve the upsampled feature map, 3D convolution can process feature maps of multiple scales at the same time, and has advantages in fusing features of different scales. It can effectively capture the correlation of feature maps in space and scale dimensions, and enhance the model's ability to detect fire points of different scales. Through 3D convolution, the model can better integrate information at various scales and improve detection accuracy.

[0065] Furthermore, the multi-scale encoder is a triple feature encoder, which fuses the features of feature maps of different sizes to improve the feature information.

[0066] Specifically, for densely populated fire points, the sizes of each feature layer in the backbone network are different, and the traditional FPN fusion mechanism only upsamples the small-sized feature map and adds it to the features of the previous layer, ignoring the rich details of the larger-sized feature layer. Therefore, the GD triple feature encoder is proposed to fuse large, medium and small features to improve the detailed feature information. Figure 4The structure of the TFE module is demonstrated. The TFE encoder uses GhostConv to downsample the shallow feature map. For small target fire points or weak fire point signals that may exist in the fire point detection task, GhostConv can enhance the model's perception of these subtle features and avoid missed detections due to unclear features. Dysample is used to upsample the deep features, and its efficient dynamic upsampling characteristics are used to help improve the performance of the model in the fire point detection task.

[0067] For large-size feature maps (Large), the GhostConv structure is used for downsampling. The GhostConv structure is as follows Figure 5 As shown in the figure, it includes using convolution to generate m feature maps Y'. The convolution operation here is similar to the ordinary convolution layer, but the number of convolution filters m is relatively small (m≤n, n is the number of feature maps that are expected to be generated in the end); after obtaining the intrinsic feature map Y', each intrinsic feature map y' is subjected to a series of cheap linear operations. i (i=1,...,s) is transformed to generate s ghost feature maps, which can greatly reduce the amount of calculation without losing too much calculation efficiency and realize efficient use of computing resources. GhostConv helps to reduce the spatial dimension characteristics of feature maps and achieve translation invariance, enhancing the robustness of the network to spatial changes and translations of input images. Moreover, compared with traditional convolution, Ghost only performs half of the convolution operations, and the remaining feature maps are obtained through cheap calculations, so the amount of calculation of the model is reduced.

[0068] For small-size feature maps (Small), use the Dysample module for upsampling. The DySample structure is as follows: Figure 6As shown, it includes grid sampling, which resamples the feature map assumed to have undergone bilinear interpolation according to the given sampling point position, thereby obtaining the upsampled feature map; sampling point generator, which is responsible for generating the offset of the sampling point, which is added to the original sampling grid to obtain the final sampling set for sampling; after the sampling point position is initialized using the nearest initialization method, it is multiplied by a range factor to constrain the moving range of the sampling position to avoid overlapping of sampling points due to excessive offsets, thereby reducing the impact of prediction near the boundary and output artifacts, more accurately restoring feature resolution, and thus better detecting the location, size and other information of the fire point. DySample is based on the design of point sampling for upsampling. This helps to retain local features and prevent the loss of small object feature information, because DYsample can achieve clearer edges by splitting a point into multiple points, and realize the upsampling process through dynamic sampling, and find the correct semantic clustering of each upsampled point. Finally, the three feature maps of the same size, large, medium and small, are spliced ​​in the channel dimension.

[0069] Further, the RFD downsampling module includes a shallow RFD unit and a deep RFD unit;

[0070] The shallow RFD unit is used to effectively process a large amount of information and redundancy in the input image in the initial stage of feature extraction, overcome the problem of feature loss of small objects in remote sensing images caused by traditional downsampling methods, and thus improve network performance.

[0071] The deep RFD unit is used for the deep RFD (DRFD) unit, which is mainly used to process feature maps rich in semantic information in the subsequent stage of feature extraction, avoiding the problem of semantic information loss caused by traditional convolution downsampling, thereby improving the network's ability to represent features.

[0072] Specifically, RFD fuses multiple feature maps extracted by different downsampling techniques to create a more robust feature map through a set of complementary features. It is used to solve the problem that using convolution to reduce the dimension of the feature map may cause information loss of small objects in the RS image and reduce performance. It overcomes the limitations of traditional convolution downsampling, thereby performing a more accurate and robust analysis of remote sensing fire point images. Two versions of the RFD module were developed: shallow RFD (SRFD) and deep RFD (DRFD). The two modules are applied to the backbone network and FDR module to improve the model's ability to analyze fire point targets. The structures of the two modules are as follows: Figure 7 (a) Figure 7 (b).

[0073] Furthermore, the detection head is improved by combining the DCNv2 module, and the modulation mechanism introduced includes:

[0074]

[0075] Among them, y(p) is the result of the output feature after the modulated deformable convolution operation at position p, and w k is the weight Δm of each sampling point in the convolution kernel k is the coefficient that modulates the input feature at the kth position, x(p+p k +Δp k ) means at position p+p k +Δp k The value sampled from the input feature map. The principle of deformable convolution in DCNv2 is as follows Figure 8 As shown, it includes deformable convolution, which can not only adjust the sampling offset (through Δpk) to adapt to the geometric deformation of the fire point, but also modulate the input features through the learned feature amplitude Δmk. This enables the network to more flexibly adjust the spatial support area and the relative influence of the samples, so that it can more accurately extract features when dealing with objects with complex geometric changes; modulate the deformable RoI pooling, and adjust the pooling area through the learned offset and modulation scalar. This enables more relevant information to be used for detection tasks. For example, in the fire point detection task, the feature information within the fire point region (RoI) can be more effectively utilized to improve the positioning accuracy of the fire point.

[0076] Specifically, traditional convolution operations are performed on a fixed grid, which may lead to the loss of some important information. DCNv2 allows the convolution kernel to be adaptively sampled on the input feature map through a flexible sampling method, which means that the network can dynamically adjust the convolution operation according to the characteristics of the input data. This flexibility enables the network to better capture the subtle changes and complex shapes in the fire point data, better retain the key information in the input features, and reduce information loss. In addition, DCNv2 supports multi-scale feature learning, which is particularly important for fire point detection because the size and shape of fire points may vary greatly. By extracting features at different scales, the network can understand the input data more comprehensively, thereby improving the accuracy of detection. At the same time, fire point data may be affected by different angles, scales, and shapes. DCNv2 can effectively handle these geometric transformations through deformable convolution, so that the network can still maintain high detection performance when facing different fire point forms. In order to further enhance the ability of deformable convolutional networks to manipulate spatial support areas, DCNv2 introduces a modulation mechanism. With it, the deformable convolutional network module can not only adjust the offset of the perceived input features, but also modulate the amplitude of input features from different spatial positions.

[0077] like Fig. 9As shown, the improved RF-YOLO network structure of the embodiment of the present invention includes: a robust downsampling RFD module is used in the backbone part, including a shallow RFD and a deep RFD module, which effectively processes a large amount of information and redundancy in the input image, overcomes the problem of loss of small object features in remote sensing images by traditional downsampling methods, and thus improves network performance; in the feature extraction stage, a scale sequence feature fusion module SSFF module is used to reduce the noise in the image by weighted averaging each pixel in the image and its neighborhood, thereby enhancing the feature extraction capability of the fire point area; in the fire point feature learning stage, a triple feature encoder is used, which can enhance the model's perception of these subtle features, avoid missed detection due to unclear features, and help improve the performance of the model in the fire point detection task; a deformable convolution is embedded in the detection head, which can more effectively utilize the feature information in the fire point area (RoI) and improve the positioning accuracy of the fire point.

[0078] Furthermore, in order to provide a comprehensive and intuitive evaluation of the improved network performance, the present invention adopts indicators commonly used in target detection tasks to evaluate the quality of model performance, such as precision, recall rate, average accuracy, number of parameters, calculation speed, etc.

[0079] Precision: This is a basic indicator to measure the performance of the model in classification tasks. It represents the proportion of correctly identified positive samples to all positive samples detected by the model.

[0080] The calculation accuracy of the fire point identification network model during the test of the test set includes:

[0081]

[0082] Among them, Precision is the ratio of correctly identified positive samples to all positive samples detected by the model, TP is the number of positive samples correctly predicted as positive samples by the model, and FP is the number of negative samples incorrectly predicted as positive samples by the model.

[0083] Recall rate: Recall rate refers to the proportion of true positive samples correctly detected by the model to all positive samples in the data set. In the task of hotspot detection, recall rate is a key indicator that reflects the model's ability to identify hotspots.

[0084] Calculating the recall rate during the testing of the fire point recognition network model using the test set includes:

[0085]

[0086] Among them, Recall is the ratio of true positive samples correctly detected by the model to all positive samples in the data set, TP is the number of positive samples correctly predicted as positive samples by the model, FP is the number of negative samples incorrectly predicted as positive samples by the model, and FN is the number of positive samples incorrectly predicted as negative samples by the model.

[0087] P (Average Precision): AP represents the average of all recall-level precision values ​​for a specific category, reflecting the overall accuracy of the model’s detection performance on that category.

[0088] The average accuracy rate calculated during the testing of the fire point recognition network model using the test set includes:

[0089] AP=∫P(R)dR,

[0090] Among them, AP is the average precision value of all recall levels of a specific category, P is the Precision, R is the Recall rate, and dR is the change in the recall rate.

[0091] GFLOPs: GFLOPs is used to describe the time complexity of the model and is positively correlated with the performance of the required hardware. The parameters calculated during the test of the fire point recognition network model using the test set include:

[0092] GFLOPs = (2C i K 2 -1)HWC0,

[0093] Among them, GFLOPs is an indicator to measure the computational complexity of the model, which is used to indicate the number of floating-point operations required by the model when running; C i is the number of channels of the input data; K is the size of the convolution kernel; HWC0 represents the height, width and number of channels of the output feature map.

[0094] Parameter size: The parameter size indicates the number of parameters used by the model, measured in millions. It affects the final size of the output model after training.

[0095] We used an NVIDIA GeForce GTX 3090 graphics card to conduct comparative experiments on different Yolo models, setting the epoch to 300, the BatchSize to 32, and the optimizer to SGD. We used the collected data to compare the performance of different mainstream models, as shown in Table 1.

[0096] Table 1

[0097]

[0098] From the experimental data, Yolov8 has better performance, so the method of the present invention is based on the Yolov8 model.

[0099] Compared with the current mainstream image detection method using faster-Rcnn, the RF-Yolo of the present invention has smaller parameters and smaller calculation amount, and its accuracy, recall rate and average accuracy are better than faster-Rcnn. Moreover, compared with the mainstream Yolo series model, the effect of my RF-Yolo is also better, as shown in Table 2.

[0100] Table 2

[0101]

[0102] Use the experimental results to draw the PR curve as follows Fig.10 .

[0103] The experimental results show that RF-Yolo can detect smaller fire spots in different geographical environments and different time periods. In the results shown, RF-Yolo detects more smaller fire spots than other mainstream Yolo networks. This is due to the fact that the TFE structure and SSFF structure in the RF-Yolo structure can make more full use of the P2 layer feature map, so that RF-Yolo has better feature extraction capabilities for small targets of different scales. In addition, the RFD structure retains more feature information of small targets, and the improved detection head of DCNv2 can better distinguish between fire spots and different backgrounds, thereby improving the overall performance. The present invention also tested the full disk image of Sunflower 8 with a resolution of 6001*6001 using RF-Yolo.

[0104] For a natural disaster as destructive as wildfire, the best solution is to take measures to curb its development in its early stage. Therefore, early detection of wildfires is crucial to protecting the natural environment, biodiversity and human health. On the one hand, from the perspective of technological development, manual detection is ineffective and costly. Chemical sensors and smoke particle sensors based on the Internet of Things are susceptible to interference from fine particles such as dust in the environment, and the battery life of the sensors limits their long-term operation, requiring personnel maintenance and inconvenient for large-scale use. The method of judging the occurrence of a fire based on thermal radiation data obtained by thermal radiation detectors carried by remote sensing satellites has been developed to be relatively mature, and there is still a problem of increased false alarm rate. Compared with traditional methods, the present invention is based on computer vision and uses real remote sensing fire images for feature extraction and recognition. It can detect fire points well, has low cost of use, and has high time resolution. A 6001*6001 resolution full disk image of the sunflower 8 satellite is input into the model, and the time to detect the fire point in the entire image will not be greater than 30 seconds, while the imaging time interval of the sunflower 8 satellite is 10min / time, so the model of the present invention can detect fire points in real time within the large observation range of the sunflower 8 satellite. On the other hand, in fire detection based on deep learning, it is noted that many scholars have used deep learning models in combination for or fire point detection. Among them, KV et al. used Landset satellite data to train and improve CNN to achieve an accuracy of 60.87%. Kang.Y et al. used 2157 sunflower 8 satellite images, containing 93270 fire point samples, to train and improve CNN, with an F1 score of 74%, but the average time used to identify the fire point on each image was about 12 minutes. In contrast, the RF-Yolo proposed in the present invention has obvious advantages in detection speed and high accuracy. In addition, the RF-Yolo proposed in the present invention can effectively detect extremely small fire points in the image, and performs well compared to other mainstream Yolo models compared by the present invention, solving the pain point that tiny fire points are difficult to identify in remote sensing images. As one of the latest target detection models, the Yolov8 model has excellent efficiency and accuracy, but there is still room for improvement in detecting targets of extremely small fire points of different scales and diverse background textures. To this end, the present invention introduces the RFD downsampling module to reconstruct the backbone network of the original model, retains more fire point information in the feature extraction stage, and uses more high-resolution p2 feature layer feature information, so that the improved model can focus on tiny fire points, greatly improving the recall rate of the model, which is crucial for remote sensing fire point tasks. By using the self-made TFE module to fuse the triple scale features of different scales, the model learns the generalized features of fire points of different scales, and combined with the high-level features extracted by the SSFF module, the model's learning ability for multi-scale targets is improved.Finally, the detection head of the original model was improved. The DCNv2 module was used to improve the detection head, so that the model can notice the diverse texture features around the fire point, further improving the detection ability of the model. The test was conducted on the Sunflower 8 satellite remote sensing fire point data set. The recall rate of key indicators was 2% better than the mainstream model, and the average accuracy was improved by 1%. After conducting fire point detection experiments on full-frame remote sensing images, it was confirmed that RF-Yolo has the ability to identify fire points in Sunflower 8 remote sensing satellite images, providing a new method for timely detection of fires.

[0105] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for real-time identification of fire points in satellite remote sensing images, characterized in that: include: Acquire a remote sensing fire point data set, and divide the remote sensing fire point data set into a training set, a test set, and a validation set; Constructing an initial fire point recognition network model, training the fire point recognition network model through the training set and the validation set to obtain a fire point recognition network model, testing the fire point recognition network model through the test set to obtain a final fire point recognition network model, wherein the final fire point recognition network model is obtained by improving the shallow feature extraction module and the multi-scale feature fusion module of YOLOv8; The final fire point recognition network model is used to identify fire points in the target area to obtain recognition results.

2. The method for real-time identification of fire points in satellite remote sensing images according to claim 1, characterized in that: The final fire point recognition network model is obtained by improving the shallow feature extraction module and multi-scale feature fusion module of YOLOv8 and includes: The RFD downsampling module is introduced into the YOLOv8 backbone network to process a large amount of information and redundancy in the input image in the initial stage of feature extraction, and to process feature maps with rich semantic information in the subsequent stage of feature extraction; Use the SSFF module to perform multi-scale feature extraction on the high-resolution feature layer P2, the medium-resolution feature layer P3, and the low-resolution feature layer P4; Design a multi-scale encoder to process feature maps of different scales and add the processed feature information element by element; The detection head is improved by combining the DCNv2 module to predict and identify fire point targets.

3. The method for real-time identification of fire points in satellite remote sensing images as claimed in claim 2, characterized in that: The convolution of feature maps P3, P4 and P5 with a series of Gaussian kernels with increasing standard deviations in the SSFF module includes: F σ (i,j)=∑ u ∑ v f(i-u,i-v)×G σ (u,v), Among them, F σ (i, j) represents the value of the convolution result at the coordinate (i, j), G σ (u, v) represents the value of the Gaussian kernel at (u, v), f(iu, iv)*G σ (u, v) means multiplying the value of the input image at (iu, jv) by the value of the Gaussian kernel at (u, v), and reducing the noise in the image by taking a weighted average of each pixel in the image and its neighborhood; G σ (x, y) is a two-dimensional Gaussian function, σ is the standard deviation, σ 2 is the variance, is the normalization constant, e is the base of the natural logarithm, (x 2 +y 2 ) represents the square of the distance from the point to the origin, and f is a two-dimensional feature map.

4. The method for real-time identification of fire points in satellite remote sensing images as claimed in claim 2, characterized in that: The multi-scale encoder is a triple feature encoder that fuses feature maps of different sizes to improve feature information.

5. The method for real-time identification of fire points in satellite remote sensing images as claimed in claim 2, characterized in that: The RFD downsampling module includes a shallow RFD unit and a deep RFD unit; The shallow RFD unit is used to process a large amount of information and redundancy in the input image at the initial stage of feature extraction; The deep RFD unit is used to process feature maps rich in semantic information in the subsequent stage of feature extraction.

6. The method for real-time identification of fire points in satellite remote sensing images as claimed in claim 2, characterized in that: The detection head is improved by combining the DCNv2 module, and feature modulation is introduced, including: Among them, y(p) is the result of the output feature after the modulated deformable convolution operation at position p, and w k is the weight Δm of each sampling point in the convolution kernel k is the coefficient that modulates the input feature at the kth position, x(p+p k +Δp k ) means at position p+p k +Δp k Values ​​sampled from the input feature map.

7. The method for real-time identification of fire points in satellite remote sensing images according to claim 1, characterized in that: The calculation accuracy Precision during the test of the fire point recognition network model through the test set includes: Among them, Precision is the ratio of correctly identified positive samples to all positive samples detected by the model, TP is the number of positive samples correctly predicted as positive samples by the model, and FP is the number of negative samples incorrectly predicted as positive samples by the model.

8. The method for real-time identification of fire points in satellite remote sensing images according to claim 1, characterized in that: The calculation of the recall rate Recall during the testing of the fire point recognition network model using the test set includes: Among them, Recall is the ratio of true positive samples correctly detected by the model to all positive samples in the data set, TP is the number of positive samples correctly predicted as positive samples by the model, FP is the number of negative samples incorrectly predicted as positive samples by the model, and FN is the number of positive samples incorrectly predicted as negative samples by the model.

9. The method for real-time identification of fire points in satellite remote sensing images as claimed in claim 1, characterized in that: The average accuracy AP is calculated during the testing of the fire point recognition network model using the test set, including: AP=∫P(R)dR, Among them, AP is the average precision value of all recall levels of a specific category, P is the Precision, R is the Recall rate, and dR is the change in the recall rate.

10. The method for real-time identification of fire points in satellite remote sensing images according to claim 1, characterized in that: The parameter amount GFLOPs calculated during the test of the fire point recognition network model using the test set includes: GFLOPs=(2C i K 2 -1)HWC0, Among them, GFLOPs is an indicator to measure the computational complexity of the model, which is used to indicate the number of floating-point operations required by the model when running; C i is the number of channels of the input data; K is the size of the convolution kernel; HWC0 represents the height, width and number of channels of the output feature map.

Images