Forest fire early warning method and system based on super-resolution neural operator

Abstract

The present invention relates to the technical field of forest fire identification and early warning, and specifically to a forest fire early warning method and system based on a super-resolution neural operator, which is as follows: extracting the initial feature map of a low-resolution forest fire image to be identified; constructing a super-resolution feature capture network, three HAFM hierarchical attention fusion modules, MSMO multi-scale mapping operator modules, super-resolution feature fusion modules and super-resolution image reconstruction modules, inputting the initial feature map into the super-resolution feature capture network, and finally outputting a super-resolution image; constructing a forest fire identification module, inputting the super-resolution image into a ResNet‑18 neural network model, obtaining identification features, and finally obtaining the identification results of the forest fire image through the fully connected layer processing of the identification features, and judging whether to issue an early warning based on the identification results. The present invention can solve the problem that low-resolution fire images cannot be accurately identified and timely warned in existing forest fire identification and detection.

Description

Technical Field

[0001] The present invention relates to the technical field of forest fire identification and early warning, and in particular to a forest fire early warning method and system based on a super-resolution neural operator. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, image recognition technology has been increasingly used in forest fire prevention systems. This protection system mainly uses image recognition technology to monitor and analyze images in the forest in real time to achieve early warning, range positioning and dynamic tracking of forest fires. By using the deep learning algorithm of artificial intelligence technology to extract and classify fire features in images, it is possible to quickly detect whether a fire has occurred in the forest, thereby improving the response speed of forest fire prevention, reducing the burden of manual patrols and improving the efficiency of forest fire fighting. The image recognition field of artificial intelligence technology has become an important part of the forest fire prevention system.

[0003] In the existing forest fire prevention scenarios, there is a problem that video surveillance cannot fully cover the entire area. If the monitoring distance is far from the location of the fire disaster, ordinary image recognition technology cannot accurately identify and detect the forest fire situation in the image. Therefore, a super-resolution neural operator fire warning method is proposed, which can parse multi-scale high-resolution images from low-resolution images and improve the recognition ability of forest fire images.

[0004] Therefore, the present invention proposes a forest fire early warning method and system based on super-resolution neural operator to solve the above problems. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention develops a forest fire early warning method and system based on a super-resolution neural operator. The present invention utilizes the high-frequency feature similarity between different scales within a low-resolution image, and through multi-branch feature extraction and efficient residual fusion, can solve the problem that low-resolution fire images cannot be accurately identified in existing forest fire recognition and detection.

[0006] On the one hand, the technical solution to the technical problem of the present invention is a forest fire early warning method based on a super-resolution neural operator, comprising the following steps:

[0007] S1. Extract low-resolution forest fire images to be identified The initial feature map ;

[0008] S2, build a super-resolution feature capture network, three HAFM hierarchical attention fusion modules, MSMO multi-scale mapping operator module, super-resolution feature fusion module and super-resolution image reconstruction module, and convert the initial feature map Input to the super-resolution feature capture network, and finally output the super-resolution image ;

[0009] S3, build a forest fire recognition module, and transform the super-resolution image Input into the ResNet-18 neural network model to obtain recognition features , and finally identify the features After processing by the fully connected layer, the recognition result of the forest fire image is obtained, and whether to issue an early warning is determined based on the recognition result.

[0010] S1 is as follows:

[0011] Collect low-resolution forest fire images to be searched , , Representing images Height, Representing images The width of Representing images The number of channels is obtained by using a convolution kernel of size , the step length is of The convolutional layer is used to retrieve the low-resolution forest fire image. Perform initial feature extraction to obtain the initial feature map , the calculation formula is as follows:

[0012] ,

[0013] in, represents the convolution kernel weight, Represents the offset corresponding to the convolution kernel, is a two-dimensional convolution operation, , Represents the initial feature map Height, Represents the initial feature map The width of Represents the initial feature map The number of channels.

[0014] S2 is as follows:

[0015] S2.1. The specific process in the HAFM hierarchical attention fusion module is as follows:

[0016] The initial feature map Input to the HAFM module, which includes a cross-scale global branch, a local branch, a fusion module, and an upsampling module;

[0017] S2.1.1. Initial feature map First, after the first HAFM module, the initial characteristics After the cross-scale global branch of the first HAFM module, the cross-scale global enhanced features are obtained , initial features After the local branch of the first HAFM module, the local features are obtained , globally enhancing features across scales and local features Through the fusion module of the first HAFM module, the fusion intermediate features are obtained , fusion of intermediate features The upsampling module after the first HAFM module outputs the final features , as follows:

[0018] (1) Initial features After the cross-scale global branch of the first HAFM module, the cross-scale global branch includes a downsampling module, a sliding window mechanism, a convolution layer, a normalization layer, a mapping module, and a deconvolution layer;

[0019] Initial feature map The low-resolution features are obtained by downsampling , the calculation formula is as follows:

[0020] ,

[0021] in, represents the downsampling operation, represents the downsampling scale factor, Indicates the assignment of the downsampling scale factor, setting is 3;

[0022] Then the initial feature map is transformed into and low-resolution features Divide into fixed-size regional blocks and obtain corresponding regional features and , regional characteristics and The dimension is , Indicates the step size of the window movement, set is 2, the calculation process is as follows:

[0023] ,

[0024] ,

[0025] in, Indicates the size of each region block, Represents the horizontal and vertical coordinates of the sliding window, Represents the initial feature map exist The regional characteristics of Represents low-resolution features exist Regional characteristics of the location;

[0026] Then through the convolution kernel The convolutional layer of the regional features and Perform linear transformation to obtain the corresponding features and , and then calculate the similarity of cross-scale global features through the inner product. The calculation process is as follows:

[0027] ,

[0028] in, Indicates Initial feature map region block features and Low-resolution region block features The similarity of , Represents the initial feature map The total number of region blocks, , Low-resolution features The total number of region blocks;

[0029] Then through Normalize the similarity and generate the cross-scale global attention weight. The calculation process is as follows:

[0030] ,

[0031] in, express Initial feature map region block features and Low-resolution region block features The cross-scale global attention weights;

[0032] Combined with the cross-scale global attention weight, the regional features Remap to the initial feature map and aggregate all features to obtain cross-scale features , and the generated cross-scale features are transformed by deconvolution Restore to the initial feature size to obtain cross-scale global enhanced features , the calculation formula is as follows:

[0033] ,

[0034] ,

[0035] in, Indicates Cross-scale features, represents the deconvolution operation;

[0036] (2) Initial feature map Input to the local branch to obtain local features The local branch contains a layer of convolution kernel size The convolutional layer, The activation function and residual connection are calculated as follows:

[0037] ,

[0038] in, Represents the activation function Operations;

[0039] (3) Local features and cross-scale global enhancement features Input into the fusion module in the first hierarchical attention fusion module to obtain the fused intermediate features , fusion is achieved through feature concatenation, and the convolution kernel size is The convolutional layer integrates the feature channels and finally uses The activation function enhances the model to learn complex image features. The calculation formula of this process is as follows:

[0040] ,

[0041] in, Represents a splicing operation, express Operation of activation function;

[0042] (4) Fusion of intermediate features Generate high-resolution features through upsampling module , the upsampling process is achieved through strided convolution and deconvolution operations, and uses The activation function introduces nonlinear relationship calculation, and the calculation process is as follows:

[0043] ,

[0044] in, represents the weight in the strided convolution, represents the weight in deconvolution, represents the bias in the stride convolution, represents the bias in deconvolution, represents strided convolution, Represents the activation function Operations;

[0045] S2.1.2. The high-resolution features obtained by the first hierarchical attention fusion module Input to the second hierarchical attention fusion module, high-resolution features After the cross-scale global branch in the second hierarchical attention fusion module, the cross-scale global enhanced features are obtained. , high-resolution features After the local branch in the second hierarchical attention fusion module, the local features are obtained , cross-scale global enhancement features and local features Through the fusion module in the second hierarchical attention fusion module, the fused intermediate features are obtained , intermediate features The final high-resolution features are output through the upsampling module in the second hierarchical attention fusion module ;

[0046] S2.1.3. The high-resolution features obtained by the second hierarchical attention fusion module , input to the third hierarchical attention fusion module, high-resolution features After the cross-scale global branch in the third hierarchical attention fusion module, the cross-scale global enhanced features are obtained. , high-resolution features After the local branch in the third hierarchical attention fusion module, the local features are obtained , cross-scale global enhancement features and local features Through the fusion module in the third hierarchical attention fusion module, the fusion intermediate features are obtained .

[0047] S2.2. The specific process in the MSMO multi-scale mapping operator module is as follows:

[0048] The MSMO multi-scale mapping operator module includes a high-dimensional mapping module, a super-resolution attention feature capture module, and an image projection module;

[0049] S2.2.1. Fusion of intermediate features Input into the MSMO module to obtain the reconstructed high-resolution features , fusion of intermediate features The process through the multi-scale mapping operator module is as follows:

[0050] Fusion of intermediate features First, it passes through the high-dimensional mapping module, which will fuse the intermediate features Mapped to a high-dimensional feature space to obtain the enhanced potential representation features ,in Represents the fusion of intermediate features The specific process is that the high-dimensional mapping module passes through the CNN encoder , the intermediate features Pixel value Promoted to high-dimensional feature space, the promoted features are obtained ,CNN encoder It consists of two convolutional layers with output channels of 128 and 256 and one The activation function is constructed, and then the improved features are Perform position encoding and weighted interpolation to obtain potential representation features , the calculation formula is as follows:

[0051] ,

[0052] ,

[0053] in, , express The set of pixel coordinates in , represents the linear transformation function, The weighting coefficient is The weighted interpolation factor when , The weighting coefficient is The coordinate offset when represents the weighted coefficient in different neighborhoods, ;

[0054] Then the super-resolution attention feature capture module uses a multi-layer attention mechanism to represent the latent features through a kernel integration operation. Capture global features and restore details to obtain high-level feature representation , Indicates The kernel integral output of the layer is The kernel integral of the layer, , the calculation formula in the kernel integral of each layer is as follows:

[0055] First layer:

[0056] ,

[0057] No. layer:

[0058] ,

[0059] in, represents a feed-forward neural network, Indicates Layer kernel integration operation, Indicates Layer kernel integral output, Indicates Layer kernel integral output, after multiple iterations, output The kernel integral output of the layer;

[0060] Then the image projection module will Kernel integral output of the layer Project back to RGB space to get high-resolution image features , this process will be projected Mapped to RGB space, the calculation formula is as follows:

[0061] ,

[0062] in, Represents a projection operation, which contains a convolution kernel size of The convolutional layer;

[0063] S2.2.2. Fusion of intermediate features Input into the MSMO module, the potential representation features are improved after the high-dimensional mapping module , express The set of pixel coordinates in , potentially representing features Then, through the super-resolution attention feature capture module, we get the high-level feature representation , high-level feature representation Finally, the reconstructed high-resolution features are obtained through the image projection module. ;

[0064] S2.2.3. Fusion of intermediate features Input into the MSMO module, the potential representation features are improved after the high-dimensional mapping module , express The set of pixel coordinates in , potentially representing features After the super-resolution attention feature capture module, high-level feature representation is obtained , high-level feature representation After the image projection module, the reconstructed high-resolution features are obtained .

[0065] S2.3, the specific process in the super-resolution feature fusion module is as follows:

[0066] The three high-resolution features obtained by the multi-scale mapping operator module , and Input to the super-resolution feature fusion module to obtain the super-resolution fusion feature map , the process uses Feature stitching combines high-resolution features , as well as Concatenate according to the channel dimension to get a new feature map , the calculation formula is as follows:

[0067] .

[0068] S2.4, the specific process in the super-resolution image reconstruction module is as follows:

[0069] The fusion features obtained by the super-resolution feature fusion module Input to the super-resolution image reconstruction module to obtain a super-resolution image , this module uses the deconvolution layer to upsample the current feature map and uses the convolution kernel size The convolution layer adjusts the feature channel to integrate the fusion features Restore to original image size while using The activation function introduces nonlinear calculations in this module to enhance the module's ability to learn complex image features. The calculation formula is as follows:

[0070] ,

[0071] in, express The convolution kernel.

[0072] S3 is as follows:

[0073] The super-resolution image obtained by the super-resolution image reconstruction module Input into the ResNet-18 neural network model to obtain the recognition features , through the fully connected layer , the identification features The feature vector is transformed into the final category prediction , and finally predict the label The recognition result of forest fire in the image is obtained through the Softmax activation function. The calculation formula is as follows:

[0074] ,

[0075] ,

[0076] ,

[0077] in, represents the scores of all categories, , Indicates The probability of the category, Indicates all categories, represents the probability of the first category, Represents a low-resolution forest fire image to be identified The probability of identifying it as a forest fire.

[0078] On the other hand, the present invention also proposes a forest fire early warning system based on a super-resolution neural operator, and implements a forest fire early warning method based on a super-resolution neural operator, which specifically includes the following modules:

[0079] Image input and initial feature extraction module: extract features from the input low-resolution fire image and capture the initial features of the image;

[0080] Hierarchical attention fusion module: input the initial features of the low-resolution image into the hierarchical attention model to obtain the global fusion features;

[0081] Multi-scale mapping operator module: The global fusion features output by the hierarchical attention fusion module are input into the multi-scale mapping operator module to obtain enhanced high-resolution features;

[0082] Super-resolution feature fusion module: The enhanced high-resolution features output by the multi-scale mapping operator module are input into the super-resolution feature fusion module to obtain super-resolution fusion features;

[0083] Super-resolution image reconstruction module: input the super-resolution fusion features output by the super-resolution feature fusion module into the super-resolution image reconstruction module to obtain a super-resolution image;

[0084] Forest fire recognition module: The super-resolution image generated by the super-resolution image reconstruction module is input into the forest fire recognition module to obtain the final recognition result;

[0085] Forest fire warning module: receives the final recognition result obtained by the forest fire recognition module, and determines whether to issue a warning based on the recognition result.

[0086] The effects provided in the content of the invention are only the effects of the embodiments, rather than all the effects of the invention. The above technical solution has the following advantages or beneficial effects:

[0087] The present invention proposes an innovative method for solving the problem that the fire monitoring in the forest is far away from the fire disaster, and the ordinary image recognition technology cannot accurately identify and judge the forest fire situation in the image. In the field of image recognition, if there is a small object in the recognition image or the image resolution is low, the image recognition technology cannot accurately identify the required object. To solve this problem, the present invention proposes a forest fire early warning method based on a super-resolution neural operator.

[0088] First, the present invention designs a deep learning network based on super-resolution neural operators for automatically identifying and extracting key features in forest fire images. The model can dynamically adjust the feature extraction process according to images of different resolutions, and can effectively capture multi-scale and high-frequency features in images even in the case of low-resolution images, thereby improving the recognition ability of images. The present invention helps to solve the problem of low resolution of recognized images due to long monitoring distances, and helps to improve the accuracy of model fire recognition. The method proposed in the present invention effectively improves the reconstruction quality of low-resolution images through the collaborative work of super-resolution feature capture networks, multi-scale mapping operator modules, and super-resolution feature fusion modules, and realizes the recognition and detection of fire images in combination with the ResNet-18 model. Finally, through experiments and tests on three tasks in the DFS dataset, the superior performance of the method in forest fire image recognition is demonstrated. By proposing an innovative super-resolution neural operator model, the present invention provides an efficient and accurate solution for the recognition of low-resolution fire images, which is of great significance to the research and practice in the field of forest fire prevention. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0090] Figure 1 It is a structural schematic diagram of the present invention. DETAILED DESCRIPTION

[0091] In order to clearly illustrate the technical features of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below.

[0092] Example 1

[0093] A forest fire early warning method based on super-resolution neural operator comprises the following steps:

[0094] S1. Extract low-resolution forest fire images to be identified The initial feature map ;

[0095] S2, build a super-resolution feature capture network, three HAFM hierarchical attention fusion modules, MSMO multi-scale mapping operator module, super-resolution feature fusion module and super-resolution image reconstruction module, and convert the initial feature map Input to the super-resolution feature capture network, and finally output the super-resolution image ;

[0096] S3, build a forest fire recognition module, and transform the super-resolution image Input into the ResNet-18 neural network model to obtain recognition features , and finally identify the features After processing by the fully connected layer, the recognition result of the forest fire image is obtained, and whether to issue an early warning is determined based on the recognition result.

[0097] S1 is as follows:

[0098] Collect low-resolution forest fire images to be searched , , Representing images Height, Representing images The width of Representing images The number of channels is obtained by using a convolution kernel of size , the step length is of The convolutional layer is used to retrieve the low-resolution forest fire image. Perform initial feature extraction to obtain the initial feature map , the calculation formula is as follows:

[0099] ,

[0100] in, represents the convolution kernel weight, Represents the offset corresponding to the convolution kernel, is a two-dimensional convolution operation, , Represents the initial feature map Height, Represents the initial feature map The width of Represents the initial feature map The number of channels.

[0101] S2 is as follows:

[0102] S2.1. The specific process in the HAFM hierarchical attention fusion module is as follows:

[0103] The initial feature map Input to the HAFM module, which includes a cross-scale global branch, a local branch, a fusion module, and an upsampling module;

[0104] S2.1.1. Initial feature map First, after the first HAFM module, the initial characteristics After the cross-scale global branch of the first HAFM module, the cross-scale global enhanced features are obtained , initial features After the local branch of the first HAFM module, the local features are obtained , globally enhancing features across scales and local features Through the fusion module of the first HAFM module, the fusion intermediate features are obtained , fusion of intermediate features The upsampling module after the first HAFM module outputs the final features , as follows:

[0105] (1) Initial features After the cross-scale global branch of the first HAFM module, the cross-scale global branch includes a downsampling module, a sliding window mechanism, a convolution layer, a normalization layer, a mapping module, and a deconvolution layer;

[0106] Initial feature map The low-resolution features are obtained by downsampling , the calculation formula is as follows:

[0107] ,

[0108] in, represents the downsampling operation, represents the downsampling scale factor, Indicates the assignment of the downsampling scale factor, setting is 3;

[0109] Then the initial feature map is transformed into and low-resolution features Divide into fixed-size regional blocks and obtain corresponding regional features and , regional characteristics and The dimension is , Indicates the step size of the window movement, set is 2, the calculation process is as follows:

[0110] ,

[0111] ,

[0112] in, Indicates the size of each region block, Represents the horizontal and vertical coordinates of the sliding window, Represents the initial feature map exist The regional characteristics of Represents low-resolution features exist Regional characteristics of the location;

[0113] Then through the convolution kernel The convolutional layer of the regional features and Perform linear transformation to obtain the corresponding features and , and then calculate the similarity of cross-scale global features through the inner product. The calculation process is as follows:

[0114] ,

[0115] in, Indicates Initial feature map region block features and Low-resolution region block features The similarity of , Represents the initial feature map The total number of region blocks, , Low-resolution features The total number of region blocks;

[0116] Then through Normalize the similarity and generate the cross-scale global attention weight. The calculation process is as follows:

[0117] ,

[0118] in, express Initial feature map region block features and Low-resolution region block features The cross-scale global attention weights;

[0119] Combined with the cross-scale global attention weight, the regional features Remap to the initial feature map and aggregate all features to obtain cross-scale features , and the generated cross-scale features are transformed by deconvolution Restore to the initial feature size to obtain cross-scale global enhanced features , the calculation formula is as follows:

[0120] ,

[0121] ,

[0122] in, Indicates Cross-scale features, represents the deconvolution operation;

[0123] (2) Initial feature map Input to the local branch to obtain local features The local branch contains a layer of convolution kernel size The convolutional layer, The activation function and residual connection are calculated as follows:

[0124] ,

[0125] in, Represents the activation function Operations;

[0126] (3) Local features and cross-scale global enhancement features Input into the fusion module in the first hierarchical attention fusion module to obtain the fused intermediate features , fusion is achieved through feature concatenation, and the convolution kernel size is The convolutional layer integrates the feature channels and finally uses The activation function enhances the model to learn complex image features. The calculation formula of this process is as follows:

[0127] ,

[0128] in, Represents a splicing operation, express Operation of activation function;

[0129] (4) Fusion of intermediate features Generate high-resolution features through upsampling module , the upsampling process is achieved through strided convolution and deconvolution operations, and uses The activation function introduces nonlinear relationship calculation, and the calculation process is as follows:

[0130] ,

[0131] in, represents the weight in the strided convolution, represents the weight in deconvolution, represents the bias in the stride convolution, represents the bias in deconvolution, represents strided convolution, Represents the activation function Operations;

[0132] S2.1.2. The high-resolution features obtained by the first hierarchical attention fusion module Input to the second hierarchical attention fusion module, high-resolution features After the cross-scale global branch in the second hierarchical attention fusion module, the cross-scale global enhanced features are obtained. , high-resolution features After the local branch in the second hierarchical attention fusion module, the local features are obtained , cross-scale global enhancement features and local features Through the fusion module in the second hierarchical attention fusion module, the fusion intermediate features are obtained , intermediate features The final high-resolution features are output through the upsampling module in the second hierarchical attention fusion module ;

[0133] S2.1.3. The high-resolution features obtained by the second hierarchical attention fusion module , input to the third hierarchical attention fusion module, high-resolution features After the cross-scale global branch in the third hierarchical attention fusion module, the cross-scale global enhanced features are obtained. , high-resolution features After the local branch in the third hierarchical attention fusion module, the local features are obtained , cross-scale global enhancement features and local features Through the fusion module in the third hierarchical attention fusion module, the fusion intermediate features are obtained .

[0134] S2.2. The specific process in the MSMO multi-scale mapping operator module is as follows:

[0135] The MSMO multi-scale mapping operator module includes a high-dimensional mapping module, a super-resolution attention feature capture module, and an image projection module;

[0136] S2.2.1. Fusion of intermediate features Input into the MSMO module to obtain the reconstructed high-resolution features , fusion of intermediate features The process through the multi-scale mapping operator module is as follows:

[0137] Fusion of intermediate features First, it passes through the high-dimensional mapping module, which will fuse the intermediate features Mapped to a high-dimensional feature space to obtain the enhanced potential representation features ,in Represents the fusion of intermediate features The specific process is that the high-dimensional mapping module passes through the CNN encoder , the intermediate features Pixel value Promoted to high-dimensional feature space, the promoted features are obtained ,CNN encoder It consists of two convolutional layers with output channels of 128 and 256 and one The activation function is constructed, and then the improved features are Perform position encoding and weighted interpolation to obtain potential representation features , the calculation formula is as follows:

[0138] ,

[0139] ,

[0140] in, , express The set of pixel coordinates in , represents the linear transformation function, The weighting coefficient is The weighted interpolation factor when , The weighting coefficient is The coordinate offset when represents the weighted coefficient in different neighborhoods, ;

[0141] Then the super-resolution attention feature capture module uses a multi-layer attention mechanism to represent the latent features through a kernel integration operation. Capture global features and restore details to obtain high-level feature representation , Indicates The kernel integral output of the layer is The kernel integral of the layer, , the calculation formula in the kernel integral of each layer is as follows:

[0142] First layer:

[0143] ,

[0144] No. layer:

[0145] ,

[0146] in, represents a feed-forward neural network, Indicates Layer kernel integration operation, Indicates Layer kernel integral output, Indicates Layer kernel integral output, after multiple iterations, output The kernel integral output of the layer;

[0147] Then the image projection module will Kernel integral output of the layer Project back to RGB space to get high-resolution image features , this process will be projected Mapped to RGB space, the calculation formula is as follows:

[0148] ,

[0149] in, Represents a projection operation, which contains a convolution kernel size of The convolutional layer;

[0150] S2.2.2. Fusion of intermediate features Input into the MSMO module, the potential representation features are improved after the high-dimensional mapping module , express The set of pixel coordinates in , potentially representing features Then, through the super-resolution attention feature capture module, we get the high-level feature representation , high-level feature representation Finally, the reconstructed high-resolution features are obtained through the image projection module. ;

[0151] S2.2.3. Fusion of intermediate features Input into the MSMO module, the potential representation features are improved after the high-dimensional mapping module , express The set of pixel coordinates in , potentially representing features After the super-resolution attention feature capture module, high-level feature representation is obtained , high-level feature representation After the image projection module, the reconstructed high-resolution features are obtained .

[0152] S2.3, the specific process in the super-resolution feature fusion module is as follows:

[0153] The three high-resolution features obtained by the multi-scale mapping operator module , and Input to the super-resolution feature fusion module to obtain the super-resolution fusion feature map , the process uses Feature stitching combines high-resolution features , as well as Concatenate according to the channel dimension to get a new feature map , the calculation formula is as follows:

[0154] .

[0155] S2.4, the specific process in the super-resolution image reconstruction module is as follows:

[0156] The fusion features obtained by the super-resolution feature fusion module Input to the super-resolution image reconstruction module to obtain a super-resolution image , this module uses the deconvolution layer to upsample the current feature map and uses the convolution kernel size The convolution layer adjusts the feature channel to integrate the fusion features Restore to original image size while using The activation function introduces nonlinear calculations in this module to enhance the module's ability to learn complex image features. The calculation formula is as follows:

[0157] ,

[0158] in, express The convolution kernel.

[0159] S3 is as follows:

[0160] The super-resolution image obtained by the super-resolution image reconstruction module Input into the ResNet-18 neural network model to obtain the recognition features , through the fully connected layer , the identification features The feature vector is transformed into the final category prediction , and finally predict the label The recognition result of forest fire in the image is obtained through the Softmax activation function. The calculation formula is as follows:

[0161] ,

[0162] ,

[0163] ,

[0164] in, represents the scores of all categories, , Indicates The probability of the category, Indicates all categories, represents the probability of the first category, Represents a low-resolution forest fire image to be identified The probability of identifying it as a forest fire.

[0165] Example 2

[0166] A forest fire early warning method based on a super-resolution neural operator includes a module for executing processing instructions of each step in a forest fire early warning method based on a super-resolution neural operator, specifically as follows:

[0167] Image input and initial feature extraction module: extract features from the input low-resolution fire image and capture the initial features of the image;

[0168] Hierarchical attention fusion module: input the initial features of the low-resolution image into the hierarchical attention model to obtain the global fusion features;

[0169] Multi-scale mapping operator module: The global fusion features output by the hierarchical attention fusion module are input into the multi-scale mapping operator module to obtain enhanced high-resolution features;

[0170] Super-resolution feature fusion module: The enhanced high-resolution features output by the multi-scale mapping operator module are input into the super-resolution feature fusion module to obtain super-resolution fusion features;

[0171] Super-resolution image reconstruction module: input the super-resolution fusion features output by the super-resolution feature fusion module into the super-resolution image reconstruction module to obtain a super-resolution image;

[0172] Forest fire recognition module: The super-resolution image generated by the super-resolution image reconstruction module is input into the forest fire recognition module to obtain the final recognition result;

[0173] Forest fire warning module: receives the final recognition result obtained by the forest fire recognition module, and determines whether to issue a warning based on the recognition result.

[0174] Example 3

[0175] In order to better demonstrate the technical effect of the present invention, the method of the present invention is compared with the existing method in different scenarios. According to the unified experimental conditions and environmental configuration, the proposed forest fire warning method based on super-resolution neural operators is compared with the current mainstream image recognition models in terms of recognition effect. The mainstream recognition models used in this experiment are: CenterNet, which realizes target detection by center point prediction and can handle smaller targets and targets with higher density, but the computational complexity of the model is relatively high; EfficientDet, which combines the backbone network and feature pyramid structure of EfficientNet with good accuracy and efficiency; Fast R-CNN, which is a two-stage target detection method with high accuracy but relatively slow inference speed; RetinaNet, which uses focal loss to solve the problem of foreground-background imbalance and is suitable for detecting sparse targets; YOLO series, including YOLOv7 and YOLOv8, are all single-stage detection models with good real-time performance and accuracy.

[0176] Four indicators are used in the experiment to measure the effectiveness of the proposed method and other recognition models, namely precision (Precision, Pre), recall (Recall), F1-score and mAP50(%). Precision indicates the proportion of samples correctly predicted as fire (Fire) or smoke (Smoke) to all samples predicted as fire or smoke (All); recall indicates the proportion of samples correctly predicted as fire or smoke to all samples actually being fire or smoke; F1-score reflects the overall performance of the model, which is the harmonic mean of precision and recall; map50(%) indicates the average value of average precision (AP) calculated when the intersection-over-union ratio of the predicted box in the model and the true box in the image is greater than or equal to 50%.

[0177] The forest fire warning method based on super-resolution neural operator proposed in this invention is experimented on the DFS (Dataset for Fire and Smoke detection, DFS) dataset. The images in this dataset cover a variety of fire scenes, such as forest fire, grass fire, building fire, road fire and various small target fires. There are 9426 fire images, smoke images and other interference images in total. In the experiment, the DFS dataset is divided into training set, validation set and test set in a ratio of 8:1:1.

[0178] The proposed forest fire warning method based on super-resolution neural operator is experimentally verified on the DFS dataset. The experimental results are shown in Table 1. The forest fire warning method proposed in the present invention has better performance than other mainstream image recognition models. In the smoke recognition task in the DFS dataset, the four indicators Pre, Recall, F1-score and mAP50(%) are all optimal; in the fire image recognition task, the three indicators Pre, Recall and F1-score are optimal, and the prediction accuracy Pre and the comprehensive performance indicator F1-score of the model are improved by 1.7% and 2.4% respectively compared with the mainstream image recognition model YOLOv8. The experimental results show the effectiveness of the forest fire warning method based on super-resolution neural operator proposed in the present invention in the field of forest fire recognition.

[0179] Table 1 Comparison results of the model of the present invention in the DFS dataset.

[0180]

[0181] Although the above describes the specific implementation mode of the invention in conjunction with the drawings, it is not intended to limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A forest fire early warning method based on super-resolution neural operator, characterized in that: The following steps are involved: S1. Extract low-resolution forest fire images to be identified The initial feature map ; S2, build a super-resolution feature capture network, three HAFM hierarchical attention fusion modules, MSMO multi-scale mapping operator module, super-resolution feature fusion module and super-resolution image reconstruction module, and convert the initial feature map Input to the super-resolution feature capture network, and finally output the super-resolution image ; S2 is as follows: S2.

1. The specific process in the HAFM hierarchical attention fusion module is as follows: The initial feature map Input to the HAFM module, which includes a cross-scale global branch, a local branch, a fusion module, and an upsampling module; S2.1.

1. Initial feature map First, after the first HAFM module, the initial characteristics After the cross-scale global branch of the first HAFM module, the cross-scale global enhanced features are obtained , initial features After the local branch of the first HAFM module, the local features are obtained , globally enhancing features across scales and local features Through the fusion module of the first HAFM module, the fusion intermediate features are obtained , fusion of intermediate features The upsampling module after the first HAFM module outputs the final features ; S2.1.

2. The high-resolution features obtained by the first hierarchical attention fusion module Input to the second hierarchical attention fusion module, high-resolution features After the cross-scale global branch in the second hierarchical attention fusion module, the cross-scale global enhanced features are obtained. , high-resolution features After the local branch in the second hierarchical attention fusion module, the local features are obtained , cross-scale global enhancement features and local features Through the fusion module in the second hierarchical attention fusion module, the fusion intermediate features are obtained , intermediate features The final high-resolution features are output through the upsampling module in the second hierarchical attention fusion module ; S2.1.

3. The high-resolution features obtained by the second hierarchical attention fusion module , input to the third hierarchical attention fusion module, high-resolution features After the cross-scale global branch in the third hierarchical attention fusion module, the cross-scale global enhanced features are obtained. , high-resolution features After the local branch in the third hierarchical attention fusion module, the local features are obtained , cross-scale global enhancement features and local features Through the fusion module in the third hierarchical attention fusion module, the fusion intermediate features are obtained ; S2.

2. The specific process in the MSMO multi-scale mapping operator module is as follows: The MSMO multi-scale mapping operator module includes a high-dimensional mapping module, a super-resolution attention feature capture module, and an image projection module; S2.2.

1. Fusion of intermediate features Input into the MSMO module to obtain the reconstructed high-resolution features ; S2.2.

2. Fusion of intermediate features Input into the MSMO module to obtain the reconstructed high-resolution features ; S2.2.

3. Fusion of intermediate features Input into the MSMO module to obtain the reconstructed high-resolution features ; S2.3, the specific process in the super-resolution feature fusion module is as follows: The three high-resolution features obtained by the multi-scale mapping operator module , and Input to the super-resolution feature fusion module to obtain the super-resolution fusion feature map ; S2.4, the specific process in the super-resolution image reconstruction module is as follows: The fusion features obtained by the super-resolution feature fusion module Input to the super-resolution image reconstruction module to obtain a super-resolution image ; S3, build a forest fire recognition module, and transform the super-resolution image Input into the ResNet-18 neural network model to obtain recognition features , and finally identify the features After processing by the fully connected layer, the recognition result of the forest fire image is obtained, and whether to issue an early warning is determined based on the recognition result.

2. The forest fire early warning method based on super-resolution neural operator according to claim 1 is characterized in that: S1 is as follows: Collect low-resolution forest fire images to be searched , , Representing images Height, Representing images The width of Representing images The number of channels is obtained by using a convolution kernel of size , the step length is of The convolutional layer is used to retrieve the low-resolution forest fire image. Perform initial feature extraction to obtain the initial feature map , the calculation formula is as follows: ， in, represents the convolution kernel weight, Represents the offset corresponding to the convolution kernel, is a two-dimensional convolution operation, , Represents the initial feature map Height, Represents the initial feature map The width of Represents the initial feature map The number of channels.

3. The forest fire early warning method based on super-resolution neural operator according to claim 2 is characterized in that: S2.1.1 is as follows: (1) Initial features After the cross-scale global branch of the first HAFM module, the cross-scale global branch includes a downsampling module, a sliding window mechanism, a convolution layer, a normalization layer, a mapping module, and a deconvolution layer; Initial feature map The low-resolution features are obtained by downsampling , the calculation formula is as follows: ， in, represents the downsampling operation, represents the downsampling scale factor, Indicates the assignment of the downsampling scale factor, setting is 3; Then the initial feature map is transformed into and low-resolution features Divide into fixed-size regional blocks and obtain corresponding regional features and , regional characteristics and The dimension is , Indicates the step size of the window movement, set is 2, the calculation process is as follows: ， ， in, Indicates the size of each region block, Represents the horizontal and vertical coordinates of the sliding window, Represents the initial feature map exist The regional characteristics of Represents low-resolution features exist Regional characteristics of the location; Then through the convolution kernel The convolutional layer of the regional features and Perform linear transformation to obtain the corresponding features and , and then calculate the similarity of cross-scale global features through the inner product. The calculation process is as follows: ， in, Indicates Initial feature map region block features and Low-resolution region block features The similarity of , Represents the initial feature map The total number of region blocks, , Low-resolution features The total number of region blocks; Then through Normalize the similarity and generate the cross-scale global attention weight. The calculation process is as follows: ， in, express Initial feature map region block features and Low-resolution region block features The cross-scale global attention weights; Combined with the cross-scale global attention weight, the regional features Remap to the initial feature map and aggregate all features to obtain cross-scale features , and the generated cross-scale features are transformed by deconvolution Restore to the initial feature size to obtain cross-scale global enhanced features , the calculation formula is as follows: ， ， in, Indicates Cross-scale features, represents the deconvolution operation; (2) Initial feature map Input to the local branch to obtain local features The local branch contains a layer of convolution kernel size The convolutional layer, The activation function and residual connection are calculated as follows: ， in, Represents the activation function Operations; (3) Local features and cross-scale global enhancement features Input into the fusion module in the first hierarchical attention fusion module to obtain the fused intermediate features , fusion is achieved through feature concatenation, and the convolution kernel size is The convolutional layer integrates the feature channels and finally uses The activation function enhances the model to learn complex image features. The calculation formula of this process is as follows: ， in, Represents a splicing operation, express Operation of activation function; (4) Fusion of intermediate features Generate high-resolution features through upsampling module , the upsampling process is achieved through strided convolution and deconvolution operations, and uses The activation function introduces nonlinear relationship calculation, and the calculation process is as follows: ， in, represents the weight in the strided convolution, represents the weight in deconvolution, represents the bias in the stride convolution, represents the bias in deconvolution, represents strided convolution, Represents the activation function operation.

4. The forest fire early warning method based on super-resolution neural operator according to claim 3 is characterized in that: S2.2 is as follows: S2.2.

1. Fusion of intermediate features The process through the multi-scale mapping operator module is as follows: Fusion of intermediate features First, it passes through the high-dimensional mapping module, which will fuse the intermediate features Mapped to a high-dimensional feature space to obtain the enhanced potential representation features ,in Represents the fusion of intermediate features The specific process is that the high-dimensional mapping module passes through the CNN encoder , the intermediate features Pixel value Promoted to high-dimensional feature space, the promoted features are obtained ,CNN encoder It consists of two convolutional layers with output channels of 128 and 256 and one The activation function is constructed, and then the improved features are Perform position encoding and weighted interpolation to obtain potential representation features , the calculation formula is as follows: ， ， in, , express The set of pixel coordinates in , represents the linear transformation function, The weighting coefficient is The weighted interpolation factor when , The weighting coefficient is The coordinate offset when represents the weighted coefficient in different neighborhoods, ; Then the super-resolution attention feature capture module uses a multi-layer attention mechanism to represent the latent features through a kernel integration operation. Capture global features and restore details to obtain high-level feature representation , Indicates The kernel integral output of the layer is The kernel integral of the layer, , the calculation formula in the kernel integral of each layer is as follows: First layer: ， No. layer: ， in, represents a feed-forward neural network, Indicates Layer kernel integration operation, Indicates Layer kernel integral output, Indicates Layer kernel integral output, after multiple iterations, output The kernel integral output of the layer; Then the image projection module will Kernel integral output of the layer Project back to RGB space to get high-resolution image features , this process will be projected Mapped to RGB space, the calculation formula is as follows: ， in, Represents a projection operation, which contains a convolution kernel size of The convolutional layer; S2.2.

2. Fusion of intermediate features Input into the MSMO module, the potential representation features are improved after the high-dimensional mapping module , express The set of pixel coordinates in , potentially representing features Then, through the super-resolution attention feature capture module, we get the high-level feature representation , high-level feature representation Finally, the reconstructed high-resolution features are obtained through the image projection module. ; S2.2.

3. Fusion of intermediate features Input into the MSMO module, the potential representation features are improved after the high-dimensional mapping module , express The set of pixel coordinates in , potentially representing features After the super-resolution attention feature capture module, high-level feature representation is obtained , high-level feature representation After the image projection module, the reconstructed high-resolution features are obtained .

5. The forest fire early warning method based on super-resolution neural operator according to claim 4 is characterized in that: S2.3 is as follows: The three high-resolution features obtained by the multi-scale mapping operator module , and Input to the super-resolution feature fusion module to obtain the super-resolution fusion feature map , the process uses Feature stitching combines high-resolution features , as well as Concatenate according to the channel dimension to get a new feature map , the calculation formula is as follows: 。 6. The forest fire early warning method based on super-resolution neural operator according to claim 5 is characterized in that: S2.4 is as follows: The fusion features obtained by the super-resolution feature fusion module Input to the super-resolution image reconstruction module to obtain a super-resolution image , this module uses the deconvolution layer to upsample the current feature map and uses the convolution kernel size The convolution layer adjusts the feature channel to integrate the fusion features Restore to original image size while using The activation function introduces nonlinear calculations in this module to enhance the module's ability to learn complex image features. The calculation formula is as follows: ， in, express The convolution kernel.

7. The forest fire early warning method based on super-resolution neural operator according to claim 6 is characterized in that: S3 is as follows: The super-resolution image obtained by the super-resolution image reconstruction module Input into the ResNet-18 neural network model to obtain the recognition features , through the fully connected layer , the identification features The feature vector is transformed into the final category prediction , and finally predict the label The recognition result of forest fire in the image is obtained through the Softmax activation function. The calculation formula is as follows: ， ， ， in, represents the scores of all categories, , Indicates The probability of the category, Indicates all categories, represents the probability of the first category, Represents a low-resolution forest fire image to be identified The probability of identifying it as a forest fire.

8. A forest fire early warning system based on a super-resolution neural operator, executing the forest fire early warning method based on a super-resolution neural operator as claimed in any one of claims 1 to 7, characterized in that: Includes the following modules: Image input and initial feature extraction module: extract features from the input low-resolution fire image and capture the initial features of the image; Hierarchical attention fusion module: input the initial features of the low-resolution image into the hierarchical attention model to obtain the global fusion features; Multi-scale mapping operator module: The global fusion features output by the hierarchical attention fusion module are input into the multi-scale mapping operator module to obtain enhanced high-resolution features; Super-resolution feature fusion module: The enhanced high-resolution features output by the multi-scale mapping operator module are input into the super-resolution feature fusion module to obtain super-resolution fusion features; Super-resolution image reconstruction module: input the super-resolution fusion features output by the super-resolution feature fusion module into the super-resolution image reconstruction module to obtain a super-resolution image; Forest fire recognition module: The super-resolution image generated by the super-resolution image reconstruction module is input into the forest fire recognition module to obtain the final recognition result; Forest fire warning module: receives the final recognition result obtained by the forest fire recognition module, and determines whether to issue a warning based on the recognition result.

Images