A method for fusing point and surface heterogeneous remote sensing data to retrieve the load of arbor combustible

Abstract

The present application relates to forest fire prevention monitoring technical field, disclose a kind of method for fusing point-surface heterogeneous remote sensing data inversion arbor combustible load, specific steps are: first, obtain multi-source remote sensing data and ground data, filter characteristic channel by correlation analysis;Image A is down-sampled by embedding layer, normalized mapping to D-dimensional vector space, then input recursive gate convolution module after patch merging, down-sampling and normalization, feature is calculated by three-stage cycle, decoder stage is up-sampled and the feature is spliced, while converging model using cross-entropy loss function;Image B is processed by the same basis, introduce SW-MSA module to enhance pixel interaction, finally fuse double-source prediction results.The present application can process multi-source heterogeneous remote sensing data, adapt to the characteristics of point-surface data fusion, and can realize comprehensive inversion of arbor combustible load of different spatial scales under complex terrain.

Description

Technical Field

[0001] This invention relates to the field of forest fire monitoring technology, and in particular to a method for inverting the combustible load of trees by fusing point and area heterogeneous remote sensing data. Background Technology

[0002] The impact of forest fires on forest ecosystems is complex and multifaceted. They can maintain the dynamic balance of forest ecosystems while simultaneously reshaping their structure and function. Mild forest fires can promote forest vegetation regeneration and nutrient cycling, increase species diversity, and maintain ecosystem stability. Severe forest fires, however, damage forest structure, reduce soil fertility, exacerbate greenhouse gas emissions, and even lead to reversed succession of forest ecosystems, making it impossible to restore them to their original healthy state.

[0003] Forest fuel load is one of the most important factors influencing the occurrence, spread, intensity, and extent of forest fires. In theoretical research, studying forest fuel load and spatial distribution not only helps in understanding the mechanisms of forest fire occurrence but also facilitates a comprehensive grasp of the propagation mechanisms of forest fire behavior. In practical application, understanding the spatial distribution of forest fuel load allows for the prediction of fire propagation paths, the scientific design of firebreaks and firebreaks, and the precise delineation of high- and low-risk fire zones, guiding daily patrols and cleanup. Therefore, studying forest fuel load and distribution is an indispensable task.

[0004] However, forest fuel load surveys and monitoring are costly and require significant investment. How to accurately and quickly obtain annual forest fuel load information has always been a pressing issue for national economic and social development. Utilizing remote sensing inversion methods to acquire forest fuel load information in complex environments over large areas is currently a hot research topic. One of the core issues of this method is to improve the universality of the inversion algorithm and address the uncertainties inherent in remote sensing inversion. Forest fuel load exhibits a strong nonlinear relationship in its spatial distribution; therefore, constructing an algorithm capable of handling the relationships between multi-source remote sensing data is crucial for forest fuel load inversion. Summary of the Invention

[0005] The purpose of this invention is to propose a method for inverting tree combustible load by fusing point-area heterogeneous remote sensing data. This method is capable of processing multi-source heterogeneous remote sensing data and is an algorithm that conforms to the characteristics of point-area remote sensing data fusion, so as to achieve comprehensive analysis of tree combustible load inverted by remote sensing at different spatial scales under complex terrain.

[0006] The technical implementation scheme of the present invention is as follows:

[0007] A method for retrieving tree combustible load by fusing heterogeneous point and area remote sensing data includes the following steps:

[0008] S1. Data Acquisition and Feature Selection: Acquire multi-source remote sensing data and ground data for inverting tree combustible load, and use correlation analysis to select feature channels.

[0009] S2, Image Normalization Mapping Processing: In step S1, Images A passes through the embedding layer, and after downsampling and image normalization processing within the embedding layer, the image Xi is mapped to a D-dimensional vector space;

[0010] S3. Patch merging process: The remote sensing image feature stitching, image normalization and linear layer parts in step S2 are processed by patch merging.

[0011] S4. Feature Preprocessing and Module Input: The features output from step S3 are downsampled and normalized, and then fed into the recursive gated convolution module.

[0012] S5, Multi-stage Feature Iterative Calculation: Repeat the three-stage calculation of the features output in step S4, with the Block iterating twice in total.

[0013] S6. Upsampling and Feature Concatenation: In the decoder stage, the features output from the fourth stage are upsampled and then concatenated with the results output from stage 3.

[0014] S7. Loss Calculation and Model Convergence: The cross-entropy loss function is used to estimate the difference between the model's predicted values ​​and the true values ​​in order to converge the model training.

[0015] S8. Dual-source feature enhancement and fusion: Images B adopts the same basic processing flow as Images A, while introducing the SW-MSA module to enhance the interaction of surface data pixels, and finally fuses the prediction results of the two.

[0016] Preferably, in step S1, the multi-source remote sensing data includes area remote sensing data and point data, and the entire original image... The images are divided into Images A and Images B. Images A includes both polygon and point data, while Images B only includes Landsat 9 remote sensing data from the polygon data. There is no spatial overlap between the two at the pixel level in the polygon layer.

[0017] Preferably, in step S2, firstly, after embedding the image into a high dimension, image normalization processing is performed on all pixels of the input image; secondly, the input features are divided into segments of size [size missing]. small image The operating range of the attention mechanism is determined, and the output features are fed into the window attention module using a trainable weight matrix. , and Generate the corresponding query matrix Q, key matrix K, and value matrix V; calculate window multi-head attention and perform residual connections; finally, pass through image normalization and multilayer perceptron and perform residual connections. The Block loops twice in total.

[0018] In step S3, a feature map is formed by selecting pixels at every other position in each batch of images, and the images are stitched together in the channel dimension. After layer normalization and linear transformation calculation, each batch of images after the patch merging layer is obtained. The rest of the Block repeats the above process, and the loop is repeated 8 times in total.

[0019] The image embedding module uses Formula 1 and Formula 2 for calculation:

[0020] Formula 1;

[0021] Formula 2;

[0022] In the formula, It is a normalization layer, whose function is to normalize the numerical values ​​so that their characteristic values ​​are within a reasonable range; It is the matrix number The average value of the dimension elements. It is the matrix number Standard deviation of dimension elements; It is a feedforward network layer. It is the first dimension of the input linear weight matrix. It is the second dimension of the input linear weight matrix. and It is the bias of the first and second dimensions;

[0023] The single-head window attention mechanism is calculated using formulas 3-5:

[0024] Formula 3;

[0025] In the formula, Q, K and These are query matrices ( ), key matrix ( ) and value matrix ( X is the high-dimensional embedding input matrix, where, , It is the length of the matrix. It is the matrix dimension. It is the matrix height; , and These are the training query matrix ( ), key matrix ( ) and value matrix ( The linear weight matrix;

[0026] Formula 4;

[0027] Formula 5;

[0028] The formula for calculating the multi-window attention mechanism is as follows:

[0029] Formula 6;

[0030] In the formula, It is a single-head attention mapping function for input and output matrices. It is a query matrix ( ) and key matrix ( ) dimensions It is a relative position offset. It is the element of the last dimension of the matrix. It is a matrix Multi-headed attention, Yes Single-headed attention The splicing, among which , The number of heads to watch out for. It is a linear weight matrix.

[0031] The normalized connectivity of the linear layer is calculated using Equations 7 and 8:

[0032] Formula 7;

[0033] Formula 8;

[0034] The multilayer perceptron is calculated using Equations 9-11:

[0035] Formula 9;

[0036] Formula 10;

[0037] Formula 11;

[0038] In the formula, It is a multilayer perceptron. It is a normalization layer. It is a matrix Multi-headed attention.

[0039] Preferably, in step S4, firstly, after entering the module, the channel C of the input feature is projected as 2C, and a two-dimensional convolution is performed on the projected x. The two-dimensional convolution is a Conv2d operation, where the order takes the value of 2-4, to obtain pws[i].

[0040] Secondly, split x into pwa in the dimension of dim=1, and split x into abc in the dimension of sum(dim); finally, perform depth convolution on abc and multiply by the scaling factor scale to obtain dw_abc; split dw_abc into dw_list in the dimension of dim=1, and calculate dw_list[i] according to the order (5 orders in total), that is, Order=0, get dw_list[0], Order=1, get dw_list[1], Order=2, get dw_list[2], Order=3, get dw_list[3], Order=4, get dw_list[4];

[0041] When Order=0, X0=pwa * dw_list[0]. Then, when Order>0, perform Hadamard product between dw_list[i] and pws[i] to obtain X1=pws[1](x) * dw_list[1], X2=pws[2](x) * dw_list[2], X3=pws[3](x) * dw_list[3], and X4=pws[4](x) * dw_list[4]. Then, perform different dimension interactions (performing Hadamard product pixel by pixel) according to the Order order to obtain X0-1= X0⊙X1, X0-2= X0-1⊙X2, X0-3= X0-2⊙X3, and X0-4= After each interaction (X0-3⊙X4), the channel is projected to 2C and downsampled with a 7*7 kernel. Finally, the output features are generated. The output features are then multiplied by the hyperparameter gamma1, and the result is randomly discarded to prevent overfitting. The result is then residually connected to the output features of the previous image normalization layer and fed into the image normalization layer and multilayer perceptron. The result is then multiplied by the hyperparameter gamma2, and the result is randomly discarded. The result is then residually connected to the output features of the previous image normalization layer. This entire calculation process is repeated 6 times.

[0042] In step S3, the calculation formulas for the deep feature extractor are formulas 12-20:

[0043] Formula 12;

[0044] Formula 13;

[0045] Formula 14;

[0046] In the formula, It is a shallow feature extractor. yes Activation function Yes The third convolution calculation, It is a linear weight matrix of input and output. It is a biased quantity. For input images, For convolution kernel, The input image contains pixel coordinates. The sliding coordinates on the convolution kernel The dot product of the convolution kernel and the input image;

[0047] Formula 15;

[0048] Formula 16;

[0049] Formula 17;

[0050] Formula 18;

[0051] Formula 19;

[0052] Formula 20;

[0053] In the formula, It is a deep feature extractor. It is a shallow feature extractor. These are features calculated by the Swintransformer module. It is a connection between the features calculated by the Swin transformer module and the features extracted by the shallow feature extractor. It is a triple convolution that extracts features from a shallow feature extractor. It is a connection of the features calculated by the Swin transformer module, the features extracted by the shallow feature extractor, and the three convolutions of the features extracted by the shallow feature extractor.

[0054] Preferably, the calculation method of the recursive gated convolution module is as follows:

[0055] Formula 21;

[0056] In the formula, It is the total number of pixels in the image. exist On each channel 3D recursively gated convolution features It is within the window On the first channel 3D target pixels Features It is in the window at the first Pixels on each channel The Features of adjacent pixels It is the first of the c channels within the window. 3D target pixels Feature convolution The weight, Represents pixels Traverse the window Adjacent pixels, Is The weights of the depthwise convolution f of adjacent features on each channel, It is the size of adjacent features. It is the window size centered on the i-th pixel.

[0057] Preferably, in step S6, the result of the first splicing is upsampled and then spliced ​​with the result of stage2; the result of the second splicing is upsampled and then spliced ​​with the result of stage1.

[0058] The formula for calculating upsampling is as follows:

[0059] Formula 22;

[0060] Preferably, in step S7, an optimizer is used to improve the training efficiency and stability of the deep learning model;

[0061] In step S7, the cross-entropy loss function is calculated using the following formula:

[0062] Formula 23;

[0063] In the formula, It is the cross-entropy loss function. It's about information content. It is a prediction probability. It is the true probability of the label;

[0064] The optimizer is used to perform calculations using formulas 24-29:

[0065] Formula 24;

[0066] Formula 25;

[0067] Formula 26;

[0068] In the formula, For the loss function in Axial gradient descent; It is an exponentially weighted average; for Squared exponent weighted average;

[0069] Formula 27;

[0070] Formula 28;

[0071] Formula 29;

[0072] In the formula, Exponentially weighted average Correction value; Squared exponent-weighted average Correction value; This is weight decay.

[0073] Preferably, in step S8, the prediction results of Images A and Images B are fused into a single image;

[0074] The formula for calculating Images B is as follows:

[0075] Formula 30;

[0076] In the formula, It is a matrix Select window attention, It is the pixel key matrix within the window. It represents the dimensions of the query matrix and key matrix within the window. It is a matrix of known pixel values ​​of neighboring pixels within the window. It is a relative position offset.

[0077] Preferably, the English abbreviations and their corresponding meanings for the modules and operations mentioned throughout the text are as follows:

[0078] The embedding layer is a Patch Embedding layer;

[0079] The downsampling operation is referred to as the Down Sampling operation.

[0080] The image is normalized using a Layer Norm operation;

[0081] The patch merging process is a Patch Merging operation.

[0082] The splicing is a Contact operation;

[0083] The recursive gated convolution module is the gnconv module;

[0084] The upsampling is an Up sample operation;

[0085] The cross-entropy loss function is the Loss Function;

[0086] The window partitioning module is the Window Partition module;

[0087] The window attention module is the SW-MSA module;

[0088] The multilayer sensor is an MLP module;

[0089] The optimizer is an AMAM optimizer;

[0090] The depthwise convolution is a dwconv operation.

[0091] The present invention has the following advantages:

[0092] 1. This invention has low requirements for data distribution assumptions, can handle multi-source heterogeneous remote sensing data, and has strong generalization ability for complex data through attention mechanisms and recursive gated convolution mechanisms, enhancing the ability to extract global and local features. Through improved attention and gating mechanisms, it enhances the modeling accuracy of large-scale combustible load characteristics in multi-source heterogeneous remote sensing images.

[0093] 2. Based on the characteristics of the combustible load of trees under complex terrain conditions, this invention dynamically adjusts the window scale to capture both global and local features simultaneously. Multi-scale feature interaction modules are introduced into each layer of the model to improve the effective extraction capability of information at different spatial scales.

[0094] 3. This invention can better integrate point and area remote sensing data to more effectively capture the spatial characteristics of tree combustible load in remote sensing data, thereby improving the accuracy of tree combustible load inversion. Attached Figure Description

[0095] Figure 1 This is a general framework diagram of the present invention.

[0096] Figure 2 This is a flowchart of the patch merging module calculation process of the present invention.

[0097] Figure 3 This is a flowchart of the calculation process for the recursive gated convolution module of this invention.

[0098] Figure 4 This is a schematic diagram of the offset window segmentation of the Swin transformer according to the present invention.

[0099] Figure 4 (a) is a schematic diagram of the window segmentation of the present invention.

[0100] Figure 4 (b) is a schematic diagram of the offset window segmentation of the present invention.

[0101] Figure 4 (c) is a schematic diagram of the window information mask, inversion and interaction of the present invention.

[0102] Figure 5 The result of calculating image A using the Swin hornet model is shown in the embodiment of this invention.

[0103] Figure 6 This is the result of calculating image A using the hornet model of this invention.

[0104] Figure 7 This is the result of calculating image A using the Swin transformer model of this invention.

[0105] Figure 8 The result diagram of image A calculated by the three models of this invention.

[0106] Figure 9 In an embodiment of the present invention, the training and validation loss maps of image A are calculated using Swing hornet.

[0107] Figure 10 This invention utilizes Hornet to calculate the training and validation loss map of image A as an implementation example.

[0108] Figure 11 This invention utilizes the Swing transformer to calculate the training and validation loss map of image A as an implementation example.

[0109] Figure 12 This is the result of calculating image B using the Swin hornet model of this invention.

[0110] Figure 13 This is the result of calculating image B using the hornet model of this invention.

[0111] Figure 14 This is the result of calculating image B using the Swing transformer model of this invention.

[0112] Figure 15The image B is the result of the calculation of the three models of this invention.

[0113] Figure 16 This invention utilizes Swing hornet to calculate the training and validation loss map of image B as an implementation example.

[0114] Figure 17 This invention utilizes Hornet to calculate the training and validation loss map of image B as an implementation example.

[0115] Figure 18 This invention utilizes a Swing transformer to calculate the training and validation loss map of image B as an implementation example. Detailed Implementation

[0116] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It is hereby declared that the directional terms such as up, down, left, right, front, back, inside, and outside used in this text are based solely on the accompanying drawings and are not intended to specifically limit the invention. Example 1:

[0117] A method for retrieving tree combustible load by fusing heterogeneous point and area remote sensing data includes the following steps:

[0118] S1. Data Acquisition and Feature Selection: Acquire multi-source remote sensing data and ground data for inverting tree combustible load, and use correlation analysis to select feature channels.

[0119] S2, Image Normalization Mapping Processing: In step S1, Images A passes through the embedding layer, and after downsampling and image normalization processing within the embedding layer, the image Xi is mapped to a D-dimensional vector space;

[0120] S3. Patch merging process: The remote sensing image feature stitching, image normalization and linear layer parts in step S2 are processed by patch merging.

[0121] S4. Feature Preprocessing and Module Input: The features output from step S3 are downsampled and normalized, and then fed into the recursive gated convolution module.

[0122] S5, Multi-stage Feature Iterative Calculation: Repeat the three-stage calculation of the features output in step S4, with the Block iterating twice in total.

[0123] S6. Upsampling and Feature Concatenation: In the decoder stage, the features output from the fourth stage are upsampled and then concatenated with the results output from stage 3.

[0124] S7. Loss Calculation and Model Convergence: The cross-entropy loss function is used to estimate the difference between the model's predicted values ​​and the true values ​​in order to converge the model training.

[0125] S8. Dual-source feature enhancement and fusion: Images B adopts the same basic processing flow as Images A, while introducing the SW-MSA module to enhance the interaction of surface data pixels, and finally fuses the prediction results of the two.

[0126] It should be noted that in step S1 of the technical solution, during the data acquisition process, the data is selected from 7 counties (cities) in 5 prefectures and cities located in the central Yunnan region (between 100°35′-104°32′ east longitude and 24°21′-24°45′ north latitude): Kunming City, Yuxi City, Qujing City, Chuxiong Yi Autonomous Prefecture and Mengzi City, Gejiu City, Mile City, Kaiyuan City, Jianshui County, Shiping County and Luxi County in the northern part of Honghe Hani and Yi Autonomous Prefecture.

[0127] I downloaded Landsat 9 Collection 2 Level-1 and Level-2 multispectral data for the central Yunnan region in December 2021 from https: / / earthexplorer.usgs.gov / ; I preprocessed the data in ENVI software; and I downloaded GEDI L2A and L4A products for the central Yunnan region in December 2021 from https: / / search.earthdata.nasa.gov / search and preprocessed the data using the Python programming language.

[0128] 2049 standard plots of tree combustibles and 1118 large sample plots of tree combustibles were established under different forest stand types, with a total area of ​​0.067 hm². 2 (25.82×25.82m), the area of ​​the large sample plot of tree combustibles is 25hm. 2 (500×500m); The survey content of standard plots and large sample plots includes: administrative region, land type, latitude and longitude, vegetation type, origin, stand density, altitude, slope, aspect, slope position, health status and tree species composition, etc.; The survey content of forest trees includes: tree species, diameter at breast height, tree height, age and growth status, etc. Sampling is carried out for different diameter classes of each dominant tree species, with 3 trees sampled for each dominant tree species. After felling, the combustible load of each organ of the whole tree is measured.

[0129] Pearson's test was used to analyze the correlation between extracted feature variables and tree combustible load, and feature channels with high correlation were selected, as shown in the table below:

[0130]

[0131] Note: B2 is the visible blue light band, B3 is the visible green light band, B4 is the visible red light band, and B8 is the near-infrared band.

[0132] Based on the highly correlated feature channels obtained in step S1, the entire original image is mosaicked and fused with raster data with a pixel value of 0 to form... ,length The width is 28674. The channel number is 28674. The value is 14, so ImagesA and ImagesB are each divided into 448*448 small image blocks. For a given channel, select 743 small image blocks where the total number of pixels is not zero. and analyzed 743 small image blocks respectively. By performing random flips 30 times, a single channel can generate 29,720 small image blocks. .

[0133] Example 2:

[0134] In step S2, after high-dimensional image embedding, image normalization is performed on all pixels of the input image; next, the input features are divided into window segmentation modules with a size of [missing information]. small image The operating range of the attention mechanism is determined, and the output features are fed into the window attention module using a trainable weight matrix. , and Generate the corresponding query matrix (Q), key matrix (K), and value matrix (V), calculate window multi-head attention, and perform residual connections; finally, pass through image normalization and a multilayer perceptron, including 2 linear layers and 1 activation layer, and perform residual connections. The Block loops twice in total.

[0135] The above process is calculated using formulas 1-11:

[0136] Formula 1;

[0137] Formula 2;

[0138] In the formula, It is a normalization layer, whose function is to normalize the numerical values ​​so that their characteristic values ​​are within a reasonable range; It is the matrix number The average value of the dimension elements. It is the matrix number Standard deviation of dimension elements. It is a feedforward network layer. It is the first dimension of the input linear weight matrix. It is the second dimension of the input linear weight matrix. and It is the bias of the first and second dimensions.

[0139] Formula 3;

[0140] In the formula, Q, K and These are query matrices ( ), key matrix ( ) and value matrix ( X is the high-dimensional embedding input matrix, where, , It is the length of the matrix. It is the matrix dimension. It is the matrix height; , and These are the training query matrix ( ), key matrix ( ) and value matrix ( The linear weight matrix.

[0141] Formula 4;

[0142] Formula 5;

[0143] Formula 6;

[0144] In the formula, It is a single-head attention mapping function for input and output matrices. It is a query matrix ( ) and key matrix ( ) dimensions It is a relative position offset. It is the element of the last dimension of the matrix. It is a matrix Multi-headed attention, Yes Single-headed attention The splicing, among which , The number of heads to watch out for. It is a linear weight matrix.

[0145] Formula 7;

[0146] Formula 8;

[0147] Formula 9;

[0148] Formula 10;

[0149] Formula 11;

[0150] In the formula, It is a multilayer perceptron. It is a normalization layer. It is a matrix Multi-headed attention.

[0151] It should be noted that in step S1, the image data with a batch size of 29720 first passes through the embedding layer. This layer reduces the spatial resolution of each image from 448x448 to 224x224 through downsampling and maps the input channels to a 96-dimensional vector space. Finally, it is processed by the normalization layer to output features with a size of [29720, 96, 224, 224]. Subsequently, these features enter the core module of the Swin Transformer.

[0152] Example 3:

[0153] In step S3, the remote sensing image features output from step S2 undergo patch merging, which includes three parts: stitching, image normalization, and a linear layer. By selecting pixels at every other position in each batch of images to form a feature map, stitching is performed along the channel dimension. After layer normalization and linear transformation calculations, each batch of images is obtained after the patch merging layer. The size of each batch of images is halved, and the number of channels is doubled, as shown in the attached figure. Figure 2 As shown, the remaining identical steps repeat the first stage process, and the Block loops a total of 8 times.

[0154] It should be further explained that the remote sensing image features output from the previous stage are processed by Advancedswin Transformer, such as patch merging, and the output size is [29720, 192, 112, 112].

[0155] Example 4:

[0156] In step S4, the features output from step S3 are downsampled and image normalized, and then fed into the recursive gated convolution module, as shown in the attached diagram. Figure 3 As shown, after entering this module, the input feature channel C is first projected to 2C, and a two-dimensional convolution is performed on the projected x. The two-dimensional convolution is a Conv2d operation, where the order value is 2-4, to obtain pws[i]. Then, x is split into pwa in the dimension of dim=1, and x is split into abc in the dimension of sum(dim).

[0157] Perform a depthwise convolution dwconv(abc) on abc and multiply by the scaling factor scale to obtain dw_abc. Split dw_abc into dw_list in the dimension with dim=1. Calculate dw_list[i] according to the order (5 orders in total). That is, Order=0, get dw_list[0], Order=1, get dw_list[1], Order=2, get dw_list[2], Order=3, get dw_list[3], Order=4, get dw_list[4].

[0158] When Order=0, X0=pwa * dw_list[0], and then Order>0, dw_list[i] and pws[i] are multiplied by Hadamard to obtain X1=pws[1](x) * dw_list[1], X2=pws[2](x) * dw_list[2], X3=pws[3](x) * dw_list[3], X4=pws[4](x) * dw_list[4]. Then, different dimensions of interaction are performed in the order of Order (Hadamard product is performed pixel by pixel) to obtain X0-1= X0⊙X1, X0-2= X0-1⊙X2, X0-3= X0-2⊙X3, X0-4= X0-3⊙X4. After each interaction, the channel is projected to 2C and downsampled with a convolution kernel of 7*7. Finally, the features are output.

[0159] The output features are multiplied by the hyperparameter gamma1 using a Hadamard product, and the result is randomly discarded to prevent overfitting. Then, a residual connection is made with the output features of the image normalization layer in the previous stage, and the result is fed into the image normalization layer and multilayer perceptron. The result is then multiplied by the hyperparameter gamma2 using a Hadamard product, and the result is randomly discarded. The result is then made a residual connection with the output features of the image normalization layer in the previous stage. This entire calculation process is repeated 6 times.

[0160] The above process is calculated using formulas 12-21:

[0161] Formula 12;

[0162] Formula 13;

[0163] Formula 14;

[0164] In the formula, It is a shallow feature extractor. yes Activation function Yes The third convolution calculation, It is a linear weight matrix of input and output. It is a biased quantity. For input images, For convolution kernel, The input image contains pixel coordinates. The sliding coordinates on the convolution kernel It is the dot product of the convolution kernel and the input image.

[0165] Formula 15;

[0166] Formula 16;

[0167] Formula 17;

[0168] Formula 18;

[0169] Formula 19;

[0170] Formula 20;

[0171] In the formula, It is a deep feature extractor. It is a shallow feature extractor. These are features calculated by the Swintransformer module. It is a connection between the features calculated by the Swin transformer module and the features extracted by the shallow feature extractor. It is a triple convolution that extracts features from a shallow feature extractor. It is a connection of the features calculated by the Swin transformer module, the features extracted by the shallow feature extractor, and the three convolutions of the features extracted by the shallow feature extractor.

[0172] Formula 21;

[0173] In the formula, It is the total number of pixels in the image. exist On each channel 3D recursively gated convolution features It is within the window On the first channel 3D target pixels Features It is in the window at the first Pixels on each channel The Features of adjacent pixels It is the first of the c channels within the window. 3D target pixels Feature convolution The weight, Represents pixels Traverse the window Adjacent pixels, Is The weights of the depthwise convolution f of adjacent features on each channel, It is the size of adjacent features. It is the window size centered on the i-th pixel.

[0174] It should be further explained that in step S4, the features output from step S3 are downsampled and image normalized, and then fed into the recursive gated convolution module, as shown in the attached diagram. Figure 3 As shown, the output size is [29720, 384, 56, 56].

[0175] Example 5:

[0176] In step S5, the features output from step S4 are calculated in three stages, and the Block is looped twice in total.

[0177] In the decoder stage (S6), the features output from step S5 are upsampled and then concatenated with the output from stage 3; the first concatenation output is upsampled and then concatenated with the output from stage 2; the second concatenation output is upsampled and then concatenated with the output from stage 1. The above process is calculated using formula 22.

[0178] Formula 22;

[0179] In the formula, The coordinates on the image are Transposed convolution result, It is a convolution kernel. The coordinates are With a convolution kernel size of The convolved image.

[0180] It should be noted that in step S5, the features output in step S1 are calculated in three stages, and the output size is [29720, 768, 28, 28].

[0181] It should be further explained that: in the decoder stage, the features output from the fourth stage are upsampled and then concatenated with the output of stage 3, with an output size of [29720, 384, 56, 56]; the output of the first concatenation is upsampled and then concatenated with the output of stage 2, with an output size of [29720, 192, 112, 112]; the output of the second concatenation is upsampled and then concatenated with the output of stage 1, with an output size of [29720, 96, 224, 224].

[0182] Example 6:

[0183] In step S7, the cross-entropy loss function is used to estimate the difference between the model's predicted values ​​and the true values ​​to converge model training. An optimizer is then employed to improve the training efficiency and stability of the deep learning model. The above process is calculated using formulas (23-29).

[0184] Formula 23;

[0185] In the formula, It is the cross-entropy loss function. It's about information content. It is a prediction probability. It represents the true probability of the label.

[0186] Formula 24;

[0187] Formula 25;

[0188] Formula 26;

[0189] In the formula, For the loss function in Axial gradient descent; It is an exponentially weighted average; for Squared exponent-weighted average.

[0190] Formula 27;

[0191] Formula 28;

[0192] Formula 29;

[0193] In the formula, Exponentially weighted average Correction value; Squared exponent-weighted average Correction value; This is weight decay.

[0194] It should be noted that the cross-entropy loss function is used to estimate the difference between the model's predicted values ​​and the true values ​​to converge model training, and an optimizer is used to improve the training efficiency and stability of the deep learning model. The Num_epochs of Swin hornet A, hornet A, and Swin transformer A are 100, 80, and 90, respectively.

[0195] Example 7:

[0196] In step S8, Images B is processed in the same way as Images A. This module introduces SW-MSA to enhance the interaction between other pixels and neighboring pixels in the pixelless data (GEDI data) (e.g., ...). Figure 4 As shown), to improve the accuracy of remote sensing inversion of tree combustible load, the above process is calculated using formula (30). Finally, the prediction results of Images A and Images B are merged into one image.

[0197] Formula 30;

[0198] In the formula, It is a matrix Select window attention, It is the pixel key matrix within the window. It represents the dimensions of the query matrix and key matrix within the window. It is a matrix of known pixel values ​​of neighboring pixels within the window. It is a relative position offset.

[0199] It should be noted that Images B is processed in the same way as Images A, and the output size is the same at each stage. The Num_epoch of Swinhornet B, Hornnet B, and Swin transformer B is all 80. Finally, the prediction results from Images A and Images B are merged into a single image.

[0200] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for retrieving the fuel load of arborous combustible by fusing point and area heterogeneous remote sensing data, characterized in that, The method comprises the following steps: S1, data acquisition and feature screening: acquiring multi-source remote sensing data for inverting the arbor combustible load and ground data for inverting the arbor combustible load, and screening feature channels by correlation analysis; S2, image normalization mapping processing: in step S1, Images A passes through an embedding layer, and the image normalization processing in the embedding layer is performed on the image Xi to map the image Xi to a D-dimensional vector space; S3, patch merging processing: the remote sensing image features in step S2 are spliced, image normalized, and subjected to patch merging processing in the linear layer part; S4, feature preprocessing and module input: the output features in step S3 are subjected to downsampling and image normalization processing, and then enter the recursive gate convolution module; S5, multi-stage feature cycle calculation: the output features in step S4 are calculated repeatedly for three stages, and Block is cycled for 2 times; S6, upsampling and feature splicing: in the decoder stage, the features output in the fourth stage are upsampled, and then spliced with the results output in stage 3; S7, loss calculation and model convergence: the cross-entropy loss function is used to estimate the difference between the model prediction value and the true value to converge the model training; S8, dual-source feature enhancement and fusion: Images B is subjected to the same basic processing procedure as Images A, and the SW-MSA module is introduced to enhance the pixel interaction of the surface data, and finally the prediction results of the two are fused; The multi-source remote sensing data in the S1 step includes surface remote sensing data and point data, and the whole original image is divided into Images A and Images B, wherein Images A includes the surface data and the point data, Images B only includes the surface data, and there is no spatial overlap between the two at the surface layer pixel level. The multi-source remote sensing data in the S1 step includes surface remote sensing data and point data, and the whole original image is divided into Images A and Images B, wherein Images A includes the surface data and the point data, Images B only includes the surface data, and there is no spatial overlap between the two at the surface layer pixel level. In the S2 step, first, after the image is embedded in a high dimension, the image normalization processing is performed on all pixels of the input image; Secondly, the input features are divided into small image blocks with a size of by the window segmentation module , the running range of the attention mechanism is determined, and the output features enter the window attention module to generate the corresponding query matrix Q, key matrix K and value matrix V using the trainable weight matrices , and , calculate the window multi-head attention, and make residual connection; Finally, then the image normalization and multilayer perceptron are performed, and residual connection is made, and Block is cycled for 2 times.

2. The method for retrieving the fuel load of the arborous combustible according to claim 1, characterized in that, In the S3 step, by selecting the pixels at every other position in each batch of images to constitute a feature map, the feature maps are spliced in the channel dimension, the image layer normalization and linear transformation calculation are performed, and each batch of images after the patch merging layer is obtained, and the remaining part of Block repeats the above procedure and is cycled for 8 times; In the S4 step, first, after entering the module, the channel C of the input feature is projected as 2C, and two-dimensional convolution is performed on the projected x, the two-dimensional convolution is a Conv2d operation, the order value is 2-4, and pws[i] is obtained; Secondly, x is split into pwa in the dim=1 dimension, and x is split into abc in the sum(dim) dimension; finally, deep convolution is performed on abc, and dw_abc is obtained by multiplying a scaling factor scale, and the scaling factor scale is 0.1-0.5; In the S3 step, the calculation formula of the deep feature extractor is formula 12-formula 20: Formula 12; Formula 13; Formula 14; In the formula, It is a shallow feature extractor. yes Activation function Yes The third convolution calculation, It is a linear weight matrix of input and output. It is a biased quantity. For input images, For convolution kernel, The input image contains pixel coordinates. The sliding coordinates on the convolution kernel The dot product of the convolution kernel and the input image; Formula 15; Formula 16; Formula 17; Formula 18; Formula 19; Formula 20; In the formula, is a deep feature extractor, is a shallow feature extractor, is a feature calculated after a Swintransformer module, is a connection of the feature calculated after the Swin transformer module and the feature extracted by the shallow feature extractor, is a third convolution of the feature extracted by the shallow feature extractor, is a connection of the feature calculated after the Swin transformer module, the feature extracted by the shallow feature extractor and the third convolution of the feature extracted by the shallow feature extractor.

3. The method for retrieving the fuel load of the arborous combustible material from the heterogeneous remote sensing data of the fusion point surface according to claim 1, characterized in that, The calculation method of the recursive gate convolution module is: Formula 21; wherein, is the pixel of the whole image In channels dimensional recurrent gated convolution feature, is the feature of the target pixel in the window channel dimensional target pixel in the window feature, is the feature of the i-th adjacent pixel of the pixel in the window channel i-th adjacent pixel of the pixel in the window feature, is the weight of the feature convolution of the target pixel in the window channel target pixel in the window feature, denotes the pixel traverses the i-th adjacent pixel in the window feature, is the weight of the depth convolution f of the adjacent features in channel adjacent feature size, is the window size centered on the i-th pixel.

4. The method for fusing point and area heterogeneous remote sensing data inversion of tree fuel load according to claim 1, characterized in that, In the S6 step, the result output by the first splicing is upsampled, and then spliced with the result output by stage 2; The result output by the second splicing is upsampled, and then spliced with the result output by stage 1.

5. The method for fusing point and area heterogeneous remote sensing data inversion of tree fuel load according to claim 1, characterized in that, In the S7 step, an optimizer is used to improve the training efficiency and stability of the deep learning model; In the S7 step, the cross-entropy loss function calculation formula is: Formula 23; wherein is a cross-entropy loss function, is an information quantity, is a predicted probability, is a label true probability; The optimizer AMAM is calculated by using the formulas 24-29: Formula 24; Formula 25; Formula 26; wherein is a loss function at axial gradient descent; is an exponentially weighted average; is is a squared exponentially weighted average; Formula 27; Formula 28; Formula 29; wherein is a modified value of the exponentially weighted average is a modified value of the exponentially weighted average is a modified value of the exponentially weighted average is a modified value of the exponentially weighted average is a weight decay.

6. The method of claim 1, wherein the method further comprises: In the S8 step, the prediction results of Images A and Images B are fused into one image; The calculation formula of Images B is: Formula 30; In the formula, It is a matrix Select window attention, It is the pixel key matrix within the window. It represents the dimensions of the query matrix and key matrix within the window. It is a matrix of known pixel values ​​of neighboring pixels within the window. It is a relative position offset.

Images