Satellite remote sensing combustible inversion method and related device

Abstract

The invention belongs to a combustible inversion method, provides a satellite remote sensing combustible inversion method and a related device, and aims to solve the technical problems that a current method for carrying out forest combustible load inversion through satellite remote sensing data cannot comprehensively reflect complex forest combustible characteristics and is high in data processing and calculation cost. The method comprises the following steps: respectively extracting corresponding modal features from an optical satellite remote sensing image and a synthetic aperture radar satellite remote sensing image of a to-be-measured region, obtaining ground feature representation in each modal by means of a convolutional neural network and a regional convolutional neural network, and then obtaining comprehensive graph feature representation through Transform; and classifying the comprehensive feature representation through a deep neural network to obtain a thematic map of the combustible loading capacity. According to the method, data from different modes are fused, and more comprehensive and accurate information than a single mode is provided, so that the prediction accuracy is improved.

Description

Technical Field

[0001] The present application relates to a combustible material inversion method, and specifically to a satellite remote sensing combustible material inversion method and related devices. Background Art

[0002] Forest fuel load refers to the total amount of easily combustible plant materials in forest ecosystems, such as dead branches and leaves, shrubs, and herbaceous plants. It is an important indicator for measuring the potential danger of forest fires and the speed of fire spread. Accurately estimating forest fuel load is of vital importance for the prevention, control, and rational allocation of fire-fighting resources.

[0003] With the development of remote sensing technology, the inversion of forest fuel load using satellite remote sensing data has become an effective method. Satellite remote sensing technology can provide large-scale and periodic surface cover information, and different types of remote sensing sensors (such as optical sensors, radar sensors, etc.) can capture surface parameters with different characteristics. However, a single type of remote sensing data is often difficult to fully reflect the complex characteristics of forest fuels, which is limited by the characteristics of the data itself and the influence of external environmental factors. In addition, the existing data fusion methods and inversion models often sacrifice real-time performance while pursuing high precision, and the data processing and calculation costs are high. Summary of the invention

[0004] The present application aims to provide a satellite remote sensing combustible inversion method and related devices to address the technical problems that the current method of inverting forest fuel load through satellite remote sensing data cannot fully reflect the complex forest fuel characteristics and has high data processing and calculation costs.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions: In the first aspect, the present application proposes a satellite remote sensing combustible inversion method, comprising: Obtain optical satellite remote sensing images and synthetic aperture radar satellite remote sensing images of the area to be measured; Extracting the first modal features from the optical satellite remote sensing images and extracting the second modal features from the synthetic aperture radar satellite remote sensing images; Inputting the first modal feature and the second modal feature into a convolutional neural network and a regional convolutional neural network to obtain a representation of the features of the ground objects in each modality; The feature representation of each modality is passed through the encoder part of the Transformer and the self-attention mechanism of the Transformer in turn to obtain the interactive feature encoding vector; The interactive feature encoding vector is passed through the self-attention layer of the Transformer to obtain a comprehensive feature representation; The comprehensive feature representation is classified through a deep neural network to obtain the fuel load thematic map.

[0006] Furthermore, the method of inputting the first modal feature and the second modal feature into a convolutional neural network and a regional convolutional neural network to obtain a representation of the ground feature under each modality includes: The optical satellite remote sensing image is passed through three convolutional layers in sequence, and each convolutional layer is connected to a maximum pooling layer to extract the first modal features; The synthetic aperture radar satellite remote sensing image is passed through two convolutional layers in sequence, and each convolutional layer is connected to a maximum pooling layer to extract the second modal features.

[0007] Furthermore, the convolution kernel size of the convolution layer used in the first modality feature extraction is 3×3, and the size of the maximum pooling layer is 2×2; The size of the maximum pooling layer used in the second modality feature extraction is 2×2.

[0008] Furthermore, the encoder part of the Transformer adopts layer normalization processing; the self-attention mechanism of the Transformer adopts a multi-head attention mechanism; The output of the multi-head attention mechanism is superimposed with the feature representation of the object in each modality and input into the self-attention layer of the Transformer.

[0009] Furthermore, the self-attention layer of the Transformer includes a connected layer normalization process and a multi-layer perceptron; The output of the multilayer perceptron is superimposed with the input of the self-attention layer of the Transformer for input into a deep neural network.

[0010] Furthermore, the classifying the comprehensive feature representation by a deep neural network includes: The combustible load thematic map is obtained by passing through the multi-scale fusion layer, feature processing layer and output layer in sequence.

[0011] Furthermore, the multi-scale fusion layer adopts a 1×1 convolution layer; the feature processing layer includes two fully connected layers; and the output layer adopts a Softmax layer.

[0012] In the second aspect, the present application proposes a satellite remote sensing combustible inversion system, comprising: An image acquisition module is used to acquire optical satellite remote sensing images and synthetic aperture radar satellite remote sensing images of the area to be measured; A feature extraction module, used for extracting first modal features from optical satellite remote sensing images and extracting second modal features from synthetic aperture radar satellite remote sensing images; A feature representation module, used for inputting the first modal feature and the second modal feature into a convolutional neural network and a regional convolutional neural network to obtain a feature representation of the ground object under each modality; The interactive module is used to make the feature representation of each modality pass through the encoder part of the Transformer and the self-attention mechanism of the Transformer in turn to obtain the interactive feature encoding vector; The comprehensive feature module is used to pass the interactive feature encoding vector through the self-attention layer of the Transformer to obtain a comprehensive feature representation; The inversion module is used to classify the comprehensive feature representation through a deep neural network to obtain the fuel load thematic map.

[0013] In a third aspect, the present application proposes an electronic device, comprising: a memory, and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the above-mentioned satellite remote sensing combustible inversion method.

[0014] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned satellite remote sensing combustible material inversion method are implemented.

[0015] Compared with the prior art, this application has the following beneficial effects: This application proposes a satellite remote sensing combustible inversion method, which extracts corresponding modal features from optical satellite remote sensing images and synthetic aperture radar satellite remote sensing images of the area to be measured, respectively, obtains the feature representation of the ground objects under each mode with the help of convolutional neural networks and regional convolutional neural networks, and then obtains the comprehensive map feature representation through Transformer, and then classifies the comprehensive feature representation through deep neural networks to obtain the combustible load thematic map. This application integrates data from different modalities, providing more comprehensive and accurate information than a single modality, thereby improving the accuracy of the prediction. With the help of Transformer and deep neural networks, information at different scales can be captured, which improves the robustness of the entire model and its adaptability to complex scenes, helps to reduce the risk of overfitting, improves the generalization ability of the model, and has the potential for lightweight model design, which is easy to run on mobile devices or edge devices, or be deployed on cloud services to achieve real-time data processing and response. This application can be widely used in environmental monitoring, disaster warning, resource management and other fields, and provides powerful remote sensing data analysis technical support for research and applications related to combustible inversion and fire testing.

[0016] The present application also proposes a satellite remote sensing combustible inversion system, an electronic device and a computer storage medium, which possess all the advantages of the above-mentioned satellite remote sensing combustible inversion method. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 This is a first schematic diagram of the satellite remote sensing combustible inversion method of this application; Figure 2 This is a second schematic diagram of the satellite remote sensing combustible material inversion method of this application; Figure 3 This is a schematic diagram of the MDLF model in the embodiment of the present application; Figure 4 A schematic diagram of the satellite remote sensing combustible inversion system of this application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0021] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0022] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0023] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0024] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0025] Forest fuel load refers to the total amount of easily combustible plant materials (such as dead branches and leaves, shrubs, and herbaceous plants) in the forest ecosystem. It is an important indicator for measuring the potential danger of forest fires and the speed of fire spread. Therefore, accurate estimation of forest fuel load is of vital importance for the prevention, control, and rational allocation of fire-fighting resources of forest fires.

[0026] Traditional forest fuel load assessment methods mainly rely on ground surveys and sampling analysis, which can provide relatively accurate data, but have many limitations, such as limited coverage, high cost, long time consumption, and difficulty in achieving large-scale and real-time monitoring. With the development of remote sensing technology, the inversion of forest fuel load using satellite remote sensing data has become an effective alternative. Satellite remote sensing technology can provide large-scale and periodic surface cover information, and different types of remote sensing sensors (such as optical sensors, radar sensors, etc.) can capture surface parameters with different characteristics. Optical remote sensing data can provide information such as vegetation index and leaf area index, while radar remote sensing data can penetrate clouds and some vegetation to provide information about surface roughness and vegetation structure.

[0027] However, a single type of remote sensing data is often difficult to fully reflect the complex characteristics of forest fuels, which is limited by the characteristics of the data itself and the influence of external environmental factors. Optical data are easily affected by clouds, atmospheric particulate matter, etc., while radar data are sensitive to vegetation type and density changes. Although traditional data fusion methods such as principal component analysis and least squares method have been tried, they are limited in effect when processing high-dimensional and nonlinear remote sensing data, and it is difficult to fully mine the useful information in the data. In addition, the existing inversion model has insufficient generalization ability and depends on specific data sets and regions. The selection and optimization of model parameters often rely on experience and lack automation and intelligence. At the same time, while pursuing high accuracy, the existing methods often sacrifice real-time performance, and the data processing and computing costs are high, which limits their popularity in practical applications. The initial application of deep learning technology provides new ideas for remote sensing data fusion, but it still needs to solve the problems of real-time and accuracy balance and computing cost. Therefore, the existing fuel load inversion technology has deficiencies in data source, fusion technology, model generalization, parameter optimization, real-time performance and computing cost, and urgently needs further research and improvement.

[0028] Based on the above situation, the present application proposes a satellite remote sensing combustible material inversion method and related devices, and the present application is described in detail below in conjunction with embodiments and drawings.

[0029] like Figure 1 As shown, it is a first schematic diagram of the satellite remote sensing combustible inversion method of the present application, which may include: S101, obtaining an optical satellite remote sensing image and a synthetic aperture radar satellite remote sensing image of a region to be measured.

[0030] It should be noted that optical satellite remote sensing images mainly capture the spectral information of the surface, and distinguish the types of land objects, such as vegetation, water bodies, buildings, etc., through the reflection of light in different bands. These images usually have rich color and texture information, which helps to identify the types of combustibles on the surface and their distribution. Synthetic Aperture Radar (SAR) satellite remote sensing images use radar beams to actively transmit and receive signals reflected from the surface, which can penetrate obstacles such as clouds and smoke and provide all-weather and all-day surface observation data. Synthetic aperture radar images are sensitive to surface roughness, humidity and other characteristics, which helps to identify the humidity state and structural characteristics of combustibles.

[0031] S102, extracting a first modal feature from the optical satellite remote sensing image, and extracting a second modal feature from the synthetic aperture radar satellite remote sensing image.

[0032] In practical applications, for optical images, image processing techniques such as edge detection, texture analysis, and color space conversion can be used to extract the spectral characteristics and texture characteristics of the surface as the first modal characteristics. These characteristics can reflect the type, distribution density, and growth state of the combustibles on the surface. For SAR images, radar scattering characteristics and roughness characteristics of the surface can be extracted as second modal characteristics through radar signal processing (such as backscatter coefficient calculation, polarization analysis, interferometry, etc.). These characteristics can reveal the physical structure, humidity state, and surface coverage of the combustibles.

[0033] S103: Input the first modal feature and the second modal feature into a convolutional neural network and a regional convolutional neural network to obtain a representation of the features of the objects in each modality.

[0034] Convolutional neural network is a deep learning model that is good at processing image data and can automatically learn and extract high-level features from images. By inputting the features of optical and SAR images into two independent convolutional neural networks, the feature representation of the objects in each mode can be obtained. At the same time, in order to better capture the spatial relationship and contextual information of surface objects, a regional convolutional neural network is introduced as a target detection model to perform more detailed recognition and classification of surface objects.

[0035] S104, the feature representation of the object under each modality is sequentially passed through the encoder part of the Transformer and the self-attention mechanism of the Transformer to obtain an interactive feature encoding vector.

[0036] Transformer is a deep learning model based on the self-attention mechanism, which performs well in processing sequence data and image data. Taking the feature representation of each modality as input, passing through the encoder part and self-attention mechanism of Transformer in sequence, the model can automatically learn and capture the correlation and complementarity between features of different modalities. Through the self-attention mechanism, the model can dynamically adjust the attention to each feature, thereby generating a more expressive interactive feature encoding vector.

[0037] S105, passing the interactive feature encoding vector through the self-attention layer of the Transformer to obtain a comprehensive feature representation.

[0038] Inputting the interactive feature encoding vector obtained in the previous step into the self-attention layer of the Transformer again can enable the model to fuse feature information of different modalities at a deeper level and generate more robust and discriminative comprehensive feature representations. These comprehensive feature representations can more comprehensively reflect the complex characteristics and spatial distribution patterns of surface combustibles.

[0039] S106, classifying the comprehensive feature representation through a deep neural network to obtain a combustible load thematic map.

[0040] In practical applications, the model will map the comprehensive feature representation to the specific category or value of the combustible load based on the learned features and rules, thereby obtaining a thematic map that reflects the distribution and load of combustibles in the tested area. In practical applications, this thematic map can provide important decision-making support for forest fire warning, ecological resource management, environmental protection, etc.

[0041] like Figure 2 As shown, it is a second schematic diagram of the satellite remote sensing combustible inversion method of the present application, which may include: S201, determine the research area.

[0042] In order to conduct remote sensing analysis and build a multimodal fusion network, it is necessary to first determine the study area. In practical applications, the study area can be determined based on the following factors: (1) Determine the appropriate research area based on the research objectives and questions; (2) Ensure that there is sufficient data availability in the selected study area, including optical remote sensing data and radar remote sensing data; (3) Ensure that the selected study area is within the coverage of satellite data.

[0043] S202, satellite remote sensing image acquisition and preprocessing.

[0044] (1) Select the satellite data source.

[0045] Taking the determined research area as the main body, the satellite remote sensing image acquisition requirements are put forward to the third-party data service provider. As an example, the acquisition quality requirements of satellite remote sensing images can adopt the following standards: 1) Data coverage: ensure that remote sensing images cover the entire study area without missing any important areas; 2) Data resolution: Select the appropriate resolution based on research needs; 3) Image output format: Images can be in Tiff or img format to ensure data accessibility and compatibility; 4) Date and time: Select images with dates and times relevant to your research objectives; 5) Image cloud cover: no more than 10%; 6) Image geometric accuracy requirements: ≤2 pixels in plain areas and ≤4 pixels in mountainous areas; 7) Image quality: Ensure that the image has no obvious noise, streaks or distortion 8) Submission of imaging results: Submit via FTP transmission or hard disk storage; 9) Imaging angle: no more than 30°; (2) Obtain images corresponding to the study area.

[0046] 1) According to the requirements for selecting satellite remote sensing data sources, obtain optical satellite remote sensing image data in the study area to form a historical optical satellite remote sensing image data set; 2) According to the requirements for selecting satellite remote sensing data sources, obtain SAR satellite remote sensing image data in the study area to form a real-time SAR satellite remote sensing image dataset.

[0047] (3) Satellite remote sensing image preprocessing After obtaining satellite remote sensing images, optical satellite remote sensing images and SAR satellite remote sensing images can be preprocessed. First, obtain image data from satellite data providers or data centers and load them into processing software. For optical images, radiation correction, atmospheric correction, geometric correction, cropping, radiation normalization, noise removal, band selection and combination, and enhancement processing can be performed. For SAR images, data decoding, slope correction, radiation correction, noise removal, polarization decomposition, geometric correction, cropping and enhancement processing can be performed, and SAR data can be converted into a format that is easier to process. Through these preprocessing steps, the quality and applicability of the image can be improved, the quality and consistency of the data can be improved, and a better basis can be provided for subsequent analysis.

[0048] S203, construct a multimodal deep learning fusion network and train it.

[0049] (1) Sample library construction As an example, a sample library for combustible monitoring and inversion can be constructed using the following method: 1) Sample size: The number of image samples of combustible samples (such as dry vegetation, wet vegetation, different types of trees, etc.) shall not be less than 1,000. The number of image samples of non-combustible samples (such as water bodies, buildings, roads, etc.) shall not be less than 500.

[0050] 2) Sample size: The size of each image sample is fixed to 1024 pixels × 1024 pixels to ensure the consistency of samples and the standardization of network input.

[0051] (2) Design the MDLF model.

[0052] like Figure 3 The figure is a schematic diagram of the MDLF model in this application. The specific implementation principle may include: In this embodiment, the MDLF model uses the Transformer architecture to achieve multimodal fusion to improve the accuracy and generalization ability of machine learning. In the MDLF model, first, the unimodal feature extraction technology is used to extract unique features from each remote sensing image. For optical satellite remote sensing images, spectral features can be extracted. For SAR satellite remote sensing images, texture features can be extracted. The following are the detailed steps for multimodal fusion using Transformer: 1) Feature extraction: Convolutional Neural Networks (CNN) and Fast Region-based Convolutional Neural Networks (FAST-RCNN) feature extraction networks are used for optical satellite remote sensing images and SAR satellite remote sensing images to obtain preliminary feature representations, which are recorded as the first modal features and the second modal features respectively.

[0053] It should be noted that convolutional neural network is a special deep neural network specially designed for processing data with grid structure, such as images. It contains components such as convolution layer, pooling layer and fully connected layer. The convolution layer slides on the input image through the convolution kernel, extracts local features and introduces nonlinearity. The pooling layer reduces the size of the feature map through the pooling operation while retaining important feature information. The fully connected layer is used to extract and integrate features to provide better input for the final output layer. The output layer usually uses the Softmax activation function for classification tasks. The advantage of the original CNN lies in its feature learning ability, which can automatically extract useful features from the original image without manually designing features. In addition, its parameter sharing and local connection characteristics enable it to reduce the amount of calculation and the number of parameters, and improve the efficiency and generalization ability of the model. With the deepening of research, the structure and design of CNN are also constantly evolving, and many improved versions have emerged, such as residual network (ResNet), Inception network, DenseNet, etc. These improved CNN structures have achieved better performance in various computer vision tasks.

[0054] The FAST-RCNN structure includes a combination of a Region Proposal Network (RPN) and a Convolutional Neural Network (CNN), followed by classification and localization steps. RPN is used to select a series of possible regions from the image, which are potential targets. Then, CNN is used to extract features for each region. Next, the features extracted by RPN and CNN are input into the classification and localization network for further classification and localization. Finally, non-maximum suppression (NMS) is performed on all detected targets to remove overlapping detection results and retain the most likely targets. The advantage of the original FAST-RCNN lies in its target detection capability, which can automatically detect multiple targets in an image, and achieves fast detection and localization of targets through the combination of RPN and CNN. However, its computational efficiency is relatively low, and feature extraction and classification are required for each possible region.

[0055] As an embodiment of the present application, when extracting the first modal feature from the optical satellite remote sensing image, the optical satellite remote sensing image is passed through three convolutional layers in sequence, and each convolutional layer is connected to a maximum pooling layer. The convolution kernel size of each convolutional layer is 3×3, and the number of feature maps is 64, 128, and 256 respectively, and each convolutional layer is connected to a 2×2 maximum pooling layer.

[0056] When extracting the second modal features from the synthetic aperture radar satellite remote sensing image, the synthetic aperture radar satellite remote sensing image is passed through two convolutional layers in sequence, and each convolutional layer is connected to a maximum pooling layer, including 2 convolutional layers, the number of feature maps is 64 and 128, and the maximum pooling layer of 2×2 is also connected.

[0057] 2) Feature encoding: The extracted first modality features and second modality features are input into the encoder part of Transformer, and each modality feature is converted into a fixed-length encoding vector.

[0058] In this embodiment, the encoder part of the Transformer adopts layer normalization processing, and the self-attention mechanism of the Transformer adopts a multi-head attention mechanism. The output of the multi-head attention mechanism is superimposed with the feature representation of the ground object under each modality and used as input to the self-attention layer of the Transformer.

[0059] 3) Multimodal interaction: The self-attention mechanism of Transformer is used to allow the feature encoding vectors of different modalities to interact with each other, thereby learning their mutual relationships and importance.

[0060] As an example, the attention mechanism module uses a 1×1 convolutional layer, and the number of output feature maps is equal to the number of input channels.

[0061] 4) Fusion representation: Through the self-attention layer of Transformer, this application obtains a comprehensive feature representation that integrates optical satellite remote sensing imagery and SAR satellite remote sensing imagery information.

[0062] In this embodiment, the self-attention layer of the Transformer includes a connected layer normalization process and a multi-layer perceptron, and the output of the multi-layer perceptron is superimposed with the input of the self-attention layer of the Transformer for input into a deep neural network.

[0063] 5) Classification: The fused comprehensive feature representation is input into the deep neural network for further processing and classification. The deep neural network can learn and abstract the fused comprehensive feature representation at a deeper level and finally output the fuel load thematic map. In this way, MDLF can not only effectively integrate information from different modalities, but also improve the model's ability to recognize complex ground features.

[0064] In this embodiment, when processing and classification are performed, the comprehensive feature representation is sequentially passed through the multi-scale fusion layer, the feature processing layer and the output layer to obtain the combustible load thematic map. As an example, the multi-scale fusion layer compresses the channel through a 1×1 convolution layer, and the number of output feature maps is 512. The feature processing layer includes 2 fully connected layers with 1024 and 512 nodes. The output layer uses a softmax activation function, and the number of output nodes is equal to the number of categories.

[0065] It should be noted that compared with traditional single-modal deep learning models such as CNN and FAST-RCNN, the Multimodal Deep Learning Fusion (MDLF) network has made significant improvements in modal fusion, feature extraction, attention mechanism, multi-scale processing, data enhancement, transfer learning, model compression and acceleration, as well as real-time and deployment. MDLF provides more comprehensive and accurate information than a single modality by fusing data from different modalities, such as optical images and radar images, thereby improving the performance and prediction accuracy of the model. In addition, the MDLF model extracts high-level features of single-modal data separately through multiple branch networks, and combines these features in the multimodal fusion layer, so that the model can understand and describe the data from different perspectives. The introduction of the attention mechanism helps the model automatically learn the key features in different modal data and assign them appropriate weights, thereby improving the model's sensitivity to key information. The multi-scale feature extraction and fusion capabilities of the MDLF model enable it to capture information at different scales, improving the model's robustness and adaptability to complex scenarios. In addition, the MDLF model uses more complex data augmentation techniques, such as rotation, scaling, and flipping, to increase the diversity of training data, thereby improving the generalization ability of the model. The MDLF model can use models pre-trained on large datasets as a starting point, which speeds up the convergence of the model and improves the performance of the model in practical tasks. The application of methods such as model pruning and quantization can reduce the model size and use more efficient inference engines to accelerate the practical application of the model. The MDLF model can be designed as a lightweight model to run on mobile devices or edge devices, or deployed to cloud services for real-time data processing and response. These improvements make the MDLF model a powerful tool in many application scenarios, such as environmental monitoring, disaster warning, and resource management.

[0066] When training the MDLF model, you can set the batch size to 32, the initial learning rate to 0.001, which decays to 0.1 every 10 epochs, the training cycle to 50, the optimizer to Adam, β1=0.9, β2=0.999, ε=10 -8 .

[0067] Data can also be enhanced during training. In this embodiment, data enhancement includes rotation of ±10°, scaling range [0.8, 1.2], and horizontal flip probability of 0.5. The validation set ratio is 20%, and the test set ratio is 10%. Model optimization uses L2 regularization, the weight decay coefficient is 0.001, and the Dropout probability is 0.5.

[0068] In actual applications, the above method can be deployed on a server with 8 CPU cores, Tesla V100 GPU, and 64GB memory, and the real-time monitoring frequency is set to update every 15 minutes. These parameters and hyperparameters will be adjusted according to actual data to optimize model performance.

[0069] S204, generation of thematic map of combustible load.

[0070] This application uses the acquired optical satellite remote sensing images and synthetic aperture radar satellite remote sensing images to extract single-modal features, and then fuses the single-modal features to integrate the information of combustibles. The features are mapped into a shared space for easy comparison and fusion. The fused features are then further processed and classified to identify and quantify combustibles.

[0071] In view of the shortcomings of existing fuel load inversion technology in terms of data source, fusion technology, model generalization, parameter optimization, real-time performance and computational cost, this application realizes high-precision, large-scale and real-time monitoring of forest fuel load, providing strong technical support for forest fire prevention and control and related management decisions. In addition, this application realizes the identification of potential high fuel load areas and can generate standardized inversion reports.

[0072] like Figure 4 As shown, it is a schematic diagram of the satellite remote sensing combustible inversion system of the present application, which may include: An image acquisition module is used to acquire optical satellite remote sensing images and synthetic aperture radar satellite remote sensing images of the area to be measured; A feature extraction module, used for extracting first modal features from optical satellite remote sensing images and extracting second modal features from synthetic aperture radar satellite remote sensing images; A feature representation module, used for inputting the first modal feature and the second modal feature into a convolutional neural network and a regional convolutional neural network to obtain a feature representation of the ground object under each modality; The interactive module is used to make the feature representation of each modality pass through the encoder part of the Transformer and the self-attention mechanism of the Transformer in turn to obtain the interactive feature encoding vector; The comprehensive feature module is used to pass the interactive feature encoding vector through the self-attention layer of the Transformer to obtain a comprehensive feature representation; The inversion module is used to classify the comprehensive feature representation through a deep neural network to obtain the fuel load thematic map.

[0073] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0074] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0075] An embodiment of the present application also provides an electronic device, which may include one or more processors, a memory, and a communication interface.

[0076] The memory, the communication interface and the processor are coupled, for example, the memory, the communication interface and the processor may be coupled together via a bus.

[0077] The communication interface is used for data transmission with other devices. The memory stores computer program code. The computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the steps of the satellite remote sensing combustible inversion method.

[0078] Wherein, the processor can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the present disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and the like. The processor can be used to support electronic devices to execute the method steps provided in the above embodiments.

[0079] The bus may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The above bus may be divided into an address bus, a data bus, a control bus, etc.

[0080] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned satellite remote sensing combustible material inversion method are implemented.

[0081] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0082] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A satellite remote sensing combustible inversion method, characterized in that: include: Obtain optical satellite remote sensing images and synthetic aperture radar satellite remote sensing images of the area to be measured; Extracting the first modal features from the optical satellite remote sensing images and extracting the second modal features from the synthetic aperture radar satellite remote sensing images; Inputting the first modal feature and the second modal feature into a convolutional neural network and a regional convolutional neural network to obtain a representation of the features of the ground objects in each modality; The feature representation of each modality is passed through the encoder part of the Transformer and the self-attention mechanism of the Transformer in turn to obtain the interactive feature encoding vector; The interactive feature encoding vector is passed through the self-attention layer of the Transformer to obtain a comprehensive feature representation; The comprehensive feature representation is classified through a deep neural network to obtain the fuel load thematic map.

2. The satellite remote sensing combustible inversion method according to claim 1, characterized in that: The method of inputting the first modal feature and the second modal feature into a convolutional neural network and a regional convolutional neural network to obtain a representation of the ground feature under each modality includes: The optical satellite remote sensing image is passed through three convolutional layers in sequence, and each convolutional layer is connected to a maximum pooling layer to extract the first modal features; The synthetic aperture radar satellite remote sensing image is passed through two convolutional layers in sequence, and each convolutional layer is connected to a maximum pooling layer to extract the second modal features.

3. The satellite remote sensing combustible inversion method according to claim 2 is characterized in that: The convolution kernel size of the convolution layer used in the first modality feature extraction is 3×3, and the size of the maximum pooling layer is 2×2; The size of the maximum pooling layer used in the second modality feature extraction is 2×2.

4. The satellite remote sensing combustible inversion method according to claim 1 is characterized in that: The encoder part of the Transformer adopts layer normalization processing; the self-attention mechanism of the Transformer adopts a multi-head attention mechanism; The output of the multi-head attention mechanism is superimposed with the feature representation of the object in each modality and input into the self-attention layer of the Transformer.

5. The satellite remote sensing combustible inversion method according to claim 1 is characterized in that: The Transformer's self-attention layer includes a connected layer normalization process and a multi-layer perceptron; The output of the multilayer perceptron is superimposed with the input of the self-attention layer of the Transformer for input into a deep neural network.

6. The satellite remote sensing combustible inversion method according to claim 1, characterized in that: The method of classifying the comprehensive feature representation by a deep neural network includes: The combustible load thematic map is obtained by passing through the multi-scale fusion layer, feature processing layer and output layer in sequence.

7. The satellite remote sensing combustible inversion method according to claim 6 is characterized in that: The multi-scale fusion layer adopts a 1×1 convolution layer; the feature processing layer includes two fully connected layers; and the output layer adopts a Softmax layer.

8. A satellite remote sensing combustible inversion system, characterized in that: include: An image acquisition module is used to acquire optical satellite remote sensing images and synthetic aperture radar satellite remote sensing images of the area to be measured; A feature extraction module, used for extracting first modal features from optical satellite remote sensing images and extracting second modal features from synthetic aperture radar satellite remote sensing images; A feature representation module, used for inputting the first modal feature and the second modal feature into a convolutional neural network and a regional convolutional neural network to obtain a feature representation of the ground object under each modality; The interactive module is used to make the feature representation of each modality pass through the encoder part of the Transformer and the self-attention mechanism of the Transformer in turn to obtain the interactive feature encoding vector; The comprehensive feature module is used to pass the interactive feature encoding vector through the self-attention layer of the Transformer to obtain a comprehensive feature representation; The inversion module is used to classify the comprehensive feature representation through a deep neural network to obtain the fuel load thematic map.

9. The satellite remote sensing combustible inversion system according to claim 8, characterized in that: The processing method in the feature representation module includes: The optical satellite remote sensing image is passed through three convolutional layers in sequence, and each convolutional layer is connected to a maximum pooling layer to extract the first modal features; The synthetic aperture radar satellite remote sensing image is passed through two convolutional layers in sequence, and each convolutional layer is connected to a maximum pooling layer to extract the second modal features.

10. The satellite remote sensing combustible inversion system according to claim 8, characterized in that: In the interaction module, the encoder part of the Transformer adopts layer normalization processing; the self-attention mechanism of the Transformer adopts a multi-head attention mechanism; the output of the multi-head attention mechanism is superimposed with the feature representation of the ground object under each modality and used as input to the self-attention layer of the Transformer; In the comprehensive feature module, the self-attention layer of the Transformer includes a connected layer normalization process and a multi-layer perceptron; the output of the multi-layer perceptron is superimposed on the input of the self-attention layer of the Transformer for input into a deep neural network.

11. The satellite remote sensing combustible inversion system according to claim 8, characterized in that: In the inversion module, the method for classifying the comprehensive feature representation by a deep neural network includes: The combustible load thematic map is obtained by passing through the multi-scale fusion layer, feature processing layer and output layer in sequence.

12. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the satellite remote sensing combustible inversion method as described in any one of claims 1-7.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the satellite remote sensing combustible inversion method as described in any one of claims 1 to 7 are implemented.

Images