Slope vegetation classification method and device based on multi-modal deep features

Abstract

The application provides a slope vegetation classification method based on multi-modal deep features, and has the characteristics that multi-source remote sensing data of a target slope region is acquired, the multi-source remote sensing data includes high-resolution optical data, hyperspectral remote sensing data, data elevation models and radar scattering data from multiple data source platforms; a preset deep learning algorithm is used to extract multi-modal deep features from the multi-source remote sensing data, the multi-modal deep features include vegetation spatial features, vegetation spectral features and vegetation depth features; a preset fusion device is used to fuse the multi-modal deep features to obtain target fusion features; and a preset vegetation classification model is used to classify the target fusion features to obtain vegetation classification information of the target slope region. In the case that current slope vegetation information extraction is difficult, the slope vegetation classification precision is improved.

Description

Technical Field

[0001] This invention relates to the field of slope vegetation classification technology, and in particular to a slope vegetation classification method and apparatus based on multimodal depth features. Background Technology

[0002] Due to natural disasters such as earthquakes and landslides, as well as human activities such as mining and construction, many slopes have become exposed, making slope restoration a frequent and routine need. Classifying slope vegetation in restoration areas is crucial for data support in simulating the carbon budget processes of slope restoration ecosystems.

[0003] Currently, slope vegetation classification research typically focuses on flat areas with relatively simple classification objects, primarily relying on single data sources collected by drones. However, the complex structure and composition of slope vegetation hinders data collection and information mining from single data sources, leading to difficulties in accurately extracting vegetation information and low accuracy in vegetation category classification. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a slope vegetation classification method and device based on multimodal depth features, which can solve the technical problem of low slope vegetation classification accuracy caused by the difficulty in extracting slope vegetation information.

[0005] In a first aspect, embodiments of the present invention provide a slope vegetation classification method based on multimodal depth features, including:

[0006] Acquire multi-source remote sensing data of the target slope area, including high-resolution optical data, hyperspectral remote sensing data, data elevation models, and radar scattering data from multiple data source platforms;

[0007] Using a preset deep learning algorithm, multimodal depth features are extracted from the multi-source remote sensing data. The multimodal depth features include vegetation spatial features, vegetation spectral features, and vegetation depth features.

[0008] The multimodal deep features are fused using a preset fusion processor to obtain the target fused features;

[0009] The target fusion features are classified by a preset vegetation classification model to obtain the vegetation classification information of the target slope area.

[0010] In some embodiments of the present invention, the step of extracting multimodal depth features from the multi-source remote sensing data using a preset deep learning algorithm includes:

[0011] Based on preset mask data, the multi-source remote sensing data is filtered to obtain target multi-source remote sensing data. The preset mask data includes vegetation mask data extracted based on historical remote sensing data.

[0012] Using a pre-defined dilated convolutional neural network, spatial features are extracted from the multi-source remote sensing data of the target to obtain the spatial features of vegetation;

[0013] Using a pre-defined dense autoencoder, spectral features are extracted from the multi-source remote sensing data of the target to obtain vegetation spectral features;

[0014] Using a preset deep neural network, based on the vegetation spatial features and the vegetation spectral features, depth features are extracted from the target multi-source remote sensing data to obtain vegetation depth features.

[0015] In some embodiments of the present invention, the step of filtering the multi-source remote sensing data based on preset mask data to obtain target multi-source remote sensing data includes:

[0016] Image segmentation is performed on multiple historical remote sensing data sets to extract vegetation mask regions from each set of historical remote sensing data sets;

[0017] The vegetation masking regions are weighted and fused to establish the vegetation masking data.

[0018] The multi-source remote sensing data and the vegetation mask data are matched, and the overlapping area is determined as the vegetation area in the multi-source remote sensing data;

[0019] The vegetation area in the multi-source remote sensing data is retained to obtain the target multi-source remote sensing data.

[0020] In some embodiments of the present invention, the step of using a preset dilated convolutional neural network to extract spatial features from the target multi-source remote sensing data to obtain vegetation spatial features includes:

[0021] Extract geometric features from the target multi-source remote sensing data and determine the geometric features as a geometric feature map; extract texture features from the target multi-source remote sensing data and determine the texture features as a geometric feature map; extract location features from the target multi-source remote sensing data and determine the location features as a location feature map.

[0022] Using the dilated convolution kernel in the preset dilated convolutional neural network, the geometric feature map, the texture feature map, and the position feature map are convolved and fully connected to obtain the vegetation spatial features.

[0023] In some embodiments of the present invention, the step of extracting spectral features from the target multi-source remote sensing data using a preset dense autoencoder to obtain vegetation spectral features includes:

[0024] The target multi-source remote sensing data is compressed by the encoder in the preset dense autoencoder to obtain a low-dimensional hidden representation;

[0025] The low-dimensional hidden representation is input into the decoder in the preset autoencoder. The decoder performs a nonlinear inverse transformation on the low-dimensional hidden representation to reconstruct the dimensionality-reduced target multi-source remote sensing data.

[0026] Spectral features were extracted from the dimensionality-reduced multi-source remote sensing data of the target to obtain the spectral features of vegetation in various bands.

[0027] In some embodiments of the present invention, the step of using a preset deep neural network to extract depth features from the target multi-source remote sensing data based on the vegetation spatial features and the vegetation spectral features to obtain vegetation depth features includes:

[0028] Using the preset deep neural network, the spatial features of the vegetation are extracted to obtain the spatial variation features of the target multi-source remote sensing data;

[0029] Using the preset deep neural network, the spectral variation features of the target multi-source remote sensing data are obtained by performing deep feature extraction on the spectral features of the vegetation.

[0030] The spatial variation features and the spectral variation features are determined as vegetation depth features.

[0031] In some embodiments of the present invention, the step of classifying the target fusion features using a preset vegetation classification model to obtain vegetation classification information for the target slope area includes:

[0032] Using the preset vegetation classification model, the target fusion features are convolutionally and fully connected to output multiple vegetation types of the target slope area;

[0033] Statistical analysis was performed on various vegetation types to obtain vegetation classification information.

[0034] Secondly, embodiments of the present invention provide a slope vegetation classification device based on multimodal depth features, including at least one control processor and a memory for communicatively connecting with the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enables the at least one control processor to perform the slope vegetation classification method based on multimodal depth features as described in the first aspect above.

[0035] Thirdly, embodiments of the present invention provide an electronic device including a slope vegetation classification device based on multimodal depth features as described in the second aspect above.

[0036] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions for performing the slope vegetation classification method based on multimodal depth features as described in the first aspect above.

[0037] The slope vegetation classification method based on multimodal depth features according to embodiments of the present invention has at least the following beneficial effects: By acquiring multi-source remote sensing data of the target slope area, the multi-source remote sensing data includes high-resolution optical data, hyperspectral remote sensing data, data elevation models, and radar scattering data from multiple data source platforms. Compared with a single data source from UAVs, this application can classify slope vegetation from multiple data source dimensions. Specifically, a preset deep learning algorithm is used to extract multimodal depth features from the multi-source remote sensing data, the multimodal depth features including vegetation spatial features, vegetation spectral features, etc. This application uses a pre-defined fusion processor to fuse multimodal depth features to obtain target fused features. This allows for the extraction of slope vegetation features from multiple feature dimensions, reducing the difficulty of slope vegetation feature extraction. Furthermore, feature extraction from spatial, spectral, and depth subspaces reduces the interference of complex slope terrain on vegetation classification. Finally, a pre-defined vegetation classification model is used to classify the target fused features, obtaining vegetation classification information for the target slope area. This improves the accuracy of slope vegetation classification and provides more reliable data support for the restoration of ecosystems in slope areas. Attached Figure Description

[0038] Figure 1 This is a flowchart of a slope vegetation classification method based on multimodal depth features provided in an embodiment of the present invention;

[0039] Figure 2 This is a flowchart of a method for extracting multimodal depth features provided in another embodiment of the present invention;

[0040] Figure 3 This is a flowchart of a method for obtaining multi-source remote sensing data of a target provided in another embodiment of the present invention;

[0041] Figure 4 This is a flowchart of a method for obtaining vegetation spatial characteristics provided in another embodiment of the present invention;

[0042] Figure 5 This is a flowchart of a method for obtaining vegetation spectral characteristics provided in another embodiment of the present invention;

[0043] Figure 6 This is a flowchart of a method for obtaining vegetation depth features according to another embodiment of the present invention.

[0044] Figure 7 This is a flowchart of a method for obtaining vegetation classification information through a preset vegetation classification model, provided in another embodiment of the present invention;

[0045] Figure 8 This is a structural diagram of a slope vegetation classification device based on multimodal depth features provided in another embodiment of the present invention. Detailed Implementation

[0046] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0047] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0048] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0049] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0050] This invention provides a slope vegetation classification method based on multimodal depth features, which has at least the following beneficial effects: By acquiring multi-source remote sensing data of the target slope area, including high-resolution optical data, hyperspectral remote sensing data, data elevation models, and radar scattering data from multiple data source platforms, this application can classify slope vegetation from multiple data source dimensions compared to a single data source like UAVs. Specifically, a preset deep learning algorithm is used to extract multimodal depth features from the multi-source remote sensing data. These multimodal depth features include vegetation spatial features, vegetation spectral features, and vegetation depth features. A preset fusion processor is used to fuse the multimodal depth features to obtain target fused features, thereby extracting slope vegetation features from multiple feature dimensions and reducing the difficulty of extracting slope vegetation features. Furthermore, this application extracts features from spatial, spectral, and depth subspaces, which can reduce the interference of complex slope terrain on vegetation classification. Finally, a preset vegetation classification model is used to classify the target fused features to obtain vegetation classification information for the target slope area, improving the accuracy of slope vegetation classification and providing more reliable data support for the restoration of ecosystems in slope areas.

[0051] The control method of the present invention will be further described below with reference to the accompanying drawings.

[0052] Reference Figure 1 , Figure 1 The flowchart illustrates a slope vegetation classification method based on multimodal depth features, provided in an embodiment of the present invention. This method includes, but is not limited to, the following steps:

[0053] Step S101: Obtain multi-source remote sensing data of the target slope area. The multi-source remote sensing data includes high-resolution optical data, hyperspectral remote sensing data, data elevation model and radar scattering data from multiple data source platforms.

[0054] It should be noted that the target slope area is the slope region requiring vegetation classification. The data source platform includes various data repositories or commercial remote sensing data platforms. Multi-source remote sensing data can be obtained based on sensors such as optical sensors (e.g., satellites), synthetic aperture radar (SAR), and thermal infrared sensors. Optionally, high-resolution optical data can specifically be high-resolution optical images, hyperspectral remote sensing data can specifically be hyperspectral remote sensing images, and radar scattering data can specifically be UAV radar images. Compared to the currently common practice of collecting slope vegetation data from a single data source using UAVs, this application achieves multi-dimensional extraction of slope vegetation features by acquiring multi-source remote sensing data of the target slope area, thereby improving the accuracy of subsequent slope vegetation classification.

[0055] Step S102: Using a preset deep learning algorithm, extract multimodal depth features from multi-source remote sensing data. The multimodal depth features include vegetation spatial features, vegetation spectral features, and vegetation depth features.

[0056] It should be noted that the preset deep learning algorithms include, but are not limited to, convolutional neural networks, autoencoders, and recurrent neural networks. Specifically, for spatial features, dilated convolutional neural networks can be used to extract spatial features at different scales; for vegetation spectral features, dense autoencoders can be used to reduce data noise in multi-source remote sensing data and improve the extraction accuracy of vegetation spectral features; and for depth features, recurrent neural networks can be used to extract deep features from multi-source remote sensing data. This embodiment extracts features from multi-source remote sensing data from spatial, spectral, and depth subspaces to comprehensively classify slope vegetation based on the multi-dimensional features of the multi-source remote sensing data, thus reducing the difficulty of feature extraction.

[0057] Step S103: Multimodal deep features are fused using a preset fusion processor to obtain the target fused features;

[0058] It should be noted that multimodal deep features are connected, superimposed, or weighted by a preset fusion unit to form a new feature vector as the target fusion feature. Optionally, feature fusion can be performed based on a simple merge fusion unit, a weighted summation fusion unit, a principal component analysis (PCA) fusion unit, or a learning fusion unit.

[0059] Step S104: Classify the target fusion features using a preset vegetation classification model to obtain vegetation classification information for the target slope area.

[0060] It should be noted that in this step, the preset vegetation classification model can be a convolutional neural network. To address the difficulties in extracting vegetation information and the low accuracy of vegetation category classification caused by the complex structure and composition of slope vegetation, the target fusion features obtained by fusing multi-level spatial, spectral, and depth features of slope vegetation are input into the convolutional neural network for processing. That is, after feature extraction from multi-source remote sensing data of the target slope area, feature fusion is then performed. The resulting target fusion features have multi-dimensional characteristics, and inputting them into the preset vegetation classification model can improve the accuracy of slope vegetation classification, achieving refined extraction of slope vegetation information.

[0061] In some embodiments, reference Figure 2 , Figure 2 This is a flowchart of a method for extracting multimodal depth features according to another embodiment of the present invention. Step S102 further includes:

[0062] Step S1021: Based on the preset mask data, the multi-source remote sensing data is filtered to obtain the target multi-source remote sensing data. The preset mask data includes vegetation mask data extracted based on historical remote sensing data.

[0063] Step S1022: Using a pre-set dilated convolutional neural network, spatial features are extracted from the target multi-source remote sensing data to obtain vegetation spatial features;

[0064] Step S1023: Using a preset dense autoencoder, spectral features are extracted from the multi-source remote sensing data of the target to obtain vegetation spectral features;

[0065] Step S1024: Using a preset deep neural network, based on vegetation spatial features and vegetation spectral features, depth features are extracted from the target multi-source remote sensing data to obtain vegetation depth features.

[0066] It should be noted that due to the complex structure and morphology of slopes, a large amount of noise interference is caused to slope vegetation classification. Therefore, this application uses the prior knowledge of the preset mask data to filter multi-source remote sensing data to reduce terrain noise interference and reduce the difficulty of extracting slope vegetation features. At the same time, multi-scale spatial features are extracted by spatial convolutional neural networks, spectral noise interference is reduced by dense autoencoders, and deeper features of vegetation spatial and spectral features are further extracted by deep neural networks to effectively extract slope vegetation features from multiple dimensions and improve the accuracy of slope vegetation classification.

[0067] In some embodiments, reference Figure 3 , Figure 3 This is a flowchart of a method for obtaining multi-source remote sensing data of a target according to another embodiment of the present invention. Step S1021 further includes:

[0068] Step S211: Perform image segmentation on multiple historical remote sensing data to extract the vegetation mask area in each historical remote sensing data.

[0069] Step S212: Perform weighted fusion on multiple vegetation mask areas to establish vegetation mask data;

[0070] Step S213: Match the multi-source remote sensing data with the vegetation mask data, and determine the overlapping area as the vegetation area in the multi-source remote sensing data.

[0071] Step S214: Preserve the vegetation areas in the multi-source remote sensing data to obtain the target multi-source remote sensing data.

[0072] It should be noted that the historical remote sensing data refers to multi-source remote sensing data of the target slope area within the most recent historical time period. Image segmentation is an image processing method used to filter out slope vegetation areas from the remote sensing data, dividing the slope into vegetation areas and non-vegetated areas. This can be implemented using instance segmentation algorithms, ultimately obtaining the vegetation mask area for each historical remote sensing data point. Since slope areas undergo topographic changes over time—for example, landslides may damage slope vegetation, resulting in bare slopes, which may later regrow vegetation—this application uses weighted fusion of vegetation mask areas from different times to establish vegetation mask data, taking into account slope area changes. Optionally, higher weights are assigned to vegetation mask areas corresponding to later historical remote sensing data, and lower weights are assigned to vegetation mask areas corresponding to earlier historical remote sensing data. This makes the vegetation mask data more consistent with the current period's multi-source remote sensing data filtering process, improving the filtering accuracy of slope vegetation areas. Multi-source remote sensing data is matched with vegetation mask data. The remote sensing areas that overlap with the vegetation mask data in the multi-source remote sensing data are taken as vegetation areas. The vegetation areas in the multi-source remote sensing data are retained to obtain the target multi-source remote sensing data.

[0073] In some embodiments, reference Figure 4 , Figure 4 This is a flowchart of a method for obtaining vegetation spatial characteristics according to another embodiment of the present invention, step S1022, including:

[0074] Step S221: Extract geometric features from the multi-source remote sensing data of the target and determine the geometric features as a geometric feature map; extract texture features from the multi-source remote sensing data of the target and determine the texture features as a geometric feature map; extract location features from the multi-source remote sensing data of the target and determine the location features as a location feature map.

[0075] Step S222: Using the dilated convolution kernel in the preset dilated convolutional neural network, convolution and full connection are performed on the geometric feature map, texture feature map and position feature map to obtain the vegetation spatial features.

[0076] It should be noted that, in this embodiment, for spatial feature extraction, geometric, positional, and textural features of the target multi-source remote sensing data are extracted using feature extraction algorithms such as Gray Level Co-occurrence Matrix (GLCM), Gabor filter, Local Binary Pattern (LBP), or GIST. Then, a dilated convolutional neural network is used to convolve and fully connect the geometric, textural, and positional feature maps with dilated convolution kernels to obtain the vegetation spatial features. The introduction of dilated convolution kernels spatially expands the receptive field of the convolution kernels, enabling more effective identification of spatial features at different scales.

[0077] It should be noted that for spatial feature extraction, geometric, location, and texture features of the target multi-source remote sensing data are extracted using feature extraction algorithms such as Gray Level Co-occurrence Matrix (GLCM), Gabor filter, Local Binary Pattern (LBP), or GIST. Then, a dilated convolutional neural network is used to convolve and fully connect the geometric, texture, and location feature maps with dilated convolution kernels to obtain the vegetation spatial features. Introducing dilated convolution kernels spatially expands the receptive field of the convolution kernel, enabling more effective identification of spatial features at different scales.

[0078] In some embodiments, reference Figure 5 , Figure 5 This is a flowchart of a method for obtaining vegetation spectral characteristics according to another embodiment of the present invention, step S1023, including:

[0079] Step S231: Compress the target multi-source remote sensing data using the encoder in the preset dense autoencoder to obtain a low-dimensional hidden representation;

[0080] Step S232: Input the low-dimensional hidden representation into the decoder in the preset autoencoder, and perform a nonlinear inverse transformation on the low-dimensional hidden representation through the decoder to reconstruct the dimensionality-reduced target multi-source remote sensing data.

[0081] Step S233: Extract spectral features from the dimensionality-reduced multi-source remote sensing data of the target to obtain the spectral features of vegetation in various bands.

[0082] It should be noted that a dense autoencoder is an unsupervised learning model consisting of an encoder and a decoder. It compresses and represents input data, reconstructing a representation that approximates the original data. The encoder portion of the dense autoencoder maps the input data to a low-dimensional hidden representation. Through nonlinear transformations of multiple hidden layers, it learns and extracts key features from the input data. The encoder typically employs a fully connected neural network structure, where the number of nodes in each hidden layer gradually decreases, ultimately compressing the input data into a lower-dimensional hidden representation. The decoder portion remaps the hidden representation to the same dimension as the original input data, reconstructing an output as close as possible to the original data through nonlinear inverse transformations of multiple hidden layers. The decoder's structure is the reverse of the encoder, with the number of nodes in the hidden layers gradually increasing, ultimately generating an output similar to the original input data. This embodiment introduces a dense autoencoder to reduce the dimensionality of multi-source remote sensing data, thereby reducing data redundancy and noise interference. Furthermore, plant spectra exhibit the following reflectance characteristics: a small reflectance peak of 10%–20% near 0.55 μm in the visible light spectrum; two distinct absorption valleys near 0.45 μm and 0.67 μm; a steep slope with a sharp increase in reflectance between 0.7 and 0.8 μm; and a high reflectance peak of 40% or greater between 0.7 and 1.1 μm in the near-infrared band; three absorption valleys at 1.4 μm, 1.9 μm, and 2.6 μm; while healthy green... Plants contain a large amount of chlorophyll, typically reflecting 40%–50% of the energy in the near-infrared (0.7–1.1 m) band and absorbing nearly 80%–90% of the energy in the visible light (0.4–0.7 m) band. This embodiment employs spectral feature extraction methods such as principal component analysis (PCA), linear discriminant analysis (LDA), and wavelet transform to extract reflectance or radiance spectral features of different vegetation types in different bands based on the plant's spectral reflectance characteristics, thereby achieving spectral feature extraction and analysis.

[0083] In some embodiments, reference Figure 6 , Figure 6 This is a flowchart of a method for obtaining vegetation depth features according to another embodiment of the present invention, step S1024, including:

[0084] Step S241: Using a preset deep neural network, perform deep feature extraction on the spatial features of vegetation to obtain the spatial variation features of the target multi-source remote sensing data.

[0085] Step S242: Using a preset deep neural network, perform deep feature extraction on the spectral features of vegetation to obtain the spectral variation features of the target multi-source remote sensing data;

[0086] Step S243: Determine the spatial variation characteristics and spectral variation characteristics as vegetation depth characteristics.

[0087] It should be noted that the preset deep neural network can be a recurrent neural network (RNN), which can be used to process time-series remote sensing data, such as meteorological data or time-series remote sensing image data, to help extract temporal information and long-term dependencies from the sequence data. Slope areas are susceptible to natural disasters or human-caused damage, and the spatial distribution and spectrum of slope vegetation exhibit certain temporal characteristics. Therefore, this embodiment uses a recurrent neural network to extract deep-level features of vegetation spatial characteristics to obtain spatial variation features, and extracts deep-level features of vegetation spectral characteristics to obtain spectral variation features. These spatial variation features and spectral variation features are used as vegetation depth features to improve the accuracy of vegetation classification.

[0088] In some embodiments, reference Figure 7 , Figure 7 This is a flowchart of a method for obtaining vegetation classification information through a preset vegetation classification model, provided in another embodiment of the present invention. Step S104 includes:

[0089] Step S1041: Using a preset vegetation classification model, perform convolution and full connection on the target fusion features to output multiple vegetation types in the target slope area;

[0090] Step S1042: Statistical analysis of multiple vegetation types is performed to obtain vegetation classification information.

[0091] It should be noted that the preset vegetation classification model classifies all target fusion features within the target slope area sequentially and outputs all vegetation types in the target slope area; by counting the number of each vegetation type, vegetation classification information is obtained.

[0092] like Figure 8 As shown, Figure 8 This is a structural diagram of a slope vegetation classification device based on multimodal depth features according to an embodiment of the present invention. The present invention also provides a slope vegetation classification device based on multimodal depth features, comprising:

[0093] The processor 801 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0094] The memory 802 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 using the methods of the embodiments of this application.

[0095] The 803 input / output interface is used to implement information input and output.

[0096] The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0097] Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804);

[0098] The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0099] This application also provides an electronic device, including the slope vegetation classification device based on multimodal depth features as described above.

[0100] This application embodiment also provides a storage medium, which is a computer-readable storage medium, storing a computer program that, when executed by a processor, implements the above-described slope vegetation classification method based on multimodal depth features.

[0101] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate, and may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0103] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A slope vegetation classification method based on multimodal depth features, characterized in that, include: Acquire multi-source remote sensing data of the target slope area, including high-resolution optical data, hyperspectral remote sensing data, data elevation models, and radar scattering data from multiple data source platforms; Using a preset deep learning algorithm, multimodal depth features are extracted from the multi-source remote sensing data. The multimodal depth features include vegetation spatial features, vegetation spectral features, and vegetation depth features. The multimodal deep features are fused using a preset fusion processor to obtain the target fused features; The target fusion features are classified by a preset vegetation classification model to obtain the vegetation classification information of the target slope area; The step of extracting multimodal depth features from the multi-source remote sensing data using a preset deep learning algorithm includes: Based on preset mask data, the multi-source remote sensing data is filtered to obtain target multi-source remote sensing data. The preset mask data includes vegetation mask data extracted based on historical remote sensing data. In addition, the filtering process based on preset mask data to obtain target multi-source remote sensing data includes: Image segmentation is performed on multiple historical remote sensing data sets to extract vegetation mask regions from each set of historical remote sensing data sets; The vegetation masking regions are weighted and fused to establish the vegetation masking data. The multi-source remote sensing data is matched with the vegetation mask data, and the overlapping area is determined as the vegetation area in the multi-source remote sensing data; The vegetation area in the multi-source remote sensing data is retained to obtain the target multi-source remote sensing data.

2. The slope vegetation classification method based on multimodal depth features as described in claim 1, characterized in that, The step of extracting multimodal depth features from the multi-source remote sensing data using a preset deep learning algorithm further includes: Using a pre-defined dilated convolutional neural network, spatial features are extracted from the multi-source remote sensing data of the target to obtain the spatial features of vegetation; Using a pre-defined dense autoencoder, spectral features are extracted from the multi-source remote sensing data of the target to obtain vegetation spectral features; Using a preset deep neural network, based on the vegetation spatial features and the vegetation spectral features, depth features are extracted from the target multi-source remote sensing data to obtain vegetation depth features.

3. The slope vegetation classification method based on multimodal depth features as described in claim 2, characterized in that, The step of using a pre-defined dilated convolutional neural network to extract spatial features from the target multi-source remote sensing data to obtain vegetation spatial features includes: Geometric features are extracted from the target multi-source remote sensing data, and the geometric features are determined as texture feature maps. Texture features are extracted from the target multi-source remote sensing data, and the texture features are determined as geometric feature maps. Location features are extracted from the target multi-source remote sensing data, and the location features are determined as location feature maps. Using the dilated convolution kernel in the preset dilated convolutional neural network, the geometric feature map, the texture feature map, and the position feature map are convolved and fully connected to obtain the vegetation spatial features.

4. The slope vegetation classification method based on multimodal depth features as described in claim 2, characterized in that, The step of extracting spectral features from the multi-source remote sensing data of the target using a preset dense autoencoder to obtain vegetation spectral features includes: The target multi-source remote sensing data is compressed by the encoder in the preset dense autoencoder to obtain a low-dimensional hidden representation; The low-dimensional hidden representation is input into the decoder in the preset dense autoencoder. The decoder performs a nonlinear inverse transformation on the low-dimensional hidden representation to reconstruct the dimensionality-reduced target multi-source remote sensing data. Spectral features were extracted from the dimensionality-reduced multi-source remote sensing data of the target to obtain the spectral features of vegetation in various bands.

5. The slope vegetation classification method based on multimodal depth features as described in claim 2, characterized in that, The method of using a preset deep neural network to extract depth features from the target multi-source remote sensing data based on the vegetation spatial features and the vegetation spectral features to obtain vegetation depth features includes: Using the preset deep neural network, the spatial features of the vegetation are extracted to obtain the spatial variation features of the target multi-source remote sensing data; Using the preset deep neural network, the spectral variation features of the target multi-source remote sensing data are obtained by performing deep feature extraction on the spectral features of the vegetation. The spatial variation features and the spectral variation features are determined as vegetation depth features.

6. The slope vegetation classification method based on multimodal depth features as described in claim 1, characterized in that, The step of classifying the target fusion features using a preset vegetation classification model to obtain vegetation classification information for the target slope area includes: Using the preset vegetation classification model, the target fusion features are convolutionally and fully connected to output multiple vegetation types of the target slope area; Statistical analysis was performed on various vegetation types to obtain vegetation classification information.

7. A slope vegetation classification device based on multimodal depth features, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the slope vegetation classification method based on multimodal depth features as described in any one of claims 1 to 6.

8. An electronic device, characterized in that, Includes the slope vegetation classification device based on multimodal depth features as described in claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the slope vegetation classification method based on multimodal depth features as described in any one of claims 1 to 6.

Images