Surface ecological data extraction method, device, equipment and storage medium

Abstract

The present invention provides a method and device for extracting surface ecological data, an apparatus, a device and a storage medium, wherein the method for extracting surface ecological data first uses the acquired hyperspectral images taken for different objects on the same surface area as the hyperspectral images of surface data to be extracted, and then obtains the target surface ecological data distribution map by inputting the hyperspectral images of surface data to be extracted into a preset surface ecological data extraction network for data extraction. Since the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral images of surface data to be extracted, and the processed hyperspectral images are semantically segmented and corrected by an attention mechanism to obtain the target surface ecological data distribution map, the surface ecological data can be accurately and reliably extracted under the premise of low model complexity, which not only reduces the serious overfitting phenomenon caused by high model complexity, but also greatly improves the extraction accuracy of surface ecological data.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for extracting surface ecological data, equipment and storage medium. Background Art

[0002] Surface ecological and environmental problems, manifested primarily in land degradation, a sharp decline in wetland and biodiversity, declining surface water quality and groundwater levels, and a prominent conflict between people and land, are impacting ecological and environmental security and sustainable development. Therefore, timely and accurate extraction of representative surface ecological data has become a critical issue that urgently needs to be addressed.

[0003] Existing methods extract typical surface ecological data from hyperspectral ground object images by learning classification models. For example, data can be extracted by sending the features extracted from the hyperspectral ground object images into a classifier for supervised learning to generate a classification model; data can also be extracted by training a multi-feature classifier for image classification through the spectral features of the hyperspectral ground object images and the high-level spatial correlation depth features extracted by the convolutional neural network.

[0004] However, since existing methods mainly focus on extracting low-dimensional features such as texture and color for supervised learning or extracting high-dimensional features when extracting ground object data, they cannot meet the growing requirements for data extraction accuracy, and deep learning models will also lead to serious overfitting due to the high complexity of the model. Summary of the Invention

[0005] The present invention provides a surface ecological data extraction method and device, equipment and storage medium to address the defects of the existing technology such as low data extraction accuracy and serious overfitting, and achieves the purpose of extracting surface ecological data with high accuracy while greatly reducing the overfitting phenomenon.

[0006] The present invention provides a method for extracting surface ecological data, comprising:

[0007] Acquiring a hyperspectral image to be extracted from surface data, wherein the hyperspectral image to be extracted from the surface data includes hyperspectral images of different objects on the same surface area;

[0008] The hyperspectral image to be extracted from the surface data is input into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map; wherein, the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image to be extracted from the surface data, and to obtain the target surface ecological data distribution map after performing semantic segmentation and attention mechanism correction on the processed hyperspectral image.

[0009] According to a surface ecological data extraction method provided by the present invention, the preset surface ecological data extraction network includes a preset multimodal principal component analysis subnetwork and a preset semantic segmentation subnetwork. The hyperspectral image of the surface data to be extracted is input into the preset surface ecological data extraction network to obtain a target surface ecological data distribution map, including:

[0010] Inputting the surface data to be extracted hyperspectral images into a preset multimodal principal component analysis subnetwork to perform multimodal principal component analysis, and obtaining each principal component analysis image after principal component analysis of each modality;

[0011] Input each principal component analysis image into the semantic segmentation subnetwork to perform semantic segmentation and spatial channel joint attention correction respectively, and obtain each predicted surface data distribution map corresponding to each principal component analysis image;

[0012] Based on each predicted surface data distribution map, a target surface ecological data distribution map is obtained.

[0013] According to a surface ecological data extraction method provided by the present invention, the preset multimodal principal component analysis subnetwork includes a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork, and the hyperspectral images of the surface data to be extracted are respectively input into the preset multimodal principal component analysis subnetwork for multimodal principal component analysis to obtain each principal component analysis image after principal component analysis of each modality, including:

[0014] After performing preset data enhancement processing on the surface data to be extracted hyperspectral image, a target enhanced image is obtained;

[0015] The target enhanced image is input into a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork for principal component analysis and segmented principal component analysis, respectively, to obtain a first principal component analysis image and the first segmented principal component analysis image.

[0016] According to a surface ecological data extraction method provided by the present invention, each convolution subnetwork in the preset semantic segmentation subnetwork is connected to a spatial channel joint attention correction subnetwork, and each principal component analysis image is input into the semantic segmentation subnetwork to perform semantic segmentation and spatial channel joint attention correction respectively, and each predicted surface data distribution map corresponding to each principal component analysis image is obtained respectively, including:

[0017] After inputting each principal component analysis image into the preset semantic segmentation subnetwork for convolution, spatial channel joint attention correction, upsampling, downsampling and jump connection, each predicted surface data distribution map corresponding to each principal component analysis image can be obtained respectively.

[0018] According to a surface ecological data extraction method provided by the present invention, the training process of the preset surface ecological data extraction network includes:

[0019] Acquire training hyperspectral images and validation hyperspectral images;

[0020] The preset initial surface ecological data extraction network is trained according to the training hyperspectral image, and the trained network is verified according to the verification hyperspectral image to obtain the preset surface ecological data extraction network.

[0021] According to a surface ecological data extraction method provided by the present invention, a preset initial surface ecological data extraction network is trained according to the training hyperspectral image, and the trained network is verified according to the verification hyperspectral image to obtain the preset surface ecological data extraction network, including:

[0022] Performing a preset number of rounds of iterative training on a preset initial surface ecological data extraction network based on the training hyperspectral image, and obtaining an intermediate surface ecological data extraction network obtained after this round of iterative training;

[0023] Verifying the intermediate surface ecological data extraction network using the verification hyperspectral image, and determining whether the parameter accuracy of the intermediate surface ecological data extraction network meets the preset accuracy requirements;

[0024] If the parameter accuracy meets the preset accuracy requirement, the intermediate surface ecological data extraction network obtained after this round of iterative training is used as the preset surface ecological data extraction network;

[0025] If the parameter accuracy does not meet the preset accuracy requirement, the intermediate surface ecological data extraction network is trained using the training hyperspectral image to obtain the preset surface ecological data extraction network.

[0026] The present invention also provides a surface ecological data extraction device, comprising:

[0027] An acquisition module is used to acquire a hyperspectral image to be extracted from the surface data, wherein the hyperspectral image to be extracted from the surface data includes hyperspectral images taken for different ground objects on the same surface area;

[0028] A determination module is used to input the hyperspectral image to be extracted from the surface data into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map; wherein, the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image to be extracted from the surface data, and to obtain the target surface ecological data distribution map after performing semantic segmentation and attention mechanism correction on the processed hyperspectral image.

[0029] The present invention also provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described surface ecological data extraction methods are implemented.

[0030] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for extracting surface ecological data.

[0031] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for extracting surface ecological data.

[0032] The present invention provides a surface ecological data extraction method, apparatus, device, and storage medium. The surface ecological data extraction method first uses hyperspectral images captured of different objects in the same surface area as hyperspectral images of surface data to be extracted. The hyperspectral images of the surface data to be extracted are then input into a preset surface ecological data extraction network for data extraction to obtain a target surface ecological data distribution map. Because the preset surface ecological data extraction network performs multimodal principal component analysis on the hyperspectral images of the surface data to be extracted, and semantic segmentation and attention mechanism correction are performed on the processed hyperspectral images to obtain the target surface ecological data distribution map, the method can accurately and reliably extract surface ecological data without high model complexity. This not only reduces the severe overfitting phenomenon caused by high model complexity, but also significantly improves the accuracy of surface ecological data extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 It is a schematic flow chart of the surface ecological data extraction method provided by the present invention;

[0035] Figure 2 This is a schematic diagram of the training process of the preset surface ecological data extraction network provided by the present invention;

[0036] Figure 3 It is a schematic diagram of the test spectrum image, ground truth image and network model prediction result provided by the present invention;

[0037] Figure 4It is a structural schematic diagram of the surface ecological data extraction device provided by the present invention;

[0038] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0039] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0040] Surface ecological and environmental issues are a major concern for humanity. The spatial distribution and temporal variations of diverse surface ecological data within a region can provide strong support for analyzing and resolving these issues. Extracting typical surface ecological data (such as buildings, vegetation, water bodies, and bare soil) not only significantly aids in strengthening ecological and environmental protection and achieving sustainable regional development, but also holds significant application prospects in ecological and environmental assessment research and the utilization and development of ecological functions.

[0041] Hyperspectral remote sensing images currently possess high spectral resolution, enabling them to distinguish subtle spectral features of different land objects. Due to this advantage, hyperspectral remote sensing has been successfully applied in many fields, such as land object classification (Kang et al., 2014; Lu et al., 2016), target detection (Nasrabadi, 2014; Liu et al., 2017), and spectral unmixing (Kizel et al., 2017). Among these applications, hyperspectral imagery land object classification has attracted considerable attention due to its importance in ecological and environmental assessment and monitoring.

[0042] Furthermore, the Hughes phenomenon, caused by the high-dimensional nature of hyperspectral data and the limited number of training samples, has always been a major challenge in hyperspectral image classification. To improve the accuracy of spectral image object classification, feature extraction or feature selection is often performed on hyperspectral images before classification. After the features are obtained, they need to be fed into a classifier for supervised learning to obtain the final classification model. Benediktsson et al. (2005) used principal component analysis (PCA) to extract multiple principal components from hyperspectral data, constructing a morphological profile (MP) for each principal component. These MPs were then combined into an extended morphological profile, which was then trained and classified using a neural network. Licciardi et al. (2012) conducted classification experiments using an extended MP constructed from features reduced by nonlinear principal component analysis (NLPCA) using an autoassociative neural network. They concluded that NLPCA achieved better classification accuracy than linear PCA. In addition, other scholars have also achieved good classification results by using dimensionality reduction methods such as independent component analysis (ICA) and singular spectrum analysis (SSA) and machine learning models such as support vector machine (SVM) (Villa et al., 2011; Zabalza et al., 2014).

[0043] With the rapid development of deep learning, many models represented by convolutional neural networks have begun to shine in the task of hyperspectral object classification. Zhao and Du (2016) proposed a classification framework based on spectral-spatial features. First, the balanced local discriminant embedding algorithm was used to extract spectral features. Secondly, the convolutional neural network was used to automatically extract high-level spatially correlated deep features. Finally, a multi-feature-based classifier was trained for image classification. Liu et al. (2017) developed an effective classification framework that combines deep belief networks (DBNs) with active learning. They designed a weighted incremental dictionary learning algorithm to actively select additional representative samples from unlabeled datasets, and DBN provided the final prediction. Gao Kuiliang et al. (2021) directly input hyperspectral high-dimensional data into the capsule network, combined the convolutional capsule layer and the residual structure to construct a deep network model, and finally introduced a three-dimensional convolutional capsule layer to fully utilize the spatial-spectral joint information in the image to further improve the classification accuracy.

[0044] However, while machine learning classification models have performed well, they are still unable to meet the growing demand for accuracy due to their inherent limitations. Deep learning models have improved the accuracy of the data in the paper, but they still suffer from relatively severe overfitting when faced with smaller sample data sets.

[0045] Based on the above problems, the present invention provides a surface ecological data extraction method. The execution subject of the surface ecological data extraction method can be a surface ecological data extraction device. The surface ecological data extraction device can be implemented as part or all of a terminal device through software, hardware, or a combination of software and hardware. Optionally, the terminal device can be a personal computer (PC), a portable device, a laptop, a smart phone, a tablet computer, a portable wearable device, or other electronic devices, such as a tablet computer, a mobile phone, etc. The present invention does not limit the specific form of the terminal device.

[0046] It should be noted that the execution subject of the following method embodiment may be part or all of the above-mentioned terminal device. The following method embodiment is described by taking the execution subject as the terminal device as an example.

[0047] Figure 1 The schematic diagram of the process of extracting surface ecological data provided by the present invention is as follows: Figure 1 As shown, the surface ecological data extraction method includes the following steps:

[0048] Step S110 : obtaining a hyperspectral image to be extracted from surface data, wherein the hyperspectral image to be extracted from the surface data includes hyperspectral images of different objects on the same surface area.

[0049] Specifically, the terminal device may receive a surface data extraction instruction sent by the client and obtain a plurality of hyperspectral images of surface data to be extracted according to the surface data extraction instruction. The terminal device may select multiple hyperspectral images of different objects in the same area from a pre-stored hyperspectral image set as the hyperspectral images of surface data to be extracted, or may activate an image acquisition device to capture multiple hyperspectral images of different objects in the same surface area as the hyperspectral images of surface data to be extracted. This is not specifically limited here.

[0050] Step S120: inputting the surface data hyperspectral image to be extracted into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map.

[0051] Among them, the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image to be extracted from the surface data, and to obtain the target surface ecological data distribution map after semantic segmentation and attention mechanism correction on the processed hyperspectral image.

[0052] Specifically, when the terminal device obtains the hyperspectral image of the surface data to be extracted, it can first perform principal component analysis processing of multiple modalities on the hyperspectral image of the surface data to be extracted according to the feature extraction algorithm and feature fusion processing algorithm of the pre-equipped device to obtain different feature images of different principal component features, and then perform semantic segmentation processing and attention mechanism correction processing on the different feature images respectively, so as to perform feature fusion of different scales of different principal component features, thereby obtaining the target surface ecological data distribution map corresponding to the hyperspectral image of the surface data to be extracted

[0053] It should be noted that, to improve extraction accuracy and precision, the feature extraction algorithm used in the present invention includes a combination of PCA and segmented PCA. This is because PCA uses orthogonal transformation to convert hyperspectral band data represented by linearly correlated variables into a small amount of "band" data represented by linear independent variables, eliminating the factors that influence the interaction between the original data components. Although data dimensionality reduction is achieved, it also changes the physical meaning of the image spectral bands, making the meaning of each principal component somewhat ambiguous. In addition, non-principal components with small variance may also contain important information that is beneficial for ground feature classification. Discarding them during dimensionality reduction may affect subsequent data processing. Therefore, segmented PCA processing is added. Segmented PCA continuously divides the original bands into several groups and then performs principal component transformation on each group separately to avoid missing important local bands during selection. This allows the classification and fusion of features obtained by PCA and segmented PCA to improve network classification accuracy. Optionally, the feature fusion processing algorithm can be a Unet++ segmentation network, with each convolutional subnetwork in the Unet++ segmentation network connected to an attention mechanism correction subnetwork.

[0054] The present invention provides a surface ecological data extraction method, apparatus, device, and storage medium. The surface ecological data extraction method first uses hyperspectral images captured of different objects in the same surface area as hyperspectral images of surface data to be extracted. The hyperspectral images of the surface data to be extracted are then input into a preset surface ecological data extraction network for data extraction to obtain a target surface ecological data distribution map. Because the preset surface ecological data extraction network performs multimodal principal component analysis on the hyperspectral images of the surface data to be extracted, and semantic segmentation and attention mechanism correction are performed on the processed hyperspectral images to obtain the target surface ecological data distribution map, the method can accurately and reliably extract surface ecological data without high model complexity. This not only reduces the severe overfitting phenomenon caused by high model complexity, but also significantly improves the accuracy of surface ecological data extraction.

[0055] Optionally, when the preset surface ecological data extraction network includes a preset multimodal principal component analysis subnetwork and a preset semantic segmentation subnetwork, step S120 can be implemented by the following process:

[0056] First, the hyperspectral images to be extracted from the surface data are respectively input into the preset multimodal principal component analysis subnetwork for multimodal principal component analysis to obtain each principal component analysis image after principal component analysis of each modality; then each principal component analysis image is input into the semantic segmentation subnetwork for semantic segmentation and spatial channel joint attention correction to obtain each predicted surface data distribution map corresponding to each principal component analysis image; finally, based on each predicted surface data distribution map, the target surface ecological data distribution map is obtained.

[0057] Specifically, when the terminal device obtains the hyperspectral image of the surface data to be extracted, it can first input the hyperspectral image of the surface data to be extracted into the preset multimodal principal component analysis subnetwork to perform principal component analysis of multiple modalities, that is, the preset multimodal principal component analysis subnetwork can simultaneously perform principal component analysis processing of different modalities on the hyperspectral image of the surface data to be extracted, such as performing PCA and segmented PCA on the hyperspectral image of the surface data to be extracted at the same time, so as to obtain each principal component analysis image after principal component analysis of each modality, and each principal component analysis image contains different component feature information with surface data attributes.

[0058] Furthermore, when the terminal device determines that the preset multimodal principal component analysis subnetwork outputs each principal component analysis image after the principal component analysis of each modality, each principal component analysis image can be further input into the preset semantic segmentation subnetwork for feature fusion processing at different levels and scales, so that the typical surface ecological data with obvious characteristics of each category and the typical surface ecological data with unclear characteristics can be accurately and precisely captured, thereby obtaining each predicted surface data distribution map; wherein, each predicted surface data distribution map includes the specific category information corresponding to each pixel and its probability.

[0059] Finally, each predicted surface data distribution map is fused and added together. For example, each predicted surface data distribution map includes a first predicted surface data distribution map and a second predicted surface data distribution map of the same size. In the first predicted surface data distribution map, the first pixel is marked as a water body with a probability of 60%, and the second pixel is marked as a non-water body with a probability of 40%. In the second predicted surface data distribution map, the first pixel is marked as a water body with a probability of 20%, and the second pixel is marked as a non-water body with a probability of 80%. In this case, the target surface ecological data distribution map obtained by fusing and adding the first and second predicted surface data distribution maps has the first pixel marked as a water body with a probability of 80%, and the second pixel marked as a non-water body with a probability of 120%. This results in a target surface ecological data distribution map with richer and more comprehensive typical surface ecological data.

[0060] The surface ecological data extraction method provided by the present invention extracts different component feature information with surface data attributes through a preset multimodal principal component analysis subnetwork with principal component analysis processing of different modalities, and fuses the different component feature information into a target surface ecological data distribution map with rich surface data categories and complete surface data through a preset semantic segmentation subnetwork with feature fusion functions of different levels and scales, thereby improving the extraction accuracy and reliability of surface ecological data.

[0061] Optionally, the preset multimodal principal component analysis subnetwork includes a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork, and the hyperspectral image to be extracted from the surface data is input into the preset multimodal principal component analysis subnetwork for multimodal principal component analysis to obtain each principal component analysis image after principal component analysis of each modality, including the following process:

[0062] After performing preset data enhancement processing on the hyperspectral images to be extracted from the surface data, target enhanced images are obtained; the target enhanced images are input into a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork for principal component analysis and segmented principal component analysis, respectively, to obtain a first principal component analysis image and the first segmented principal component analysis image.

[0063] Among them, the preset data enhancement processing includes channel enhancement and data enhancement.

[0064] Specifically, the terminal device first performs preset data enhancement processing on the hyperspectral image to be extracted from the surface data, that is, the channel enhancement purpose is achieved by selecting any band in the hyperspectral image to be extracted from the surface data for elimination processing, and then the data enhancement purpose is achieved by randomly performing horizontal mirroring, vertical mirroring and diagonal data enhancement on the hyperspectral image to be extracted from the eliminated surface data, so as to obtain the target enhanced image.

[0065] Then, PCA and segmented PCA are performed on the target enhanced image, wherein the process of performing PCA on the target enhanced image includes:

[0066] For a target enhanced image X consisting of n pixels and m bands, first calculate the covariance matrix C of the target enhanced image X, then find the n eigenvalues ​​of the covariance matrix C and the eigenvectors corresponding to each eigenvalue, and then arrange the n eigenvectors from large to small according to the size relationship between the corresponding eigenvalues, and then arrange them row by row to form the first matrix. Select the first k rows of the first matrix to form the second matrix P. The second matrix P includes the eigenvectors corresponding to the k largest eigenvalues. Then, the result of multiplying the second matrix P with the target enhanced image X is used as the first principal component analysis image after PCA processing. The calculation formula of the covariance matrix C is:

[0067]

[0068] Among them, x i represents the i-th pixel of the target enhanced image X, i = 1, 2, ... n, X j represents the j-th band of the target enhanced image X, the superscript T represents the transposition operation, ∑(·) represents the summation operation, and j = 1, 2, …, m.

[0069] The process of performing segmented PCA on the target enhanced image X includes:

[0070] For a target enhanced image X consisting of n pixels and m bands, first assume that each of the m bands consists of n′ pixels, i′∈[1,n′], and then calculate the Pearson product-moment correlation coefficient r between each two adjacent bands in the target enhanced image X j，j+1 , and its calculation formula is:

[0071]

[0072] Among them, r j，j+1 represents the Pearson product-moment correlation coefficient between the jth band and the j+1th band, j and j+1 represent two adjacent bands in the m bands, and the initial value of j is 1; each band in the m bands includes n′ pixels, i′∈[1,n′]; A ji′ represents the pixel value of the i′th pixel in the jth band, represents the mean value of the jth band, B (j+1)i′ represents the pixel value of the i′th pixel in the j+1th band, Represents the mean of the j+1th band, j∈[1,m].

[0073] Furthermore, the Pearson product-moment correlation coefficient r between each two adjacent bands is determined. j，j+1Is it greater than the preset coefficient threshold T? j，j+1 >T, the jth band is stored in the segment index linked list, and the value of j is increased by 1, and the Pearson product-moment correlation coefficient between the next two adjacent bands and the size judgment of the coefficient threshold T are continued until m bands are traversed. At this time, the target enhanced image X is divided into a hyperspectral image F containing D groups of bands using each band stored in the segment index linked list; for example, when the segment index linked list contains the 1st band, the 3rd band, the 7th band, and the 11th band, the hyperspectral image can be divided into 4 groups. At this time, it can be considered that the hyperspectral image F includes the 1st-2nd band, the 3rd-6th band, the 7th-10th band, and the 11th-mth band.

[0074] Then, each group of bands in the hyperspectral image F is further subjected to PCA processing. The PCA processing process is the same as the above process, that is, the eigenvectors of the covariance matrix of each group of bands are first sorted from large to small according to the size relationship between the corresponding eigenvalues, and then arranged in rows to form a first matrix, and then the first k rows of the first matrix are respectively formed into a second matrix, and the second matrix includes the eigenvectors corresponding to the k largest eigenvalues. Then, the result of multiplying each second matrix with the corresponding group of bands is obtained to obtain D PCA processing results. Finally, the D PCA processing results are superimposed (such as channel fusion processing), and the superimposed result is used as the first segmented principal component analysis image obtained after segmented PCA processing.

[0075] It should be noted that since the surface data to be extracted hyperspectral images are usually acquired for different land features in a certain surface area with surface ecological and environmental problems, and the surfaces with surface ecological and environmental problems are mostly characterized by land quality degradation, sharp decline in wetlands and biodiversity, decline in surface water quality and groundwater levels, and prominent conflicts between people and land, it is necessary to enhance the acquired surface data to be extracted hyperspectral images to avoid the disadvantage of network overfitting caused by insufficient data; and when it is determined that the surface data to be extracted hyperspectral images have fuzzy bands, the fuzzy bands should also be removed. This will improve the detection accuracy of surface data to be extracted hyperspectral images in areas with surface ecological and environmental problems.

[0076] In addition, if the surface data to be extracted hyperspectral image contains b fuzzy bands, the mth band is the last band in the remaining bands after removing the b fuzzy bands in the surface data to be extracted hyperspectral image and performing channel enhancement processing; conversely, if the data to be extracted hyperspectral image does not contain fuzzy bands, the mth band is the last band in the remaining bands after performing channel enhancement processing on the surface data to be extracted hyperspectral image.

[0077] The surface ecological data extraction method provided by the present invention avoids network overfitting while improving the classification balance of surface data by first performing preset enhancement processing on the hyperspectral image to be extracted, and then performing principal component analysis and segmented principal component analysis processing on the surface data, thereby providing a reliable basis for subsequently obtaining a high-precision target surface ecological data distribution map.

[0078] Optionally, each convolution subnetwork in the preset semantic segmentation subnetwork is connected to a spatial channel joint attention correction subnetwork, and each principal component analysis image is input into the semantic segmentation subnetwork to perform semantic segmentation and spatial channel joint attention correction respectively, and each predicted surface data distribution map corresponding to each principal component analysis image is obtained respectively, including:

[0079] After inputting each principal component analysis image into the preset semantic segmentation subnetwork for convolution, spatial channel joint attention correction, upsampling, downsampling and jump connection, each predicted surface data distribution map corresponding to each principal component analysis image can be obtained respectively.

[0080] Specifically, the preset semantic segmentation subnetwork includes a preset Unet++ subnetwork and a preset scSE subnetwork. The preset Unet++ subnetwork is a network trained on the initial Unet++ subnetwork, and the preset scSE subnetwork is a network trained on the initial scSE subnetwork. The Unet++ subnetwork retains each convolutional subnetwork, each downsampling subnetwork, each upsampling subnetwork and each jump connection subnetwork, and also includes multiple scSE subnetworks. Each convolutional subnetwork is connected to an scSE subnetwork, and the number of convolutional subnetworks and scSE subnetworks is the same.

[0081] Therefore, when each principal component analysis image is input into the preset semantic segmentation subnetwork, the preset convolution subnetwork performs convolution calculations on the principal component analysis image to obtain a convolved feature map. The scSE subnetwork performs spatial and channel correction on the feature map output by the corresponding preset convolution subnetwork to obtain a corrected feature map. The preset downsampling subnetwork performs a pooling operation on the input feature map to reduce the size of the feature map to obtain a reduced-size feature map. The preset upsampling subnetwork doubles the size of the input feature map to obtain a doubled-size feature map. The preset skip connection subnetwork fuses feature maps of different scales. Finally, the last preset scSE subnetwork in the preset semantic segmentation subnetwork performs spatial and channel corrections to output each predicted surface data distribution map corresponding to each principal component analysis image. For example, when each principal component analysis image includes a first principal component analysis image and a first segmented principal component analysis image, each corresponding predicted surface data distribution map can include a first principal component predicted surface data distribution map and a first segmented predicted surface data distribution map, respectively.

[0082] It should be noted that the preset scSE subnetwork performs spatial and channel correction on the feature maps output by the preset convolution subnetwork; the preset jump connection subnetwork fuses feature maps of different scales to obtain a fused feature map, and the scale of the feature map obtained after each fusion is different; feature maps of different scales can be obtained after multiple upsampling, multiple downsampling, multiple fusions, or multiple spatial and channel corrections, and the more times upsampling is performed, the greater the depth of the scale will be as the number of processing operations increases, and the depth of the current feature map is greater than the depth of the previous feature map. For example, the scale of the feature map obtained after the second convolution operation is deeper than the scale of the feature map obtained after the first convolution operation, and the scale of the feature map obtained after the fourth fusion is deeper than the scale of the feature map obtained after the third fusion.

[0083] The surface ecological data extraction method provided by the present invention uses a preset semantic segmentation subnetwork to perform convolution, spatial channel joint attention correction, upsampling, downsampling and jump connection on each principal component analysis image, thereby achieving the purpose of fusing different feature information extracted by multimodal principal component analysis at different scales and depths, thereby effectively improving the extraction accuracy and category richness of surface ecological data.

[0084] Optionally, the training process of the preset surface ecological data extraction network includes:

[0085] First, a training hyperspectral image and a verification hyperspectral image are obtained; then, a preset initial surface ecological data extraction network is trained according to the training hyperspectral image, and the trained network is verified according to the verification hyperspectral image to obtain the preset surface ecological data extraction network.

[0086] Specifically, M hyperspectral images of different objects on the same surface area are obtained in advance, each hyperspectral image includes K channels, and the M hyperspectral images contain an original remote sensing image and a labeled remote sensing image y. Each pixel point in the labeled remote sensing image y is marked with a corresponding category value. For example, when a pixel in the labeled remote sensing image y is a tree and the vegetation category value is preset to 1, the pixel can be marked as 1; the M hyperspectral images are divided into a training image set consisting of M1 training hyperspectral images (for example, M1 takes a value of 72) and a verification image set consisting of M2 verification hyperspectral images (for example, M2 can take a value of 18) according to a preset ratio (preferably 4:1).

[0087] Furthermore, M1 training hyperspectral images are used to train the preset initial surface ecological data extraction network, wherein the preset initial surface ecological data extraction network includes an initial multimodal principal component analysis subnetwork and an initial semantic segmentation subnetwork, the initial semantic segmentation subnetwork includes an initial Unet++ subnetwork and an initial scSE subnetwork, and the initial multimodal principal component analysis subnetwork includes an initial principal component analysis subnetwork and an initial segmented principal component analysis subnetwork. When the preset initial surface ecological data extraction network is trained to convergence, a preset surface ecological data extraction network including a preset multimodal principal component analysis subnetwork and a preset semantic segmentation subnetwork can be obtained.

[0088] The surface ecological data extraction method provided by the present invention achieves the purpose of quickly obtaining a preset surface ecological data extraction network by using training hyperspectral images to train the initial surface ecological data extraction network and using verification hyperspectral images to verify the trained network. It can not only utilize the high spectral resolution of hyperspectral images to achieve the purpose of distinguishing the subtle spectral characteristics of different land objects, but also can reduce the complexity of network training and select appropriate training samples based on a combination of training and verification. It greatly reduces the overfitting phenomenon and improves the training speed and accuracy of the network, laying the foundation for the subsequent rapid and accurate extraction of surface ecological data.

[0089] Optionally, the training of a preset initial surface ecological data extraction network based on the training hyperspectral image, and the verification of the trained network based on the verification hyperspectral image to obtain the preset surface ecological data extraction network include:

[0090] First, the preset initial surface ecological data extraction network is iteratively trained for a preset number of rounds based on the training hyperspectral image to obtain an intermediate surface ecological data extraction network obtained after this round of iterative training; then the intermediate surface ecological data extraction network is verified using the verification hyperspectral image, and it is determined whether the parameter accuracy of the intermediate surface ecological data extraction network meets the preset accuracy requirements; if the parameter accuracy meets the preset accuracy requirements, the intermediate surface ecological data extraction network obtained after this round of iterative training is used as the preset surface ecological data extraction network; if the parameter accuracy does not meet the preset accuracy requirements, the intermediate surface ecological data extraction network is trained using the training hyperspectral image to obtain the preset surface ecological data extraction network.

[0091] Specifically, the terminal device uses M1 training hyperspectral images to train a preset initial surface ecological data extraction network. After all M1 training hyperspectral images train the initial surface ecological data extraction network once, it is called a round of training. In addition, the verification process can be performed after multiple rounds of training for the initial surface ecological data extraction network, or after one round of training. Preferably, the verification process is performed after one round of training.

[0092] After each round of training using M1 training hyperspectral images, the terminal device will obtain the intermediate network after this round of training, and further use M2 verification hyperspectral images to verify the intermediate network after this round of training to determine whether the network parameter accuracy of the intermediate network after this round of training meets the preset accuracy requirements. When the network parameter accuracy of the intermediate network after this round of training meets the preset accuracy requirements, the training is stopped, and the corresponding intermediate network when the training stops is determined to be the preset surface ecological data extraction network; on the contrary, if the intermediate network after this round of training does not meet the preset accuracy requirements, the M1 training hyperspectral images are used again to perform the number of rounds for the intermediate network after this round of training, until the network parameter accuracy of the network after verification training using M2 verification hyperspectral images meets the preset accuracy requirements, and the preset surface ecological data extraction network is obtained.

[0093] In addition, for each training hyperspectral image, each training hyperspectral image participates in one training in each round of training, and a loss function is calculated after each training hyperspectral image participates in one training. The process of each training hyperspectral image participating in one training and the calculation process of the corresponding loss function include:

[0094] like Figure 2 As shown in the figure, the training hyperspectral image is processed with PCA and segmented PCA respectively to obtain the first principal component analysis training image after PCA processing and the first segmented principal component analysis training image after segmented PCA. After the first principal component analysis training image and the first segmented principal component analysis training image are input into the initial semantic segmentation subnetwork respectively, the first principal component predicted surface data training distribution map corresponding to the first principal component analysis training image and the first segmented predicted surface data training distribution map corresponding to the first segmented principal component analysis training image are output respectively.

[0095] Before performing PCA and segmented PCA on the training hyperspectral image, the training hyperspectral image is first subjected to blur band removal. Then, one channel of the training hyperspectral image after noise band removal is optionally removed to obtain an offline enhanced training image. The offline enhanced training image is then subjected to random horizontal and vertical mirroring and diagonal data enhancement to obtain the target enhanced training image. If blur bands exist in the training hyperspectral image, the removal process is performed first and then the enhancement process is performed. If blur bands do not exist, the removal process is not performed and the enhancement process is performed directly.

[0096] Then, the process of performing PCA on the target enhanced training image includes: for the target enhanced training image Y including n pixels and m bands, calculating the training covariance matrix of the target enhanced training image Y, then calculating the n eigenvalues ​​of the training covariance matrix and the eigenvector corresponding to each eigenvalue, arranging the n eigenvectors from large to small according to the corresponding eigenvalue size and arranging them by row to form a first training matrix, taking the first k rows of the first training matrix to form a second training matrix, the second training matrix includes the eigenvectors corresponding to the k largest eigenvalues, and then multiplying the second training matrix by the target enhanced training image Y as the first principal component analysis training image after PCA processing.

[0097] The process of performing segmented PCA on the target enhanced training image Y includes: for the target enhanced training image X including n pixels and m bands, each of the m bands includes n' pixels, i'∈[1,n'], calculating the Pearson product moment correlation coefficient r between each adjacent two bands in the target enhanced training image Y j',j'+1 , and further determine when r j',j'+1 When ≥T, the j'th band is stored in the segment index linked list, the value of j' is incremented by 1, and the Pearson product-moment correlation coefficient between the next two adjacent bands is calculated and compared with the threshold T until m bands are traversed, j'∈[1,m]. At this time, the target enhanced training image Y is divided into a hyperspectral training image consisting of D groups of bands using each band stored in the segment index linked list. PCA processing is then performed on each group of bands in the hyperspectral training image. The PCA processing process is the same as the above process, thereby obtaining D PCA processing training results. The D PCA processing training results are then superimposed (for example, channel fusion processing). The superimposed result can be used as the first segment principal component analysis training image after segment PCA. The PCA processing training results of each group of bands in the hyperspectral training image include one band, and the first segment principal component analysis training image after superimposition includes D bands.

[0098] The process of inputting the first principal component analysis training image and the first segmented principal component analysis training image into the initial semantic segmentation subnetwork for semantic segmentation processing includes:

[0099] like Figure 2As shown in the figure, the triangular mesh structure in the middle part is the initial semantic segmentation subnetwork, each upward arrow represents the initial upsampling subnetwork, each downward arrow represents the initial downsampling subnetwork, each dotted line represents the initial jump connection subnetwork, each circle represents the initial convolution subnetwork, and the small square connected after each circle represents the initial scSE subnetwork. Therefore, when the first principal component analysis training image and the first segmented principal component analysis training image enter the initial semantic segmentation subnetwork, they will undergo convolution, downsampling, spatial channel joint attention correction, upsampling and jump connection processing. Finally, after the spatial channel joint attention correction is performed by the last initial scSE subnetwork, the first principal component predicted surface data training distribution map corresponding to the first principal component analysis training image and the first segmented predicted surface data training distribution map corresponding to the first segmented principal component analysis training image are output respectively.

[0100] Furthermore, the first principal component predicted surface data training distribution map and the first segmented predicted surface data training distribution map respectively include specific category information and its probability corresponding to each pixel. For example, in the first principal component predicted surface data training distribution map, the first pixel is marked as bare soil with a probability of 40%, and the second pixel is marked as non-bare soil with a probability of 70%. In the first segmented predicted surface data training distribution map, the first pixel is marked as bare soil with a probability of 50%, and the second pixel is marked as non-bare soil with a probability of 80%. The first principal component predicted surface data training distribution map and the first segmented predicted surface data training distribution map are then fused and added to obtain a training surface ecological data distribution map P obtained after the training hyperspectral image is trained. In the training surface ecological data distribution map P, the first pixel is marked as bare soil with a probability of 90%, and the second pixel is marked as non-bare soil with a probability of 150%.

[0101] Then, the label remote sensing image y is used to calculate the loss function Loss of the training surface ecological data distribution map P ce , the formula is:

[0102]

[0103] Where N represents the number of pixels in the labeled remote sensing image y and the training surface ecological data distribution map P, respectively; K represents the number of preset categories (for example, when including building, vegetation, water, and bare soil categories, K is 4). The cth value in the one-hot encoding representing the actual category of the i'th pixel in the training surface ecological data distribution map P (for example, when the actual category of the i'th pixel is vegetation and is 1, the first bit in the one-hot encoding is 1 and the rest are 0. In this case, c is 1 and refers to the vegetation category); Represents the predicted probability value of category c at the i'th pixel in the labeled remote sensing image y (for example, when c is 1, it refers to the predicted probability value of the vegetation class). Optionally, the joint function of soft cross entropy and Dice, Lossseg, can also be used as the loss function:

[0104]

[0105] Among them, TP is the positive class that is accurately classified, FP is the negative class that is misclassified as the positive class, and FN is the positive class that is misclassified as the negative class.

[0106] Based on the above-mentioned process of training hyperspectral images participating in one training and the calculation process of the corresponding loss function, the training process and loss function calculation operations of each training hyperspectral image can be executed, and the corresponding parameters of the network after this training can be adaptively adjusted according to the value of the loss function calculated each time, so that the number of training rounds is as small as possible and the network training accuracy is higher.

[0107] The surface ecological data extraction method provided by the present invention achieves the purpose of quickly and efficiently obtaining a preset surface ecological data extraction network by training a preset initial surface ecological data extraction network with hyperspectral images for a preset number of rounds of training, and using verification hyperspectral images to verify the accuracy of the intermediate surface ecological data extraction network obtained after iterative training. This not only ensures the network accuracy, but also greatly reduces the training complexity and speeds up the network training speed.

[0108] In actual processing, to verify the effectiveness of the proposed method, a training image set consisting of 72 training hyperspectral images was constructed by dividing M hyperspectral images into a 4:1 ratio. Spectral comparative analysis revealed that the hyperspectral images consist of 32 bands, and that the first and 32nd bands exhibited significant noise relative to the other bands. This noise is detrimental to healthy network learning, so the proposed method eliminated these two bands. Furthermore, one channel was selected from the remaining 30 channels after the band elimination operation, and a multimodal principal component analysis was then performed on the remaining 29 channels.

[0109] To alleviate the class imbalance, we performed a multimodal principal component analysis (MPCA) transformation with a probability of 1 on images with a bare soil ratio greater than 1% and a background ratio less than 50%, and a multimodal principal component analysis transformation with a probability of 1 / 2 on all other images. This yielded approximately 72 + 5 * 30 + 67 * 15 = 1092 images. Furthermore, during iterative network training, we used the AdamW optimization algorithm with an initial learning rate of 0.001 and a weight decay of 0.001, performing 100 backpropagation iterations to optimize the network parameters layer by layer. The network parameters obtained after the optimal number of rounds of Intersection over Union (IoU) of the validation images were saved.

[0110] In order to quantitatively describe the ecological data extraction accuracy of the model, this paper uses three indicators: Kappa coefficient, intersection over union (IoU), and mean intersection over union (mIoU) to evaluate the performance of the results. The Kappa coefficient represents the ratio of error reduction of the extraction results obtained by the network compared with the results obtained by completely random classification. IoU is expressed as the ratio of the intersection and union of the actual positive samples and the predicted positive samples. mIoU is the average value of the IoU of each category. The calculation formulas of each indicator are as follows:

[0111]

[0112] Among them, P o It represents the number of correctly classified pixels divided by the total number of pixels, P e It represents the quotient of the sum of the products of the actual and predicted number of samples of each category and the square of the total number of pixels. TP represents the accurately classified positive class, FN represents the positive class that is misclassified as the negative class, FP represents the negative class that is misclassified as the positive class, and K represents the preset number of categories.

[0113] The accuracy of the surface ecological data evaluation using the present invention is shown in Table 1. Among the four ecological data, the water body has the highest IoU of 0.833, followed by the vegetation IoU of 0.827, the building IoU of 0.781, and the bare soil IoU of 0.507 ranks last. The overall Kappa value of the network reaches 0.822, and the mIoU is 0.737. Figure 3 A schematic diagram of the network model prediction results of the test spectral image, the corresponding ground truth image and the network ecological element extraction results is given. Figure 3 In the figure, (a) is a schematic diagram of a column of test spectral images, (b) is a schematic diagram of a column of ground truth images, and (c) is a schematic diagram of a column of network model prediction results.

[0114] Table 1

[0115]

[0116] Through precision comparison analysis, it is proved that the extraction precision of surface ecological data using the preset surface ecological data extraction network in the method of the present invention is high and the effect is better.

[0117] The surface ecological data extraction device provided by the present invention is described below. The surface ecological data extraction device described below and the surface ecological data extraction method described above can be referenced to each other.

[0118] Figure 4 An example of a surface ecological data extraction device is provided. Figure 4As shown, the surface ecological data extraction device 400 includes: an acquisition module 410, used to obtain the hyperspectral image of the surface data to be extracted, and the hyperspectral image of the surface data to be extracted includes hyperspectral images taken for different objects on the same surface area; a determination module 420, used to input the hyperspectral image of the surface data to be extracted into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map; wherein, the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image of the surface data to be extracted, and obtain the target surface ecological data distribution map after performing semantic segmentation and attention mechanism correction on the processed hyperspectral image.

[0119] Optionally, the determination module 420 can be specifically used to input the hyperspectral images to be extracted from the surface data into a preset multimodal principal component analysis subnetwork for multimodal principal component analysis, and obtain each principal component analysis image after principal component analysis of each modality; input each principal component analysis image into the semantic segmentation subnetwork for semantic segmentation and spatial channel joint attention correction, and obtain each predicted surface data distribution map corresponding to each principal component analysis image; based on each predicted surface data distribution map, obtain the target surface ecological data distribution map.

[0120] Optionally, the determination module 420 can also be used to perform preset data enhancement processing on the hyperspectral image to be extracted from the surface data to obtain a target enhanced image; the target enhanced image is input into a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork to perform principal component analysis and segmented principal component analysis respectively to obtain a first principal component analysis image and the first segmented principal component analysis image.

[0121] Optionally, the determination module 420 can also be used to input each principal component analysis image into a preset semantic segmentation subnetwork for convolution, spatial channel joint attention correction, upsampling, downsampling and jump connection, so as to obtain each predicted surface data distribution map corresponding to each principal component analysis image.

[0122] Optionally, the acquisition module 410 may be further configured to acquire training hyperspectral images and verification hyperspectral images.

[0123] Optionally, the determination module 420 can also be used to train a preset initial surface ecological data extraction network based on the training hyperspectral image, and verify the trained network based on the verification hyperspectral image to obtain the preset surface ecological data extraction network.

[0124] Optionally, the determination module 420 can also be specifically used to perform a preset number of rounds of iterative training on the preset initial surface ecological data extraction network based on the training hyperspectral image to obtain an intermediate surface ecological data extraction network obtained after this round of iterative training; use the verification hyperspectral image to verify the intermediate surface ecological data extraction network, and judge whether the parameter accuracy of the intermediate surface ecological data extraction network meets the preset accuracy requirements; if the parameter accuracy meets the preset accuracy requirements, the intermediate surface ecological data extraction network obtained after this round of iterative training is used as the preset surface ecological data extraction network; if the parameter accuracy does not meet the preset accuracy requirements, use the training hyperspectral image to train the intermediate surface ecological data extraction network to obtain the preset surface ecological data extraction network.

[0125] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device 500 may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other via the communications bus 540. The processor 510 may call logic instructions in the memory 530 to execute a surface ecological data extraction method, which includes: obtaining a hyperspectral image of surface data to be extracted, wherein the hyperspectral image of surface data to be extracted includes hyperspectral images of different land features in the same surface area; inputting the hyperspectral image of surface data to be extracted into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map; wherein the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image of surface data to be extracted, and performing semantic segmentation and attention mechanism correction on the processed hyperspectral image to obtain the target surface ecological data distribution map.

[0126] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0127] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the surface ecological data extraction method provided by the above methods, which includes: obtaining a hyperspectral image of surface data to be extracted, and the hyperspectral image of surface data to be extracted includes hyperspectral images of different objects on the same surface area; inputting the hyperspectral image of surface data to be extracted into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map; wherein the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image of surface data to be extracted, and obtain the target surface ecological data distribution map after performing semantic segmentation and attention mechanism correction on the processed hyperspectral image.

[0128] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the surface ecological data extraction method provided by the above-mentioned methods, the method comprising: obtaining a hyperspectral image of surface data to be extracted, the hyperspectral image of surface data to be extracted comprising hyperspectral images of different objects on the same surface area; inputting the hyperspectral image of surface data to be extracted into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map; wherein the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image of surface data to be extracted, and to obtain the target surface ecological data distribution map after performing semantic segmentation and attention mechanism correction on the processed hyperspectral image.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for extracting surface ecological data, characterized in that: include: Acquire a hyperspectral image to be extracted from surface data, wherein the hyperspectral image to be extracted from surface data includes hyperspectral images of different objects on the same surface area; The hyperspectral image to be extracted from the surface data is input into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map; wherein the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image to be extracted from the surface data, and to perform semantic segmentation and attention mechanism correction on the processed hyperspectral image to obtain the target surface ecological data distribution map; The preset surface ecological data extraction network includes a preset multimodal principal component analysis subnetwork and a preset semantic segmentation subnetwork. The hyperspectral image of the surface data to be extracted is input into the preset surface ecological data extraction network to obtain a target surface ecological data distribution map, including: The hyperspectral images to be extracted from the surface data are respectively input into a preset multimodal principal component analysis subnetwork for multimodal principal component analysis to obtain each principal component analysis image after principal component analysis of each modality; Input each principal component analysis image into the semantic segmentation subnetwork to perform semantic segmentation and spatial channel joint attention correction respectively, and obtain each predicted surface data distribution map corresponding to each principal component analysis image respectively; Based on each predicted surface data distribution map, a target surface ecological data distribution map is obtained; The preset multimodal principal component analysis subnetwork includes a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork. The surface data to be extracted hyperspectral images are respectively input into the preset multimodal principal component analysis subnetwork for multimodal principal component analysis to obtain each principal component analysis image after each modal principal component analysis, including: After performing preset data enhancement processing on the surface data to be extracted hyperspectral images, target enhanced images are obtained; Inputting the target enhanced image into a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork to perform principal component analysis and segmented principal component analysis respectively, to obtain a first principal component analysis image and a first segmented principal component analysis image; Each convolution subnetwork in the preset semantic segmentation subnetwork is respectively connected to a spatial channel joint attention correction subnetwork, and each principal component analysis image is input into the semantic segmentation subnetwork to perform semantic segmentation and spatial channel joint attention correction respectively, and each predicted surface data distribution map corresponding to each principal component analysis image is obtained respectively, including: After inputting each principal component analysis image into the preset semantic segmentation subnetwork for convolution, spatial channel joint attention correction, upsampling, downsampling and jump connection, each predicted surface data distribution map corresponding to each principal component analysis image can be obtained respectively.

2. The method for extracting surface ecological data according to claim 1, characterized in that: The training process of the preset surface ecological data extraction network includes: Acquire training hyperspectral images and validation hyperspectral images; The preset initial surface ecological data extraction network is trained according to the training hyperspectral image, and the trained network is verified according to the verification hyperspectral image to obtain the preset surface ecological data extraction network.

3. The method for extracting surface ecological data according to claim 2, characterized in that: The step of training a preset initial surface ecological data extraction network according to the training hyperspectral image, and verifying the trained network according to the verification hyperspectral image to obtain the preset surface ecological data extraction network includes: Performing a preset number of rounds of iterative training on a preset initial surface ecological data extraction network according to the training hyperspectral image, and obtaining an intermediate surface ecological data extraction network obtained after this round of iterative training; Using the verification hyperspectral image to verify the intermediate surface ecological data extraction network, and determining whether the parameter accuracy of the intermediate surface ecological data extraction network meets the preset accuracy requirements; If the parameter accuracy reaches the preset accuracy requirement, the intermediate surface ecological data extraction network obtained after this round of iterative training is used as the preset surface ecological data extraction network; If the parameter accuracy does not meet the preset accuracy requirement, the intermediate surface ecological data extraction network is trained using the training hyperspectral image to obtain the preset surface ecological data extraction network.

4. A surface ecological data extraction device, characterized in that: include: An acquisition module is used to acquire a hyperspectral image to be extracted from surface data, wherein the hyperspectral image to be extracted from surface data includes hyperspectral images taken for different objects on the same surface area; A determination module is used to input the hyperspectral image to be extracted from the surface data into a preset surface ecological data extraction network to obtain a target surface ecological data distribution map; wherein the preset surface ecological data extraction network is used to perform multimodal principal component analysis on the hyperspectral image to be extracted from the surface data, and to perform semantic segmentation and attention mechanism correction on the processed hyperspectral image to obtain the target surface ecological data distribution map; The preset surface ecological data extraction network includes a preset multimodal principal component analysis subnetwork and a preset semantic segmentation subnetwork. The determination module is used to input the hyperspectral image to be extracted from the surface data into the preset surface ecological data extraction network to obtain a target surface ecological data distribution map, including: The hyperspectral images to be extracted from the surface data are respectively input into a preset multimodal principal component analysis subnetwork for multimodal principal component analysis to obtain each principal component analysis image after principal component analysis of each modality; Input each principal component analysis image into the semantic segmentation subnetwork to perform semantic segmentation and spatial channel joint attention correction respectively, and obtain each predicted surface data distribution map corresponding to each principal component analysis image respectively; Based on each predicted surface data distribution map, a target surface ecological data distribution map is obtained; The preset multimodal principal component analysis subnetwork includes a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork. The determination module is used to input the surface data to be extracted hyperspectral images into the preset multimodal principal component analysis subnetwork for multimodal principal component analysis, and obtain each principal component analysis image after each modal principal component analysis, including: After performing preset data enhancement processing on the surface data to be extracted hyperspectral images, target enhanced images are obtained; Inputting the target enhanced image into a preset principal component analysis subnetwork and a preset segmented principal component analysis subnetwork to perform principal component analysis and segmented principal component analysis respectively, to obtain a first principal component analysis image and a first segmented principal component analysis image; Each convolution subnetwork in the preset semantic segmentation subnetwork is respectively connected to a spatial channel joint attention correction subnetwork, and the determination module is used to input each principal component analysis image into the semantic segmentation subnetwork to perform semantic segmentation and spatial channel joint attention correction respectively, and obtain each predicted surface data distribution map corresponding to each principal component analysis image respectively, including: After inputting each principal component analysis image into the preset semantic segmentation subnetwork for convolution, spatial channel joint attention correction, upsampling, downsampling and jump connection, each predicted surface data distribution map corresponding to each principal component analysis image can be obtained respectively.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the surface ecological data extraction method as described in any one of claims 1 to 3 are implemented.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for extracting surface ecological data as described in any one of claims 1 to 3 are implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for extracting surface ecological data as described in any one of claims 1 to 3 are implemented.

Images