Regional vegetation type detection method and system

Abstract

The invention discloses a regional vegetation type detection method and system. The method comprises the steps that regional remote sensing image data are acquired and preprocessed; performing hierarchical feature extraction and fusion according to the preprocessed regional remote sensing image data to obtain fusion features; constructing a deep hierarchical semantic segmentation model coupled with vegetation type priori knowledge to input fusion features and output a vegetation type classification result; wherein the vegetation type priori knowledge is a hierarchical structure of the vegetation type. According to the technical scheme provided by the invention, features can be extracted from the angles of space, channels and scales based on the attention mechanism, differential features of remote sensing image data can be extracted from multiple dimensions, and the classification effect is improved; by constructing the deep hierarchical semantic segmentation model coupled with vegetation type priori knowledge, the problem of unbalanced distribution of vegetation type samples can be overcome, the training speed is increased, the classification effect is optimized, and the detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method and system for detecting regional vegetation types. Background Art

[0002] Vegetation type detection is an indispensable part of forest resource investigation and monitoring. The comprehensive use of multi-source, multi-temporal and multi-spectral remote sensing data can obtain vegetation type information, which can provide an important scientific basis for the rational utilization and protection of my country's forest resources.

[0003] However, when using this type of data for vegetation type detection, the commonly used data fusion method is usually used to extract information, which often ignores the differentiated characteristics of multi-source, multi-temporal and multi-spectral remote sensing images. At the same time, due to the objective conditions of vegetation distribution and the limitations of the field survey environment, the classification accuracy is often reduced due to the uneven distribution of training samples.

[0004] In addition, when vegetation types are usually detected, they are usually roughly detected in large categories such as forests, shrubs, and grasses, lacking more fine-grained detection and classification. This general classification method can only assess the vegetation profile of a large area, but it is inadequate for in-depth research on ecosystem health, biodiversity conservation, and precision agricultural management. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the current technology, the present invention provides a method for detecting regional vegetation types. By hierarchically extracting and fusing the features of remote sensing image data, a multi-level semantic segmentation model coupled with prior knowledge of vegetation types is constructed, which can optimize the classification effect and improve the detection accuracy.

[0006] To achieve the above object, the embodiments of the present invention adopt the following technical solutions: A method for detecting regional vegetation types comprises the following steps: Acquire regional remote sensing image data and perform preprocessing; According to the pre-processed regional remote sensing image data, hierarchical feature extraction and fusion are performed to obtain fusion features; A deep hierarchical semantic segmentation model coupled with vegetation type prior knowledge is constructed to input fusion features and output vegetation type classification results; Among them, the prior knowledge of vegetation types is the hierarchical structure of vegetation types.

[0007] According to one aspect of the present invention, the step of extracting and fusing hierarchical features to obtain fused features based on the preprocessed regional remote sensing image data includes: A spatiotemporal feature extraction module is constructed based on joint spatial attention and channel attention to extract spatiotemporal features of the preprocessed regional remote sensing image data.

[0008] According to one aspect of the present invention, the step of extracting and fusing hierarchical features to obtain fused features based on the preprocessed regional remote sensing image data includes: A scale attention module is constructed to calculate the contribution weights of spatiotemporal features of different scales, and the spatiotemporal features of different image data are fused to obtain fused features.

[0009] According to one aspect of the present invention, the construction of a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types includes: Obtain the multi-level structure of vegetation types; Based on the fusion features and the number of vegetation type levels N, an N-level semantic segmentation model coupled with prior knowledge of vegetation types was constructed using the U-Net model as the basic architecture.

[0010] According to one aspect of the present invention, the N-level semantic segmentation model includes: The previous layer of the N-layer semantic segmentation model serves as a constraint for the next layer; Each level of the N-level semantic segmentation model outputs the corresponding vegetation type result.

[0011] According to one aspect of the present invention, the N-level semantic segmentation model includes: The loss function of the N-level semantic segmentation model is obtained by considering the losses of all levels and weighted summing the losses of the coarse-grained type level and the fine-grained type level; Control the distribution of loss weights when training the model.

[0012] According to one aspect of the present invention, the step of constructing a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types to input fusion features and output vegetation type classification results further includes: Based on the hierarchical structure of vegetation types, the vegetation type classification results are cascaded and fused to build an optimization model to obtain optimized vegetation type detection results.

[0013] According to one aspect of the present invention, the regional vegetation type detection method further includes: The detection results are verified by combining samples collected on the ground, and the deep hierarchical semantic segmentation model is fine-tuned and improved.

[0014] According to one aspect of the present invention, the regional vegetation type detection method further includes: Based on the vegetation type detection results, a regional vegetation type distribution map is constructed.

[0015] A regional vegetation type detection system, characterized in that, based on the regional vegetation type detection method as described above, it comprises: Data acquisition module, used to acquire regional remote sensing image data and perform preprocessing; The feature extraction module is used to extract and fuse hierarchical features based on the pre-processed regional remote sensing image data to obtain fusion features; The detection module is used to build a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types to input fusion features and output vegetation type classification results.

[0016] Advantages of the present invention: The present invention provides a regional vegetation type detection method, which can extract features from the perspectives of space, channel and scale based on the attention mechanism, and can extract differentiated features of remote sensing image data from multiple dimensions to improve classification effect; by constructing a deep hierarchical semantic segmentation model coupled with vegetation type prior knowledge, it can overcome the problem of uneven distribution of vegetation type samples, improve training speed, optimize classification effect, and improve detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is a flow chart of a method for detecting regional vegetation types according to Embodiment 1 of the present invention; Figure 2 The process of the vegetation type detection method described in the present invention; Figure 3 The network structure of the spatiotemporal feature extraction module described in the present invention; Figure 4 It is the three-level semantic segmentation model described in the present invention; Figure 5 This is a flow chart of a method for detecting regional vegetation types according to Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Embodiment 1 like Figure 1 and 2As shown, a method for detecting regional vegetation types includes the following steps: S1: Obtain regional remote sensing image data and perform preprocessing.

[0021] In practical applications, the collected regional remote sensing image data are multi-temporal GF-1 WFV, GF-1 PMS and GF-2PMS images.

[0022] GF-1 WFV, GF-1 PMS and GF-2 PMS images refer to several different types of remote sensing image data acquired by Earth observation satellites launched by China. GF stands for the "Gaofen" series of satellites, which are high-resolution satellites that can acquire high-resolution surface images for a variety of scientific research and applications. Specifically, "GF-1" and "GF-2" stand for Gaofen-1 and Gaofen-2 satellites, respectively, while "WFV" and "PMS" stand for different types of sensors.

[0023] WFV is the abbreviation of "Wide Field of View". This sensor can cover a wider ground area and is suitable for large-scale environmental monitoring and resource surveys.

[0024] PMS is the abbreviation of "Panchromatic and Multispectral Camera". This sensor can acquire panchromatic and multispectral images. Panchromatic images have high resolution, while multispectral images can provide spectral information of ground objects, which is helpful for land cover classification and environmental monitoring.

[0025] In practical applications, the preprocessing methods for regional remote sensing image data include: noise removal, geometric correction, radiation calibration, atmospheric correction and image registration.

[0026] S2: Based on the preprocessed regional remote sensing image data, hierarchical feature extraction and fusion are performed to obtain fusion features.

[0027] Preferably, the feature extraction part of the method realizes hierarchical feature extraction and fusion by constructing a spatiotemporal feature extraction module and a scale attention module. In the method, the feature extraction part is also referred to as a hierarchical scale-sensitive attention spatiotemporal feature extraction model.

[0028] The attention mechanism in the field of computer vision can extract differentiated features from multiple dimensions. Based on the attention mechanism, features can be extracted from the perspectives of space, channel, and scale, which can enhance the differentiated features of remote sensing images and improve classification results.

[0029] 1. Spatiotemporal feature extraction module By constructing a spatiotemporal feature extraction module based on joint spatial attention and channel attention, spatiotemporal features are extracted from the preprocessed regional remote sensing image data.

[0030] Specifically, the network structure of the spatiotemporal feature extraction module is as follows: Figure 3 As shown, the input feature is A, and the final output feature is E. The dimensions of A and E are both c×h×w, where c is the number of channels, h is the number of pixels in the image height, and w is the number of pixels in the image width.

[0031] Preferably, the spatiotemporal feature extraction module includes: (1) Spatial attention module: considers the correlation between different locations in the image and extracts the spatial features of regional remote sensing image data.

[0032] like Figure 3 As shown in the figure, the spatial attention module inputs feature A, and generates three new feature maps B, F, and D respectively after convolution. The dimensions of B, F, and D are consistent with those of A and E, which are c×h×w. B and F generate feature S through operation. S is a 2h×w feature map, that is, the dimension is 2×h×w. Finally, after transportation, the output feature P is consistent with the dimension of E, which is also c×h×w. Specifically, the calculation method of the spatial attention model is as follows:

[0033]

[0034] Where i and j represent the i-th and j-th pixel positions of the image; α is the scale parameter, which is initially 0; A j is the input feature of the jth position; P j is the output feature of the jth position; S ji It is the core calculation step of the spatial attention model, which measures the correlation between the i-th position and the j-th position; M is the number of samples.

[0035] (2) Channel attention module: considers the correlation between different channels in the image and extracts the channel features of regional remote sensing image data.

[0036] Generally, the input image is an RGB image, which has three channels. However, remote sensing images are different from RGB images and have multiple channels, also called bands. High-resolution images generally have four bands, that is, c is 4; other sensors, such as Sentinel-2, have 11 bands, that is, c is 11. Remote sensing equipment can capture electromagnetic waves of different wavelength ranges, and each band corresponds to data of a specific wavelength range.

[0037] like Figure 3As shown in the figure, the channel attention module inputs feature A, generates feature X through intermediate calculation, and the dimension of X is c×c; finally, after calculation, it outputs feature Q, and the dimension of Q is consistent with the dimension of E, which is also c×h×w. Specifically, the calculation method of the channel attention module is as follows:

[0038]

[0039] Among them, i and j represent the i-th and j-th bands of the image; β is the weight parameter, which is initially 0; A j is the input feature of the jth position; Q j is the output feature of the jth position; X ji Measures the correlation between the i-th band and the j-th band.

[0040] (3) Add the output results of the spatial attention module and the channel attention module to obtain the final output feature E.

[0041] By integrating the processing results of the spatial attention module and the channel attention module, not only the spatial consistency of pixels of the same category is improved, but also the band distinguishability is enhanced, thus realizing the spatiotemporal feature extraction of remote sensing input images.

[0042] 2. Scale Attention Module By constructing a scale attention module, the contribution weights of spatiotemporal features of different scales are calculated, and the spatiotemporal features of different image data are fused to obtain fused features.

[0043] Specifically, by constructing a scale attention model, we learn the contribution weights of spatiotemporal features generated by remote sensing images with different spatial resolutions to the results, perform weighted summation of spatiotemporal features, and achieve spatiotemporal feature fusion. The weights are calculated as follows:

[0044] Among them, k∈{1,...,K}, represents the level of remote sensing images with different spatial resolutions. In the specific implementation, it includes 5 scale levels, K=5, k∈{1,...,5}; w k is the weight; h k 、h t It is the classification probability results at different scale levels.

[0045] S3: Construct a deep hierarchical semantic segmentation model coupled with vegetation type prior knowledge to input fusion features and output vegetation type classification results.

[0046] Among them, the prior knowledge of vegetation types is the hierarchical structure of vegetation types.

[0047] This method uses a semantic segmentation network to detect vegetation types. Before building the semantic segmentation network, it is necessary to classify the vegetation types first; then the remote sensing image data is annotated to construct a sample data set.

[0048] In forest ecosystems, traditional vegetation type detection only divides forests, shrubs, and grasses into broad categories, which is too general. For example, the broad category of "forest" alone cannot effectively distinguish between different types such as coniferous forests, broad-leaved forests, and mixed forests, which have significant differences in ecological processes, species composition, and responses to environmental changes. Therefore, a more fine-grained detection and classification of vegetation types is needed to conduct in-depth research on ecosystem health, biodiversity conservation, and precision agricultural management.

[0049] In this method, a classification table from rough classification to detailed classification is listed according to the detection requirements of vegetation types. In the specific implementation, the vegetation types are classified into three levels, and the specific classification is shown in the following table:

[0050] Users can also classify and subdivide vegetation types in a more detailed and scientific manner based on their actual needs and research purposes.

[0051] The training data were selected from the regional remote sensing image data, and the three-level classification was annotated according to the above vegetation type classification table to construct a sample data set.

[0052] This method introduces the semantic segmentation network in detail from the aspects of network model architecture design, loss function design and loss weight design.

[0053] Building a semantic segmentation network includes: (1) Obtain the multi-level structure of vegetation types.

[0054] In this method, when constructing the sample data set, the vegetation types are specifically classified. In the specific implementation, the vegetation types are classified into three levels, that is, the number of levels N of the vegetation types is 3.

[0055] (2) Based on the fusion features and the number of vegetation type levels N, an N-level semantic segmentation model coupled with prior knowledge of vegetation types is constructed using the U-Net model as the basic architecture.

[0056] In this method, the semantic segmentation network is multi-layered and utilizes the hierarchical structure of vegetation types. Therefore, it is also called a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types.

[0057] In the specific implementation, the number of vegetation type levels N is 3. According to the classification level of vegetation types, a three-level semantic segmentation model is constructed with the U-Net model as the basic architecture, such as Figure 4 As shown in Figure 2, the input of the three-level semantic segmentation model is the fused features finally obtained in the feature extraction part of step S2 mentioned above.

[0058] Preferably, the N-level semantic segmentation model includes: The previous layer of the N-layer semantic segmentation model serves as a constraint for the next layer; Each level of the N-level semantic segmentation model outputs the corresponding vegetation type result.

[0059] Specifically, the first layer of the model serves as a constraint for the second layer, and the second layer serves as a constraint for the third layer; the first layer outputs the results of the first-level classification of vegetation types, the second layer outputs the results of the second-level classification of vegetation types, and the third layer outputs the results of the third-level classification of vegetation types.

[0060] Preferably, the N-level semantic segmentation model further includes: The loss function of the N-level semantic segmentation model is obtained by considering the losses of all levels and weighted summing the losses of the coarse-grained type level and the fine-grained type level; When training the model, control the distribution of loss weights to overcome the problem of sample imbalance.

[0061] In the specific implementation, the prior structure information of vegetation types can be used to guide the training of the entire model, making it easier for gradients to be back-propagated, thereby reducing the number of training times and accelerating the convergence speed. The loss function of the model considers the losses at all levels and is obtained by weighted summing the losses at the coarse-grained type level and the fine-grained type level. Specifically, the loss function L U The calculation method is as follows:

[0062] Where N is the number of vegetation types, N=3; V n is the loss weight, each V n The value is between 0 and 1, the sum is 1, V n It defines the contribution of a specific level to the final loss function; G is the number of classes to be classified; Y is the expected output result, with dimensions G×h×w; Output results for the model.

[0063] In order to overcome the problem of sample imbalance, the loss weight V is controlled n The distribution of can realize model training. This can not only flexibly control the level type of model output, but also more flexibly extract the level type that needs to be paid attention to during training, effectively prevent gradient diffusion, and overcome sample imbalance.

[0064] Finally, the vegetation type detection results output by each layer of the N-level semantic segmentation model are integrated to obtain the final output result of the detection network. Specifically, the final output result dimension is G×h×w, where G is the number of categories to be classified and the number of channels of the output result. Each channel corresponds to the probability of whether each position belongs to the category.

[0065] In practical applications, a comparison threshold can be pre-set based on the output results of the detection model. If all classification results of a pixel position do not reach the threshold, the position is an unidentified type or other type. If a classification result reaches the threshold, the probabilities of each category are compared, and the vegetation type with the highest probability is selected as the actual category of the current pixel position.

[0066] Preferably, the method further comprises: The detection results are verified by combining samples collected on the ground, and the deep hierarchical semantic segmentation model is fine-tuned and improved.

[0067] The beneficial effects of this embodiment are: Based on the attention mechanism, this method can extract features from the perspectives of space, channel and scale respectively. It can extract differentiated features of remote sensing image data from multiple dimensions and improve the classification effect. By constructing a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types, it can overcome the problem of uneven distribution of vegetation type samples, increase training speed, optimize classification effect and improve detection accuracy.

[0068] This method provides more detailed data support for ecologists, forestry managers, environmental protection agencies, etc. by finely dividing vegetation types, which helps to formulate more scientific and reasonable ecological protection and restoration strategies. In the agricultural field, fine-grained vegetation type detection can also help farmers achieve precise management of crop planting and improve resource utilization efficiency.

[0069] Embodiment 2 like Figure 2 and 5 As shown, a method for detecting regional vegetation types includes the following steps: S1: Obtain regional remote sensing image data and perform preprocessing.

[0070] In practical applications, the collected regional remote sensing image data are multi-temporal GF-1 WFV, GF-1 PMS and GF-2PMS images.

[0071] GF-1 WFV, GF-1 PMS and GF-2 PMS images refer to several different types of remote sensing image data acquired by Earth observation satellites launched by China. GF stands for the "Gaofen" series of satellites, which are high-resolution satellites that can acquire high-resolution surface images for a variety of scientific research and applications. Specifically, "GF-1" and "GF-2" stand for Gaofen-1 and Gaofen-2 satellites, respectively, while "WFV" and "PMS" stand for different types of sensors.

[0072] WFV is the abbreviation of "Wide Field of View". This sensor can cover a wider ground area and is suitable for large-scale environmental monitoring and resource surveys.

[0073] PMS is the abbreviation of "Panchromatic and Multispectral Camera". This sensor can acquire panchromatic and multispectral images. Panchromatic images have high resolution, while multispectral images can provide spectral information of ground objects, which is helpful for land cover classification and environmental monitoring.

[0074] In practical applications, the preprocessing methods for regional remote sensing image data include: noise removal, geometric correction, radiation calibration, atmospheric correction and image registration.

[0075] S2: Based on the preprocessed regional remote sensing image data, hierarchical feature extraction and fusion are performed to obtain fusion features.

[0076] Preferably, the feature extraction part of the method realizes hierarchical feature extraction and fusion by constructing a spatiotemporal feature extraction module and a scale attention module. In the method, the feature extraction part is also referred to as a hierarchical scale-sensitive attention spatiotemporal feature extraction model.

[0077] The attention mechanism in the field of computer vision can extract differentiated features from multiple dimensions. Based on the attention mechanism, features can be extracted from the perspectives of space, channel, and scale, which can enhance the differentiated features of remote sensing images and improve classification results.

[0078] 1. Spatiotemporal feature extraction module By constructing a spatiotemporal feature extraction module based on joint spatial attention and channel attention, spatiotemporal features are extracted from the preprocessed regional remote sensing image data.

[0079] Specifically, the network structure of the spatiotemporal feature extraction module is as follows: Figure 3 As shown, the input feature is A, and the final output feature is E. The dimensions of A and E are both c×h×w, where c is the number of channels, h is the number of pixels in the image height, and w is the number of pixels in the image width.

[0080] Preferably, the spatiotemporal feature extraction module includes: (1) Spatial attention module: considers the correlation between different locations in the image and extracts the spatial features of regional remote sensing image data.

[0081] like Figure 3 As shown in the figure, the spatial attention module inputs feature A, and generates three new feature maps B, F, and D respectively after convolution. The dimensions of B, F, and D are consistent with those of A and E, which are c×h×w. B and F generate feature S through operation. S is a 2h×w feature map, that is, the dimension is 2×h×w. Finally, after transportation, the output feature P is consistent with the dimension of E, which is also c×h×w. Specifically, the calculation method of the spatial attention model is as follows:

[0082]

[0083] Where i and j represent the i-th and j-th pixel positions of the image; α is the scale parameter, which is initially 0; A j is the input feature of the jth position; P j is the output feature of the jth position; S ji It is the core calculation step of the spatial attention model, which measures the correlation between the i-th position and the j-th position; M is the number of samples.

[0084] (2) Channel attention module: considers the correlation between different channels in the image and extracts the channel features of regional remote sensing image data.

[0085] Generally, the input image is an RGB image, which has three channels. However, remote sensing images are different from RGB images and have multiple channels, also called bands. High-resolution images generally have four bands, that is, c is 4; other sensors, such as Sentinel-2, have 11 bands, that is, c is 11. Remote sensing equipment can capture electromagnetic waves of different wavelength ranges, and each band corresponds to data of a specific wavelength range.

[0086] like Figure 3 As shown in the figure, the channel attention module inputs feature A, generates feature X through intermediate calculation, and the dimension of X is c×c; finally, after calculation, it outputs feature Q, and the dimension of Q is consistent with the dimension of E, which is also c×h×w. Specifically, the calculation method of the channel attention module is as follows:

[0087]

[0088] Among them, i and j represent the i-th and j-th bands of the image; β is the weight parameter, which is initially 0; A j is the input feature of the jth position; Qj is the output feature of the jth position; X ji Measures the correlation between the i-th band and the j-th band.

[0089] (3) Add the output results of the spatial attention module and the channel attention module to obtain the final output feature E.

[0090] By integrating the processing results of the spatial attention module and the channel attention module, not only the spatial consistency of pixels of the same category is improved, but also the band distinguishability is enhanced, thus realizing the spatiotemporal feature extraction of remote sensing input images.

[0091] 2. Scale Attention Module By constructing a scale attention module, the contribution weights of spatiotemporal features of different scales are calculated, and the spatiotemporal features of different image data are fused to obtain fused features.

[0092] Specifically, by constructing a scale attention model, we learn the contribution weights of spatiotemporal features generated by remote sensing images with different spatial resolutions to the results, perform weighted summation of spatiotemporal features, and achieve spatiotemporal feature fusion. The weights are calculated as follows:

[0093] Among them, k∈{1,...,K}, represents the level of remote sensing images with different spatial resolutions. In the specific implementation, it includes 5 scale levels, K=5, k∈{1,...,5}; w k is the weight; h k 、h t It is the classification probability results at different scale levels.

[0094] S3: Construct a deep hierarchical semantic segmentation model coupled with vegetation type prior knowledge to input fusion features and output vegetation type classification results.

[0095] Among them, the prior knowledge of vegetation types is the hierarchical structure of vegetation types.

[0096] This method uses a semantic segmentation network to detect vegetation types. Before building the semantic segmentation network, it is necessary to classify the vegetation types first; then the remote sensing image data is annotated to construct a sample data set.

[0097] In forest ecosystems, traditional vegetation type detection only divides forests, shrubs, and grasses into broad categories, which is too general. For example, the broad category of "forest" alone cannot effectively distinguish between different types such as coniferous forests, broad-leaved forests, and mixed forests, which have significant differences in ecological processes, species composition, and responses to environmental changes. Therefore, a more fine-grained detection and classification of vegetation types is needed to conduct in-depth research on ecosystem health, biodiversity conservation, and precision agricultural management.

[0098] In this method, a classification table from rough classification to detailed classification is listed according to the detection requirements of vegetation types. In the specific implementation, the vegetation types are classified into three levels, and the specific classification is shown in the following table:

[0099] Users can also classify and subdivide vegetation types in a more detailed and scientific manner based on their actual needs and research purposes.

[0100] The training data were selected from the regional remote sensing image data, and the three-level classification was annotated according to the above vegetation type classification table to construct a sample data set.

[0101] This method introduces the semantic segmentation network in detail from the aspects of network model architecture design, loss function design and loss weight design.

[0102] Building a semantic segmentation network includes: (1) Obtain the multi-level structure of vegetation types.

[0103] In this method, when constructing the sample data set, the vegetation types are specifically classified. In the specific implementation, the vegetation types are classified into three levels, that is, the number of levels N of the vegetation types is 3.

[0104] (2) Based on the fusion features and the number of vegetation type levels N, an N-level semantic segmentation model coupled with prior knowledge of vegetation types is constructed using the U-Net model as the basic architecture.

[0105] In this method, the semantic segmentation network is multi-layered and utilizes the hierarchical structure of vegetation types. Therefore, it is also called a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types.

[0106] In the specific implementation, the number of vegetation type levels N is 3. According to the classification level of vegetation types, a three-level semantic segmentation model is constructed with the U-Net model as the basic architecture, such as Figure 4 As shown in Figure 2, the input of the three-level semantic segmentation model is the fused features finally obtained in the feature extraction part of step S2 mentioned above.

[0107] Preferably, the N-level semantic segmentation model includes: The previous layer of the N-layer semantic segmentation model serves as a constraint for the next layer; Each level of the N-level semantic segmentation model outputs the corresponding vegetation type result.

[0108] Specifically, the first layer of the model serves as a constraint for the second layer, and the second layer serves as a constraint for the third layer; the first layer outputs the results of the first-level classification of vegetation types, the second layer outputs the results of the second-level classification of vegetation types, and the third layer outputs the results of the third-level classification of vegetation types.

[0109] Preferably, the N-level semantic segmentation model further includes: The loss function of the N-level semantic segmentation model is obtained by considering the losses of all levels and weighted summing the losses of the coarse-grained type level and the fine-grained type level; When training the model, control the distribution of loss weights to overcome the problem of sample imbalance.

[0110] In the specific implementation, the prior structure information of vegetation types can be used to guide the training of the entire model, making it easier for gradients to be back-propagated, thereby reducing the number of training times and accelerating the convergence speed. The loss function of the model considers the losses at all levels and is obtained by weighted summing the losses at the coarse-grained type level and the fine-grained type level. Specifically, the loss function L U The calculation method is as follows:

[0111] Where N is the number of vegetation types, N=3; V n is the loss weight, each V n The value is between 0 and 1, the sum is 1, V n It defines the contribution of a specific level to the final loss function; G is the number of classes to be classified; Y is the expected output result, with dimensions G×h×w; Output results for the model.

[0112] In order to overcome the problem of sample imbalance, the loss weight V is controlled n The distribution of can realize model training. This can not only flexibly control the level type of model output, but also more flexibly extract the level type that needs to be paid attention to during training, effectively prevent gradient diffusion, and overcome sample imbalance.

[0113] (3) Based on the hierarchical structure of vegetation types, the vegetation type classification results are cascaded and fused to construct an optimization model to obtain optimized vegetation type detection results.

[0114] In this step, by constructing an optimization model, based on the mutual relationship between the types at each level, the classification results of each layer of the N-level semantic segmentation model are cascaded and integrated to ensure the consistency of the detection results at each level in hierarchical logic, and to construct the optimized output of the N-level semantic segmentation model to obtain the optimized vegetation type detection results. In this method, this step is also called the classification result optimization method combining the uncertainty of each layer result.

[0115] Specifically, assuming is the hierarchical classification result probability generated by the N-level semantic segmentation model. Cascade and use it as input to build the optimization model, which is calculated as follows:

[0116] in, is the final output classification result; Z is the newly constructed optimization model. In order to enable the newly constructed network model to complete synchronous end-to-end network training with the N-level semantic segmentation model, this method uses ResNet as the basic architecture to build the optimization model.

[0117] The detection network of this method combines the N-level semantic segmentation model with the above optimization model. The final loss function L consists of two parts. One part is the weighted loss L generated by the N-level semantic segmentation model. U , and the other part is the loss L of the optimization model Z , the specific calculation method is as follows:

[0118]

[0119] in, To optimize the hyperparameters of the model, it is used to control the classification results of each layer and the weight of the results obtained by the optimization model.

[0120] Finally, the output dimension of the detection network is G×h×w, where G is the number of objects to be classified and the number of channels of the output result. Each channel corresponds to the probability of whether each position belongs to that category.

[0121] In practical applications, a comparison threshold can be pre-set based on the output results of the detection model. If all classification results of a pixel position do not reach the threshold, the position is an unidentified type or other type. If a classification result reaches the threshold, the probabilities of each category are compared, and the vegetation type with the highest probability is selected as the actual category of the current pixel position.

[0122] Preferably, the method further comprises: The detection results are verified by combining samples collected on the ground, and the deep hierarchical semantic segmentation model is fine-tuned and improved.

[0123] S4: Construct a regional vegetation type distribution map based on the vegetation type detection results.

[0124] The vegetation type detection results of the detection area are visualized and a vegetation type distribution map is constructed to provide assistance for environmental monitoring and decision analysis, and facilitate subsequent analysis and processing.

[0125] The beneficial effects of this embodiment are as follows: this method also constructs an optimization model to cascade and fuse the classification results of each layer of the N-level semantic segmentation model, thereby ensuring the consistency of the detection results of each layer in hierarchical logic and optimizing the output results; constructs a visual regional vegetation type distribution map to provide assistance for environmental monitoring and decision-making analysis, and facilitates subsequent other analysis and processing.

[0126] Embodiment 3 A regional vegetation type detection system, based on the regional vegetation type detection method as described in Embodiment 1 or 2, comprises: Data acquisition module, used to acquire regional remote sensing image data and perform preprocessing; The feature extraction module is used to extract and fuse hierarchical features based on the pre-processed regional remote sensing image data to obtain fusion features; The detection module is used to build a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types to input fusion features and output vegetation type classification results.

[0127] Preferably, the system further comprises: Model adjustment module, used to verify the detection results by combining field samples, and to fine-tune and improve the regional vegetation type detection model; The distribution map construction module is used to construct a regional vegetation type distribution map based on the vegetation type detection results.

[0128] Embodiment 4

[0129] A computer program, which, when executed, implements the steps of the regional vegetation type detection method as described in embodiment one or two.

[0130] Embodiment 5

[0131] A readable storage medium stores a computer program as described in Example 4, and when the computer program is executed, the steps of the regional vegetation type detection method as described in Example 1 or 2 are implemented.

[0132] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with the art within the technical scope disclosed in the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting regional vegetation types, characterized in that: The following steps are involved: Acquire regional remote sensing image data and perform preprocessing; According to the pre-processed regional remote sensing image data, hierarchical feature extraction and fusion are performed to obtain fusion features; A deep hierarchical semantic segmentation model coupled with vegetation type prior knowledge is constructed to input fusion features and output vegetation type classification results; Among them, the prior knowledge of vegetation types is the hierarchical structure of vegetation types.

2. The method for detecting regional vegetation types according to claim 1, characterized in that: The step of extracting and fusing hierarchical features to obtain fusion features based on the preprocessed regional remote sensing image data includes: A spatiotemporal feature extraction module is constructed based on joint spatial attention and channel attention to extract spatiotemporal features of the preprocessed regional remote sensing image data.

3. The method for detecting regional vegetation types according to claim 2, characterized in that: The step of extracting and fusing hierarchical features to obtain fusion features based on the preprocessed regional remote sensing image data includes: A scale attention module is constructed to calculate the contribution weights of spatiotemporal features of different scales, and the spatiotemporal features of different image data are fused to obtain fused features.

4. The method for detecting regional vegetation types according to claim 1, characterized in that: The deep hierarchical semantic segmentation model coupled with vegetation type prior knowledge includes: Obtain the multi-level structure of vegetation types; Based on the fusion features and the number of vegetation type levels N, an N-level semantic segmentation model coupled with prior knowledge of vegetation types was constructed using the U-Net model as the basic architecture.

5. The method for detecting regional vegetation types according to claim 4, characterized in that: The N-level semantic segmentation model includes: The previous layer of the N-layer semantic segmentation model serves as a constraint for the next layer; Each level of the N-level semantic segmentation model outputs the corresponding vegetation type result.

6. The method for detecting regional vegetation types according to claim 5, characterized in that: The N-level semantic segmentation model includes: The loss function of the N-level semantic segmentation model is obtained by considering the losses of all levels and weighted summing the losses of the coarse-grained type level and the fine-grained type level; Control the distribution of loss weights when training the model.

7. The method for detecting regional vegetation types according to any one of claims 1 to 6, characterized in that: The step of constructing a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types to input fusion features and output vegetation type classification results also includes: Based on the hierarchical structure of vegetation types, the vegetation type classification results are cascaded and fused to build an optimization model to obtain optimized vegetation type detection results.

8. The method for detecting regional vegetation types according to claim 1, characterized in that: The regional vegetation type detection method also includes: The detection results are verified by combining samples collected on the ground, and the deep hierarchical semantic segmentation model is fine-tuned and improved.

9. The method for detecting regional vegetation types according to claim 1, characterized in that: The regional vegetation type detection method also includes: Based on the vegetation type detection results, a regional vegetation type distribution map is constructed.

10. A regional vegetation type detection system, characterized in that: The method for detecting regional vegetation types according to any one of claims 1 to 9 comprises: Data acquisition module, used to acquire regional remote sensing image data and perform preprocessing; The feature extraction module is used to extract and fuse hierarchical features based on the pre-processed regional remote sensing image data to obtain fusion features; The detection module is used to build a deep hierarchical semantic segmentation model coupled with prior knowledge of vegetation types to input fusion features and output vegetation type classification results.

Images