A ConvNeXt-based remote sensing image vegetation classification method and device

Abstract

The application provides a ConvNeXt-based remote sensing image vegetation classification method and device, relates to the technical field of vegetation classification, and comprises the following steps: acquiring sample remote sensing image data, expanding the sample remote sensing image data by using an Fmix mixed sample data enhancement algorithm, and obtaining a sample data set; training a vegetation classification model by using the sample data set to obtain a target vegetation classification model, wherein the vegetation classification model comprises a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet; after acquiring to-be-classified remote sensing image data, inputting the to-be-classified remote sensing image data into the target vegetation classification model to obtain an initial classification result; performing adjacent category fusion processing on target objects in the initial classification result, and performing contour simplification processing on graph patches in the category fusion processing result to obtain a target classification result, thereby solving the technical problem of low precision of the existing vegetation classification method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vegetation classification, and in particular to a ConvNeXt-based remote sensing image vegetation classification method and device. BACKGROUND

[0002] Vegetation classification is an important part of studying land cover, resource utilization and change analysis. Although traditional field surveys can accurately classify vegetation categories, they consume a lot of manpower and time. Satellite remote sensing technology provides a large amount of data basis for vegetation classification, and how to quickly and accurately extract different vegetation categories becomes a difficult problem to be solved.

[0003] Current vegetation classification research mainly uses multispectral information of remote sensing images. The spectral characteristics of different vegetation have consistency and difference, and vegetation feature indexes such as NDVI normalized difference vegetation index and GVI greenness vegetation index can be calculated accordingly. Then, traditional machine learning algorithms such as K-nearest neighbor, support vector machine and random forest are used to classify different vegetation. Such algorithms are often suitable for smaller research scope, time interval and category classification. Vegetation classification categories rely on spectral differences and do not fully utilize other image information. Once the spectral characteristics change, the classification accuracy may be greatly reduced.

[0004] Deep learning methods can maximize the extraction of information from images and realize automatic learning of features, which is one of the more effective methods for vegetation classification under massive remote sensing data. High spatial resolution remote sensing images cannot provide more band radiation characteristics, but can supplement more detailed spatial texture information. At present, there are few deep learning model researches on vegetation classification based on high-resolution remote sensing images. There are problems such as unbalanced vegetation categories, easily confused categories, and poor edge effect of classification results.

[0005] No effective solution has been proposed to solve the above problems. SUMMARY

[0006] Therefore, the present application aims to provide a ConvNeXt-based remote sensing image vegetation classification method and device to alleviate the technical problem of low vegetation classification accuracy and efficiency.

[0007] In a first aspect, an embodiment of the present application provides a ConvNeXt-based remote sensing image vegetation classification method, comprising: obtaining sample remote sensing image data, and expanding the sample remote sensing image data using an Fmix mixed sample data enhancement algorithm to obtain a sample data set; training a vegetation classification model using the sample data set to obtain a target vegetation classification model, wherein the vegetation classification model comprises a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet; after obtaining to-be-classified remote sensing image data, inputting the to-be-classified remote sensing image data into the target vegetation classification model to obtain an initial classification result; performing adjacent class fusion processing on a target object in the initial classification result, and performing contour simplification processing on a graph patch in the class fusion processing result to obtain a target classification result, wherein the target object includes patches and holes with an area less than a preset threshold.

[0008] Further, the sample remote sensing image data is expanded using the Fmix mixed sample data enhancement algorithm to obtain a sample data set, comprising: manually interpreting and labeling vegetation types in the sample remote sensing image data to obtain target remote sensing image data; segmenting the target remote sensing image data according to a preset size to obtain an initial sample data set; processing the initial sample data set using the Fmix mixed sample data enhancement algorithm to obtain a sample data set.

[0009] Further, the vegetation classification model is trained by using the sample data set to obtain a target vegetation classification model, including: dividing the sample data set into a training set and a validation set; the calculation step, inputting a preset number of sample data in the training set into the vegetation classification model, and calculating the sum of cross-entropy losses; the optimization step, based on the sum of cross-entropy losses and AdamW function, the parameter optimization of the vegetation classification model is carried out to obtain an initial vegetation classification model; the first execution step, the initial vegetation classification model is determined as the vegetation classification model, and the calculation step and the optimization step are repeatedly executed until the number of repeated executions reaches a first preset number, and an intermediate vegetation classification model is obtained; the second execution step, the intermediate vegetation classification model is determined as the vegetation classification model, and the validation set is determined as the training set, and the calculation step, the optimization step and the first execution step are repeatedly executed until the number of repeated executions reaches a second preset number, and the intermediate vegetation classification model with the maximum intersection over union is determined as the target vegetation classification model. Further, the target object in the initial classification result is subjected to adjacent class fusion processing, including: inputting each class in the initial classification result into a corresponding channel; calculating the connected domain area of each channel, and determining the graph spot and hole in each channel with a connected domain area less than the preset threshold value as the target object; based on the target object, a first mask is constructed, and the graph spot and hole with an area greater than the preset threshold value in the first mask are removed to obtain a second mask; based on a preset vegetation class order, the target object in the second mask data is subjected to adjacent class fusion processing.

[0010] Further, the graph spot in the class fusion processing result is subjected to contour simplification processing, including: extracting the internal boundary and the external boundary of the graph spot in the class fusion processing result; using the Visvalingam-Whyatt algorithm to simplify the boundary points of the internal boundary and the external boundary.

[0011] In a second aspect, the embodiments of the present application also provide a ConvNeXt-based remote sensing image vegetation classification device, comprising: an acquisition unit, a training unit, a classification unit and an optimization unit, wherein the acquisition unit is configured to acquire sample remote sensing image data, and expand the sample remote sensing image data by using a Fmix mixed sample data enhancement algorithm to obtain a sample data set; the training unit is configured to train a vegetation classification model by using the sample data set to obtain a target vegetation classification model, wherein the vegetation classification model comprises a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet; the classification unit is configured to input the to-be-classified remote sensing image data into the target vegetation classification model after acquiring the to-be-classified remote sensing image data to obtain an initial classification result; and the optimization unit is configured to perform adjacent class fusion processing on a target object in the initial classification result, and perform contour simplification processing on a graph patch in the class fusion processing result to obtain a target classification result, wherein the target object comprises a patch and a hole with an area less than a preset threshold.

[0012] Further, the acquisition unit is configured to: manually interpret and label the vegetation types in the sample remote sensing image data to obtain target remote sensing image data; segment the target remote sensing image data according to a preset size to obtain an initial sample data set; and expand the initial sample data set by using the Fmix mixed sample data enhancement algorithm to obtain a sample data set.

[0013] Further, the training unit is configured to: divide the sample data set into a training set and a validation set; in a calculation step, input a preset number of sample data in the training set into the vegetation classification model and calculate the sum of cross-entropy losses; in an optimization step, based on the sum of cross-entropy losses and an AdamW function, perform parameter optimization on the vegetation classification model to obtain an initial vegetation classification model; in a first execution step, determine the initial vegetation classification model as the vegetation classification model, repeatedly execute the calculation step and the optimization step until the number of repeated executions reaches a first preset number, and obtain an intermediate vegetation classification model; and in a second execution step, determine the intermediate vegetation classification model as the vegetation classification model, determine the validation set as the training set, repeatedly execute the calculation step, the optimization step and the first execution step until the number of repeated executions reaches a second preset number, and determine an intermediate vegetation classification model with the maximum intersection over union among the intermediate vegetation classification models as the target vegetation classification model.

[0014] In a third aspect, the embodiments of the present application also provide an electronic device, comprising a memory and a processor, the memory is configured to store a program supporting the processor to execute the method in the first aspect, and the processor is configured to execute the program stored in the memory.

[0015] In a fourth aspect, an embodiment of the present application also provides a computer readable storage medium, and the computer readable storage medium stores a computer program.

[0016] In the embodiment of the present application, sample remote sensing image data is acquired, and the sample remote sensing image data is expanded by using a Fmix mixed sample data enhancement algorithm to obtain a sample data set; a target vegetation classification model is obtained by training a vegetation classification model by using the sample data set, wherein the vegetation classification model comprises a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet; after acquiring remote sensing image data to be classified, the remote sensing image data to be classified is input into the target vegetation classification model to obtain an initial classification result; a target object in the initial classification result is subjected to adjacent class fusion processing, and a graph patch in the class fusion processing result is subjected to contour simplification processing to obtain a target classification result, wherein the target object comprises a patch and a hole with an area less than a preset threshold, so that the purpose of accurately classifying vegetation in a remote sensing image is achieved, thereby solving the technical problem of low accuracy of the existing vegetation classification method, and improving the technical effect of improving the accuracy of the vegetation classification method.

[0017] Other features and advantages of the present application will be set forth in the descriptions below, and in part will become apparent to those skilled in the art from the descriptions, or can be learned by practice of the present application. The objects and other advantages of the present application can be realized and achieved by the structure particularly pointed out in the descriptions, claims and drawings.

[0018] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0020] Figure 1 A flowchart of a remote sensing image vegetation classification method based on ConvNeXt provided by an embodiment of the present application is shown in the figure;

[0021] Figure 2 A schematic diagram of a remote sensing image vegetation classification device based on ConvNeXt provided by an embodiment of the present application is shown in the figure;

[0022] Figure 3A structural schematic diagram of a ConvNeXt encoder provided for an embodiment of the present application is shown in FIG. 1.

[0023] Figure 4 A structural schematic diagram of a UPerNet decoder provided for an embodiment of the present application is shown in FIG. 2.

[0024] Figure 5 A structural schematic diagram of a PPM module provided for an embodiment of the present application is shown in FIG. 3.

[0025] Figure 6 A schematic diagram of an electronic device provided for an embodiment of the present application is shown in FIG. 4. DETAILED DESCRIPTION

[0026] To make the objectives, technical solutions, and advantages of embodiments of the present application clearer, the technical solutions of the present application will be described below in conjunction with the accompanying drawings, which apparently are some embodiments of the present application but not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of the present application.

[0027] Embodiment One

[0028] According to an embodiment of the present application, an embodiment of a ConvNeXt-based remote sensing image vegetation classification method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0029] Figure 1 A flowchart of a ConvNeXt-based remote sensing image vegetation classification method according to an embodiment of the present application is shown in FIG. 1, which includes the following steps: Figure 1

[0030] Step S102, sample remote sensing image data is obtained, and the sample remote sensing image data is expanded using a Fmix mixed sample data enhancement algorithm to obtain a sample data set.

[0031] Step S104, the sample data set is used to train a vegetation classification model to obtain a target vegetation classification model, wherein the vegetation classification model includes a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet.

[0032] Step S106, after obtaining the remote sensing image data to be classified, the remote sensing image data to be classified is input into the target vegetation classification model to obtain an initial classification result.

[0033] ​In step S108, the target objects in the initial classification result are subjected to a nearby class fusion processing, and the graph patches in the class fusion processing result are subjected to a contour simplification processing, to obtain a target classification result, wherein the target objects include patches and holes with an area less than a preset threshold.

[0034] In the embodiment of the present application, sample remote sensing image data is acquired, and the sample remote sensing image data is expanded by using an Fmix mixed sample data enhancement algorithm to obtain a sample data set; a vegetation classification model is trained by using the sample data set to obtain a target vegetation classification model, wherein the vegetation classification model includes a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet; after acquiring remote sensing image data to be classified, the remote sensing image data to be classified is input into the target vegetation classification model to obtain an initial classification result; target objects in the initial classification result are subjected to a nearby class fusion processing, and graph patches in the class fusion processing result are subjected to a contour simplification processing, to obtain a target classification result, wherein the target objects include patches and holes with an area less than a preset threshold, thereby achieving the purpose of accurately classifying vegetation in remote sensing images, and further solving the technical problem of low accuracy of existing vegetation classification methods, thereby improving the technical effect of vegetation classification method accuracy.

[0035] In the embodiment of the present application, step S102 includes the following steps:

[0036] The vegetation types in the sample remote sensing image data are subjected to artificial interpretation and labeling to obtain target remote sensing image data;

[0037] The target remote sensing image data is segmented according to a preset size to obtain an initial sample data set;

[0038] The initial sample data set is expanded by using the Fmix mixed sample data enhancement algorithm to obtain a sample data set.

[0039] In the embodiment of the present application, after acquiring sample remote sensing image data, the coverage of grassland, shrubs and economic forests in the vegetation categories of the sample remote sensing image data is less, so even if images containing the above three types of vegetation regions are selectively selected in sample making, the sample amount is generally low, resulting in sample imbalance. In vegetation classification model training, sample imbalance may result in low classification accuracy of the model for weak categories and poor model generalization.

[0040] The present application adopts the Fmix method to perform data enhancement on weak class samples in view of the sample imbalance problem: since vegetation is widely distributed and can exist in any background, and the edges of vegetation are mostly irregular, the present application adopts the Fmix method to crop the target samples into any shape and paste them into any background, thereby improving the data quantity and richness of weak class samples. The following steps are included:

[0041] Step 1a: high-resolution satellite remote sensing images are collected, the resolution is 0.5-0.8 m, and the wave bands include red, green and blue three wave bands; forest land, grassland, farmland, shrub and economic forest are manually interpreted and labeled; the labeled images are sliced, the slice size is 512*512, and 6013 pairs of sample data are obtained;

[0042] Step 1b: the pixel number of each class in the vegetation sample data is counted, and the proportion of each class is calculated, and the grassland, shrub and economic forest with a proportion less than 5% are considered as weak classes, and samples with a proportion of three types of surface features greater than 20% in the sample are extracted as weak class samples;

[0043] Step 1c: a low-frequency image is obtained from the Fourier space, a binary mask image mask is obtained by using the low-frequency image, two samples image1 and image2 and corresponding labels label1 and label2 are randomly extracted from the weak class samples and the original samples respectively, image is calculated as image1*mask+image2*(1-mask), label is calculated as label1*mask+label2*(1-mask), and is output, until 1987 pairs of samples are generated, a total of 8000 pairs of sample data (i.e., sample data set).

[0044] The vegetation classification model is described in detail below.

[0045] The embodiment of the present application provides a deep learning network structure taking ConvNeXt as an encoder and UPerNet as a decoder, the ConvNeXt encoder is used as a backbone network to obtain semantic features F1, F2, F3 and F4 of different levels, the UPerNet decoder extracts multi-scale features of high-level semantic information through a PPM module, and low-level high-resolution features and high-level semantic features are fused through an FPN network, so that a high-precision vegetation classification result is obtained.

[0046] As Figure 2As shown, the ConvNeXt network is composed of a Stem layer and 4 convolution block groups in series: the stem layer is an overlapping-free convolution with a convolution kernel size of 4x4 and an output channel of 128, which down-samples the features to 1 / 4 of the original size; the 4 convolution block groups are connected by overlapping-free convolution with a convolution kernel size of 2x2, which down-samples the output of the previous block group to 1 / 2, and the outputs of the 4 convolution block groups are feature maps F1, F2, F3, and F4, respectively, with channel numbers of 128, 256, 512, and 1024, respectively; the 4 convolution block groups are composed of 3, 3, 27, and 3 convolution blocks, respectively, and two convolution blocks are added by jump connection as in the ResNet residual structure;

[0047] The inverse bottleneck layer and the depth separable convolution are used in the convolution block, Figure 2 As shown in the right, the structure of one convolution block is as follows: first, a 1x1 point-wise convolution is used to increase the dimension of the feature map to 4 times the original, then a 7x7 depth convolution is used to extract features, and finally a 1x1 point-wise convolution is used to reduce the dimension to the original dimension; among them, the depth convolution adopts the idea of group convolution in ResNeXt, that is, every 4 channels are divided into a group, and convolution is performed on each group, and finally the features of each group are spliced in the channel dimension to improve the calculation speed of the model; the depth convolution layer is followed by a Layer Norm (LN) normalization layer and a ReLU activation function.

[0048] As shown in the right, Figure 3 As shown in the right, the main body of the UPerNet network is an FPN network, and the highest level feature F4 is output to the FPN network through the PPM module.

[0049] As shown in the right, Figure 4 As shown in the right, the output is the highest level feature F4 (1024xH / 32xW / 32) extracted by the encoder, and 1x1 convolution is performed on the F4 after average pooling with sizes of 1, 2, 3, and 6, respectively, and then the dimension is reduced to 512, and then the four size feature maps are up-sampled to the size of the F4 feature map, and the up-sampled feature map is spliced with the F4, and then a convolution with a kernel size of 3x3, a stride of 1, and an output channel of 512 is performed to obtain a feature map B4;

[0050] The 1x1 convolution is performed on the secondary high-level semantic feature F3, so that the number becomes 512, and the 2 times up-sampling B4 is added to obtain the fusion feature B3, and the F2 and F1 are dimensioned and added to the 2 times up-sampled B3 and B2 to obtain B2 and B1. The feature maps B1, B2, B3 and B4 are up-sampled to be the same as B1 and are spliced along the channel dimension. A convolution with a convolution kernel size of 3x3, a step of 1 and an output channel of 512 is used to fuse the four levels of features to obtain a feature map B. After 4 times up-sampling and a classification head with a class number of 6, a prediction classification result can be obtained.

[0051] In the embodiment of the present application, step S104 comprises the following steps:

[0052] The sample data set is divided into a training set and a validation set.

[0053] The calculation step inputs a preset number of sample data in the training set into the vegetation classification model and calculates the sum of cross-entropy losses.

[0054] The optimization step optimizes the parameters of the vegetation classification model based on the sum of cross-entropy losses and an AdamW function to obtain an initial vegetation classification model.

[0055] The first execution step determines the initial vegetation classification model as the vegetation classification model, repeatedly executes the calculation step and the optimization step until the number of repeated executions reaches a first preset number, and obtains an intermediate vegetation classification model.

[0056] The second execution step determines the intermediate vegetation classification model as the vegetation classification model and determines the validation set as the training set, repeatedly executes the calculation step, the optimization step and the first execution step until the number of repeated executions reaches a second preset number, and determines the intermediate vegetation classification model with the maximum intersection-over-union ratio as the target vegetation classification model.

[0057] Specifically, step 1a: using transfer learning, loading a ConvNeXt model trained on ImageNet as initial model parameters;

[0058] Step 1b: inputting a batch of training data with a size of 4 into the vegetation classification model to obtain a vegetation classification result, and calculating the loss according to the true label:

[0059] The loss used in the present application is the sum of cross-entropy loss CE Loss and Dice Loss:

[0060] Loss=Loss CE +Loss Dice

[0061] The CE loss is calculated as follows:

[0062]

[0063] M represents the number of categories, p c represents the probability of being predicted as category c, y c is a one-hot vector, which is 1 when the category is the same as the category of the sample, and 0 otherwise.

[0064] The Dice loss is calculated as follows:

[0065]

[0066]

[0067]

[0068] q i is the network prediction value after Sigmoid or Softmax, taking values (0, 1), t i is the true value, taking values 0 or 1.

[0069] The Dice loss can alleviate the problem of sample imbalance. In the process of calculating Dice, the intersection (product) of the prediction result and the true value of each category is equivalent to masking other categories, and is independent of the proportion of the background (other categories). The training is more inclined to explore the foreground area. However, when the target is small, once the prediction of some pixels of this category is wrong, the Dice loss will be large, further causing the gradient to change dramatically and the training to be unstable. The CE loss is the overall average, and when the target is small, the impact on the loss is also small, and the training relies more on the background area. Therefore, the present application uses the strategy of CE+Dice to calculate the loss.

[0070] Step 3c: use AdamW function as optimization function, update network model parameters;

[0071] Use AdamW method as optimization function:

[0072] m t = β1 × m t-1 + (1-β1) × g t

[0073] v t = β2 × v t-1 + (1-β2) × g t 2

[0074]

[0075]

[0076]

[0077] g t is the gradient of the loss to the weight, m t is the first momentum, v t is the second momentum, α is the initial learning rate 0.001, the experience values of hyperparameters β1 and β2 are 0.9 and 0.999 respectively, and λ is the weight decay factor, which is generally set to 0.01. According to the AdamW optimizer, the current gradient g t is first calculated, and then the current first momentum m t and the second momentum v t are calculated based on the current gradient and the historical momentum, the first momentum controls the learning direction, and the second momentum controls the learning rate, v t is larger, the update step is smaller; thirdly, the initial momentum is biased and corrected; finally, the last parameter update is introduced into the current model parameter θ t .

[0078] Step 1d: iterate the steps 1b-1c until a certain number of iterations is reached, input the validation data set into the model to calculate the classification accuracy of the model;

[0079] Step 1e: iterate the step 1d until the maximum training batch is reached, and select the model with the highest average intersection over union score on the validation set as the target vegetation classification model.

[0080] The step S106 is described below.

[0081] After obtaining the to-be-classified remote sensing image data, the to-be-classified remote sensing image data is input into the target vegetation classification model, the predicted label is obtained, and is written into the prediction result block by block. To ensure good continuity between two blocks, the target block size is 512x512 when reading, and an image of 128 size is expanded around it to be sent into the model for inference and prediction.

[0082] In the embodiment of the present application, the step S108 includes the following steps:

[0083] Each class in the initial classification result is input into the corresponding channel;

[0084] The connected domain area of each channel is calculated, and the image patches and holes in each channel with a connected domain area less than the preset threshold are determined as the target object;

[0085] A first mask is constructed based on the target object, and the image patches and holes with an area greater than the preset threshold in the first mask are removed to obtain a second mask;

[0086] The target formation in the second mask data is subjected to a nearby category fusion processing based on a preset vegetation category sequence.

[0087] The internal boundary and the external boundary of the graph patch in the category fusion processing result are extracted;

[0088] The internal boundary and the external boundary are subjected to boundary point simplification processing by using a Visvalingam-Whyatt algorithm.

[0089] Specifically, step 2a: a preset vegetation category sequence is set, when the category sequence is in the front, the small graph patches and the holes adjacent to the category are preferentially assigned to the category, and the preset vegetation category sequence is set as woodland, farmland, economic forest, grassland and shrub according to the reliability of the prediction result of each category;

[0090] Step 2b: the output classification result is split into six channels of binary graph (five vegetation categories and one background category), and the region with a value of 1 in each channel is the region predicted to be the category;

[0091] Step 2c: the connected domain of the graph patch in each channel is calculated, the area of each connected domain is calculated, the graph patch with a pixel area less than 200 is considered as a to-be-fused graph patch, and the union set of all to-be-fused graph patches in all channels is mask1;

[0092] Step 2d: in order to avoid two or more adjacent small graph patches being fused into a large graph patch not belonging to any small graph patch category, the graph patch with a pixel area greater than the threshold value of 200 in the mask1 is removed to obtain a mask;

[0093] Step 2e: the pixels in the mask region in the prediction result are equal to the category i according to the fusion sequence;

[0094] Step 2f: iteration is performed on steps 2b-2e, if the small graph patch is adjacent to the category i, the graph patch will not be identified in the next small graph patch extraction after the fusion in step 2e is performed, otherwise the small graph patch will enter the next iteration to try to be fused into the next category, until all categories are iterated, and a single-channel classification graph is regenerated;

[0095] Step 2g: the external boundary and the internal boundary of the classification graph patch after the fusion processing are extracted, and the boundary points are simplified by using a Visvalingam-Whyatt algorithm;

[0096] Step 2h: the position information and the attribute information of the boundary points are written into a vector file to obtain a target classification result, and the shapefile format of ESRI is used here.

[0097] The current vegetation classification algorithm based on machine learning relies on spectral features, and the distinguishable vegetation categories are limited and the generalization is low. In order to improve the accuracy, generalization and classification category of vegetation classification, the application provides a vegetation classification method and system for high-resolution satellite remote sensing image: for the problem of unbalanced vegetation sample categories, the Fmix method is used to expand the weak category samples; for the problem of easy confusion between vegetation categories, the ConvNeXt+UPerNet network structure is used to extract multi-level fusion features to realize high-precision classification extraction; for the problem of a large number of ambiguous small patches at the classification edge, a technical process of fusing small patches by category is proposed.

[0098] The remote sensing image vegetation classification method and system provided by the embodiment of the application optimize from the three stages of "front (sample preparation and enhancement) → middle (model building and training) → rear (edge fusion and smoothing)", which improves the accuracy and efficiency of vegetation classification in high-resolution remote sensing images.

[0099] Embodiment two:

[0100] The embodiment of the application also provides a ConvNeXt-based remote sensing image vegetation classification device for executing the ConvNeXt-based remote sensing image vegetation classification method provided by the above content of the embodiment of the application. The following is a specific introduction to the ConvNeXt-based remote sensing image vegetation classification device provided by the embodiment of the application.

[0101] As shown in Figure 5 , 2 is a schematic diagram of the above-mentioned ConvNeXt-based remote sensing image vegetation classification device, which includes an acquisition unit 10, a training unit 20, a classification unit 30 and an optimization unit 40.

[0102] The acquisition unit is configured to acquire sample remote sensing image data, and expand the sample remote sensing image data by using an Fmix mixed sample data enhancement algorithm to obtain a sample data set.

[0103] The training unit is configured to train a vegetation classification model by using the sample data set to obtain a target vegetation classification model, wherein the vegetation classification model includes a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet.

[0104] The classification unit is configured to input the to-be-classified remote sensing image data into the target vegetation classification model after acquiring the to-be-classified remote sensing image data to obtain an initial classification result.

[0105] The optimization unit is configured to perform adjacent class fusion processing on a target object in the initial classification result, and perform contour simplification processing on a graph spot in the class fusion processing result, to obtain a target classification result, wherein the target object includes a patch and a hole with an area less than a preset threshold.

[0106] In the embodiment of the present application, sample remote sensing image data is acquired, and the sample remote sensing image data is expanded by using a Fmix mixed sample data enhancement algorithm to obtain a sample data set; a vegetation classification model is trained by using the sample data set to obtain a target vegetation classification model, wherein the vegetation classification model includes a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet; after acquiring remote sensing image data to be classified, the remote sensing image data to be classified is input into the target vegetation classification model to obtain an initial classification result; a target object in the initial classification result is subjected to adjacent class fusion processing, and a graph spot in the class fusion processing result is subjected to contour simplification processing to obtain a target classification result, wherein the target object includes a patch and a hole with an area less than a preset threshold, thereby achieving the purpose of accurately classifying vegetation in a remote sensing image, and solving the technical problem of low accuracy of existing vegetation classification methods, thereby improving the technical effect of improving the accuracy of the vegetation classification method.

[0107] Embodiment three:

[0108] The embodiment of the present application also provides an electronic device, including a memory and a processor, the memory is used to store the program supporting the processor to execute the method described in the above embodiment one, and the processor is configured to execute the program stored in the memory.

[0109] Referring to Figure 6 The embodiment of the present application also provides an electronic device 100, including a processor 60, a memory 61, a bus 62 and a communication interface 63, the processor 60, the communication interface 63 and the memory 61 are connected through the bus 62; the processor 60 is used to execute the executable module stored in the memory 61, such as a computer program.

[0110] The memory 61 may include a high-speed random access memory (RAM, Random Access Memory) and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 63 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.

[0111] The bus 62 can be an ISA bus, a PCI bus, an EISA bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 6 Only one bidirectional arrow is used to represent the bus in the figure, but it does not mean that there is only one bus or only one type of bus.

[0112] The memory 61 is configured to store a program, and the processor 60 executes the program after receiving an execution instruction. The method executed by the device for defining the flow process according to any one of the embodiments of the present application can be applied to the processor 60 or implemented by the processor 60.

[0113] The processor 60 can be an integrated circuit chip with processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 60 or an instruction in the form of software. The processor 60 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), and the like; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory 61, and the processor 60 reads the information in the memory 61 and combines the hardware to complete the steps of the above method.

[0114] Embodiment four:

[0115] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is run by a processor to execute the steps of the method described in the above embodiment one.

[0116] In addition, in the description of the embodiments of the present application, unless specifically defined and limited, the terms "mount", "connect", "connection" should be understood as a broad meaning, for example, can be fixedly connected, can be detachably connected, or integrally connected; can be mechanically connected, can be electrically connected; can be directly connected, can be indirectly connected through an intermediate medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0117] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0118] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interfaces, devices or units, and can be electrical, mechanical or other forms.

[0119] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0120] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0121] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the same. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that any person skilled in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some of the technical features, within the technical scope disclosed by the present application. The modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A ConvNeXt-based remote sensing image vegetation classification method, characterized in that, The method comprises the following steps: acquiring sample remote sensing image data, and expanding the sample remote sensing image data by using an Fmix mixed sample data enhancement algorithm to obtain a sample data set; training a vegetation classification model by using the sample data set to obtain a target vegetation classification model, wherein the vegetation classification model comprises a feature encoder constructed based on ConvNeXt and a decoder constructed based on UperNet; after acquiring remote sensing image data to be classified, inputting the remote sensing image data to be classified into the target vegetation classification model to obtain an initial classification result; performing adjacent class fusion processing on target objects in the initial classification result, and performing contour simplification processing on graph patches in the class fusion processing result to obtain a target classification result, wherein the target objects include patches and holes with areas less than a preset threshold; the adjacent class fusion processing on the target objects in the initial classification result comprises: inputting each class in the initial classification result into a corresponding channel; calculating the area of the connected domain of each channel, and determining the graph patches and holes with an area less than the preset threshold in each channel as the target objects; constructing a first mask based on the target objects, and removing the graph patches and holes with an area greater than the preset threshold in the first mask to obtain a second mask; performing adjacent class fusion processing on the target objects in the second mask data based on a preset vegetation class order.

2. The method of claim 1, wherein, The method for expanding the sample remote sensing image data by using the Fmix mixed sample data enhancement algorithm to obtain the sample data set comprises the following steps: performing artificial interpretation and labeling on the vegetation types in the sample remote sensing image data to obtain target remote sensing image data; segmenting the target remote sensing image data according to a preset size to obtain an initial sample data set; expanding the initial sample data set by using the Fmix mixed sample data enhancement algorithm to obtain a sample data set.

3. The method of claim 1, wherein, The method for training the vegetation classification model by using the sample data set to obtain the target vegetation classification model comprises the following steps: dividing the sample data set into a training set and a validation set; a calculation step of inputting a preset number of sample data in the training set into the vegetation classification model and calculating the sum of cross-entropy losses; an optimization step of performing parameter optimization on the vegetation classification model based on the sum of cross-entropy losses and an AdamW function to obtain an initial vegetation classification model; a first execution step of determining the initial vegetation classification model as the vegetation classification model, repeatedly executing the calculation step and the optimization step until the number of repeated executions reaches a first preset number to obtain an intermediate vegetation classification model; a second execution step of determining the intermediate vegetation classification model as the vegetation classification model, determining the validation set as the training set, repeatedly executing the calculation step, the optimization step and the first execution step until the number of repeated executions reaches a second preset number, and determining the intermediate vegetation classification model with the maximum intersection over union as the target vegetation classification model.

4. The method of claim 1, wherein, The method for performing contour simplification processing on the graph patches in the class fusion processing result comprises the following steps: extracting an inner boundary and an outer boundary of a map spot in the category fusion processing result; performing boundary point simplification processing on the inner boundary and the outer boundary by using a Visvalingam-Whyatt algorithm.

5. A ConvNeXt-based remote sensing image vegetation classification device, characterized in that, comprise: an acquisition unit, a training unit, a classification unit, and an optimization unit, wherein the acquisition unit is configured to acquire sample remote sensing image data, and expand the sample remote sensing image data by using an Fmix mixed sample data enhancement algorithm to obtain a sample data set; the training unit is configured to train a vegetation classification model by using the sample data set to obtain a target vegetation classification model, wherein the vegetation classification model comprises a feature encoder constructed based on a ConvNeXt and a decoder constructed based on a UperNet; the classification unit is configured to, after acquiring remote sensing image data to be classified, input the remote sensing image data to be classified into the target vegetation classification model to obtain an initial classification result; the optimization unit is configured to perform adjacent category fusion processing on a target object in the initial classification result, and perform contour simplification processing on a map spot in the category fusion processing result to obtain a target classification result, wherein the target object comprises a patch and a hole with an area less than a preset threshold; the optimization unit is specifically configured to: input each category in the initial classification result into a corresponding channel; calculate the area of a connected domain of each channel, and determine a map spot and a hole in each channel with an area of the connected domain less than the preset threshold as the target object; construct a first mask based on the target object, and remove a map spot and a hole with an area greater than the preset threshold in the first mask to obtain a second mask; perform adjacent category fusion processing on the target object in the second mask data based on a preset vegetation category order.

6. The apparatus of claim 5, wherein, the acquisition unit is configured to: perform artificial interpretation and annotation on the vegetation types in the sample remote sensing image data to obtain target remote sensing image data; segment the target remote sensing image data according to a preset size to obtain an initial sample data set; expand the initial sample data set by using the Fmix mixed sample data enhancement algorithm to obtain a sample data set.

7. The apparatus of claim 5, wherein, the training unit is configured to: divide the sample data set into a training set and a validation set; a calculation step of inputting a preset number of sample data in the training set into the vegetation classification model and calculating the sum of cross-entropy losses; an optimization step of performing parameter optimization on the vegetation classification model based on the sum of cross-entropy losses and an AdamW function to obtain an initial vegetation classification model; a first execution step of determining the initial vegetation classification model as the vegetation classification model, repeatedly executing the calculation step and the optimization step until the number of repeated executions reaches a first preset number of times to obtain an intermediate vegetation classification model; A second execution step is to determine the intermediate vegetation classification model as the vegetation classification model and determine the verification set as the training set, repeatedly execute the calculation step, the optimization step and the first execution step until the number of repetitions reaches a second preset number, and determine the intermediate vegetation classification model with the maximum intersection over union as the target vegetation classification model.

8. An electronic device, comprising: A computer program product comprising a memory for storing a program supporting a processor to execute the method of any one of claims 1 to 4, and the processor configured to execute the program stored in the memory.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that A computer program which, when executed by a processor, performs the steps of the method of any one of claims 1 to 4.

Images