A method for distinguishing the characteristics of bald spots in plateau pika-type degraded meadows

Abstract

The invention discloses a method for distinguishing the characteristics of bald spots in plateau pika-type degraded meadows. Based on the image data of high-cold degraded meadows remotely sensed by unmanned aerial vehicles, the semantic segmentation-related method is used to classify and predict different types of bald spots in high-cold degraded meadows. Different types of patches are annotated according to expert experience, and each image collected by the unmanned aerial vehicle is annotated; using the deep learning framework and the idea of ​​transfer learning, the network models Resnet50, MobileNetV2 and Xception are used as the backbone feature extraction network, and the convolutional neural network PSPNet and Deeplabv3+ are respectively used to train the data set; the expected purpose is achieved through experiments. PSPNet cooperates with the backbone feature extraction network MobileNetV2 to perform best on this data set, achieving good results of 87.14% precision, 86.57% recall, 86.85% f1‑score, 99.10% accuracy, and 81.40% frequency-weighted intersection-and-union. The invention provides a theoretical basis and scientific support for the intelligent diagnosis and rapid repair of high-cold degraded meadows, and lays a foundation for the application of intelligent technology in regional ecological environment.

Description

Technical Field

[0001] The invention relates to the technical field of plateau pika type degraded meadow grade discrimination, and in particular to a method for discriminating bald spot characteristics of plateau pika type degraded meadow. Background Art

[0002] The Qinghai-Tibet Plateau is the "third pole of the earth". The current mainstream cognition is that the degradation of its grassland is due to the combined influence of natural factors and human activities. Natural factors include pika activities and climate change; human activities mainly include overgrazing, excessive digging and mining. Among them, the plateau pika is the dominant rodent burrowing animal in the alpine meadows of the Qinghai-Tibet Plateau. Its feeding and burrowing behaviors cause a large number of soil bare spots to be embedded in the grassland vegetation, resulting in spatial heterogeneity of vegetation distribution. In addition, the study found that the plateau pika tends to live in areas with high soil moisture content, rich vegetation, and suitable land surface temperature during the growing season. At the same time, pikas gnaw underground rhizomes to destroy the grass layer. In this process, the soil structure is strongly changed, the permafrost layer becomes thinner, and the spread of "black soil beach" is catalyzed. "Black soil beach" is a name for large areas of secondary bare land formed after the severe degradation of alpine meadows in the range of 3600-4500 meters above sea level on the Qinghai-Tibet Plateau. Rodent damage is the main factor in the formation of bald spots, which accelerates the formation process of "black soil beach". However, the alpine meadows on the Qinghai-Tibet Plateau have low system vitality, are fragile and have great inertia. Once destroyed, it is extremely difficult to manage. In short, the occurrence of rodent damage will further deteriorate the grassland ecological environment. Therefore, clarifying the life trajectory of plateau pikas in alpine meadows and the resulting bald spots on the meadows has a significant impact on the formulation of measures and methods to restore the alpine meadow ecosystem.

[0003] UAV remote sensing technology has the advantages of wide shooting range, high efficiency and high resolution in the observation process, and has broad application prospects in various industries. The grassland is vast and the weather often changes. In this case, the grassland information can be quickly and accurately identified by drones, and detailed and accurate experimental data can be obtained. In traditional methods, manual measurement and empirical judgment are often used to determine the identification and classification of bald spots in alpine pika-type degraded meadows based on UAV remote sensing. Obviously, the classification results after using this method often rely on rich expert experience, and are time-consuming and labor-intensive, requiring a lot of time and manpower. With the deepening of deep learning theory and the development of computer-related technologies, computer vision and images are applied to the grassland field. Based on the semantic segmentation of convolutional neural networks, the identification and classification of bald spots in degraded meadows can be conveniently, quickly and accurately processed, and the typical characteristics of bald spot types caused by plateau pika interference can be further clarified, which is of great significance for the intelligent diagnosis and precise restoration of alpine degraded grasslands.

[0004] Semantic segmentation is an accurate pixel-level image classification task, which means accurately classifying each pixel, giving label information of similar feature areas and determining their location information. In the field of computer vision, semantic segmentation is an important research direction with very high practical value in life. For example, in the judgment of tumors in medical images; in the prediction of pedestrians and vehicles in unmanned driving; in remote sensing images, the segmentation of environments such as houses, roads and lakes. Traditional semantic segmentation algorithms are based on surface features such as texture, edge, and color, and mainly include common segmentation algorithms based on thresholds, graph theory, clustering, and regions. With the complexity of segmentation scenes and the improvement of segmentation requirements, deep learning has gradually been introduced into semantic segmentation tasks, and convolutional neural networks (CNNs) have become an important means of extracting image features, achieving efficient and accurate results. The basic structure of CNN: input layer, convolution layer, pooling layer, fully connected layer and output layer. The input feature map obtains the local feature map through the convolution operation in the convolution layer and the nonlinear fitting of the activation function; the pooling layer extracts spatial invariant features through downsampling operations to reduce the number of neurons; the fully connected layer obtains the global information of the image.

[0005] In summary, the research on semantic segmentation-based detection and recognition of bald spot characteristics in alpine pika-type degraded meadows is still in a new realm that needs to be explored. Summary of the invention

[0006] In order to solve the above technical problems, the present invention uses semantic segmentation and correlation methods based on the image data of alpine degraded grassland remote sensing by unmanned aerial vehicles to classify and predict different types of bald spots in alpine degraded grassland. The purpose is to provide a method for distinguishing the characteristics of bald spots in plateau pika-type degraded meadows.

[0007] The present invention provides a method for distinguishing the characteristics of bald spots in plateau pika-type degraded meadows, which specifically comprises the following steps:

[0008] S1, Data source and acquisition: The experimental site recorded and observed the changing patterns of different types of patches in different plots under two disturbances, simulated grazing and plateau pika, and then evaluated the grassland degradation. The experimental site is 30 meters * 25 meters in size, with a total of 27 plots. The grassland images are taken by drones from high altitudes, once a month, with two photos taken for each plot each time.

[0009] S2, classification of the data set: The data set includes grassland degradation bald spots caused by simulated grazing and disturbance by plateau pikas. Based on the experience of experts in the field, the status of the meadows shown in the remote sensing images is divided into four types, which are described as active areas, inactive areas, recovery areas and alpine meadow background areas;

[0010] S3, image preprocessing: In order to obtain standardized data, the original image is processed through the editing functions of Photoshop and the photos in the Windows system. The specific operations include: rotation, cropping, spot removal and light adjustment;

[0011] S4, data set annotation: the preprocessed images are combined with the bald spots classified by the data set, and the polygon annotation method in the software labelme is used to finally generate a josn file to save the required location information of each area;

[0012] S5, data set preprocessing: convert the annotated josn file into a label image in png format through labelme related programs; organize the data set in voc format, and finally normalize the data set to 512*512 size;

[0013] S6, data enhancement: select images with fewer classification labels for data enhancement, including horizontal and vertical flipping of images, counterclockwise rotation of 90 degrees, counterclockwise rotation of 180 degrees, and counterclockwise rotation of 270 degrees;

[0014] S7, build a deep learning framework: use the semantic segmentation algorithm to train the alpine pika type degraded meadow dataset, mainly using PSPNet with the backbone feature extraction network MobileNetV2 and Resnet50, Deeplabv3+ with the backbone feature extraction network MobileNetV2 and Xception; the optimizer selects the Adam (Adaptive Moment Estimation) algorithm to dynamically adjust the learning rate gradient descent to ensure the stability of the parameters; in addition, in order to obtain a more precise segmentation result, the downsampling factor is selected to 8 times; in the training process, the cross entropy loss function is used as a reference for observing the applicability of the algorithm on the dataset and whether to stop the iteration;

[0015] S8, indicator evaluation: the data set is randomly divided into training set: validation set = 8:2 ratio, and the test set and validation set are combined into one for comprehensive testing, and the precision (Precision), recall rate (Recall), f1-score, accuracy (Accuracy) and frequency weighted intersection over union (FWIoU) are used to evaluate the effect; in addition, in order to reduce the problem of imbalanced classification labels in the data set, that is, the uneven number of each patch, the evaluation indicators are divided into weight proportions, and different weights are calculated for each category, that is, recovery area: active area: inactive area: background area = 1:1:1:7; the indicator evaluation results are displayed through the confusion matrix.

[0016] Furthermore, in step S1, images of four months from June to September in four years are obtained, and unqualified images and images that cannot be used for experiments are removed.

[0017] Furthermore, in step S2, the active area is the area where plateau pikas frequently move, where the soil is exposed and loose and few plants grow; the inactive area is the area formed when plateau pikas stop disturbing, where there are a small amount of miscellaneous grass plants; the restoration area is the area where there is no plateau pika activity and natural ecological restoration is being carried out, and generally the vegetation coverage rate is less than 60%; the alpine meadow area is the alpine meadow background area.

[0018] Compared with the existing technology, the present invention has the following beneficial effects:

[0019] Based on the image data of alpine degraded meadows remotely sensed by unmanned aerial vehicles, the present invention adopts semantic segmentation-related methods to realize the classification and prediction of different types of bald spots in alpine degraded meadows. First, different types of patches are annotated according to expert experience, and each image collected by the unmanned aerial vehicle is annotated as active area, inactive area, recovery area and alpine meadow background area; secondly, using the deep learning framework and the idea of ​​transfer learning, the pre-trained network models Resnet50, MobileNetV2 and Xception are used as the backbone feature extraction network, and the convolutional neural network PSPNet and Deeplabv3+ are used to train the data set to complete the semantic segmentation task; finally, the expected purpose is achieved through experiments. Among them, PSPNet with the backbone feature extraction network MobileNetV2 performs best on this data set, achieving the best results of 87.14% precision, 86.57% recall, 86.85% f1-score, 99.10% accuracy and 81.40% frequency-weighted intersection-over-intersection ratio. Therefore, the present invention provides a theoretical basis and scientific support for the intelligent diagnosis and rapid restoration of alpine degraded meadows, and at the same time lays a foundation for the application of intelligent technology in the regional ecological environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of the method of the present invention;

[0021] Figure 2 Four types of regions classified for the dataset; in the figure, A, active region; B, inactive region; C, recovery region; D, alpine meadow background region;

[0022] Figure 3 The images are abnormally exposed. In the figure, A is an image with too low brightness; B is an image with too high brightness;

[0023] Figure 4 Annotate the process diagram for the dataset;

[0024] Figure 5 This is a preprocessed image of the dataset in voc format. A is an unprocessed image; B is a processed image in voc format;

[0025] Figure 6 It is the Epoch-loss graph of Algorithm 1;

[0026] Figure 7 Epoch-loss graph of Algorithm 2;

[0027] Figure 8 Epoch-loss graph of Algorithm 3;

[0028] Fig. 9 Epoch-loss graph of Algorithm 4;

[0029] Fig.10 Epoch-loss graph of Algorithm 5;

[0030] Fig.11 The f1-score result graphs of the five algorithms are shown below;

[0031] Fig.12 This is a prediction result diagram; in the diagram, A, algorithm 1; B, algorithm 2; C, algorithm 3; D, algorithm 4; E, algorithm 5. DETAILED DESCRIPTION

[0032] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Example 1

[0034] A method for distinguishing the characteristics of bald spots in plateau pika-type degraded meadows

[0035] 1. Hardware environment

[0036] Table 1 Hardware environment

[0037]

[0038] 2. Software environment

[0039] The programming language is Python, and the deep learning framework is Pytorch. The specific software environment configuration is shown in Table 2.

[0040] Table 2 Software environment

[0041]

[0042] 3. Data source and acquisition

[0043] The dataset comes from drone aerial images of grassland plots in a test site in Henan County, Huangnan Tibetan Autonomous Prefecture, Qinghai Province. The test site records and observes the change patterns of different types of patches in different plots under two disturbances, simulated grazing and plateau pikas, and then evaluates the degradation of the grassland. The size of the test site is 30 meters * 25 meters, with a total of 27 plots. The grassland images are taken by drones from high altitude, once a month, and two photos are taken for each plot each time. Images of four months from June to September each year from 2018 to 2021 were obtained, and unqualified images and images that cannot be used for experiments were removed, totaling 756 original images;

[0044] 4. Classification of Datasets

[0045] The dataset includes grassland degradation bald spots caused by simulated grazing and disturbance by plateau pikas. Based on the experience of experts in the field, the status of meadows shown in remote sensing images is divided into four types, namely active areas, inactive areas, recovery areas and alpine meadow background areas. Among them, active areas are areas where plateau pikas frequently move, with loose and exposed soil and few plants growing; inactive areas are areas formed when plateau pikas stop disturbing, with a small amount of miscellaneous grass plants; recovery areas are areas where there is no plateau pika activity and natural ecological restoration is being carried out, with a general vegetation coverage rate of less than 60%; alpine meadow areas are alpine meadow background areas (see Appendix for details). Figure 2 ).

[0046] 5. Image preprocessing

[0047] Due to the influence of shooting angle and environment and noise from the tools used by the staff, the pictures may contain white spots or part of the image may be skewed or exposed abnormally. In order to obtain standardized data, the original images were processed using the editing functions of Photoshop and the photos in the Windows system. The specific operations include: rotation, cropping, spot removal and light adjustment. Figure 3 The abnormally exposed image shown above has its brightness and contrast adjusted up (down) in batches through the program.

[0048] 6. Dataset Annotation

[0049] Combine the preprocessed image with the bald spots classified by the dataset, and use the polygon annotation method in the software labelme to finally generate a josn file to save the location information of each area. Labelme is a cross-platform image annotation tool based on the graphical interface QT written in python. It can be used to annotate common visual tasks such as classification, detection, segmentation, key points, etc. It supports export in formats such as VOC and COCO. Red represents inactive areas, green represents restored areas, and yellow represents active areas. The specific annotation process is shown in the attached figure. Figure 4 shown.

[0050] 7. Dataset preprocessing

[0051] The josn file formed by the annotation is converted into a label image in png format through the labelme related program; the data set is organized in voc format (see the attached Figure 5 ), and finally normalize the data set to 512*512 size;

[0052] 8. Data Augmentation

[0053] In order to improve the generalization and robustness of the algorithm model used in the experiment, expand the scale of the data set, and reduce the degree of overfitting, data enhancement is necessary; however, considering the impact of unbalanced label data, especially the need to combat the large area of ​​the normal alpine meadow background area, we selected pictures with a small number of classification labels for data enhancement. The operations include horizontal and vertical flipping of the image, counterclockwise rotation of 90 degrees, counterclockwise rotation of 180 degrees, and counterclockwise rotation of 270 degrees. This data enhancement method can effectively avoid the possibility of random rotation leading to the reduction or disappearance of annotation information without reducing the area of ​​the annotation label. After data enhancement, the data set size reached 2036 pictures.

[0054] 9. Build a deep learning framework

[0055] The semantic segmentation algorithm was used to train the alpine pika-type degraded meadow dataset. PSPNet was mainly used with the backbone feature extraction network MobileNetV2 and Resnet50, and Deeplabv3+ was used with the backbone feature extraction network MobileNetV2 and Xception. The optimizer selected the Adam (Adaptive Moment Estimation) algorithm to dynamically adjust the learning rate gradient descent to ensure the stability of the parameters. In addition, in order to obtain more precise segmentation results, the downsampling factor was selected to 8 times. In the training process, the cross entropy loss function was used as a reference for observing the applicability of the algorithm on the dataset and whether to stop the iteration. The algorithm and other parameter settings used are shown in Table 3.

[0056] Table 3 Algorithm parameters

[0057]

[0058] 10. Index evaluation

[0059] The dataset was randomly divided into training set and validation set in the ratio of 8:2, and the test set and validation set were combined into one. Therefore, 20% of the 2036 images, i.e., 408 images, were comprehensively tested, and the precision, recall, f1-score, accuracy, and frequency-weighted intersection-over-union (FWIoU) were used to evaluate the effect. In addition, in order to reduce the problem of imbalanced classification labels in the dataset, i.e., the uneven number of each patch, the evaluation indicators were divided into weight proportions, and different weights were given to each category, i.e., recovery area: active area: inactive area: background area = 1:1:1:7. The indicator evaluation results were displayed through the confusion matrix.

[0060] The structure of the confusion matrix is ​​shown in Table 4. For the restored area, the evaluation indicators are explained as follows:

[0061] TP (True Positive): True positive refers to the positive tuple correctly classified by the classifier. That is, the type detected and segmented is the restoration area, which is consistent with the labeled type. TN (True Negative): True negative refers to the negative tuple correctly classified by the classifier, that is, the type detected and segmented is not the restoration area, and is consistent with the labeled type. FP (False Positive): False positive refers to the negative tuple that is mistakenly labeled as a positive tuple. That is, the type detected and segmented is the restoration area, which is inconsistent with the labeled type. FN (False Negative): False negative refers to the positive tuple that is mistakenly labeled as a negative tuple. That is, the type detected and segmented is not the restoration area, but the labeled type is the restoration area.

[0062] Table 4 Confusion matrix

[0063]

[0064] Precision: It is a measure of accuracy, that is, the percentage of tuples marked as positive that are actually positive. The calculation formula is as follows:

[0065]

[0066] Recall: Also known as sensitivity, it is the percentage of positive tuples marked as positive. The calculation formula is as follows:

[0067]

[0068] f1-score: Since Precision and Recall are contradictory, usually, when Precision is high, Recall tends to be low; and when Precision is low, Recall tends to be high. When the classification confidence is high, Precision tends to be high; when the classification confidence is low, Recall tends to be high. Therefore, the F measure was proposed, which is the weighted harmonic mean of Precision and Recall. The calculation formula is as follows:

[0069]

[0070] Accuracy: The percentage of tuples correctly classified by the classifier, reflecting the correct recognition of various tuples by the classifier. The calculation formula is as follows:

[0071]

[0072] Frequency-weighted intersection over union (FWIoU): IOU stands for Intersection over Union, which is the ratio of intersection over union. In the problem of semantic segmentation, the IoU of a single class is the ratio of the true label and the predicted value of that class. The frequency-weighted IoU is obtained by setting weights according to the frequency of each class, multiplying the weights by the IoU of each class and summing them up. The calculation formula is as follows:

[0073]

[0074] 11. Result Analysis

[0075] In this study, five algorithms with different parameters were set for the alpine pika degraded meadow dataset. During the training process, the cross entropy loss function was used as a reference to observe the applicability of the algorithm on the dataset and whether to stop the iteration. The details of the loss value change with the number of iterations are shown in the attached figure. Figure 6-10 shown.

[0076] From the attached Figure 6 , 8 In Figures 1 and 2, all algorithms that use MobileNetV2 as the backbone feature extraction network have a significant decrease in loss value in the early stage of iterative training; while Resnet50 is used as the backbone feature extraction network, as shown in the attached figure. Figure 7 The loss value of Algorithm 2 shown in the figure has been decreasing slowly during the iterative training process; Xception is used as the backbone feature extraction network, as shown in the attached figure. Fig.10In the algorithm 5 shown, the loss value has a significant decrease in the early stage of iterative training, but the change is relatively slow in the later stage. For Algorithm 1 and Algorithm 3, compared with the two convolutional neural networks, the loss value of PSPNet changes more rapidly during training; for Algorithm 3 and Algorithm 4, both use the same convolutional neural network and backbone feature extraction network, but set a larger batch size and learning rate for gradient descent, which can speed up the training speed and reduce the training time.

[0077] Table 5 shows the calculation results of the evaluation indicators for the test set. From Table 5, it can be concluded that PSPNet with the backbone feature extraction network MobileNetV2 is the most suitable for this dataset. The precision, recall and frequency weight intersection and union values ​​are all good. The second best indicator is Deeplabv3+ with the backbone feature extraction network MobileNetV2. The experimental effect of PSPNet with the backbone feature extraction network Resnet50 is the least outstanding for this grassland dataset. Of course, all five experimental algorithms have high accuracy.

[0078] Table 5 Experimental results of five algorithms

[0079]

[0080] Attached Fig.11 The following is the evaluation result of the comprehensive index f1-score of the five algorithms on the Grassland dataset. Algorithm 1, PSPNet with the backbone feature extraction network MobileNetV2, has the highest f1-score of 86.85%. Algorithm 2, PSPNet with the backbone feature extraction network Resnet50, has the lowest f1-score of 77.59%, which may be caused by the contradiction between the too deep network layers of Resnet50 and the small size of the dataset.

[0081] Attached Fig.12 The following are the prediction graphs of the algorithms used in the experiment for each image. It can be seen that the convolutional neural network PSPNet combined with the backbone feature extraction network MobileNetV2 is indeed the most applicable to this dataset. However, the recognition effect of the algorithm using the network Resnet50 and Xception for backbone feature extraction needs to be improved.

[0082] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for distinguishing the characteristics of bald spots in plateau pika-type degraded meadows, characterized in that: The specific steps include: S1, Data source and acquisition: The experimental site recorded and observed the changing patterns of different types of patches in different plots under two disturbances, simulated grazing and plateau pika, and then evaluated the grassland degradation. The experimental site is 30 meters * 25 meters in size, with a total of 27 plots. The grassland images are taken by drones from high altitudes, once a month, with two photos taken for each plot each time. S2, classification of the data set: The data set includes grassland degradation bald spots caused by simulated grazing and disturbance factors of plateau pikas. According to the experience of experts in the field, the status of the meadows shown in the remote sensing images is divided into four types, which are described as active areas, inactive areas, recovery areas and alpine meadow background areas; S3, image preprocessing: In order to obtain standardized data, the original image is processed through the editing functions of Photoshop and the photos in the Windows system. The specific operations include: rotation, cropping, spot removal and light adjustment; S4, data set annotation: the preprocessed images are combined with the bald spots classified by the data set, and the polygon annotation method in the software labelme is used to finally generate a josn file to save the required location information of each area; S5, data set preprocessing: convert the annotated josn file into a label image in png format through labelme related programs; organize the data set in voc format, and finally normalize the data set to 512*512 size; S6, data enhancement: select images with fewer classification labels for data enhancement, including horizontal and vertical flipping of images, counterclockwise rotation of 90 degrees, counterclockwise rotation of 180 degrees, and counterclockwise rotation of 270 degrees; S7, build a deep learning framework: use the semantic segmentation algorithm to train the alpine pika type degraded meadow dataset, use PSPNet with the backbone feature extraction network MobileNetV2 and Resnet50, and Deeplabv3+ with the backbone feature extraction network MobileNetV2 and Xception; the optimizer selects the Adam algorithm to dynamically adjust the learning rate gradient descent to ensure the stability of the parameters; in addition, in order to obtain more precise segmentation results, the downsampling factor is selected to 8 times; in the training process, the cross entropy loss function is used as a reference for observing the applicability of the algorithm on the dataset and whether to stop iteration; S8, indicator evaluation: the dataset is randomly divided into training set: validation set = 8:2 ratio, and the test set and validation set are combined into one for comprehensive testing, and the precision, recall, f1-score, accuracy and frequency-weighted intersection over union (FWIoU) are used to evaluate the effect; in addition, in order to reduce the problem of imbalanced classification labels in the dataset, that is, the uneven number of each patch, the evaluation indicators are divided into weight proportions, and different weights are given to each category, that is, recovery area: active area: inactive area: background area = 1:1:1:7; the indicator evaluation results are displayed through the confusion matrix.

2. The method for distinguishing the characteristics of bald spots in plateau pika-type degenerate meadows according to claim 1 is characterized in that: In step S1, images of four months from June to September in four years are obtained, and unqualified images and images that cannot be used for experiments are removed.

3. The method for distinguishing the characteristics of bald spots in plateau pika-type degenerate meadows according to claim 1, characterized in that: In step S2, the active area is the area where the plateau pikas frequently move, where the soil is exposed and loose, and where few plants grow; the inactive area is the area where the plateau pikas stop disturbing, where there are a small amount of miscellaneous grass plants; the restoration area is the area where there is no plateau pika activity and where natural ecological restoration is being carried out, where the vegetation coverage rate is less than 60%; Alpine meadow area, that is, alpine meadow background area.

Images