An image recognition monitoring method based on UAV remote sensing technology

Abstract

The present invention discloses an image recognition monitoring method based on unmanned aerial vehicle remote sensing technology, which belongs to the field of deep learning technology. The present invention adopts unmanned aerial vehicle and hyperspectral imager to collect grassland object images, obtain original image data of target area; pre-process the collected original image to obtain pre-processed data set; classify grassland objects based on vegetation index, perform time series information analysis and cross-scale analysis on the pre-processed data set; make grassland object classification data set based on grassland object characteristics and deep learning classification data set production rules, and classify grassland images; make grassland object segmentation data set based on the pre-processed data set, and construct grassland object segmentation model based on deep learning segmentation method and grassland object definition; perform weighted fusion and expert voting on the processing results to obtain the final segmentation results of grassland objects, and calculate vegetation coverage according to the results.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to an image recognition monitoring method based on unmanned aerial vehicle remote sensing technology. Background Art

[0002] As one of the important ecosystems, grassland plays an important role in maintaining ecological balance, supporting biodiversity, regulating climate and providing resources for human activities. However, grassland ecosystems are susceptible to factors such as climate change and human activities, leading to grassland degradation and desertification. Therefore, timely and accurate monitoring of grassland features and understanding of changes in grassland ecosystems are of great significance for grassland protection and management.

[0003] Traditional grassland feature classification and monitoring methods mainly rely on manual labeling and analysis, which is inefficient and cannot meet the needs of large-scale, real-time monitoring. Some existing automated classification methods are not accurate enough when dealing with complex grassland features, and are prone to misclassification and missed classification, affecting the reliability of monitoring results. Grassland ecosystems have significant seasonal and long-term changes. Existing methods lack in-depth analysis of these temporal changes and cannot effectively monitor the dynamic changes of grasslands. Summary of the invention

[0004] The purpose of the present invention is to provide an image recognition monitoring method based on unmanned aerial vehicle remote sensing technology to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] An image recognition monitoring method based on unmanned aerial vehicle remote sensing technology, characterized in that the method comprises the following steps:

[0007] S100, using a UAV and a hyperspectral imager to collect grassland features images and obtain original image data of the target area; preprocessing the collected original images according to the flight altitude and image type to obtain a preprocessed data set;

[0008] S200, classifying grassland features based on vegetation index using the preprocessed data set to achieve classification of bare soil and vegetation communities; performing time series information analysis, calculating vegetation index at different time points using time series data, and applying trend analysis to model; performing cross-scale analysis, extracting and analyzing features of data at different scales using multi-scale analysis methods, and obtaining the distribution and change patterns of grassland features at different spatial scales;

[0009] S300, using the preprocessed data set to produce a grassland feature classification data set based on grassland feature characteristics and deep learning classification data set production rules; building, training, verifying and testing a deep learning model, and using the optimal classification model with optimized parameters to classify grassland images to obtain grassland feature classification results;

[0010] S400, using the preprocessed data set to create a grassland feature segmentation data set; constructing a grassland feature segmentation model based on the deep learning segmentation method and the grassland feature definition, training, verifying and testing the model, and using the optimal segmentation model to segment the grassland image after comparative analysis to obtain a grassland feature segmentation result;

[0011] S500, weighted fusion and expert voting are performed on the processing results of S200-S400, and the category of each pixel is determined by using the weighted fusion and expert voting scheme to obtain the final segmentation results of each feature in the grassland, and the vegetation coverage is calculated according to the final segmentation results of each feature in the grassland.

[0012] According to step S100, a drone and a hyperspectral imager are used to collect images of the target grassland area several times, covering different seasons, to obtain original image data of the target area in multiple time phases; according to the flight altitude and the image type, the collected original images are spliced, cropped and geometrically corrected to eliminate geometric distortion and splicing errors, and radiation correction and atmospheric correction are performed; histogram equalization is applied to improve the contrast and clarity of the image to obtain a preprocessed data set.

[0013] In step S200, the grassland features are classified based on the vegetation index NDVI to obtain the category of each pixel, which is mainly divided into bare soil and vegetation communities. A grassland feature classification map is generated to mark the spatial distribution of bare soil and vegetation communities. NDVI data of different periods are calculated and trend analysis is performed to obtain the seasonal changes and long-term trends of the grassland ecosystem. A time series NDVI curve graph is generated to show the changes in the vegetation index over time and reveal the dynamic changes of the grassland ecosystem. Using a multi-scale analysis method, feature extraction and analysis are performed on data of different scales to obtain the distribution and change laws of grassland features at different spatial scales. Distribution maps of grassland features at different scales are generated to show the spatial distribution characteristics of features at each scale.

[0014] According to step S200, the vegetation index NDVI is calculated using the near infrared band NIR and the red band Red data, and the formula is as follows:

[0015]

[0016] NDVI values ​​range from -1 to 1, with larger values ​​indicating denser vegetation cover and smaller values ​​indicating bare soil or other non-vegetated features;

[0017] According to the range of NDVI values, thresholds were set to distinguish bare soil and vegetation communities. NDVI calculation and threshold determination were applied to each pixel to obtain preliminary bare soil and vegetation classification results.

[0018] According to step S200, NDVI data of different seasons are obtained to form a time series data set {NDVI t}, where t represents the time point; linear trend model, polynomial trend model and exponential smoothing model are used to model the time series NDVI data respectively;

[0019] The linear trend model assumes that the NDVI value changes linearly over time, and the model form is:

[0020] NDVI t =β0+β1t+∈ t ;

[0021] Among them, β0 is the intercept, β1 is the slope, ∈ t is the error term;

[0022] The least squares method is used to estimate the model parameters β0 and β1, and the formula is as follows:

[0023]

[0024] Where T is the total number of time points, is the average value of NDVI, is the average value of the time point; according to the fitted model, the predicted NDVI value at each time point is calculated

[0025]

[0026] in, and are the predicted values ​​of β0 and β1 respectively;

[0027] Calculate the goodness of fit R of the model 2 , evaluate the explanatory power of the model:

[0028]

[0029] If the NDVI data presents a nonlinear trend, a polynomial trend model is used, specifically a quadratic polynomial trend model, in the form of:

[0030] NDVI t =β0+β1t+β2t 2 +∈ t ;

[0031] β2 is the coefficient of the quadratic term, indicating the tendency of quadratic change over time;

[0032] The least squares method is used to estimate the model parameters β0, β1, and β2, and the predicted NDVI value at each time point is calculated based on the fitted model:

[0033]

[0034] in, is the predicted value of β2; calculate the goodness of fit R 2 , evaluate the explanatory power of the model;

[0035] The exponential smoothing model is suitable for time series with random fluctuations but no obvious trend or periodicity. Specifically, a simple exponential smoothing model is used, which is in the form of:

[0036]

[0037] Where α is the smoothing parameter, 0<α<1;

[0038] The goodness of fit of each model was calculated, and the optimal model was selected. Based on the parameters and prediction results of the optimal model, the seasonal changes and long-term trends of the grassland ecosystem were revealed.

[0039] According to step S200, grassland object data of different spatial scales are collected, and features are extracted from data of different scales, wherein the features include: texture features, shape features, spectral features and spatial features, wherein the texture features describe the roughness and pattern of the grassland object surface, the shape features describe the geometric shape of the grassland object, the spectral features reflect the reflectivity of the grassland object in different spectral bands, and the spatial features describe the spatial distribution and relative position relationship of the grassland object; the fractal dimension of the grassland object is calculated to reflect the complexity of the object at different scales, and the formula is as follows:

[0040]

[0041] Among them, D is the fractal dimension, N(∈) is the minimum number of units covering the features when the scale is ∈; the changing trend of the fractal dimension can reveal the changes in the structural complexity of grassland features from small scale to large scale. By calculating and comparing the fractal dimensions of different regions, the differences in structural complexity of different regions can be identified; the terrain index at different scales is calculated, the fractal dimensions and terrain index at different scales are integrated, and a multi-scale analysis model is established to show the distribution and change laws of grassland features at different scales;

[0042] Integrate multi-scale features, establish a multi-scale analysis model, and explore the relationships and dynamic changes of grassland features at different spatial scales.

[0043] In step S300, a deep learning model is built and trained. The model performs well on the validation set and the test set, with high accuracy and few misclassifications and missed classifications. A classification result map is generated to show the spatial distribution of vegetation, bare soil, and other objects.

[0044] According to step S300, the categories of the grassland object classification data set include vegetation, bare soil and others, and the Canny edge detection is used to identify the features therein; based on the Canny edge detection result, the unclear or erroneous parts are manually corrected, and the extracted features are annotated to form a grassland object classification data set, and each image or image segment needs to have a corresponding label; the annotated data set is divided into a training set, a validation set and a test set;

[0045] Use the deep learning framework TensorFlow to build a neural network model, define the number of network layers, the number of neurons in each layer, the activation function, and the loss function; input the training set data into the model for training. During the training process, update the model parameters through forward propagation and back propagation to minimize the loss function; increase the diversity of training data through random cropping, rotation, and flipping to prevent model overfitting.

[0046] Optimize model performance by adjusting hyperparameters. Use grid search to find the optimal hyperparameter combination, including learning rate, batch size, and regularization parameters; use validation set data to evaluate the performance of the model during training, adjust the model structure and hyperparameters based on the validation results, and use cross-validation methods to ensure the generalization ability of the model; evaluate the performance of the final model on the test set, and calculate the classification accuracy, recall rate, and F1 score; deploy the trained model to actual applications, classify new grassland features, and integrate it into the image processing system of the drone to achieve real-time classification.

[0047] In step S400, a grassland feature segmentation model is constructed and trained. The model performs well on the validation set and the test set, and can accurately segment different types of features. A segmentation result map is generated, showing in detail the spatial distribution of soil mounds, secondary bare land, restored patches, rat holes and other features.

[0048] According to step S400, the categories of the grassland feature segmentation dataset include earth mounds, secondary bare land, restored patches, rat holes and others; the image is annotated using a semi-automatic tool, and each pixel needs to have a corresponding label; the annotated dataset is divided into a training set, a validation set and a test set;

[0049] U-Net was selected as the deep learning model, and the deep learning framework TensorFlow was used to build the segmentation model. The number of model layers, the number of convolution kernels in each layer, pooling and upsampling operations were defined. The model was trained using the training set data. During the training process, the cross-entropy loss function was used to evaluate the segmentation effect of the model, and the model parameters were optimized through back propagation. Random cropping, rotation, flipping and color jittering were used to increase the diversity of training data to prevent model overfitting. Hyperparameters were adjusted to optimize model performance. The performance of the model during training was evaluated using the validation set data, and the model structure and hyperparameters were adjusted according to the validation results. The generalization ability of the model was ensured through cross-validation. The segmentation performance of the final model was evaluated on the test set, and the model was applied to actual grassland images for ground object segmentation. The model output the category of each pixel and generated a detailed segmentation result map.

[0050] According to step S500, based on the accuracy index of the classification and segmentation results, a corresponding weight is assigned to each result of S200-S400, and the weight is adjusted according to the performance of the validation set to satisfy the sum of the weights being 1, wherein the weights are respectively the weight of the vegetation index classification, the weight of the grassland feature classification result, and the weight of the grassland feature segmentation result; for each pixel, a weighted fusion result is calculated based on the classification and segmentation results from different sources and their corresponding weights, and its final category is determined by weighting and voting.

[0051] According to step S500, according to the actual situation of grassland features, expert rules are set, each classification and segmentation result is regarded as a vote, and voting is performed in combination with expert rules, and the final category is determined by the category with the most votes; for each pixel, if the weighted fusion and expert voting results are consistent, the pixel category is directly determined, and if the results are inconsistent, certain rules are introduced or further analysis is performed to determine;

[0052] The grassland object classification results and grassland object segmentation results are mapped to the geographic coordinate system using the location information recorded when the image is taken, and the position of each pixel in the image corresponds to its position in the actual geographic space; according to the grassland object classification results and grassland object segmentation results, the number of pixels of each category of the grassland object classification data set is counted, and the actual area of ​​each pixel in the geographic space is determined according to the resolution and flight altitude of the image taken by the UAV to obtain the total area of ​​the target area; the number of pixels of each category is multiplied by the actual geographic area of ​​each pixel to obtain the total area of ​​vegetation category pixels; and the vegetation coverage is calculated, and the vegetation coverage represents the ratio of the total area of ​​vegetation category pixels to the total area of ​​the target area.

[0053] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0054] The present invention adopts deep learning technology to construct and optimize grassland object classification and segmentation models, significantly improving the accuracy of classification and segmentation. Through weighted fusion and expert voting schemes, the classification results are further optimized to reduce misclassification and missed classification.

[0055] The present invention comprehensively utilizes multi-source data such as hyperspectral data, time series data and vegetation index to conduct comprehensive grassland feature monitoring. The fusion of multi-source data makes the classification and monitoring results more comprehensive and accurate.

[0056] The present invention reveals the seasonal changes and long-term trends of grassland ecosystems through trend analysis of time-series NDVI data, provides more in-depth dynamic monitoring and analysis, and helps to timely discover problems such as grassland degradation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0058] Figure 1 It is a schematic diagram of the steps of an image recognition monitoring method based on UAV remote sensing technology of the present invention;

[0059] Figure 2 It is a time series NDVI curve diagram of an image recognition monitoring method based on unmanned aerial vehicle remote sensing technology of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1 and Figure 2 , the present invention provides a technical solution:

[0062] According to one embodiment of the present invention, Figure 1 A schematic diagram of the steps of an image recognition monitoring method based on UAV remote sensing technology is shown in the figure.

[0063] An image recognition monitoring method based on unmanned aerial vehicle remote sensing technology, characterized in that the method comprises the following steps:

[0064] S100, using a UAV and a hyperspectral imager to collect grassland features images and obtain original image data of the target area; preprocessing the collected original images according to the flight altitude and image type to obtain a preprocessed data set;

[0065] S200, classifying grassland features based on vegetation index using the preprocessed data set to achieve classification of bare soil and vegetation communities; performing time series information analysis, calculating vegetation index at different time points using time series data, and applying trend analysis to model; performing cross-scale analysis, extracting and analyzing features of data at different scales using multi-scale analysis methods, and obtaining the distribution and change patterns of grassland features at different spatial scales;

[0066] S300, using the preprocessed data set to produce a grassland feature classification data set based on grassland feature characteristics and deep learning classification data set production rules; building, training, verifying and testing a deep learning model, and using the optimal classification model with optimized parameters to classify grassland images to obtain grassland feature classification results;

[0067] S400, using the preprocessed data set to create a grassland feature segmentation data set; constructing a grassland feature segmentation model based on the deep learning segmentation method and the grassland feature definition, training, verifying and testing the model, and using the optimal segmentation model to segment the grassland image after comparative analysis to obtain a grassland feature segmentation result;

[0068] S500, weighted fusion and expert voting are performed on the processing results of S200-S400, and the category of each pixel is determined by using the weighted fusion and expert voting scheme to obtain the final segmentation results of each feature in the grassland, and the vegetation coverage is calculated according to the final segmentation results of each feature in the grassland.

[0069] According to step S100, a drone and a hyperspectral imager are used to collect images of the target grassland area several times, covering different seasons, to obtain original image data of the target area in multiple time phases; according to the flight altitude and the image type, the collected original images are spliced, cropped and geometrically corrected to eliminate geometric distortion and splicing errors, and radiation correction and atmospheric correction are performed; histogram equalization is applied to improve the contrast and clarity of the image to obtain a preprocessed data set.

[0070] According to step S200, the vegetation index NDVI is calculated using the near infrared band NIR and the red band Red data, and the formula is as follows:

[0071]

[0072] NDVI values ​​range from -1 to 1, with larger values ​​indicating denser vegetation cover and smaller values ​​indicating bare soil or other non-vegetated features;

[0073] According to the range of NDVI values, a threshold is set to distinguish between bare soil and vegetation communities, and NDVI calculation and threshold determination are applied to each pixel to obtain preliminary bare soil and vegetation classification results. In this embodiment, NDVI<0.2 indicates bare soil, and NDVI≥0.2 indicates vegetation.

[0074] According to step S200, grassland feature data of different resolutions and spatial scales are collected, and features are extracted from data of different scales; the fractal dimension of grassland features is calculated to reflect the complexity of features at different scales, and the formula is as follows:

[0075]

[0076] Where D is the fractal dimension, N(∈) is the minimum number of units covering the features when the scale is ∈; calculate the terrain index at different scales, and analyze the distribution and change rules of the features at different scales;

[0077] Integrate multi-scale features, establish a multi-scale analysis model, and explore the relationships and dynamic changes of grassland features at different spatial scales.

[0078] In this embodiment, we collected grassland feature data of different resolutions and spatial scales, including high-resolution images from drones, medium-resolution images from MODIS, and low-resolution images from Landsat. Feature extraction was performed on the data at each scale, including but not limited to texture features, morphological features, and color features. These features are designed to capture the spatial and visual characteristics of features at different scales. We used the fractal dimension formula to analyze the features at each scale and calculated the fractal dimensions at different scales. At the same time, we calculated terrain indices at different scales, such as terrain slope, terrain curvature, etc., to analyze the distribution and change patterns of features at different scales.

[0079] Based on the collected feature data, we built a multi-scale analysis model. The model combines features at different scales to explore the interrelationships and dynamic changes of grassland features at different spatial scales. Through model training and testing, we were able to predict the spatial distribution of features at different scales and analyze the changing trends of features at different scales.

[0080] According to step S300, the categories of the grassland object classification data set include vegetation, bare soil and others, and edge detection is used to identify the features therein; the extracted features are annotated to form a grassland object classification data set, and each image or image fragment needs to have a corresponding label; the annotated data set is divided into a training set, a verification set and a test set; among them, 70% is used for training, 15% is used for verification, and 15% is used for testing.

[0081] Use the deep learning framework TensorFlow to build a neural network model, define the number of network layers, the number of neurons in each layer, the activation function, and the loss function; input the training set data into the model for training. During the training process, update the model parameters through forward propagation and back propagation to minimize the loss function; increase the diversity of training data through random cropping, rotation, and flipping to prevent model overfitting.

[0082] Optimize model performance by adjusting hyperparameters. Use grid search to find the optimal hyperparameter combination, including learning rate, batch size, and regularization parameters; use validation set data to evaluate the performance of the model during training, adjust the model structure and hyperparameters based on the validation results, and use cross-validation to ensure the generalization ability of the model; evaluate the performance of the final model on the test set, and calculate the classification accuracy, recall rate, and F1 score; the evaluation results show that the model achieves 90% accuracy on the test set. Deploy the trained model to practical applications, classify new grassland features, and integrate it into the image processing system of the drone to achieve real-time classification.

[0083] According to step S400, the categories of the grassland feature segmentation dataset include earth mounds, secondary bare land, restored patches, rat holes and others; the image is annotated using a semi-automatic tool, and each pixel needs to have a corresponding label; the annotated dataset is divided into a training set, a validation set and a test set; and the division is performed in a ratio of 70%, 15%, and 15%.

[0084] U-Net was selected as the deep learning model, and the deep learning framework TensorFlow was used to build the segmentation model. The number of model layers, the number of convolution kernels in each layer, pooling and upsampling operations were defined. The model was trained using the training set data. During the training process, the cross-entropy loss function was used to evaluate the segmentation effect of the model, and the model parameters were optimized through back propagation. Random cropping, rotation, flipping and color jittering were used to increase the diversity of training data to prevent model overfitting. Hyperparameters were adjusted to optimize model performance. The performance of the model during training was evaluated using the validation set data, and the model structure and hyperparameters were adjusted according to the validation results. The generalization ability of the model was ensured through cross-validation. The segmentation performance of the final model was evaluated on the test set, and the model was applied to actual grassland images for ground object segmentation. The model output the category of each pixel and generated a detailed segmentation result map.

[0085] According to step S500, based on the accuracy index of the classification and segmentation results, a corresponding weight is assigned to each result of S200-S400, and the weight is adjusted according to the performance of the validation set to satisfy the sum of the weights being 1, wherein the weights are respectively the weight of the vegetation index classification, the weight of the grassland feature classification result, and the weight of the grassland feature segmentation result; for each pixel, a weighted fusion result is calculated based on the classification and segmentation results from different sources and their corresponding weights, and its final category is determined by weighting and voting.

[0086] According to step S500, according to the actual situation of grassland features, expert rules are set, each classification and segmentation result is regarded as a vote, and voting is performed in combination with expert rules, and the final category is determined by the category with the most votes; for each pixel, if the weighted fusion and expert voting results are consistent, the pixel category is directly determined, and if the results are inconsistent, certain rules are introduced or further analysis is performed to determine;

[0087] The grassland object classification results and grassland object segmentation results are mapped to the geographic coordinate system using the location information recorded when the image is taken, and the position of each pixel in the image corresponds to its position in the actual geographic space; according to the grassland object classification results and grassland object segmentation results, the number of pixels of each category of the grassland object classification data set is counted, and the actual area of ​​each pixel in the geographic space is determined according to the resolution and flight altitude of the image taken by the UAV to obtain the total area of ​​the target area; the number of pixels of each category is multiplied by the actual geographic area of ​​each pixel to obtain the total area of ​​vegetation category pixels; and the vegetation coverage is calculated, and the vegetation coverage represents the ratio of the total area of ​​vegetation category pixels to the total area of ​​the target area.

[0088] According to another embodiment of the present invention, Figure 2 The time series NDVI curve of an image recognition monitoring method based on UAV remote sensing technology is shown in Table 1.

[0089] Table 1 Time series NDVI data table

[0090] Time t NDVI value 1 0.45 2 0.50 3 0.52 4 0.55 5 0.60

[0091] According to step S200, NDVI data of different seasons are obtained to form a time series data set {NDVI t}, where t represents the time point; linear trend model, polynomial trend model and exponential smoothing model are used to model the time series NDVI data respectively;

[0092] The linear trend model assumes that the NDVI value changes linearly over time, and the model form is:

[0093] NDVI t =β0+β1t+∈ t ;

[0094] Among them, β0 is the intercept, β1 is the slope, ∈ t is the error term;

[0095] The least squares method is used to estimate the model parameters β0 and β1, and the formula is as follows:

[0096]

[0097] Where T is the total number of time points, is the average value of NDVI, is the average value of the time point; according to the fitted model, the predicted NDVI value at each time point is calculated

[0098]

[0099] in, and are the predicted values ​​of β0 and β1 respectively;

[0100] Calculate the goodness of fit R of the model 2 , evaluate the explanatory power of the model:

[0101]

[0102] If the NDVI data presents a nonlinear trend, a polynomial trend model is used, specifically a quadratic polynomial trend model, in the form of:

[0103] NDVI t =β0+β1t+β2t 2 +∈ t ;

[0104] β2 is the coefficient of the quadratic term, indicating the tendency of quadratic change over time;

[0105] The least squares method is used to estimate the model parameters β0, β1, and β2, and the predicted NDVI value at each time point is calculated based on the fitted model:

[0106]

[0107] in, is the predicted value of β2; calculate the goodness of fit R 2 , evaluate the explanatory power of the model;

[0108] The exponential smoothing model is suitable for time series with random fluctuations but no obvious trend or periodicity. Specifically, a simple exponential smoothing model is used, which is in the form of:

[0109]

[0110] Where α is the smoothing parameter, 0<α<1;

[0111] The goodness of fit of each model was calculated, and the optimal model was selected. Based on the parameters and prediction results of the optimal model, the seasonal changes and long-term trends of the grassland ecosystem were revealed.

[0112] In terms of seasonal changes, we can observe that the NDVI value fluctuates over time. In the given data, the NDVI value gradually increases from 0.45 at the first time point to 0.50 at the last time point. This seasonal change may be affected by seasonal changes and climatic factors such as rainfall, temperature, and light. Since the growth cycle and climatic conditions of grassland ecosystems may change with the change of seasons, the seasonal changes in NDVI values ​​can reflect the seasonal characteristics of grassland ecosystems.

[0113] All three models indicate a long-term trend: NDVI values ​​increase over time. This may indicate that vegetation cover in grassland areas has gradually increased during the observation period. The existence of long-term trends may be affected by a variety of factors, including changes in precipitation patterns, improvements in soil quality, climate change, and the degree of human disturbance. Understanding long-term trends is crucial for the management and protection of grassland ecosystems because it can help us predict future changes and take appropriate measures to maintain ecological balance.

[0114] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0115] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image recognition monitoring method based on UAV remote sensing technology, characterized in that: The method comprises the following steps: S100, using a UAV and a hyperspectral imager to collect grassland features images and obtain original image data of the target area; preprocessing the collected original images according to the flight altitude and image type to obtain a preprocessed data set; According to step S100, a drone and a hyperspectral imager are used to collect images of the target grassland area several times, covering different seasons, to obtain original image data of the target area in multiple time phases; according to the flight altitude and the image type, the collected original images are spliced, cropped and geometrically corrected to eliminate geometric distortion and splicing errors, and radiation correction and atmospheric correction are performed; histogram equalization is applied to improve the contrast and clarity of the image, and a preprocessed data set is obtained; S200, classifying grassland features based on vegetation index using the preprocessed data set to achieve classification of bare soil and vegetation communities; performing time series information analysis, calculating vegetation index at different time points using time series data, and applying trend analysis to model; performing cross-scale analysis, extracting and analyzing features of data at different scales using multi-scale analysis methods, and obtaining the distribution and change patterns of grassland features at different spatial scales; S300, using the preprocessed data set to produce a grassland feature classification data set based on grassland feature characteristics and deep learning classification data set production rules; building, training, verifying and testing a deep learning model, and using the optimal classification model with optimized parameters to classify grassland images to obtain grassland feature classification results; S400, using the preprocessed data set to create a grassland feature segmentation data set; constructing a grassland feature segmentation model based on the deep learning segmentation method and the grassland feature definition, training, verifying and testing the model, and using the optimal segmentation model to segment the grassland image after comparative analysis to obtain a grassland feature segmentation result; According to step S400, the categories of the grassland feature segmentation dataset include earth mounds, secondary bare land, restored patches, rat holes and others; the image is annotated using a semi-automatic tool, and each pixel needs to have a corresponding label; the annotated dataset is divided into a training set, a validation set and a test set; Select U-Net as the deep learning model, use the deep learning framework TensorFlow to build the segmentation model, define the number of model layers, the number of convolution kernels in each layer, pooling and upsampling operations; use the training set data to train the model. During the training process, use the cross entropy loss function to evaluate the segmentation effect of the model, and optimize the model parameters through back propagation; use random cropping, rotation, flipping and color jittering, adjust the hyperparameters, use the validation set data to evaluate the performance of the model during the training process, adjust the model structure and hyperparameters according to the validation results, and perform cross-validation; evaluate the segmentation performance of the final model on the test set, and apply the model to actual grassland images for ground object segmentation; the model outputs the category of each pixel and generates a segmentation result map; S500, weighted fusion and expert voting are performed on the processing results of S200-S400, and the category of each pixel is determined by using the weighted fusion and expert voting scheme to obtain the final segmentation results of each feature in the grassland, and the vegetation coverage is calculated according to the final segmentation results of each feature in the grassland.

2. The image recognition monitoring method based on UAV remote sensing technology according to claim 1 is characterized in that: According to step S200, the vegetation index NDVI is calculated using the near infrared band NIR and the red band Red data, and the formula is as follows: NDVI values ​​range from -1 to 1, with larger values ​​indicating denser vegetation cover and smaller values ​​indicating bare soil or other non-vegetated features; According to the range of NDVI values, thresholds were set to distinguish bare soil and vegetation communities. NDVI calculation and threshold determination were applied to each pixel to obtain preliminary bare soil and vegetation classification results.

3. The image recognition monitoring method based on UAV remote sensing technology according to claim 2 is characterized in that: According to step S200, NDVI data of different seasons are obtained to form a time series data set {NDVI t }, where t represents the time point; The linear trend model, polynomial trend model and exponential smoothing model were used to model the time series NDVI data respectively; The linear trend model assumes that NDVI values ​​change linearly over time, and the model form is: Among them, β0 is the intercept, β1 is the slope, and ϵ t is the error term; The least squares method is used to estimate the model parameters β0 and β1, and the formula is as follows: Where T is the total number of time points, is the average NDVI value, is the average value at the time point; According to the fitted model, the predicted NDVI value at each time point is calculated : Calculate the goodness of fit R of the model 2 , evaluate the explanatory power of the model: If the NDVI data presents a nonlinear trend, a polynomial trend model is used, specifically a quadratic polynomial trend model, in the form of: β2 is the coefficient of the quadratic term, indicating the tendency of quadratic change over time; The least squares method is used to estimate the model parameters β0, β1, and β2, and the predicted NDVI value at each time point is calculated based on the fitted model: in, is the predicted value of β2; calculate the goodness of fit R 2 , evaluate the explanatory power of the model; The exponential smoothing model is suitable for time series with random fluctuations but no obvious trend or periodicity. Specifically, a simple exponential smoothing model is used, which is in the form of: Where α is the smoothing parameter, 0<α<1; The goodness of fit of each model was calculated, the optimal model was selected, and the seasonal changes of the grassland were revealed based on the parameters and prediction results of the optimal model.

4. The image recognition monitoring method based on UAV remote sensing technology according to claim 3 is characterized in that: According to step S200, grassland object data of different spatial scales are collected, and features are extracted from data of different scales, wherein the features include: texture features, shape features, spectral features and spatial features, wherein the texture features describe the roughness and pattern of the grassland object surface, the shape features describe the geometric shape of the grassland object, the spectral features reflect the reflectivity of the grassland object in different spectral bands, and the spatial features describe the spatial distribution and relative position relationship of the grassland object; the fractal dimension of the grassland object is calculated to reflect the complexity of the object at different scales, and the formula is as follows: Among them, D is the fractal dimension, N(ϵ) is the minimum number of units covering the features when the scale is ϵ; the changing trend of the fractal dimension can reveal the changes in the structural complexity of grassland features from small scale to large scale. By calculating and comparing the fractal dimensions of different regions, the differences in structural complexity of different regions can be identified; the terrain index at different scales is calculated, the fractal dimensions and terrain index at different scales are integrated, and a multi-scale analysis model is established to show the distribution and change laws of grassland features at different scales.

5. The image recognition monitoring method based on UAV remote sensing technology according to claim 1 is characterized in that: According to step S300, the categories of the grassland object classification data set include vegetation, bare soil and others, and the features thereof are identified using Canny edge detection; Based on the Canny edge detection results, the unclear or erroneous parts are manually corrected, and the extracted features are annotated to form a grassland feature classification dataset. Each image or image fragment needs to have a corresponding label; the annotated dataset is divided into a training set, a validation set, and a test set; Use the deep learning framework TensorFlow to build a neural network model, define the number of network layers, the number of neurons in each layer, the activation function, and the loss function; input the training set data into the model for training; During the training process, the model parameters are updated through forward propagation and back propagation to minimize the loss function; random cropping, rotation and flipping are used to increase the diversity of training data to prevent model overfitting.

6. The image recognition monitoring method based on UAV remote sensing technology according to claim 5 is characterized in that: Optimize model performance by adjusting hyperparameters; use grid search to find the optimal hyperparameter combination, which includes learning rate, batch size and regularization parameter; use validation set data to evaluate the performance of the model during training, adjust the model structure and hyperparameters according to the validation results, and ensure the generalization ability of the model through cross-validation method; evaluate the performance of the final model on the test set, and calculate the classification accuracy, recall rate and F1 score; deploy the trained model to practical applications, classify new grassland object images, and integrate it into the image processing system of the drone to achieve real-time classification.

7. The image recognition monitoring method based on UAV remote sensing technology according to claim 1 is characterized by: According to step S500, based on the accuracy index of the classification and segmentation results, a corresponding weight is assigned to each result of S200-S400, and the weight is adjusted according to the performance of the validation set to satisfy that the sum of the weights is 1, wherein the weights are the weight of the vegetation index classification, the weight of the grassland feature classification result, and the weight of the grassland feature segmentation result; For each pixel, the weighted fusion result is calculated based on the classification and segmentation results from different sources and their corresponding weights, and its final category is determined by weighting and voting.

8. The image recognition monitoring method based on UAV remote sensing technology according to claim 1 is characterized by: According to step S500, according to the actual situation of grassland features, expert rules are set, each classification and segmentation result is regarded as a vote, and voting is conducted in combination with the expert rules, and the final category is determined by the category with the most votes; For each pixel, if the results of weighted fusion and expert voting are consistent, the pixel category is directly determined. If the results are inconsistent, certain rules are introduced or further analysis is performed to determine the category. Using the location information recorded when the image was taken, the grassland object classification results and grassland object segmentation results were mapped to the geographic coordinate system, and the position of each pixel in the image corresponded to its position in the actual geographic space; According to the grassland feature classification results and grassland feature segmentation results, the number of pixels in each category of the grassland feature classification data set is counted. According to the resolution and flight altitude of the drone image, the actual area of ​​each pixel in the geographic space is determined to obtain the total area of ​​the target area. Multiply the number of pixels in each category by the actual geographic area of ​​each pixel to obtain the total area of ​​vegetation category pixels; The vegetation coverage is calculated, where the vegetation coverage represents the ratio of the total area of ​​vegetation category pixels to the total area of ​​the target area.

Images