Remote sensing image vegetation coverage prediction method and system based on deep learning

Abstract

The invention discloses a remote sensing image vegetation coverage prediction method and system based on deep learning, and relates to the technical field of vegetation coverage prediction, and the method comprises the steps: collecting multi-source remote sensing image data and a corresponding truth value label, carrying out the preprocessing, and extracting the features of spectrums, textures and the like to construct a data set; constructing a prediction model containing a convolutional neural network (extracting spatial features) and a recurrent neural network (processing time sequence features), and training the model through cross validation and regularization; and finally inputting new data to realize prediction. In the preprocessing, the optical remote sensing image is subjected to radiometric calibration, atmospheric correction and geometric correction, aperture radar image denoising and laser radar data point cloud filtering classification. The system comprises a data collection module, a preprocessing module, a feature extraction module, a model construction module, a training module and a prediction module. According to the scheme, multi-source data and deep learning are fused, the vegetation coverage prediction precision is improved, and technical support is provided for ecological monitoring and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of vegetation coverage prediction, and specifically relates to a method and system for predicting vegetation coverage of remote sensing images based on deep learning. Background Art

[0002] The prediction of vegetation coverage of remote sensing images is of great significance for ecological environment monitoring, climate change research, etc. Traditional methods have limitations such as insufficient accuracy. Deep learning brings new possibilities to this field due to its powerful feature extraction ability. It can be used for real-time monitoring of regional vegetation dynamic changes, providing data support for the evaluation of ecological protection projects; assisting in agricultural yield estimation and promoting the development of precision agriculture; in natural disaster early warning, predicting risks such as soil erosion through changes in vegetation coverage, and providing a scientific basis for resource management and sustainable development decision-making.

[0003] Existing methods and systems for predicting vegetation coverage of remote sensing images based on deep learning rely on a single remote sensing data source, making it difficult to comprehensively capture the spatial structure, temporal dynamics, and physical properties of vegetation, resulting in incomplete feature representation. At the same time, the system lacks refined processing of specific noises in different sensor data during the preprocessing process, leading to low data quality and easily interfering with model training. Therefore, a method and system for predicting vegetation coverage of remote sensing images based on deep learning are needed to solve the above problems. Summary of the Invention

[0004] To solve the above technical problems, a method and system for predicting vegetation coverage of remote sensing images based on deep learning are provided. The present technical solution solves the problems in the above background art that existing methods and systems for predicting vegetation coverage of remote sensing images based on deep learning rely on a single remote sensing data source, making it difficult to comprehensively capture the spatial structure, temporal dynamics, and physical properties of vegetation, resulting in incomplete feature representation, and at the same time, the system lacks refined processing of specific noises in different sensor data during the preprocessing process, leading to low data quality and easily interfering with model training.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for predicting vegetation coverage of remote sensing images based on deep learning, comprising:

[0007] S1. Collect multi-source remote sensing image data, where the multi-source remote sensing image data includes optical remote sensing images, synthetic aperture radar images, and lidar data, and at the same time collect ground measured vegetation coverage data of the corresponding area as the true value label;

[0008] S2. Preprocess the collected multi-source remote sensing image data, including radiometric calibration, atmospheric correction, and geometric correction for optical remote sensing images, denoising for synthetic aperture radar images, and point cloud filtering and classification for lidar data, to obtain preprocessed multi-source remote sensing image data;

[0009] S3. Extract spectral features, texture features, spatial features, and time series features from the preprocessed multi-source remote sensing image data to construct a multi-feature dataset;

[0010] S4. Construct a vegetation coverage prediction model based on deep learning. The vegetation coverage prediction model includes a convolutional neural network unit and a recurrent neural network unit. The convolutional neural network unit is used to extract the spatial features of remote sensing images, and the recurrent neural network unit is used to process time series features;

[0011] S5. Input the multi-feature dataset and the ground truth labels into the constructed deep learning model for training, and optimize the model parameters using cross-validation and regularization methods to obtain a trained vegetation coverage prediction model;

[0012] S6. After preprocessing and feature extraction of the newly acquired multi-source remote sensing image data, input it into the trained vegetation coverage prediction model to obtain the predicted vegetation coverage results.

[0013] In an alternative embodiment, the radiometric calibration of the optical remote sensing image specifically includes:

[0014] S2.1.1. Based on the optical remote sensing image, determine the calibration parameter file of the acquisition sensor corresponding to the optical remote sensing image;

[0015] S2.1.2. Obtain the gain coefficient and offset coefficient corresponding to the optical remote sensing image from the calibration parameter file;

[0016] S2.1.3. Obtain the original digital quantization value corresponding to each pixel in the optical remote sensing image and convert it into a radiance value:

[0017] L i = G·DN i + O, where L i represents the radiance value converted from the original digital quantization value corresponding to the i-th pixel in the optical remote sensing image, G represents the gain coefficient corresponding to the optical remote sensing image, DN i represents the original digital quantization value corresponding to the i-th pixel in the optical remote sensing image, and O represents the offset coefficient corresponding to the optical remote sensing image.

[0018] In an alternative embodiment, the atmospheric correction of the optical remote sensing image specifically includes:

[0019] S2.2.1. Based on the optical remote sensing image, obtain the relevant atmospheric parameters when collecting the optical remote sensing image, including atmospheric water vapor content, aerosol optical depth, solar zenith angle, observation zenith angle, and relative azimuth angle;

[0020] S2.2.2. Input the relevant atmospheric parameters corresponding to the optical remote sensing image into the 6S model to obtain the atmospheric transmittance, ground object transmittance, and atmospheric path radiance required for atmospheric correction;

[0021] S2.2.3. Obtain the radiance value corresponding to the optical remote sensing image, and perform atmospheric correction on the radiance value corresponding to the optical remote sensing image based on the atmospheric transmittance, ground object transmittance, and atmospheric path radiance to obtain the surface reflectance:

[0022]

[0023] In the formula, ρ i represents the surface reflectance corresponding to the i-th pixel in the optical remote sensing image, Π is the pi, L i represents the radiance value converted from the original digital quantization value corresponding to the i-th pixel in the optical remote sensing image, L p represents the atmospheric path radiance corresponding to the optical remote sensing image, d represents the sun-earth distance corresponding to the optical remote sensing image, τ s represents the atmospheric transmittance, τ v represents the ground object transmittance, E s represents the solar spectral irradiance at the top of the atmosphere corresponding to the optical remote sensing image, θ s represents the solar zenith angle corresponding to the optical remote sensing image.

[0024] In an optional embodiment, the geometric correction of the optical remote sensing image specifically includes:

[0025] S2.3.1. Obtain the high-precision reference topographic map corresponding to the optical remote sensing image, and select at least 6 homologous points as ground control points from the optical remote sensing image and the high-precision reference topographic map respectively;

[0026] S2.3.2. Obtain the image coordinate system and the reference map coordinate system corresponding to the optical remote sensing image and the high-precision reference topographic map;

[0027] S2.3.3. Obtain the coordinates (x, y) and (X, Y) of the 6 ground control points selected from the optical remote sensing image and the high-precision reference topographic map in the image coordinate system and the reference map coordinate system respectively;

[0028] S2.3.4. Establish a polynomial transformation model:

[0029]

[0030] In the formula, a jand b j are both polynomial coefficients, where j = j = 0, 1, …, 5;

[0031] S2.3.5. Use the least squares method to fit the coordinates (x, y) and (X, Y) of 6 ground control points in the image coordinate system and the reference map coordinate system to determine the polynomial coefficients a j and b j ;

[0032] S2.3.6. According to the polynomial coefficients a j and b j , map each pixel in the optical remote sensing image to the reference map coordinate system;

[0033] S2.3.7. Obtain the new positions of each pixel in the optical remote sensing image mapped to the reference map coordinate system, and perform resampling using bilinear interpolation to obtain the geometrically corrected optical remote sensing image.

[0034] In an alternative embodiment, the denoising process for the synthetic aperture radar image specifically includes:

[0035] S2.4.1. Obtain the central pixel in the synthetic aperture radar image, and select a sliding window W of size m×m with the central pixel as the reference;

[0036] S2.4.2. Slide and intercept the synthetic aperture radar image using the sliding window W, with a sliding step of 1 pixel, and simultaneously determine the mean and variance of the pixels within each window;

[0037] S2.4.3. Determine the equivalent number of looks ENL corresponding to each window through the mean difference and variance of the pixels within each window:

[0038]

[0039] where, is the square value of the mean of the pixels within each window, is the variance of the pixels within each window;

[0040] S2.4.4. According to the equivalent number of looks corresponding to each window, combined with the mean difference and variance of the pixels within each window, obtain the filtering coefficient C corresponding to each window:

[0041]

[0042] S2.4.5. Use the filtering coefficient C corresponding to each window and the mean of the pixels within each window to filter the corresponding central pixel in each window to obtain the denoised central pixel value:

[0043] y mid = μ W+C(x mid -μ W );

[0044] where y mid is the denoised central pixel value of the central pixel corresponding to each window, and x mid is the central pixel value corresponding to each window;

[0045] S2.4.6. Remove the denoised central pixel in each window, and obtain the local mean and local variance corresponding to all the remaining pixels;

[0046] S2.4.7. Determine the local equivalent number of looks L corresponding to each window according to the local mean and local variance corresponding to all the remaining pixels:

[0047]

[0048] where μ 2 is the square value of the local mean corresponding to each window, and σ 2 is the local variance corresponding to each window; S2.4.8. Filter all the remaining pixels in each window using the local equivalent number of looks to obtain the denoised local pixel values:

[0049]

[0050] where y sr is the denoised local pixel value of the pixel values remaining in each window, and x sr are the pixel values remaining in each window, and s and r are index labels.

[0051] In an alternative embodiment, the point cloud filtering and classification processing of the lidar data specifically includes:

[0052] S2.5.1. Divide the lidar data into grids of size n×n;

[0053] S2.5.2. Traverse each n×n grid, and respectively find the point with the smallest Z coordinate therein as the lowest point p of the corresponding n×n grid min ;

[0054] S2.5.3. For each point p in the grid, calculate its slope S with the lowest point:

[0055]

[0056] where Z p and Zp min are the Z coordinate values of point p and the lowest point respectively, and D is the distance between point p and the lowest point p min on the plane;

[0057] S2.5.6. Set the slope threshold T s If the slope S of point p is greater than T s then determine this point as a noise point and remove it to obtain the lidar data after point cloud filtering;

[0058] S2.5.7. Extract the features corresponding to each point from the lidar data after point cloud filtering, including elevation, echo intensity, and neighborhood point density;

[0059] S2.5.8. Collect points of known categories as training samples, construct a training dataset to train the SVM classifier, and obtain a classification model;

[0060] S2.5.9. Input the features of each point in the lidar data after point cloud filtering into the classification model to obtain the category label of this point, and complete the point cloud classification process.

[0061] Furthermore, a remote sensing image vegetation coverage prediction system based on deep learning is proposed to implement the prediction method as described in any one of the above, including:

[0062] Data collection module, which is used to collect multi-source remote sensing image data;

[0063] Data preprocessing module, which is used to preprocess the collected multi-source remote sensing image data;

[0064] Feature extraction module, which is used to extract spectral features, texture features, spatial features, and time series features from the preprocessed multi-source remote sensing image data, and construct a multi-feature dataset;

[0065] Model construction module, which is used to construct a vegetation coverage prediction model based on deep learning;

[0066] Model training module, which is used to input the multi-feature dataset and the ground truth label into the constructed deep learning model for training, and optimize the model parameters by using cross-validation and regularization methods to obtain a trained vegetation coverage prediction model;

[0067] Prediction module, which is used to input the newly acquired multi-source remote sensing image data after preprocessing and feature extraction into the trained vegetation coverage prediction model to obtain the predicted vegetation coverage result.

[0068] In an alternative embodiment, the data collection module includes:

[0069] Remote sensing data acquisition unit, which is used to collect optical remote sensing images, synthetic aperture radar images, and lidar data;

[0070] True value data acquisition unit, which is used to collect ground measured vegetation coverage data corresponding to multi-source remote sensing image data and provide accurate label information for the model training module.

[0071] In an optional embodiment, the data preprocessing module includes:

[0072] Optical remote sensing image preprocessing unit, which is used to perform radiometric calibration, atmospheric correction and geometric correction on optical remote sensing images;

[0073] Synthetic aperture radar image denoising unit, which is used to perform denoising processing on synthetic aperture radar images;

[0074] LiDAR data processing unit, which is used to perform point cloud filtering and classification processing on LiDAR data.

[0075] In an optional embodiment, the model construction module includes:

[0076] Convolutional neural network construction unit, which is used to construct the input layer, convolutional layer, pooling layer, activation function and fully connected layer of the convolutional neural network for extracting spatial features of remote sensing images;

[0077] Recurrent neural network construction unit, which is used to construct the input layer, recurrent layer and output layer of the recurrent neural network for processing time series features;

[0078] Model fusion unit, which is used to fuse the outputs of the convolutional neural network module and the recurrent neural network module to construct a complete vegetation coverage prediction model.

[0079] Compared with the prior art, the beneficial effects of the present invention are:

[0080] A method for predicting vegetation coverage of remote sensing images based on deep learning proposed by this solution, by fusing multi-source remote sensing data such as optical, synthetic aperture radar, and LiDAR, combining spectral, texture, spatial and time series features, breaks through the limitations of single data, and significantly improves the accuracy and comprehensiveness of vegetation coverage prediction;

[0081] A method for predicting vegetation coverage of remote sensing images based on deep learning proposed by this solution designs a dedicated preprocessing process for different types of data (such as radiometric calibration and geometric correction of optical images, denoising of radar images, and LiDAR point cloud classification), effectively eliminates noise and distortion, and ensures the reliability of the input data;

[0082] A method for predicting vegetation coverage of remote sensing images based on deep learning proposed in this solution adopts a model architecture that combines a convolutional neural network and a recurrent neural network. The former efficiently extracts spatial features, and the latter processes time-series dynamic information to achieve in-depth mining of multi-dimensional features of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 It is a flowchart of a method for predicting vegetation coverage of remote sensing images based on deep learning proposed by the present invention;

[0084] Figure 2 It is a flowchart of radiometric calibration and atmospheric correction of optical remote sensing images in the present invention;

[0085] Figure 3 It is a flowchart of geometric correction of optical remote sensing images in the present invention;

[0086] Figure 4 It is a flowchart of denoising processing of synthetic aperture radar images in the present invention;

[0087] Figure 5 It is a flowchart of point cloud filtering and classification processing of lidar data in the present invention;

[0088] Figure 6 It is a system framework diagram of a system for predicting vegetation coverage of remote sensing images based on deep learning proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0089] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art can think of other obvious variations.

[0090] Referring to Figure 1 - Figure 6 As shown, a method for predicting vegetation coverage of remote sensing images based on deep learning includes:

[0091] S1. Collect multi-source remote sensing image data, where the multi-source remote sensing image data includes optical remote sensing images, synthetic aperture radar images, and lidar data. At the same time, collect the ground-measured vegetation coverage data of the corresponding area as the true value label;

[0092] S2. Preprocess the collected multi-source remote sensing image data, including radiometric calibration, atmospheric correction, and geometric correction of optical remote sensing images, denoising processing of synthetic aperture radar images, and point cloud filtering and classification processing of lidar data to obtain the preprocessed multi-source remote sensing image data;

[0093] Further, the radiometric calibration of the optical remote sensing image specifically includes:

[0094] S2.1.1. Determine the calibration parameter file of the acquisition sensor corresponding to the optical remote sensing image based on the optical remote sensing image;

[0095] S2.1.2. Obtain the gain coefficient and offset coefficient corresponding to the optical remote sensing image from the calibration parameter file;

[0096] S2.1.3. Obtain the original digital quantization value corresponding to each pixel in the optical remote sensing image and convert it into a radiance value:

[0097] L i = G·DN i + O, where L i represents the radiance value converted from the original digital quantization value corresponding to the i-th pixel in the optical remote sensing image, G represents the gain coefficient corresponding to the optical remote sensing image, DN i represents the original digital quantization value corresponding to the i-th pixel in the optical remote sensing image, and O represents the offset coefficient corresponding to the optical remote sensing image.

[0098] Specifically, radiometric calibration is the process of converting the original digital quantization value (DN value) recorded by the sensor into a radiance value or reflectance value with physical significance. In steps S2.1.1 and S2.1.2, according to the acquisition sensor corresponding to the optical remote sensing image, query the corresponding sensor manufacturer, and obtain the gain coefficient G and offset coefficient O from the calibration parameter file provided by the sensor manufacturer.

[0099] Furthermore, perform atmospheric correction on the optical remote sensing image, specifically including:

[0100] S2.2.1. Based on the optical remote sensing image, obtain the relevant atmospheric parameters when acquiring the optical remote sensing image, including atmospheric water vapor content, aerosol optical thickness (usually measured at a wavelength of 550 nm), solar zenith angle, observation zenith angle, and relative azimuth angle;

[0101] S2.2.2. Input the relevant atmospheric parameters corresponding to the optical remote sensing image into the 6S model to obtain the atmospheric transmittance, surface transmittance, and path radiance required for atmospheric correction;

[0102] S2.2.3. Obtain the radiance value corresponding to the optical remote sensing image, and perform atmospheric correction on the radiance value corresponding to the optical remote sensing image based on the atmospheric transmittance, surface transmittance, and path radiance to obtain the surface reflectance:

[0103]

[0104] In the formula, ρ i represents the surface reflectance corresponding to the i-th pixel in the optical remote sensing image, Π is the pi, Li represents the radiance value converted from the original digital quantization value corresponding to the \(i\)-th pixel in the optical remote sensing image, \(L\) p represents the path radiance corresponding to the optical remote sensing image, \(d\) represents the sun-earth distance (astronomical unit) corresponding to the optical remote sensing image, \(\tau\) s represents the atmospheric transmittance, \(\tau\) v represents the surface transmittance, \(E\) s represents the solar spectral irradiance at the top of the atmosphere corresponding to the optical remote sensing image, \(\theta\) s represents the solar zenith angle corresponding to the optical remote sensing image.

[0105] Specifically, the purpose of atmospheric correction is to eliminate the influence of factors such as atmospheric scattering and absorption on the radiation value of the remote sensing image and obtain the true reflectance of the ground object. The atmospheric transmittance is the transmittance from the sun to the ground object, which reflects the proportion of the remaining radiation after the solar radiation is absorbed and scattered by the atmosphere during the process from the sun to the ground. It is a dimensionless value with a range of 0 to 1. The surface transmittance is the transmittance from the ground object to the sensor, that is, the proportion of the remaining radiation after the radiation reflected by the ground is absorbed and scattered by the atmosphere during the process from the ground to the sensor. It is also a dimensionless value with a range of 0 to 1. \(E\) s represents the solar spectral irradiance at the top of the atmosphere corresponding to the optical remote sensing image, which refers to the solar radiation energy received per unit area per unit time perpendicular to the sun's rays at the top of the earth's atmosphere. A pixel is the smallest unit of an image. An image is composed of a large number of pixels arranged in a matrix, and each pixel corresponds to a specific area on the ground (spatial resolution). For different bands, \(E\) s has different values. The solar zenith angle, that is, the angle between the sun's rays and the zenith direction, is in radians or degrees. The solar zenith angle affects the path length and energy distribution of the solar radiation reaching the ground, and \(\cos\theta\) s is used to correct the oblique irradiation of the solar radiation.

[0106] It can be understood that each pixel corresponds to a radiance value, which is the original signal recorded by the sensor and reflects the radiation energy of the ground area corresponding to the pixel. This value is converted into a physically measurable radiance after radiometric calibration.

[0107] Furthermore, geometric correction of the optical remote sensing image specifically includes:

[0108] S2.3.1. Obtain a high-precision reference topographic map corresponding to the optical remote sensing image, and select at least 6 homologous points as ground control points from the optical remote sensing image and the high-precision reference topographic map respectively;

[0109] S2.3.2. Obtain the image coordinate system and the reference map coordinate system corresponding to the optical remote sensing image and the high-precision reference topographic map;

[0110] S2.3.3. Respectively obtain the coordinates (x, y) and (X, Y) of the 6 ground control points selected from the optical remote sensing image and the high-precision reference topographic map in the image coordinate system and the reference map coordinate system;

[0111] S2.3.4. Establish a polynomial transformation model:

[0112]

[0113] where a j and b j are both polynomial coefficients, j = 0, 1,..., 5;

[0114] S2.3.5. Use the least squares method to fit the coordinates (x, y) and (X, Y) of the 6 ground control points in the image coordinate system and the reference map coordinate system, and determine the polynomial coefficients a j and b j ;

[0115] S2.3.6. According to the polynomial coefficients a j and b j , map each pixel in the optical remote sensing image to the reference map coordinate system;

[0116] S2.3.7. Obtain the new positions of each pixel in the optical remote sensing image mapped to the reference map coordinate system, and perform resampling using bilinear interpolation to obtain the geometrically corrected optical remote sensing image.

[0117] Specifically, the role of geometric correction is to eliminate the geometric distortion of the remote sensing image caused by factors such as sensor attitude and terrain undulation, so that the image matches the geographic coordinate system. The high-precision reference topographic map can be obtained from the Geographic Information System (GIS). In step S2.3.7, bilinear interpolation is used for resampling to obtain the geometrically corrected optical remote sensing image. The nearest neighbor difference method can also be used for resampling.

[0118] Furthermore, perform denoising processing on the synthetic aperture radar image, specifically including:

[0119] S2.4.1. Obtain the central pixel in the synthetic aperture radar image, and select a sliding window W of size m×m with the central pixel as the reference;

[0120] S2.4.2. Use the sliding window W to slide and intercept the synthetic aperture radar image, with a sliding step of 1 pixel, and synchronously determine the mean and variance of the pixels in each window;

[0121] S2.4.3. Determine the equivalent number of looks ENL corresponding to each window through the mean difference and variance of the pixels in each window:

[0122]

[0123] In the formula, is the square value of the mean of the pixels in each window, is the variance of the pixels in each window;

[0124] S2.4.4. According to the equivalent number of looks corresponding to each window, combined with the mean difference and variance of the pixels in each window, obtain the filtering coefficient C corresponding to each window:

[0125]

[0126] S2.4.5. Use the filtering coefficient C corresponding to each window and the mean of the pixels in each window to filter the central pixel corresponding to each window to obtain the denoised central pixel value:

[0127] y mid = μ W + C(x mid - μ W );

[0128] In the formula, y mid is the denoised central pixel value of the central pixel corresponding to each window, and x mid is the central pixel value corresponding to each window;

[0129] S2.4.6. Remove the denoised central pixel in each window to obtain the local mean and local variance corresponding to all the remaining pixels;

[0130] S2.4.7. Determine the local equivalent number of looks L corresponding to each window according to the local mean and local variance corresponding to all the remaining pixels:

[0131]

[0132] In the formula, μ 2 is the square value of the local mean corresponding to each window, and σ 2 is the local variance corresponding to each window;

[0133] S2.4.8. Use the local equivalent number of looks to filter all the remaining pixels in each window to obtain the denoised local pixel values:

[0134]

[0135] In the formula, y sr is the denoised local pixel value of the remaining pixel values in each window, and x sr is the remaining pixel value in each window, and s and r are index labels.

[0136] Specifically, there is multiplicative noise in the synthetic aperture radar (SAR) image, and filtering such as Lee filtering is required to remove the noise. In step S2.4.1, a sliding window W of size m×m is selected, where m is an odd number such as 3, 5, or 7. It needs to be selected according to the image resolution and the scale of the ground object. For example, for a low-resolution image (100m), 5×5 can be selected, and for a high-resolution image (1m), 3×3 can be selected. The equivalent number of looks (ENL) reflects the degree of speckle suppression in the SAR image. The larger the value, the more uniform the area (such as flat farmland) and the weaker the noise; the smaller the value, the rougher the area (such as vegetation canopy) and the stronger the noise. In a high-ENL area (such as water body): the noise is low, and the original value is mainly retained during filtering. In a low-ENL area (such as forest): the noise is strong, and the filtering intensity needs to be increased.

[0137] It can be understood that the central pixel in the synthetic aperture radar image is obtained, and with the central pixel as the reference, a sliding window W of size m×m is selected. The central pixel here is the pixel at the geometric center position of the synthetic aperture radar image. In step S2.4.5, using the filtering coefficient C corresponding to each window and the mean value of the pixels within each window, the central pixel corresponding to each window is filtered (adaptive weighted averaging is performed on the central pixel. The stronger the noise (the larger C), the more it depends on the neighborhood mean value; the purer the signal (the smaller C), the more the original value is retained), and the denoised central pixel value is obtained. The central pixel here is the pixel at the geometric center position in a specified local area (such as a sliding window, a neighborhood window). In steps S2.4.6 - S2.4.8, the secondary filtering of non-central pixels is to avoid the influence of the filtered value of the central pixel on the neighborhood statistics and ensure that the filtering of the remaining pixels is based on the undisturbed original data. For each remaining pixel, weighted averaging is performed according to the local equivalent number of looks L (noise intensity) of its local area to further suppress the speckle noise at non-central positions while maintaining the spatial consistency within the window.

[0138] Furthermore, point cloud filtering and classification processing are performed on the lidar data, specifically including:

[0139] S2.5.1. Divide the lidar data into grids of size n×n;

[0140] S2.5.2. Traverse each n×n grid and find the point with the smallest Z coordinate as the lowest point p of the corresponding n×n grid min ;

[0141] S2.5.3. For each point p within the grid, calculate its slope S with the lowest point:

[0142]

[0143] where Z p and They are the Z coordinate values of point p and the lowest point respectively, and D is the distance between point p and the lowest point p min on the plane;

[0144] S2.5.6. Set the slope threshold T s , if the slope S of point p is greater than T s , then this point is determined as a noise point and removed, and the lidar data after point cloud filtering is obtained;

[0145] S2.5.7. Extract the features corresponding to each point from the lidar data after point cloud filtering, including elevation, echo intensity, and neighborhood point density;

[0146] S2.5.8. Collect points of known categories as training samples, construct a training dataset to train the SVM classifier, and obtain a classification model;

[0147] S2.5.9. Input the features of each point in the lidar data after point cloud filtering into the classification model to obtain the category label of this point, and complete the point cloud classification process.

[0148] Specifically, the purpose of point cloud filtering is to remove noise points and outliers in the lidar data, while retaining effective information such as ground points and vegetation points. In step S2.5.3, D is the distance between point p and the lowest point pmin on the plane, that is, the distance on the X-Y plane in the point cloud coordinate system. In steps S2.5.7 - S2.5.9, point cloud classification of the filtered lidar data is to classify the lidar data into different categories, such as ground points, vegetation points, building points, etc. Set the slope threshold Ts, which is used to distinguish noise from ground objects in point cloud filtering and needs to be set in combination with terrain features (for example, a larger value is taken for mountains and a smaller value is taken for plains)

[0149] S3. Extract spectral features, texture features, spatial features, and time series features from the preprocessed multi-source remote sensing image data, and construct a multi-feature dataset;

[0150] Specifically, for spectral features, common vegetation indices such as the Normalized Difference Vegetation Index (NDVI), Enhanced Vegetation Index (EVI), etc. need to be calculated. Taking NDVI as an example, the calculation formula is NDVI = (NIR + RED) / (NIR - RED), where NIR is the reflectance in the near-infrared band and RED is the reflectance in the red band. These vegetation indices can reflect the growth status and coverage degree of vegetation. It is also possible to directly extract the reflectance values of each band of the remote sensing image, which are the direct manifestations of the spectral characteristics of the ground objects. For texture features, the co-occurrence relationship between different gray levels in the image needs to be calculated. By statistically analyzing the gray combination frequencies of pixel pairs at different distances and directions, a gray-level co-occurrence matrix is obtained. Then, texture features such as contrast, correlation, energy, and homogeneity are extracted from the matrix, which can describe information such as the texture roughness and regularity of the image. For spatial features, algorithms such as the Canny edge detection algorithm are used to detect the edge information in the image. Edges usually correspond to the boundaries of different ground objects and can provide information about the shape and contour of the ground objects. It is also possible to perform morphological operations such as dilation, erosion, opening, and closing on the image to extract spatial structure information such as connected regions and holes in the image. For time series feature extraction, for multi-temporal remote sensing images, the reflectance difference or vegetation index difference of pixels at the same location between different time phases is calculated to reflect the changes of ground objects over time. For long-term remote sensing data, trend analysis is carried out, such as using linear regression to analyze the change trend of the vegetation index over time to obtain features such as slope and intercept. The extracted spectral features, texture features, spatial features, and time series features are combined to form a multi-dimensional feature vector. For each pixel or image region, there is a corresponding feature vector. The feature vectors of all pixels or image regions and the corresponding true value labels (the measured vegetation coverage on the ground) are combined to construct a multi-feature dataset.

[0151] S4. Construct a vegetation coverage prediction model based on deep learning. The vegetation coverage prediction model includes a convolutional neural network unit and a recurrent neural network unit. The convolutional neural network unit is used to extract the spatial features of the remote sensing image, and the recurrent neural network unit is used to process the time series features;

[0152] Specifically, the convolutional neural network unit includes:

[0153] Input layer: The preprocessed remote sensing image or the extracted features are used as inputs. The number of neurons in the input layer depends on the dimension of the input data.

[0154] Convolutional layer: Multiple convolutional kernels are used to perform convolutional operations on the input data to extract spatial features at different scales and directions. Each convolutional kernel corresponds to a feature map, and multiple feature maps can be obtained through convolutional operations.

[0155] Pooling layer: Samples the feature map output by the convolutional layer, such as max pooling or average pooling, to reduce the size of the feature map while retaining important feature information and reducing the computational load.

[0156] Activation function: An activation function, such as the ReLU function, is used between the convolutional layer and the fully connected layer to introduce non-linearity and enhance the expressive power of the model.

[0157] Fully connected layer: Integrates the features extracted by the convolutional layer and the pooling layer, and maps the features to a low-dimensional feature space through the fully connected layer.

[0158] The recurrent neural network unit includes:

[0159] Input layer: Takes time series features (such as multi-temporal vegetation indices) as input. The number of neurons in the input layer depends on the dimension of the time series.

[0160] Recurrent layer: Uses recurrent units (such as LSTM or GRU) to process time series data. The recurrent units can remember previous state information and handle long-term dependencies in the sequence.

[0161] Output layer: Maps the output of the recurrent layer to the final prediction result. The number of neurons in the output layer is usually 1, representing the predicted vegetation coverage.

[0162] A vegetation coverage prediction model is obtained by fusing the convolutional neural network unit and the recurrent neural network unit: The outputs of the convolutional neural network unit and the recurrent neural network unit are fused. This can be done through methods such as concatenation or weighted summation to combine spatial features and time series features, and then input into a fully connected layer for the final prediction.

[0163] S5. Input the multi-feature dataset and the ground truth labels into the constructed deep learning model for training, and use cross-validation and regularization methods to optimize the model parameters to obtain a trained vegetation coverage prediction model, specifically including:

[0164] S5.1. Divide the multi-feature dataset into a training set, a validation set, and a test set. Usually, the training set is used for model training, the validation set is used to adjust the hyperparameters of the model (such as learning rate, batch size, etc.), and the test set is used to evaluate the final performance of the model;

[0165] S5.2. Conduct model training:

[0166] S5.2.1. Select an appropriate loss function, such as the mean squared error (MSE) loss function, to measure the difference between the model prediction result and the ground truth labels. The calculation formula of the loss function is where y i is the ground truth label of the i-th sample, is the prediction result of the model, and M is the number of samples;

[0167] S5.2.2. Use an optimizer (such as Stochastic Gradient Descent SGD, Adam, etc.) to update the parameters of the model to minimize the value of the loss function. The optimizer adjusts the weights and biases of the model according to the gradient information of the loss function;

[0168] S5.2.3. Input the data of the training set into the model in batches for training. The data of each batch undergoes forward propagation to calculate the prediction result, then calculate the value of the loss function, and then calculate the gradient through backpropagation. Finally, use the optimizer to update the parameters of the model. Repeat this process until the loss function converges or reaches the preset number of training epochs;

[0169] S5.3. Adopt the k-fold cross-validation method to divide the training set data into k subsets. Each time, select k - 1 of these subsets as the training data, and the remaining 1 subset as the validation data, and perform k times of training and validation. Finally, average the results of the k times of validation to obtain the cross-validation performance of the model. Through cross-validation, the generalization ability of the model can be evaluated more accurately and overfitting can be avoided;

[0170] S5.4. Add a regularization term to the loss function, such as the L1 regularization term λ∑ i |θ i | or the L2 regularization term λ∑ i θ i 2 , where are the parameters of the model and is the regularization coefficient. The regularization term can constrain the parameters of the model to prevent overfitting. At the same time, during the training process, randomly set the outputs of some neurons to 0, so that the model cannot overly rely on certain neurons, thereby improving the generalization ability of the model.

[0171] S6. After preprocessing and feature extraction of the newly obtained multi-source remote sensing image data, input it into the trained vegetation coverage prediction model to obtain the predicted vegetation coverage result, which specifically includes:

[0172] S6.1. Perform the same preprocessing operations on the newly obtained multi-source remote sensing image data as the training data, including radiometric calibration, atmospheric correction, geometric correction, etc., to ensure the quality and consistency of the data;

[0173] S6.2. Extract spectral features, texture features, spatial features, and time series features from the preprocessed remote sensing image data to construct a multi-feature dataset in the same format as the training data;

[0174] S6.3. Input the extracted multi-feature dataset into the trained vegetation coverage prediction model, and obtain the predicted vegetation coverage result through forward propagation calculation;

[0175] S6.4. Post-process the prediction results, such as removing outliers and performing spatial smoothing operations, to improve the accuracy and reliability of the prediction results. Median filtering or Gaussian filtering can be used for spatial smoothing.

[0176] Furthermore, a remote sensing image vegetation coverage prediction system based on deep learning is proposed to implement the prediction method as described in any of the above items, including:

[0177] Data collection module, which is used to collect multi-source remote sensing image data;

[0178] Data preprocessing module, which is used to preprocess the collected multi-source remote sensing image data;

[0179] Feature extraction module, which is used to extract spectral features, texture features, spatial features and time series features from the preprocessed multi-source remote sensing image data to construct a multi-feature data set;

[0180] Model construction module, which is used to construct a vegetation coverage prediction model based on deep learning;

[0181] Model training module, which is used to input the multi-feature data set and the ground truth label into the constructed deep learning model for training, and optimize the model parameters by using cross-validation and regularization methods to obtain a trained vegetation coverage prediction model;

[0182] Prediction module, which is used to input the newly acquired multi-source remote sensing image data after preprocessing and feature extraction into the trained vegetation coverage prediction model to obtain the predicted vegetation coverage result.

[0183] Furthermore, the data collection module includes:

[0184] Remote sensing data acquisition unit, which is used to collect optical remote sensing images, synthetic aperture radar images and lidar data;

[0185] Ground truth data acquisition unit, which is used to collect the ground measured vegetation coverage data corresponding to the multi-source remote sensing image data to provide accurate label information for the model training module.

[0186] Furthermore, the data preprocessing module includes:

[0187] Optical remote sensing image preprocessing unit, which is used to perform radiometric calibration, atmospheric correction and geometric correction on the optical remote sensing image;

[0188] Synthetic aperture radar image denoising unit, which is used to denoise the synthetic aperture radar image;

[0189] A lidar data processing unit, which is used to perform point cloud filtering and classification processing on lidar data.

[0190] Furthermore, the model construction module includes:

[0191] A convolutional neural network construction unit, which is used to construct the input layer, convolutional layer, pooling layer, activation function and fully connected layer of the convolutional neural network for extracting the spatial features of remote sensing images;

[0192] A recurrent neural network construction unit, which is used to construct the input layer, recurrent layer and output layer of the recurrent neural network for processing time series features;

[0193] A model fusion unit, which is used to fuse the outputs of the convolutional neural network module and the recurrent neural network module to construct a complete vegetation coverage prediction model.

[0194] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection required by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting the vegetation coverage of remote sensing images based on deep learning, characterized in that, Including: S1. Collect multi-source remote sensing image data, where the multi-source remote sensing image data includes optical remote sensing images, synthetic aperture radar images, and lidar data. At the same time, collect the ground measured vegetation coverage data of the corresponding area as the ground truth label; S2. Preprocess the collected multi-source remote sensing image data, including radiometric calibration, atmospheric correction, and geometric correction for optical remote sensing images, denoising for synthetic aperture radar images, and point cloud filtering and classification for lidar data, to obtain the preprocessed multi-source remote sensing image data; S3. Extract spectral features, texture features, spatial features, and time series features from the preprocessed multi-source remote sensing image data to construct a multi-feature data set; S4. Construct a vegetation coverage prediction model based on deep learning. The vegetation coverage prediction model includes a convolutional neural network unit and a recurrent neural network unit. The convolutional neural network unit is used to extract the spatial features of remote sensing images, and the recurrent neural network unit is used to process time series features; S5. Input the multi-feature data set and the ground truth label into the constructed deep learning model for training, and use cross-validation and regularization methods to optimize the model parameters to obtain a trained vegetation coverage prediction model; S6. After preprocessing and feature extraction of the newly acquired multi-source remote sensing image data, input it into the trained vegetation coverage prediction model to obtain the predicted vegetation coverage result.

2. A method for predicting vegetation coverage of remote sensing images based on deep learning according to claim 1, characterized in that, The radiometric calibration of the optical remote sensing image specifically includes: S2.1.

1. Based on the optical remote sensing image, determine the calibration parameter file of the acquisition sensor corresponding to the optical remote sensing image; S2.1.

2. Obtain the gain coefficient and offset coefficient corresponding to the optical remote sensing image from the calibration parameter file; S2.1.

3. Obtain the original digital quantization value corresponding to each pixel in the optical remote sensing image and convert it into radiance value: L i = G·DN i + O, where L i represents the radiance value converted from the original digital quantization value corresponding to the i-th pixel in the optical remote sensing image, G represents the gain coefficient corresponding to the optical remote sensing image, and DN i represents the original digital quantization value corresponding to the i-th pixel in the optical remote sensing image, and O represents the offset coefficient corresponding to the optical remote sensing image.

3. A method for predicting vegetation coverage of remote sensing images based on deep learning according to claim 1, characterized in that The atmospheric correction of the optical remote sensing image specifically includes: S2.2.

1. Based on the optical remote sensing image, obtain the relevant atmospheric parameters when collecting the optical remote sensing image, including atmospheric water vapor content, aerosol optical thickness, solar zenith angle, observation zenith angle, and relative azimuth angle; S2.2.

2. Input the relevant atmospheric parameters corresponding to the optical remote sensing image into the 6S model to obtain the atmospheric transmittance, surface transmittance, and atmospheric path radiance required for atmospheric correction; S2.2.

3. Obtain the radiance value corresponding to the optical remote sensing image, and perform atmospheric correction on the radiance value corresponding to the optical remote sensing image based on the atmospheric transmittance, surface transmittance, and atmospheric path radiance to obtain the surface reflectance; Where ρ i represents the surface reflectance corresponding to the i-th pixel in the optical remote sensing image, Π is the pi, L i represents the radiance value converted from the original digital quantization value corresponding to the i-th pixel in the optical remote sensing image, L p represents the path radiance of the atmosphere corresponding to the optical remote sensing image, d represents the sun-earth distance corresponding to the optical remote sensing image, τ s represents the atmospheric transmittance, τ v represents the ground object transmittance, E s represents the solar spectral irradiance at the top of the atmosphere corresponding to the optical remote sensing image, θ s represents the solar zenith angle corresponding to the optical remote sensing image.

4. A method for predicting vegetation coverage of remote sensing images based on deep learning according to claim 1, characterized in that The geometric correction of the optical remote sensing image specifically includes: S2.3.

1. Obtain the high-precision reference topographic map corresponding to the optical remote sensing image, and select at least 6 homologous points as ground control points from the optical remote sensing image and the high-precision reference topographic map respectively; S2.3.

2. Obtain the image coordinate system and reference map coordinate system corresponding to the optical remote sensing image and the high-precision reference topographic map; S2.3.

3. Respectively obtain the coordinates (x, y) and (X, Y) of the 6 ground control points selected from the optical remote sensing image and the high-precision reference topographic map in the image coordinate system and the reference map coordinate system; S2.3.

4. Establish a polynomial transformation model: where a j and b j are both polynomial coefficients, and j = 0, 1, …, 5; S2.3.

5. Use the least squares method to fit the coordinates (x, y) and (X, Y) of the 6 ground control points in the image coordinate system and the reference map coordinate system, and determine the polynomial coefficients a j and b j ; S2.3.

6. Map each pixel in the optical remote sensing image to the reference map coordinate system according to the polynomial coefficients a j and b j . S2.3.

7. Obtain the new position of each pixel in the optical remote sensing image mapped to the reference map coordinate system, and perform resampling using bilinear interpolation to obtain the geometrically corrected optical remote sensing image.

5. A method for predicting vegetation coverage of remote sensing images based on deep learning according to claim 1, characterized in that, The denoising process for the synthetic aperture radar image specifically includes: S2.4.

1. Obtain the central pixel in the synthetic aperture radar image, and select a sliding window W of size m×m with the central pixel as the reference; S2.4.

2. Slide and intercept the synthetic aperture radar image using the sliding window W, with a sliding step of 1 pixel each time, and simultaneously determine the mean and variance of the pixels within each window; S2.4.

3. Determine the equivalent number of looks ENL corresponding to each window through the mean difference and variance of the pixels within each window; In the formula, is the square value of the mean of the pixels within each window, is the variance of the pixels within each window; S2.4.

4. According to the equivalent number of looks corresponding to each window, combined with the mean difference and variance of the pixels within each window, obtain the filtering coefficient C corresponding to each window; S2.4.

5. Use the filtering coefficient C corresponding to each window and the mean of the pixels within each window to filter the central pixel corresponding to each window to obtain the denoised central pixel value; y mid = μ W + C(x mid - μ W ); where y mid is the denoised central pixel value of the central pixel corresponding to each window, and x mid is the central pixel value corresponding to each window; S2.4.

6. Remove the denoised central pixel in each window, and obtain the local mean and local variance corresponding to all the remaining pixels; S2.4.

7. Determine the local equivalent number of looks L corresponding to each window according to the local mean and local variance corresponding to all the remaining pixels; where, μ 2 is the square value of the local mean corresponding to each window, and σ 2 is the local variance corresponding to each window; S2.4.

8. Filter all the remaining pixels in each window using the local equivalent number of looks to obtain the denoised local pixel values: where y sr is the local pixel value after denoising the remaining pixel values of each window, and x sr is the remaining pixel value of each window, and s and r are index labels.

6. A method for predicting the vegetation coverage of remote sensing images based on deep learning according to claim 1, characterized in that, The point cloud filtering and classification process for the lidar data specifically includes: S2.5.

1. Divide the lidar data into grids of size n×n; S2.5.

2. Traverse each n×n grid, and find the point with the minimum Z coordinate in it as the lowest point p of the corresponding n×n grid min ; S2.5.

3. For each point p within the grid, calculate its slope S with respect to the lowest point; where Z p and Z pmin are the Z - coordinate values of point p and the lowest point respectively, and D is the distance between point p and the lowest point p min on the plane; S2.5.

6. Set the slope threshold T s If the slope S of point p is greater than T s then this point is determined as a noise point and removed to obtain the lidar data after point cloud filtering; S2.5.

7. Extract the features corresponding to each point from the lidar data after point cloud filtering, including elevation, echo intensity, and neighborhood point density; S2.5.

8. Collect the points of known categories as training samples, construct a training dataset to train the SVM classifier, and obtain a classification model; S2.5.

9. Input the features of each point in the lidar data after point cloud filtering into the classification model to obtain the category label of this point, and complete the point cloud classification process.

7. A remote sensing image vegetation coverage prediction system based on deep learning, which is used to implement the prediction method described in any one of claims 1-6, and is characterized in that, Including: A data collection module, which is used to collect multi-source remote sensing image data; A data preprocessing module, which is used to preprocess the collected multi-source remote sensing image data; A feature extraction module, which is used to extract spectral features, texture features, spatial features, and time series features from the preprocessed multi-source remote sensing image data, and construct a multi-feature dataset; A model construction module, which is used to construct a vegetation coverage prediction model based on deep learning; A model training module, which is used to input a multi-feature dataset and ground truth labels into a constructed deep learning model for training, and optimize the model parameters by using cross-validation and regularization methods to obtain a trained vegetation coverage prediction model; A prediction module, which is used to input the newly acquired multi-source remote sensing image data after preprocessing and feature extraction into the trained vegetation coverage prediction model to obtain the predicted vegetation coverage result.

8. The vegetation coverage prediction system for remote sensing images based on deep learning according to claim 7, characterized in that The data collection module includes: A remote sensing data acquisition unit, which is used to collect optical remote sensing images, synthetic aperture radar images, and lidar data; A ground truth data acquisition unit, which is used to collect the ground-measured vegetation coverage data corresponding to the multi-source remote sensing image data to provide accurate label information for the model training module.

9. The remote sensing image vegetation coverage prediction system based on deep learning according to claim 7, characterized in that The data preprocessing module includes: An optical remote sensing image preprocessing unit, which is used to perform radiometric calibration, atmospheric correction, and geometric correction on the optical remote sensing images; A synthetic aperture radar image denoising unit, which is used to perform denoising processing on the synthetic aperture radar images; A lidar data processing unit, which is used to perform point cloud filtering and classification processing on the lidar data.

10. A remote sensing image vegetation coverage prediction system based on deep learning according to claim 7, characterized in that, The model construction module includes: A convolutional neural network construction unit, which is used to construct the input layer, convolutional layer, pooling layer, activation function, and fully connected layer of the convolutional neural network to extract the spatial features of the remote sensing images; A recurrent neural network construction unit, which is used to construct the input layer, recurrent layer, and output layer of the recurrent neural network to process the time series features; A model fusion unit, which is used to fuse the outputs of the convolutional neural network module and the recurrent neural network module to construct a complete vegetation coverage prediction model.

Images