Soil humidity multi-source remote sensing monitoring method and device and storage medium

Abstract

The invention relates to the field of remote sensing monitoring, and discloses a soil humidity multi-source remote sensing monitoring method and device and a storage medium, and the method comprises the steps: obtaining multi-source satellite remote sensing data; preprocessing the multi-source satellite remote sensing data to obtain preprocessed data; constructing a data set according to the preprocessed data; constructing a three-dimensional joint convolutional neural network, and training the three-dimensional joint convolutional neural network based on the data set to obtain a trained network; and predicting the soil humidity by using the trained network. The method has the beneficial effects that the problem of insufficient multi-source remote sensing data fusion and information complementation in the prior art is effectively solved, information complementation and feature fusion of the multi-source remote sensing data are realized, the robustness of the model is enhanced, and the prediction precision and stability are improved; the method has the characteristics of simplicity and effectiveness, does not need to research and analyze a complex remote sensing mechanism, and can more accurately approach a complex nonlinear function between remote sensing information and ground feature parameters.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing monitoring, and in particular to a soil moisture multi-source remote sensing monitoring method, equipment and storage medium. Background Art

[0002] Soil is an important natural resource on the earth's surface, and its parameters (such as humidity, temperature, organic matter content, etc.) are of great significance to agricultural production, ecological environment protection and water resource management. Although traditional soil parameter measurement methods, such as field sampling and laboratory analysis, can provide relatively accurate data, their operation process is cumbersome, time-consuming and labor-intensive, and it is difficult to achieve large-scale, high-frequency real-time monitoring.

[0003] Remote sensing technology has been used for soil moisture estimation in recent decades. For example, Hassan-Esfahani et al. developed a neural network method to provide reasonably accurate soil moisture estimates for irrigated farmland using high spatial resolution UAV system imagery, with a root mean square error of less than 0.3 m. 3 / m 3 ; Thermal infrared remote sensing inversion; Sadeghi and Babaeian et al. proposed a new optical trapezoidal model to replace the traditional triangle / trapezoidal method to estimate soil moisture from the reflectance data of Sentinel-2, Landsat-8 and MODIS shortwave infrared bands, and this method is applicable to new high-resolution optical satellites that do not contain thermal infrared bands; Microwave remote sensing technology captures the emissivity and backscattering information of microwaves on the soil surface to obtain the dielectric properties of water, air and solids. Le Hégarat-Mascle et al. proposed a soil moisture monitoring method based on ERS / SAR data based on the prior information of the linear relationship between soil moisture and SAR signals. The research results show that multi-source remote sensing technology provides a very powerful means for monitoring near-surface soil moisture.

[0004] However, remote sensing inversion of soil moisture involves multi-source data fusion and information complementarity, and few studies have considered the impact of multi-source data on soil moisture inversion. Summary of the invention

[0005] The purpose of the present invention is to propose a soil moisture multi-source remote sensing monitoring method, device and storage medium to solve the technical problem of inaccurate soil moisture inversion results caused by lack of multi-source information fusion in current soil moisture remote sensing inversion research.

[0006] The present invention provides a soil moisture multi-source remote sensing monitoring method, comprising the following steps:

[0007] S1. Acquire multi-source satellite remote sensing data;

[0008] S2, preprocessing multi-source satellite remote sensing data to obtain preprocessed data;

[0009] S3, constructing a data set based on the preprocessed data;

[0010] S4, constructing a three-dimensional joint convolutional neural network, and training the three-dimensional joint convolutional neural network based on the data set to obtain a trained network;

[0011] S5. Use the trained network to predict soil moisture.

[0012] A storage medium stores instructions and data for implementing a soil moisture multi-source remote sensing monitoring method.

[0013] A soil moisture multi-source remote sensing monitoring device comprises: a processor and the storage medium; the processor loads and executes instructions and data in the storage medium to implement a soil moisture multi-source remote sensing monitoring method.

[0014] The beneficial effects provided by the present invention are as follows: through the parallel structure of the spatial-spectral feature extraction block and the complex-valued three-dimensional convolution block, the spatial-spectral structure of the multispectral remote sensing image and the polarization complex image is extracted, which not only includes the context information of the remote sensing image, but also ensures the existence of the spectral information and the phase information, effectively solves the problem of insufficient multi-source remote sensing data fusion and information complementarity in the prior art, realizes the information complementarity and feature fusion of multi-source remote sensing data, enhances the robustness of the model, and improves the prediction accuracy and stability; at the same time, the present invention is based on the convolutional neural network technology, has the characteristics of simplicity and effectiveness, does not require research and analysis of complex remote sensing mechanisms, and can more accurately approximate the complex nonlinear functions between remote sensing information and ground object parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic flow chart of the method of the present invention;

[0016] Figure 2 It is the structural diagram of the 3D-Unified-CNN model;

[0017] Figure 3 It is the structural diagram of the spatial spectrum feature extraction block;

[0018] Figure 4 It is a structural diagram of the complex-valued 3D convolutional block;

[0019] Figure 5 It is a working schematic diagram of the hardware device of the present invention. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0021] Before formally explaining the present invention, relevant technical terms in the present invention are explained. In addition, the scheme of the present invention is also generally explained first for easy understanding.

[0022] Please refer to Figure 1 , Figure 1 It is a schematic diagram of the process of the present invention.

[0023] The present invention provides a soil moisture multi-source remote sensing monitoring method, comprising the following steps:

[0024] S1. Acquire multi-source satellite remote sensing data;

[0025] It should be noted that step S1 is specifically as follows:

[0026] S11. Obtaining experimental data of multiple soil samples;

[0027] S12. Obtain radar complex image IMG corresponding to radar data source based on soil sample experimental data CV and multispectral band image IMG MSSI .

[0028] As an embodiment, the present invention selects experimental data of n soil samples, and the sample collection position is P i =(x i ,y i ), i = 1, 2, ..., n, the corresponding soil moisture data is Y = {Y1, Y2, ..., Y n}, the collection date is D = {D1, D2..., D n}.

[0029] According to the collection date of each sample D i With geographical location P i , get the same radar data source D i Multi-polarization / full-polarization radar complex image IMG of the day CV ; At the same time, the distance D under the same optical data source is obtained i Multispectral band image IMG of the most recent date MSSI . IMG CV Contains at least the amplitude and phase information of two polarization channels, HH, HV, and VV, IMG MSSI It contains at least four band information: blue band, green band, red band and near-infrared band.

[0030] S2, preprocessing multi-source satellite remote sensing data to obtain preprocessed data;

[0031] It should be noted that step S2 specifically includes:

[0032] S21, radar complex image IMG CV Perform radiometric calibration, multi-view processing, speckle filtering, and geocoding processing in sequence to obtain the processed image IMG' MSSI ;

[0033] S22, multispectral band image IMG MSSI Perform radiation calibration, atmospheric correction, and orthorectification processing in sequence to obtain the processed image IMG' MSSI .

[0034] As an embodiment, the present invention uses ENVI 5.3 or other remote sensing professional software to complete radiation calibration, atmospheric correction, and orthorectification processing of multispectral images; and uses ESA SNAP or other radar image processing software to complete radiation calibration, multi-view processing, coherent speckle filtering, and geocoding processing of polarimetric radar images.

[0035] S3. Build a dataset based on the preprocessed data:

[0036] It should be noted that the process of constructing the data set in step S3 is as follows:

[0037] S31, the longitude and latitude coordinates P of the i-th data in the experimental data of the soil sample i =(x i ,y i ), i = 1, 2..., n is the center point, from the multispectral band image IMG' processed in step S2 MSSI Cut out the data of H1×W1 window X MSSI (H1, W1, D1), where D1 ≥ 4, H1 = W1 = 2k + 1, k∈Z;

[0038] S32, P i =(x i ,y i ), i = 1, 2..., n is the center of the radar complex image IMG' CV Cut out the data of H2×W2 window X CV (H2, W2, D2, 2), where D2 ≥ 2, H2 = W2 = 2k + 1, k∈Z, and the last dimension 2 represents the real and imaginary parts of the complex number;

[0039] S33, taking the center of the window H2×W2 as the origin, setting the length and width of a single pixel to 1, and calculating the geographic coordinates (x left , x right ,y up ,y down ) Calculate each sampling point P i Relative position in the window The calculation formula is as follows:

[0040]

[0041] Finally, we get n data sets A = [A1, A2..., A n};

[0042] in The data set is divided into a training set and a validation set according to a predetermined ratio by random division. Preferably, the predetermined ratio in the present invention is set to 8:2.

[0043] As an embodiment, in step S3, it is necessary to find the corresponding pixel position from the image according to the geographical location of the soil sample, and extract the data of the specified window size with this as the center. Assuming that all the sampling points are distributed within the range of an image, the latitude and longitude information of the sampling points have been stored in a CSV file. The exemplary pseudo code of the present invention is listed as follows:

[0044]

[0045]

[0046] The above is for illustrative purposes only.

[0047] Similarly, for local data extraction of polarization radar images, the second implementation method can also be referred to. After extracting the data of the corresponding area, the matrix can be reshaped by the reshape function.

[0048] When calculating the relative position of the sampling point in the local window of the complex image, the following calculation formula can be used for implementation:

[0049]

[0050] Where (x left , x right ,y up ,y down ) is the corner coordinate of the center pixel of the local window, (x i ,y i ) are the geographic coordinates of the sampling points.

[0051] After merging all the data sets, you can use the scikit-learn library in Python to randomly divide the data sets into training sets and validation sets in proportion. The pseudo code is as follows:

[0052] train_set, validation_set=train_test_split(dataset_A, test_size=0.2, random_state=42).

[0053] S4, constructing a three-dimensional joint convolutional neural network, and training the three-dimensional joint convolutional neural network based on the data set to obtain a trained network;

[0054] It should be noted that the construction process of the three-dimensional joint convolutional neural network is as follows:

[0055] Construct two input layers to accept multispectral band images X MSSI (H1, W1, D1), complex image X CV (H2, W2, D2, 2);

[0056] Construct an empty spectrum feature extraction block, accept X MSSI After being processed by the spatial spectrum feature extraction block, the spatial spectrum feature F is obtained. MSSI (C1, 3, 3, 1), where C1 is the number of channels of the empty spectrum feature.

[0057] Construct a complex-valued 3D convolution block, accepting X CV After being processed by the complex-valued 3D convolution block, the complex-valued feature F is obtained. CV (N2, 3, 3, 1, 2), where C2 is the number of channels of the complex-valued feature.

[0058] The empty spectrum feature F MSSI and the complex-valued characteristic F CV With the relative position parameter P REL After the variable convolution layer, the empty spectrum feature vector I of the target position is obtained MSSI (1, C1) and the real eigenvector I CV-real (1, C2), imaginary eigenvector I CV-imag (1, C2);

[0059] Finally, the empty spectrum eigenvector I is merged MSSI and the complex-valued eigenvector I CV Get the composite eigenvector I COM (1, C1+2×C2) and then three fully connected layers FC1, FC2, and FC3 are used to achieve accurate prediction of soil moisture.

[0060] As an example, please refer to Figure 2 , Figure 2 The structure diagram of the 3D-Unified-CNN deep learning model proposed in the present invention is shown. In establishing the 3D-Unified-CNN deep learning model (S4), two input layers are defined to receive the multispectral band image X MSSI (H1, W1, D1), complex image X CV (H2, W2, D2, 2); construct a spatial spectrum feature extraction block and a complex-valued three-dimensional convolution block, and use a parallel structure to improve computational efficiency.

[0061] After obtaining the spatial spectrum features and complex-valued features, they are input into the variable convolution layer to obtain three one-dimensional vectors, which are merged and input into the fully connected layer. After three fully connected layers FC1, FC2, and FC3, accurate prediction of soil moisture is achieved.

[0062] For further information, please refer to Figure 3 , Figure 3 The structural diagram of the spatial spectrum feature extraction block in the model is shown, which consists of 3 consecutive convolutional layers, 1 pooling layer, and 2 consecutive convolutional layers. The input of the first 3 consecutive convolutional layers is the set of input and output of the upper layer.

[0063] For further information, please refer to Figure 4 , Figure 4 It is a structural diagram of the complex-valued 3D convolutional block.

[0064] Figure 4 (a) in the figure shows the structural diagram of the complex-valued three-dimensional convolution block in the model, which is a full convolutional layer structure. Figure 4 (b) shows the steps of complex-valued convolution processing, for the input image X CV The complex-valued three-dimensional convolution block processed by (H2, W2, D2, 2) is a full convolution layer structure, and in the complex-valued three-dimensional convolution processing, the input tensor is first separated into real and imaginary parts, and then three-dimensional convolution operations are performed on the real and imaginary parts respectively, and the real and imaginary parts of the output complex numbers are calculated, and finally the results are merged into a complex tensor.

[0065] In step S4, the activation function of the convolutional layer in the 3D-Unified-CNN deep learning model is ReLU, and the activation function of the first two fully connected layers is Softplus. The formula is as follows:

[0066] ReLU(x)=max(0,x),

[0067] Softplus(x)=log(1+e x ),

[0068] The training set is used to train the 3D-Unified-CNN model. The model is optimized by the Adam optimizer. The loss function is the Huber function. The formula is as follows:

[0069]

[0070] Among them, Δy is the difference between the predicted value and the true value, and δ is a hyperparameter with a default value of 1.

[0071] In the prediction and evaluation of soil moisture (S5), the multi-source remote sensing image data in the validation set data is input into the trained 3D-Unified-CNN model. The root mean square error of the model output is used to evaluate the prediction performance. The calculation formula is:

[0072]

[0073] Where k is the number of test samples, Y i is the actual value of the test sample, is the predicted value of the test sample.

[0074] S5. Use the trained network to predict soil moisture.

[0075] As an embodiment, the present invention selects 200 sets of soil moisture sample data, collects multispectral band images of Gaofen-1 satellite IMG according to the spatiotemporal information of the samples MSSI (blue band, green band, red band, near infrared band, 8m resolution), and Gaofen-3 satellite full polarization strip (QPS I) HH, HV, VV polarization channel complex image IMG CV (Amplitude, phase, 8m resolution).

[0076] After preprocessing the Gaofen-1 image and Gaofen-3 image using ENVI 5.3 and ESA SNAP, a 23x23 window of data was cropped from the Gaofen-1 image. MSSI (23, 23, 4), crop the 7x7 window data X from the Gaofen-3 image CV (7, 7, 6) and then reshaped to X CV (7, 7, 3, 2).

[0077] And calculate each sampling point P according to the geographic coordinates of the four corner points of the central pixel i Relative position in the window Finally, we get 200 data sets A = {A1, A2..., A 200},

[0078] in

[0079] The data set A is randomly divided into a training set and a validation set in a ratio of 8:2. The training set contains 160 samples and the test set contains 40 samples.

[0080] When building the 3D-Unified-CNN deep learning model, two input layers are defined to accept multispectral band images X MSSI (23, 23, 4), the fully polarized complex image X CV(7, 7, 3, 2); construct a spatial spectrum feature extraction block and a complex-valued three-dimensional convolution block, and use a parallel structure to improve computational efficiency.

[0081] X MSSI After multiple extraction blocks are processed, the empty spectrum feature F is obtained. MSSI (64,3,3,1),X CV After processing by multiple convolution blocks, the complex-valued feature F is obtained. CV (16, 3, 3, 1, 2).

[0082] According to relative position After the variable convolution layer, the empty spectrum feature vector I of the target position is obtained MSSI (1, 64) and the real eigenvector I CV-real (1, 16), imaginary eigenvector I CV-imag (1, 16); after merging, we get the feature vector I ALL (1, 96) is input into the fully connected layer, and the accurate prediction of soil moisture is achieved through three fully connected layers FC1, FC2, and FC3. The number of neurons in FC1 is 96, the number of neurons in FC2 is 32, and the number of neurons in FC3 is 1, which is the predicted value of soil moisture.

[0083] The structures of the spatial spectrum feature extraction block and the complex-valued three-dimensional convolution block are shown in the following table.

[0084] Table 1 Structure of the spatial spectrum feature extraction block and the complex-valued 3D convolution block

[0085]

[0086] 160 training samples were input into the 3D-Unified-CNN deep learning model for training, with a batch size of 16 and an iteration number of 100. The model was optimized using the Adam optimizer, with the Huber loss function, and the early stopping method was used to stop training in advance when the performance of the validation set no longer improved, to avoid overfitting caused by too many iterations.

[0087] The 40 validation samples were input into the trained 3D-Unified-CNN model for prediction, and the root mean square error was used to evaluate the prediction performance. Finally, the model that passed the performance evaluation was used for soil moisture inversion prediction.

[0088] See also Figure 5 , Figure 5 40 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically comprises: a soil moisture multi-source remote sensing monitoring device 401, a processor 402 and a storage medium 403.

[0089] A soil moisture multi-source remote sensing monitoring device 401: The soil moisture multi-source remote sensing monitoring device 401 implements the soil moisture multi-source remote sensing monitoring method.

[0090] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the soil moisture multi-source remote sensing monitoring method.

[0091] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the soil moisture multi-source remote sensing monitoring method.

[0092] The beneficial effects of the present invention are as follows: through the parallel structure of the spatial-spectral feature extraction block and the complex-valued three-dimensional convolution block, the spatial-spectral structure of the multispectral remote sensing image and the polarization complex image is extracted, which not only includes the context information of the remote sensing image, but also ensures the existence of the spectral information and the phase information, effectively solves the problem of insufficient multi-source remote sensing data fusion and information complementarity in the prior art, realizes the information complementarity and feature fusion of multi-source remote sensing data, enhances the robustness of the model, and improves the prediction accuracy and stability; at the same time, the present invention is based on convolutional neural network technology, has the characteristics of simplicity and effectiveness, does not require research and analysis of complex remote sensing mechanisms, and can more accurately approximate the complex nonlinear functions between remote sensing information and ground object parameters.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A soil moisture multi-source remote sensing monitoring method, characterized in that: The method comprises the following steps: S1. Acquire multi-source satellite remote sensing data; S2, preprocessing multi-source satellite remote sensing data to obtain preprocessed data; S3, constructing a data set based on the preprocessed data; S4, constructing a three-dimensional joint convolutional neural network, and training the three-dimensional joint convolutional neural network based on the data set to obtain a trained network; S5. Use the trained network to predict soil moisture.

2. A soil moisture multi-source remote sensing monitoring method as claimed in claim 1, characterized in that: Step S1 is specifically as follows: S11. Obtaining experimental data of multiple soil samples; S12. Obtain radar complex image IMG corresponding to radar data source based on soil sample experimental data CV and multispectral band image IMG MSSI .

3. A soil moisture multi-source remote sensing monitoring method as claimed in claim 2, characterized in that: Step S2 specifically includes: S21, radar complex image IMG CV Perform radiometric calibration, multi-view processing, speckle filtering, and geocoding processing in sequence to obtain the processed image IMG' MSSI ; S22, multispectral band image IMG MSSI Perform radiation calibration, atmospheric correction, and orthorectification processing in sequence to obtain the processed image IMG' MSSI .

4. A soil moisture multi-source remote sensing monitoring method as claimed in claim 3, characterized in that: The process of constructing the data set in step S3 is as follows: S31, the longitude and latitude coordinates P of the i-th data in the experimental data of the soil sample i =(x i ,y i ), i=1,2……,n is the center point, from the multispectral band image IMG' processed in step S2 MSSI Cut out the data of H1×W1 window X MSSI (H1, W1, d1), where D1 ≥ 4, H1 = W1 = 2k + 1, k∈Z; S32, P i =(x i ,t i ), i=1,2……,n is the center of the radar complex image IMG' CV Cut out the data of H2×W2 window X CV (H2,W2,D2,2), where D2≥2,H2=W2=2k+1,k∈Z, and the last dimension 2 represents the real and imaginary parts of the complex number; S33, taking the center of the window H2×W2 as the origin, setting the length and width of a single pixel to 1, and calculating the geographic coordinates (x left ,x right ,y up ,y down ) Calculate each sampling point P i Relative position in the window The calculation formula is as follows: Finally, we get n data sets A={A1,H2……,A n }; in The data set is divided into a training set and a validation set according to a predetermined ratio by random partitioning.

5. A soil moisture multi-source remote sensing monitoring method as claimed in claim 4, characterized in that: The construction process of the three-dimensional joint convolutional neural network is as follows: Construct two input layers to accept multispectral band images X MSSI (W1,W1,D1), complex image X CV (H2,W2,D2,2); Construct an empty spectrum feature extraction block, accept X MSSI After being processed by the spatial spectrum feature extraction block, the spatial spectrum feature F is obtained. MSSI (C1,3,3,1), where C1 is the number of channels of the empty spectrum feature; Construct a complex-valued 3D convolution block, accepting X CV After being processed by the complex-valued 3D convolution block, the complex-valued feature F is obtained. CV (N2,3,3,1,2), where C2 is the number of channels of the complex-valued feature; The empty spectrum feature F MSSI and the complex-valued characteristic F CV With the relative position parameter P RRL After the variable convolution layer, the empty spectrum feature vector I of the target position is obtained MSSI (1,C1) and the real eigenvector I CV-real (1,C2), imaginary eigenvector I CV-imag (1,C2); Finally, the empty spectrum eigenvector I is merged MSSI and the complex-valued eigenvector I CV Get the composite eigenvector I COM (1, C1+2×C2) and then three fully connected layers FC1, FC2, and FC3 are used to achieve accurate prediction of soil moisture.

6. A soil moisture multi-source remote sensing monitoring method as claimed in claim 5, characterized in that: For the input image X CV The complex-valued three-dimensional convolution block processed by (H2, W2, D2, 2) is a full convolution layer structure, and in the complex-valued three-dimensional convolution processing, the input tensor is first separated into real and imaginary parts, and then three-dimensional convolution operations are performed on the real and imaginary parts respectively, the real and imaginary parts of the output complex numbers are calculated, and finally the results are merged into a complex tensor.

7. A soil moisture multi-source remote sensing monitoring method as claimed in claim 1, characterized in that: The three-dimensional joint convolutional neural network is trained using the training set. The model is optimized using the Adam optimizer. The loss function is the Huber function, and the formula is as follows: Among them, Δy is the difference between the predicted value and the true value, δ is a hyperparameter, and the default value is; after training, the activation value obtained from the fully connected layer FC3 is the monitoring result of soil moisture by the three-dimensional joint convolutional neural network based on the extracted image features.

8. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a soil moisture multi-source remote sensing monitoring method as described in any one of claims 1 to 7.

9. A soil moisture multi-source remote sensing monitoring device, characterized in that: include: Processor and storage medium; the processor loads and executes instructions and data in the storage medium to implement a soil moisture multi-source remote sensing monitoring method as described in any one of claims 1 to 7.

Images