A soil profile layer division method based on indoor imaging spectroscopy technology

Abstract

The present invention discloses a soil profile occurrence layer division method based on indoor imaging spectral technology, including: S1, using a pixel mosaic-based method to obtain a complete soil profile hyperspectral image; S2, geometrically correcting the image, removing the wooden frame, and retaining the original soil profile image of a fixed width; S3, using a random forest algorithm to remove invalid pixels such as cracks and shadows; S4, smoothing and denoising the retained spectral data; S5, extracting the row average spectral curve, obtaining the fixed depth row average spectral curve as the spectral data of the layered strip soil sample; S6, generating a spectral surface; S7, dividing the occurrence layer based on a semantic segmentation network model. The present invention can realize the accurate division of various soil profile occurrence layers, and provides strong technical support for large-scale soil classification and sustainable management of soil resources.

Description

Technical Field

[0001] The invention belongs to the field of soil informatics, and in particular relates to a soil profile occurrence layer division method based on indoor imaging spectroscopy technology. Background Art

[0002] The study of soil science originates from the in-depth observation of soil profiles and their morphological characteristics. The genetic layers in the profiles provide valuable information for revealing the formation mechanism and development process of soil diversity. The types and number of genetic layers are the basis for studying the soil evolution process and conducting soil classification. Their accurate division is of great guiding significance for assessing soil health, predicting crop growth potential, and optimizing the utilization of water and fertilizer resources. Soil horizon identification is generally performed by soil experts through on-site description and delineation of soil profile morphological characteristics, and further calibration can be performed based on laboratory analysis results. However, this method is highly dependent on the soil science knowledge and practical experience of field observers, is labor-intensive and has a certain degree of subjectivity, and individual differences pose a challenge to the accuracy and repeatability of soil classification. Therefore, a standardized method and tool are needed to minimize its impact so that the division results of the genetic layers are more objective, convenient, and accurate.

[0003] As a cutting-edge tool in the field of soil analysis, imaging spectroscopy overcomes the limitations of traditional fixed-depth sampling and can obtain soil property variation characteristics at continuous depths. This technology can simultaneously capture the spatial distribution and spectral characteristics of soil, and through high-resolution imaging in multiple narrow and continuous spectral bands, it provides an accurate detection method for in-depth research on soil physical and chemical properties and their spatial variability distribution. Imaging spectroscopy can achieve rapid and accurate classification of soil types, and reduces large-scale field sampling and laboratory physical and chemical analysis. It is an effective new way to build a soil profile imaging spectral database and achieve accurate soil classification. Due to its rapid and non-destructive nature, and the advantages of unified image and spectrum and high spectral resolution, imaging spectroscopy has been widely used in the prediction of many key soil property studies and mapping.

[0004] However, there are still some problems to be solved in the current application of soil layer identification based on imaging spectroscopy technology. The patent application with application number 201710192544.6, "A soil type identification method based on spectral surface matching", realizes soil type identification by interpolating and derivation of the point spectral reflectance of the soil profile layer. The paper "Preliminary Study on the Division of Soil Profile Layers Based on Imaging Spectroscopy Technology" discloses the use of support vector machines to model and classify the principal components extracted from imaging spectral data, proving the feasibility of using imaging spectral data for profile layer division. The limitations of the above two studies are: (1) Due to limited sampling representativeness, insufficient consideration of spatial variability, and large amount of computational effort in spectral data processing, the recognition accuracy and efficiency still need to be improved. (2) The existing technology still faces some challenges when processing hyperspectral images, such as image noise, crack interference, and the influence of invalid pixels, which limit the accuracy and efficiency of soil layer division. Therefore, using imaging spectra to generate spectral surfaces and applying them to soil profile layer identification will be a new exploration process. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a soil profile layer division method based on indoor imaging spectroscopy technology, which can automatically divide and identify soil profile layers based on spectral images, and provides strong technical support for large-scale soil classification and sustainable management of soil resources.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a soil profile layer division method based on indoor imaging spectroscopy technology, comprising the following steps:

[0007] S1, using pixel mosaic method to obtain complete soil profile spectral image;

[0008] S2, geometrically correct the spectral image and retain the original soil profile image with a fixed width;

[0009] S3, using the random forest model to remove invalid pixels; specifically including the following sub-steps:

[0010] S3.1. Use the random forest model to classify the image into two categories, remove invalid crack pixels and invalid shadow pixels, and set the spectral reflectance value of the invalid pixel area to 0; the random forest model uses grid search to perform hyperparameter tuning, and uses grid search to find the best parameter combination. The formula is as follows:

[0011] ,

[0012] Among them, c is the category label, is an indicator function, which is 1 when the prediction result of the i-th tree is equal to c, otherwise it is 0;

[0013] S3.2. The accuracy of the random forest model is evaluated by the overall accuracy OA and Kappa coefficient, as follows:

[0014] ,

[0015] ;

[0016] in, r Indicates the total number of classification categories; N is the total number of samples, that is, the sum of samples in all categories; x ii It is i The number of samples correctly classified in the class; Po It represents the observed consistency ratio, that is, the ratio of the sum of correctly classified samples in each category to the total number of samples; P It represents the expected consistency ratio, that is, the ratio of the sum of the products of the number of true samples and the number of predicted samples in each category to the total number of samples if the classifier is randomly classified.

[0017] S4, performing smoothing and denoising processing on the spectral data of the soil profile spectral image after removing invalid pixels;

[0018] S5, extracting a row average spectrum curve from the smoothed spectrum image as spectrum data of the layered strip soil sample;

[0019] S6, generating a spectral surface and a depth function curve for the spectral data of the layered strip soil sample;

[0020] S7, constructing a semantic segmentation network, taking the normalized spectral surface image as input and the multi-channel classification result as output to train the semantic segmentation network, and obtaining a soil profile layer division model; inputting the soil profile spectral image into the soil profile layer division model, and obtaining a layered spectrum. Specifically including the following sub-steps:

[0021] S7.1. Normalize the spectral surface image to ensure that the value of the data when it is input into the network is in the range of [0,1]. According to the depth and wavelength, mark the spectral surface image with different categories according to the set occurrence layer boundary as the training label for semantic segmentation.

[0022] S7.2. Construct a semantic segmentation network model U-Net++, and set the model input to the normalized spectral surface image, and the output to the multi-channel classification result, where each channel corresponds to an occurrence layer category; set the model loss function to the cross entropy loss function, and select the adaptive moment estimation as the optimizer;

[0023] S7.3. Use the training set data to train the model to ensure that the model can better identify different occurrence layer areas after each iteration;

[0024] S7.4. Use the validation set to validate the model, evaluate the accuracy of the model, and adjust the model's hyperparameters based on the validation results.

[0025] S7.5. Apply the trained model to the unlabeled spectral surface image, predict the occurrence layer category of each pixel, generate the occurrence layer division result, color each pixel according to the category output by the model, and obtain the stratified spectrum.

[0026] Furthermore, the aforementioned step S1 includes the following sub-steps:

[0027] S1.1. Collecting undisturbed soil profile imaging spectral images according to a preset spectral sampling interval and a preset spectral curve detection coverage range;

[0028] S1.2, rotating and mosaicking the original soil profile imaging spectrum image;

[0029] S1.3, using a pixel mosaic method, adjusting the size of the mosaic area, adjusting image overlap and color balance, and outputting the mosaicked image;

[0030] S1.4, return to step S1.3, until the mosaic processing of all original soil profile imaging spectral images is completed to obtain a complete soil full profile spectral image.

[0031] Furthermore, the aforementioned step S2 is specifically as follows: geometrically correcting the image to eliminate the distortion generated during the shooting process, removing the wooden frame and the background outside the field of view in the image by cropping, and retaining the original image of the soil profile with a fixed width.

[0032] Furthermore, the aforementioned step S4 includes the following sub-steps:

[0033] S4.1. For any soil profile spectral image with invalid pixels removed, read the soil spectral reflectance and remove the noise components at both ends of the initial spectrum.

[0034] S4.2, use Savitzky-Golay filter to smooth the spectral data, as follows:

[0035] ,

[0036] in, i Indicates that when smoothing, the current wavelength k The offset between the value at and its adjacent value, is the wavelength k The smoothed average value is h i is the smoothing coefficient, wis the width of the smoothing window, H To normalize the coefficients, the number of windows is set to 25 and the polynomial degree is 3.

[0037] Furthermore, the aforementioned step S5 specifically comprises: extracting corresponding row average spectra at fixed depth intervals from the smoothed spectral image, and extracting principal components of the spectral data using principal component analysis (PCA) according to the soil profile occurrence layer.

[0038] Furthermore, the aforementioned step S6 includes the following sub-steps:

[0039] S6.1. For the processed spectral reflectance data, a spectral surface in tiff format is generated with the band as the abscissa and the profile depth as the ordinate;

[0040] S6.2. Extract the average spectral curve of the soil formation layer according to the knowledge and experience of soil experts until the average spectral curve of all layers in the profile is extracted.

[0041] Compared with the prior art, the beneficial technical effects of the present invention using the above technical solution are as follows:

[0042] The present invention proposes a soil profile layer division method based on indoor imaging spectroscopy technology, which can address the problems of strong human subjectivity, high labor intensity, and lack of consistency and repeatability in the field of soil profile layer division in the past. Based on the semantic segmentation algorithm, a model is generated to achieve accurate division of various soil profile layers, providing strong technical support for large-scale soil classification and sustainable management of soil resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of a flow chart of an implementation mode of the present invention.

[0044] Figure 2 It is a schematic diagram of the spectral surface of the soil profile M46 in this embodiment. DETAILED DESCRIPTION

[0045] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0046] Various aspects of the invention are described herein with reference to the accompanying drawings, in which many illustrative embodiments are shown. The embodiments of the invention are not limited to those described in the accompanying drawings. It should be understood that the invention is implemented by any of the various concepts and embodiments described above, as well as the concepts and embodiments described in detail below, because the concepts and embodiments disclosed in the invention are not limited to any implementation. In addition, some aspects disclosed in the invention may be used alone or in any appropriate combination with other aspects disclosed in the invention.

[0047] The soil profile samples used in this embodiment were collected from 108 profiles in 25 provincial administrative regions in China. Based on the knowledge, experience and laboratory physical and chemical analysis results of soil experts, these profiles were divided into 14 soil orders, 39 suborders, and 105 soil types according to the Chinese soil system classification, and the occurrence layer level of each profile was divided. The 389-1045 nm visible near-infrared spectrum of each undisturbed soil profile was measured using an INFINITY V10E imaging spectrometer (Lumenera, Canada), with a spectral interval of 2.60 nm.

[0048] This embodiment selects one typical soil profile M46 (simple ever-wet prototype soil) to further explain the present invention in detail with respect to the entire process of data preprocessing, obtaining row average spectral curve, principal component analysis, generating spectral surface, etc. Figure 1 This is a flow chart of an embodiment of the present invention. This embodiment selects a typical soil profile M46 (simple wet soil) and the entire process of data preprocessing, obtaining the average spectrum curve, principal component analysis, generating the spectrum surface, etc. is as follows:

[0049] The present invention provides a soil profile layer division method based on indoor imaging spectroscopy technology, comprising the following steps:

[0050] S1, using pixel mosaic method to obtain complete soil profile spectral image;

[0051] S2, geometrically correct the spectral image and retain the original soil profile image with a fixed width;

[0052] S3, using the Random Forest (RF) model to remove invalid pixels;

[0053] S4, performing smoothing and denoising processing on the spectral data of the soil profile spectral image after removing invalid pixels;

[0054] S5, extracting a row average spectrum curve from the smoothed spectrum image as spectrum data of the layered strip soil sample;

[0055] S6, generating a spectral surface and a depth function curve for the spectral data of the layered strip soil sample;

[0056] S7. Construct a semantic segmentation network, take the normalized spectral surface image as input, and use the multi-channel classification result as output to train the semantic segmentation network to obtain a soil profile layer division model; input the soil profile spectral image into the soil profile layer division model to obtain a stratified spectrum.

[0057] As a preferred embodiment of the present invention, step S1 includes the following sub-steps:

[0058] S1.1. The INFINITY V10E imaging spectrometer (Lumenera, Canada) was used to collect undisturbed soil profile imaging spectral images. The detection range of each spectral curve covered 389-1045 nm, and the spectral sampling interval was 2.60 nm.

[0059] S1.2. The spectral image of the original soil profile was rotated and mosaicked using ENVI Classic software (V5.6). To ensure mosaicking accuracy, image registration was performed according to the overlapping areas of the two images, and the pixel positions, i.e., row and column numbers, were obtained based on the same ground feature landmarks.

[0060] S1.3, using the pixel mosaic method, appropriately adjust the size of the mosaic area, adjust the image overlap and color balance through the 'Edit Entry' option, and output the mosaicked image;

[0061] S1.4, return to step S1.3, until the mosaic processing of all original soil profile imaging spectral images is completed, and a complete 1m soil full profile spectral image is obtained.

[0062] As a preferred embodiment of the present invention, step S2 specifically includes: geometrically correcting the image by using ENVI Classic software (V5.6) to eliminate the distortion generated during the shooting process. The wooden frame and the background outside the field of view in the image are removed by cropping, and the original soil profile image with a fixed width (400 columns) is retained.

[0063] Typical areas with cracks in different profiles are selected for perceptual interest areas of soil and cracks, and training and validation sets are established. Based on the statistical data of the areas of interest in the data set, the wavelength, soil reflectivity mean, and crack reflectivity mean are extracted to train the model. As shown in Table 1, the training data example table:

[0064] Table 1

[0065]

[0066] As a preferred embodiment of the present invention, step S3 includes the following sub-steps:

[0067] S3.1. Use the random forest algorithm to classify the image into two categories, remove invalid pixels such as cracks and shadows, and set the spectral reflectance values ​​of these areas to 0. The training data and test data of the random forest model come from manually annotated valid and invalid pixels. Through the grid search method, different parameter combinations are systematically traversed to find the optimal parameter combination of the model. The formula is as follows:

[0068] ,

[0069] Among them, c is the category label, is an indicator function, which is 1 when the prediction result of the i-th tree is equal to c, otherwise it is 0.

[0070] S3.2. The accuracy of the model is evaluated by Overall Accuracy (OA) and Cohen's Kappa coefficient (Kappa). The specific formula is as follows:

[0071] S3.2. The accuracy of the random forest model is evaluated by the overall accuracy OA and Kappa coefficient, as follows:

[0072] ,

[0073] ;

[0074] in, r Indicates the total number of classification categories; N is the total number of samples, that is, the sum of samples in all categories; x ii It is i The number of samples correctly classified in the class; Po It represents the observed consistency ratio, that is, the ratio of the sum of correctly classified samples in each category to the total number of samples; P It represents the expected consistency ratio, that is, the ratio of the sum of the products of the number of true samples and the number of predicted samples in each category to the total number of samples if the classifier is randomly classified.

[0075] As a preferred embodiment of the present invention, step S4 includes the following sub-steps:

[0076] S4.1. For any soil profile spectral image with invalid pixels removed, read the soil spectral reflectance, remove the noise components at both ends of the initial spectrum, and the retained spectral data is the 428 to 970 nm band in the initial spectrum;

[0077] S4.2, the spectral data is smoothed using the Savitzky-Golay filter, the number of windows is set to 25, and the polynomial degree is 3; the specific formula is as follows:

[0078] ,

[0079] in, i Indicates that when smoothing, the current wavelength k The offset between the value at and its adjacent value, is the wavelength k The smoothed average value is h iis the smoothing coefficient, w is the width of the smoothing window, H To normalize the coefficients, the number of windows is set to 25 and the polynomial degree is 3.

[0080] As a preferred embodiment of the present invention, step S5 specifically includes: dividing the 1m full profile into 200 depths with a fixed depth interval of 0.5cm. For the smoothed spectral reflectance, the row average spectrum of each layer is obtained according to the occurrence layer divided by soil experts. In this embodiment, the number of occurrence layers of profile M46 is 5, such as Figure 2 For any occurrence layer of the cross section, the principal component analysis of the reflectance spectrum data is performed to obtain the principal components of the spectrum data of the five occurrence layers.

[0081] As a preferred embodiment of the present invention, step S6 includes the following sub-steps:

[0082] S6.1. For the processed spectral reflectance data, a spectral surface in tiff format is generated with the band as the abscissa and the profile depth as the ordinate;

[0083] S6.2. Extract the average spectral curve of the soil formation layer according to the knowledge and experience of soil experts until the average spectral curve of all layers in the profile is extracted.

[0084] As a preferred embodiment of the present invention, step S7 includes the following sub-steps:

[0085] S7.1. Normalize the spectral surface image to ensure that the value of the data when it is input into the network is in the range of [0,1]. According to the depth and wavelength, mark the spectral surface image with different categories according to the set occurrence layer boundaries as training labels for semantic segmentation.

[0086] S7.2. Construct a semantic segmentation network model U-Net++, and set the model input to the normalized spectral surface image, and the output to the multi-channel classification result, where each channel corresponds to an occurrence layer category; set the model loss function to the cross entropy loss function, and select the optimizer Adaptive Moment Estimation (Adam);

[0087] S7.3. Use the training set data to train the model to ensure that the model can better identify different occurrence layer areas after each iteration;

[0088] S7.4. Use the validation set to validate the model, evaluate the accuracy of the model, and adjust the model's hyperparameters based on the validation results.

[0089] S7.5. Apply the trained model to the unlabeled spectral surface image, predict the occurrence layer category of each pixel, generate the occurrence layer division result, color each pixel according to the category output by the model, and obtain the stratified spectrum.

[0090] Although the present invention has been described above with preferred embodiments, it is not intended to limit the present invention. A person skilled in the art of the present invention may make various modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the definition of the claims.

Claims

1. A soil profile layer division method based on indoor imaging spectroscopy technology, characterized in that: The following steps are involved: S1, using pixel mosaic method to obtain complete soil profile spectral image; S2, geometrically correct the spectral image and retain the original soil profile image with a fixed width; S3, using random forest model to remove invalid pixels; specific sub-steps are as follows: S3.

1. Use the random forest model to classify the image into two categories, remove invalid crack pixels and invalid shadow pixels, and set the spectral reflectance value of the invalid pixel area to 0; the random forest model uses grid search to perform hyperparameter tuning, and uses grid search to find the best parameter combination. The formula is as follows: , Among them, c is the category label, is an indicator function, which is 1 when the prediction result of the i-th tree is equal to c, otherwise it is 0; S3.

2. The accuracy of the random forest model is evaluated by the overall accuracy OA and Kappa coefficient, as follows: ， ； in, r Indicates the total number of classification categories; N is the total number of samples, that is, the sum of samples in all categories; x ii It is i The number of samples correctly classified in the class; Po It represents the observed consistency ratio, that is, the ratio of the sum of correctly classified samples in each category to the total number of samples; P It represents the expected consistency ratio, that is, the ratio of the sum of the product of the number of true samples and the number of predicted samples in each category to the total number of samples if the classifier is randomly classified; S4, performing smoothing and denoising processing on the spectral data of the soil profile spectral image after removing invalid pixels; S5, extracting a row average spectrum curve from the smoothed spectrum image as spectrum data of the layered strip soil sample; S6, generating a spectral surface and a depth function curve for the spectral data of the layered strip soil sample; S7, constructing a semantic segmentation network, taking the normalized spectral surface image as input and the multi-channel classification result as output to train the semantic segmentation network, and obtaining a soil profile layer division model; inputting the soil profile spectral image into the soil profile layer division model to obtain a layered spectrum; specifically comprising the following sub-steps: S7.

1. Normalize the spectral surface image to ensure that the value of the data when it is input into the network is in the range of [0,1]. According to the depth and wavelength, mark the spectral surface image with different categories according to the set occurrence layer boundary as the training label for semantic segmentation. S7.

2. Construct a semantic segmentation network model U-Net++, and set the model input to the normalized spectral surface image, and the output to the multi-channel classification result, where each channel corresponds to an occurrence layer category; set the model loss function to the cross entropy loss function, and select the adaptive moment estimation as the optimizer; S7.

3. Use the training set data to train the model to ensure that the model can better identify different occurrence layer areas after each iteration; S7.

4. Use the validation set to validate the model, evaluate the accuracy of the model, and adjust the model's hyperparameters based on the validation results. S7.

5. Apply the trained model to the unlabeled spectral surface image, predict the occurrence layer category of each pixel, generate the occurrence layer division result, color each pixel according to the category output by the model, and obtain the stratified spectrum.

2. The method for dividing soil profile layers based on indoor imaging spectroscopy technology according to claim 1, characterized in that: Step S1 includes the following sub-steps: S1.

1. Collecting undisturbed soil profile imaging spectral images according to a preset spectral sampling interval and a preset spectral curve detection coverage range; S1.2, rotating and mosaicking the original soil profile imaging spectrum image; S1.3, using a pixel mosaic method, adjusting the size of the mosaic area, adjusting image overlap and color balance, and outputting the mosaicked image; S1.4, return to step S1.3, until the mosaic processing of all original soil profile imaging spectral images is completed to obtain a complete soil full profile spectral image.

3. The method for dividing soil profile layers based on indoor imaging spectroscopy technology according to claim 1, characterized in that: Step S2 specifically includes: performing geometric correction on the image to eliminate the distortion generated during the shooting process, removing the wooden frame and the background outside the field of view in the image by cropping, and retaining the original soil profile image with a fixed width.

4. The method for dividing soil profile layers based on indoor imaging spectroscopy technology according to claim 1, characterized in that: Step S4 includes the following sub-steps: S4.

1. For any soil profile spectral image with invalid pixels removed, read the soil spectral reflectance and remove the noise components at both ends of the initial spectrum. S4.2, use Savitzky-Golay filter to smooth the spectral data, as follows: ， in, i Indicates that when smoothing, the current wavelength k The offset between the value at and its adjacent value, is the wavelength k The smoothed average value is h i is the smoothing coefficient, w is the width of the smoothing window, H To normalize the coefficients, the number of windows is set to 25 and the polynomial degree is 3.

5. The method for dividing soil profile layers based on indoor imaging spectroscopy technology according to claim 1, characterized in that: Step S5 specifically includes: extracting the corresponding row average spectrum at fixed depth intervals from the smoothed spectral image, and extracting the principal components of the spectral data using principal component analysis (PCA) according to the soil profile layer.

6. The method for dividing soil profile layers based on indoor imaging spectroscopy technology according to claim 1, characterized in that: Step S6 includes the following sub-steps: S6.

1. For the processed spectral reflectance data, a spectral surface in tiff format is generated with the band as the abscissa and the profile depth as the ordinate; S6.

2. Extract the average spectral curve of the soil formation layer according to the knowledge and experience of soil experts until the average spectral curve of all layers in the profile is extracted.

Images