A method for classifying particulate organic matter into fresh POM and old POM based on CT images
Through CT images and machine learning technology, the accurate distinction between fresh particulate organic matter and old POM in the soil is solved, and the problem of difficult to identify fresh and old POM in the soil is achieved, and in-depth research on soil organic matter and optimization of straw return measures is achieved.
Patent Information
- Application Number
- CN202410419965.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-09
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-04-09
AI Technical Summary
The prior art is difficult to accurately identify and distinguish the differences between fresh particulate organic matter (POM) and old POM in soil, which affects the physical protection mechanism of soil organic carbon and the optimization of straw return measures.
Using CT images combined with machine learning technology, the particle organic matter is identified and segmented through sample CT scan, image preprocessing, machine learning, and its morphological characteristic parameters are calculated, and the random forest model is used to divide it into fresh residues and old POM to obtain its space Distribution information.
The accurate distinction between the particulate organic matter in the soil is achieved, and the decomposition and transformation process and spatial morphological structure is deeply understood, which promotes in-depth research on the source of POM and the circulation of soil organic matter, and optimizes the measures for returning straw to the fields.
Smart Images

Figure CN118521657B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of soil science, and in particular to a method for separating particulate organic matter into fresh POM and old POM based on CT images. Background Art
[0002] In traditional soil science research, soil is usually regarded as a "black box". Although the morphological characteristics of POM can be observed through certain methods, it is difficult to obtain the spatial distribution information of POM in different forms. Therefore, directly and accurately quantifying the amount of particulate organic matter (POM) and its spatial distribution information in the soil is of great significance to better understand the physical protection mechanism of soil organic carbon. The applicant's previous patent (application number ZL202311203105.2) provides a method for in-situ quantification of soil particulate organic matter using CT technology combined with machine learning, which provides a powerful tool for accurately quantifying the spatial distribution of POM in soil.
[0003] However, POM comes from a variety of sources, including plant roots, stems, and leaf residues, so its shape characteristics show great differences. Although X-ray CT technology can separate larger organic matter (fresh POM) such as roots, stems, and leaf residues, it is difficult to accurately identify old POM with a higher degree of decomposition. In order to identify the differences between POMs with different degrees of decomposition, some researchers have proposed osmium-stained POM, which identifies the position of osmium in the image and then observes the spatial distribution of POMs with different degrees of decomposition. However, the concentration and signal intensity of osmium staining, as well as the sample size, have a great impact on the test results; secondly, osmium vapor is highly toxic, which limits the possibility of further studying POM. Therefore, there is an urgent need for a method to accurately identify particulate organic matter with different degrees of decomposition.
[0004] Based on the morphological characteristics of POM, spherical POM comes from the decomposition of plant stems and leaves, plate-like POM usually comes from leaf residues, and columnar POM usually comes from straw and root residues. Fresh residues (roots, straw, and leaf residues) become hot spots for microbial decomposition due to their high organic matter and nutrient content, and a large amount of nutrients are released during the decomposition of fresh residues. In contrast, old POM is a product of residues such as straw that has undergone long-term microbial decomposition and mineralization. It has a low organic matter content but is tightly bound to the soil matrix, so it is relatively stable in the soil. Although CT technology currently has potential in distinguishing the morphological characteristics of POM in soil, it still faces the challenge of how to accurately distinguish fresh residues from old POM. Solving this problem is crucial to understanding the source of POM under straw return conditions and its impact on soil organic matter circulation. Therefore, developing a method to directly and accurately distinguish fresh POM from old POM from CT images can not only promote a deeper understanding of the spatial distribution and source of POM, but also help optimize straw return measures. Summary of the invention
[0005] The purpose of the present invention is to classify particulate organic matter into fresh POM and old POM according to morphological characteristics, so as to provide an important means for in-depth study of the decomposition, transformation and spatial morphological structure characteristics of particulate organic matter.
[0006] To achieve the above object, the present invention provides a method for classifying particulate organic matter into fresh POM and old POM based on CT images, comprising the following steps:
[0007] Step S1: Scan the sample CT and perform image preprocessing;
[0008] Step S2: Identify and segment particulate organic matter based on machine learning;
[0009] Step S3: calculating the morphological characteristic parameters of the particulate organic matter;
[0010] Step S4: Select a data training set;
[0011] Step S5: using the random forest model to classify particulate organic matter into fresh residues and old POM;
[0012] Step S6: Calculate the spatial distribution characteristics of fresh residues and old POM;
[0013] The specific contents of steps S1 to S3 are the method contents of the applicant's previous patent, with patent number ZL202311203105.2. In this patent method, the soil particulate organic matter can be quantified in situ, but the method cannot distinguish which of the particulate organic matter is fresh residue and old POM. This application continues the research based on the previous research results, and further screens the screened particulate organic matter to identify fresh POM and old POM.
[0014] Furthermore, in step S1, performing CT scanning on the sample and performing image preprocessing includes the following steps:
[0015] Step S11: Take a soil aggregate sample, scan it with a CT scanner, reconstruct the scanned image with Datos|x2 Rec software, and convert it into a tiff format that can be analyzed. The image resolution is 8 μm.
[0016] Step S12: importing the image reconstructed in step S11 into Image J / Fiji software;
[0017] Step S13: Use Adjust-Brightness / Contrast to adjust the brightness and contrast of the image;
[0018] Step S14: using a non-mean filter to perform noise reduction;
[0019] Step S15: using an unsharp mask filter to reduce local volume effects;
[0020] Step S16: Import the image processed by S15 into VG Studio MAX 2022 software, use the adaptive rectangle tool to crop the aggregate boundary, and obtain the aggregate image with the boundary interference removed; identify and judge the soil matrix, pores and particulate organic matter in the aggregate image by human eyes, and use this as the basis for whether the machine learning automatic identification and segmentation in the subsequent step S2 is misclassified.
[0021] Furthermore, in step S2, identifying and segmenting particulate organic matter based on machine learning includes the following steps:
[0022] Step S21: Open the Pixel Classification process of the ilastik software, select Add a single 3D / 4D Volume from Sequence, and import all images;
[0023] Step S22: Select pixel feature values: grayscale value (Intensity), edge information (Edge) and texture feature (Texture), and the Gaussian smoothing standard deviations are δ=0.3, 0.7, and 1.0 respectively;
[0024] Step S23: First, three labels are added, namely soil matrix, particulate organic matter and pores. Under each label, the corresponding pixel category is annotated using a brush. The system performs training classification in an iterative manner. The training process lasts about 5 seconds. After the training is completed, the soil matrix, particulate organic matter and pores are displayed in the image in different colors.
[0025] Step S24: If any misclassification occurs during the training process, the annotation is corrected using a brush or eraser function until the classification result matches the recognition result in step S16;
[0026] Step S25: After correcting the misclassification and completing the training, the result is exported to Tiff Sequence format, and the data type is converted to 8-bit integer (Integer 8bit).
[0027] ilastik is an open source interactive tool for bioinformatics image analysis based on supervised machine learning. Its pixel classification process can annotate user-defined labels to each pixel point, perform random forest classification based on user annotations, and thus complete the segmentation of different types of pixels.
[0028] Step S26: Open the segmented image in Fiji software, and use Color-Split Channels to split the RGB channels of the image into soil matrix, pores, and particulate organic matter, respectively, and keep the image of particulate organic matter for the next step of analysis;
[0029] Step S27: filtering using a median filter;
[0030] Step S28: Invert the grayscale value of the filtered image, and use the Sauvola algorithm in the automatic local threshold method to binarize the image. The parameter settings use the default values. The black part in the binarized image is particulate organic matter, and its volume is larger than the voxel size corresponding to the sample at the CT scan image resolution.
[0031] Step S29: Use the Connected Components Labelling function of the MorphoLibJ plug-in to add labels to the black pixels representing the particulate organic matter, and store the image data in floating point type; use the Label Size Filtering function to screen out the particulate organic matter with a volume larger than 50×50×50μm3; finally, convert the image into a binary image, that is, screen out the particulate organic matter with a volume larger than 50×50×50μm3.
[0032] Furthermore, in step S3, calculating the morphological characteristic parameters of the particulate organic matter includes the following steps:
[0033] Step S31: importing the image of particulate organic matter with a volume larger than 50×50×50 μm3 into Fiji software;
[0034] Step S32: using the Analyze Regions 3D function of the MorphoLibJ plug-in to calculate the morphological characteristics of particulate organic matter, including volume, surface area, and equivalent ellipse radius, and exporting them to csv format;
[0035] Step S33: Open the csv format file in step S32 in Excel, and further calculate the morphological characteristics of the particulate organic matter, including agglomeration B, compactness C, sphericity S and plate-likeness P. The calculation formula is as follows:
[0036]
[0037]
[0038] S=36πV 2 / A 3
[0039] P=|R 2 | / |R 1 |
[0040] In the formula, each particle of organic matter is replaced by an equivalent ellipsoid, which is an imaginary ellipsoid with the same volume as the particle of organic matter. The greater the agglomeration, the less elongated the object is; the greater the compactness, the more compact the particle of organic matter is and the fewer internal pores; the greater the sphericity, the more the particle of organic matter is similar to a sphere; the greater the plate-likeness, the more the particle of organic matter resembles old POM or leaves.
[0041] Step S34: retain the following morphological characteristics of the particulate organic matter: volume, agglomeration, compactness, sphericity and plate-likeness, and save them as an Excel file.
[0042] Furthermore, in step S4, selecting a data training set based on the morphological characteristics of particulate organic matter in the image includes the following steps:
[0043] Step S41: After adding labels to the particulate organic matter in step S32, use the LabelEdition function of the MorphoLibJ plug-in to manually select some fresh residues and old POM, and calculate the morphological characteristics of these fresh residues and old POM, which are used as data training sets and test sets, that is, pre-assigned samples. The data training set is divided into a training set and a test set of the model in a ratio of 7:3; some data of the manually selected fresh residues and old POM and the calculation results of their morphological characteristics are shown in Table 1.
[0044] Table 1 Examples of fresh residues and old POM
[0045]
[0046]
[0047] Here, some fresh residues and old POM are manually screened out based on the morphology in the image, and then the morphological characteristics of these selected fresh residues and old POM are recalculated. Although the morphological characteristics of all particulate organic matter have been calculated before the fresh residues and old POM are manually selected in step S34, it takes a lot of time to match the selected fresh residues and old POM with the previous calculation results one by one. It is faster to recalculate these fresh residues and old POM. The calculation results in Table 2 are recalculated.
[0048] The data training set and test set generally require more than 300-600 samples.
[0049] Step S42: The morphological characteristics of the particulate organic matter determined in step S34 are divided into data training set samples (based on step S41) and unclassified data (prediction samples) and stored in the same excel file.
[0050] Further, in step S5, using the random forest model to classify the particulate organic matter into fresh residues and old POM includes the following steps:
[0051] Step S51: Use Python language to write the random forest classification prediction model code, which mainly includes the following contents: read the excel file containing the morphological characteristics of particulate organic matter, extract the training set data, extract the morphological characteristics and labels of each particulate organic matter, divide the pre-assigned samples into data training set and data test set, create a random forest model to predict the unclassified data, export the prediction results, calculate the model accuracy and OOB (Out of Bag) accuracy. The prediction results are exported as an excel file with a new column added on the basis of the original file.
[0052] The morphological characteristics of particulate organic matter used for classification are: volume, agglomeration, compactness, sphericity and platyness; the model accuracy and OOB (Out of bag) accuracy are generally above 90%, indicating that the model prediction effect is better;
[0053] When building a random forest model in Python, you need to import the RandomForestClassifier class and the train_test_split function for data segmentation in the scikit-learn library; first, use the train_test_split function to divide the dataset into a training set and a test set; then, use the training set to train the RandomForestClassifier to learn the parameters of the model; finally, use the test set to evaluate the performance of the model and adjust the parameters of the model as needed to optimize the performance.
[0054] The RandomForestClassifier class is a class that implements the random forest classification algorithm. It uses multiple decision trees to train samples and predict categories. Through ensemble learning, random forests can reduce the risk of overfitting that may occur in a single decision tree and improve the generalization ability of the model. When building a random forest, RandomForestClassifier automatically performs random sampling of data and random selection of features to build multiple different decision trees. These decision trees are trained independently and the final classification result is obtained by voting during prediction.
[0055] The train_test_split function is a practical tool for dividing data sets in scikit-learn. It divides the original data set into two parts, a training set and a test set, so that the training set can be used to learn model parameters during model training, and the test set is used to evaluate the performance of the model. By setting a suitable division ratio (such as the test_size parameter, the ratio of the training set to the test set is 7:3 in the present invention), it can be ensured that the model has enough data to learn during training, and also has enough data to evaluate the generalization ability of the model. The train_test_split function also allows the random seed to be set by the random_state parameter to ensure that the result of each division is reproducible.
[0056] Step S52: Open the classified file, mark all POMs classified as fresh residues as 1, mark all POMs classified as old POMs as 0, change the title of the column to Classification, and save it as a csv file.
[0057] Fiji software cannot open excel files and needs to be saved as csv files.
[0058] Step S53: Open the csv file in step S52 in Fiji software, and after adding labels to the particulate organic matter in step S32, open the Assign Measure to Label function in the MorphoLibj plug-in, select the opened csv file in the Table Name option, select Classification for Column, set Min Value and Max Value to 0 and 0.5 respectively, and then click OK to screen out the old POM and name it as old POM;
[0059] Min Value and Max Value represent the minimum value and maximum value in the Classification column data respectively. In step S52, the fresh residue and the old POM are marked as 1 and 0 respectively, so the Min Value and MaxValue values can be set by yourself to separate the fresh POM or the old POM.
[0060] Step S54: Open Process-Image Calculator, select the total POM in Image1, select Subtract in Operation, select the old POM in Image2, click OK to get the fresh POM, and rename it to fresh POM;
[0061] Step S55: Open Process-Binary-Make binary, binarize the fresh residue and the old POM image respectively and save them.
[0062] Further, in step S6, the classified fresh POM and old POM are respectively imported into VG Studio MAX2022 for 3D visualization. The morphological characteristic parameters of the fresh POM and the old POM are obtained in step S51, including volume, surface area, agglomeration, compactness, sphericity and plate-likeness.
[0063] Compared with the prior art, the present invention has the following beneficial effects:
[0064] The present invention combines CT technology with a random forest model. Through CT scanning images, particulate organic matter identification and segmentation, and then manually selecting training set samples, the random forest model is trained through the training set samples. The trained model predicts that the particulate organic matter in the sample is classified into fresh POM and old POM, and their spatial distribution information is obtained respectively.
[0065] Starting from the source and decomposition and transformation process of particulate organic matter, the present invention divides the particulate organic matter in soil samples into fresh POM and old POM according to the form of the particulate organic matter. Fresh POM and old POM can respectively represent different sources of particulate organic matter, which can lay a foundation for in-depth research on particulate organic matter in soil science. BRIEF DESCRIPTION OF THE DRAWINGS
[0066] Figure 1 The soil sample CT scan image is reconstructed and converted into an 8-bit grayscale image.
[0067] Figure 2 It is the grayscale image of the soil sample after preprocessing.
[0068] Figure 3 This is the effect of classifying the image of the soil sample, where red represents the soil matrix, green represents the pores, and blue represents the particulate organic matter.
[0069] Figure 4 The soil sample is screened to find the volume larger than 50×50×50μm 3 Image of particulate organic matter.
[0070] Figure 5 These are manually selected examples of fresh stumps and old POMs.
[0071] Figure 6 is the spatial distribution of fresh debris after classification.
[0072] Figure 7 It is the spatial distribution of old POM after classification.
[0073] Figure 8 is a schematic diagram of an equivalent ellipse. DETAILED DESCRIPTION
[0074] In order to make the technical solutions and advantages of the invention more clear, the invention is further described below in conjunction with specific examples and drawings.
[0075] Soil samples of 0-40 cm soil layer were randomly collected from the tillage and fertilization experimental base of Longkang Farm in Huaiyuan County, Anhui Province. The samples were naturally air-dried, and 9 aggregates with a diameter of 6-10 mm were selected for CT scanning. The method for classifying particulate organic matter into fresh POM and old POM based on CT images in this embodiment includes the following steps:
[0076] Step S1: Scan the 9 aggregate samples separately using micro-CT (Phoenix Nanotom, Germany) with a scanning voltage of 100 kV, a current of 90 μA, and an exposure time of 1.25 s. Each sample is rotated 360° at a constant speed on the sample stage. The resolution of the CT scan is related to the diameter of the sample. Although the diameter of the 9 aggregates in this embodiment is 6-10 mm, due to the irregular shape of the aggregates and the placement and other reasons, the CT scanning range will be expanded to ensure that all parts of the aggregates are scanned, so the image resolution of the CT scan will be improved. In this embodiment, the image resolution of the CT scan is 8 μm. Image reconstruction is performed using Datos|x2Rec software, and 1208 to 2135 8-bit grayscale images are obtained for each sample, which are stored in Tiff format, such as Figure 1 shown.
[0077] The 1420 grayscale images of the first sample were imported into the open source software Image J / Fiji. First, the brightness and contrast of the image were adjusted by Brightness / Contrast. The noise reduction was performed by the non-local mean filter (Non-Local Means Denoising) in the Biomedgroup plug-in. The parameters were set to noise variance 15 (Sigma=15) and smoothing factor 1 (δ=1). Then, edge enhancement was performed, and an unsharp mask filter (UnsharpMask Filter) with a radius equal to 1 was applied to reduce the local volume effect caused by image blur, and the processed image was saved. The image was then imported into the VG Studio MAX 2022 software. Since the aggregates were irregular in shape, the cropped ROI could only calculate the particulate organic matter in part of the space, and the particulate organic matter content of the entire aggregate could not be obtained by volume conversion. Therefore, the entire aggregate was analyzed here. The adaptive rectangle tool was used to crop the aggregate boundary, and the equal value was set to 128 and the depth was set to 2. At this time, the agglomerate is selected as ROI, the ROI is inverted and filled with white pixels, and the agglomerate image with boundary interference removed is obtained after export, as shown in Figure 2 shown.
[0078] exist Figure 2 On this basis, the soil matrix, pores and particulate organic matter are identified and judged by human eyes, which serves as the basis for whether the machine learning automatic identification and segmentation in the subsequent step S3 is misclassified.
[0079] Step S2: The segmentation of particulate organic matter is completed in the Pixel Classification process of the ilastik software. In Input Data, select Add a single 3D / 4D Volume from Sequence, import all images, and select pixel features including grayscale value, edge information, and texture features. The Gaussian smoothing standard deviations are δ = 0.3, 0.7, and 1.0, respectively. The original image is classified into soil matrix, pores, and particulate organic matter, and three labels of soil matrix, pores, and particulate organic matter are added, such as Figure 3 , each label corresponds to a pixel category, and then a brush is used to annotate each pixel category separately and perform classification training (Live Update).
[0080] Misclassification refers to the deviation of classification of some pixels after classification training is completed, such as identifying soil matrix as POM, or POM as soil matrix, which all rely on human eye recognition, e.g. Figure 2 The gray value of particulate organic matter is between pores and soil matrix. Some particulate organic matter is the product of decomposition or semi-decomposition of plant straw and leaves, and has a unique texture structure, which can be easily separated from the soil matrix or pores. If the human eye cannot identify the particulate organic matter or it cannot be identified due to poor image quality, then machine learning cannot perform segmentation, because the premise of machine learning is that the instructions are given manually, and the system classifies all pixels according to the instructions. If an incorrect classification occurs, the pixels are re-marked using the brush or eraser function until the training result is accurate. The data type of the exported image is Integer 8bit, and the image format is Tiff Sequence.
[0081] The image segmented in ilastik was imported into Fiji software, and Color-Split Channels was used to split the RGB channels of the image into soil matrix, pores, and particulate organic matter. The image sequence of particulate organic matter was retained, filtered using a median filter with a radius of 2 pixels, and the image grayscale value was inverted (Invert). The image was binarized using the Sauvola algorithm in the Auto Local Threshold method (Auto Local Threshold). The parameter settings used the default values. The black part of the image after binarization was larger than 1 voxel (8×8×8μm 3 ) of particulate organic matter.
[0082] To screen volumes larger than 50×50×50μm 3 The particle organic matter of the image is labeled using the Connected Components Labelling function of the MorphoLibJ plug-in. Then, the particle organic matter with a volume larger than 244 voxels is screened out using Label Size Filtering and converted into a binary image, as shown in the figure. Figure 4 .
[0083] Step S3: Import the particle organic matter image into Fiji software, use the Connected Components Labelling function in the MorphoLibJ plug-in to add labels to each particle organic matter, and name the labeled image as total POM. Then use Analyze Region 3D to calculate the morphological characteristics of each particle organic matter, including volume, surface area, equivalent ellipsoid radius (R 1 , R 2 , R 3 ) and exported to csv format. The equivalent ellipsoids of different shapes and the types of particulate organic matter they represent are shown in Table 2;
[0084] Table 2 Equivalent ellipsoids of different shapes and the types of particulate organic matter they represent
[0085]
[0086] like Figure 8 As shown, R1 is the major radius, which is the radius of the ellipsoid along the longest axis, also known as the major axis or semi-major axis. R2 is the median radius, which is the radius of the ellipsoid along the axis perpendicular to the major radius, also known as the minor axis or semi-minor axis. R3 is the minor radius, which is the radius of the ellipsoid along the shortest axis, also known as the radius, R 1 >R 2 >R 3 .
[0087] Open the csv format file in Excel and further calculate the morphological characteristics of particulate organic matter, including agglomeration B, compactness C, sphericity S and plate-likeness P. The calculation formula is as follows:
[0088]
[0089]
[0090] S=36πV 2 / A 3
[0091] P=|R 2 | / |R 1 |
[0092] In the formula, R 1 , R 2 , R 3 Represent the three radii of the equivalent ellipsoid, where R 1 >R 2 >R 3 . A and V are the actual surface area and actual volume of each POM, respectively. Because a single POM characteristic parameter cannot accurately distinguish between new and old POM, multiple characteristic parameters need to be combined for classification. The greater the agglomeration, the less the object is elongated; the greater the compactness, the more compact the particulate organic matter and the fewer internal pores; the greater the sphericity, the more the particulate organic matter is similar to a sphere; the greater the plate-likeness, the more the particulate organic matter is similar to a plate, as shown in Table 2. The greater the plate-likeness, the more the particulate organic matter resembles old POM or leaves. Due to repeated crushing and translocation, the old POM has high compactness, less internal pore space, and is more circular or spherical in shape; fresh residues are usually continuous, elongated objects (straw, roots) or have a large surface area (leaves).
[0093] There are 9717 particles of organic matter in this sample, that is, 9717 samples need to be classified. The following morphological characteristics of the particle organic matter are retained: volume, agglomeration, compactness, sphericity and plate-likeness, and saved as an Excel file.
[0094] Step S4: For the total POM images in step S3, use the Label Edition function of the MorphoLibJ plug-in to manually select some images, such as Figure 5 , Analyze Region 3D was used to calculate the morphological characteristics (volume, mass B, compactness C, sphericity S, and plate-likeness P) of the training set samples, which were used as the data training set, i.e., the pre-assigned samples. A total of 914 samples were selected as the data training set, of which 326 samples were fresh residues and 588 were old POMs. Among the 9717 samples, 914 were the data training set and 8803 were the prediction set data, i.e., the data to be classified. Their morphological characteristics were all saved in an excel file.
[0095] Fresh residues mainly include undecomposed or semi-decomposed plant residues (straw, roots, leaves, etc.), which are large in size and complex in shape, such as tubular roots and plate-like leaves. After further decomposition, fresh residues become smaller, more compact, and spherical old POMs, which are tightly combined with the soil matrix or tiny pores and are more stably preserved in the soil.
[0096] Step S5: using the random forest model to classify particulate organic matter into fresh residues and old POM;
[0097] Step S51: Use Python language to write the random forest classification prediction model code, which mainly includes the following parts: (1) import the pandas library, import the train_test_split, RandomForestClassifier and accuracy_score functions in the scikit-learn library; (2) read the Excel file containing the morphological characteristics of particulate organic matter; (3) data No. 1-914 is the training set (classified), and data No. 915-9717 is the prediction set (unclassified); (4) extract each The morphological characteristics and labels of particulate organic matter. The morphological characteristics of particulate organic matter used for classification are: volume, agglomeration, compactness, sphericity and plate-likeness; (5) The training set data is divided into a training set and a test set with a ratio of 7:3, that is, 640 data are for the training set and 274 data are for the test set; (6) A random forest model with 100 decision trees is created, the model is trained using the training set data, and the prediction accuracy of the model is tested on the test set; (7) The prediction set data is predicted; the prediction results are exported as an excel file by adding a new column on the basis of the original file (i.e., the excel file in step S4).
[0098] The pandas library provides a large number of data structures and data analysis tools. In this invention, the functions of the pandas library are used as follows:
[0099] Use the pandas read_excel function to read and store data in Excel files;
[0100] Use the indexing function of pandas to extract the data of the training set and the prediction set;
[0101] Use pandas column selection function to extract feature columns from training set and prediction set data;
[0102] Use pandas column selection functionality to extract the label column;
[0103] Use the pandas function to add a new column Predicted_Class to the original file, including the predicted results;
[0104] Use the to_excel function of the pandas library to write the prediction results to a new Excel file.
[0105] Accuracy and OOB are both evaluations of the performance of the random forest model, providing the quality of the prediction results. OOB (Out-of-Bag) accuracy is a performance evaluation indicator unique to the random forest model. During the construction of the random forest model, each sample has the opportunity to be selected as a training sample or an out-of-bag sample. OOB accuracy is calculated based on the prediction results of out-of-bag samples that are not used to train a specific tree. Since OOB samples are unknown during the model training process, OOB accuracy provides an estimate of the generalization performance of the model on unseen data. It does not require additional validation sets or test sets and can be calculated directly during the model training process. The higher the OOB accuracy, the better the classification performance of the random forest model on out-of-bag samples, which also implies that the model may have a strong generalization ability on the overall data set. Accuracy is a basic indicator in the performance evaluation of machine learning models. It measures the proportion of samples correctly classified by the model. In classification tasks, accuracy is obtained by comparing the model prediction results with the true labels and then calculating the percentage of correctly classified samples to the total number of samples. The higher the accuracy, the better the performance of the model on the classification task.
[0106] Because it is impossible to manually classify all POMs into fresh residues and old POMs, the workload is huge and cannot be achieved. It is also impossible to directly compare the predicted results with the actual results. We can only manually select a part of them and test them on the selected part. After the model is trained, we can use the model to predict the remaining large number of POMs that cannot be manually classified.
[0107] In this embodiment, the accuracy of the model tested on the test set is as high as 97.43%, and the OOB accuracy is 97.22%. The accuracy and OOB accuracy are generally above 90%, indicating that the model prediction effect is better.
[0108] Step S52: Open the classified file, mark all POMs classified as fresh residues as 1, mark all POMs classified as old POMs as 0, change the title of the classification result column to Classification, and save it as a csv file. Fiji software cannot open excel files, so they need to be saved as csv files.
[0109] Step S53: Open the classified csv file in Fiji software, and for the total POM image in step S3, open the Assign Measure to Label function in the MorphoLibj plug-in, select the opened csv file in the Table Name option, select Classification for Column, set Min Value and Max Value to 0 and 0.5 respectively, and then click OK to screen out the old POM and name it as old POM;
[0110] Step S54: Then open Process-Image Calculator, select the total POM in Image1, select Subtract in Operation, select the old POM in Image2, click OK to get the fresh POM, and rename it to fresh POM;
[0111] Step S55: Open Process-Binary-Make binary, binarize the fresh POM and old POM images and save them. The classification results of fresh POM and old POM are shown in Table 3:
[0112] Table 3 Classification results
[0113] Fresh POM Old POM quantity 4303 5414 <![CDATA[Total volume (mm 3 )]]> 14.19 1.36
[0114] Fresh residues account for the vast majority of particulate organic matter, indicating that returning straw to the fields can quickly increase the content of particulate organic matter in farmland soil.
[0115] The classified fresh residues and old POM images were imported into VG Studio MAX 2022 for 3D visualization, as shown in Figure 6 , Figure 7 The characteristic parameters of the fresh POM and the old POM are obtained in step S4, including volume, surface area, agglomeration, compactness, sphericity, and plate-likeness.
[0116] The present invention divides particulate organic matter into fresh POM and old POM, which provides an important basis for analyzing the contribution of plant residues to the source of particulate organic matter and the physical protection mechanism of particulate organic matter. ilastik, Fiji and Python used in the present invention are all open source software, and as long as you are slightly familiar with the software usage method and image processing flow, it is easy to promote and apply.
Claims
1. A method for classifying particulate organic matter into fresh POM and old POM based on CT images, characterized in that: The following steps are involved: Step S1: perform CT scanning on the sample and perform image preprocessing; Step S2: Identify and segment particulate organic matter based on machine learning; Step S3: Calculate the morphological characteristic parameters of the particulate organic matter, including: volume, agglomeration, compactness, sphericity and plate-likeness; Step S4: Selecting a data training set, specifically including: Step S41: manually select part of the fresh POM and old POM from the particulate organic matter screened in step S2, and calculate the morphological characteristics of these fresh POM and old POM, which are used as data training sets. The data training sets are divided into a training set and a test set of the model in a ratio of 7:3; the remaining unselected particulate organic matter is used as a prediction sample; Step S42: storing the morphological features of the data training set and the prediction sample in the same excel file; Step S5: Using the random forest model to classify particulate organic matter into fresh POM and old POM; specifically including: Step S51: Use Python language to build a random forest model. The model first extracts training set data for training, learns the model parameters, then uses the test set data to evaluate the performance of the model, and adjusts the model parameters to optimize the performance; finally, the prediction sample in step S41 is predicted, and the prediction results are recorded in a new column in the excel file of step S42; Step S52: Open the excel file of step S51, mark all the particulate organic matter classified as fresh POM as 1, mark all the particulate organic matter classified as old POM as 0, change the title of the column to Classification, and save it as a csv file; Step S53: Open the csv file in step S52 in Fiji software, open the AssignMeasure to Label function in the MorphoLibj plug-in, select the opened csv file in the Table Name option, select Classification as Column, set Min Value and Max Value to 0 and 0.5 respectively, and then click OK to filter out the old POM and name it as Old POM; Step S54: Open Process-Image Calculator, select the total POM in Image1, select Subtract in Operation, select the old POM in Image2, click OK to get the fresh POM, and rename it to fresh POM; Step S55: Open Process-Binary-Make binary, binarize the fresh POM and old POM images respectively and save them; Step S6: Calculate the spatial distribution characteristics of fresh POM and old POM.
2. The method for classifying particulate organic matter into fresh POM and old POM based on CT images according to claim 1, characterized in that: In step S51, the accuracy and OOB accuracy are calculated using the test set data.
3. The method for classifying particulate organic matter into fresh POM and old POM based on CT images according to claim 1, characterized in that: In step S6, the classified fresh POM and old POM are respectively imported into VG Studio MAX2022 for three-dimensional visualization.
4. The method for classifying particulate organic matter into fresh POM and old POM based on CT images according to claim 1, characterized in that: In step S1, performing CT scanning on the sample and image preprocessing includes the following steps: Step S11: Take a sample, scan it with a CT scanner, reconstruct the scanned image with Datos | x2 Rec software, and convert it into a tiff format that can be analyzed; Step S12: importing the image reconstructed in step S11 into Image J / Fiji software; Step S13: Use Adjust-Brightness / Contrast to adjust the brightness and contrast of the image; Step S14: using a non-mean filter to perform noise reduction; Step S15: using an unsharp mask filter to reduce local volume effects; Step S16: Import the image processed by S15 into VG Studio MAX 2022 software, use the adaptive rectangle tool to crop the aggregate boundary, and obtain the aggregate image with the boundary interference removed; identify and judge the soil matrix, pores and particulate organic matter in the aggregate image by human eyes, and use this as the basis for whether the machine learning automatic identification and segmentation in the subsequent step S2 is misclassified.
5. The method for classifying particulate organic matter into fresh POM and old POM based on CT images according to claim 4, characterized in that: In step S2, identifying and segmenting particulate organic matter based on machine learning includes the following steps: Step S21: Open the pixel classification process of the ilastik software, select Add Image Sequence, and import all images; Step S22: Select pixel feature values: gray value, edge information and texture features, and Gaussian smoothing standard deviation are δ=0.3, 0.7, and 1.0 respectively; Step S23: Add three labels, namely soil matrix, particulate organic matter and pores. Use a brush to annotate the corresponding pixel category under each label. The system performs training classification in an iterative manner. The training process lasts about 5 seconds. After the training is completed, the soil matrix, particulate organic matter and pores are displayed in the image in different colors; Step S24: If any misclassification occurs during the training process, the annotation is corrected using a brush or eraser function until the classification result matches the recognition result in step S1; Step S25: After correcting the misclassification and completing the training, the result is exported to Tiff Sequence format, and the data type is converted to 8-bit integer; Step S26: Open the segmented image in Fiji software, and use Color-Split Channels to split the RGB channels of the image into soil matrix, pores, and particulate organic matter, respectively, and keep the image of particulate organic matter for the next step of analysis; Step S27: filtering using a median filter; Step S28: inverting the grayscale value of the filtered image, and binarizing the image using the Sauverra algorithm in the automatic local threshold method. The parameter settings use the default values. The black part of the binarized image is particulate organic matter, and its volume is larger than the voxel size corresponding to the sample at the CT scan image resolution; Step S29: Use the Connected Components Labelling function of the MorphoLibJ plug-in to add labels to the black pixels representing the organic matter of the particles, and store the image data in floating point type; use the Label Size Filtering function to filter the pixels with a volume larger than 50×50×50 μm. 3 Finally, the image is converted into a binary image, that is, the particles with a volume larger than 50×50×50 μm are screened out. 3 of particulate organic matter.
6. The method for classifying particulate organic matter into fresh POM and old POM based on CT images according to claim 5, characterized in that: In step S3, calculating the morphological characteristic parameters of the particulate organic matter includes the following steps: Step S31: Refine the volume larger than 50×50×50 μm 3 The particle organic matter images were imported into Fiji software; Step S32: using the Analyze Regions 3D function of the MorphoLibJ plug-in to calculate the morphological characteristics of particulate organic matter, including volume, surface area, and equivalent ellipse radius, and exporting them to csv format; Step S33: Open the csv file in step S32 in Excel to further calculate the morphological characteristics of particulate organic matter, including agglomeration B , tightness C , Sphericity S and plate-like P , the calculation formula is as follows: In the formula, each particle of organic matter is replaced by an equivalent ellipsoid, which is an imaginary ellipsoid with the same volume as the particle of organic matter; R 1 , R 2 , R 3 are the three radii of the equivalent ellipsoid, A represents the surface area of each POM, V Indicates the volume of each POM; Step S34: retain the following morphological characteristics of the particulate organic matter: volume, agglomeration, compactness, sphericity and plate-likeness, and save them as an Excel file.
