Soil porosity in-situ detection method based on multi-source data fusion and machine learning

Abstract

The application discloses a kind of based on multi-source data fusion and machine learning's soil porosity in-situ detection method, comprising: soil porosity in-situ detection machine learning model is deployed in the industrial computer of soil porosity in-situ detection device, and soil sample to be measured is collected;Using time domain reflection sensor probe insertion obtains the soil volume water content, soil temperature and volume conductivity of nearby soil;Data acquisition module acquires the layered image of internal horizontal section of soil sample to be measured and the change curve of soil resistance;Data processing module processes data, respectively obtains image feature, the average value of volume conductivity and soil resistance at standard temperature 25 DEG C;Soil volume water content and obtained data are used as the characteristic data of soil sample to be measured, and the characteristic data is substituted into the trained soil porosity in-situ detection machine learning model, and the soil porosity of soil sample to be measured is calculated.The method can quickly detect soil porosity, and detection efficiency is fast, cost is low, and detection precision is high.

Description

Technical Field

[0001] This invention relates to the field of soil testing technology, specifically to an in-situ method for detecting soil porosity based on multi-source data fusion and machine learning. Background Technology

[0002] Soil pores are the spaces between soil particles. They are closely related to soil fertility and serve as important channels and storage sites for water and air in the soil. They provide space and nutrients for plant root development and microbial reproduction, and directly affect the migration of water and air phases in the soil and the growth of roots in the soil.

[0003] Soil porosity is the percentage of soil pore volume to soil volume. It is one of the most important physical properties of soil and a fundamental data point for soil research. Currently, the main methods for determining soil porosity include the ring sample method, computed tomography (CT) scan, and nuclear magnetic resonance (NMR) imaging. The ring sample method is a traditional experimental method with simple and readily available tools; however, for soils of different textures, testing a single sample using the ring sample method takes three to four days, which is time-consuming, labor-intensive, and inefficient. CT scan and NMR are both imaging methods; for scientific research requiring large samples and large-scale soil surveys, their use is prohibitively expensive. Therefore, there is an urgent need for a lower-cost and faster technology to accurately and promptly detect soil porosity, guiding scientific decisions regarding cropping systems and fertilization methods, and promoting increased agricultural production and efficiency. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned problems and provide a method for in-situ detection of soil porosity based on multi-source data fusion and machine learning. This method utilizes the speed and convenience of image acquisition and, through a trained machine learning model for in-situ detection of soil porosity, can quickly detect soil porosity with high efficiency, low cost, and high accuracy.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for in-situ detection of soil porosity based on multi-source data fusion and machine learning includes the following steps:

[0007] (1) The trained machine learning model for in-situ detection of soil porosity is deployed in the industrial control computer of the in-situ detection device for soil porosity, wherein the industrial control computer includes a data acquisition module and a data processing module.

[0008] (2) Use a standard ring cutter to collect soil samples at the test point; use a time-domain reflectometry sensor probe to insert into the soil near the collection location of the soil sample to obtain the soil volume water content, soil temperature and volume conductivity.

[0009] (3) The soil sample to be tested in the ring cutter is pushed upward using the soil porosity in-situ detection device. Then, the part of the soil sample to be tested is horizontally sheared so that the original soil column is exposed to the internal horizontal cross-section. The data acquisition module acquires the layered image of the internal horizontal cross-section of the soil sample to be tested and the curve of the change of soil resistance during the shearing process.

[0010] (4) The data processing module processes the layered image of the soil sample to be tested, the volumetric conductivity measured at different temperatures, and the change curve of soil resistance to obtain the image features, the volumetric conductivity at the standard temperature of 25℃, and the average soil resistance. The soil volumetric water content, image features, volumetric conductivity at the standard temperature of 25℃, and the average soil resistance are used as the feature data of the soil sample to be tested. The feature data are substituted into the trained in-situ detection machine learning model of soil porosity to calculate the soil porosity of the soil sample to be tested.

[0011] One preferred embodiment of the present invention, the method for constructing the in-situ detection machine learning model for soil porosity, includes the following steps:

[0012] (S1) Using a standard ring cutter, undisturbed soil columns were collected at sampling points in the field. The time domain reflectance sensor probe was inserted into the soil near the sampling location of the undisturbed soil column to obtain soil volume water content, soil temperature and volume conductivity.

[0013] (S2) The soil porosity in-situ detection device is used to push out the undisturbed soil column in the ring cutter at the same height in sequence. After each undisturbed soil column is pushed out, the soil porosity in-situ detection device will horizontally shear the part of the undisturbed soil column so that the internal horizontal cross-section of the undisturbed soil column is exposed. The soil porosity in-situ detection device collects the layered image of the internal horizontal cross-section of the undisturbed soil column and the curve of the change of soil resistance during the shearing process.

[0014] (S3) Python-OpenCV is used to preprocess the layered images and extract image features. The volumetric conductivity measured at different temperatures is corrected to the volumetric conductivity at the standard temperature of 25℃. After removing outliers from the soil resistance change curve, the average soil resistance during the shearing process is calculated. The image features, soil volumetric water content, volumetric conductivity at the standard temperature of 25℃, and average soil resistance are used as feature values ​​of the machine learning dataset, and the actual soil porosity measured in the laboratory is used as the label value of the dataset.

[0015] (S4) Randomly divide the dataset from step (S3) into a training set and a test set in a ratio of 7:3;

[0016] (S5) Z-score standardization is used to standardize the feature values ​​of the dataset;

[0017] (S6) Select multiple different models, determine the prediction error of each model by calculating the average of the five-fold cross-validation regression coefficient and the root mean square error, and finally determine the most suitable machine learning model based on the model error.

[0018] (S7) Substitute the most suitable machine learning model into the swarm intelligence optimization algorithm, use the swarm intelligence optimization algorithm to perform feature selection on the data after standardization in step (S5), remove irrelevant or redundant features, obtain the optimal feature subset, substitute the optimal feature subset into the machine learning model for retraining, and obtain the in-situ detection machine learning model for soil porosity.

[0019] In a preferred embodiment of the present invention, the industrial control computer further includes a result display module; the data acquisition module also collects GPS information of the detection points; after obtaining GPS information of a limited number of detection points and the soil porosity of the soil sample to be tested, Kriging interpolation is used to fit the variogram and covariance function to simulate the spatial variability of soil porosity and to plot a heat map of the spatial distribution of soil porosity at different depths; the result display module is used to display the detected soil porosity value, the change curve of soil resistance during shearing of the soil sample to be tested, and the heat map of the spatial distribution of soil porosity. The purpose of the above steps is to obtain the detected soil porosity value, the change curve of soil resistance, and the heat map of the spatial distribution of soil porosity more intuitively.

[0020] When used in the field, a single testing point can collect one or more soil samples as needed. By interpolating multiple testing points in the field using the Kriging interpolation method, the soil porosity of the entire field can be predicted, providing a holistic understanding of the porosity of the entire field and enabling prediction from point to surface. With the addition of depth information, prediction from surface to volume can be achieved.

[0021] In a preferred embodiment of the present invention, in step (S1), before collecting the undisturbed soil column, a layer of petroleum jelly is applied to the inner wall of the ring cutter; two undisturbed soil columns are collected adjacent to each sampling point, one of which is sent to the laboratory to test the true value of soil porosity; the other undisturbed soil column is used to perform step (S2). In the above steps, applying petroleum jelly can lubricate the undisturbed soil column, which reduces the friction between the inner wall of the ring cutter and the soil when the ring cutter is pressed into the soil during sampling, and also facilitates the removal of the undisturbed soil column from the ring cutter; furthermore, the obtained true value is used as the label value of the dataset, and the true value of soil porosity and the feature value of the dataset are substituted into each model for training.

[0022] Furthermore, since the vertical distribution of pores in the soil is "loose at the top and solid at the bottom", in step (S1), undisturbed soil columns at two depths, 0-20cm and 20-30cm, are collected at each sampling location to obtain different porosity distributions. This allows the established in-situ soil porosity detection machine learning model to learn the characteristics under various porosities, thereby improving the generalization ability of the in-situ soil porosity detection machine learning model. In the above structure, that is, four undisturbed soil columns are collected at each sampling point.

[0023] Preferably, the in-situ soil porosity detection device includes a light box, a soil sample pushing and cutting mechanism, a control system, and an image acquisition system. The light box provides a sealed lighting environment. The soil sample pushing and cutting mechanism is located inside the light box and includes a frame fixed to the bottom of the light box, a pushing mechanism mounted on the frame, and a cutting mechanism. The pushing mechanism precisely pushes the undisturbed soil column or the soil sample to be tested from the ring cutter to a set height. The cutting mechanism cuts off the pushed-out undisturbed soil column or the soil sample to be tested. A force sensor is installed on the cutting mechanism to acquire the change curve of soil resistance during the shearing process. The control system is connected to the pushing mechanism and the cutting mechanism to control their movement. The image acquisition system includes an industrial camera located at the top center of the light box and an industrial computer located inside the control box. The industrial camera acquires layered images of the horizontal cross-section of the undisturbed soil column or the soil sample to be tested. In steps (S2)-(S3), the control system sets the bulldozing height of the bulldozing mechanism, and the bulldozing mechanism pushes out the undisturbed soil column at the same height each time. Specifically, the control system controls the bulldozing mechanism to push the undisturbed soil column in the ring cutter upwards precisely to the set height. Then, the control system controls the soil cutting mechanism to horizontally shear the pushed-out undisturbed soil column, exposing the horizontal cross-section inside the undisturbed soil column. An industrial camera captures the horizontal cross-section inside the undisturbed soil column to obtain layered images. During the shearing process, the force sensor collects the change curve of soil resistance. The above operation is repeated, and multiple layered images and multiple change curves of soil resistance can be obtained for each undisturbed soil column. Then, the layered images of the undisturbed soil column and multiple change curves of soil resistance are sent to the terminal device. The layered images are preprocessed and image features are extracted using the Python-OpenCV library on the terminal device. After removing outliers from the change curves of soil resistance, the average value of soil resistance during the shearing process is calculated.In steps (3)-(4), the control system sets the bulldozing height of the bulldozing mechanism, and the bulldozing mechanism pushes out the soil sample to be tested at the same height each time. Specifically, the control system controls the bulldozing mechanism to push the soil sample to be tested in the ring cutter upwards at the set height, and then the control system controls the cutting mechanism to horizontally shear the pushed-out soil sample to expose the horizontal cross-section inside the soil sample. The industrial camera collects the horizontal cross-section inside the soil sample to obtain a layered image. During the shearing process, the force sensor collects the change curve of soil resistance. The industrial camera is connected to the industrial computer. At this time, the data acquisition module of the industrial computer will collect the change curve of soil resistance of the soil sample in the force sensor and the data from the industrial camera. The data processing module processes the layered images of the soil sample to be tested, along with the changes in volumetric conductivity and soil resistance. After processing, the soil volumetric water content, image features, volumetric conductivity at a standard temperature of 25℃, and average soil resistance are used as the feature data of the soil sample. These feature data are then substituted into a pre-trained machine learning model for in-situ detection of soil porosity to calculate the soil porosity. By repeating the above steps, multiple layered images can be obtained for each soil sample, and each layered image can yield a soil porosity result. This allows for the study of the specific conditions of soil porosity in the depth direction. Alternatively, the average value of multiple soil porosities can be used as the final result of the soil porosity.

[0024] Preferably, in step (S3), the image features include color features and texture features; wherein, the color features include gray-level histogram features, gray-level features, and HSV color moments; the texture features include local binary mode features, gray-level co-occurrence matrix features, and fractal dimension. In the above structure, by extracting multiple image features, the generalization ability of the model is improved.

[0025] Preferably, in step (S3), the method for extracting grayscale histogram features is as follows:

[0026] (S3.1) Convert the layered image to grayscale;

[0027] (S3.2) Perform Gaussian fitting on the grayscale histogram and extract five function parameters—offset, center, width, area, and height—as the grayscale histogram features of the layered image; the specific formula for Gaussian fitting is as follows:

[0028]

[0029] Where y0 is the offset, x c With ω as the center and A as the width, y c -y0 represents the height.

[0030] Preferably, in step (S3), the correction formula for correcting the volume conductivity measured at different temperatures to the volume conductivity at the standard temperature of 25°C is as follows:

[0031] EC 25 =f t EC t

[0032] Among them, EC 25 Volume conductivity at 25°C, in μS / cm; f t EC is the correction factor. t The volumetric conductivity measured at different temperatures is expressed in μS / cm.

[0033] f t Use the following empirical formula to estimate:

[0034] f t =1-0.020346(T-25)+0.003822(T-25) 2 +0.000555(T-25) 3

[0035] Where T is the temperature at the time of measurement, in °C.

[0036] Preferably, in step (S5), the standardized data follows a normal distribution, as shown in the formula:

[0037]

[0038] Where, x * σ represents the standardized data; x represents the original feature data; μ represents the mean of a certain feature variable; and σ represents the variance of a certain feature variable.

[0039] Preferably, in step (S6), suitable models include linear regression, ridge regression, support vector regression, backpropagation neural network, and gradient boosting regression tree. Standardized feature data and the actual soil porosity values ​​detected in the laboratory are substituted into each machine learning model for training. The prediction error of each model is determined by calculating the average of the five-fold cross-validation regression coefficients and the root mean square error. The results show that the backpropagation neural network significantly outperforms other models. Therefore, the backpropagation neural network is a suitable machine learning model. The backpropagation neural network is then substituted into a swarm intelligence optimization algorithm for feature selection to construct a machine learning model for in-situ detection of soil porosity.

[0040] Furthermore, the training model of the backpropagation neural network consists of five layers: an input layer, a first hidden layer, a second hidden layer, a third hidden layer, and an output layer. The first hidden layer has 100 neurons, the second hidden layer has 80 neurons, the third hidden layer has 50 neurons, and the output layer has only one neuron, outputting the predicted soil porosity value. The backpropagation process uses the Adam algorithm to update and optimize the weights of the backpropagation neural network, continuously reducing the loss function. The learning rate is set to 0.001, the activation function is the ReLU function, and the maximum number of iterations is 2000.

[0041] Compared with the prior art, the present invention has the following advantages:

[0042] 1. The soil porosity in-situ detection method of the present invention utilizes the speed and convenience of image acquisition, and uses the soil volume water content, image features, volume conductivity at a standard temperature of 25℃ and the average soil resistance as feature data of the soil sample to be tested. The feature data is substituted into the trained soil porosity in-situ detection machine learning model, which can quickly detect soil porosity with high detection efficiency, low cost and high detection accuracy.

[0043] 2. The soil porosity in-situ detection method of the present invention uses image features, soil volumetric water content, volumetric conductivity at standard temperature of 25℃ and average soil resistance as the dataset for machine learning, and establishes a machine learning model for soil porosity in-situ detection based on multi-source data fusion and machine learning, thereby improving the detection accuracy.

[0044] 3. The in-situ soil porosity detection method of this invention, due to the "black box" nature of the soil's internal pore structure, utilizes layered images of the internal horizontal cross-section of the undisturbed soil column, soil volumetric water content, volumetric conductivity, and soil resistance curves, and employs machine learning and swarm intelligence optimization algorithms (genetic algorithms) to establish a machine learning model for in-situ soil porosity detection. This model studies the mapping relationship between image features after damage and soil porosity, a method not previously reported domestically or internationally. Furthermore, by acquiring layered images of the internal horizontal cross-section of the undisturbed soil column, the accuracy of the in-situ soil porosity detection machine learning model can be further improved, thereby enhancing detection accuracy.

[0045] 4. The in-situ soil porosity detection method of the present invention adopts a multi-sensor information fusion approach, which integrates and complements the information from the image sensor, force sensor and time domain reflectance sensor in the industrial camera, thereby improving the perception capability of the multi-sensor system of soil pore structure. Attached Figure Description

[0046] Figure 1This is a flowchart of an in-situ soil porosity detection method based on multi-source data fusion and machine learning, as described in this invention.

[0047] Figure 2 The flowchart of constructing an in-situ machine learning model for soil porosity detection in this invention.

[0048] Figure 3 This is a flowchart of the image feature extraction process in this invention.

[0049] Figure 4 The prediction performance of different models in this invention is shown.

[0050] Figures 5-6 This is a comparison chart of the model performance before and after using the genetic algorithm in this invention. Figure 5 The graph shows the model performance before using the genetic algorithm. Figure 6 The graph shows the performance of the model after using the genetic algorithm.

[0051] Figure 7 This is a flowchart of the spatial interpolation process for drawing a spatial distribution map of soil porosity using the Kriging interpolation method in this invention.

[0052] Figure 8 This is a three-dimensional structural diagram of the soil porosity in-situ detection device of the present invention.

[0053] Figures 9-10 These are perspective views of the soil sample pushing and cutting mechanism in this invention from different viewpoints.

[0054] Figure 11 This is a schematic diagram of the internal structure of the bulldozing mechanism in this invention. Detailed Implementation

[0055] To enable those skilled in the art to fully understand the technical solutions of the present invention, the present invention will be further described below in conjunction with embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0056] See Figure 1 This embodiment discloses an in-situ soil porosity detection method based on multi-source data fusion and machine learning, including the following steps:

[0057] (1) The trained machine learning model for in-situ detection of soil porosity is deployed in the industrial control computer of the in-situ detection device for soil porosity, wherein the industrial control computer includes a data acquisition module and a data processing module.

[0058] (2) Use a standard ring cutter 15 to collect soil samples at the test point; use a time domain reflectance sensor probe to insert into the soil near the collection location of the soil sample to obtain the soil volume water content, soil temperature and volume conductivity.

[0059] (3) The soil sample to be tested in the ring cutter is pushed upward using the soil porosity in-situ detection device. Then, the part of the soil sample to be tested is horizontally sheared so that the original soil column is exposed to the internal horizontal cross-section. The data acquisition module acquires the layered image of the internal horizontal cross-section of the soil sample to be tested and the curve of the change of soil resistance during the shearing process.

[0060] (4) The data processing module processes the layered image of the soil sample to be tested, the volumetric conductivity measured at different temperatures, and the change curve of soil resistance to obtain the image features, the volumetric conductivity at the standard temperature of 25℃, and the average soil resistance. The soil volumetric water content, image features, volumetric conductivity at the standard temperature of 25℃, and the average soil resistance are used as the feature data of the soil sample to be tested. The feature data are substituted into the trained in-situ detection machine learning model of soil porosity to calculate the soil porosity of the soil sample to be tested.

[0061] See Figures 2-10 The method for constructing the in-situ detection machine learning model for soil porosity includes the following steps:

[0062] (S1) Using a standard ring cutter 15, undisturbed soil columns were collected at sampling points in the field. A time-domain reflectometry (TDR) sensor probe was inserted into the soil near the sampling location of the undisturbed soil column to obtain soil volume water content, soil temperature and volume electrical conductivity.

[0063] (S2) The soil porosity in-situ detection device is used to push out the undisturbed soil column in the ring cutter 15 at the same height. After each undisturbed soil column is pushed out, the soil porosity in-situ detection device will horizontally shear the part of the undisturbed soil column so that the internal horizontal cross-section of the undisturbed soil column is exposed. The soil porosity in-situ detection device collects the layered image of the internal horizontal cross-section of the undisturbed soil column and the curve of the change of soil resistance during the shearing process.

[0064] (S3) Python-OpenCV was used to preprocess the layered images and extract image features. The volumetric conductivity measured at different temperatures was corrected to the volumetric conductivity (EC) at the standard temperature of 25℃. 25 After removing outliers from the soil resistance variation curve, the average soil resistance (f) during the shearing process is calculated; the image features, soil volumetric water content, and volumetric conductivity (EC) at a standard temperature of 25°C are then considered. 25 The average soil resistance (f) and the soil porosity measured in the laboratory are used as the feature values ​​of the machine learning dataset, and the actual soil porosity values ​​are used as the label values ​​of the dataset.

[0065] (S4) Randomly divide the dataset from step (S3) into a training set and a test set in a ratio of 7:3;

[0066] (S5) Z-score standardization is used to standardize the feature values ​​of the dataset to reduce the impact of different dimensions and orders of magnitude between variables on model parameter learning;

[0067] (S6) Select multiple different models and calculate the regression coefficient R through five-fold cross-validation. 2 The prediction error of each model is determined by the average of the root mean square error (RMSE), and finally the most suitable machine learning model is determined based on the model error.

[0068] (S7) Substitute the most suitable machine learning model into the swarm intelligence optimization algorithm, and use the swarm intelligence optimization algorithm to select features from the standardized feature data in step (S5), remove irrelevant or redundant features, thereby finding the most important information and reducing the data dimensionality, that is, finding the optimal feature subset, and substituting the optimal feature subset into the model to retrain and obtain the in-situ detection machine learning model for soil porosity.

[0069] See Figures 1-10 The industrial control computer also includes a result display module. The soil sample to be tested is collected from the detection points, and the data acquisition module also collects the GPS information (location information) of the detection points. After obtaining the GPS information of a limited number of detection points and the soil porosity of the soil sample, Kringring interpolation is used to fit the variogram and covariance function to simulate the spatial variability of soil porosity and to draw a heat map of the spatial distribution of soil porosity at different depths. The result display module is used to display the detected soil porosity value, the change curve of soil resistance during shearing of the soil sample, and the heat map of the spatial distribution of soil porosity. The purpose of the above steps is to obtain the detected soil porosity value, the change curve of soil resistance, and the heat map of the spatial distribution of soil porosity more intuitively.

[0070] When used in the field, a single testing point can collect one or more soil samples as needed. By interpolating multiple testing points in the field using the Kriging interpolation method, the soil porosity of the entire field can be predicted, providing a holistic understanding of the porosity of the entire field and enabling prediction from point to surface. With the addition of depth information, prediction from surface to volume can be achieved.

[0071] See Figure 2In step (S1), before collecting the undisturbed soil column, a layer of Vaseline is applied to the inner wall of the ring cutter 15. Two undisturbed soil columns are collected adjacent to each sampling point. One undisturbed soil column is sent to the laboratory to test the true value of soil porosity; the other undisturbed soil column is used to perform step (S2). In the above steps, applying Vaseline can lubricate the undisturbed soil column. On the one hand, it can reduce the friction between the inner wall of the ring cutter 15 and the soil when the ring cutter 15 is pressed into the soil during sampling. On the other hand, it makes it easier to push the undisturbed soil column out of the ring cutter 15. In addition, the obtained true value is used as the label value of the dataset, and the true value of soil porosity and the feature value of the dataset are substituted into each model for training.

[0072] See Figure 2 Since the vertical distribution of pores in the soil is "loose at the top and solid at the bottom", in step (1), undisturbed soil columns at two depths of 0-20cm and 20-30cm are collected at each sampling location to obtain different porosity distributions. This allows the established in-situ soil porosity detection machine learning model to learn the characteristics under various porosities and improve the generalization ability of the in-situ soil porosity detection machine learning model. In the above structure, that is, the number of undisturbed soil columns collected at each sampling point is four.

[0073] See Figures 8-11The in-situ soil porosity detection device includes a light box 1, a soil sample pushing and cutting mechanism, a control system, and an image acquisition system 2. The light box 1 provides a sealed lighting environment. The soil sample pushing and cutting mechanism is located inside the light box 1 and includes a frame 3 fixed to the bottom of the light box 1, a pushing mechanism 4 mounted on the frame 3, and a cutting mechanism 5. The pushing mechanism 4 precisely pushes out the undisturbed soil column or the soil sample to be tested from the ring cutter 15 to a set height. The cutting mechanism 5 removes the pushed-out undisturbed soil column or the soil sample to be tested. The soil sample is cut off; the cutting mechanism 5 is equipped with a force sensor 13, which is used to acquire the change curve of soil resistance during the shearing process of the undisturbed soil column or the soil sample to be tested; the control system is connected to the bulldozing mechanism 4 and the cutting mechanism 5, and is used to control the movement of the bulldozing mechanism 4 and the cutting mechanism 5; the image acquisition system 2 includes an industrial camera 2-1 set in the center of the top of the optical box 1 and an industrial control computer set in the control box 20; the industrial camera 2-1 is used to acquire layered images of the horizontal cross-section of the undisturbed soil column or the soil sample to be tested. In steps (S2)-(S3), the control system sets the bulldozing height of the bulldozing mechanism 4, and the bulldozing mechanism 4 pushes out the undisturbed soil column at the same height each time. Specifically, the control system controls the bulldozing mechanism 4 to push the undisturbed soil column in the ring cutter 15 upwards precisely at the set height. Then, the control system controls the soil cutting mechanism 5 to horizontally shear the pushed-out undisturbed soil column, exposing the horizontal cross-section inside the undisturbed soil column. The industrial camera 2-1 collects the horizontal cross-section inside the undisturbed soil column to obtain layered images. During the shearing process of the undisturbed soil column, the force sensor 13 collects the change curve of soil resistance. The above operation is repeated, and multiple layered images and multiple change curves of soil resistance can be obtained for each undisturbed soil column. Then, the layered images of the undisturbed soil column and multiple change curves of soil resistance are sent to the terminal device. The layered images are preprocessed and image features are extracted by the Python-OpenCV library on the terminal device. After removing outliers from the change curves of soil resistance, the average value of soil resistance during the shearing process is calculated.In steps (3)-(4), the control system sets the bulldozing height of the bulldozing mechanism 4, and the bulldozing mechanism 4 pushes out the soil sample to be tested at the same height each time; specifically, the control system controls the bulldozing mechanism 4 to push the soil sample to be tested in the ring cutter 15 upwards precisely at the set height, and then the control system controls the cutting mechanism 5 to horizontally shear the pushed-out soil sample to expose the horizontal cross-section inside the soil sample; the industrial camera 2-1 collects the horizontal cross-section inside the soil sample to obtain a layered image; during the shearing process of the soil sample, the force sensor 13 collects the change curve of soil resistance; the industrial camera 2-1 Connect to the industrial control computer; at this time, the data acquisition module of the industrial control computer will acquire the soil resistance change curve of the soil sample to be tested in the force sensor 13 and the layered image of the soil sample to be tested in the industrial camera 2-1. The data processing module processes the layered image, volume conductivity, and soil resistance change curve of the soil sample to be tested. The specific processing process is as follows: the layered image is preprocessed and image features are extracted, the volume conductivity measured at different temperatures is corrected to the volume conductivity at the standard temperature of 25℃, and the outliers of the soil resistance change curve are removed before calculating the average soil resistance during the shearing process. Using soil volumetric water content, image features, volumetric conductivity at a standard temperature of 25℃, and average soil resistance as characteristic data of the soil sample to be tested, the characteristic data are substituted into a trained in-situ soil porosity detection machine learning model to calculate the soil porosity of the soil sample. Next, the control system controls the cutting mechanism 5 to push out the soil sample and then performs horizontal shearing to expose the internal horizontal cross-section of the soil sample. An industrial camera 2-1 acquires the internal horizontal cross-section of the soil sample to obtain layered images, and a sensor 13 acquires the change curve of soil resistance. The above steps are repeated to obtain the soil porosity of the layered image. Multiple layered images can be obtained for each soil sample, and each layered image can yield one soil porosity result, allowing for the study of the specific soil porosity in the depth direction. Alternatively, the average soil porosity of multiple layered images can be used as the final soil porosity result.

[0074] The terminal device is a computer.

[0075] See Figures 8-11 The bulldozing mechanism 4 includes a piston 4-1 and an electric bulldozing push rod. The electric bulldozing push rod includes a push rod 4-2 and a stepper motor 4-3. The stepper motor 4-3 is mounted on the frame 3. The upper end of the push rod 4-2 is connected to the piston 4-1, and the lower end of the push rod 4-2 is connected to the power end of the stepper motor 4-3. The piston 4-1 slides against the inner wall of the ring cutter 15. When the detection device starts working, the lower surface of the piston 4-1 is flush with the bottom surface of the ring cutter 15. The stroke of the push rod 4-2 can be controlled according to the number of pulses of the stepper motor 4-3. The stepper motor 4-3 is connected to the control system, which controls the movement of the stepper motor 4-3.

[0076] See Figures 8-11 The soil cutting mechanism 5 includes a soil cutting blade 5-1 and a soil cutting electric push rod 5-1 mounted on the frame 3 for driving the soil cutting blade 5-1 to move. By driving the soil cutting blade 5-1 to move via the soil cutting electric push rod 5-1, the soil cutting blade 5-1 can cut off the soil extending from the ring cutter 15.

[0077] See Figures 8-11 The telescopic rod of the soil-cutting electric push rod 5-1 is connected to the soil-cutting blade 5-1 via the force sensor 13. The purpose of the force sensor 13 is to collect data on changes in soil resistance during shearing of the undisturbed soil column or the soil sample to be tested, in order to analyze soil porosity information.

[0078] See Figures 8-11 The frame 3 is provided with a clamping mechanism 6 for clamping the ring cutter 15. Its purpose is to clamp the ring cutter 15 and prevent the ring cutter 15 from being pushed out during the upward pushing of soil.

[0079] See Figures 8-11 The frame 3 is equipped with a collection mechanism 9 for collecting waste soil. In this mechanism, the soil cut by the cutting blade 5-1 will fall into the collection mechanism 9 for collection.

[0080] See Figure 3 In step (S3), the image features include color features and texture features; wherein, the color features include gray-level histogram features, gray-level features, and HSV color moments; the texture features include Local Binary Pattern (LBP) features, Gray-Level Co-occurrence Matrix (GLCM) features, and fractal dimension. The feature extraction process is as follows: Figure 2 As shown, the above structure improves the model's generalization ability by extracting various image features.

[0081] See Figure 3 In step (S3), the method for extracting grayscale histogram features is as follows:

[0082] (3.1) Convert the layered image to grayscale;

[0083] (3.2) Gaussian fitting is performed on the gray-level histogram to extract five function parameters: offset, center, width, area, and height, as gray-level histogram features of the layered image; the specific formula for Gaussian fitting is as follows:

[0084]

[0085] Where y0 is the offset, x c With ω as the center and A as the width, y c-y0 is the height, y is the number of pixels with various grayscale values, and x is the grayscale value.

[0086] Specifically, in step (S3), the correction formula for correcting the volume conductivity measured at different temperatures to the volume conductivity at the standard temperature of 25℃ is as follows:

[0087] EC 25 =f t EC t

[0088] Among them, EC 25 Volume conductivity at 25°C, in μS / cm; f t EC is the correction factor. t The volumetric conductivity measured at different temperatures is expressed in μS / cm.

[0089] f t Use the following empirical formula to estimate:

[0090] f t =1-0.020346(T-25)+0.003822(T-25) 2 +0.000555(T-25) 3

[0091] Where T is the temperature at the time of measurement, in °C.

[0092] Specifically, in step (S5), the standardized data follows a normal distribution, as shown in the formula:

[0093]

[0094] Where, x * σ represents the standardized data; x represents the original feature data, i.e., the feature values ​​of the dataset before standardization; μ represents the mean of a certain feature variable; and σ represents the variance of a certain feature variable.

[0095] See Figure 4In step (S6), suitable models include linear regression (LR), ridge regression (RR), support vector regression (SVR), backpropagation neural network (BPNN), and gradient boosting regression tree (GBDT). Standardized feature data and the actual soil porosity values ​​detected in the laboratory are substituted into each machine learning model for training. The prediction error of each model is determined by calculating the average of the five-fold cross-validation regression coefficients and the root mean square error. The results show that the backpropagation neural network (BPNN) significantly outperforms the other models. Therefore, the backpropagation neural network (BPNN) is the most suitable machine learning model. The backpropagation neural network (BPNN) is then substituted into a swarm intelligence optimization algorithm for feature selection to construct a machine learning model for in-situ detection of soil porosity.

[0096] Furthermore, the backpropagation neural network (BPNN) training model consists of five layers: an input layer, a first hidden layer, a second hidden layer, a third hidden layer, and an output layer. The first hidden layer has 100 neurons, the second hidden layer has 80 neurons, the third hidden layer has 50 neurons, and the output layer has only one neuron, outputting the predicted soil porosity value. The backpropagation process uses the Adam algorithm to update and optimize the weights of the backpropagation neural network, continuously reducing the loss function. The learning rate is set to 0.001, the activation function is the ReLU function, and the maximum number of iterations is 2000.

[0097] Specifically, in step (S7), feature selection plays a crucial role in machine learning. Its main goal is to maximize model performance and minimize the number of features. The aim is to remove irrelevant or redundant features from a given set of features, thereby finding the most important information and reducing the data dimensionality. Finding the optimal subset is a key issue in feature selection.

[0098] Swarm intelligence optimization algorithms are an evolutionary computation technique inspired by the survival and lifestyle of biological groups in nature.

[0099] See Figures 5-6Swarm optimization algorithms are evolutionary computation techniques inspired by the survival and lifestyles of biological groups in nature. This embodiment uses a genetic algorithm (GA), binary bat algorithm (BBA), and particle swarm optimization algorithm (PSO) to extract the feature subset with the highest information content from the full feature space to improve the regression accuracy and reduce the model complexity of the machine learning model for in-situ soil porosity detection. The feature selection results are shown in Table 1. It can be seen that on the dataset of this embodiment, the feature selection ability of the genetic algorithm (GA) is superior to that of the binary bat algorithm (BBA) and particle swarm optimization algorithm (PSO), reducing the number of features from 52 to 25, eliminating redundant features, and improving the model's prediction accuracy. On the same test set, the R-value of the BPNN with all features is... 2 The RMSE values ​​were 0.908 and 3.452%, respectively. The R-value of the BPNN using a feature subset selected by a genetic algorithm was... 2 The RMSE and RMSE were 0.957% and 2.365% respectively. Figures 5-6 As shown.

[0100] Table 1 Comparison of Feature Selection Results for GA, BBA, and PSO

[0101]

[0102] In this embodiment, the specific steps for using a genetic algorithm to select the optimal feature subset are as follows:

[0103] The standardized features are treated as genes on chromosomes and encoded in binary. The total number of "0"s and "1"s is the same as the number of features. A gene "1" indicates that the feature is selected, and a gene "0" indicates that the feature is not selected. The number of gene "1"s represents the number of features to be selected. An initial population is then randomly generated. To ensure genetic diversity, the initial population size is set to 50 in this embodiment. The fitness function is used to calculate the fitness of each individual in the population to evaluate the quality of individuals and serve as the criterion for feature combinations. This embodiment employs a backpropagation neural network (BPNN) with five-fold cross-validation. 2 The average value is used as the fitness function. During the genetic algorithm's execution, the algorithm terminates when a preset stopping condition is met, outputting the optimal feature subset. In this embodiment, the preset stopping condition is reaching the maximum number of generations (100). Otherwise, gene operations continue, including selection, crossover, and mutation. Specific gene operations and parameter settings are as follows:

[0104] (a) Selection Operation: The principle of selection is that individuals with higher fitness have a higher probability of being selected, and therefore a higher probability of being inherited by the next generation, realizing Darwin's principle of survival of the fittest. Tournament selection is used as the selection operator. This strategy involves taking a certain number of individuals N from the parent population and selecting the individual with the highest fitness from these individuals to enter the offspring population. This operation is repeated until the size of the offspring population is the same as that of the parent population. In this embodiment, the number of individuals N selected each time is set to 3.

[0105] (b) Crossover operation: Two parents selected by the crossover probability exchange dissimilar genes to produce new individuals. The resulting new generation of individuals inherits the superior traits of their parents. If the crossover probability is too high, the high-fitness gene string structure will be quickly destroyed, while if it is too low, the search will stop. Therefore, in this embodiment, the crossover probability is set to 0.5.

[0106] (c) Mutation operation: Mutation is the process of changing the value of a gene in a chromosome with a certain probability to produce a new individual. In this embodiment, the mutation probability is set to 0.01.

[0107] In step (S1), the time domain reflectance sensor probe is inserted into the soil near the sampling location of the undisturbed soil column, that is, the time domain reflectance sensor probe is inserted into the soil at a distance of 5-10 cm from the sampling location of the undisturbed soil column. In step (1), the time domain reflectance sensor probe is inserted into the soil near the sampling location of the soil sample to be tested, that is, the time domain reflectance sensor probe is inserted into the soil at a distance of 5-10 cm from the sampling location of the soil sample to be tested.

[0108] The above are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above content. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for in-situ detection of soil porosity based on multi-source data fusion and machine learning, characterized in that, Includes the following steps: (1) The trained in-situ soil porosity detection machine learning model is deployed in the industrial control computer of the in-situ soil porosity detection device. The industrial control computer includes a data acquisition module and a data processing module. (2) Use a standard ring cutter to collect soil samples at the test point; use a time-domain reflectometry sensor probe to insert into the soil near the collection location of the soil sample to obtain the soil volume water content, soil temperature and volume conductivity. (3) The soil sample to be tested in the ring cutter is pushed upward using the soil porosity in-situ detection device. Then, the part of the soil sample to be tested is horizontally sheared so that the original soil column is exposed to the internal horizontal cross-section. The data acquisition module acquires the layered image of the internal horizontal cross-section of the soil sample to be tested and the curve of the change of soil resistance during the shearing process. (4) The data processing module processes the layered image of the soil sample to be tested, the volumetric conductivity measured at different temperatures and the change curve of soil resistance, and obtains the image features, the volumetric conductivity at the standard temperature of 25℃ and the average soil resistance, respectively; the soil volumetric water content, image features, volumetric conductivity at the standard temperature of 25℃ and the average soil resistance are used as the feature data of the soil sample to be tested, and the feature data are substituted into the trained soil porosity in-situ detection machine learning model to calculate the soil porosity of the soil sample to be tested; The method for constructing the in-situ detection machine learning model for soil porosity includes the following steps: (S1) Using a standard ring cutter, undisturbed soil columns were collected at sampling points in the field. The time domain reflectance sensor probe was inserted into the soil near the collection location of the undisturbed soil column to obtain soil volume water content, soil temperature and volume electrical conductivity. (S2) The soil porosity in-situ detection device is used to push out the undisturbed soil column in the ring cutter at the same height in sequence. After each undisturbed soil column is pushed out, the soil porosity in-situ detection device will horizontally shear the part of the undisturbed soil column so that the internal horizontal cross-section of the undisturbed soil column is exposed. The soil porosity in-situ detection device collects the layered image of the internal horizontal cross-section of the undisturbed soil column and the curve of the change of soil resistance during the shearing process. (S3) Python-OpenCV is used to preprocess the layered images and extract image features. The volumetric conductivity measured at different temperatures is corrected to the volumetric conductivity at the standard temperature of 25℃. After removing outliers from the soil resistance change curve, the average soil resistance during the shearing process is calculated. The image features, soil volumetric water content, volumetric conductivity at the standard temperature of 25℃ and the average soil resistance are used as feature values ​​of the machine learning dataset. The true value of soil porosity of the undisturbed soil column measured in the laboratory is used as the label value of the dataset. (S4) Randomly divide the dataset from step (S3) into a training set and a test set in a 7:3 ratio; (S5) Z-score standardization is used to standardize the feature values ​​of the dataset; (S6) Select multiple different models, calculate the average of the five-fold cross-validation regression coefficients and root mean square error to determine the prediction error of each model, and finally determine the most suitable machine learning model based on the model error. (S7) Substitute the most suitable machine learning model into the swarm intelligence optimization algorithm, and use the swarm intelligence optimization algorithm to perform feature selection on the data after standardization in step (S5), remove irrelevant or redundant features, obtain the optimal feature subset, and substitute the optimal feature subset into the machine learning model for retraining to obtain the in-situ detection machine learning model for soil porosity.

2. The method for in-situ detection of soil porosity based on multi-source data fusion and machine learning according to claim 1, characterized in that, The industrial control computer also includes a result display module; the data acquisition module also collects GPS information of the detection points; after obtaining GPS information of a limited number of detection points and the soil porosity of the soil sample to be tested, the Kriging interpolation method is used to fit the variogram and covariance function to simulate the spatial variability of soil porosity and draw a heat map of the spatial distribution of soil porosity at different depths; the result display module is used to display the detected value of soil porosity, the change curve of soil resistance during shearing of the soil sample to be tested, and the heat map of the spatial distribution of soil porosity.

3. The method for in-situ detection of soil porosity based on multi-source data fusion and machine learning according to claim 1, characterized in that, In step (S1), before collecting the undisturbed soil column, a layer of petroleum jelly is applied to the inner wall of the ring cutter; two undisturbed soil columns are collected adjacent to each sampling point, one of which is sent to the laboratory to test the true value of soil porosity; the other undisturbed soil column is used to perform step (S2).

4. The method for in-situ detection of soil porosity based on multi-source data fusion and machine learning according to claim 1, characterized in that, The in-situ soil porosity detection device includes a light box, a soil sample pushing and cutting mechanism, a control system, and an image acquisition system. The light box provides a sealed lighting environment. The soil sample pushing and cutting mechanism, located inside the light box, includes a frame fixed to the bottom of the light box, a pushing mechanism mounted on the frame, and a cutting mechanism. The pushing mechanism precisely pushes the undisturbed soil column or the soil sample to be tested from the ring cutter to a set height. The cutting mechanism removes the pushed-out undisturbed soil column or the soil sample. A force sensor is installed on the cutting mechanism to acquire the change curve of soil resistance during the shearing process. The control system is connected to the pushing and cutting mechanisms and controls their movement. The image acquisition system includes an industrial camera located at the top center of the light box and an industrial computer located inside the control box. The industrial camera acquires layered images of the horizontal cross-section of the undisturbed soil column or the soil sample to be tested.

5. The method for in-situ detection of soil porosity based on multi-source data fusion and machine learning according to claim 1, characterized in that, In step (S3), the image features include color features and texture features; wherein, the color features include grayscale histogram features, grayscale features, and HSV space color moments; the texture features include local binary mode features, grayscale co-occurrence matrix features, and fractal dimension.

6. The method for in-situ detection of soil porosity based on multi-source data fusion and machine learning according to claim 5, characterized in that, In step (S3), the method for extracting grayscale histogram features is as follows: (S3.1) Convert the layered image to grayscale; (S3.2) Perform Gaussian fitting on the grayscale histogram and extract five function parameters—offset, center, width, area, and height—as the grayscale histogram features of the layered image; the specific formula for Gaussian fitting is as follows: Where is the offset. Centered on, For width, For area, For height, The number of pixels for each grayscale value. This is the grayscale value.

7. The method for in-situ detection of soil porosity based on multi-source data fusion and machine learning according to claim 1, characterized in that, In step (S3), the correction formula for adjusting the volume conductivity measured at different temperatures to the volume conductivity at the standard temperature of 25℃ is as follows: in, The volume conductivity at 25°C; For correction factors; Volume conductivity measured at different temperatures; Use the following empirical formula to estimate: in, The temperature at the time of measurement.

8. The method for in-situ detection of soil porosity based on multi-source data fusion and machine learning according to claim 1, characterized in that, In step (S5), the standardized data follows a normal distribution, as shown in the formula: in, The data has been standardized. This refers to the original feature data; It is the average value of a certain characteristic variable; Let V be the variance of a certain characteristic variable.

9. The method for in-situ detection of soil porosity based on multi-source data fusion and machine learning according to claim 1, characterized in that, In step (S6), various models, including linear regression, ridge regression, support vector regression, backpropagation neural network, and gradient boosting regression tree, are used to train each machine model by substituting standardized data and the actual values ​​of soil porosity detected in the laboratory.

Images