Online Soil Type Identification Method and System Based on One-Dimensional Force Data and Self-Attention Mechanism

Abstract

The present invention discloses an online soil type identification method and system based on one-dimensional force data and self-attention mechanism, which at least includes a data sensing and acquisition unit and a data analysis unit. The data sensing and acquisition unit is used to collect real-time pressure data during soil sampling. After sequentially converting the pressure data electrical signal into a standard analog signal and a digital signal, it is sent to the data analysis unit. The data analysis unit includes a machine learning model based on the transformer model. The machine learning model only adopts the encoder part of the transformer model, uses the self-attention mechanism and multi-head attention to improve the model's ability to obtain temporal information in the time series, outputs through a linear fully connected layer, and realizes the identification of soil types through model training and adjustment of model hyperparameters. The present invention makes better use of the time information of the detection data, the input vectors can be calculated in parallel, improves the identification accuracy, exceeds the traditional machine learning model, and meets the real-time monitoring requirements for the soil density at the sampler location.

Description

Technical Field

[0001] The present invention belongs to the fields of information technology and geological and geotechnical engineering surveying and mapping technology, and mainly relates to an online soil type identification method and system based on one-dimensional force data. Background Art

[0002] Sampling on exoplanets is an important task in space science, and the analysis of soil samples is conducive to our better understanding of the universe. However, due to the complexity and unknownness of the soil geological structure of extraterrestrial celestial bodies, when drilling for sampling, the sampler probe is very likely to touch hard objects such as rocks. If the drilling instruction continues to be executed, the sampler may be damaged. Therefore, the soil type at the position of the sampler probe should be accurately identified to guide the sampling decision.

[0003] Visual information such as images often plays an important role in sensing and recognition. However, during drilling and sampling, it is difficult to collect images of underground soil at the position of the sampler probe. Limited by lighting conditions and occlusion, it is unrealistic to use visual information to identify drilling conditions during the drilling and sampling process.

[0004] Currently, in geological and geotechnical engineering surveys and mapping, the cone penetration test (CPT) is often used to identify soil types. The CPT device includes a cylindrical probe and a conical tip with different sensors. It can push it into the ground to measure the stress at the top, sleeve friction, and pore water pressure in real time, and use traditional machine learning models to evaluate formation parameters. However, the CPT device requires many parameters to be measured, and the size and weight of the sampler are large, which do not meet the requirements of extraterrestrial sampling.

[0005] In addition, traditional machine learning algorithms such as multi-layer perceptron (MLP) are often used for soil classification at present. However, traditional machine learning algorithms have poor learning effects and low accuracy when facing multi-classification tasks with many soil types. They fail to fully utilize the temporal connection of the input feature vectors, and require multiple feature vectors and a large amount of data for training, with high costs and low efficiency.

[0006] ​The present invention precisely aims at the problems existing in the prior art and provides an online soil type identification method and system based on one-dimensional force data, which at least includes a data induction and acquisition unit and a data analysis unit. The data induction and acquisition unit is used to collect real-time pressure data during soil sampling. After sequentially converting the pressure data electrical signal into a standard analog signal and a digital signal, it is sent to the data analysis unit. The data analysis unit includes a machine learning model based on the Transformer model. The machine learning model only adopts the encoder part of the Transformer model, uses the self-attention mechanism and multi-head attention to improve the model's ability to obtain temporal information in the time series, and outputs through a linear fully connected layer. Through model training and adjusting model hyperparameters, the identification of soil types is achieved. The present invention makes better use of the time information of the detection data, improves the identification accuracy, exceeding that of traditional machine learning models; at the same time, the input vectors can be calculated in parallel, improving the identification efficiency, meeting the real-time monitoring requirements of the soil density at the sampler location, and being able to provide a reference for the decision-making of drilling.

[0007] To achieve the above object, the technical solution adopted by the present invention is: an online soil type identification system based on one-dimensional force data, which at least includes a data induction and acquisition unit and a data analysis unit.

[0008] The data induction and acquisition unit: is used to collect real-time pressure data during soil sampling. After sequentially converting the pressure data electrical signal into a standard analog signal and a digital signal, it is sent to the data analysis unit.

[0009] The data analysis unit: includes a machine learning model based on the Transformer model. The machine learning model only adopts the encoder part of the Transformer model, uses the self-attention mechanism and multi-head attention to improve the model's ability to obtain temporal information in the time series, and outputs through a linear fully connected layer. Through model training and adjusting model hyperparameters, the identification of soil types is achieved; the structure of the machine learning model in the data analysis unit is: after the input matrix is subjected to position encoding, it enters a dropout layer, enters the encoder structure, calculates through a multi-head attention, passes through the dropout and Add&Norm layers, then passes through a fully connected layer and a Relu activation function, and then sequentially passes through a dropout layer, a fully connected layer, a dropout layer and an Add&Norm layer. This is one cycle of the encoder structure. Finally, it outputs through an added fully connected layer.

[0010] As an improvement of the present invention, the data induction and acquisition unit includes an S-shaped tension and compression strain gauge sensor, a transmitter and a single-chip microcomputer; in the machine learning model of the data analysis unit, the cross-entropy of the model prediction classification result is used as the loss function, and the ADAM optimization algorithm is used to reduce the gradient.

[0011] As an improvement of the present invention, in the machine learning model, the adjustable model hyperparameters include the maximum number of types of input data, the maximum length of the input vector, the size mapped to the Transformer model, the number of heads in the multi-head attention mechanism, the size of the fully connected layer in the feed-forward layer, the number of soil types, the number of encoder layers, and the number of training epochs.

[0012] To achieve the above object, the technical solution adopted by the present invention is also: an online soil type recognition method based on one-dimensional force data, including the following steps:

[0013] S1: Measure the one-dimensional force data during soil drilling sampling through a force sensor. The time of the one-dimensional force data lasts for several seconds to obtain a set of time series, constituting a set of feature vectors;

[0014] S2: Record the feature vectors and the corresponding soil types, construct a data set for the multi-classification task of soil types, and obtain a batch of one-dimensional force data;

[0015] S3: Process the batch of feature vectors obtained in step S2. Add the label of the soil corresponding to the feature vector to the starting position of the feature vector, and combine the combined feature vectors into a sample matrix to complete the construction of the data set;

[0016] S4: Use 80% of the feature vectors of the sample matrix as the training set and 20% as the test set. Input the training set data into the data analysis unit for model training, input the test set data into the trained model, select the best hyperparameters by comparing the classification accuracy of the model with different hyperparameters, and save the best model;

[0017] S5: Collect data through the data induction acquisition unit and input the data into the best model obtained in step S4 for online recognition of soil categories.

[0018] As an improvement of the present invention, in the data processing process of step S3, use the numerical value 0 to fill the beginning of the feature vector, fill a certain number of "0"s before the valid data to make the dimensions of each group of feature vectors the same. The processed feature vector x is expressed as follows:

[0019] x i =(y i , f1, f2, ……, f 124 ) T , x i ∈R 125

[0020] Among them, y i is the label of the simulated soil corresponding to the feature vector, and x i represents the i-th group of feature vectors.

[0021] As an improvement of the present invention, in the model training of step S4, the labels and feature vectors are first separated to obtain a one-dimensional label matrix.

[0022] Y labels =(y1,y2,……,y n )

[0023] Among them, n is the number of classes of feature vectors in the data set, y i Represents the label of the i-th group of eigenvectors and a two-dimensional eigenvector matrix;

[0024] X features =(x1,x2,……,x n ),x i ∈R 124

[0025] in x i Represents the i-th group of eigenvectors; perform this operation on the training set and the test set respectively, and obtain 4 matrices, named train labels , train features , test labels and test features , train represents the training set data, and test represents the test set data;

[0026] Multiply the eigenvector matrix by 1000:

[0027] train features =train features ×1000;

[0028] test features =test features ×1000;

[0029] The value of the valid data in the feature vector is increased to enhance the difference from the leading zero.

[0030] As another improvement of the present invention, in the model training of step S4, the output result of the model is a matrix O∈R n*num_class , where n is the number of eigenvectors, num_class is the number of classes required; the output matrix O gives the probability that each input eigenvector corresponds to a soil type of 0 to num_class-1, q(x i )=(p(x i =0),p(x i =1),p(x i =2),……,p(x i =num_class-1)), p(xi = a), where a = {0, 1, 2, ……, num_class - 1}, representing the probability that the corresponding soil type of the feature vector x i is a, and satisfies

[0031] As an improvement of the present invention, step S5 further includes:

[0032] S51: When the sampler drills for soil sampling, the pressure sensor on the sampler obtains real-time pressure data and outputs an electrical signal;

[0033] S52: The output electrical signal is connected to a transmitter, and after being amplified and filtered, it is converted into a standard analog signal, and the standard analog signal is 0 - 5V;

[0034] S53: The standard analog signal is converted into a digital signal by a single-chip microcomputer through analog-to-digital conversion and sent to the data analysis unit using the serial communication protocol;

[0035] S54: The machine learning model in the data analysis unit is trained using the digital signal. The input matrix first performs position encoding, then enters a dropout layer, enters the encoder structure, calculates through a multi-head attention, then passes through the dropout and Add&Norm layers, then passes through a fully connected layer and a ReLU activation function, and then passes through the dropout layer, fully connected layer, dropout layer and Add&Norm layer in sequence to form a cycle of the encoder structure;

[0036] S55: Set the number of training rounds to achieve N cycles;

[0037] S56: Finally, add a fully connected layer to map the dimension of the training result to the required number of soil types to achieve the identification of soil types.

[0038] As an improvement of the present invention, the signal sampling frequency in step S51 is 10Hz, and the range of the pressure sensor is 0 - 100N; the voltage of the transmitter in step S52 is 12V.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) Compared with traditional soil classification methods, the present invention does not use visual information such as images and spectrograms, and reduces the number of sensors. Compared with CPT geological exploration, the present invention only relies on a small one-dimensional force sensor, and the acquisition equipment has low cost and light weight;

[0041] (2) The present invention constructs a machine learning model based on the self-attention mechanism, which improves the ability of the model to obtain temporal information in time series, fully utilizes the temporal connection of pressure data, and improves the recognition accuracy. Compared with traditional machine learning models, this model has a higher accuracy for multi-classification problems of time series and can accurately identify various soil types.

[0042] (3) The present invention constructs a machine learning model based on the self-attention mechanism. The input feature vectors can be calculated in parallel, and the calculation of the next step does not depend on the calculation result of the previous step, which improves the training time and recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic structural diagram of the experimental platform in Embodiment 2 of the present invention;

[0044] Figure 2 It is a schematic structural diagram of the S-shaped tensile and compressive strain sensor used in the present invention;

[0045] Figure 3 It is a schematic structural diagram of the machine learning model based on the transformer model of the present invention;

[0046] Figure 4 It is a flowchart of the steps of the method of the present invention.

[0047] In the figure:

[0048] 1. Stepper motor 2. Slide block. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] The present invention will be further clarified below in conjunction with the drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0050] Embodiment 1

[0051] An on-line soil type recognition system based on one-dimensional force data includes at least a data induction and acquisition unit and a data analysis unit.

[0052] The data induction and acquisition unit includes an S-shaped tensile and compressive strain sensor, a transmitter, and a single-chip microcomputer, and is used to collect real-time pressure data during soil sampling. After converting the pressure data electrical signal into a standard analog signal and a digital signal in sequence, it is sent to the data analysis unit; the pressure sensor is installed on the sampler to collect real-time pressure data during drilling sampling of the sampler and output an electrical signal; the output signal is connected to the transmitter and converted into a 0-5V standard analog signal after processes such as amplification and filtering; the standard analog signal is converted into a digital signal by the single-chip microcomputer through analog-to-digital conversion, and then sent to the data analysis unit using the serial communication protocol.

[0053] The data analysis unit includes a machine learning model based on the Transformer model. The data analysis unit processes and analyzes the one-dimensional force data during the drilling and sampling process. Since the mechanical data collected in different soils has certain differences, a model network is constructed based on the encoder part of the Transformer model to make full use of the temporal relationship of the mechanical data to identify this difference, thereby realizing the identification of soil types;

[0054] The main structure of the model is as follows: The input matrix first undergoes positional encoding, then enters a dropout layer, and then enters the encoder structure. First, it passes through a multi-head attention calculation, then through dropout and Add&Norm layers, then through a fully connected layer and a Relu activation function, and then through dropout layers, fully connected layers, dropout layers, and Add&Norm layers in sequence. This is one cycle of the encoder structure, and the number of cycles N can be set by parameters; finally, a fully connected layer is added at the end to map the dimension of the result to the number of required soil types to realize the identification of soil types.

[0055] During the data analysis process, during training, the data of the training set is used as the input matrix and input into the model network of the present invention. The cross-entropy of the model prediction classification result is used as the loss function, and the ADAM optimization algorithm is used to reduce the gradient, so that the model can learn the force data corresponding to different soil types.

[0056] The system of the present invention is based on the encoder part of the Transformer model, realizing the autonomous classification and recognition of time series. In the case of only using the one-dimensional force data obtained by the sensor during the drilling process, we make better use of the time information of the detection data, improve the recognition accuracy, and exceed the traditional machine learning model. At the same time, the input vectors can be calculated in parallel, improving the recognition efficiency, meeting the real-time monitoring requirements of the soil compactness at the sampler location, and being able to provide a reference for the decision-making of drilling.

[0057] Embodiment 2:

[0058] In this embodiment, five kinds of granular materials with different particle diameters are used to establish a simulated soil data set. Among them, the particle diameter of the cultivated soil is about 0.1 - 0.15 mm, the particle diameter of the cement is 0.40 - 0.55 mm, the particle diameter of the sand is 0.5 - 1.0 mm, the particle diameter of the mung beans is 3 - 4 mm, and the particle diameter of the gravel is 10 - 20 mm; the experimental platform is as Figure 1 shown, including a sliding rail platform that can move in the vertical direction, a one-dimensional force sensor as [[ID=ID=18]] Figure 2 shown, a cylindrical probe, and a container filled with cement powder, sand, gravel, mung beans or ordinary soil.

[0059] Online soil type identification method based on one-dimensional force data, the specific steps are as follows:

[0060] Step S1: Measure the one-dimensional force data during simulated soil drilling sampling on the experimental platform through a force sensor. Determine the time duration of a set of one-dimensional force data for several seconds to obtain a set of time series, which records the pressure change of the position where the drill bit is located during the sampling process. Such a set of time series constitutes a set of feature vectors;

[0061] Figure 1 The shown sliding platform can simulate the drilling and sampling process of a planetary surface sampler by adjusting the stepper motor 1 to make the slider 2 rise or fall at a specified speed. Figure 2 The shown force sensor is an S-shaped tensile pressure strain sensor, which can measure the pressure in the vertical direction. Its upper end is connected to the slider 2 of the sliding platform through a connector, and its lower end is connected to a cylindrical probe to simulate the sampler. When the slider 2 moves downward at a uniform speed, one end of the probe will contact the test soil to simulate the process of drilling and sampling.

[0062] When collecting data, make the slider move downward at a uniform speed. One end of the probe will contact the test soil to simulate the process of drilling and sampling. At this time, the sensor will be squeezed and pressure data will be obtained. The data is sent to the computer through the serial communication protocol after amplification and filtering and stored in the form of a file for subsequent construction of the data set and model training process.

[0063] In the data collection stage, first place the sampler so that its bottom is close to the simulated soil without contacting the soil, and then start the sliding table to let the sampler descend at a specified speed. Once the slider 2 starts to move, the data of the force sensor will be immediately transmitted to the computer. When the slider moves past the preset distance, immediately close the serial port tool of the computer to stop receiving data. Repeat the measurement process, test different soil samples, and obtain enough data.

[0064] Step S2: Repeat the process of this step, collect the pressure data during drilling and sampling in a variety of simulated soils. The number of simulated soils should be no less than 2 types. Record the feature vectors and the corresponding soil types for constructing the data set of the soil type multi-classification task to obtain a large amount of one-dimensional force data; In this embodiment, 5 different simulated soils are used. The types of simulated soils used and their labels are: cement ash is 0, sand is 1, cultivated soil is 2, mung beans are 3, and gravel is 4;

[0065] Step S3: Process the batch of feature vectors obtained in Step S2, add the label of the soil corresponding to the feature vector to the starting position of the feature vector, and combine the combined feature vectors into a sample matrix to complete the construction of the data set;

[0066] The value 0 is used to pad the beginning of the feature vector, that is, a number of "0"s are padded before the valid data (zero-padding), so that the dimensions of each set of feature vectors are the same. In this embodiment, the dimension of the feature vector is 124. In addition, the first bit is the label of the simulated soil type, and the length of each set of feature vectors is 125. The processed feature vector x is expressed as follows:

[0067] x i =(y i ,f1,f2,……,f 124 ) T ,x i ∈R 125

[0068] Among them, y i is the label of the simulated soil corresponding to the eigenvector, x i represents the i-th group of eigenvectors, f1,f2,……,f 124 The data is the one-dimensional force data after leading zero padding. The data set is divided into training set and test set in a ratio of 8:2. Part of the data used for training is shown in Table 1 below:

[0069] Table 1 Part of the data set

[0070]

[0071]

[0072] Each column in the table represents pressure data collected at one time, representing a set of feature vectors. The first row contains the label for the soil type corresponding to the feature vector (0 for cement, 1 for sand, 2 for culture soil, 3 for mung bean, and 4 for gravel). Excluding the label, the length of each feature vector should be 124. This table only captures a portion of the feature vectors.

[0073] S4: 80% of the eigenvectors of the sample matrix are used as the training set, and 20% as the test set. The training set data is input into the data analysis unit for model training, and the test set data is input into the trained model. The optimal hyperparameters are selected by comparing the classification accuracy of the model with different hyperparameters, and the optimal model is saved.

[0074] The above is the offline training process. The specific description of the training process is as follows:

[0075] First, separate the label and the feature vector to obtain a one-dimensional label matrix.

[0076] Y labels =(y1,y2,……,y n )

[0077] Among them, n is the number of classes of feature vectors in the dataset, and y i represents the label of the i-th group of feature vectors and a two-dimensional feature vector matrix

[0078] X features =(x1, x2, ……, x n ), where x i ∈R 124

[0079] Among them x i represents the i-th group of feature vectors. Perform this operation on the training set and the test set respectively to obtain 4 matrices, which are named train labels and train features , test labels and test features respectively. train represents the training set data, and test represents the test set data;

[0080] Multiply the feature vector matrix by 1000:

[0081] train features [[ID=�5]]= train features ×1000;

[0082] test features = test features ×1000;

[0083] Increase the value of the valid data in the feature vector to enhance the difference from the leading 0.

[0084] In the machine learning model based on the self-attention mechanism in this embodiment, the learnable parameters are initialized with a Gaussian distribution with a mean of 0 and a standard deviation of 0.09; then set the hyperparameters of the model; select the ADAM optimization algorithm and implement the model training process based on the pytorch library of python. First, zero the gradients through the optimizer.zero_grad() function; then put train features as the input matrix into the model to obtain the output matrix O, and calculate the cross-entropy between the output matrix O and the label vector train labels as the loss function; then calculate the gradients of the loss function through the loss.backward() function by backpropagation to obtain the gradients of each parameter; finally, implement gradient descent and update the model parameters through the optimizer.step() function to complete one round of training;

[0085] Repeat the training for num_epochs rounds to obtain the trained model; use the test set data test featuresPut the trained model, compare the predicted classification results with the actual results. If the accuracy rate is higher than the preset value, save the model; otherwise, adjust the hyperparameters of the model and repeat the above training process. The saved model will be applied to the subsequent online recognition process.

[0086] The output result of the model is a matrix O ∈ R n*num_class , where n is the number of feature vectors, and num_class is the number of classes to be divided. In this embodiment, it is taken as 5. The output matrix O gives the probabilities corresponding to the soil types 0 to 4 for each input feature vector, q(x i ) = (p(x i = 0), p(x i = 1), i(x i = 2), p(x i = 3), p(x i = 4)), p(x i = a), a = {0, 1, 2, 3, 4}, representing the probability that the soil type corresponding to the feature vector x i is of type a.

[0087] The machine learning model is constructed based on the self-attention mechanism and multi-head attention, which can improve the ability of the model to make full use of the time information in the time series, thereby improving the classification accuracy. The main structure of the model is described as follows: The input matrix first maps the dimension to the dimension of the model (d_embed) through word embedding, then performs positional encoding on the input matrix, and then enters the encoder structure. The input matrix at this time is recorded as src1. After that, it enters a dropout layer, then performs multi-head attention calculation, and then passes through another dropout layer. Denote the matrix at this time as src, and perform the operation src = src + src1[1]. Then perform layer normalization [2] on src (the operations [1] and [2] together are called Add&Norm). Denote the matrix at this time as src’. src’ passes through the fully connected layer 1, relu activation function, dropout layer, fully connected layer 2, and dropout layer in sequence and then becomes src3. Execute the operation src’ = src’ + src3 to obtain the output matrix src’, forming a cycle of the encoder structure; The number of cycles of the encoder structure can be set to N by setting the hyperparameter num_Iayers; Finally, add a fully connected layer to map the dimension of the training result to the number of soils to be classified, so as to realize the recognition of soil types. The framework of the model is as Figure 3 shown.

[0088] This embodiment uses the self-attention mechanism to improve the classification accuracy of univariate time series. Different from traditional classifiers, the present invention makes full use of the time information in the force signal to improve the classification accuracy. The detailed implementation process of the model is described as follows:

[0089] The input matrix is first mapped using input embeddings. The input matrix \(X\in\mathbb{R}\) w×m =\([x_1, x_2, \ldots, x\) w \) T , where each vector \(x\) t \(\in\mathbb{R}\) m is linearly mapped to a \(d\)-dimensional vector. \(u\) t \(\in\mathbb{R}\) d fully connected layer, and \(d\) is the dimension (\(d_{embed}\)) of the Transformer model. That is:

[0090] \(U = XW\) p + b p ; \(W\) p \(\in\mathbb{R}\) mΔd , \(b\) p \(\in\mathbb{R}\) d .

[0091] Then, positional encoding is added to the matrix \(U\). Since the model used is based on a feed - forward architecture and does not use any convolutional or recurrent neural networks, the model adopted in the present invention is insensitive to the positional information of the input sequence. In this model, trigonometric functions are used to encode the positions:

[0092] \(PE\) (pos，2i) = \(\sin(\text{pos} / 1000\) 2i / d_embed )

[0093] \(PE\) (pos，2i+1) = \(\cos(\text{pos} / 1000\) 2i / d_embed )

[0094] The generated positional encoding is added to the mapped matrix \(U\). For different input vectors, although there are different elements at the same position, the positional encoding vector \(PE\in\mathbb{R}\) d is always the same and generates a positional matrix \(W\) pos \(\in\mathbb{R}\) w×d . The positional encoding adds the positional matrix \(W\) pos to the matrix \(U\in\mathbb{R}\) w×d = \([u_1, u_2, \ldots, u\) w \) T as follows:

[0095] \(U' = U + W\) pos .

[0096] The input vector after positional encoding first enters the multi - head attention operation. For the input vector \(U'\in\mathbb{R}\) w×d = \([u_1, u_2, \ldots, u\) w \) T , it is multiplied by three matrices Map to a new matrix:

[0097] Q = U'W q ,

[0098] K = U'W k ,

[0099] V = U'W v ,

[0100] Then its dot product attention score is expressed as:

[0101]

[0102] The matrices Q, K, V obtained from dot product attention are then mapped to h groups, i.e.:

[0103] Q i = QW q，i ,

[0104] K i = KW k，i ,

[0105] V i = VW v，i ,

[0106] For the matrices with the same value of i (i.e., within the same head) numbered i = 1, 2,..., h, calculate their dot product attention for the output matrices Q i , K i , V i Concatenate them, and then pass through a fully connected layer to change the dimension to d_embed, which is the dimension of the model.

[0107] MultiHead(Q, K, V) = Concat(head1, head2,......, head h )W o

[0108] where head i = Attention(QW q，i , KW k，i , VW v，i ),

[0109] where d q = d k , d = d_embed.

[0110] Then pass through a dropout layer, add the result to the input vector of the positional encoding, and then pass through layer normalization (i.e., the Add&Norm layer). After that, pass through the fully connected layer W liner1 ∈ R d_embed×d_feed_forwardAfter applying the ReLU activation function and then dropout, it passes through the fully connected layer \(W\). liner2 ∈R d_eed_forward×d_embed After passing through the dropout layer, it then undergoes Add&Norm. This is the operation for one encoder layer.

[0111] Repeat the operation of the encoder layer \(num\_layers\) times.

[0112] Finally, when outputting, add another fully connected layer \(W\). fc ∈R d_embed×d_class , and map the dimension of the result to the required number of classification categories to achieve the classification of soil.

[0113] Among them, the loss function is the cross - entropy between the model prediction result and the actual label result. Using the output result \(O\in R\) of the model n*num_class and the label vector \(train\). labels Calculate the cross - entropy. For each group of feature vectors, the cross - entropy between the model output result and the actual result is expressed as:

[0114]

[0115] Among them, \(p(x\) i ) is the actual probability distribution function, and \(q(x\) i ) is the probability distribution function predicted by the model. In the classification problem of the present invention, if the soil type is the same as the label (i.e., \(x\) i = label serial number), then \(p(x\) i ) is 1, otherwise the probability is 0.

[0116] The hyperparameters in the model are: the maximum number of types of input data \(len\_traj\); the maximum length of the input vector \(d\_obs\); the dimension \(d\_embed\) mapped to the Transformer model; the number of heads \(n\_heads\) in the multi - head attention mechanism; the dimension \(d\_hidden\) of the fully connected layer in the feed - forward layer; the number of soil types \(d\_class\); the number of encoder layers \(n\_layers\) and the number of training epochs \(num\_epochs\), as well as the learning rate \(lr\).

[0117] The model parameters of this embodiment are set as shown in Table 2.

[0118] Table 2 Model Parameter Table

[0119]

[0120] [[ID=S5]]S5: Collect data through the data sensing and acquisition unit, and input the data into the best model obtained in step S4 for online identification of soil categories. The specific steps are as follows:

[0121] S51: When the sampler drills for soil sampling, the pressure sensor on the sampler obtains real-time pressure data and outputs an electrical signal. The signal sampling frequency is 10 Hz, and the range of the pressure sensor is 0 - 100 N;

[0122] S52: The output electrical signal is connected to a transmitter, which is converted into a standard analog signal after amplification and filtering. The standard analog signal is 0 - 5 V, and the voltage of the transmitter is 12 V;

[0123] S53: The standard analog signal is converted into a digital signal by a single-chip microcomputer and sent to the data analysis unit using the serial communication protocol;

[0124] S54: The machine learning model in the data analysis unit is trained using the digital signal. The input matrix first undergoes positional encoding, then enters a dropout layer, enters the encoder structure, undergoes a multi-head attention calculation, and then passes through dropout and Add&Norm layers. After that, it passes through a fully connected layer and a ReLU activation function, and then successively passes through dropout layers, fully connected layers, dropout layers, and Add&Norm layers to form one cycle of the encoder structure;

[0125] S55: Set the number of training epochs to achieve N cycles;

[0126] S56: Finally, add a fully connected layer to map the dimension of the training result to the number of required soil types to achieve soil type recognition.

[0127] During the verification process, the data in the test set is input into the trained model for prediction. The prediction results are compared with the actual results, and the accuracy, macro-precision, macro-recall, and F1 score are calculated as indicators to evaluate the classification effect of the model. By adjusting the hyperparameters of the model, the model can have the best classification effect.

[0128] Our model can correctly identify all the data in the test set containing 5 soil types. The classification accuracy rate exceeds 99%. Compared with traditional machine learning models, the model of the present invention has a higher accuracy rate when classifying multiple soil types.

[0129] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements all fall within the protection scope of the claims of the present invention.

Claims

1. An on-line soil type recognition system based on one-dimensional force data and self-attention mechanism, characterized in that It includes at least a data sensing and acquisition unit and a data analysis unit. The data sensing and acquisition unit: is used to collect real-time pressure data during soil sampling, convert the pressure data electrical signal into a standard analog signal and then a digital signal in sequence, and send it to the data analysis unit. The data analysis unit: at least includes a machine learning model based on the transformer model. The machine learning model only adopts the encoder part of the transformer model, uses the self-attention mechanism and multi-head attention to improve the model's ability to obtain temporal information in the time series, and outputs through a linear fully connected layer. Through model training and adjusting the model hyperparameters, the recognition of soil types is achieved. The structure of the machine learning model is as follows: After the input matrix is position-encoded, it enters a dropout layer, then enters the encoder structure, calculates through a multi-head attention, passes through the dropout and Add&Norm layers, then passes through a fully connected layer and the Relu activation function, and then passes through the dropout layer, fully connected layer, dropout layer and Add&Norm layer in sequence. This is one cycle of the encoder structure, and finally outputs through an added fully connected layer.

2. The on-line soil type identification system based on one-dimensional force data and self-attention mechanism according to claim 1, characterized in that: The data sensing and acquisition unit at least includes an S-shaped tension and compression strain sensor, a transmitter and a single-chip microcomputer; in the machine learning model of the data analysis unit, the cross-entropy of the model prediction classification result is used as the loss function, and the ADAM optimization algorithm is used to reduce the gradient.

3. The on-line soil type identification system based on one-dimensional force data and self-attention mechanism according to claim 2, characterized in that: In the machine learning model, the adjustable model hyperparameters include the maximum number of input data types, the maximum length of the input vector, the size mapped to the transformer model, the number of heads in the multi-head attention mechanism, the size of the fully connected layer in the feed-forward layer, the number of soil types, the number of encoder layers, and the number of training epochs.

4. An online soil type identification method based on one-dimensional force data and self-attention mechanism using the system according to claim 1, characterized in that It includes the following steps: S1: Measure the one-dimensional force data during soil drilling sampling through a force sensor. The time of the one-dimensional force data lasts for several seconds to obtain a set of time series, which constitutes a set of feature vectors. S2: Record the feature vectors and the corresponding soil types, construct a dataset for the multi-classification task of soil types, and obtain a batch of one-dimensional force data. S3: Process the batch of feature vectors obtained in step S2. Add the label of the soil corresponding to the feature vector at the beginning of the feature vector, and combine the combined feature vectors into a sample matrix to complete the construction of the dataset. S4: Use 80% of the feature vectors of the sample matrix as the training set and 20% as the test set. Input the training set data into the data analysis unit for model training, input the test set data into the trained model, select the best hyperparameters by comparing the classification accuracy of the model with different hyperparameters, and save the best model. S5: Collect data through the data sensing and acquisition unit and input the data into the best model obtained in step S4 for online identification of soil types.

5. The online soil type identification method based on one-dimensional force data and self-attention mechanism according to claim 4, characterized in that: During the data processing in step S3, use the numerical value 0 to fill the head of the feature vector, fill several "0"s before the valid data to make the dimension of each group of feature vectors the same. The processed feature vector x is represented as follows: x i = (y i , f1, f2, ……, f 124 ), T , x i ∈R 125 where y i is the label of the simulated soil corresponding to the eigenvector, and xi represents the i-th group of eigenvectors.

6. The online soil type identification method based on one-dimensional force data and self-attention mechanism according to claim 5, characterized in that: In the model training of step S4, first, the labels and feature vectors are separated to obtain a one-dimensional label matrix. Y labels =(y1, y2, ……, y n ) where n is the number of classes of feature vectors in the dataset, and y i represents the label of the i-th group of feature vectors and a two-dimensional feature vector matrix; X features =(x1, x2, ……, x n ), x i ∈R 124 Among them x i represents the i-th group of feature vectors; perform this operation on the training set and the test set respectively to obtain 4 matrices, named train labels , train features , test labels and test features respectively, where train represents the training set data and test represents the test set data; Multiply the feature vector matrix by 1000: train features = train features × 1000; teSt features = test features × 1000; Increase the values of the valid data in the feature vectors to enhance the difference from the leading 0s.

7. The online soil type recognition method based on one-dimensional force data and self-attention mechanism according to claim 5, characterized in that: In the model training of step S4, the output result of the model is a matrix O ∈ R n*num_class , where n is the number of feature vectors, and num_class is the number of classes to be classified; the output matrix O gives the probabilities that for each input feature vector, it corresponds to soil types from 0 to num_class - 1 respectively. q(x i ) = (p(x i = 0), p(x i = 1), p(x i = 2), …, p(x i = hum_class - 1)), p(x i = a), a = {0, 1, 2, ……, hum_class - 1}, represents the probability that the corresponding soil type of the feature vector x i is a, and satisfies 8. The online soil type identification method based on one-dimensional force data and self-attention mechanism according to claim 7, characterized in that: Step S5 further includes: S51: When the sampler drills for soil sampling, the pressure sensor on the sampler obtains real-time pressure data and outputs an electrical signal. S52: The output electrical signal is connected to a transmitter, and after being amplified and filtered, it is converted into a standard analog signal, and the standard analog signal is 0 - 5V. S53: The standard analog signal is converted into a digital signal by a microcontroller through analog-to-digital conversion and sent to the data analysis unit using the serial communication protocol. S54: The machine learning model in the data analysis unit is trained using the digital signal. The input matrix first undergoes positional encoding, then enters a dropout layer, enters the encoder structure, performs a multi-head attention calculation, then passes through the dropout and Add&Norm layers, then passes through a fully connected layer and the Relu activation function, and then successively passes through the dropout layer, fully connected layer, dropout layer, and Add&Norm layer to form one cycle of the encoder structure. S55: Set the number of training epochs to achieve N cycles. S56: Finally, add a fully connected layer to map the dimension of the training result to the number of required soil types to achieve the identification of soil types.

9. The online soil type identification method based on one-dimensional force data and self-attention mechanism according to claim 8, characterized in that: In step S51, the signal sampling frequency is 10Hz, and the range of the pressure sensor is 0 - 100N; in step S52, the voltage of the transmitter is 12V.

Images