Texture surface recognition method based on multi-dimensional friction tactile signal

Abstract

The application discloses a texture surface recognition method based on multidimensional friction tactile signal, comprising the following steps: acquiring the friction coefficient, contact feedback pressure and friction-induced vibration signal of the texture surface by using a bionic finger device, then performing pretreatment and analysis such as calibration, filtering, integration and frame division on the collected tactile data through a program, and adopting a long short-term memory superposition convolutional neural network deep learning algorithm to construct a mathematical model for texture surface classification and recognition.The application has the advantages that the spatial characteristics of the texture surface are characterized as multidimensional time domain signals by multiple sensors, and the neural network algorithm based on the signal time sequence characteristics is combined to construct the model, so that the accurate recognition of the texture surface is realized; the time sequence superposition convolutional network model of unsupervised learning is adopted, the complex data feature extraction process is omitted, and the texture surface is intelligently and quickly recognized by using a small amount of samples, so that a new method is provided for robot tactile recognition.

Description

Technical Field

[0001] This invention relates to the field of tactile recognition in human-computer interaction, and more specifically, to a method for textured surface recognition based on multi-dimensional friction tactile signals. Background Technology

[0002] Texture is a fundamental characteristic of an object's surface, possessing both visual and tactile attributes. It contains multidimensional information reflecting the surface state, such as roughness, hardness, and friction. In the industrial field, material surface texture is a crucial factor influencing product appearance and performance. The rapid development of technologies such as human-computer interaction, intelligent robots, and intelligent prosthetics has led to the widespread application of texture recognition technology in computer vision and pattern recognition. However, as texture recognition is increasingly used in complex working environments, single visual image-based texture recognition methods can no longer meet the demands for texture recognition. Tactile texture recognition identifies and distinguishes textures based on tactile feedback information obtained from machine-material surface interaction. In fields such as force-tactile human-computer interaction, robotic tactile sensing, and weak visual feature texture recognition, tactile texture recognition plays a role that is difficult to replace with visual texture recognition. Therefore, the development of tactile texture recognition methods based on tactile feedback information from interaction with material surfaces has significant application value.

[0003] Several tactile classification and recognition methods for textured surfaces have been developed: Patent CN114898219A uses tactile pressure data from a robotic arm to develop a machine learning classification method that extracts data features using a multi-head encoding self-attention layer and a feedforward neural network layer, and employs a Support Vector Machine (SVM) classification model to achieve object recognition. Patent CN117170517A maps force tactile signals collected by a flexible tactile sensor into a two-dimensional grayscale image, and uses a deep neural network learning algorithm to classify the image to achieve tactile recognition. Patent CN116738281A combines force tactile and sound signals, extracts Mel-frequency cepstral coefficients as feature vectors, and uses an SVM classification model to achieve object recognition.

[0004] Analysis of existing texture tactile classification methods reveals that while significant progress has been made, considerable limitations remain. For instance, using supervised learning SVM models requires manual design to extract tactile-related texture features, making them unsuitable for classifying unknown textures. The models exhibit weak generalization ability, and the entire process is cumbersome and complex, lacking intelligence. Furthermore, the data used for texture recognition is mostly single-modal force-tactile information, failing to provide rich and accurate texture information and being susceptible to environmental influences, resulting in poor anti-interference capabilities and limited adaptability to various application scenarios. Additionally, the temporal characteristics and spatial geometric relationships of transformed texture tactile information are ignored, leading to models that are insensitive to similar textures. Therefore, a texture surface recognition method based on multi-dimensional tactile signals and possessing unsupervised learning characteristics is urgently needed. This method should extract multi-scale geometric information and temporal correlation features from tactile sequence information as classification reference factors to improve model generalization ability and achieve intelligent and highly accurate texture surface recognition. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a texture surface recognition method based on multi-dimensional frictional tactile signals. By combining frictional tactile signals containing information about the spatial geometry of the texture, such as friction coefficient, contact feedback pressure, and friction-induced vibration, the representational dimension of the texture surface's spatial geometry is increased, thereby improving the accuracy of texture surface recognition. Furthermore, by incorporating a Long Short-Term Memory Stacked Convolutional Neural Network (LSTM-CNN) with unsupervised learning characteristics, the method automatically extracts the temporal series dependencies and spatial geometric scale features of the multi-dimensional frictional tactile signals in the temporal domain, establishing a texture surface classification model to achieve frictional tactile recognition of textured surfaces.

[0006] The specific technical solution adopted in this invention includes the following steps: Step S1: The drive module controls the bionic finger to scan the sample under test with different contact pressures, sliding speeds, and contact angles. Step S2: The signal acquisition module acquires multi-dimensional friction tactile signals, including: a three-dimensional force sensor placed above the bionic finger to acquire contact pressure and friction force signals in the sliding direction; a flexible tactile sensor embedded in the flexible fingertip of the bionic finger to acquire feedback pressure signals in contact with the sample surface; and a three-axis accelerometer fixed above the flexible fingertip of the bionic finger to acquire three-dimensional acceleration signals scanning the sample surface. Step S3 involves analyzing and processing the friction tactile signal data, including: using a first-order low-pass filter to filter the three-dimensional force sensor data acquired by the signal acquisition module. F Flexible tactile sensor data T Triaxial accelerometer data K The formula for filtering out high-frequency interference signals in the system is as follows:

[0007] In the formula, f ji The first three-dimensional force data after filtering i One value, k ji The first of the filtered triaxial acceleration data i One value, j Refers to the filtered data on the x, y, and z axes. t i The first of the filtered contact feedback pressure data i One value, F ji This represents the i-th value of the original three-dimensional force data. K ji This represents the i-th value of the original triaxial acceleration data. j Refers to the original x, y, and z axis data. T i For the first i The original contact feedback pressure values ​​are used, with the first value of the original data being used as the initial value for filtering. α For filter factors, h c The cutoff frequency, P The sampling period; The filtered 3D mechanical data is converted into friction coefficients. μ The formula is as follows: (5) In the formula, μ i Indicates the first i One friction coefficient value, f xi , f yi , f zi These are the first, second, and third axes of the filtered three-dimensional force sensor data. i One value, a, b, c These are the mean zero-position deviations of the forces along the x, y, and z axes, respectively. The contact feedback pressure is obtained by performing baseline calibration on the filtered contact feedback mechanical data. q The formula is as follows: (6) In the formula, q i For the first i A flexible tactile sensor provides contact feedback of pressure values. d This represents the average zero-position deformation deviation of the flexible tactile sensor. The filtered triaxial acceleration data is converted into vertical vibration of the bionic fingertip. s The formula is as follows: (7) In the formula, s i For the vibration data of the first i One value, k xi , k yi , k zi These are the first, second, and third axes of the filtered triaxial accelerometer data. i One value; The types of samples to be tested are used as labels and integrated with friction coefficient, feedback pressure, and vibration data into a four-dimensional array to serve as a data sample library for training the classification model. Step S4: Perform frame segmentation on the data samples to augment the data. Then, use the LSTM-CNN network classification model trained and evaluated on the sample database to identify the newly collected samples.

[0008] Step S1, the driving module controls the bionic finger to scan the surface of the sample to be tested, specifically includes: Controlling the test machine's drive device to achieve bionic fingers using pressure F z 0.8~3 N, sliding speed V x 10~40 mm / s, relative displacement L x 30 mm, angle P j : Scan the surface of the sample under test with a contact factor of 10~70°.

[0009] The data framing process in step S4 specifically includes: The data sampling method using a sliding window is used to perform frame-by-frame processing of the samples, thereby amplifying the sample data. The formula for calculating the amplified sample number n is as follows: (8) In the formula, L For the length of the sample data, q The sampling window length, w This is the sliding step size.

[0010] The training and evaluation of the classification model in step S4 specifically includes: The data was standardized to map the tactile data to a range of 0 to 1; the sample type labels were one-hot encoded, and the dataset was divided into training data and test data in a 7:3 ratio. The LSTM-CNN network classification model is trained using training data. The model includes an input layer, a parallel combination layer of the LSTM-CNN network, a feature fusion layer, a fully connected layer, and a Softmax classification layer. The gradient descent Adam algorithm is used for training, and the model is evaluated using cross-entropy and accuracy. After evaluating and optimizing the model using test data, new sample data is collected for classification and recognition.

[0011] Compared with the prior art, the beneficial effects of the present invention are: 1. By using frictional tactile signals containing information about the spatial geometry of the texture, such as friction coefficient, contact feedback pressure, and friction-induced vibration, the spatial geometry of the textured surface is characterized from multiple dimensions, providing multi-dimensional features for textured surface recognition and improving the anti-interference ability and application scenario adaptability of the classification model.

[0012] 2. Consider the influence of different contact factors on tactile signals when fingertips come into contact with objects, such as contact pressure, sliding speed, and contact angle, to increase the diversity of sample data and improve the generalization ability of the classification model.

[0013] 3. By using a deep learning model with unsupervised learning characteristics, the neural network autonomously extracts the geometric features of the friction tactile signal, eliminating the step of manually extracting signal features, which greatly improves the intelligence level of the recognition method and reduces the development time of the classification model.

[0014] 4. Using a composite neural network model with temporal series dependencies and spatial geometric scale features of the extracted signal, more features are used for model training compared to methods that manually extract geometric features from tactile signals. The recognition rate of spatially similar textures reaches over 90%, effectively improving the accuracy of texture recognition in classification models. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the process of implementing the textured surface recognition method of the present invention; Figure 2 This is a schematic diagram of the signal acquisition device and recognition method of the present invention; Figure 3 This is a schematic diagram of the processing procedure and signals for acquiring tactile data according to the present invention; Figure 4 This is a diagram of the LSTM-CNN network model architecture constructed in this invention; Figure 5 This is a flowchart of the process for training and evaluating the neural network recognition model of this invention; Figure 6 This is a confusion matrix diagram showing the accuracy of the identification results of the sample test data.

[0016] As shown in the icons, S100 is the device control step, S200 is the signal acquisition step, S300 is the data processing step, S400 is the model training step, S500 is the sample identification step, 101 is the drive module, 102 is the bionic finger, 103 is the three-dimensional force sensor, 104 is the flexible tactile sensor, 105 is the triaxial accelerometer, 106 is the stage, 201 is the time domain diagram of the friction coefficient, 202 is the time domain diagram of the contact feedback pressure, and 203 is the time domain diagram of the vibration signal. Detailed Implementation

[0017] To make the technical solution of the present invention more intuitive and clear, the present invention will be described more comprehensively below with reference to the accompanying drawings and embodiments.

[0018] The specific implementation process of this invention is as follows: like Figure 1 The diagram shows the workflow of the texture surface recognition method of the present invention, which includes the following steps: In device control step S100, the drive module controls the bionic finger to scan the sample under test with different contact pressures, sliding speeds, and contact angles; for example... Figure 2 As shown, the drive module 101 is controlled to enable the bionic finger 102 to exert pressure. F z (0.8 N, 1.5 N, 2.5 N) Sliding speed along the x-axis V x (10 mm / s, 20 mm / s, 30 mm / s, 40 mm / s), relative displacement L x 30mm, angle P j The surface of the sample to be tested is scanned at contact factors of (10°, 20°, 30°, 40°). The sample to be tested is fixed on the stage 106. The sample includes a grid texture surface with line spacing of 1 mm, 2 mm, 3 mm, and 4 mm. Each sample is scanned 3 times.

[0019] In device control step S200, the signal acquisition module acquires multi-dimensional friction tactile signals; using a USB-6002 acquisition card and adopting a reference single-ended measurement method, the sensor data is read; such as... Figure 2 As shown, a three-dimensional force sensor 103 placed above the bionic finger 102 collects contact pressure and friction force signals in the sliding direction at a sampling frequency of 1000 Hz; a flexible tactile sensor 104 embedded in the flexible fingertip of the bionic finger 102 collects feedback pressure signals from the contact sample surface at a sampling frequency of 1000 Hz; and a triaxial accelerometer 105 fixed above the flexible fingertip of the bionic finger 102 collects three-dimensional acceleration signals from the scanning sample surface at a sampling frequency of 4000 Hz.

[0020] Data processing step S300 involves analyzing and processing the friction tactile signal data, obtaining a sample database for training the classification model through methods such as calibration, filtering, and integration; for example... Figure 3 The diagram illustrates the process of collecting tactile data. Details of each step are as follows: A first-order low-pass filter with a cutoff frequency of 400 Hz and a sampling period of 0.01 s was used to filter the three-dimensional force sensor data acquired by the signal acquisition module. F Flexible tactile sensor data T Triaxial accelerometer data K The formula for filtering out high-frequency interference signals in the system is as follows:

[0021] In the formula, f ji The first three-dimensional force data after filtering i One value, k ji The first of the filtered triaxial acceleration data i One value, j Refers to the data on the x, y, and z axes. t i The first of the filtered contact feedback pressure data i One value, F ji The first of the original three-dimensional force data i One value, K ji The first of the original triaxial acceleration data i One value, j Refers to the data on the x, y, and z axes. T i For the first i The original contact feedback pressure value is used as the initial value for filtering, with the first value of the original data serving as the initial value. f j1 = F j1 , k j1 = K j1 , t 1 = T 1); α For filter factors, h c The cutoff frequency, P The sampling period; The filtered 3D mechanical data is converted into friction coefficients. μ The formula is as follows: (5) In the formula, μ i Indicates the first i One friction coefficient value, f xi , f yi , f zi These are the i-th values ​​of the filtered 3D force sensor data along the x, y, and z axes, respectively. a, b, c These are the mean zero-position deviations of the forces along the x, y, and z axes, respectively. The contact feedback pressure is obtained by performing baseline calibration on the filtered contact feedback mechanical data. q The formula is as follows: (6) In the formula, q i For the first i A flexible tactile sensor provides contact feedback of pressure values. d The mean value of the zero-position deformation deviation of the flexible tactile sensor 104; The filtered triaxial acceleration data is converted into vertical vibration of the bionic fingertip. s The formula is as follows: (7) In the formula, s i For the vibration data of the first i One value, k xi , k yi , k zi These are the filtered data from the triaxial accelerometer, representing the 105x, y, and z axes. i One value; The data processing flow is as follows Figure 3 As shown, 201 is the time-domain plot of the friction coefficient, 202 is the time-domain plot of the contact feedback pressure signal, and 203 is the time-domain plot of the vibration signal. The types of samples to be tested are used as labels and integrated with friction coefficient, feedback pressure, and vibration data into a four-dimensional array to serve as a data sample library for training the classification model.

[0022] In model training step S400, the data samples are processed by frame segmentation to achieve data augmentation. The LSTM-CNN network classification model, trained and evaluated using the sample database, is used to identify the newly collected samples. The specific implementation method of frame segmentation is as follows: A sliding window sampling method with a window length of 100 sample points and a step size of 50 sample points is used to perform frame segmentation of the samples, thereby amplifying the sample data and increasing the number of samples. nThe calculation formula is as follows: (8) In the formula, L For the length of the sample data, q The sampling window length, w This is the sliding step size; The data is standardized by mapping the friction tactile data to a range of 0 to 1. This is done by subtracting the minimum value from each dimension of tactile data and then dividing by the difference between the maximum and minimum values. One-hot encoding was performed on the sample category labels, and the dataset was divided into training data and test data in a 7:3 ratio. The LSTM-CNN network classification model is trained using training data. The model architecture is as follows: Figure 4 As shown, the model includes an input layer, an LSTM-CNN parallel ensemble layer, a feature fusion layer, a fully connected layer, and a softmax classification layer. In the LSTM-CNN parallel ensemble layer, Dropout layers are added to the LSTM network branches to prevent overfitting, and ReLU layers are added to the CNN network branches to suppress network nodes with limited effectiveness. The model is trained using the gradient descent Adam algorithm, evaluated using cross-entropy and accuracy, optimized using test data, and then applied to the classification of new samples. The overall execution process of the LSTM-CNN network classification model is as follows: Figure 5 As shown; In optimizing the LSTM-CNN network classifier, the optimal number of units and layers for the LSTM and CNN base classifiers is obtained through a traversal method. The confusion matrix of the recognition accuracy of the test data is shown in the figure below. Figure 6 As shown, a simple classification model trained using this method achieves an accuracy of over 90% in recognizing samples with similar textures.

[0023] In sample identification step S500, the texture recognition model obtained above is used to classify and identify new samples. In this step, the three-dimensional force signal, soft contact feedback force signal and triaxial acceleration signal of the sample to be tested are collected through steps S100 and S200. Then, a multi-dimensional friction tactile dataset is generated through step S300. Finally, the preprocessed input is used to train the classification model to realize the classification and identification of the sample to be tested.

[0024] In summary, this embodiment controls a bionic finger to scan the sample under test with different contact pressures, sliding speeds, and contact angles via a drive module; multi-dimensional tactile information is obtained through multiple sensors; after analyzing and processing the tactile information, a tactile signal classification training sample dataset is generated, and an LSTM-CNN network model is used to train and learn the dataset to achieve classification and recognition of the tested pattern shape. This invention combines multi-dimensional tactile signals and an LSTM-CNN network to train a classification model, which can quantitatively predict the spatial changes in the surface geometry of the tested object, the contact factors of the bionic fingertip, and the corresponding classification label of the sample. The model trained using multi-dimensional tactile information has a high recognition accuracy for objects with similar textured surfaces.

Claims

1. A method for textured surface recognition based on multi-dimensional friction tactile signals, characterized in that, Includes the following steps: Step S1: The drive module controls the bionic finger to scan the sample under test with different contact pressures, sliding speeds, and contact angles. Step S2, the signal acquisition module acquires multi-dimensional friction tactile signals, including: a three-dimensional force sensor (103) placed above the bionic finger to acquire contact pressure and friction force signals in the sliding direction; a flexible tactile sensor (104) embedded in the flexible fingertip of the bionic finger to acquire feedback pressure signals on the sample surface; and a three-axis accelerometer (105) fixed above the flexible fingertip of the bionic finger to acquire three-dimensional acceleration signals scanning the sample surface. Step S3 involves analyzing and processing the aforementioned tactile signal data, including: A first-order low-pass filter is used to filter the three-dimensional force sensor data acquired by the signal acquisition module. F Flexible tactile sensor data T Triaxial accelerometer data K The formula for filtering out high-frequency interference signals in the system is as follows: (1) (2) (3) (4) In the formula, f ji The first three-dimensional force data after filtering i One value, k ji The first of the filtered triaxial acceleration data i One value, j Refers to the filtered data on the x, y, and z axes. t i The first of the filtered contact feedback pressure data i One value, F ji This represents the i-th value of the original three-dimensional force data. K ji This represents the i-th value of the original triaxial acceleration data. j Refers to the original x, y, and z axis data. T i For the first i The original contact feedback pressure values ​​are used, with the first value of the original data being used as the initial value for filtering. α For filter factors, h c The cutoff frequency, P The sampling period; The filtered 3D mechanical data is converted into friction coefficients. μ The formula is as follows: (5) In the formula, μ i Indicates the first i One friction coefficient value, f xi , f yi , f zi These are the first, second, and third axes of the filtered three-dimensional force sensor data. i One value, a, b, c These are the mean zero-position deviations of the forces along the x, y, and z axes, respectively. The contact feedback pressure is obtained by performing baseline calibration on the filtered contact feedback mechanical data. q The formula is as follows: (6) In the formula, q i For the first i A flexible tactile sensor (104) provides contact feedback of pressure values. d The mean value of the zero-position deformation deviation of the flexible tactile sensor (104); The filtered triaxial acceleration data is converted into vertical vibration of the bionic fingertip. s The formula is as follows: (7) In the formula, s i For the vibration data of the first i One value, k xi , k yi , k zi These are the first x, y, and z axis data of the filtered triaxial accelerometer (105). i One value; The types of samples to be tested are used as labels and integrated with friction coefficient, feedback pressure, and vibration data into a four-dimensional array to serve as a data sample library for training the classification model. Step S4: The data samples are processed by frame segmentation to achieve data augmentation. The newly collected samples are identified by a classification model with a parallel architecture of long short-term memory neural network and convolutional neural network, which is trained and evaluated by the sample database.

2. The texture surface recognition method based on multi-dimensional friction tactile signals according to claim 1, characterized in that, Step S1, the driving module controls the bionic finger to scan the surface of the sample to be tested, including: The control test bench drive module enables the bionic finger to operate under pressure. F z 0.8~3 N, sliding speed V x 10~40 mm / s, relative displacement L x 30 mm, angle P j : Scan the surface of the sample under test with a contact factor of 10~70°.

3. The texture surface recognition method based on multi-dimensional friction tactile signals according to claim 1, characterized in that, Step S4, data framing processing, includes: The data sampling method using a sliding window is used to perform frame-by-frame processing of the samples, thereby amplifying the sample data. The formula for calculating the amplified sample number n is as follows: (8) In the formula, L For the length of the sample data, q The sampling window length, w This is the sliding step size.

4. The texture surface recognition method based on multi-dimensional friction tactile signals according to claim 1, characterized in that, The training and evaluation of the classification model in step S4 includes: The data was standardized, mapping the tactile data to a range of 0 to 1. Sample category labels were one-hot encoded, and the dataset was divided into training and testing data in a 7:3 ratio. A classification model using a parallel architecture of Long Short-Term Memory (LSTM) neural networks and Convolutional Neural Networks (CNNs) was trained using the training data. This model included an input layer, a parallel combination layer of LSTM and CNNs, a feature fusion layer, a fully connected layer, and a Softmax classification layer. Gradient descent was used for training, and cross-entropy and accuracy were used to evaluate the model. After optimizing the model using the testing data, new sample data was collected for classification and recognition.

Images