A hardness identification method, hardness identification system, device and storage medium

Abstract

The present application relates to the field of robot hardness recognition, and discloses a hardness recognition method, a hardness recognition system, a device and a storage medium. The method includes: acquiring tactile data; wherein the tactile data is obtained based on the torque data and pressure data collected by the manipulator pressing the sample object; preprocessing the tactile data to obtain a tactile image; inputting the tactile image into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object; wherein the hardness recognition network includes at least one fully connected layer, and the classification category of the fully connected layer is determined by Poisson distribution coding. In the above manner, the present application can learn the ordered relationship between hardness levels and classify the hardness.

Description

Technical Field

[0001] The present application relates to the field of robot hardness identification, and in particular to a hardness identification method, a hardness identification system, a hardness identification device and a computer-readable storage medium. Background Art

[0002] When the environment is dark or the visual sensor cannot work, tactile recognition plays an important role in the robot's exploration of the environment. When the robot touches an object, it can obtain many characteristics of the object, such as the texture, roughness, and hardness of the surface. Among them, hardness is one of the most important tactile properties for the robot to recognize objects.

[0003] Tactile hardness recognition can supplement information that visual sensors cannot perceive. For example, some fruits have no obvious difference in appearance before and after ripening, and it is difficult to judge the ripeness of fruits only through visual sensors. However, with the help of tactile hardness recognition, it is easy to judge the ripeness of fruits, because the hardness of fruits with different ripeness varies greatly. So far, deep learning has shown superior performance in object hardness recognition.

[0004] Traditional deep learning-based hardness recognition methods have certain limitations, as these methods are unable to learn the ordered information between the hardness levels of objects. Summary of the invention

[0005] The main technical problem solved by the present application is to provide a hardness identification method that can correctly identify the hardness category and reflect the relationship between different hardnesses.

[0006] In order to solve the above technical problems, a technical solution adopted by the present application is: providing a hardness identification method, comprising:

[0007] Acquire tactile data; wherein the tactile data is obtained based on torque data and pressure data collected by the manipulator when pressing the sample object; preprocess the tactile data to obtain a tactile image; input the tactile image into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object; wherein the hardness recognition network includes at least one fully connected layer, and the classification category of the fully connected layer is determined by Poisson distribution coding.

[0008] Among them, Poisson distribution encoding is implemented based on the following formula;

[0009]

[0010] Among them, K represents the number of predicted categories, H(k) represents the logarithm of the probability density function of the Poisson distribution, It represents the sum of the probabilities of all predicted categories, which is 1, and exp(-H(k)) represents the probability of encoding the correct category.

[0011] The method also includes that, among a preset number of fingers of the manipulator, a torque sensor and a plurality of array-distributed pressure sensors are arranged on each finger; wherein, preprocessing the tactile data to obtain a tactile image includes: forming a first pressure data matrix based on data collected by the plurality of array-distributed pressure sensors of each finger; splicing the first pressure data matrices corresponding to the preset number of fingers to form a second pressure data matrix; and forming a first torque data matrix based on data collected by the torque sensors of the preset number of fingers; copying the first torque data matrix to form a second torque data matrix; splicing the second pressure data matrix with the second torque data matrix to obtain a tactile image.

[0012] Among them, downsampling processing is performed on tactile data at multiple time points; multiple downsampled tactile images are combined to form a tactile image sequence; the tactile image is input into a pre-trained hardness recognition network to classify the tactile image, and further obtain the hardness of the sample object, including: inputting the tactile image sequence into a pre-trained hardness recognition network to classify the tactile image sequence, and further obtain the hardness of the sample object.

[0013] The tactile data is subjected to denoising to remove the data between the manipulator contacting the sample object.

[0014] This also includes regularizing the tactile data.

[0015] The method also includes establishing a hardness recognition network; obtaining a data set; wherein the data set includes a plurality of tactile data; preprocessing the data set to obtain a plurality of tactile image sequences; and inputting the plurality of tactile image sequences into the hardness recognition network to train the hardness recognition network.

[0016] Among them, the hardness recognition network is a CNN-LSTM network, which includes a CNN module, an LSTM layer and a fully connected layer. The CNN module includes a convolutional layer, a batch normalization layer and a pooling layer.

[0017] Among them, multiple tactile image sequences are divided into a training data set and a test data set; the training data set is input into a hardness recognition network to train the hardness recognition network; and the test data set is input into the hardness recognition network to test the hardness recognition network.

[0018] Wherein, a plurality of tactile image sequences are classified according to the shapes of sample objects; and each type of tactile image sequence is divided into a training data set and a test data set.

[0019] In order to solve the above problems, another technical solution adopted in the present application is: to provide a hardness recognition system, including: an acquisition module, used to acquire tactile data; wherein the tactile data is obtained based on the torque data and pressure data collected by the manipulator pressing the sample object; a preprocessing module, used to preprocess the tactile data to obtain a tactile image; a recognition module, used to input the tactile image into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object; wherein the hardness recognition network includes at least one fully connected layer, and the classification category of the fully connected layer is determined by Poisson distribution encoding.

[0020] In order to solve the above problem, another technical solution adopted in the present application is: to provide a hardness identification device, including: a processor and a memory coupled to the processor, the memory stores a computer program, and the processor is used to execute the computer program to implement the above method.

[0021] In order to solve the above problem, another technical solution adopted by the present application is: providing a computer-readable storage medium, wherein the computer-readable storage medium stores program data, and the program data is used to implement the above method when executed by a processor.

[0022] The beneficial effects of the present application are as follows: Different from the prior art, the present application provides a hardness identification method. The method obtains tactile data; wherein the tactile data is obtained according to the torque data and pressure data collected by the manipulator pressing the sample object; the tactile data is preprocessed to obtain a tactile image; the tactile image is input into a pre-trained hardness recognition network, and the tactile image is classified to achieve the effect of hardness identification of the object. At the same time, the hardness recognition network adopted in the present application includes at least one fully connected layer, and the classification category of the fully connected layer is determined by Poisson distribution coding. Poisson distribution coding is based on Poisson distribution, which can not only reflect the correct category, but also reflect the ordered information between different hardness levels. In the above manner, the present application can fully learn the ordered information between hardness levels and classify the hardness. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0024] Figure 1 It is a flow chart of an embodiment of a hardness identification method provided by the present application;

[0025] Figure 2 It is a flow chart of an embodiment of a tactile image construction method provided by the present application;

[0026] Figure 3 It is a flow chart of an embodiment of a preprocessing method provided by the present application;

[0027] Figure 4 It is a flow chart of another embodiment of a pretreatment method provided by the present application;

[0028] Figure 5 It is a flow chart of another embodiment of a pretreatment method provided by the present application;

[0029] Figure 6 It is a flow chart of an embodiment of a hardness recognition network construction method provided by the present application;

[0030] Figure 7 It is a structural schematic diagram of an embodiment of a hardness identification network provided by the present application;

[0031] Figure 8 It is a flow chart of another embodiment of a hardness recognition network construction method provided by the present application;

[0032] Fig. 9 It is a structural schematic diagram of an embodiment of a hardness identification system provided by the present application;

[0033] Fig.10 It is a structural schematic diagram of an embodiment of a hardness identification device provided by the present application;

[0034] Fig.11 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application; DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all methods and processes related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0036] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0037] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0038] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a hardness identification method provided by the present application. The hardness identification method of this embodiment specifically includes steps 100 to 300:

[0039] Step 100: Acquire tactile data; wherein the tactile data is obtained based on torque data and pressure data collected when the manipulator performs a pressing operation on the sample object.

[0040] A manipulator is a robot that can imitate certain movements and functions of human hands and arms. It is an automatic operating device used to grasp, carry objects or operate tools according to a fixed program. It can be programmed to complete various expected operations. It combines the advantages of both human and mechanical machines in terms of structure and performance. In the field of hardness identification, when collecting experimental data, sensors are usually installed on the fingers of the manipulator to collect data values ​​with different attribute characteristics that can characterize the hardness of the object. Commonly used robot finger sensors include pressure sensors, torque sensors, joint angle sensors, etc.

[0041] In this embodiment, pressure sensors and torque sensors are installed on the fingers of the manipulator, and pressure data and torque data are obtained by pressing the sample object, and these two types of data are used as tactile data.

[0042] Optionally, a joint angle sensor may be installed on the manipulator to obtain joint angle data as tactile data by pressing the sample object.

[0043] In this embodiment, the silicone sample is pressed to obtain tactile data. The silicone sample is made by mixing AB glue in a ratio of 1:1 and pouring it into a 3D printing mold, and standing it at room temperature for 6 hours. The silicone sample has the following three different shapes, namely a cuboid, a cuboid with protrusions, and a flower shape. Each shape of the silicone sample has 13 different hardnesses, ranging from 0A to 60A (shoreA scale). A total of 39 silicone samples were made using this method.

[0044] The specific process of data collection is as follows: first, place the silicone on the turntable, set the robot at a fixed vertical height H from the silicone sample, press the silicone downward and return to the initial height position; adjust the turntable to rotate the silicone 7.5 degrees, and repeat the pressing process; perform the above experiment 60 times on each sample. In this way, a total of 2340 data were collected.

[0045] Step 200: preprocessing the tactile data to obtain a tactile image;

[0046] After collecting tactile data, some preprocessing is usually performed to remove invalid data from the tactile data, or to transform the data to a data form that can more intuitively reflect the hardness category. The tactile data can be constructed in the form of a tactile image or in the form of a tactile image sequence.

[0047] Step 300: Input the tactile image into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object; wherein the hardness recognition network includes at least one fully connected layer, and the classification category of the fully connected layer is determined by Poisson distribution coding.

[0048] After obtaining the tactile image, the tactile image is input into the hardness recognition network to train the hardness recognition network. The hardness recognition network contains a fully connected layer. After the feature is input into the fully connected layer, a K-dimensional prediction vector is output, where K represents the type of classification task. In traditional solutions, one-hot encoding is usually used to encode the correct category.

[0049] One-hot encoding is also called one-bit effective encoding. It uses an N-bit state register to encode N states. Each state has its own independent register bit, and at any time, only one of them is valid. For example, in an event, 5 categories are encoded, and the correct category is the third category, then the encoding sequence is 00100.

[0050] In the hardness recognition network, if one-hot encoding is used to encode the correct category, since only the encoding at the position corresponding to the correct hardness category is 1 and the other positions are 0, in each iteration, learning can only be performed based on the encoding of the only correct category, and the hardness information of other categories is lost.

[0051] Poisson distribution coding is derived and evolved based on the Poisson distribution function. The probability density function image of the Poisson distribution is similar to the normal distribution, which is high in the middle and low at both ends. Using Poisson distribution to encode the correct hardness category can reflect the correct category without losing the ordered information of other hardness categories. For example, in a specific embodiment, the highest position in the middle can be set to represent the correct hardness category 5, and the nearest positions on the left and right sides represent the other two categories 4 and 6 with the smallest difference in hardness, and so on. The use of Poisson distribution coding avoids the direct loss of hierarchical ordered information of other hardness categories while retaining the most identifiable correct category coding. Therefore, the use of Poisson distribution coding can fully learn the ordered information between hardness levels, and the hardness recognition network can be trained by calculating the cross entropy loss between the prediction vector and the Poisson distribution coding, which can effectively classify the hardness.

[0052] Among them, Poisson distribution encoding is implemented based on the following formula:

[0053]

[0054] Where K represents the number of predicted categories, H(k) represents the logarithm of the probability density function of the Poisson distribution, It represents the sum of the probabilities of all predicted categories, which is 1, and exp(-H(k)) represents the probability of encoding the correct category.

[0055] Poisson distribution coding is derived from the following formula:

[0056] The probability density function of the Poisson distribution is defined as:

[0057]

[0058] Where 1≤k≤K, K represents the number of predicted categories, and λ represents the peak position control value of Poisson distribution. In this application, the λ value is not directly taken at the position where the index value of the category is aligned with the peak value of Poisson distribution, but the λ value obtained when the ratio of the index value of the correct category to the sum of the index values ​​of the left and right categories is minimized. The solution process is as follows:

[0059]

[0060] Further, it is equivalent to solving:

[0061]

[0062] You can get:

[0063]

[0064] make Substituting it into the probability density function of the Poisson distribution, we get:

[0065]

[0066] Furthermore, taking the logarithm of D(k), we obtain:

[0067]

[0068] Finally, a normalization function is applied to H(k) so that the sum of H(k) is 1, and E(k) can be obtained.

[0069] The present application obtains tactile data; wherein the tactile data is obtained by collecting torque data and pressure data when the manipulator presses the sample object; the tactile data is preprocessed to obtain a tactile image; and finally the tactile data is input into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object. Through this method, not only the hardness of the sample object can be identified, but also the relationship between the sample object and different hardness can be reflected.

[0070] In the process of preprocessing, the present application proposes a tactile image construction method, such as Figure 2 This embodiment specifically includes steps 210 to 250:

[0071] Tactile data is collected according to the following steps by arranging a torque sensor and a plurality of array-distributed pressure sensors on each of a preset number of fingers of the manipulator.

[0072] Step 210: forming a first pressure data matrix according to the data collected by the pressure sensors distributed in a plurality of arrays on each finger;

[0073] Step 220: splicing the first pressure data matrices corresponding to a preset number of fingers to form a second pressure data matrix;

[0074] Since pressure sensors and torque sensors are installed on more than one finger of the manipulator during the experiment, different first pressure data matrices are collected on each finger during each pressing of the sample object. In order to simultaneously apply different data on multiple manipulator fingers, the first pressure data matrices corresponding to a preset number of manipulator fingers are spliced ​​into a second pressure data matrix.

[0075] Step 230: forming a first torque data matrix according to data collected by torque sensors of a preset number of fingers;

[0076] Step 240: copying the first moment data matrix to form a second moment data matrix;

[0077] A torque sensor is installed on a preset number of fingers of the manipulator, wherein each torque sensor can collect a torque data, so in one experiment, a preset number of torque data can be collected to form a first torque data matrix. In order to form a tactile image with the data collected in one experiment, the first torque data matrix is ​​copied to form a second torque data matrix, so that the first matrix dimension of the second torque data matrix is ​​the same as the first matrix dimension of the second pressure data matrix.

[0078] Step 250: splicing the second pressure data matrix and the second moment data matrix to obtain a tactile image.

[0079] The second pressure data matrix and the second moment data matrix are spliced ​​in the first matrix dimension to obtain a tactile image.

[0080] Through the above embodiments, a tactile image can be constructed for each time point in each acquisition experiment. In the preprocessing process, other methods can also be used to process the tactile data. This application proposes a preprocessing method in Example 3, please refer to Figure 3 , the specific steps include 260a to 280:

[0081] Generally speaking, in the process of data collection by the robot, a lot of invalid experimental data will be included and there will be noise in the data. Through the preprocessing operation steps, valid data can be obtained and the noise information can be effectively removed.

[0082] Step 260a: down-sampling the tactile data at multiple time points;

[0083] In the process of collecting tactile data, the collection time points are very dense, which often causes data oversampling and introduces some noise. In order to remove such noise, the tactile data is downsampled and the tactile data of some time points are retained for subsequent operations.

[0084] Step 270a: combining the downsampled tactile images to form a tactile image sequence;

[0085] According to Embodiment 2, the tactile data can be constructed into a tactile image. In each acquisition process, the tactile data is acquired at multiple time points, so a tactile image can be constructed at each time point. The downsampled tactile images are combined to form a tactile image sequence.

[0086] Step 280: Input the tactile image sequence into a pre-trained hardness recognition network, classify the tactile image sequence, and further obtain the hardness of the sample object.

[0087] The tactile image sequence contains temporal and spatial information that can characterize the hardness category. Inputting the tactile image sequence into a pre-trained hardness recognition network can effectively extract the spatial and temporal features of the tactile data, thereby more effectively identifying the hardness category.

[0088] This application also proposes a pretreatment method in Example 4, please refer to Figure 4 , the specific steps include 260b to 280:

[0089] Step 260b: performing denoising processing on the data at multiple time points;

[0090] Since the data in the period from when the manipulator starts to descend to when it contacts the sample object cannot represent the hardness of the sample object, the data in this period is invalid data and needs to be truncated. In this operation step, this invalid data is removed.

[0091] Step 270b: combining the multiple denoised tactile image sequences to form a tactile image sequence;

[0092] Step 280: Input the tactile image sequence into a pre-trained hardness recognition network, classify the tactile image sequence, and further obtain the hardness of the sample object.

[0093] This application also proposes a pretreatment method in Example 5, please refer to Figure 5 , the specific steps include 260c to 280:

[0094] Step 260c: performing regularization processing on the tactile data at multiple time points;

[0095] The regularization process calculates the p-norm of each tactile image sequence, and divides each data in the tactile image sequence by the norm, so that the p-norm of each processed tactile image sequence is equal to 1. After this processing, the hardness category information can be effectively extracted during the training of the hardness recognition network, so as to classify the hardness.

[0096] Step 270c: combining the regularized multiple tactile image sequences to form a tactile image sequence;

[0097] Step 280: Input the tactile image sequence into a pre-trained hardness recognition network, classify the tactile image sequence, and further obtain the hardness of the sample object.

[0098] Optionally, step 260a, step 260b and step 260c may apply one or more of them to pre-process the tactile data, and step 260a may be performed before step 260b, or after step 260b.

[0099] After the tactile data is preprocessed, it is input into the hardness recognition network for training and testing. This application proposes a method for constructing a hardness recognition network, such as Figure 6 As shown, it can be used to construct a hardness recognition network, which specifically includes steps 310 to 330:

[0100] Step 310: Obtain a data set;

[0101] Wherein, the data set includes a plurality of tactile data;

[0102] Step 320: preprocessing the data set to obtain a plurality of tactile image sequences;

[0103] The data set may be preprocessed according to embodiments 2-5 to obtain multiple tactile image sequences.

[0104] Step 330: Input the tactile image sequence into the hardness recognition network to train the hardness recognition network.

[0105] For details on the composition of the hardness recognition network, please refer to Figure 7 , Figure 7 It is a structural diagram of an embodiment of a hardness recognition network proposed by the present invention.

[0106] The structure includes: a tactile image sequence 40, a hardness recognition network 50 and a Poisson distribution code 60. The hardness recognition network 50 includes: a first CNN module 51, a second CNN module 52, an LSTM layer 53 and a fully connected layer 54. First, the tactile image sequence 40 is input into the hardness recognition network 50 for feature extraction, and the feature data processed by the first CNN module 51 and the second CNN module 52 is input into the LSTM layer 53 with 100 units, and finally the feature data is input into the fully connected layer 54. The fully connected layer 54 converts high-dimensional data into a K-dimensional prediction vector, and trains the hardness recognition network 50 by calculating the cross entropy loss between the K-dimensional prediction vector and the correct vector encoded by the Poisson distribution code 60.

[0107] The first CNN module 51 and / or the second CNN module 52 include a convolutional layer, a batch normalization layer and a pooling layer.

[0108] In the process of training the hardness recognition network, the present application proposes another embodiment of a hardness recognition network construction method, such as Figure 8As shown, it can be used to construct a hardness recognition network, which specifically includes steps 331 to 333:

[0109] Step 331: Divide the plurality of tactile image sequences into a training data set and a test data set;

[0110] Before training and testing the hardness recognition network, the tactile image sequence needs to be divided into non-overlapping training data sets and test data sets to obtain more accurate classification results. First, the tactile image sequences of different shapes are trained and tested respectively, and the data of each shape is randomly divided into 10 parts. In each iteration, 9 of them are used as training sets and the remaining one is used as a test set. Then, all tactile image sequences of all shapes are trained and tested, and all the data are randomly divided into 10 parts. In each iteration, 9 of them are used as training sets and the remaining one is used as a test set.

[0111] Step 332: inputting the training data set into the hardness recognition network to train the hardness recognition network;

[0112] In this application, a ten-fold cross-validation method is used to train the hardness recognition network.

[0113] Step 333 inputs the test data set into the hardness recognition network to test the hardness recognition network;

[0114] Through Example 8, the hardness recognition network can be trained and tested in a data set containing only one shape and in a data set containing all shapes, so as to verify the classification performance of the network and make the classification result of the hardness recognition network more reliable.

[0115] According to Examples 1-8, see Fig. 9 , Fig. 9 It is a structural diagram of an embodiment of the hardness identification system proposed in this application. The system 10 includes: an acquisition module 11, a preprocessing module 12 and an identification module 13;

[0116] The acquisition module 11 is used to acquire tactile data; wherein the tactile data is obtained based on the torque data and pressure data collected by the manipulator when pressing the sample object; the preprocessing module 12 is used to preprocess the tactile data to obtain a tactile image; the recognition module 13 is used to input the tactile image into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object; wherein the hardness recognition network includes at least one fully connected layer, and the classification category of the fully connected layer is determined by Poisson distribution coding.

[0117] Specific reference Fig.10 , Fig.10 It is a structural schematic diagram of an embodiment of a hardness identification device provided in the present application.

[0118] The hardness identification device 20 includes a processor 21 and a memory 22. The processor 21 and the memory 22 are coupled. The memory 22 stores a computer program, which is used to execute the hardness identification method.

[0119] Specific reference Fig.11 , Fig.11 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application.

[0120] The computer-readable storage medium 30 includes program data 31. When the program data 31 is executed by the processor, the hardness identification method described above can be implemented.

[0121] Different from the prior art, the present application proposes a hardness recognition method. First, tactile data is obtained; wherein, the tactile data is obtained according to the torque data and pressure data collected by the manipulator when pressing the sample object; the tactile data is preprocessed to obtain a tactile image; the tactile image is input into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object. In addition, the present application proposes to use Poisson distribution to encode the correct category as a correct category vector to train the hardness recognition network. The correct vector encoded by Poisson distribution contains not only the correct category information, but also the ordered information between different hardness levels. Therefore, Poisson distribution encoding can effectively classify hardness. At the same time, the present invention inputs the tactile image sequence into the hardness recognition network, which can obtain the spatial and temporal characteristics of tactile hardness, so as to better classify the hardness. In general, the present invention can fully learn the ordered information of the hardness level and classify the hardness, and has the advantages of high classification accuracy and good stability.

[0122] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A hardness identification method, characterized in that: The method comprises: Acquiring tactile data; wherein the tactile data is obtained based on the torque data and pressure data collected by the manipulator performing a pressing operation on the sample object; Preprocessing the tactile data to obtain a tactile image; Inputting the tactile image into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object; Wherein, the hardness recognition network includes at least one fully connected layer, and the classification category of the fully connected layer is determined by Poisson distribution coding; The Poisson distribution encoding is implemented based on the following formula: Where K represents the number of predicted categories, H(k) represents the logarithm of the probability density function of the Poisson distribution, It represents the sum of the probabilities of all predicted categories, which is 1, and exp(-H(k)) represents the probability of encoding the correct category.

2. The method according to claim 1, characterized in that: Among the preset number of fingers of the manipulator, each of the fingers is provided with a torque sensor and a plurality of pressure sensors distributed in an array; The preprocessing of the tactile data to obtain a tactile image includes: Forming a first pressure data matrix according to data collected by a plurality of array-distributed pressure sensors of each finger; splicing the first pressure data matrices corresponding to a preset number of fingers to form a second pressure data matrix; and Forming a first torque data matrix according to data collected by torque sensors of a preset number of fingers; Copying the first moment data matrix to form a second moment data matrix; The second pressure data matrix and the second moment data matrix are spliced ​​to obtain a tactile image.

3. The method according to claim 2, characterized in that The method further comprises: Down-sampling the tactile data at multiple time points; Combining the downsampled tactile images to form a tactile image sequence; The step of inputting the tactile image into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object includes: The tactile image sequence is input into a pre-trained hardness recognition network to classify the tactile image sequence, and further obtain the hardness of the sample object.

4. The method according to claim 2, characterized in that: The method further comprises: The tactile data is subjected to denoising processing to remove data before the manipulator contacts the sample object.

5. The method according to claim 2, characterized in that: The method further comprises: Regularization processing is performed on the tactile data.

6. The method according to claim 1, characterized in that The method further comprises: Establish a hardness identification network; Acquire a data set; wherein the data set includes a plurality of the tactile data; Preprocessing the data set to obtain a plurality of tactile image sequences; The plurality of tactile image sequences are input into the hardness recognition network to train the hardness recognition network.

7. The method according to claim 6, characterized in that The hardness recognition network is a CNN-LSTM network, which includes a CNN module, an LSTM layer and a fully connected layer, and the CNN module includes a convolutional layer, a batch normalization layer and a pooling layer.

8. The method according to claim 6, characterized in that The step of inputting the plurality of tactile image sequences into the hardness recognition network to train the hardness recognition network comprises: Dividing the plurality of tactile image sequences into a training data set and a test data set; Inputting the training data set into the hardness recognition network to train the hardness recognition network; The test data set is input into the hardness recognition network to test the hardness recognition network.

9. The method according to claim 8, characterized in that The step of dividing the plurality of tactile image sequences into a training data set and a test data set comprises: classifying the plurality of tactile image sequences according to the shape of the sample object; The tactile image sequences of all shapes are divided into a training data set and a test data set.

10. A hardness identification system, characterized in that: The hardness identification system comprises: An acquisition module, used for acquiring tactile data; wherein the tactile data is obtained based on the torque data and pressure data collected by the manipulator performing a pressing operation on the sample object; A preprocessing module, used for preprocessing the tactile data to obtain a tactile image sequence; The recognition module is used to input the tactile image into a pre-trained hardness recognition network to classify the tactile image and further obtain the hardness of the sample object; wherein the hardness recognition network includes at least one fully connected layer, and the classification category of the fully connected layer is determined by Poisson distribution coding; the Poisson distribution coding is implemented based on the following formula: Where K represents the number of predicted categories, H(k) represents the logarithm of the probability density function of the Poisson distribution, It represents the sum of the probabilities of all predicted categories, which is 1, and exp(-H(k)) represents the probability of encoding the correct category.

11. A hardness identification device, characterized in that: The hardness identification device includes a processor and a memory coupled to the processor, wherein a computer program is stored in the memory, and the processor is configured to execute the computer program to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program data, and when the program data is executed by a processor, it is used to implement the method according to any one of claims 1 to 9.

Images