A piezoresistive glove intelligent recognition device based on a six-branch deep capsule network

Abstract

This invention discloses a piezoresistive glove intelligent recognition device based on a six-branch deep capsule network, relating to the fields of intelligent robot tactile sensing technology and perception algorithm technology. The device includes a knitted glove, a flexible tactile piezoresistive sensor array, a resistance-to-voltage conversion module, a data acquisition card, and an artificial intelligence algorithm module. Twenty-six flexible tactile piezoresistive sensors are sewn onto the knitted glove at specific points and connected to the resistance-to-voltage conversion module and the data acquisition card. The data is ultimately transmitted to a host computer. After establishing a grasping dataset using pre-drawn tactile information images, the six-branch deep capsule network is trained to achieve accurate object recognition. This invention's six-branch deep capsule network piezoresistive glove intelligent recognition device utilizes a flexible tactile sensor array to grasp and recognize different objects, and improves recognition accuracy through the six-branch deep capsule network. Its excellent human-computer interaction capabilities provide a novel approach for the development of intelligent robot perception and recognition applications.

Description

Technical Field

[0001] This invention relates to the fields of intelligent robot tactile sensing technology and perception algorithm technology, specifically to an intelligent identification device for piezoresistive gloves based on a six-branch deep capsule network. Background Technology

[0002] With the rapid development of sensor technology and micro / nano manufacturing, flexible tactile sensors, exhibiting high sensitivity, low cost, and high flexibility, have found widespread application in fields such as electronic skin, human activity monitoring, and healthcare. Flexible tactile sensors can be mainly classified into capacitive, piezoelectric, triboelectric, and piezoresistive effects based on their working principles. Comparatively, flexible piezoresistive tactile sensors indirectly change the distribution and contact state of the internal conductive material by deforming the active material under external force, resulting in a regular change in the device's resistance. Due to their ability to respond to static stress, high reliability, and wide testing pressure range, flexible piezoresistive tactile sensors offer increasing possibilities for the research and development of flexible electronic products.

[0003] To increase sensing contact points and expand application range, flexible piezoresistive tactile sensors are arrayed and combined with advanced artificial intelligence algorithms. This enables convenient detection and recognition of objects with different characteristics and is widely used in the field of intelligent wearable robots. Smart gloves with tactile sensor arrays worn on robotic arms serve as an important medium between external objects and the robotic arm. They convert complex mechanical signals into computationally processable electrical signals, further extracting tactile features. Deep learning algorithms are used to process tactile information data, perceiving the contact state of the target object and its own characteristic information, enabling object recognition, exploration, stability estimation, and force control. Therefore, exploring flexible sensors with high sensitivity, high reliability, and a wide testing pressure range to increase sensing capabilities and testing range, and using deep learning algorithms to adjust object recognition strategies, has significant scientific and engineering value for the widespread application of intelligent recognition and autonomous operation in intelligent robots. Summary of the Invention

[0004] To address the shortcomings of existing technologies, such as the limited number of sensing contact points and narrow application range of tactile sensors on intelligent robot manipulators, which result in insufficient sensitivity and the need for improved deep learning algorithms for information processing, this invention provides a piezoresistive glove intelligent recognition device based on a six-branch deep capsule network. Furthermore, it also improves the deep learning algorithm for information processing based on this device.

[0005] This invention is achieved through the following technical solution: a piezoresistive glove intelligent recognition device based on a six-branch deep capsule network, comprising a knitted glove, a flexible tactile sensor array, a resistance-to-voltage conversion module, a data acquisition card, and an artificial intelligence algorithm module. The flexible tactile sensor array is distributed on the palm surface and the inner surface of the finger bones of the knitted glove. The flexible tactile sensor array outputs resistance information to the resistance-to-voltage conversion module. The resistance-to-voltage conversion module outputs voltage information to the data acquisition card. The data acquisition card outputs the collected voltage data information to the artificial intelligence algorithm module of the host computer.

[0006] Furthermore, the flexible tactile sensing array includes 26 flexible tactile piezoresistive sensors, distributed as follows: one flexible tactile piezoresistive sensor is provided on the inner surface of each of the three phalanges of the index finger, middle finger, ring finger and little finger; one flexible tactile piezoresistive sensor is provided on the inner surface of each of the two phalanges of the thumb; and the remaining twelve flexible tactile piezoresistive sensors are evenly distributed on the palm surface.

[0007] Preferably, each flexible tactile piezoresistive sensor is rectangular and includes an encapsulation film, an interdigital electrode layer, an isolation layer, and a flexible piezoresistive film. The encapsulation film has two layers, namely the uppermost layer and the lowermost layer. Between the two encapsulation film layers, from top to bottom, are the interdigital electrode layer and the flexible piezoresistive film. The isolation layer is an adhesive strip, distributed on the outer side of three sides of the flexible piezoresistive film and adhered to the upper and lower encapsulation films. The remaining side exposes the electrode connection points of the interdigital electrode layer.

[0008] Preferably, the encapsulation film is made of flexible polyimide film, the interdigitated electrode layer is formed on the encapsulation layer using screen printing technology with interdigitated electrodes spaced 100μm apart, and the isolation layer is made of double-sided 3M tape.

[0009] Preferably, the method for preparing the flexible piezoresistive film includes the following steps:

[0010] (1) Disperse carbon fibers and carbon nanotubes in acetone at a mass ratio of 6:1 and stir for 4-6 hours. Then, ultrasonically disperse the mixed solution after stirring for 30-60 minutes. Heat in a fume hood until the acetone solution is completely evaporated to obtain a mixture of carbon fiber and carbon nanotube powders.

[0011] (2) Add the carbon fiber and carbon nanotube powder mixture with a mass ratio of 3:20 to polydimethylsiloxane in 10 batches and stir for 3-6 hours.

[0012] (3) Add the curing agent with a mass ratio of 1:10 to the carbon fiber / carbon nanotube / polydimethylsiloxane mixture and stir for 20min-40min. Pour it onto a cloth with a horizontal and vertical interwoven pattern and coat it into a 300μm thick film using an adjustable film scraper. Then cure it on a heating table at 100℃ for 30min-50min to form a flexible piezoresistive film with a cloth texture pattern.

[0013] The present invention discloses a piezoresistive glove intelligent recognition device based on a six-branch deep capsule network. The artificial intelligence algorithm module of this device uses a six-branch deep capsule network for computation. The six-branch deep capsule network includes a base capsule layer, a 3D routing layer, a classification routing layer, and an image reconstruction layer. The algorithm flow includes the following steps:

[0014] (1) Using Python's matplotlib2D plotting library, the data input to the host computer is plotted on a hand contour image according to the position of 26 flexible tactile piezoresistive sensors on the piezoresistive glove and the force of the sensors to form a sub-image; the body contact with the object is reflected by six touchable surface sub-images, the six sub-images are stitched together to form a large image grasping dataset, and the grasping dataset is divided into training set, validation set and test set in a ratio of 7:2:1;

[0015] (2) The grasping training set is fed into the basic capsule layer and the 3D routing layer. The basic capsule layer includes a regular convolutional layer and a convolutional capsule layer. The 3D routing layer applies a 3D routing protocol. The results of the basic capsule layer and the 3D routing layer are compressed into 2D matrices and horizontally concatenated. The 2D matrices of each branch are fed into the classification routing layer. After obtaining the results, they are input into the image reconstruction layer to reconstruct the images of each branch. The reconstruction process is to input the target vector into the fully connected network to obtain the output neuron. After five transposed convolutions, the image is continuously upsampled to obtain the final result. The reconstruction loss is then calculated.

[0016] (3) The 2D matrices of each branch are vertically concatenated and fed into the classification routing layer to obtain the final prediction result. The edge loss calculation function is used to predict the relative distance between the input sample and the output sample. The loss functions used in the network are the edge loss calculation function and the reconstruction loss. The edge loss calculation function is shown in Equation (1):

[0017] Margin_loss=Y*max(0,0.9-V)+λ(1-Y)*max(0,V-0.1) (1)

[0018] V is the magnitude of the predicted vector, Y is the one-hot encoding of the true label, and λ is used to control the effect of gradient backpropagation in the initial stage of training.

[0019] The reconstruction loss is obtained by comparing the reconstructed predicted image of each branch with the original image using the mean square error loss at the pixel level, as shown in Equation (2).

[0020]

[0021] In equation (2), x and y represent the tensors of the original image and the reconstructed image, respectively; x ij and y ij These represent the elements in the tensor; M and N represent the number of rows and columns of the tensor.

[0022] After obtaining the mean square error and edge loss, the overall loss of the multi-branch deep capsule network is calculated as shown in Equation (3). Since the mean square error is much larger than the edge loss, the weight value is used to balance the impact on the network.

[0023] Loss = Margin loss +αMSE_loss (3)

[0024] (4) Input the validation set into the trained six-branch deep capsule network to adjust the model performance and hyperparameters to obtain the trained network model;

[0025] (5) The divided test set is input into the trained six-branch deep capsule network model for testing, and the prediction output of the test set is obtained with an accuracy of 99.80%.

[0026] Furthermore, in step (2), the calculation process of the 3D routing protocol includes the following steps:

[0027] Set the feature map received by the 3D routing layer to Φ l (w l w l c l n l w is the size of the feature map; c represents the depth of the 3D capsule; n represents Φ. l The number of capsules;

[0028] Set the size of the 3D convolution kernel to (3×3×c). l The step size in the horizontal and vertical directions is 1, and the step size in the depth direction is c. l The quantity is c l *n l ;

[0029] Φ l The number of capsules is increased after passing through a 3D convolutional layer, and the result is shown in equation (4):

[0030] U=(w l w l c l cl ×n l (4)

[0031] Each n in U l The set of matrices can be considered as a capsule T, as shown in equation (5):

[0032] T = (w u w u n l (5)

[0033] The prior probability matrix is ​​set as shown in equation (6):

[0034] B = (w u ×w u ×n l c l ×n l (6)

[0035] The coupling coefficient is calculated using the Softmax function as shown in equation (7);

[0036]

[0037] The coupling coefficient matrix K obtained after transformation by B is shown in equation (8):

[0038] K = (w u w u n l c l ×n l (8)

[0039] Multiplying K and U by the dot product yields S, as shown in equation (9):

[0040] S = (T, c) l n l (9)

[0041] Arrange the results in S according to n l Dimensions added together;

[0042] Using the Squash function as shown in equation (10), the prior matrix B is activated to obtain V;

[0043]

[0044] After obtaining V, multiply it by U and add it to B to obtain a new matrix B as shown in equation (11);

[0045] B = V * U + B (11)

[0046] Repeat steps (6)-(11) three times to update V and make Φ l+1 =V.

[0047] Furthermore, in step (3), the classification routing layer calculation process includes the following steps:

[0048] The input data of the fully connected layer in the classification routing layer is set as a 2D capsule M = (a, c) of the image, and the data of the next layer is calculated in a fully connected manner as shown in Equation (12);

[0049] y = Wx + b 12)

[0050] Output multiple 2D capsules as shown in equation (13):

[0051] U = (n, a, v) (13)

[0052] In equation (12), n is the final number of categories;

[0053] The prior matrix is ​​set as shown in equation (14):

[0054] B = (n × v, a) (14)

[0055] B is fed into the Softmax function to obtain the coupling coefficient K;

[0056] The coefficient matrix K is transformed as shown in equation (15):

[0057] K = (n, a, v) (15)

[0058] Take the dot product of K and U to obtain S;

[0059] Add the data in S along dimension a to get (n, v);

[0060] V is obtained by activating B using the Squash function;

[0061] After obtaining v, multiply it by U and add it to B to obtain a new matrix as shown in equation (16), and update the coupling coefficients;

[0062] B = V * U + B (16)

[0063] Repeat steps (15)-(16) three times to obtain the final classification result V.

[0064] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a piezoresistive glove intelligent recognition device based on a six-branch deep capsule network:

[0065] (1) The flexible tactile piezoresistive sensor proposed in this invention utilizes the difference in the three-dimensional size structure of carbon fiber and carbon nanotube to uniformly disperse the two in polydimethylsiloxane after mixing, which has high sensitivity and wide pressure testing range.

[0066] (2) The capsule network in the intelligent recognition device for piezoresistive gloves based on a six-branch deep capsule network proposed in this invention not only considers the relationships between neurons but also focuses on the relationships between features. Each capsule represents a specific feature, and the relationships between feature components can be calculated inside the capsule, thereby effectively capturing the spatial layout of the object. In addition, the multi-branch deep capsule network used in this invention represents the sensor signals collected by the flexible tactile sensing array on a hand-shaped background image and establishes a dataset by using a multi-subgraph connection method. The dataset is then trained and learned to obtain a network model with strong generalization ability.

[0067] (3) The present invention proposes a smart identification device for piezoresistive gloves based on a six-branch deep capsule network. It involves sewing 26 flexible tactile piezoresistive sensors with high sensitivity and wide pressure testing range onto the knitted glove according to the points, and connecting them with a resistance-to-voltage conversion module and a data acquisition card. The monitored tactile signals are transmitted to the host computer, and the object is accurately identified by the well-trained six-branch deep capsule network. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of the intelligent identification device for piezoresistive gloves based on a six-branch deep capsule network according to the present invention.

[0069] Figure 2 This is a schematic diagram of the flexible tactile piezoresistive sensor structure prepared according to the present invention.

[0070] Figure 3 In the image, (1) is a scanning electron microscope image of carbon fiber, (2) is a scanning electron microscope image of carbon nanotubes, and (3) is a scanning electron microscope image of flexible piezoresistive membrane.

[0071] Figure 4 This is a schematic diagram of a six-branch deep capsule network model.

[0072] Figure 5 The set of 30 objects to be identified.

[0073] Figure 6 This is the confusion matrix used to identify 30 objects.

[0074] The following labels are used in the diagram: 1-knitted glove, 2-flexible tactile piezoresistive sensor, 3-resistance-to-voltage conversion module, 4-data acquisition card, 5-artificial intelligence algorithm, 6-encapsulation film, 7-flexible piezoresistive film, 8-isolation layer, 9-interdigital electrode layer. Detailed Implementation

[0075] The present invention will be further described below with reference to specific embodiments.

[0076] A piezoresistive glove intelligent recognition device based on a six-branch deep capsule network, with the structure as follows: Figure 1 As shown: It includes a knitted glove 1, a flexible tactile sensor array 2, a resistance-to-voltage conversion module 3, a data acquisition card 4, and an artificial intelligence algorithm module 5. The flexible tactile sensor array 2 is distributed on the palm surface and the inner surface of the finger bones of the knitted glove 1. The flexible tactile sensor array 2 outputs resistance information to the resistance-to-voltage conversion module 3. The resistance-to-voltage conversion module 3 outputs voltage information to the data acquisition card 4. The data acquisition card 4 outputs the collected voltage data information to the artificial intelligence algorithm module 5 of the host computer.

[0077] In this embodiment, the flexible tactile sensing array 2 includes 26 flexible tactile piezoresistive sensors, which are distributed as follows: a flexible tactile piezoresistive sensor is provided on the inner surface of each of the three phalanges of the index finger, middle finger, ring finger and little finger; a flexible tactile piezoresistive sensor is provided on the inner surface of each of the two phalanges of the thumb; and the remaining twelve flexible tactile piezoresistive sensors are evenly distributed on the palm surface.

[0078] Each flexible tactile piezoresistive sensor in this embodiment is rectangular, such as... Figure 2 As shown, the system includes an encapsulation film 6, an interdigitated electrode layer 9, an isolation layer 8, and a flexible piezoresistive film 7. The encapsulation film 6 has two layers, an uppermost layer and a lowermost layer. Between the two encapsulation film layers 6, from top to bottom, are the interdigitated electrode layer 9 and the flexible piezoresistive film 7. The isolation layer 8 is an adhesive strip distributed on the outer three sides of the flexible piezoresistive film 7 and adhered to the upper and lower encapsulation films 6, with the remaining side exposing the electrode connection points of the interdigitated electrode layer 9. The encapsulation film 6 is made of flexible polyimide film. The interdigitated electrode layer 9 is formed on the encapsulation layer using screen printing technology with interdigitated electrodes spaced 100μm apart. The isolation layer 8 uses double-sided 3M tape.

[0079] The preparation method of the flexible piezoresistive film 7 in this embodiment includes the following steps:

[0080] (1) Carbon fibers and carbon nanotubes with a mass ratio of 6:1 were dispersed in acetone and stirred for 4-6 hours. The stirred mixture was then ultrasonically dispersed for 30-60 minutes. The mixture was heated in a fume hood until the acetone solution was completely evaporated, yielding a mixture of carbon fiber and carbon nanotube powders. The scanning electron microscope image of the carbon fibers is shown below. Figure 3 As shown in (1), the scanning electron microscope image of carbon nanotubes is as follows. Figure 3 As shown in (2).

[0081] (2) Add the carbon fiber and carbon nanotube powder mixture with a mass ratio of 3:20 to polydimethylsiloxane in 10 batches and stir for 3-6 hours.

[0082] (3) Add a curing agent at a mass ratio of 1:10 to the carbon fiber / carbon nanotube / polydimethylsiloxane mixture and stir for 20-40 minutes. Pour the mixture onto a fabric with a cross-hatched pattern and coat it into a 300 μm thick film using an adjustable film scraper. Cure the film on a heating stage at 100°C for 30-50 minutes to form a flexible piezoresistive film with a fabric texture. A scanning electron microscope image of the flexible piezoresistive film is shown below. Figure 3 As shown in (3).

[0083] In the piezoresistive glove intelligent recognition device designed in this embodiment based on a six-branch deep capsule network, the artificial intelligence algorithm module 5 uses a six-branch deep capsule network for calculation, such as... Figure 4 As shown, the six-branch deep capsule network includes a base capsule layer, a 3D routing layer, a classification routing layer, and an image reconstruction layer. The algorithm flow includes the following steps:

[0084] (1) Using Python's matplotlib2D plotting library, the data input to the host computer was plotted onto a hand contour image based on the positions of 26 flexible tactile piezoresistive sensors on the piezoresistive glove and the force applied to the sensors, forming a sub-image. Six touchable surface sub-images were used to reflect the body contact with the object. These six sub-images were stitched together to form a large grasping dataset, which was then divided into a training set, a validation set, and a test set in a 7:2:1 ratio. The dataset contained 30 objects, such as... Figure 5 As shown;

[0085] (2) The grasping training set is fed into the basic capsule layer and the 3D routing layer. The basic capsule layer includes a regular convolutional layer and a convolutional capsule layer, and the 3D routing layer applies a 3D routing protocol. The results of the basic capsule layer and the 3D routing layer are compressed into 2D matrices and horizontally concatenated. The 2D matrices of each branch are fed into the classification routing layer. After obtaining the results, they are input into the image reconstruction layer to reconstruct the images of each branch. The reconstruction process involves inputting the target vector into a fully connected network to obtain the output neuron. After five transposed convolutions, the image is continuously upsampled to obtain the final result, and the reconstruction loss is calculated. The calculation process of the 3D routing protocol includes the following steps:

[0086] Set the feature map received by the 3D routing layer to Φ l (w l w l c l n l w is the size of the feature map; c represents the depth of the 3D capsule; n represents Φ. l The number of capsules;

[0087] Set the size of the 3D convolution kernel to (3×3×c). l The step size in the horizontal and vertical directions is 1, and the step size in the depth direction is c.l The quantity is c l *n l ;

[0088] Φ l The number of capsules is increased after passing through a 3D convolutional layer, and the result is shown in equation (1):

[0089] U=(w l w l c l c l ×n l (1)

[0090] Each n in U l The set of matrices is considered as a capsule T, as shown in equation (2):

[0091] T = (w u w u n l (2)

[0092] The prior probability matrix is ​​set as shown in equation (3):

[0093] B = (w u ×w u ×n l c l ×n l (3)

[0094] The coupling coefficient is calculated using the Softmax function as shown in equation (4);

[0095]

[0096] The coupling coefficient matrix K obtained after transformation by B is shown in equation (5):

[0097] K = (w u w u n l c l ×n l (5)

[0098] Multiplying K and U by the dot product yields S, as shown in equation (6):

[0099] S = (T, c) l n l (6)

[0100] Arrange the results in S according to n l Dimensions added together;

[0101] Using the Squash function as shown in equation (7), the prior matrix B is activated to obtain V;

[0102]

[0103] After obtaining V, multiply it by U and add it to B to obtain a new matrix B as shown in equation (8);

[0104] B = V * U + B (8)

[0105] Repeat steps (3)-(8) three times to update V and make Φ l+1 =V.

[0106] (3) The 2D matrices of each branch are vertically concatenated and fed into the classification routing layer to obtain the final prediction result. The edge loss calculation function is used to predict the relative distance between the input sample and the output sample. The loss functions used in the network are the edge loss calculation function and the reconstruction loss. The edge loss calculation function is shown in Equation (9):

[0107] Margin_loss=Y*max(0,0.9-V)+λ(1-Y)*max(0,V-0.1) (9)

[0108] V is the magnitude of the predicted vector, Y is the one-hot encoding of the true label, and λ is used to control the effect of gradient backpropagation in the initial stage of training.

[0109] The reconstruction loss is obtained by comparing the reconstructed predicted image of each branch with the original image using the mean square error loss at the pixel level, as shown in Equation (10).

[0110]

[0111] In equation (10), x and y represent the tensors of the original image and the reconstructed image, respectively; x ij and y ij These represent the elements in the tensor; M and N represent the number of rows and columns of the tensor.

[0112] After obtaining the mean square error and edge loss, the overall loss of the multi-branch deep capsule network is calculated as shown in Equation (11). Since the mean square error is much larger than the edge loss, the weight value is used to balance the impact on the network.

[0113] Loss = Margin loss +αMSE_loss (11)

[0114] The classification routing layer calculation process includes the following steps:

[0115] The input data of the fully connected layer in the classification routing layer is set as a 2D capsule M = (a, c) of the image, and the data of the next layer is calculated in a fully connected manner as shown in Equation (12);

[0116] y = Wx + b 12)

[0117] Output multiple 2D capsules as shown in equation (13):

[0118] U = (n, a, v) (13)

[0119] In equation (12), n is the final number of categories;

[0120] The prior matrix is ​​set as shown in equation (14):

[0121] B = (n × v, a) (14)

[0122] B is fed into the Softmax function to obtain the coupling coefficient K;

[0123] The coefficient matrix K is transformed as shown in equation (15):

[0124] K = (n, a, v) (15)

[0125] Take the dot product of K and U to obtain S;

[0126] Add the data in S along dimension a to get (n, v);

[0127] V is obtained by activating B using the Squash function;

[0128] After obtaining V, multiply it by U and add it to B to obtain a new matrix as shown in equation (16), and update the coupling coefficients;

[0129] B = V * U + B (16)

[0130] Repeat steps (15)-(16) three times to obtain the final classification result V.

[0131] (4) Input the validation set into the trained six-branch deep capsule network to adjust the model performance and hyperparameters to obtain the trained network model;

[0132] (5) The partitioned test set is input into the trained six-branch deep capsule network model for testing, and the prediction output of the test set is obtained with an accuracy of 99.80%; the confusion matrix used to identify 30 objects is as follows: Figure 6 As shown.

[0133] The scope of protection claimed by this invention is not limited to the specific embodiments described above. Moreover, for those skilled in the art, this invention can have various modifications and alterations. Any modifications, improvements, and equivalent substitutions made within the concept and principles of this invention should be included within the scope of protection of this invention without any creative effort.

Claims

1. A piezoresistive glove intelligent recognition device based on a six-branch deep capsule network, characterized in that: The system includes a knitted glove (1), a flexible tactile sensor array (2), a resistance-to-voltage conversion module (3), a data acquisition card (4), and an artificial intelligence algorithm module (5). The flexible tactile sensor array (2) is distributed on the palm surface and the inner side of the finger bones of the knitted glove (1). The flexible tactile sensor array (2) outputs resistance information to the resistance-to-voltage conversion module (3). The resistance-to-voltage conversion module (3) outputs voltage information to the data acquisition card (4). The data acquisition card (4) outputs the collected voltage data information to the artificial intelligence algorithm module (5) of the host computer. The flexible tactile sensor array (2) includes 26 flexible tactile piezoresistive sensors, which are distributed as follows: a flexible tactile piezoresistive sensor is provided on the inner surface of the three phalanges of the index finger, middle finger, ring finger and little finger, a flexible tactile piezoresistive sensor is provided on the inner surface of the two phalanges of the thumb, and the remaining twelve flexible tactile piezoresistive sensors are evenly distributed on the palm. The artificial intelligence algorithm module (5) uses a six-branch deep capsule network for computation. The six-branch deep capsule network includes a base capsule layer, a 3D routing layer, a classification routing layer, and an image reconstruction layer. The algorithm flow includes the following steps: (1) Using Python's matplotlib2D plotting library, the data passed to the host computer is plotted on a hand contour image according to the position of 26 flexible tactile piezoresistive sensors on the piezoresistive glove and the force of the sensors to form a sub-image; the six touchable surface sub-images are used to reflect the body contact with the object, the six sub-images are stitched together to form a large image grasping dataset, and the grasping dataset is divided into training set, validation set and test set in a ratio of 7:2:1; (2) The grasping training set is fed into the basic capsule layer and the 3D routing layer. The basic capsule layer includes a regular convolutional layer and a convolutional capsule layer. The 3D routing layer applies a 3D routing protocol. The results of the basic capsule layer and the 3D routing layer are compressed into 2D matrices and horizontally stitched together. The 2D matrices of each branch are fed into the classification routing layer. After obtaining the results, they are input into the image reconstruction layer to reconstruct the images of each branch. The reconstruction process is to input the target vector into the fully connected network to obtain the output neuron. After five transposed convolutions, the image is continuously upsampled to obtain the final result. The reconstruction loss is then calculated. (3) The 2D matrices of each branch are vertically concatenated and fed into the classification routing layer to obtain the final prediction result. The edge loss calculation function is used to predict the relative distance between the input sample and the output sample. The loss functions used in the network are the edge loss calculation function and the reconstruction loss. The edge loss calculation function is shown in Equation (1): （1）； It is the magnitude of the predicted vector. It is a unique hot code for the real label. Used to control the effect of gradient backpropagation in the initial stage of training; The reconstruction loss is obtained by comparing the reconstructed predicted image of each branch with the original image using the mean square error loss at the pixel level, as shown in Equation (2); （2）； In equation (2), and These represent the tensors of the original image and the reconstructed image, respectively; and Each of these represents an element in the tensor; Represent the number of rows and columns of a tensor; After obtaining the mean square error and edge loss, the overall loss of the multi-branch deep capsule network is calculated as shown in Equation (3). Since the mean square error is much larger than the edge loss, the weight value is used to balance the impact on the network. （3）； (4) Input the validation set into the trained six-branch deep capsule network to adjust the model performance and hyperparameters to obtain the trained network model; (5) The divided test set is input into the trained six-branch deep capsule network model for testing, and the prediction output of the test set is obtained with an accuracy of 99.80%.

2. The intelligent identification device for piezoresistive gloves based on a six-branch deep capsule network according to claim 1, characterized in that: Each flexible tactile piezoresistive sensor is rectangular and includes an encapsulation film (6), an interdigital electrode layer (9), an isolation layer (8), and a flexible piezoresistive film (7). The encapsulation film (6) has two layers, namely the uppermost layer and the lowermost layer. Between the two encapsulation films (6), from top to bottom, are the interdigital electrode layer (9) and the flexible piezoresistive film (7). The isolation layer (8) is an adhesive strip, distributed on the outer side of three sides of the flexible piezoresistive film (7) and adhered to the upper and lower encapsulation films (6). The remaining side exposes the electrode connection point of the interdigital electrode layer (9).

3. The intelligent identification device for piezoresistive gloves based on a six-branch deep capsule network according to claim 2, characterized in that: The encapsulation film (6) is made of flexible polyimide film. The interdigitated electrode layer (9) is formed on the encapsulation layer using screen printing technology with an interdigitated electrode spacing of 100μm. The isolation layer (8) is made of double-sided 3M tape.

4. The intelligent identification device for piezoresistive gloves based on a six-branch deep capsule network according to claim 2, characterized in that: The preparation method of the flexible piezoresistive film (7) includes the following steps: (1) Disperse carbon fibers and carbon nanotubes in acetone at a mass ratio of 6:1 and stir for 4-6 hours. Then, ultrasonically disperse the mixed solution after stirring for 30-60 minutes. Heat in a fume hood until the acetone solution is completely evaporated to obtain a mixture of carbon fiber and carbon nanotube powders. (2) Add the carbon fiber and carbon nanotube powder mixture with a mass ratio of 3:20 to polydimethylsiloxane in 10 batches and stir for 3-6 hours. (3) Add the curing agent with a mass ratio of 1:10 to the carbon fiber / carbon nanotube / polydimethylsiloxane mixture and stir for 20 min-40 min. Pour it onto a cloth with a horizontal and vertical interwoven pattern and coat it into a 300 μm thick film using an adjustable film scraper. Then cure it on a heating table at 100°C for 30 min-50 min to form a flexible piezoresistive film with a cloth texture pattern.

5. The intelligent identification device for piezoresistive gloves based on a six-branch deep capsule network according to claim 1, characterized in that: In step (2), the calculation process of the 3D routing protocol includes the following steps: Set the feature map received by the 3D routing layer as follows: ; It is the size of the feature map; Indicates the depth of the 3D capsule; express The number of capsules; Set the size of the 3D convolution kernel to... The step size in the horizontal and vertical directions is 1, and the step size in the depth direction is... The quantity is ; The number of capsules is increased after passing through a 3D convolutional layer, and the result is shown in equation (4): （4）； Will each The set of matrices can be viewed as a capsule. As shown in equation (5): （5）； The prior probability matrix is ​​set as shown in equation (6): （6）； use The coupling coefficient is calculated using the function shown in equation (7); （7）； Get through The coupling coefficient matrix is ​​transformed into As shown in equation (8): （8）； Will and Dot product As shown in equation (9): （9）； Will The results in Dimensions added together; use The function is shown in equation (10), activating the prior matrix. get ; （10）； get Then, combine it with Dot product and union The summation yields a new B matrix as shown in equation (11); （11）； Repeat steps (7) to (11) three times for updates. and make .

6. The intelligent identification device for piezoresistive gloves based on a six-branch deep capsule network according to claim 1, characterized in that: In step (3), the classification routing layer calculation process includes the following steps: The input data for the fully connected layer in the classification routing layer is a 2D capsule of images. The next layer data is calculated in a fully connected manner as shown in equation (12); （12）； Output multiple 2D capsules as shown in equation (13): （13）； In equation (13), This represents the final number of categories; The prior matrix is ​​set as shown in equation (14): （14）； Will Send in The function obtains the coupling coefficient. ; coefficient matrix The transformation is shown in equation (15): （15）； Will and dot product ; Will The data in the middle are obtained by adding them along dimension a. ; use Function activation get ; get Then, combine it with Dot product and union The summation yields a new matrix as shown in equation (16), and the coupling coefficients are updated. (16)； Repeat steps (15)-(16) three times to obtain the final classification result. .

Images