A soft robot gripper state detection system and applications

Abstract

The application discloses a soft manipulator gripping state detection system, comprising: a feature extraction module, which is used for extracting the features of a touch signal and a motion state signal of the soft manipulator in a gripping state, and converting the touch signal features and the motion state features into corresponding pulse data; an association module, which is used for calculating the membership degree of the pulse data, and obtaining fuzzy pulse data, wherein the fuzzy pulse data is data obtained by performing a Cartesian product operation on the pulse data and an excitation level; and a classifier, which is used for obtaining the gripping state of the soft manipulator based on the fuzzy pulse data. Through the soft manipulator gripping state detection system disclosed by the application, the Bernoulli function is designed as the membership degree function, the characteristics of the pulse data are retained, and accurate detection of the gripping state is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of tactile signal analysis and processing based on soft robot hand, and more particularly, relates to a soft robot hand grasping state detection system, a training method, a detection method and a control method. BACKGROUND

[0002] Robot technology has penetrated into many fields of modern society, including manufacturing, medical and transportation. Soft robots use flexible materials such as silicone rubber, providing higher flexibility and adaptability than traditional metal robots, especially suitable for human-robot collaboration environments, improving safety. Among them, soft robot hand is a practical choice in this field because of its simple manufacturing and low cost. Accurate assessment of the grasping state of the soft robot hand is crucial for guiding subsequent operations and ensuring safe operation of the system. Existing researches are mostly focused on rigid structure fingers, which are either made of hard materials or only use soft materials on the contact surface, and there is little research on the grasping state of real soft robot hand. In addition, existing researches mostly ignore the complex nonlinear response and high noise level of soft robot hand, and mainly focus on slip detection, so they are not comprehensive enough. In view of the wide practical application of soft robot hand, it is necessary to conduct in-depth research on its grasping state.

[0003] Using multi-modal sensors for state perception is an important direction of current robot sensing research. For grasping state detection, most current researches rely on methods such as triboelectricity and tactile array, mainly focusing on the tactile signals generated by the contact surface position. Due to the material properties of soft robot hand, it often cannot completely fit the surface of the grasped object when wrapped, resulting in that the sensor signal is greatly affected by the installation position and the contact condition with the object. However, at the same time, its material properties also determine that the grasping state can be more comprehensively reflected by obtaining the position change and vibration of the soft robot hand itself. In addition, many researches integrate sensors inside the robot hand, resulting in highly customized algorithms that are difficult to apply to commercial soft robot hands, which is not conducive to the transplantation of algorithms. Therefore, designing a simple and easy-to-implement multi-modal sensing scheme and realizing effective analysis of the grasping state on this basis is still a key problem to be solved.

[0004] It is crucial to design an algorithm that can accurately perceive the soft robotic gripper's grasping state. Some studies have proposed a fuzzy neural network (FNN)-based approach, which provides flexibility, robustness, explainability, and high-dimensional data processing capabilities. However, FNN may require a large set of rules for high-dimensional time series data, increasing training time. Convolutional neural networks (CNNs) and other deep learning (DL) methods can automatically identify patterns through multi-level feature extraction without explicit rules, enhancing perception capabilities, but at a higher computational cost, especially for long time series or event-driven tasks. Spiking neural networks (SNNs), as the third generation of neural networks, mimic the operation of mammalian brains, providing sparse computation, event-driven mechanisms, and ultra-low power consumption. Integrating FNN, DL, and SNN into a new hybrid network can combine their strengths, such as the explainability of fuzzy systems, the powerful representation capabilities of deep learning, and the efficiency and low power consumption of SNNs. This integration is particularly complex because it requires optimizing all components simultaneously, managing the differences between rule-based and data-driven methods, and maintaining the explainability of fuzzy rules to meet the unique input requirements of SNNs. To this end, developing a complex joint training strategy that can handle complex information flows, maintain the transparency of fuzzy logic, and provide a more comprehensive and effective solution for soft robotic gripper state analysis is a significant challenge. SUMMARY

[0005] In view of the defects of the prior art and the need for improvement, the present application provides a soft robotic gripper grasping state detection system, a training method, a detection method and a control method, which designs a special membership function to improve the perception ability of the soft robotic gripper and further improve the control of the soft robotic gripper.

[0006] To achieve the above-mentioned purpose, the present application provides a soft robotic gripper grasping state detection system, comprising:

[0007] A feature extraction module is used to extract the features of the tactile signal and the motion state signal of the soft robotic gripper in the grasping state, and convert the tactile signal features and the motion state features into corresponding pulse data;

[0008] An association module is used to calculate the membership degree of the pulse data and obtain fuzzy pulse data, which is the data obtained by performing Cartesian product operation on the pulse data and the excitation level;

[0009] A classifier is used to obtain the grasping state of the soft robotic gripper based on the fuzzy pulse data.

[0010] Preferably, the feature extraction module comprises one or more cascaded feature extraction units, wherein each feature extraction unit comprises a convolution layer, a normalization layer, an activation layer and a down-sampling layer connected in cascade.

[0011] Preferably, the membership function used to calculate the membership of the pulse data is a Bernoulli function, and the expression is as follows:

[0012]

[0013] wherein p r,a represents the probability that the a-th feature is 0 under the r-th rule, Y s,a represents the a-th feature of the s-th input, M r,a represents the membership of the a-th feature under the r-th rule.

[0014] Preferably, the calculation formula of the excitation level is as follows:

[0015]

[0016] wherein d represents the dimension of the input feature, F r represents the excitation level of the r-th rule.

[0017] The application discloses a training method of a soft robot gripper state detection system, comprising:

[0018] The tactile signal and the motion state signal in the training set are input into the soft robot gripper state detection system to obtain the corresponding gripping state of the soft robot; the tactile signal features and the motion state features are converted into corresponding pulse data; when the data is forward propagated, the membership of the pulse data is calculated by designing a membership function; when the data is backward propagated, the membership function is replaced by gradient by designing a sigmoid curve function, and the soft robot gripper state detection system is trained.

[0019] wherein the upper limit and the lower limit of the sigmoid curve function correspond to a first membership value and a second membership value, the first membership value is the larger one of the probability that the input feature is 0 and the probability that the input feature is 1, and the second membership value is the smaller one of the two.

[0020] Preferably, the membership function is a Bernoulli function, and the expression is as follows:

[0021]

[0022] The sigmoid function is an arctangent function, and the expression is as follows:

[0023]

[0024] wherein p r,aP (Y = 0 | r) represents the probability of the a-th feature being 0 under the r-th rule, Y s,a M represents the a-th feature of the s-th input r,a P (Y = 0 | r) represents the membership degree of the a-th feature under the r-th rule, and a represents the steepness factor of the arctangent function.

[0025] Preferably, the training set is obtained by the following method:

[0026] The sensor data of the soft manipulator in the gripping state is collected to obtain an original data set, wherein the sensor data includes soft manipulator tactile signals, acceleration, gyroscope readings and Euler angle data, wherein the acceleration, gyroscope readings and Euler angle data correspond to the motion state of the soft manipulator.

[0027] The original data set is divided into a training set and a test set, and the corresponding label of the training set is manipulator gripping jitter, stable gripping and gripping sliding.

[0028] The application also discloses a soft manipulator gripping state detection method, comprising: inputting the tactile signal and the motion state signal to be detected into the soft manipulator gripping state detection system to obtain the gripping state of the soft manipulator, wherein the tactile signal is a signal collected by a flexible sensor on the soft manipulator, and the motion state signal is a signal collected by an IMU sensor, including: acceleration, gyroscope readings and Euler angle data.

[0029] The application also discloses a soft manipulator gripping state control method, comprising: detecting the current gripping state of the flexible manipulator by using the soft manipulator gripping state detection method.

[0030] Controlling the gripping state of the flexible manipulator based on the detection result.

[0031] The application also discloses an electronic device comprising: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to execute the above detection method, training method and control method.

[0032] Overall, the above technical solutions conceived by the application can achieve the following beneficial effects:

[0033] The membership function designed in the application ingeniously combines the advantages of discrete probability distribution and continuous function, which not only retains the sensitivity to pulse data, but also ensures the efficiency and stability of the algorithm in the training process. This innovation solves the challenges encountered by traditional membership functions in processing pulse data, and provides new possibilities for the application of fuzzy neural networks in pulse neural networks.

[0034] The soft robot gripper state detection system proposed by the present application combines the powerful feature extraction capability of convolutional neural network, the efficient processing of time information of pulse neural network and the explainable advantage of fuzzy system, forming an innovative hybrid intelligent system. Through the automatic learning of complex patterns of input data by convolutional layer, the conversion of these patterns into pulse data by pulse coding, and the simulation of the information processing mechanism of biological neural system, the time dynamics and biological rationality of the model are enhanced. At the same time, the fuzzy reasoning system introduced can effectively deal with the uncertainty of data, and improve the robustness and adaptability of the decision-making process.

[0035] The present application improves the accuracy of soft robot gripper state detection, and provides a solid foundation for realizing more stable and reliable grasping operation. By introducing a multi-modal perception system, i.e. multiple sensors, the gripper state of the soft robot under different working conditions can be monitored in real time, including gripping jitter, stable gripping and gripping sliding, etc. The adaptability to object grasping in complex environments is enhanced, and the safety and efficiency of operation are also greatly improved. In particular, the system can quickly identify changes in the gripping state in minor changes, and timely adjust the control strategy to avoid operation failure caused by unstable gripping or sliding. In addition, it also supports precise state monitoring of soft robots with different degrees of freedom, ensuring that each action can achieve the best effect, thereby promoting the wide application of soft robot technology in industrial automation, medical assistance and other precision operation fields. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 A multi-degree-of-freedom soft robot hardware module schematic diagram containing multi-modal sensing according to an embodiment of the present application.

[0037] Figure 2 A soft robot gripper state detection method schematic diagram according to an embodiment of the present application.

[0038] Figure 3 A Bernoulli membership degree and arctangent function schematic diagram designed according to an embodiment of the present application.

[0039] Figure 4 A soft robot gripper state detection system structure schematic diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0041] In the present application, the terms "first", "second", and the like (if any) in the present application and the accompanying drawings are used to distinguish similar objects, and are not necessarily used to describe a particular order or sequence.

[0042] Figure 1 A schematic diagram of a multi-degree-of-freedom gas-driven soft robot hardware module containing multi-mode sensing according to an embodiment of the present application. As shown Figure 1 The air compressor is connected to the air source driver through the air pipe, and the driver controls the soft robot to bend and other actions by setting the output air pressure. The IMU (Inertial Measurement Unit) is built into the fingers of the soft robot, which is used to measure the Euler angle, angular acceleration and gyroscope reading data of the soft robot in three axes, to judge the motion state of the soft robot according to the above data, and to transmit the measured data to the host computer through serial connection; The flexible sensor is connected in series with a resistor, such as a 3.3kΩ resistor, to form a voltage division circuit, which is used to detect the tactile information of the soft robot, and collect and upload the tactile information in the form of a voltage signal to the host computer through a data acquisition card. In an embodiment of the present application, the sensors include: flexible sensors and IMU sensors, and the tactile information and motion state combination reflects the gripping state of the robot, such as: gripping jitter, stable gripping, and gripping sliding, etc. Each IMU sensor includes 3 data channels to transmit the measured Euler angle, angular acceleration and gyroscope reading, respectively, and each soft finger includes at least one IMU sensor and 3 flexible sensors. Figure 1 A schematic diagram of the layout of the sensors on the fingers and the collection of tactile information and motion information is shown in the figure. In the present application, a three-finger soft robot is used as an example to illustrate the gripping state detection method, but it is not limited to this. Other multi-finger soft robots, and soft robots including but not limited to gas-driven, fluid-driven, and wire-driven driving methods are also applicable.

[0043] Figure 2 A schematic diagram of a soft robot gripping state detection method according to an embodiment of the present application. First, the flexible sensors and IMU sensors installed on the fingers of the soft robot collect tactile information and acceleration, gyroscope reading and Euler angle data as raw data, respectively. The raw data is time-aligned through a synchronization module to ensure that the data from different sensors is consistent at the same time point. The standardization module pre-processes the raw data to meet the requirements of the robot gripping state detection system, corresponding to Figure 2The convolutional pulse blur network DSHTFN in the figure. The preprocessed data, acceleration, gyroscope readings, Euler angles and voltage signals are input into the trained soft manipulator grip state detection system. The soft manipulator grip state detection system identifies the grip state of the soft manipulator according to the data collected by the current sensor: such as grip jitter, stable grip and grip slip. Here, grip jitter refers to the shaking of the manipulator when grasping due to external force interference, but the grasped object is still firm and will not fall off; stable grip refers to the smooth and undisturbed grasping of the object; grip slip refers to the sliding motion of the object relative to the manipulator under the influence of external force.

[0044] Embodiment 1:

[0045] The application discloses a soft manipulator grip state detection system, comprising: a feature extraction module for extracting features of tactile signals and motion state signals of a soft manipulator in a grip state, and converting the tactile signal features and motion state features into corresponding pulse data; an association module for calculating the membership of the pulse data and obtaining fuzzy pulse data, which is data obtained by performing Cartesian product operation on the pulse data and the excitation level; and a classifier for obtaining the grip state of the soft manipulator based on the fuzzy pulse data.

[0046] Specifically, the feature extraction module includes one or more cascaded feature extraction units, wherein the feature extraction unit integrates the structure of a convolutional neural network and a pulse neural network, such as a convolutional layer, a normalization layer, an activation layer and a down-sampling layer. In an optional implementation, the convolutional layer is used to extract features of tactile signals and motion state signals of a soft manipulator in a grip state, the activation layer converts continuous signals into pulse data, and the down-sampling layer further performs down-sampling processing on the pulse data, and the down-sampling layer can be a max-pooling layer. The pulse data output by the feature extraction module is input into the association module, the membership of the pulse data is calculated, the excitation level is obtained through mean operation, and the fuzzy pulse data is obtained by performing Cartesian product operation on the excitation level and the pulse data. In an optional implementation, the association module further includes normalization processing of the excitation level, further standardization processing, and removal of negative values through an activation function. The association module inputs the fuzzy pulse data into the classifier, wherein the classifier is composed of a linear layer mapping, a softmax operation and an argmax operation, and the classifier outputs a classification result according to the fuzzy pulse data, i.e. the grip state of the soft manipulator. Figure 4 As shown in the figure, the feature extraction module includes two cascaded feature extraction units, wherein each feature extraction unit includes cascaded convolutional layers, normalization layers, activation layers and down-sampling layers.

[0047] It should be noted that the membership degree of the pulse data is calculated in the association module, and the Bernoulli function is used. Since the Bernoulli function is the most basic probability model for describing binary random events, its core feature is that there are only two possible results (0 or 1), and the pulse data processed in the application is a discrete value, that is, 0 and 1, which is highly consistent with the Bernoulli function. Therefore, the Bernoulli function is designed as the membership degree function, which can fully retain the characteristics of the pulse data.

[0048] In order to further understand the process of obtaining the gripping state of the soft robot gripper gripping state detection system, the training process of the detection system will be described through the following embodiments, and then the working process will be understood. The training process refers to training the detection system through the training set, and adjusting the parameters to the best value. The working process of the detection system refers to inputting the to-be-detected data into the trained detection system, and predicting the classification result. In the application, the classification result is the gripping state of the soft robot gripper. The feature extraction module in the detection system is two cascaded feature extraction units. Those skilled in the art should understand that this is only one embodiment and is not limited. The feature extraction unit corresponds to block I and block II in Figure 4 , and block III corresponds to the association module and the classifier.

[0049] Specifically, referring to Figure 4 , in the optional embodiment of the application, an offline case is considered, that is, the data of the flexible sensor and the IMU sensor multi-channel and the corresponding gripping state label are recorded as , where n s represents the number of data set samples, X s is the sample data, and l s is the label. Each dimension in corresponds to the value of each flexible sensor or IMU sensor signal, and d represents the number of dimensions. For a data set with N p categories, the label is represented as l s ∈{1,2,...,N p}, and here there are three categories. The training process of the soft robot gripper gripping state detection system includes the following steps:

[0050] S1: Collect experimental data. For example, fix the finger-type soft robotic hand and the three-finger soft robotic hand to ensure that the position and posture of the robotic hand remain stable during the grasping process. Then, collect sensor data, including flexible sensors and IMU sensors, for these two types of robotic hands in three different grasping states when performing the grasping action. In each grasping state, record data from multiple sensors mounted on the soft robotic hand. The sensor data of the finger-type robotic hand is referred to as raw data Q1, and the sensor data of the three-finger robotic hand is referred to as raw data Q2. The above selection of finger-type and three-finger robotic hands is only one embodiment of the present invention; other multi-finger robotic hands are also applicable for experimental data.

[0051] S2: Perform Z-score standardization on the dataset. Z-score standardization is a method of standardizing data by subtracting the mean and then dividing by the standard deviation. This process converts the data into dimensionless values, where the mean becomes 0 and the standard deviation becomes 1. During standardization, the data is split into training and test sets. For standardization of the test set, the mean and standard deviation from the training set are used. The standardized dataset is represented as follows: The training set is represented as The test set is represented as Where n train and n test These represent the number of samples in the training set and the test set, respectively.

[0052] S3: Let the output of the l-th layer be represented as Y. (l) Use convolutional layers (such as...) Figure 4 The convolutional layer in block I extracts features from the original data. Let the input data be represented as X, and the convolutional kernel as W. Then we get:

[0053]

[0054] Here, (i,j) represents the position in the generated feature map, and (m,n) is the size of the convolution kernel, with * indicating the convolution operation. In the above formula, Y (1) (i,j) represents the value of the output feature map of the first convolutional layer Conv1 at position (i,j), obtained by performing a convolution operation between the input data X and the convolution kernel W at this position. The convolution operation is calculated by sliding the convolution kernel at each position (i,j) of the input data, multiplying the input data within the kernel's coverage area by the corresponding kernel weights, and then summing the results. This process is repeated for each position on the feature map, thereby extracting local features from the entire input data. Next, the data will be processed through an adaptive normalization layer SN to obtain standardized values:

[0055]

[0056] where e is used to avoid division by zero error and is close to 0. Ω = {IN, LN, BN} represents the space of three standardization algorithms: BN represents batch normalization, IN represents instance normalization and LN represents layer normalization. IN ,σ IN are the mean and standard deviation of the sample s in individual channel respectively. LN LN are the mean and standard deviation of the whole sample s. BN BN are the mean vector and standard deviation vector of the current batch features. Here k e Ω. Similarly, ω k ' is also determined in the same way, parameters λ k , λ k ', γ and β are updated in the training process.

[0057] Further, the results of Y (2) are input to an IF (Integrate-and-Fire) spiking neuron layer. In this layer, the membrane potential V[t] is updated according to the membrane potential V[t-1] of the previous time step and the current input Y (2) [t]:

[0058] V[t] = V[t-1] + Y (2) [t]

[0059] When the membrane potential exceeds the threshold voltage V threshold , the neuron fires a spike and resets the membrane potential. The firing (reset) equation is defined by the Heaviside step function Θ(x), S[t] = Θ(V[t]-V threshold ) where The reset equation of the membrane potential is: V[t] = V[t]-V threshold S[t], during backpropagation, the gradient of the alternative needs to be used to solve the problem of continuous derivable function in backpropagation, such as arctangent Atan function or sigmod function, but not limited to this. Finally, the output after the above three layers of processing is recorded as Y (3) .

[0060] Further, maximum pooling is applied to Y (3) , which represents the information of the region by selecting the maximum value from the local region to achieve down-sampling, and the operation process is as follows:

[0061]

[0062] Specifically, for each new output position Y (4) (i,j), we select the maximum value from Y (3) ​​The maximum value in a 2*2 local region is selected as the value of the position, so as to reduce the data amount while retaining the most important feature information.

[0063] S4: On the basis of S3, further convolution, self-adaptive standardization, pulse activation and maximum pooling processing (as shown in block II of FIG. 2) are performed. Figure 4 In an embodiment of the present application, the convolution layer Conv2 is applied to Y (4) to obtain Y (5) . In order to reduce the number of optimization parameters in the network, Y (5) is applied to SN to obtain Y (6) . Then, the pulse neuron activation layer SNA is used to activate Y (6) to obtain Y (7) . Y (7) is subjected to a maximum pooling operation by the maximum pooling layer MP to obtain Y (8) . The data in Y (8) is flattened into a one-dimensional vector to generate Y (9) , and then a linear layer is connected for linear expansion. For example, assuming that the dimension of the input vector is d i , the dimension of the output vector after linear expansion is d o , the weight matrix and the bias term are

[0064] Y (10) = Y (9) W (9) +b (9)

[0065] Y (10) is subjected to pulse activation by the pulse neuron activation layer SNA to obtain Y (11) , which is used as the input of the fuzzy system (as shown in block III of FIG. 2). Figure 4

[0066] S5: A membership function is designed to process the pulse data Y (11) obtained in step S4. In an embodiment of the present application, the forward propagation function of the membership function is a Bernoulli function, and the backward propagation is a sigmoid function. The Bernoulli function is used to adapt to the binary characteristics of the pulse data, and the sigmoid function makes the function continuous and derivable in the backward propagation. The membership function is not limited to the above two functions, and any function that meets the requirements of adapting to the binary characteristics of the pulse data in the forward propagation and being continuous and derivable in the backward propagation belongs to the protection idea of the present application.

[0067] ​It should be noted that the upper and lower bounds of the S-shaped curve function here correspond to the first membership value and the second membership value, the first membership value is the larger value of the probability of the input feature being 0 and the probability of the input feature being 1, and the second membership value is the smaller value.

[0068] Specifically, since a TSK (Takagi-Sugeno-Kang) fuzzy system with d inputs and a single output, it contains R rules, as follows:

[0069]

[0070] where M r,a is the membership function of the a-th input feature in the r-th rule, Y r (12) (Y (11) ) is the conclusion of the r-th rule. The parameters w r,0 and w r,a are optimized, where r = 1, 2,..., R, and a = 1, 2,..., d. In order to adapt to the input characteristics of pulse data, the definition of pulse neuron activation is referred to, and in an embodiment of the present application, the membership function is designed to be a forward propagation Bernoulli function, as shown in Figure 3 Since each dimension of the data is 0 or 1, it is assumed that the result of each dimension follows a Bernoulli distribution Y s,a (11) ~ B (1, 1-p a ), that is, the probability of the input feature being 0 is p a , and the probability of the input feature being 1 is 1-p a , and the expression for the forward propagation of the membership function is defined as:

[0071]

[0072] where p r,a represents the probability of the a-th feature being 0 under the r-th rule, Y s,a represents the a-th feature of the s-th input, and M r,a represents the membership of the a-th feature under the r-th rule. It should be noted that the definition of the membership function is in discrete form and cannot be used for gradient operation in backpropagation. In the backpropagation process, the substitute function of the membership function is set to Φ(Y s,a (11) ), and the specific design method is shown in Figure 3 When the probability of the input feature being 0 is p a , and the probability of the input feature being 1 is 1-p a , as long as the upper and lower bounds of the membership function in the backpropagation process are p a and 1-p a , (pa and 1-p a The larger number is the upper bound, and the smaller number is the lower bound. A suitable steepness can be designed, where the appropriate steepness is determined by the magnitude of the derivative during the inverse differentiation process, and is not unique. For example, in an embodiment of this invention, when the arctangent function is chosen for the S-curve function, α represents the kurtosis factor of the arctangent function. The value of the parameter α affects the steepness of the curve, and its range is between 1 and 20. The expression for the arctangent function is as follows:

[0073]

[0074] This expression translates the original arctangent function by shifting and phase adjustment, with the upper bound set to p. r,a and 1-p r,a The larger value in the middle is the smaller value below. The steepness is limited by the value of α to ensure that the backpropagation gradient does not explode. The central symmetric point is shifted from the origin to (0.5, 0.5) to realize the membership fitting function of the backpropagation process, which solves the problem that the original discrete signal cannot be solved for gradient.

[0075] Furthermore, the excitation level F of the above membership function calculation rule... r (Y s (11) The formula for calculating ) is:

[0076]

[0077] After calculating the excitation levels of all rules, normalization is performed (e.g., ...). Figure 4 (As shown in Block III):

[0078]

[0079] To avoid the gradient vanishing problem, we further use the adaptive standard layer SN processing to obtain:

[0080]

[0081] Furthermore, to avoid the negative impact of introducing negative activations on the interpretability of the results, and to ensure that outliers do not affect the results, a ReLU layer was added at the end to filter out negative values:

[0082]

[0083] Those skilled in the art will understand that other activation functions can also be used here. For input Y... (11) Applying a Cartesian product to its activation level yields Y. (12) The output of the blurred layer is obtained using a linear layer.

[0084] S6: On the basis of S5, Y (13) The softmax function is applied to obtain the probability of each sample belonging to each class:

[0085]

[0086] Y (14) The argmax function is applied to obtain the final classification result, as follows:

[0087]

[0088] Here, Y s (15) represents the final classification result of the s-th sample, and the argmax function returns the class index l that makes the maximum value. In other words, for each sample, we select the class with the highest probability as the predicted class of the sample. This step converts the probability distribution into a specific class label, completing the entire process from input data to class prediction.

[0089] After training the above detection system through the above steps, a trained soft robot gripper state detection system is obtained. The working process of the detection system is to input the to-be-detected tactile signal and motion state signal into the trained detection system, and then perform data processing according to the above steps to obtain the gripping state of the soft robot.

[0090] The above embodiment illustrates the training process of the soft robot gripper state detection system with 2 feature extraction units as an example, and the output of the l-th layer in the cascaded feature extraction units is represented as Y (l) , and l represents the number of layers in this embodiment. When the number of feature extraction units is 1 or other numbers, the number of layers changes with the number of feature extraction units.

[0091] Example 2:

[0092] The application further discloses a training method of the soft manipulator gripping state detection system, which comprises the following steps: inputting the touch signal and the motion state signal in a training set into the soft manipulator gripping state detection system in the embodiment 1, so as to obtain the gripping state of the soft manipulator; converting the touch signal feature and the motion state feature into corresponding pulse data; calculating the membership degree of the pulse data by designing the membership function when the data is forward propagated; training the soft manipulator gripping state detection system by designing the S-shaped curve function to perform gradient substitution on the membership function when the data is backward propagated; and wherein the upper limit and the lower limit of the S-shaped curve function correspond to the first membership value and the second membership value, the first membership value is the larger one of the probability that the input feature is 0 and the probability that the input feature is 1, and the second membership value is the smaller one of the two. For reference of the meanings of the first membership value and the second membership value in the embodiment 1, the meanings will not be repeated here. For the training process and the expression of the membership function, reference can be made to the training process of the soft manipulator gripping state detection system in the embodiment 1, and the training process will not be described in detail again.

[0093] Embodiment 3

[0094] The application further discloses a soft manipulator gripping state detection method, which comprises the following steps: inputting the touch signal and the motion state signal to be detected into the soft manipulator gripping state detection system, so as to obtain the gripping state of the soft manipulator, wherein the touch signal is a signal collected by a flexible sensor on the soft manipulator, and the motion state signal is a signal collected by an IMU sensor, comprising acceleration, gyroscope reading and Euler angle data.

[0095] Embodiment 4

[0096] The application further discloses a soft manipulator gripping state control method, which comprises the following steps: detecting the current gripping state of the flexible manipulator by using the soft manipulator gripping state detection method in the embodiment 3; and controlling the gripping state of the flexible manipulator based on the detection result.

[0097] Embodiment 5

[0098] The application further discloses an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor executes the method in the embodiments 2-4.

[0099] Further, in the experiment, the collected sensor data set is divided into 80% training set and 20% test set, the sensor dimension of the three-fingered manipulator is set to 24, the sensor dimension of the three-fingered manipulator is set to 36, and the number of sample categories is 3. In the designed experiment, the results are shown in the following table, when the finger is 95.66%, and when the three fingers are 96.05%. Compared with the traditional support vector machine algorithm, the detection system or network has a significant advantage.

[0100] Using algorithms Our network Support vector machines To the finger 95.66％ 77.33％ Three fingers 96.05％ 85.65％

[0101] In summary, through the above technical solutions conceived by the present application, the following beneficial effects can be achieved:

[0102] The membership function designed in the present application ingeniously combines the advantages of discrete probability distribution and continuous function, preserving both sensitivity to pulse data and ensuring high efficiency and stability during the training process. This innovative solution addresses the challenges encountered by traditional membership functions when dealing with pulse data, providing new possibilities for the application of fuzzy neural networks in spiking neural networks.

[0103] The present application proposes a soft gripper state detection system that integrates the powerful feature extraction capability of convolutional neural networks, the efficient processing of temporal information by spiking neural networks, and the interpretability advantage of fuzzy systems, forming an innovative hybrid intelligent system. Through convolutional layers, the system automatically learns the complex patterns of input data, converts these patterns into pulse data using pulse coding, and then simulates the information processing mechanism of biological neural systems, thereby enhancing the temporal dynamics and biological rationality of the model. At the same time, the introduced fuzzy reasoning system can effectively deal with data uncertainty, improving the robustness and adaptability of the decision-making process.

[0104] The present application improves the accuracy of soft gripper state detection, providing a solid foundation for achieving more stable and reliable grasping operations. By introducing a multi-modal perception system, i.e., multiple sensors, the system can monitor the grasping state of the soft gripper in different working conditions in real time, including grasping jitter, grasping stability, and grasping sliding. This enhances the adaptability of object grasping in complex environments and significantly improves the safety and efficiency of operations. In particular, the system can quickly identify changes in grasping state in minor changes, adjust control strategies in a timely manner, and avoid operation failures caused by unstable or slipping grasping. In addition, it supports precise state monitoring of soft grippers with different degrees of freedom, ensuring that each motion achieves optimal results, thereby promoting the widespread application of soft robot technology in industrial automation, medical assistance, and other precision operation fields.

[0105] Those skilled in the art will readily understand that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A soft robot gripper grasp state detection system, comprising: The method comprises the following steps: a feature extraction module for extracting features of tactile signals and motion state signals of the soft manipulator in a gripping state, and converting the tactile signal features and the motion state features into corresponding pulse data, wherein the feature extraction module comprises one or more cascaded feature extraction units, and each feature extraction unit comprises a convolution layer, a normalization layer, an activation layer, and a down-sampling layer connected in cascade; an association module for calculating the membership of the pulse data and obtaining fuzzy pulse data, wherein the fuzzy pulse data is obtained by performing a Cartesian product operation on the pulse data and the excitation level data, and the membership function used to calculate the membership of the pulse data is a Bernoulli function, and the expression is as follows: in Indicates the first Rule # The probability that a feature dimension is 0. Indicates the first The first input Dimensional features, Indicates the first Rule # Membership degree of a feature; The calculation formula of the excitation level is as follows: where d denotes the dimensionality of the input features, denotes the activation level of the th rule. a classifier for obtaining the gripping state of the soft manipulator based on the fuzzy pulse data.

2. A training method of a soft robot hand grasp state detection system, characterized by, The method comprises the following steps: inputting the tactile signals and the motion state signals in the training set into the soft manipulator gripping state detection system to obtain the corresponding gripping state of the soft manipulator; converting the tactile signal features and the motion state features into corresponding pulse data; when the data is forward propagated, the membership of the pulse data is calculated by designing a membership function; when the data is backward propagated, the membership function is replaced by a gradient of an S-shaped curve function, and the soft manipulator gripping state detection system is trained, wherein the membership function is a Bernoulli function, and the expression is as follows: The S-shaped function is an inverse tangent function, and the expression is as follows: wherein represents the dimensional feature of the dimensional feature under the dimensional feature of the input, dimensional feature of the dimensional feature under the dimensional feature of the dimensional feature, and α represents the steepness factor of the arctangent function; wherein the upper limit and the lower limit of the S-shaped curve function correspond to a first membership value and a second membership value, the first membership value is the larger one of the probability that the input feature is 0 and the probability that the input feature is 1, and the second membership value is the smaller one of the two probabilities.

3. The training method of a soft robot gripper grasping state detection system according to claim 2, characterized in that, The training set is obtained by the following method: collecting sensor data of the soft manipulator in the gripping state to obtain an original data set, wherein the sensor data includes tactile signals of the soft manipulator, acceleration, gyroscope readings, and Euler angle data, wherein the acceleration, gyroscope readings, and Euler angle data correspond to the motion state of the soft manipulator; the original data set is divided into a training set and a test set, and the label corresponding to the training set is a manipulator gripping shaking, stable gripping, and gripping sliding.

4. A method for detecting a gripping state of a soft robot hand, characterized by, The method comprises the following steps: inputting the tactile signals and the motion state signals to be detected into the soft manipulator gripping state detection system of claim 1 to obtain the gripping state of the soft manipulator, wherein the tactile signals are signals collected by a flexible sensor on the soft manipulator, and the motion state signals are signals collected by an IMU sensor, including acceleration, gyroscope readings, and Euler angle data.

5. A method for controlling a gripping state of a soft robot hand, characterized by, The method comprises the following steps: detecting the current gripping state of the flexible manipulator by using the soft manipulator gripping state detection method of claim 4; controlling the gripping state of the flexible manipulator based on the detection result.

6. An electronic device, comprising: The method comprises the following steps: a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to execute the method of any one of claims 2-5.

Images