A method for detecting the grasping state of a flexible manipulator in different postures and motion states

Abstract

The present invention discloses a method for detecting the grasping state of a flexible manipulator in different postures and motion states, and belongs to the field of tactile signal analysis and processing based on flexible manipulators. The present invention conducts grasping state classification and recognition experiments in five static scenarios: horizontally upward, horizontally downward, horizontally left, horizontally right, and horizontally; and in six dynamic scenarios: translational reciprocating motion in three axial directions, front-back, left-right, and up-down, and semicircular arc reciprocating motion in three orthogonal planes in space. The present invention greatly broadens the experimental scenarios, making the experimental results highly generalizable and universal. The present invention proposes a self-adaptive standardized high-dimensional fuzzy system, and for the first time uses a fuzzy system to solve the problem of grasping state detection of a flexible manipulator. Compared with the grasping state detection results using other machine learning methods, the classification accuracy of the results is higher.

Description

Technical Field

[0001] The present invention belongs to the field of tactile signal analysis and processing based on flexible manipulators, and more specifically, relates to a method for detecting the grasping state of a flexible manipulator in different postures and motion states. Background Art

[0002] With the advancement of robotics, traditional manipulators based on rigid joints and linkages are no longer sufficient in many grasping scenarios. Consequently, many researchers have studied the design and application of flexible manipulators. Evaluating the gripping state of a manipulator during the grasping process is a fundamental and critical component of robotics research. On the one hand, analyzing the gripping state helps ensure system safety, particularly by preventing the grasped object from falling due to external forces, which could lead to serious safety accidents. On the other hand, monitoring the gripping state of the manipulator provides the necessary signals for controlling the manipulator's next actions. Currently, many researchers have studied the gripping state of manipulators. Most of these researchers narrowly define gripping state analysis as detecting the slipping state of the grasped object, and extensive research has been conducted on slip detection. A few researchers have also categorized gripping states into several categories, including slipping, impending slipping, and stable states.

[0003] Compared to detecting the grasping state of rigid manipulators, detecting the grasping state of flexible manipulators is more difficult and complex. For one thing, while many researchers have been researching flexible sensors, the performance of these sensors remains inferior to many tactile sensors that can only be used with rigid manipulators. Researchers in this field remain puzzled as to how to integrate the latest research findings in flexible sensing with flexible manipulators. Furthermore, flexible manipulators have a higher degree of freedom, making their modeling more challenging than that of rigid manipulators. While rigid manipulators can be easily analyzed through analytical models for their kinematic and inverse kinematic equations, this is significantly more difficult for flexible manipulators. Furthermore, grasping with flexible manipulators differs from that with rigid manipulators in both hardware systems and signal formats, making methods for analyzing the grasping state of flexible manipulators more complex.

[0004] To date, most research has focused on the design of flexible manipulators. A small number of studies have incorporated tactile sensors into flexible manipulators, and even fewer have investigated the use of sensor information for grasping control. Similarly, research on how to use tactile information to analyze the grasping state of flexible manipulators is relatively limited. However, due to its wide range of potential applications, studying the grasping state of flexible manipulators is of great significance.

[0005] There are many machine learning models available today, and fuzzy systems are among the most promising. Based on a comprehensive theoretical foundation and system, they employ rule-based models to simulate human decision-making. During decision-making, they display the activation level of each rule to determine the corresponding output, offering a certain degree of interpretability. Among these, the TSK fuzzy system is highly flexible and widely popular. Recent research has proposed a TSK fuzzy system based on layer normalization, but its performance still has room for improvement. Fuzzy systems, with their advantages such as strong interpretability, have never been used in the field of grasp state analysis, demonstrating their significant application value. Summary of the Invention

[0006] In response to the above defects or improvement needs of the prior art, the present invention provides a method for detecting the grasping state of a flexible robotic arm in different postures and motion states, the purpose of which is to improve the tactile perception ability of the flexible robotic arm and make the grasping of the flexible hand more stable and reliable.

[0007] To achieve the above objectives, according to one aspect of the present invention, a method for detecting the gripping state of a flexible manipulator in different postures and motion states is provided, comprising:

[0008] S1. Collect tactile sensor data under different grasping states of the flexible manipulator, with the manipulator in horizontal, slightly above, slightly below, slightly to the left, and slightly to the right, to obtain static experimental data.

[0009] S2. Collect tactile sensor data from different grasping states while the flexible manipulator moves in three mutually perpendicular axial directions and back-and-forth semicircular arcs in three orthogonal planes, to obtain dynamic experimental data; combine the static and dynamic experimental data as raw data;

[0010] S3. Dividing the original data into a training set and a test set;

[0011] S4. Establishing a self-adaptive standardized high-dimensional fuzzy system; the self-adaptive standardized high-dimensional fuzzy system is provided with an SN normalization layer and a ReLU linear activation layer based on HTSK; the SN normalization layer is used to map the normalized activation of HTSK to obtain the new rule activation; the ReLU linear activation layer is used to further process the new rule activation;

[0012] S5. Using the training set and the test set to train the adaptive standardized high-dimensional fuzzy system;

[0013] S6. Input the tactile sensing signal of the flexible manipulator in the current grasping state into the trained high-dimensional fuzzy system to obtain the grasping state detection result in real time.

[0014] Furthermore, the grasping states include inflation bending of the flexible hand, forced shaking, stable grasping, and forced sliding of the object.

[0015] Furthermore, the method also includes performing Z-Score normalization on the training set, and normalizing the test set using the expectation and standard deviation of the training set.

[0016] Furthermore, the flexible manipulator is pneumatically driven, and pressurizes the air cavity inside the manipulator to expand and bend it, so as to envelop or pinch the object, thereby achieving grasping of the object.

[0017] Furthermore, the flexible manipulator adopts a piezoresistive tactile sensing array as a tactile sensor.

[0018] Furthermore, the method sets the value range of the fuzzy rule number to be 300-1800.

[0019] Furthermore, the tactile sensor of the flexible manipulator is embedded in the flexible manipulator.

[0020] The present invention also provides a flexible manipulator grasping state detection system in different postures and motion states, comprising:

[0021] The static experimental data acquisition module is used to collect tactile sensor data under different grasping states when the flexible manipulator is in the horizontal, horizontally upward, horizontally downward, horizontally to the left, and horizontally to the right postures, and obtain static experimental data;

[0022] The dynamic experimental data acquisition module is used to collect tactile sensor data under different grasping states when the flexible manipulator moves back and forth in three mutually perpendicular axial directions in space, and moves back and forth in a semicircular arc in three orthogonal planes, to obtain dynamic experimental data;

[0023] A training data construction module is used to combine the static experimental data and the dynamic experimental data as raw data; and divide the raw data into a training set and a test set;

[0024] A model building module is used to establish a self-adaptive standardized high-dimensional fuzzy system, and train the self-adaptive standardized high-dimensional fuzzy system using a training set and a test set; the self-adaptive standardized high-dimensional fuzzy system is provided with an SN normalization layer and a ReLU linear activation layer based on HTSK; the SN normalization layer is used to map the normalized activation of HTSK to obtain the new rule activation; the ReLU linear activation layer is used to further process the new rule activation;

[0025] The real-time detection module is used to input the tactile sensing signal of the flexible manipulator in the current grasping state into the trained high-dimensional fuzzy system to obtain the grasping state detection result in real time.

[0026] The present invention also provides a method for controlling the gripping state of a flexible manipulator in different postures and motion states, comprising: detecting the current gripping state of the flexible manipulator using the above-mentioned method for detecting the gripping state of a flexible manipulator in different postures and motion states;

[0027] The grasping state of the flexible manipulator is controlled based on the detection results.

[0028] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art.

[0029] The present invention conducted grasping state classification and recognition experiments in five static scenarios, namely, horizontally upward, horizontally downward, horizontally left, horizontally right, and horizontally; as well as six dynamic scenarios, namely, translational reciprocating motion in three axial directions of front-to-back, left-to-right, and up-to-down in space, and semicircular reciprocating motion in three orthogonal planes. This greatly broadened the experimental scenarios and made the experimental results highly generalizable and applicable.

[0030] This paper proposes a self-adaptive, standardized high-dimensional fuzzy system, and for the first time uses a fuzzy system to solve the problem of grasping state detection in flexible manipulators. This fuzzy system offers advantages such as high input dimensionality, precise classification results, strong interpretability, and high generalizability. Compared to grasping state detection results using other machine learning methods, its results are more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the flexible manipulator with flexible tactile sensing used in the present invention.

[0032] Figure 2 Four grasping states for the flexible manipulator used in the present invention.

[0033] Figure 3 Schematic diagram of the dynamic and static experimental scenarios proposed in this invention.

[0034] Figure 4 This is a schematic diagram of the high-dimensional fuzzy system structure based on adaptive standardization proposed in the present invention. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0036] The present invention uses a flexible manipulator with tactile sensing and designs various experimental scenarios such as static and dynamic. Tactile sensor data is collected in these experimental scenarios, and a method based on a self-adaptive standardized high-dimensional fuzzy system is proposed to identify and classify the grasping state of the flexible manipulator.

[0037] As Figure 1 Take, for example, the flexible manipulator with flexible tactile sensors shown in the figure. Made of high-strength silicone rubber, it includes a chamber cover, a chamber body, fiberglass, and a restraining layer. When the manipulator is inflated and pressurized by an external air source, the manipulator's air chambers expand, causing the exterior of the manipulator to bend inward. The restraining layer prevents bending within the manipulator, allowing it to perform operations such as grasping and pinching objects. The tactile sensors in the flexible manipulator are embedded within the manipulator itself, rather than simply attached to the manipulator's surface using adhesive or other means. The flexible manipulator and sensors form an integrated grasping system. The tactile sensors are made of nanocarbon black, sodium chloride, and thermoplastic polyurethane elastomer rubber. After mixing and heating the powders of these three, the sodium chloride particles dissolve, resulting in a porous nanocarbon film. This film is coated with double-sided tape and conductive fabric for easy wiring. The flexible manipulator uses a tactile sensor array based on the piezoresistive principle as its tactile sensor, providing experimental data. When the manipulator expands, the film bends, causing the internal pore gaps to increase, thereby increasing resistance. Conversely, when the manipulator deflates or its inner wall is squeezed, the film's pore gaps decrease, increasing the conductive path and reducing resistance. This change in resistance can be used to sense the manipulator's bending and gripping, and its changes are sensitive to the manipulator's deformation. The method of this invention is also applicable to flexible manipulators of other structures.

[0038] like Figure 2Shown are the four grasping states used in the embodiment of the present invention. The four grasping states are: inflation bending (the air pressure inside the manipulator rises to cause the manipulator to bend), force shaking (the manipulator will not fall when grasping an object, but will be vibrated by external forces), stable grasping (the object is grasped steadily and undisturbed) and object force sliding (the object is affected by external forces and slides relative to the manipulator). The four grasping states used in the embodiment of the present invention are different from the grasping states of traditional rigid manipulators, and can basically include all states of the grasping process of flexible manipulators, while also facilitating experiments. The implementation of the method of the present invention is not limited to the above four grasping states, and is defined according to actual needs.

[0039] like Figure 3 As shown, it is a scene demonstration of the two major types of experiments set by the present invention, static and dynamic. The static experiment refers to the situation where the flexible manipulator is installed on the rigid manipulator arm, and the position where the flexible manipulator and the manipulator arm are fixed is called the wrist of the manipulator arm. The flexible manipulator is placed horizontally as the initial position of the wrist, and the wrist is rotated in different directions of up, down, left and right, and experiments of four grasping states are carried out in these directions respectively. The different directions of the static experiment are named as the horizontal upper direction, the horizontal lower direction, the horizontal left direction, the horizontal right direction and the horizontal direction. The dynamic experiment refers to the situation where the flexible manipulator is installed on the rigid manipulator arm, and the manipulator arm drives the manipulator to move in different directions, and experiments of four grasping states are carried out during these movements. The different directions of movement refer to three kinds of translational reciprocating movements, namely, back and forth, left and right, and up and down in space, as well as semicircular arc reciprocating movements in three orthogonal planes in space.

[0040] like Figure 4 The figure shows the algorithm flow chart of the self-adaptive normalized high-dimensional fuzzy system proposed in the present invention. The present invention proposes to use the self-adaptive normalized high-dimensional fuzzy system to classify the grasping state. Consider an offline situation, that is, the data X={x i}, i = 1, 2, ..., N and the corresponding grasping state label Y = {y i}, i = 1, 2, ..., N, where N represents the number of samples. The method comprises the following steps:

[0041] S1. Static experimental data collection. The flexible manipulator is fixed to the end of a robotic arm. The location where the flexible manipulator and the robotic arm are fixed is called the wrist of the manipulator. The wrist of the manipulator is rotated in five different directions: horizontally upward, horizontally downward, horizontally to the left, horizontally to the right, and horizontally. Tactile sensor data is collected in these directions under four different gripping states of the flexible manipulator grasping an object. This data is then combined to form a dataset, referred to as raw data S1.

[0042] S2. Collection of dynamic experimental data. Similarly, the flexible manipulator is fixed to the end of a robotic arm. The robotic arm drives the flexible manipulator to move back and forth in three mutually perpendicular axial directions in space, namely, translational reciprocating motion in three axial directions: front-to-back, left-to-right, and up-and-down. The robotic arm then drives the flexible manipulator to perform semicircular reciprocating motion in three orthogonal planes in space. During these six different directional movements, data from the tactile sensor is collected in four different gripping states when the flexible manipulator grasps an object. These data are then mixed to form a data set, which is referred to as raw data S2.

[0043] S3. For the original data S (S can be either original data S1 or original data S2), split the training set and the test set, perform Z-Score normalization on the training set, and use the expectation and standard deviation of the training set to normalize the test set. Suppose the normalized training set is in N t is the number of training set samples, and D is the vector dimension.

[0044] S4. Establish the TSK fuzzy system HTSK based on high-dimensional defuzzification algorithm, and set its normalized activation as Consider a TSK system with D-dimensional input and 1-dimensional output and R rules:

[0045] Rule r: If x i,1 It's X r,1 , x i,2 It's X r,2 ,…,x i,D It's X r,D ;

[0046] Then we have:

[0047] where X r,d is the fuzzy set corresponding to the d-dimensional feature of the r-th rule, whose center is m r,d , with a standard deviation of σ r,d .y r (x) is the output of the rth rule, which needs to be optimized r,d , σ r,d And the corresponding parameter w r,0 and w r,d , where r=1,2,...,R,d=1,2,...,D. Assume Define the activation function Specifies the output of the HTSK system

[0048] For the scenario of this embodiment, setting the value range of the fuzzy rule number to 300-1800 can ensure the accuracy of the results to a large extent, while balancing the time and space consumption required by the algorithm to a certain extent.

[0049] S5. Based on the adaptive normalization algorithm SN, the normalized activation of HTSK is mapped to obtain the new rule activation. Where Ω={in,ln,bn}. ε is a small quantity, μ in and σ in is the mean and standard deviation of the single channel of sample x, μ ln and σ ln is the mean and standard deviation of sample x, μ bn and σ bn is the mean and standard deviation vector of the current batch features, w k Is the activation function calculated by softmax k∈{in,ln,bn}, similarly we can get w k ′. γ,β,λ in ,λ ln ,λ bn ,λ′ in ,λ′ ln ,λ′ bn All are updated and optimized during training. Thus, the new activation is

[0050] S6. Use the ReLU linear activation function to further process the regular normalized activation as follows:

[0051]

[0052] S7. For this algorithm, the training set is used for training to obtain the final training network, that is, the SN normalization layer and the ReLU linear activation layer are added to the HTSK network.

[0053] S8. Based on the trained fuzzy system, the tactile sensing signal is input to the network in real time to obtain the grasping state detection result in real time

[0054] For example, this experiment splits the dataset into an 80% training set and a 20% test set, with a sensor dimension D of 12 and a sample category count of 4. In both the dynamic and static experiments, the classification results using the adaptive normalized high-dimensional fuzzy system are shown in Table 1: 97.27% in the static experiment and 94.96% in the dynamic experiment.

[0055] Table 1

[0056] Four-category grasping state accuracy

[0057] Static 97.27％ dynamic 94.96％

[0058] The tactile sensors of a rigid manipulator are relatively separate from the manipulator itself, so the position and movement of the rigid manipulator typically have little impact on the tactile sensor signals. However, due to the special characteristics of a flexible manipulator, its different position and posture will cause the shape of the flexible hand to change, thereby changing the signals of its internal tactile sensors. Similarly, when in motion, the flexible hand is more susceptible to deformation due to the movement process, which interferes with the signals of its internal tactile sensors. Mathematically describing the different postures and movements of a flexible hand using analytical methods is inherently difficult, and finding the mathematical relationship between the posture and movement of the flexible hand and its tactile sensor signals is even more difficult. Therefore, the present invention conducted grasping state classification and recognition experiments in five static scenarios: horizontally upward, horizontally downward, horizontally leftward, horizontally rightward, and horizontally; as well as six dynamic scenarios: translational reciprocating motion in three axial directions in space: forward, backward, leftward, and upward, and reciprocating motion in a semicircular arc in three orthogonal planes. The present invention greatly broadens the experimental scenarios, making the experimental results highly generalizable and applicable.

[0059] Existing research on grasping states is basically based on rigid manipulators, while research on flexible manipulators usually uses common machine learning methods based on feature extraction. This method is not suitable for real-time detection of grasping states and is usually an offline analysis. Conventional machine learning methods have poor performance and poor interpretability. At the same time, as the dimension of the input signal increases, it is easy to produce the problem of dimensionality disaster. Therefore, the present invention proposes a self-adaptive standardized high-dimensional fuzzy system, and for the first time uses a fuzzy system to solve the problem of grasping state detection of flexible manipulators. The fuzzy system has the advantages of high input dimension, accurate classification results, strong interpretability, and high generalization. Compared with the grasping state detection results of other machine learning methods, its classification accuracy is higher. In addition, based on the trained fuzzy system, the tactile sensing signal can be input to the network in real time, and the grasping state detection results can be obtained in real time.

[0060] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for detecting the gripping state of a flexible manipulator in different postures and motion states, characterized in that: include: S1. Collect tactile sensor data under different grasping states of the flexible manipulator, with the manipulator in horizontal, slightly above, slightly below, slightly to the left, and slightly to the right, to obtain static experimental data. S2. Collect tactile sensor data from different grasping states while the flexible manipulator moves in three mutually perpendicular axial directions and back-and-forth semicircular arcs in three orthogonal planes, to obtain dynamic experimental data; combine the static and dynamic experimental data as raw data; S3. Dividing the original data into a training set and a test set; S4. Establishing a self-adaptive standardized high-dimensional fuzzy system; the self-adaptive standardized high-dimensional fuzzy system is provided with an SN normalization layer and a ReLU linear activation layer based on HTSK; the SN normalization layer is used to map the normalized activation of HTSK to obtain the new rule activation; the ReLU linear activation layer is used to further process the new rule activation; S5. Using the training set and the test set to train the adaptive standardized high-dimensional fuzzy system; S6. Input the tactile sensing signal of the flexible manipulator in the current grasping state into the trained high-dimensional fuzzy system to obtain the grasping state detection result in real time.

2. The method for detecting the gripping state of a flexible manipulator in different postures and motion states according to claim 1 is characterized in that: The grasping states include inflation bending of the flexible hand, forced shaking, stable grasping, and forced sliding of the object.

3. The method for detecting the gripping state of a flexible manipulator in different postures and motion states according to claim 2 is characterized in that: The method further includes performing Z-Score normalization on the training set, and normalizing the test set using the expectation and standard deviation of the training set.

4. The method for detecting the gripping state of a flexible manipulator in different postures and motion states according to claim 2 is characterized in that: The flexible manipulator is pneumatically driven, and pressurizes the air cavity inside the manipulator to expand and bend it, so as to envelop or pinch an object, thereby achieving grasping of the object.

5. The method for detecting the gripping state of a flexible manipulator in different postures and motion states according to claim 4 is characterized in that: The flexible manipulator adopts a piezoresistive tactile sensing array as a tactile sensor.

6. The method for detecting the gripping state of a flexible manipulator in different postures and motion states according to claim 1 is characterized in that: In the method, the value range of the number of fuzzy rules is set to 300-1800.

7. The method for detecting the gripping state of a flexible manipulator in different postures and motion states according to claim 5 is characterized in that: The tactile sensor of the flexible manipulator is embedded in the flexible manipulator.

8. A flexible manipulator grasping state detection system in different postures and motion states, characterized by: include: The static experimental data acquisition module is used to collect tactile sensor data under different grasping states when the flexible manipulator is in the horizontal, horizontally upward, horizontally downward, horizontally to the left, and horizontally to the right postures, and obtain static experimental data; The dynamic experimental data acquisition module is used to collect tactile sensor data under different grasping states when the flexible manipulator moves back and forth in three mutually perpendicular axial directions in space, and moves back and forth in a semicircular arc in three orthogonal planes, to obtain dynamic experimental data; A training data construction module is used to combine the static experimental data and the dynamic experimental data as raw data; and divide the raw data into a training set and a test set; A model building module is used to establish a self-adaptive standardized high-dimensional fuzzy system, and train the self-adaptive standardized high-dimensional fuzzy system using a training set and a test set; the self-adaptive standardized high-dimensional fuzzy system is provided with an SN normalization layer and a ReLU linear activation layer based on HTSK; the SN normalization layer is used to map the normalized activation of HTSK to obtain the new rule activation; the ReLU linear activation layer is used to further process the new rule activation; The real-time detection module is used to input the tactile sensing signal of the flexible manipulator in the current grasping state into the trained high-dimensional fuzzy system to obtain the grasping state detection result in real time.

9. A method for controlling the gripping state of a flexible manipulator in different postures and motion states, characterized in that: include: The current grasping state of the flexible manipulator is detected by using the method for detecting the grasping state of the flexible manipulator in different postures and motion states according to any one of claims 1 to 7; The grasping state of the flexible manipulator is controlled based on the detection results.

Images