A pneumatic soft multi-fingered hand with cross-modal tactile sensing function on the palm

Abstract

A pneumatic soft multi-fingered hand with a cross-modal tactile sensing function on the palm includes an inflatable film sensing device as the palm, a plurality of pneumatic curved soft fingers arranged around the edges of the palm, an air pressure regulating device connected to the palm and the pneumatic curved soft fingers, and a control device connected to the inflatable film sensing device. The present invention can not only achieve flexible grasping and protect the grasped object from being damaged, but also establish tactile feedback based on vision, realize cross-modal transformation from visual mode to tactile mode, establish tactile sensing function, identify the shape of the grasped object, and thus classify the object.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic hands, and more specifically, to a pneumatic soft multi-fingered hand with a palm having a cross-modal tactile sensing function. Background Art

[0002] With the development of robotics and the increasing demand for automation, multi-fingered hands are widely used in automated grasping operations. As a substitute for human hands, multi-fingered hands need to be flexible enough and have tactile perception to adapt to unstructured and complex contact environments.

[0003] The Chinese patent document with publication number CN110919687A discloses a soft gripper with an embedded flexible multimodal tactile sensor, including at least one gripper assembly, characterized in that the gripper assembly includes a soft gripper, a connecting plate and a multimodal tactile sensor, the soft gripper is two, relatively symmetrically arranged at both ends of the connecting plate, the soft gripper includes a hollow soft gripper body, the hollow part constitutes a first air chamber, the outer surface of the inner side wall of the soft gripper body is provided with barb tooth grooves, and the outer side wall is provided with multiple groups of continuous curved structures. However, the above scheme can only realize flexible grasping of the finger part, and cannot realize the recognition of the shape of the object for classification. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of existing robotic hands that cannot realize flexible grasping and can recognize the shape of objects for classification, and to provide a pneumatic soft multi-fingered hand with a cross-modal tactile sensing function on the palm. The present invention can not only realize flexible grasping and protect the grasped objects from being damaged, but also establish tactile feedback based on vision, realize cross-modal transformation from visual mode to tactile mode, establish tactile sensing function, recognize the shape of the grasped objects, and thus classify the objects.

[0005] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0006] A pneumatic soft multi-fingered hand with a cross-modal tactile sensing function on the palm includes an inflatable film sensing device as the palm, a plurality of pneumatic curved soft fingers arranged around the edges of the palm, an air pressure regulating device connected to the palm and the pneumatic curved soft fingers, and a control device connected to the inflatable film sensing device. The palm and the pneumatic curved soft fingers are inflated by the air pressure regulating device to achieve object grasping, and the air pressure regulating device can control the air pressure not to be too high to damage the grasped object. The pneumatic curved soft fingers can be bent and wrap around the grasped object after being filled with a certain air pressure by the air pressure regulating device, and the air pressure can be adjusted according to the size and weight of the object to achieve different grasping forces. The inflatable film sensing device and the control device can determine whether the object is grasped successfully and sense the three-dimensional geometric shape of the grasped object, thereby classifying the grasped object.

[0007] Furthermore, the inflatable film sensor device includes a bracket, a film arranged on the top of the bracket as a palm surface, and a sensing and identification device arranged inside the bracket and used in conjunction with the film. The pneumatic bending soft finger is arranged on the circumferential outer side wall of the bracket.

[0008] Furthermore, the film is provided with a plurality of columnar protrusions on the side close to the sensing and identifying device. The sensing and identifying device includes a camera and a lighting board. The camera and the lighting board are directly opposite to the columnar protrusions. The camera is connected to the control device for communication. When grasping an object, the inflatable film device as the palm is squeezed, causing the film to deform and drive the columnar protrusions fixed on the film to move. After the lighting board illuminates the film and its columnar protrusions, the camera can take pictures of the state of the columnar protrusions before and after the displacement, and transmit them to the control device for comparison and calculation, and finally obtain the contact area between the object and the film and the three-dimensional geometric shape information. In this way, vision-based tactile feedback can be established to achieve the transformation from visual mode to tactile mode.

[0009] Furthermore, the color of the end face of the cylindrical protrusion facing the camera is different from the color of the other parts of the cylindrical protrusion and the color of the side of the film facing the camera. In this way, the end face of the cylindrical protrusion is easier to distinguish from the background color composed of the side of the film facing the camera and the surface color of the cylindrical protrusion other than the end face, and the displacement of the cylindrical protrusion captured by the camera is clearer and easier to be recognized by the control device.

[0010] Furthermore, the end face of the cylindrical protrusion is circular, and the film is parallel to the camera in the uninflated state, and the end face of the cylindrical protrusion photographed by the camera is circular; when the inflatable film sensor device is inflated or grabs an object, the film is deformed so that the cylindrical protrusion is displaced, and the end face of the cylindrical protrusion photographed by the camera is elliptical. In this way, the control device applies image processing operations such as filtering, expansion, and convex hull operation to the image captured by the camera to obtain the contact area between the grasped object and the palm film. At the same time, the end face of the cylindrical protrusion of the palm film is formed into an ellipse in the image captured by the camera, and its ellipse equation is obtained by conventional graphic fitting means. Combined with the perspective projection law of the space circle, the position of each cylindrical protrusion end face in space is calculated according to the ellipse equation, so as to fit the deformation surface of the film using the least squares method, that is, the contact surface between the palm and the object, analyze the geometric features of the contact surface, obtain the local three-dimensional geometric information of the object, realize cross-modal tactile perception, and realize the classification of the object according to the geometric information of the object.

[0011] Furthermore, the color of the end face of the columnar protrusion facing the camera is white, and the color of the other parts of the columnar protrusion except its end face and the color of the side of the film facing the camera are black. In this way, the color of the columnar protrusion is most different from the background color, and it is easiest to identify the control device.

[0012] Furthermore, the columnar protrusions on the film are distributed in a polygonal array, with a distribution spacing of 2.0-4.0 mm, a height of 1.0-3.0 mm, and a diameter of 1.0-2.0 mm. In this way, the columnar protrusions will not be too thick to affect the elasticity of the film, and at the same time, the columnar protrusions are neatly arranged, which is conducive to the position change information of the end face of the columnar protrusion caused by the film being deformed by force, so as to calculate the contact area of ​​the object with the film and the three-dimensional geometric shape information.

[0013] Furthermore, the columnar protrusions on the film are distributed in a hexagonal array with a distribution spacing of 3 mm, the height of the columnar protrusions is 2 mm, and the diameter of the top end face of the columnar protrusions is 1.5 mm. This arrangement is most conducive to the control device combining the perspective projection law of the space circle, and calculating the position of the end face of the columnar protrusion in space according to the imaging ellipse equation, thereby deriving the three-dimensional geometric shape information of the palm film.

[0014] Furthermore, the pneumatic bending soft finger includes a connection part for fixing on the palm, a main air cavity connected to the connection part, and a plurality of secondary air cavities arranged on a side of the main air cavity away from the palm and connected to the main air cavity, wherein the secondary air cavities are closely arranged in sequence along the length direction of the pneumatic bending soft finger. Thus, when the finger is inflated, the secondary air cavity expands and squeezes, so that the finger bends toward the palm, and the plurality of fingers bend to cover the object to achieve the object grabbing.

[0015] Furthermore, the film and the pneumatic bending soft fingers are made of soft elastomers, so that when the pneumatic soft multi-fingered hand grasps an object, all places in contact with the object are made of soft materials, which better protects the object and achieves flexible grasping.

[0016] The present invention also provides a method for grasping an object by a pneumatic soft multi-finger hand having a cross-modal tactile sensing function on the palm, which comprises the following steps:

[0017] Step 1: first move the pneumatic soft multi-finger hand to the grasping position, set the palm contact area threshold U on the control device, set the air pressure threshold P1 of the pneumatic bending soft finger, the air pressure threshold P2 of the palm and the initial air pressure value of the palm on the air pressure regulating device, inflate the pneumatic bending soft finger and grasp the target object. The air pressure threshold P1 of the finger and the air pressure threshold P2 of the palm are empirical values, which are used to protect the grasped object and the pneumatic soft multi-finger hand itself, otherwise the air pressure of the palm and fingers is too high, which can easily crush the grasped object and break the palm film and fingers. The palm contact area threshold U is also an empirical value, which is used by the control device to determine whether the object is grasped successfully. If the contact area between the object and the film calculated by the control device is less than the threshold U, it is determined that the grasping is unsuccessful; if the calculated contact area is not less than the threshold U, it is considered that the grasping is successful.

[0018] Step 2: Calculate the contact area between the object and the film through the inflatable film sensor device and the control device, and determine whether the palm contact area is less than the threshold U. If so, proceed to step 3; if not, proceed to step 5. If the palm contact area is less than the threshold U, it is determined that the grasping is not successful, and proceed to step 3 to continue grasping; if the palm contact area is not less than the threshold U, it is determined that the grasping is successful and proceed to step 5.

[0019] Step 3: Determine whether the air pressure of the pneumatic bending soft finger is less than the threshold value P1. If so, continue to inflate the finger and then proceed to step 2 again; if not, proceed to step 4. Before continuing to inflate the finger, first determine whether the air pressure of the finger has reached the threshold value P1. If the air pressure of the finger has reached the threshold value P1, proceed to step 4 to further determine whether to increase the air pressure of the palm to grasp the object; if the air pressure of the finger is not yet at the threshold value P1, continue to increase the air pressure of the finger and again determine whether the contact area between the palm and the object reaches the threshold value U.

[0020] Step 4: Determine whether the palm air pressure is less than the threshold value P2. If so, continue to inflate the palm and then repeat step 2; if not, the control device will report an error and stop, waiting for manual intervention. Before continuing to inflate the palm, first determine whether the palm air pressure has reached the threshold value P2. If the palm air pressure is less than the threshold value P2, continue to increase the palm air pressure, and then perform step 2 to again determine whether the contact area between the object and the palm reaches the threshold value U; if the palm air pressure is not less than the threshold value P2, at this time, the air pressure of the palm and fingers has reached the threshold value. In order to prevent the object from being crushed or the palm and fingers from being broken, the air pressure cannot be increased further, and the contact area of ​​the object and the palm has not reached the threshold value, so it is determined that the grasping has failed, and the device will stop reporting an error and wait for manual intervention.

[0021] Step 5: The three-dimensional geometric shape information of the palm film is derived through the control device, and the objects are classified according to their geometric shapes. The pneumatic soft multi-fingered hand is moved to the corresponding position, the object is put down, and finally the air pressure of the pneumatic bending soft fingers and palm is released. The end face of the columnar protrusion on the side of the palm film close to the sensing device is circular. When the film is not inflated, it is parallel to the camera, and the end face of the columnar protrusion photographed by the camera is circular; when the film is inflated or grabs an object, the film is deformed so that the columnar protrusion is displaced, and the end face of the columnar protrusion photographed by the camera is elliptical. In this way, the control device can calculate the position of the end face of the columnar protrusion in space according to the imaging ellipse equation, thereby deriving the three-dimensional geometric shape information of the palm film and classifying the objects.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] The present invention uses an inflatable film sensing device and a control device as the palm to establish visual-based tactile feedback, realize the transformation from visual mode to tactile mode, identify the geometric shape of the grasped object, and thus classify the grasped object. At the same time, the pneumatically curved soft fingers and palm film that contact the grasped object of the present invention are both made of flexible materials, and the inflatable device can control the grasping force to achieve fully flexible grasping. Moreover, through tactile sensing, it is possible to determine whether the object is grasped successfully while achieving flexible grasping. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the overall structure of the present invention.

[0025] Figure 2 It is a schematic diagram of the internal structure of the present invention.

[0026] Figure 3 It is a schematic diagram of the present invention when grasping an object.

[0027] Figure 4 This is an effect diagram of the film and its columnar protrusions photographed by the camera of the present invention.

[0028] Figure 5 This is a flow chart of the object grabbing method of the present invention.

[0029] The illustrations are as follows:

[0030] 1-pneumatic bending soft finger, 2-bracket, 3-film, 4-columnar protrusion, 5-camera, 6-illumination board, 7-end face, 8-connecting part, 9-main air cavity, 10-secondary air cavity. DETAILED DESCRIPTION

[0031] The present invention is further described below in conjunction with specific implementation methods. The accompanying drawings are only used for exemplary descriptions and are only schematic diagrams, not actual drawings, and cannot be understood as limiting this patent; in order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the accompanying drawings may be omitted.

[0032] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limitations on this patent. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0033] Example 1

[0034] like Figures 1 to 4 As shown, a pneumatic soft multi-fingered hand with a cross-modal tactile sensing function on the palm includes an inflatable film sensing device as the palm, a plurality of pneumatic bending soft fingers 1 arranged on the edges of the palm, an air pressure regulating device connected to the palm and the pneumatic bending soft fingers 1, and a control device connected to the inflatable film sensing device. The palm and the pneumatic bending soft fingers 1 are inflated by the air pressure regulating device to grasp the object, and the air pressure regulating device can control the air pressure not to be too high to damage the grasped object. The pneumatic bending soft fingers 1 can be bent and wrap around the grasped object after being filled with a certain air pressure by the air pressure regulating device, and can adjust the air pressure according to the size and weight of the object to achieve different grasping forces. The inflatable film sensing device and the control device can determine whether the grasped object is grasped successfully and sense the three-dimensional geometric shape of the grasped object, so as to classify the grasped object.

[0035] like Figures 1 to 4 As shown, the inflatable film sensing device includes a bracket 2, a film 3 provided on the top of the bracket 2 as a palm surface, and a sensing and identification device provided inside the bracket 2 and used in conjunction with the film 3. The pneumatic bending soft finger 1 is provided on the circumferential outer wall of the bracket 2.

[0036] like Figures 1 to 4As shown, the film 3 is provided with several columnar protrusions 4 on the side close to the sensing and recognition device. The sensing and recognition device includes a camera 5 and a lighting board 6. The camera 5 and the lighting board 6 are directly facing the columnar protrusions 4. The camera 5 is connected to the control device for communication. When grasping an object, the inflatable film device as the palm is squeezed, causing the film 3 to deform and drive the columnar protrusions 4 fixed on the film 3 to move. After the lighting board 6 illuminates the film 3 and its columnar protrusions 4, the camera 5 can take pictures of the state of the end face 7 of the columnar protrusion 4 before and after the displacement, and transmit it to the control device for comparison and calculation, and finally obtain the contact area of ​​the object with the film 3 and the three-dimensional geometric shape information.

[0037] like Figures 1 to 4 As shown, the color of the end face 7 of the columnar protrusion 4 facing the camera 5 is different from the color of the other parts of the columnar protrusion 4 and the color of the side of the film 3 facing the camera 5. In this way, the end face 7 of the columnar protrusion 4 is easier to distinguish from the background color composed of the side of the film 3 facing the camera 5 and the surface color of the columnar protrusion 4 except the end face 7, and the displacement of the columnar protrusion 4 captured by the camera 5 is clearer and easy to identify by the control device. In this embodiment, the color of the end face 7 of the columnar protrusion 4 facing the camera 5 is white, and the color of the other parts of the columnar protrusion 4 except the end face 7 and the color of the side of the film 3 facing the camera 5 are black. In this way, the color of the columnar protrusion 4 is most different from the background color, which is easiest to identify by the control device.

[0038] like Figures 1 to 4 As shown, the end face 7 of the columnar protrusion 4 is circular, and in the uninflated state, the film 3 is parallel to the camera 5, and the end face 7 of the columnar protrusion 4 photographed by the camera 5 is circular; when the inflatable film sensor device is inflated or grabs an object, the film 3 is deformed to cause the columnar protrusion 4 to move, and at this time, the end face 7 of the columnar protrusion 4 photographed by the camera 5 is elliptical. In this way, the control device applies image processing operations such as filtering, dilation, and convex hull operations to the image captured by the camera 5 to obtain the contact area between the grasped object and the palm film 3. At the same time, the end face 5 of the columnar protrusion 4 of the palm film 3 forms an ellipse in the image captured by the camera 5. The ellipse equation is obtained by conventional graphic fitting means, and combined with the perspective projection law of the space circle, the position of the end face 5 of each columnar protrusion 4 in space is calculated according to the ellipse equation. The least squares method is then used to fit the deformed surface of the film 3, that is, the contact surface between the palm and the object. The geometric features of the contact surface are analyzed to obtain the local three-dimensional geometric information of the object, realize cross-modal tactile perception, and classify the object according to its geometric information.

[0039] In this way, the control device can calculate the position of the end face 7 of the cylindrical protrusion 4 in space according to the imaged ellipse equation, thereby deriving the three-dimensional geometric shape information of the palm film 3. In this way, a tactile feedback based on vision can be established to achieve the transformation from the visual mode to the tactile mode.

[0040] like Figures 1 to 4 As shown, the columnar protrusions 4 on the film 3 are distributed in a hexagonal array with a distribution spacing of 3 mm, the height of the columnar protrusions 4 is 2 mm, and the diameter of the top end surface 7 of the columnar protrusions 4 is 1.5 mm. This arrangement is most conducive to the control device combining the perspective projection law of the space circle, and calculating the position of the end surface 7 of the columnar protrusion 4 in space according to the ellipse equation of the imaging, thereby deriving the three-dimensional geometric shape information of the palm film 3.

[0041] like Figures 1 to 4 As shown, the pneumatic bending soft finger 1 includes a connection part 8 for fixing on the palm, a main air cavity 9 connected to the connection part 8, and a plurality of secondary air cavities 10 arranged on a side of the main air cavity 9 away from the palm and connected to the main air cavity 9, and the secondary air cavities 10 are arranged closely in sequence along the length direction of the pneumatic bending soft finger 1. In this way, when the pneumatic bending soft finger 1 is inflated, the secondary air cavity 10 expands and squeezes, so that the pneumatic bending soft finger 1 bends toward the palm, and the plurality of pneumatic bending soft fingers 1 bend and wrap around an object to achieve object grasping.

[0042] like Figures 1 to 4 As shown, the film 3 and the pneumatic bending soft finger 1 are made of soft elastomer, so that when the pneumatic soft multi-fingered hand grasps an object, all places in contact with the object are made of soft materials, which better protects the object and realizes flexible grasping.

[0043] Example 2

[0044] This embodiment is similar to the embodiment 1, except that, in this embodiment, the columnar protrusions 4 on the film 3 are distributed in a polygonal array, with a distribution spacing of 2.0 mm, a height of the columnar protrusions 4 of 1.0 mm, and a diameter of the columnar protrusions 4 of 1.0 mm. The side of the film 3 facing the camera 5 is red, and the end surface 7 of the columnar protrusions 4 facing the camera 5 is green.

[0045] Example 3

[0046] This embodiment is similar to Embodiment 1, except that, in this embodiment, the columnar protrusions 4 on the film 3 are distributed in a polygonal array with a distribution spacing of 4.0 mm, the height of the columnar protrusions 4 is 3.0 mm, and the diameter of the columnar protrusions 4 is 2.0 mm.

[0047] Example 4

[0048] like Figure 5 As shown, a method for grasping an object by a pneumatic soft multi-finger hand having a cross-modal tactile sensing function on the palm includes the following steps:

[0049] Step 1: first move the pneumatic soft multi-finger hand to the grasping position, set the palm contact area threshold U on the control device, set the air pressure threshold P1 of the pneumatic bending soft finger 1, the palm air pressure threshold P2 and the initial air pressure value of the palm on the air pressure regulating device, inflate the pneumatic bending soft finger 1 and grasp the target object. The air pressure threshold P1 of the finger and the air pressure threshold P2 of the palm are empirical values, which are used to protect the grasped object and the pneumatic soft multi-finger hand itself, otherwise the palm and finger air pressure is too high, which can easily crush the grasped object and break the palm film 3 and the pneumatic bending soft finger 1. The palm contact area threshold U is also an empirical value, which is used by the control device to determine whether the object is grasped successfully. If the contact area between the object and the film 3 calculated by the control device is less than the threshold U, it is determined that the grasping is unsuccessful; if the calculated contact area is not less than the threshold U, it is considered that the grasping is successful.

[0050] Step 2: Calculate the contact area between the object and the film 3 through the inflatable film sensor device and the control device, and determine whether the palm contact area is less than the threshold value U. If so, proceed to step 3, if not, proceed to step 5. If the palm contact area is less than the threshold value U, it is determined that the grasping is not successful, and proceed to step 3 to continue grasping; if the palm contact area is not less than the threshold value U, it is determined that the grasping is successful and proceed to step 5.

[0051] Step 3: Determine whether the air pressure of the pneumatic bending soft finger is less than the threshold value P1. If so, continue to inflate the pneumatic bending soft finger 1 and then proceed to step 2 again; if not, proceed to step 4. Before continuing to inflate the pneumatic bending soft finger 1, first determine whether the air pressure of the pneumatic bending soft finger 1 has reached the threshold value P1. If the air pressure of the pneumatic bending soft finger 1 has reached the threshold value P1, proceed to step 4 to further determine whether to increase the air pressure of the palm to grasp the object; if the air pressure of the pneumatic bending soft finger 1 is not yet at the threshold value P1, continue to increase the air pressure of the pneumatic bending soft finger 1, and then determine again whether the contact area between the film 3 and the object has reached the threshold value U.

[0052] Step 4: Determine whether the palm air pressure is less than the threshold value P2. If so, continue to inflate the palm and then proceed to step 2 again; if not, the control device will report an error and stop, waiting for manual intervention. Before continuing to inflate the palm, first determine whether the palm air pressure has reached the threshold value P2. If the palm air pressure is less than the threshold value P2, continue to increase the palm air pressure, and then proceed to step 2 to determine again whether the contact area between the object and the palm reaches the threshold value U; if the palm air pressure is not less than the threshold value P2, at this time, the air pressure of the palm and the pneumatic bending soft finger 1 have reached the threshold value. In order to prevent the object from being crushed or the palm and fingers from being broken, the air pressure cannot be increased further, and the contact area of ​​the object and the palm has not reached the threshold value, so it is determined that the grasping has failed, and the machine will stop reporting an error and wait for manual intervention.

[0053] Step 5: The three-dimensional geometric shape information of the palm film 3 is derived through the control device, the objects are classified according to their geometric shapes, the pneumatic soft multi-fingered hand is moved to the corresponding position, the object is put down, and finally the air pressure of the pneumatic bending soft fingers 1 and the palm is released. The end face 7 of the columnar protrusion 4 on the side of the palm film 3 close to the sensing device is circular. In the uninflated state, the film 3 is parallel to the camera 5, and the end face 7 of the columnar protrusion 4 photographed by the camera 5 is circular; when the palm is inflated or an object is grasped, the film 3 is deformed so that the columnar protrusion 4 is displaced, and the end face 7 of the columnar protrusion 4 photographed by the camera 5 is elliptical. The cylindrical protrusion 4 and the film 3 are illuminated by the lighting board 6, and the camera 5 photographs the shapes of all the end faces 7, so that the control device can calculate the position of the end face 7 of the columnar protrusion 4 in space according to the imaging ellipse equation, thereby deriving the three-dimensional geometric shape information of the palm film 3 and classifying the objects.

[0054] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A pneumatic soft multi-fingered hand with cross-modal tactile sensing function on the palm, characterized in that: The invention comprises an inflatable film sensor device as a palm, a plurality of pneumatically curved soft fingers (1) arranged on the edges of the palm, an air pressure regulating device connected to the palm and the pneumatically curved soft fingers (1), and a control device connected to the inflatable film sensor device; the inflatable film sensor device comprises a bracket (2), a film (3) arranged on the top of the bracket (2) as a palm surface, and a sensing and recognition device arranged inside the bracket (2) and used in conjunction with the film (3); the pneumatically curved soft fingers (1) are arranged on the circumferential outer side wall of the bracket (2); a plurality of columnar protrusions (4) are arranged on the side of the film (3) close to the sensing and recognition device, and the sensing and recognition device comprises a camera (5) and a lighting board (6), the camera (5) and the lighting board (6) are directly opposite to the columnar protrusion (4), and the camera (5) is communicatively connected with the control device; the color of the end face (7) of the columnar protrusion (4) facing the camera (5) is different from the color of the other parts of the columnar protrusion (4) and the color of the side of the film (3) facing the camera (5); the color of the end face (7) of the columnar protrusion (4) facing the camera (5) is white, and the color of the other parts of the columnar protrusion (4) except the end face (7) and the color of the side of the film (3) facing the camera (5) are black; the material of the film (3) and the pneumatic bending soft finger (1) is a soft elastomer.

2. The pneumatic soft multi-fingered hand with cross-modal tactile sensing function on the palm according to claim 1, characterized in that: The columnar protrusions (4) on the film (3) are distributed in a polygonal array with a distribution spacing of 2.0-4.0 mm. The height of the columnar protrusions (4) is 1.0-3.0 mm, and the diameter of the columnar protrusions (4) is 1.0-2.0 mm.

3. The pneumatic soft multi-fingered hand with cross-modal tactile sensing function on the palm according to claim 1, characterized in that: The columnar protrusions (4) on the film (3) are distributed in a hexagonal array with a distribution spacing of 3 mm. The height of the columnar protrusions (4) is 2 mm, and the diameter of the top end surface (7) of the columnar protrusions (4) is 1.5 mm.

4. The pneumatic soft multi-fingered hand with cross-modal tactile sensing function on the palm according to claim 1, characterized in that: The pneumatically bendable soft finger (1) comprises a connecting portion (8) for fixing on a palm, a main air cavity (9) connected to the connecting portion (8), and a plurality of secondary air cavities (10) arranged on a side of the main air cavity (9) away from the palm and connected to the main air cavity (9), wherein the secondary air cavities (10) are closely arranged in sequence along the length direction of the pneumatically bendable soft finger (1).

5. The method for grasping an object by a pneumatic soft multi-finger hand having a cross-modal tactile sensing function on the palm as claimed in any one of claims 1 to 4, characterized in that: The following steps are involved: Step 1: first move the pneumatic soft multi-finger hand to the grasping position, set the contact area threshold U of the palm on the control device, set the air pressure threshold P1 of the pneumatic bending soft finger (1), the air pressure threshold P2 of the palm and the initial air pressure value of the palm on the air pressure regulating device, inflate the pneumatic bending soft finger (1) and grasp the target object; Step 2: Calculate the contact area between the object and the film (3) through the inflatable film sensor device and the control device, and determine whether the palm contact area is less than the threshold value U. If so, proceed to step 3; if not, proceed to step 5; Step 3: Determine whether the air pressure of the pneumatic bending soft finger (1) is less than the threshold value P1. If so, continue to inflate the pneumatic bending soft finger (1) and then proceed to step 2 again; if not, proceed to step 4; Step 4: Determine whether the palm air pressure is less than the threshold value P2. If so, increase the palm air pressure and then repeat step 2; if not, the control device reports an error and stops, waiting for manual intervention; Step 5: Through the control device, the three-dimensional geometric shape information of the palm film (3) is derived, the objects are classified according to their geometric shapes, the pneumatic soft multi-fingered hand is moved to the corresponding position, the object is placed down, and finally the air pressure of the pneumatic bending soft fingers (1) and the palm is released.

Images