Robot touch device based on gradient convex touch structure and identification method thereof

Abstract

The invention discloses a robot touch device based on a gradient convex touch structure and an identification method of the robot touch device. An integrated circuit chip and a printed circuit board of the device are stacked on a bottom supporting layer, and a force sensing layer covers the bottom supporting layer; the force sensing layer includes an outer insulating layer and an inner convex contact layer having a gradient convex contact structure. The six-dimensional force of the contact force, the true value of the position and the capacitance gray level image of the tactile device under the contact force are obtained through an established calibration platform, a data set is established for training of a convolutional neural network, and finally the six-dimensional force and position information of the contact force to be recognized is obtained through the convolutional neural network and a nearest neighbor algorithm. And tactile recognition is completed. The force sensing layer with the gradient structure improves the normal force range and the tangential force precision in tactile recognition; the system has the characteristics of miniaturization, compact structure and high resolution. The stability of multi-mode grabbing and the reliability of flexible operation can be improved through the contact force and contact position information sensed by the device.

Description

Technical Field

[0001] The invention relates to a robot tactile device and the field of tactile sensors, in particular to a robot tactile device based on a gradient convex tactile structure and an identification method thereof. Background Art

[0002] Robots have reached new levels of dexterity in recent years, and roboticists are still working to develop robots that can sort and pack items, cut vegetables, and fold clothes in unstructured and dynamic environments. This enhancement is due to mechanical breakthroughs, sensors that can sense touch on the robot's body, and new mechanisms that can provide natural compliance. Most notably, this development has taken advantage of huge advances in machine learning to encapsulate uncertainty models and support further developments in adaptive and robust control.

[0003] Therefore, integrating tactile devices on the hands of intelligent robots to provide the robots with information such as force and touch perception (force / position information) and to improve the operational dexterity and reliability of the robot hands in unstructured environments has become a necessary and key link in realizing further intelligence of robots. Summary of the invention

[0004] In order to solve the problems existing in the background technology, the present invention provides a robot tactile device based on a gradient convex touch structure and its recognition method. The present invention produces a tactile device with compact structure, miniaturization and high resolution through structural design; designs a gradient convex touch soft structure to improve the range and accuracy of the robot tactile device, and integrates the robot tactile device on the fingertips of the robot's dexterous hand to sense the contact force information and contact position information during grasping and dexterous operation, thereby improving the reliability and stability of grasping and dexterous operation.

[0005] The technical solution adopted by the present invention is:

[0006] 1. A robot tactile device based on gradient convex touch structure:

[0007] The robot tactile device comprises a force sensing layer, an integrated circuit chip, a printed circuit board (PCB) and a bottom support layer. The integrated circuit chip and the printed circuit board are sequentially stacked and arranged on one side of the bottom support layer and are electrically connected to each other. The force sensing layer covers the bottom support layer and covers the integrated circuit chip and the printed circuit board. The inner wall surface of the force sensing layer is opposite to the integrated circuit chip interval. The force sensing layer comprises an external insulating layer and an internal convex touch layer provided with a plurality of gradient convex touch structures for improving the tactile normal force range and tangential force accuracy. The external insulating layer and the internal convex touch layer are stacked, and each gradient convex touch structure on the internal convex touch layer is located on the inner wall surface of the internal convex touch layer and is opposite to the integrated circuit chip interval. The printed circuit board is electrically connected to an external data acquisition device.

[0008] The integrated circuit chip and the printed circuit board are welded together and fixed to the bottom support layer by solder glue. The force sensing layer and the bottom support layer are connected together by epoxy conductive glue, and there is also a flying wire at the bottom of the printed circuit board that is also connected to the inside of the force sensing layer by conductive glue, forming an external path of the capacitor. The internal convex contact layer 6 is composed of a uniform arrangement of hemispherical cylindrical gradient convex contact structures arranged in an array. The gradient convex contact structure is a gradient arrangement. Specifically, the gradient convex contact structures in the center and the surroundings are closer to the integrated circuit chip and increase toward the middle, and the overall distribution is annular.

[0009] The external insulating layer and the internal convex contact layer are both cylindrical structures, the bottom surface of the cylindrical structure is an arc curved toward the integrated circuit chip, the four sides of the cylindrical structure are sheet-shaped and form a square opening, each gradient convex contact structure is evenly spaced and arranged on the arc-shaped inner wall surface of the internal convex contact layer in a square array, and the end of each gradient convex contact structure is vertically facing the integrated circuit chip; the external insulating layer is sleeved on the internal convex contact layer, the internal convex contact layer is sleeved on the bottom support layer, and the integrated circuit chip and the printed circuit board are located between each gradient convex contact structure and the bottom support layer. The cross-sections of the integrated circuit chip, the printed circuit board and the bottom support layer are all square.

[0010] In the square array formed by the gradient convex contact structures of the internal convex contact layer, the distance between the end of the gradient convex contact structure located in the center and the integrated circuit chip is smaller than the distance between the end of the gradient convex contact structure located at the surrounding edges and the integrated circuit chip, the distance between the end of the gradient convex contact structure located at the surrounding edges and the integrated circuit chip is smaller than the distance between the remaining gradient convex contact structures and the integrated circuit chip, and the distance between the remaining gradient convex contact structures and the integrated circuit chip gradually increases from the center of the square array toward the surrounding edges.

[0011] The outer insulating layer is made of carbon black non-conductive silicone material, and the inner convex contact layer is made of carbon black conductive silicone material.

[0012] N*N capacitor plates are arrayed on the integrated circuit chip, and each gradient convex contact structure faces a plurality of respective capacitor plates. The integrated circuit chip as the lower plate is electrically connected to the printed circuit board.

[0013] The internal convex contact layer of the force sensing layer serves as the upper plate and the integrated circuit chip (serves as the lower plate, thereby forming many capacitors. When the force sensing layer is subjected to external contact force, the convex contact structure of the upper plate will be displaced, contact the lower plate, and deformed at the same time. At this time, due to the change in the spacing between the two plates of the capacitor, the capacitance value collected by the capacitor also changes. The capacitance of these large numbers of capacitors is collected, and after analog-to-digital conversion and data processing, a grayscale image converted from the capacitance value is obtained. The deformation and contact condition of the force sensing layer can be reflected through the radius size, center position, number and other information of the circle in the grayscale image, that is, the external contact force information and contact position information are indirectly reflected. The greater the external contact force, the smaller the spacing between the upper and lower plates, the larger the radius of the circle, the larger the capacitance value, the larger the grayscale value, and the brighter the circle.

[0014] When the force sensing layer is subjected to a normal force and begins to displace, the convex contact structure closer to the electrode will first contact the lower electrode and deform. When the deformation accumulates to a certain amount, other convex contact structures farther away from the electrode will begin to contact the electrode. The gradient distribution of the convex contact structure allows these convex contact structures to deform in batches and reach saturation. While ensuring the accuracy of the device, the range of the device is greatly improved. In addition, when the force sensing layer is subjected to normal force and tangential force at the same time, the gradient convex contact structure ensures that when receiving different normal forces, there will always be individual convex contact structures that have not reached deformation saturation and can be offset under the action of tangential force, which greatly improves the device's perception of tangential force.

[0015] 2. A tactile recognition method of a robot tactile device based on a gradient convex tactile structure, comprising:

[0016] 1) Build a calibration platform and install it on a horizontal and stable platform, and install the robot tactile device on the top of the calibration platform through the bottom support layer.

[0017] 2) Apply different contact forces to the force sensing layer of the robot tactile device through the spherical indenter at the end of the robotic arm, and for each moment of the application process of each contact force, obtain the true value of the six-dimensional force information of the contact force at the current moment through the calibration platform, obtain the true value of the contact position information of the contact force at the current moment during the application process through the spherical indenter, obtain the various capacitance values ​​generated by the various gradient convex contact structures of the force sensing layer at the current moment being pressed on the various capacitor plates on the integrated circuit chip through external data acquisition equipment and the printed circuit board of the robot tactile device, and convert them into a grayscale image of the capacitance value at the current moment, use the six-dimensional force information of the contact force at the current moment as the label of the grayscale image of the capacitance value at the current moment, construct each capacitance value grayscale image with the label of the six-dimensional force information as a six-dimensional force data set, use the true value of the contact position information of the contact force at the current moment as the label of the grayscale image of the capacitance value at the current moment, and construct each capacitance value grayscale image with the contact position information as a position data set.

[0018] 3) The six-dimensional force data set is input into the convolutional neural network (CNN) for training to obtain the trained convolutional neural network (CNN).

[0019] 4) The robot tactile device is integrated into the fingertips of the robot. When the robot drives the robot tactile device to contact an object, the capacitance values ​​generated when each gradient convex contact structure of the force sensing layer is pressed onto each capacitor plate on the integrated circuit chip are obtained in real time through external data acquisition equipment and the printed circuit board of the robot tactile device, and converted into a continuous capacitance value grayscale image; for each capacitance value grayscale image, the capacitance value grayscale image is input into the trained convolutional neural network CNN for processing, and the trained convolutional neural network CNN outputs the six-dimensional force information when contacting the object after processing, and the capacitance value grayscale image and the position data set are processed by the K nearest neighbor classification algorithm KNN (k-Nearest Neighbor) to obtain the contact position information when contacting the object, and finally complete the tactile recognition.

[0020] In the step 1), the calibration platform includes a first connecting member, a second connecting member, a rigid six-dimensional force sensor and a fixing member. The robot tactile device, the first connecting member, the second connecting member, the rigid six-dimensional force sensor, the fixing member and the stabilizing platform are arranged in sequence from top to bottom. The robot tactile device and the first connecting member are connected by a plurality of first bolts, the first connecting member and the second connecting member are connected by a plurality of second bolts, and the rigid six-dimensional force sensor and the fixing member are connected by a plurality of third bolts.

[0021] In the step 2), when the contact force is applied to the robot tactile device, each gradient convex touch structure of the robot tactile device moves downward and gradually contacts each capacitor plate on the integrated circuit chip. At each moment of the contact force application process, the capacitance value of each capacitor plate is transmitted to the external data acquisition device in real time via the printed circuit board, and then each capacitance value is converted into a capacitance value grayscale image after analog-to-digital conversion and normalization processing. The capacitance value grayscale image includes a number of circles with array distribution with brightness, the position of each circle is the position where the gradient convex touch structure contacts the capacitor plate, that is, the contact position information of the contact force, the number of each circle is the number of the gradient convex touch structures contacting the capacitor plate, that is, the contact area of ​​the contact force, and the size of each circle is the contact area where the gradient convex touch structure contacts the capacitor plate, that is, the contact pressure of the contact force, and the contact area and contact pressure of the contact force are used as the contact force information.

[0022] In the step 2), when a contact force is applied to the robot tactile device, when the gradient convex contact structures of the force sensing layer are subjected to the normal force of the contact force and produce a downward displacement, the gradient convex contact structures closer to the integrated circuit chip first contact the integrated circuit chip in turn and deform, and then the other gradient convex contact structures farther from the integrated circuit chip contact the integrated circuit chip in turn and deform; when the gradient convex contact structures of the force sensing layer are subjected to the normal force and the tangential force of the contact force at the same time, the gradient convex contact structures with larger deformation do not produce a displacement, and the other gradient convex contact structures with smaller deformation or no deformation produce a displacement, thereby sensing the tangential force.

[0023] By integrating the robot tactile device on the robot's dexterous fingertips, a stable grasping based on multi-modality can be achieved. The stable grasping based on multi-modality means that before the formal grasping, the stability of the grasping is judged by sensing information through different robot tactile devices in a "pre-grasping" manner, so as to decide whether to continue grasping or adjust a new grasping posture. In the application of the present invention, in the pre-grasping stage, the stability of the grasping is judged by the contact form and contact position sensed by the robot tactile device. The contact situation of the robot's dexterous fingertips can be judged based on the contact position information and contact form information obtained in real time, and the grasping force and grasping posture of the robot's dexterous hand can be adjusted in real time, which greatly improves the stability of the grasping. However, grasping is usually not the ultimate goal of robot operation. After stable grasping, it is often hoped to complete some operation tasks by changing the posture and position of the grasped object in the hand, taking bolt tightening as an example. Under the premise of stable grasping of the bolt, at the initial stage of the bolt being screwed in, the torque information sensed by the force tactile perception of the robot tactile device can be used to determine whether the bolt has an axial deviation. If an axial deviation occurs, the grasping posture is readjusted to prevent damage to the workpiece. In the tightening stage, the contact position sensed by the force tactile perception of the robot tactile device is used to ensure that the bolt does not exceed the fingertip safety edge area when rotating, that is, the arc center area of ​​the force sensing layer, to ensure the stability and safety of the operation; at the same time, as the bolt is tightened, the grasping force of the fingertips is adjusted in real time according to the sensed six-dimensional force information to prevent slippage.

[0024] The beneficial effects of the present invention are:

[0025] 1) The present invention designs a force sensing layer with a gradient structure and applies it to a force tactile sensor, thereby improving the normal force range and tangential force accuracy of the force tactile sensor.

[0026] 2) The robot tactile device of the present invention has the characteristics of miniaturization, compact structure and high resolution, and is easy to be integrated in the robot's dexterous fingertips, etc., and is suitable for narrow and complex environments. The contact force information and contact position information sensed by the robot can improve the stability and reliability of the robot's dexterous hand's multi-modal grasping and dexterous operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a schematic diagram of the overall structure of the robot tactile device of the present invention, wherein: Figure 1 (a) is an exploded view of the robot tactile device. Figure 1 (b) is a cross-sectional view of the robot tactile device;

[0028] Figure 2 It is a schematic diagram of a grayscale image of the capacitance value of the force sensing layer with a gradient structure when a force is applied;

[0029] Figure 3 This is a schematic diagram of a contact force application and six-dimensional force and contact position calibration platform for a robot tactile device of the present invention, wherein: Figure 3 (a) is a schematic diagram of the robot tactile device of the present invention when contact force is applied, Figure 3 (b) is a schematic diagram of a six-dimensional force sensing and contact position calibration platform of the robot tactile device of the present invention;

[0030] Figure 4 It is a visual schematic diagram of the robot tactile device of the present invention sensing different contact positions. Figure 4 (a) is a visual schematic diagram of the robot tactile device of the present invention sensing the contact position of the upper left area, Figure 4 (b) is a visual schematic diagram of the robot tactile device of the present invention sensing the contact position of the central area, Figure 4 (c) is a visual schematic diagram of the robot tactile device of the present invention sensing the contact position of the lower right area;

[0031] Figure 5 A schematic diagram of the robot tactile device of the present invention being integrated into the robot's dexterous hand;

[0032] Figure 6 This is a visualization diagram of the six-dimensional force perceived by the robot hand with integrated tactile device when grasping a bolt. Figure 6 (a) is a visualization diagram of the six-dimensional force perceived by the robot hand with integrated tactile device when stably grasping the bolt. Figure 6 (b) and (c) are visualization diagrams of the six-dimensional force sensed by the robot hand with integrated tactile device when grasping the bolt and the axis is offset.

[0033] Figure 7 A schematic diagram of the control flow of the robot tactile device of the present invention for applying stable grasping;

[0034] Figure 8 A schematic diagram of the control flow of tightening bolts in the robot tactile device of the present invention;

[0035] In the figure: 1. force sensing layer, 2. integrated circuit chip, 3. printed circuit board, 4. bottom support layer, 5. external insulation layer, 6. internal convex contact layer, 7. first connecting member, 8. first bolt, 9. second connecting member, 10. second bolt, 11. rigid six-dimensional force sensor, 12. fixing member, 13. third bolt, 14. robotic arm, 15. spherical pressure head, 16. stable platform, 17. dexterous robot hand. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] like Figure 1(a) and Figure 1 As shown in (b), the robot tactile device based on the gradient convex touch structure of the present invention comprises a force sensing layer 1, an integrated circuit chip 2, a printed circuit board 3 and a bottom support layer 4, wherein the integrated circuit chip 2 and the printed circuit board 3 are sequentially stacked and arranged on one side of the bottom support layer 4 and are electrically connected to each other, the force sensing layer 1 covers the bottom support layer 4 and covers the integrated circuit chip 2 and the printed circuit board 3, and the inner wall surface of the force sensing layer 1 is spaced and directly opposite to the integrated circuit chip 2; the force sensing layer 1 comprises an external insulating layer 5 and an internal convex touch layer 6 provided with a plurality of gradient convex touch structures for improving the tactile normal force range and tangential force accuracy, the external insulating layer 5 and the internal convex touch layer 6 are stacked, and each gradient convex touch structure on the internal convex touch layer 6 is located on the inner wall surface of the internal convex touch layer 6 and is spaced and directly opposite to the integrated circuit chip 2; the printed circuit board 3 is electrically connected to an external data acquisition device. The external insulating layer 5 is made of carbon black non-conductive silicone material, and the internal convex touch layer 6 is made of carbon black conductive silicone material.

[0038] In the specific implementation, the integrated circuit chip 2 is an integrated circuit chip for active capacitance measurement manufactured by complementary metal oxide semiconductor CMOS (Complementary Metal Oxide Semiconductor) technology; the printed circuit board 3 is made according to the chip packaging design and is used to collect and transmit real-time capacitance values; the bottom support layer 4 is used to support the printed circuit board 3, and is packaged together with the bottom of the convex contact layer through epoxy conductive glue to form the external circuit of the capacitor. The overall size of the device is 15mm*15mm*8mm.

[0039] The integrated circuit chip 2 and the printed circuit board 3 are welded together and fixed on the bottom support layer 4 by solder glue. The force sensing layer 1 and the bottom support layer 4 are connected together by epoxy conductive glue, and there is also a flying wire at the bottom of the printed circuit board 3 that is also connected to the inside of the force sensing layer 1 by conductive glue, forming an external path of the capacitor. The internal convex contact layer 6 is composed of a uniform arrangement of hemispherical cylindrical gradient convex contact structures arranged in an array. The gradient convex contact structure is a gradient arrangement. Specifically, the gradient convex contact structures in the center and the surroundings are closer to the integrated circuit chip 2 and increase toward the middle, and the overall distribution is annular.

[0040] The external insulating layer 5 and the internal convex contact layer 6 are both cylindrical structures, the bottom surface of the cylindrical structure is an arc curved toward the integrated circuit chip 2, the four sides of the cylindrical structure are sheet-shaped and form a square opening, each gradient convex contact structure is evenly spaced and arranged on the arc-shaped inner wall surface of the internal convex contact layer 6 in a square array, and the end of each gradient convex contact structure is vertically facing the integrated circuit chip 2; the external insulating layer 5 is mounted on the internal convex contact layer 6, and the internal convex contact layer 6 is mounted on the bottom support layer 4, and the integrated circuit chip 2 and the printed circuit board 3 are located between each gradient convex contact structure and the bottom support layer 4. The cross-sections of the integrated circuit chip 2, the printed circuit board 3 and the bottom support layer 4 are all square.

[0041] In the square array formed by the gradient convex contact structures of the internal convex contact layer 6, the distance between the end of the gradient convex contact structure located in the center and the integrated circuit chip 2 is smaller than the distance between the end of the gradient convex contact structure located at the surrounding edges and the integrated circuit chip 2, the distance between the end of the gradient convex contact structure located at the surrounding edges and the integrated circuit chip 2 is smaller than the distance between the remaining gradient convex contact structures and the integrated circuit chip 2, and the distance between the remaining gradient convex contact structures and the integrated circuit chip 2 gradually increases from the center of the square array toward the surrounding edges.

[0042] N*N capacitor plates are arrayed on the integrated circuit chip 2, and each gradient convex contact structure faces its own plurality of capacitor plates. The integrated circuit chip 2 as the lower plate is electrically connected to the printed circuit board 3. In the specific implementation, 7*7 gradient convex contact structures and 192*192 capacitor plates are used. Since the integrated circuit chip 2 has the characteristics of high resolution, it can fully obtain all the extrusion deformation and displacement of the convex contact structure of the force sensing layer 1 when it is subjected to force. This enables the robot tactile device of the present invention to have a high tactile resolution on the basis of a small size, and can sense the contact force distribution (contact force information and contact force position information).

[0043] The internal convex contact layer 6 of the force sensing layer 1 serves as an upper electrode plate and the integrated circuit chip 2 serves as a lower electrode plate, thereby forming a plurality of capacitors. When the force sensing layer 1 is subjected to an external contact force, the convex contact structure of the upper electrode plate will be displaced, contact the lower electrode plate, and deform at the same time. At this time, due to the change in the spacing between the two electrodes of the capacitor, the capacitance value collected by the capacitor also changes. The capacitance of these large numbers of capacitors is collected, and after analog-to-digital conversion and data processing, a grayscale image converted from the capacitance value is obtained. The deformation and contact condition of the force sensing layer 1 can be reflected through the radius size, center position, number and other information of the circle in the grayscale image, that is, the external contact force information and contact position information are indirectly reflected. The greater the external contact force, the smaller the spacing between the upper and lower electrodes, the larger the radius of the circle, the larger the capacitance value, the larger the grayscale value, and the brighter the circle.

[0044] When the force sensing layer 1 is displaced by the normal force, the convex contact structure closer to the electrode will first contact the lower electrode and deform. When the deformation accumulates to a certain amount, other convex contact structures farther away from the electrode will begin to contact the electrode. The gradient distribution of the convex contact structure allows these convex contact structures to deform in batches and reach saturation. While ensuring the accuracy of the device, the range of the device is greatly improved. In addition, when the force sensing layer 1 is subjected to normal force and tangential force at the same time, the gradient convex contact structure ensures that when receiving different normal forces, there will always be individual convex contact structures that have not reached deformation saturation and can be offset under the action of tangential force, which greatly improves the device's perception of tangential force.

[0045] The tactile recognition method of the robot tactile device based on the gradient convex tactile structure of the present invention is specifically as follows:

[0046] 1) Build a calibration platform and install it on a horizontal stable platform 16, and install the robot tactile device on the top of the calibration platform through the bottom support layer 4. The calibration platform includes a first connecting member 7, a second connecting member 9, a rigid six-dimensional force sensor 11 and a fixing member 12. The robot tactile device, the first connecting member 7, the second connecting member 9, the rigid six-dimensional force sensor 11, the fixing member 12 and the stable platform 16 are arranged in sequence from top to bottom. The robot tactile device and the first connecting member 7 are connected by a plurality of first bolts 8, the first connecting member 7 and the second connecting member 9 are connected by a plurality of second bolts 10, and the rigid six-dimensional force sensor 11 and the fixing member 12 are connected by a plurality of third bolts 13. The sensor 11 adopts a commercial rigid six-dimensional force sensor nano17.

[0047] 2) applying different contact forces to the force sensing layer 1 of the robot tactile device through the spherical indenter 15 at the end of the robotic arm 14, and obtaining the true value of the six-dimensional force information of the contact force at the current moment through the calibration platform at each moment of the application process of each contact force, and obtaining the true value of the contact position information of the contact force at the current moment during the application process through the spherical indenter 15, and obtaining the various capacitance values ​​generated by the various gradient convex contact structures of the force sensing layer 1 at the current moment being pressed on the various capacitor plates on the integrated circuit chip 2 through external data acquisition equipment and the printed circuit board 3 of the robot tactile device, thereby converting them into a grayscale image of the capacitance value at the current moment, and using the six-dimensional force information of the contact force at the current moment as a label of the grayscale image of the capacitance value at the current moment, and constructing each grayscale image of the capacitance value with the label of the six-dimensional force information as a six-dimensional force data set, and using the true value of the contact position information of the contact force at the current moment as a label of the grayscale image of the capacitance value at the current moment, and constructing each grayscale image of the capacitance value with the contact position information as a position data set.

[0048] When a contact force is applied to the robot tactile device, each gradient convex touch structure of the robot tactile device moves downward and gradually contacts each capacitor plate on the integrated circuit chip 2. At each moment of the contact force application process, the capacitance value of each capacitor plate is transmitted to the external data acquisition device in real time via the printed circuit board 3, and then each capacitance value is converted into a capacitance value grayscale image after analog-to-digital conversion and normalization processing. The capacitance value grayscale image includes a number of circles with array distribution with brightness. The position of each circle is the position where the gradient convex touch structure contacts the capacitor plate, that is, the contact position information of the contact force. The number of each circle is the number of the gradient convex touch structures contacting the capacitor plate, that is, the contact area of ​​the contact force. The size of each circle is the contact area where the gradient convex touch structure contacts the capacitor plate, that is, the contact pressure of the contact force. The contact area and contact pressure of the contact force serve as the contact force information.

[0049] When a contact force is applied to the robot tactile device, when the various gradient convex contact structures of the force sensing layer 1 are displaced downward by the normal force of the contact force, the various gradient convex contact structures closer to the integrated circuit chip 2 first contact the integrated circuit chip 2 in turn and deform, and then the other various gradient convex contact structures farther away from the integrated circuit chip 2 contact the integrated circuit chip 2 in turn and deform; when the various gradient convex contact structures of the force sensing layer 1 are simultaneously subjected to the normal force and tangential force of the contact force, the gradient convex contact structures with larger deformation do not deviate, while other gradient convex contact structures with smaller deformation or no deformation deviate, thereby sensing the tangential force.

[0050] 3) Input the six-dimensional force data set into the convolutional neural network (CNN) for training to obtain the trained convolutional neural network (CNN).

[0051] 4) The robot tactile device is integrated into the fingertips of the robot. When the robot drives the robot tactile device to contact an object, the capacitance values ​​generated when each gradient convex contact structure of the force sensing layer 1 is pressed onto each capacitor plate on the integrated circuit chip 2 are obtained in real time through external data acquisition equipment and the printed circuit board 3 of the robot tactile device, and converted into a continuous capacitance value grayscale image; for each capacitance value grayscale image, the capacitance value grayscale image is input into the trained convolutional neural network CNN for processing, and the trained convolutional neural network CNN outputs the six-dimensional force information when contacting the object after processing, and the capacitance value grayscale image and the position data set are processed by the K nearest neighbor classification algorithm KNN to obtain the contact position information when contacting the object, and finally complete the tactile recognition.

[0052] By integrating the robot tactile device on the fingertips of the robot dexterous hand 17, a stable grasping based on multi-modality can be realized. The stable grasping based on multi-modality means that before the formal grasping, the stability of the grasping is judged by sensing information through different robot tactile devices in a "pre-grasping" manner, so as to decide whether to continue grasping or adjust a new grasping posture. In the application of the present invention, in the pre-grasping stage, the stability of the grasping is judged by the contact form and contact position sensed by the robot tactile device. The contact situation of the fingertips of the robot dexterous hand 17 can be judged according to the contact position information and contact form information obtained in real time, and the grasping force and grasping posture of the robot dexterous hand 17 can be adjusted in real time, which greatly improves the stability of the grasping. However, grasping is usually not the ultimate goal of robot operation. After stable grasping, it is often hoped to complete some operation tasks by changing the posture and position of the grasped object in the hand, taking bolt tightening as an example. Under the premise of stably grasping the bolt, at the initial stage of the bolt being screwed in, the torque information sensed by the force tactile perception of the robot tactile device can be used to determine whether the bolt has an axial deviation. If an axial deviation occurs, the grasping posture is readjusted to prevent damage to the workpiece. In the tightening stage, the contact position sensed by the force tactile perception of the robot tactile device is used to ensure that the bolt does not exceed the fingertip safety edge area when rotating at the fingertips, that is, the arc center area of ​​the force sensing layer 1, to ensure the stability and safety of the operation; at the same time, as the bolt is tightened, the grasping force of the fingertips is adjusted in real time according to the sensed six-dimensional force information to prevent slippage.

[0053] like Figure 2 As shown, when the robot tactile device is subjected to an increasing normal force, the force sensing layer 1 with a gradient structure is squeezed and displaced, and the gradient convex contact structure in the central area closest to the integrated circuit chip 2 first contacts the integrated circuit chip 2 and deforms; when the normal force continues to increase, the gradient convex contact structures around the device begin to contact the integrated circuit chip 2 and deform; and when the deformation of these gradient convex contact structures tends to saturation, the gradient convex contact structure in the middle begins to deform in contact. Such a force sensing layer 1 with a gradient structure not only increases the range of the robot tactile device, but also ensures the sensitivity of the normal force perception within the range, and does not reduce the accuracy of the device.

[0054] Secondly, when the device is subjected to different normal forces, the force sensing layer 1 with a gradient structure always has a part of the gradient convex contact structure that is not in a state of deformation saturation, which makes it possible for the unsaturated gradient convex contact structure to still deflect on the surface of the integrated circuit chip 2 when the device is subjected to a smaller tangential force. This greatly improves the perception of the robot tactile device for tangential forces under different normal forces, and can greatly improve the tangential force accuracy of the robot tactile device.

[0055] like Figure 3(a) and Figure 3 As shown in (b), in order to calibrate the six-dimensional force perception ability and contact position perception ability of the robot tactile device, a calibration platform is built. First, in order to connect the robot tactile device and the nano17 rigid six-dimensional force sensor 11 together and place them in the same Cartesian coordinate system, a sensor connection system is formed using three components: a first connector 7, a second connector 9 and a fixing member 12. The components are fixedly connected by bolts 8, 10, and 13 respectively. The connected sensor connection system is fixed on a stable platform 16, and a spherical pressure head 15 is connected to the end of the mechanical arm 14. During the calibration process, the mechanical arm 14 is controlled to drive the spherical pressure head 15 to perform cyclic pressurization at various positions on the surface of the robot tactile device. The true value of the six-dimensional force at each moment is provided by the nano17 rigid six-dimensional force sensor 11, and the true value of the contact position is provided by the end coordinate value of the mechanical arm. Through data processing, the grayscale image of the capacitance value collected by the robot tactile device is matched one by one with the true value of the six-dimensional force and the true value of the contact position, forming two data sets that can be used for machine learning.

[0056] like Figure 4 (a) Figure 4 (b) and Figure 4 As shown in (c), when the robot tactile device is touched by an external object, the KNN algorithm is used to compare the grayscale image of the capacitance value collected by the sensor with the data set of the contact position obtained by the calibration experiment, and the respective Euclidean distances are calculated to obtain the k smallest data, and the label with the highest frequency of occurrence among them is taken as the predicted value of the algorithm. Finally, the real-time visualization of the contact position results predicted by the sensor is realized, such as the upper left area, the center area and the lower right area. When the robot dexterous hand 17 wants to stably grasp objects of different sizes and shapes, and perform some subsequent dexterous operations similar to human hands, the position information of the object at the fingertips plays a very important role. It provides a basis for judging whether it can be grasped stably, and can also provide guidance for changing the posture and dexterous operation in the hand. The contact position perception ability of the device improves the operational dexterity and reliability of the robot hand.

[0057] like Figure 5 As shown, the robot tactile device of the present invention is miniaturized, has a simple and compact structure, and has good integrability. It can be integrated into the five fingertips of the robot's dexterous hand 17 to provide tactile information for the stable grasping and operation of the robot's dexterous hand 17.

[0058] like Figure 6 (a) Figure 6 (b) and Figure 6As shown in (c), when the robot hand 17 integrated with the robot tactile device of the present invention grasps a bolt, it can judge whether it can grasp the bolt stably through the six-dimensional contact force information sensed by the tactile device, and can judge the specific situation of the axis deviation of the bolt according to the torque of the three axes, which provides a basis for the subsequent adjustment of the posture of the robot hand 17 to grasp the bolt stably again. Figure 6 As shown in (b), when the robot hand 17 integrated with the robot tactile device of the present invention grasps the bolt and rotates around the y-axis, the sensed three-axis torque Ty will increase significantly, which is reflected in the line width of the torque Ty in the visualization diagram. Figure 6 As shown in (c), when the robot dexterous hand 17 integrated with the robot tactile device of the present invention rotates around the z-axis when grasping a bolt, Tz in the sensed three-axis torque will increase significantly, which is reflected in the line width of the Tz torque in the visualization diagram.

[0059] like Figure 7 As shown, when the robot tactile device is applied to the fingertips of the robot dexterous hand 17, objects of different shapes can be stably grasped. First, the fingertips of the robot dexterous hand 17 move, and the fingertips touch the object for pre-contact. Before the formal grasping, the robot tactile device will transmit the sensed contact position information and contact force information to the control module through the "pre-grasping" method. The control module will analyze the sensed information to determine whether the current contact situation can complete stable grasping and whether the contact area is the center area of ​​the tactile device. If it is determined that stable grasping cannot be achieved, the grasping posture and position of the robot dexterous hand 17 are adjusted according to the sensed contact position and contact force information, and the fingertips of the robot dexterous hand 17 are continuously controlled to move, and grasping is attempted again. If it is determined that stable grasping can be achieved, the robot hand is controlled to start grasping. After the fingers of the robot's dexterous hand 17 are adjusted to the appropriate position and posture, the grasping operation begins. During the grasping process, the robot's tactile device continues to sense the contact force information in real time and transmits it to the control module. The control module adjusts the grasping force of the fingertips in real time according to the sensed six-dimensional force information to avoid slippage and ensure the stability of the grasping.

[0060] like Figure 8As shown, when the robot tactile device is applied to the fingertips of the robot dexterous hand 17, the dexterous operation of tightening the bolt can be realized. First of all, in order to accurately screw the bolt into the appropriate position, it is crucial to stably grasp the head of the bolt, so the bolt is first stably grasped at the center of the fingertips through the above-mentioned control strategy, and then the robot dexterous hand 17 is moved to the appropriate position with the bolt through the mechanical arm. The two fingers of the robot dexterous hand 17 drive the bolt to rotate through lateral movement and start to move downward to screw it in. At the same time, the robot tactile device transmits the sensed six-dimensional force information and contact position information to the control module in real time, and adjusts the size of the fingertip grasping force in real time according to the sensed tangential force information to prevent slippage; the bolt is judged according to the sensed three-axis torque when being screwed in. During the tightening process, check whether the axis is offset. If so, abandon the operation to prevent damage to the workpiece, readjust the position and grasping posture of the robot hand 17 and restart the second operation, and drive the bolt to rotate downward through the gripper; judge the stability of the bolt at the fingertips based on the sensed contact position information. If the contact position is close to the edge of the fingertips, release the grip, adjust the posture of the robot hand 17, grasp the bolt in the center of the fingertips based on the stable grasping strategy, and continue to perform the bolt tightening operation. The fingertips of the robot hand 17 adjust the grasping force in real time. Finally, judge whether the tightening force of the bolt has reached the requirement based on the sensed six-dimensional force information, and complete the bolt tightening operation.

[0061] In view of the fact that the current robot dexterous hand 17 needs force tactile perception to complete some stable grasping and dexterous operations, the present invention has produced a miniaturized, compact, and high-resolution tactile device through structural design; the force sensing layer with a gradient structure is utilized to significantly improve the normal force range and tangential force accuracy of the robot tactile device; and an application method and control strategy for integrating this robot tactile device on the fingertips of the robot dexterous hand 17 to achieve stable grasping and dexterous operations of bolt tightening are proposed, thereby improving the reliability of the stable grasping and dexterous operations of the manipulator.

Claims

1. A robot tactile device based on a gradient convex touch structure, characterized in that: The invention comprises a force sensing layer (1), an integrated circuit chip (2), a printed circuit board (3) and a bottom support layer (4); the integrated circuit chip (2) and the printed circuit board (3) are sequentially stacked and arranged on one side of the bottom support layer (4) and are electrically connected to each other; the force sensing layer (1) covers the bottom support layer (4) and covers the integrated circuit chip (2) and the printed circuit board (3); the inner wall surface of the force sensing layer (1) and the integrated circuit chip (2) are spaced and face each other; the force sensing layer (1) comprises an external insulating layer (5) and an internal convex touch layer (6) provided with a plurality of gradient convex touch structures for improving the tactile normal force range and tangential force accuracy; the external insulating layer (5) and the internal convex touch layer (6) are stacked and arranged; each gradient convex touch structure on the internal convex touch layer (6) is located on the inner wall surface of the internal convex touch layer (6) and is spaced and face each other; the printed circuit board (3) is electrically connected to an external data acquisition device.

2. The robot tactile device based on the gradient convex touch structure according to claim 1, characterized in that: The external insulating layer (5) and the internal convex contact layer (6) are both tubular structures, the bottom surface of the tubular structure is an arc that bends toward the integrated circuit chip (2), the four side surfaces of the tubular structure are sheet-shaped and form square openings, and each gradient convex contact structure is evenly spaced and arranged on the arc-shaped inner wall surface of the internal convex contact layer (6) in a square array, and the end of each gradient convex contact structure is vertically facing the integrated circuit chip (2); the external insulating layer (5) is mounted on the internal convex contact layer (6), and the internal convex contact layer (6) is mounted on the bottom support layer (4), and the integrated circuit chip (2) and the printed circuit board (3) are located between each gradient convex contact structure and the bottom support layer (4).

3. The robot tactile device based on the gradient convex tactile structure according to claim 2, characterized in that: In the square array formed by the gradient convex contact structures of the internal convex contact layer (6), the distance between the end of the gradient convex contact structure located in the center and the integrated circuit chip (2) is smaller than the distance between the end of the gradient convex contact structure located at the edges and the integrated circuit chip (2), the distance between the end of the gradient convex contact structure located at the edges and the integrated circuit chip (2) is smaller than the distance between the remaining gradient convex contact structures and the integrated circuit chip (2), and the distance between the remaining gradient convex contact structures and the integrated circuit chip (2) gradually increases from the center of the square array to the edges.

4. The robot tactile device based on the gradient convex tactile structure according to claim 2, characterized in that: The outer insulating layer (5) is made of non-conductive silicone material, and the inner convex contact layer (6) is made of conductive silicone material.

5. The robot tactile device based on the gradient convex tactile structure according to claim 1, characterized in that: N*N capacitor plates are arrayed on the integrated circuit chip (2), and each gradient convex contact structure faces a plurality of respective capacitor plates.

6. The tactile recognition method of a robot tactile device based on a gradient convex tactile structure according to any one of claims 1 to 5, characterized in that: include: 1) constructing a calibration platform and installing it on a horizontal stable platform (16), and installing the robot tactile device on the top of the calibration platform through a bottom support layer (4); 2) applying different contact forces to the force sensing layer (1) of the robot tactile device through a spherical indenter (15) at the end of the robot arm (14); for each moment of the application process of each contact force, obtaining the true value of the six-dimensional force information of the contact force at the current moment through the calibration platform; obtaining the true value of the contact position information of the contact force at the current moment during the application process through the spherical indenter (15); obtaining the capacitance values ​​generated by the various gradient convex contact structures of the force sensing layer (1) being pressed onto the various capacitor plates on the integrated circuit chip (2) through an external data acquisition device and a printed circuit board (3) of the robot tactile device at the current moment, thereby converting the capacitance values ​​into a grayscale image of the capacitance value at the current moment; using the six-dimensional force information of the contact force at the current moment as a label of the grayscale image of the capacitance value at the current moment; constructing each grayscale image of the capacitance value with the six-dimensional force information label as a six-dimensional force data set; using the true value of the contact position information of the contact force at the current moment as a label of the grayscale image of the capacitance value at the current moment; and constructing each grayscale image of the capacitance value with the contact position information label as a position data set; 3) Inputting the six-dimensional force data set into the convolutional neural network CNN for training to obtain a trained convolutional neural network CNN; 4) A robot tactile device is integrated into the fingertips of the robot. When the robot drives the robot tactile device to contact an object, the capacitance values ​​generated when each gradient convex contact structure of the force sensing layer (1) is pressed onto each capacitor electrode on the integrated circuit chip (2) are obtained in real time through an external data acquisition device and a printed circuit board (3) of the robot tactile device, and converted into a continuous capacitance value grayscale image; for each capacitance value grayscale image, the capacitance value grayscale image is input into a trained convolutional neural network (CNN) for processing, and the trained convolutional neural network (CNN) outputs six-dimensional force information when contacting the object after processing, and the capacitance value grayscale image and the position data set are processed using a K nearest neighbor classification algorithm (KNN) to obtain contact position information when contacting the object, and finally complete tactile recognition.

7. The tactile recognition method of the robot tactile device based on the gradient convex tactile structure according to claim 6, characterized in that: In the step 1), the calibration platform comprises a first connecting member (7), a second connecting member (9), a rigid six-dimensional force sensor (11) and a fixing member (12), and the robot tactile device, the first connecting member (7), the second connecting member (9), the rigid six-dimensional force sensor (11), the fixing member (12) and the stabilizing platform (16) are arranged in sequence from top to bottom.

8. The tactile recognition method of the robot tactile device based on the gradient convex tactile structure according to claim 6, characterized in that: In the step 2), when a contact force is applied to the robot tactile device, each gradient convex touch structure of the robot tactile device moves downward and gradually contacts each capacitor plate on the integrated circuit chip (2). At each moment of the contact force application process, the capacitance value of each capacitor plate is transmitted to an external data acquisition device in real time via the printed circuit board (3), and then each capacitance value is converted into a capacitance value grayscale image after analog-to-digital conversion and normalization processing. The capacitance value grayscale image includes a plurality of circles with array distribution with brightness. The position of each circle is the position where the gradient convex touch structure contacts the capacitor plate, that is, the contact position information of the contact force. The number of each circle is the number of the gradient convex touch structures contacting the capacitor plate, that is, the contact area of ​​the contact force. The size of each circle is the contact area where the gradient convex touch structure contacts the capacitor plate, that is, the contact pressure of the contact force. The contact area and contact pressure of the contact force serve as the contact force information.

9. The tactile recognition method of the robot tactile device based on the gradient convex tactile structure according to claim 6, characterized in that: In the step 2), when a contact force is applied to the robot tactile device, when each gradient convex contact structure of the force sensing layer (1) is subjected to the normal force of the contact force and produces a downward displacement, each gradient convex contact structure closer to the integrated circuit chip (2) first contacts the integrated circuit chip (2) in sequence and is deformed, and then other gradient convex contact structures farther from the integrated circuit chip (2) contact the integrated circuit chip (2) in sequence and are deformed; when each gradient convex contact structure of the force sensing layer (1) is subjected to the normal force and the tangential force of the contact force at the same time, the gradient convex contact structure with a larger deformation does not produce a displacement, while other gradient convex contact structures with a smaller deformation or no deformation produce a displacement, thereby sensing the tangential force.

Images