A fusion sensor system for an underactuated five-fingered anthropomorphic hand

Abstract

The application relates to the field of automatic control, in particular to a fusion sensor system for an underactuated five-fingered anthropomorphic hand; the underactuated five-fingered anthropomorphic hand comprises a wrist and a palm rotatably connected to the wrist, a plurality of fingers capable of completing a gripping action are arranged on the palm, a sensor array board is installed at the palm center of the palm, a visual perception board is arranged on the lower side of the sensor array board, and a lens of the visual perception board collects visual information through a center hole of the sensor array board; the sensor array board is a flexible circuit board, the sensor array board is composed of a 5*5 proximity sensor array, and the proximity sensor realizes measurement of object proximity distance based on a reflection principle; the actual needs of environment control and adaptive grasping of the underactuated five-fingered anthropomorphic hand can be met.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automation control, in particular to a fusion sensor system for an underactuated five-fingered anthropomorphic hand. BACKGROUND

[0002] The underactuated five-fingered anthropomorphic hand is a mechanical hand design that imitates the structure and function of human hands, and its core feature is the "underactuated" mechanism; underactuation means that the number of actuators is less than the degrees of freedom, and through ingenious design, the passive adaptability of the mechanical structure is used to achieve complex grasping and operation tasks; however, in the actual use of the underactuated five-fingered anthropomorphic hand, the traditional global camera used in the use process of the anthropomorphic robot hand faces the problem of limited field of view due to occlusion, and the traditional visual closed-loop control strategy faces the problem that the grasping stage cannot be controlled in real time, and various problems caused by the loss of information due to the occlusion of the camera position. SUMMARY

[0003] The purpose of the present application is to provide a fusion sensor system for an underactuated five-fingered anthropomorphic hand, which can meet the actual needs of environmental control and adaptive grasping of the underactuated five-fingered anthropomorphic hand.

[0004] The purpose of the present application is achieved by the following technical solutions:

[0005] A fusion sensor system for an underactuated five-fingered anthropomorphic hand, comprising an underactuated five-fingered anthropomorphic hand, the underactuated five-fingered anthropomorphic hand comprising a wrist and a palm rotatably connected to the wrist, a plurality of fingers capable of completing a grasping action are arranged on the palm, a sensor array board is installed at the palm center of the palm, a visual perception board is arranged on the lower side of the sensor array board, and the lens of the visual perception board collects visual information through the center hole of the sensor array board.

[0006] The sensor array board is a flexible circuit board, and the sensor array board is composed of a 5*5 proximity sensor array, and the proximity sensor measures the proximity distance of an object based on the reflection principle.

[0007] The plurality of fingers are respectively a little finger, a ring finger, a middle finger, an index finger and a thumb, and a plurality of DC brushless motors for driving the little finger, the ring finger, the middle finger, the index finger and the thumb to move are arranged on the palm.

[0008] The palm is installed with a main control board, and the DC brushless motors and the sensor array board are driven and controlled and sensor data are collected through the main control board, and the sensor array board and the main control board are connected by FPC flexible wires to communicate in I2C mode.

[0009] The visual perception board transmits the processed visual information to the main control board through wireless Bluetooth.

[0010] The micro control unit (MCU) integrated on the master control board is used to process raw data of the sensor array board, communicate with the upper computer and control the movement of multiple fingers, and the master control board receives the visual information transmitted by Bluetooth and outputs the extracted visual features after processing;

[0011] When the underactuated five-fingered artificial hand is far away from the object, the end-side reasoning is performed according to the visual perception board, the original image data collected by the lens are input, the original image data contain three-channel RGB images of the target object and surrounding environment information, the original image features are extracted and classified and aggregated through offline visual perception board processing at the end-side, the semantic perception classification information of the target object and the surrounding environment and the relative position relationship in the pixel coordinate system compared with the palm are obtained, and the corresponding processing results are transmitted to the master control board through Bluetooth for optimization and then transmitted to the upper computer through the universal asynchronous receiver-transmitter (UART).

[0012] When the palm reaches the minimum detection distance threshold of the sensor array board, the depth information of each point of the array is obtained through interpolation based on the raw data of the sensor array board through a fitting function on the basis of the features obtained in the first stage of visual information processing of the visual perception board, the surface contour features of the point array collected are obtained through morphological processing and fitting, the visual features of the visual perception board are transmitted through Bluetooth, the sensor array board is transmitted through the FPC flexible line in the I2C communication mode, and the two are fused at the feature level in the master control board, and finally the output results reflect the relative pose relationship between the underactuated five-fingered artificial hand and the target object, which is transmitted to the upper computer through the universal asynchronous receiver-transmitter (UART).

[0013] When the underactuated five-fingered artificial hand is close to the object, the visual information density of the visual perception board is greatly reduced due to the short distance, at this time, the various processed features obtained through visual information processing are no longer significant, and the distance change of the short distance based on the raw data of the sensor array board is more sensitive, when the visual information cannot obtain enough processed features, the real-time visual information no longer plays a role, at this time, the surface contour features obtained by the point array depth information are compared with the features perceived in the first stage, and the two are matched and approximated in the master control board, and finally the output results reflect the relative pose relationship between the underactuated five-fingered artificial hand and the target object, which is transmitted to the upper computer through the universal asynchronous receiver-transmitter (UART).

[0014] The palm is a hollow structure, the sensor array board is fixedly connected to the shell of the palm through the positioning hole, and the master control board is fixed to the shell on the back of the palm.

[0015] The beneficial effects of the present application are:

[0016] The application is based on the actual demand of environment control and adaptive grasping of the underactuated five-fingered anthropomorphic hand, and designs the underactuated five-fingered anthropomorphic hand with high sensitivity and fusion of visual-proximity sensor array system based on the principle of mechatronics design.

[0017] The sensing system comprises 5*5 equidistantly arranged infrared proximity sensors, a visual perception board and a main control board; the visual information of the system can effectively improve the distinguishing perception ability of the anthropomorphic mechatronic artificial hand to the target object during grasping in a complex environment through end-side information processing; the proximity information makes up for the low information density of the visual information at a close distance, improves the perception ability of the underactuated five-fingered anthropomorphic hand to the grasping space, and improves the operation performance of the underactuated five-fingered anthropomorphic hand.

[0018] The sensor information processing circuit comprises a sensor array board, a visual perception board and a main control board, wherein the sensor array board is a flexible circuit board, the sensor array board is installed at the palm and comprises a 5*5 proximity sensor array, the middle part is hollow for placing the lens of the visual perception board, 25 sensors are controlled for data transmission through a multiplexing chip in the flexible board, the high-coverage deep data acquisition is realized in the limited resource space and installation space of the embedded chip in the hand, the collected data is communicated through an I2C mode and connected to the information processing integrated circuit board of the underactuated five-fingered anthropomorphic hand through a FPC flexible line, the visual information acquisition board is arranged below the flexible circuit board, the image data acquisition and processing are realized in the board, and the visual information acquisition board is connected to the control board of the underactuated five-fingered anthropomorphic hand through BLE Bluetooth, so as to effectively relieve the information processing pressure of the main control board and improve the efficiency of the overall control system.

[0019] The sensor control system can obtain the relative pose relationship between the target object and the underactuated five-fingered anthropomorphic hand through visual information processing at a long distance, effectively solves the limited field of view problem of the global camera in the use process of the anthropomorphic robot hand, improves the intelligent degree of the underactuated five-fingered anthropomorphic hand control, simplifies the complex coordinate transformation involved in the traditional visual scheme, enriches the traditional hand sensor information level, and improves the control performance of the underactuated five-fingered anthropomorphic hand control; the proximity array realizes the palm area depth information acquisition at a close distance, realizes the closed-loop control for the grasping target, compared with the traditional visual closed-loop control strategy which cannot realize real-time closed-loop control in the grasping stage and loses information due to the camera position shielding, effectively improves the grasping performance of the underactuated five-fingered anthropomorphic hand, widens the application scene of the underactuated five-fingered anthropomorphic hand, and improves the reliability of the mechatronic system. BRIEF DESCRIPTION OF DRAWINGS

[0020] The application will be further described in detail in combination with the drawings and specific implementation methods.

[0021] Figure 1It is a fusion sensor system structure schematic diagram for underactuated five-fingered anthropomorphic hand of the present application;

[0022] Figure 2 It is a side view of the fusion sensor system for underactuated five-fingered anthropomorphic hand of the present application;

[0023] Figure 3 It is a sensor array board structure schematic diagram of the present application;

[0024] Figure 4 It is a visual dominant stage algorithm framework diagram of the present application;

[0025] Figure 5 It is an array fusion stage algorithm framework diagram of the present application;

[0026] Figure 6 It is an array stage algorithm framework diagram of the present application.

[0027] In the figure: little finger 1; ring finger 2; middle finger 3; index finger 4; DC brushless motor 5; main control board 6; sensor array board 7; visual perception board 8; thumb 9; palm 10; wrist 11. DETAILED DESCRIPTION

[0028] The present application will be further described in detail below in combination with the drawings.

[0029] As Figures 1 to 6 shown, in order to realize the technical effect of "meeting the actual needs of underactuated five-fingered anthropomorphic hand in environmental control and adaptive grasping", the structure and function of a fusion sensor system for underactuated five-fingered anthropomorphic hand are described in detail below;

[0030] As Figure 1 and Figure 2 shown; a fusion sensor system for underactuated five-fingered anthropomorphic hand, comprising an underactuated five-fingered anthropomorphic hand, the underactuated five-fingered anthropomorphic hand comprising a wrist 11 and a palm 10 rotatably connected to the wrist 11, a plurality of fingers capable of completing a grasping action are arranged on the palm 10, a sensor array board 7 is installed at the palm 10, a visual perception board 8 is arranged on the lower side of the sensor array board 7, and the lens of the visual perception board 8 collects visual information through the center hole of the sensor array board 7; the shell of the palm 10 is designed as a hollow structure according to the configuration of the sensor array board 7, which can reduce the weight and signal interference of the signal acquisition circuit, the sensor array board 7 is fixedly connected to the shell of the palm 10 through the positioning hole, and the main control board 6 is fixed to the shell on the back of the palm 10 through the positioning hole;

[0031] Among them, the palm 10 is made of composite material, preferably organic high polymer material, which can reduce the weight of the overall system while meeting the rigidity requirements;

[0032] The sensor array board 7 is a flexible circuit board, and the sensor array board 7 is composed of a 5*5 proximity sensor array, and the proximity sensor measures the distance of an object based on a reflection principle;

[0033] The plurality of fingers are respectively a little finger 1, a ring finger 2, a middle finger 3, an index finger 4 and a thumb 9, and a plurality of DC brushless motors 5 for driving the little finger 1, the ring finger 2, the middle finger 3, the index finger 4 and the thumb 9 to move are arranged on a palm 10; the finger body is fixed to the shell of the palm 10 and the shell of the back of the palm 10 through bolts;

[0034] The palm 10 is provided with a main control board 6, the DC brushless motor 5 and the sensor array board 7 are driven and controlled and sensor data are collected through the main control board 6, and the sensor array board 7 and the main control board 6 are connected through FPC flexible lines to communicate in an I2C mode;

[0035] The visual perception board 8 transmits the processed visual information to the main control board 6 through wireless Bluetooth;

[0036] The main control board 6 is integrated with a micro control unit MCU for processing raw data of the sensor array board 7, communicating with an upper computer and controlling the plurality of fingers to move, the main control board 6 receives visual feature extraction output results of the processed visual information transmitted through Bluetooth, a plurality of sensor data such as current and hall position sensors are arranged on the main control board 6, the motor is driven based on a corresponding control strategy to realize intelligent grabbing of the humanoid five-fingered hand, corresponding control instructions are transmitted to a working carrier such as a mechanical arm through a plurality of modes such as CAN / UART, and necessary control information is provided for closed-loop motion control of the working carrier facing a grabbing task;

[0037] Further, compared with the prior art, only a single sensor is usually used for information acquisition in the prior art, such as only a local visual perception board 8 is used, complete relative depth information of a target and the humanoid hand cannot be obtained, and when the distance is close, the visual field of the visual perception board 8 is limited because only a local target object can be seen in the visual field, so it is difficult to accurately judge the position, shape and state of the target object, and it is difficult to generate corresponding prompt information when the distance is close; the sensor array board 7 is further added, the sensor array board 7 can obtain a discrete depth sequence near the gripping range of the palm, the object depth condition of the array region can be obtained by analyzing the sequence data, the judgment of whether there is a target object and the relative pose relationship between the target object and the humanoid hand when the distance is close can be obtained based on the complete local visual target result, and the guided grabbing of the whole process of the closed loop is realized, the information collected by the visual perception board 8 and the sensor array board 7 is converged and transmitted on the main control board 6 through mutual cooperation of the visual perception board 8 and the sensor array board 7, and effective grabbing of the target object is realized;

[0038] The following is an example of a disabled person using a humanoid five-finger hand to operate. The specific implementation process is as follows:

[0039] As shown in Figure 4 , when the underactuated five-finger humanoid hand is far away from the object, the end-side reasoning is performed according to the visual perception board 8, the original image data collected by the lens is input, which contains three-channel RGB images of the target object and surrounding environment information, and the original image features are extracted and classified and aggregated after being processed by the offline visual perception board 8 at the end side, obtaining semantic perception classification information of the target object and the surrounding environment and relative position relationship in the pixel coordinate system compared with the palm 10. At this stage, the proximal sensor value is not significant, so the control is mainly based on visual information, and the corresponding processing results are transmitted to the main control board 6 through Bluetooth for optimization and then transmitted to the host computer through the universal asynchronous receiver-transmitter UART; the user is guided by voice to gradually approach the target using the underactuated five-finger humanoid hand.

[0040] As shown in Figure 5 , after the motion control reaches a certain distance, the palm 10 reaches the minimum detection distance threshold of the sensor array board 7. At this time, based on the original data of the sensor array board 7, the depth information of each point in the array is obtained by interpolation through the fitting function on the basis of the features obtained in the first stage of visual information processing of the visual perception board 8. The surface contour features of the point array collected are obtained by morphological processing and fitting. The visual features of the visual perception board 8 are transmitted through Bluetooth, and the sensor array board 7 is transmitted through FPC flexible line in I2C communication mode. The two are fused at the feature level in the main control board 6, and the final output result reflects the relative pose relationship between the underactuated five-finger humanoid hand and the target object, which is transmitted to the host computer through the universal asynchronous receiver-transmitter UART; the user is guided by voice to gradually approach the target using the underactuated five-finger humanoid hand.

[0041] As shown in Figure 6As shown, as the distance approaches, i.e. when the underactuated five-fingered anthropomorphic hand is close to the object, the visual information density of the visual perception board 8 is greatly reduced due to the close distance, at this time the various processed features obtained by the visual information processing are no longer significant, and the distance change of the close distance based on the original data of the sensor array board 7 is more sensitive, when the visual information cannot obtain enough processed features, the real-time visual information no longer plays a role, at this time the surface contour features obtained by the dot matrix depth information are mainly compared with the features perceived in the first stage, and the two are matched and approximated in the main control board 6, and the final output result reflects the relative pose relationship between the underactuated five-fingered anthropomorphic hand and the target object containing details, which is transmitted to the upper computer through the universal asynchronous receiver-transmitter UART; through voice guidance, the user uses the anthropomorphic five-fingered hand until the palm 10 of the five-fingered hand reaches the closest distance to the target object and the relative pose of the five-fingered hand to the target object reaches the graspable range, and the MCU sends a control command to realize the closing of the little finger 1, the ring finger 2, the middle finger 3, the index finger 4 and the thumb 9, so as to realize the grasping of the underactuated five-fingered anthropomorphic hand;

[0042] Further, as shown in Figures 4 to 6 The sensor array board 7 and the visual perception board 8 are integrated on the palm 10, visual perception is performed through the visual perception board 8, distance perception is performed through the sensor array board 7, the two are combined with each other, the sensor array board 7 obtains the judgment of the presence or absence of the target object and the relative pose relationship between the target object and the anthropomorphic hand based on the complete local visual target result of the visual perception board 8 when the distance is close; in the human-computer interaction scene, it is necessary to obtain the corresponding guide information and interact with the human in real time, the sensor array board 7 and the visual perception board 8 can guide the human-computer interaction in real time, and the sensor array board 7 and the visual perception board 8 are integrated in the palm 10, and the integration level is high; first, the existing technology based on vision for automatic grasping mostly combines the anthropomorphic hand with the mechanical arm, without considering human interaction, and the corresponding path planning from approaching to closing grasping can be obtained after coordinate transformation based on vision, and when used, only the mechanical arm and other equipment are controlled based on the path planning result to realize closed-loop control, but in the present application, human-computer interaction can be effectively completed through voice control, and the information feedback transmitted through the sensor array board 7 and the visual perception board 8 can guide the human-computer interaction in real time.

Claims

1. A fusion sensor system for an underactuated five-fingered anthropomorphic hand, comprising an underactuated five-fingered anthropomorphic hand, characterized in that: The underactuated five-fingered humanoid hand includes a wrist (11) and a palm (10) rotatably connected to the wrist (11), a plurality of fingers capable of completing a gripping action are arranged on the palm (10), a sensor array board (7) is installed at the palm center of the palm (10), a visual perception board (8) is arranged on the lower side of the sensor array board (7), the lens of the visual perception board (8) collects visual information through the center hole of the sensor array board (7), the sensor array board (7) obtains the judgment of whether there is a target object and the relative pose relationship between the target object and the humanoid hand based on the complete local visual target result of the visual perception board (8), and the information feedback transmitted by the sensor array board (7) and the visual perception board (8) can guide the human-computer interaction in real time; When the underactuated five-fingered humanoid hand is far away from the object, end-side reasoning is performed according to the visual perception board (8), original image data collected by the lens is input, the original image data contains three-channel RGB images of the target object and surrounding environment information, the original image features are extracted and classified and aggregated through offline visual perception board (8) processing at the end side, semantic perception classification information of the target object and the surrounding environment and the relative position relationship in the pixel coordinate system compared with the palm (10) are obtained, and the corresponding processing result is transmitted to the main control board (6) through Bluetooth for optimization and then transmitted to the upper computer through the universal asynchronous receiver-transmitter (UART). When the palm (10) reaches the minimum detection distance threshold of the sensor array board (7), the depth information of each point of the array is obtained through the fitting function based on the original data of the sensor array board (7) on the basis of the features obtained in the first stage visual information processing of the visual perception board (8), the point array depth information is morphologically processed and fitted to obtain the surface contour features of the point array collection, the visual features of the visual perception board (8) are transmitted through Bluetooth, the sensor array board (7) is transmitted through FPC flexible line in I2C communication mode, and the two are fused at the feature level in the main control board (6), and finally the output result reflects the relative pose relationship between the underactuated five-fingered humanoid hand and the target object, which is transmitted to the upper computer through the universal asynchronous receiver-transmitter (UART). When the underactuated five-fingered humanoid hand is close to the object, the visual information density of the visual perception board (8) is greatly reduced due to the short distance, at this time the various processed features obtained by visual information processing are no longer significant, and the distance change of the short distance based on the original data of the sensor array board (7) is more sensitive, when the visual information cannot obtain enough processed features, the real-time visual information no longer plays a role, at this time the surface contour features obtained by the point array depth information are compared with the features perceived in the first stage, and the two are matched and approximated in the main control board (6), and finally the output result reflects the relative pose relationship between the underactuated five-fingered humanoid hand and the target object, which is transmitted to the upper computer through the universal asynchronous receiver-transmitter (UART).

2. The fused sensor system for an underactuated five-fingered anthropomorphic hand according to claim 1, wherein: The sensor array board (7) is a flexible circuit board, the sensor array board (7) is composed of a 5*5 proximity sensor array, and the proximity sensor realizes the measurement of the object proximity distance based on the reflection principle.

3. The fused sensor system for an underactuated five-fingered anthropomorphic hand according to claim 2, wherein: The plurality of fingers are respectively a little finger (1), a ring finger (2), a middle finger (3), an index finger (4) and a thumb (9), and a plurality of DC brushless motors (5) for driving the little finger (1), the ring finger (2), the middle finger (3), the index finger (4) and the thumb (9) to move are arranged on a palm (10).

4. The fused sensor system for an underactuated five-fingered anthropomorphic hand according to claim 3, wherein: The palm (10) is provided with a main control board (6), the DC brushless motors (5) and a sensor array board (7) are driven and controlled and sensor data is collected through the main control board (6), the sensor array board (7) and the main control board (6) are connected through FPC flexible lines to communicate in an I2C mode.

5. The fused sensor system for an underactuated five-fingered anthropomorphic hand according to claim 4, wherein: The visual perception board (8) transmits the processed visual information to the main control board (6) through wireless Bluetooth.

6. The fused sensor system for an underactuated five-fingered anthropomorphic hand according to claim 5, wherein: The main control board (6) is integrated with a micro control unit MCU to process raw data of the sensor array board (7), communicate with an upper computer and control the plurality of fingers to move, and the main control board (6) receives the visual feature extraction output result of the processed visual information transmitted through Bluetooth.

7. The fused sensor system for an underactuated five-fingered anthropomorphic hand according to claim 1, wherein: The palm (10) is a hollow structure, the sensor array board (7) is fixedly connected to an outer shell of the palm (10) through positioning holes, and the main control board (6) is fixed to an outer shell on the back of the palm (10) through positioning holes.

Images