Humanoid robot based on vision and teleoperation

Abstract

The utility model discloses a humanoid robot based on vision and teleoperation, which relates to the field of transfer robots and comprises a humanoid robot body and a flexible handle. The humanoid robot body comprises a head part, a trunk part, operation arms and a first wireless receiving and transmitting module; the trunk part is provided with an image acquisition device; the operation arm comprises a plurality of support arms which are rotationally connected in sequence; an independent driving module is arranged in each supporting arm, and the number of the supporting arms is larger than or equal to four; the first wireless transceiver module is in communication connection with the operation arm and the image acquisition device; the dexterous handle comprises a shell, a handle and / or function keys, a display module and a second wireless receiving and transmitting module, the second wireless receiving and transmitting module is in communication connection with the handle and / or the function keys and the display module, and the second wireless receiving and transmitting module is in communication connection with the first wireless receiving and transmitting module. The mechanical arm is high in degree of freedom, large in operation space, good in flexibility and strong in immediacy sense of operators, and labor force is saved.

Description

Technical Field

[0001] The utility model belongs to the technical field of handling robots, and in particular relates to a humanoid robot based on vision and remote operation. Background Art

[0002] At present, the handling task can be completed by combining a three-axis or four-axis robot with an AGV trolley. The method of using a robot to carry objects is relatively mature, usually by guiding the robot to complete the handling task through a camera. However, the operating space of a single robot arm is limited. Although the combination of two arms and AGV expands the operating space, a three-degree-of-freedom or four-degree-of-freedom robot arm is usually used, with a small range of activity, and is developed for specific application scenarios. The function is single, the system portability is poor, and the operator's sense of presence for handling is weak. For example, the patent with publication number CN117355392A has only one robot arm, which can only grab objects through the end effector (suction cup), and the function is relatively single. The robot arm named Fetch and the robot arm named Freight of Fetch Robotics are both combined with AGV, but they only have one robot arm and a two-finger gripper, which can only clamp smaller objects. Although ARMAR-6 is equipped with two robot arms, its waist lacks degrees of freedom, resulting in a small space for its arm activities and cannot complete autonomous handling tasks. Utility Model Content

[0003] In order to solve the above problems, the utility model provides a humanoid robot based on vision and remote operation, with a high degree of freedom of the mechanical arm, a large operating space, good flexibility, a strong sense of presence for operators, and labor saving.

[0004] The utility model provides a humanoid robot based on vision and remote operation, comprising a humanoid robot body and an operating handle;

[0005] The humanoid robot body comprises:

[0006] head;

[0007] A trunk portion, arranged at the lower end of the head portion, wherein the trunk portion is provided with an image acquisition device;

[0008] An operating arm is arranged on the trunk, and the operating arm comprises a plurality of supporting arms connected in sequence and rotationally; wherein each supporting arm is respectively provided with an independent driving module, and the number of the supporting arms is greater than or equal to 4;

[0009] A dexterous hand, disposed at the lower end of the operating arm, the index of the dexterous hand being greater than or equal to 3;

[0010] A first wireless transceiver module is respectively connected to the operating arm and the image acquisition device for communication;

[0011] The operating handle comprises:

[0012] shell;

[0013] A handle and / or a function button, arranged on the housing;

[0014] A display module, arranged on the housing;

[0015] The second wireless transceiver module is respectively connected to the handle and / or the function button and the display module for communication, and the second wireless transceiver module is used to connect to the first wireless transceiver module for communication.

[0016] In an optional embodiment, the operating handle further includes a posture sensing module, and the posture sensing module includes any one or a combination of the following: a gyroscope, an accelerometer, a posture sensor, and an electronic compass.

[0017] In an optional embodiment, the image acquisition device includes a depth camera, and a receiving cavity for receiving the image acquisition device is provided on the torso, and the image acquisition device is rotatably installed in the receiving cavity.

[0018] In an optional embodiment, a plurality of the arms are connected in rotation in sequence, wherein a rotation plane at one end of each arm is perpendicular to the axial direction of the arm, and a rotation plane at the other end of each arm is parallel to the axial direction of the arm.

[0019] In an optional embodiment, the support arm includes at least one transverse support arm and at least five vertical support arms; one end of the transverse support arm is rotatably connected to the trunk, and the other end of the transverse support arm is rotatably connected to the upper ends of at least five vertical support arms that are rotatably connected in sequence;

[0020] In an optional embodiment, the vertical support arm includes a first vertical support arm and a second vertical support arm, the upper end of the first vertical support arm is adapted to the lower end of the second vertical support arm, and the lower end of the first vertical support arm is adapted to the upper end of the second vertical support arm; wherein the rotation surface of the upper end of the first vertical support arm and the rotation surface of the lower end of the second vertical support arm are respectively perpendicular to the axial direction of the support arm, and the rotation surface of the lower end of the first vertical support arm and the rotation surface of the upper end of the second vertical support arm are respectively parallel to the axial direction of the support arm.

[0021] In an optional embodiment, the number of the operating arms is two, and the two operating arms are respectively arranged on both sides of the torso.

[0022] In an optional embodiment, the first wireless transceiver module and the second wireless transceiver module include any one or a combination of the following: a Wi-Fi transceiver, a Z-Wave transceiver, an infrared transceiver, a ZigBee transceiver, a Bluetooth component, and a global mobile communication system terminal.

[0023] In an optional embodiment, the display module includes a touch display screen.

[0024] In an optional embodiment, the humanoid robot body further includes:

[0025] Lower limbs, arranged at the lower part of the trunk;

[0026] A foot, disposed at the bottom of the lower limb;

[0027] The lower limbs and / or the feet are used to drive the trunk to move.

[0028] The beneficial effects brought about by the technical solution provided by the embodiment of the utility model are as follows: the humanoid robot based on vision and remote operation of the utility model comprises a humanoid robot body and an operating handle, the humanoid robot body is provided with an operating arm, since the operating arm comprises a plurality of arms connected in sequence and an independent driving module is respectively provided in each arm, the number of the arms is greater than or equal to 4, thereby having a high degree of freedom and being able to reach any position within the working space, thereby increasing the operating space of the robotic arm; and since a dexterous hand is connected to the end of the robotic arm, the flexibility of the carrying operation can be improved; an operating handle is also provided, and a display module is provided on the operating handle, the display module can display the image collected by the image acquisition device installed on the torso, so that the operator controls the humanoid robot body through the handle to carry the target object to different positions, thereby enhancing the operator's sense of presence when controlling the humanoid robot body through the operating handle, and can meet a variety of application scenarios and save labor. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the structure of a humanoid robot based on vision and teleoperation provided by an embodiment of the utility model;

[0030] Figure 2 A system schematic diagram of a humanoid robot based on vision and teleoperation provided in an embodiment of the utility model;

[0031] Figure 3 Another system schematic diagram of a humanoid robot based on vision and teleoperation provided by an embodiment of the utility model;

[0032] Figure 4 A schematic structural diagram of a first vertical support arm provided in an embodiment of the utility model;

[0033] Figure 5 A schematic structural diagram of a second vertical support arm provided in an embodiment of the utility model.

[0034] In the figure: 10-head; 20-trunk; 30-operating arm; 301-upper end of the first vertical arm; 302-lower end of the first vertical arm; 303-upper end of the second vertical arm; 304-lower end of the second vertical arm; 341-first end of the fourth arm; 31-first arm; 32-second arm; 33-third arm; 34-fourth arm; 35-fifth arm; 36-sixth arm; 37-seventh arm; 38-connecting rod; 39-dexterous hand; 391-fingers; 40-lower limbs; 50-foot; 60-operating handle ;61-handle;62-function button;63-display module;64-second wireless transceiver module;65-handle control module;66-gesture sensing module;70-image acquisition module;71-depth camera;72-image acquisition control module;73-third wireless transceiver module;80-humanoid robot control module;81-first wireless transceiver module;83-robot control module;84-arm;85-drive module;86-EtherCAT master module;87-EtherCAT slave module;88-status sensor. DETAILED DESCRIPTION

[0035] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0036] See also Figure 1-Figure 5 The present embodiment provides a humanoid robot based on vision and remote operation, including a humanoid robot body and an operating handle 60; the humanoid robot body includes a head 10, a trunk 20, an operating arm 82, a dexterous hand 39 and a first wireless transceiver module 81. The trunk 20 is connected to the lower end of the head 10, and an image acquisition device 70 is arranged in the trunk 20, and the image acquisition device 70 is used to collect the handling operation image at the end of the operating arm 82. Two operating arms 82 are respectively arranged on both sides of the trunk 20, and the operating arm 82 includes a plurality of supporting arms 84 connected in sequence; wherein, each supporting arm 84 is respectively provided with an independent driving module 85 to drive the supporting arm 84 to move, and the number of supporting arms 84 is 7. Each operating arm 82 includes a dexterous hand 39, and the index of each dexterous hand 39 is 5. The first wireless transceiver module 81 is respectively connected to the operating arm 30 and the image acquisition device for communication; the operating handle 60 includes a shell and a handle 61 and function buttons 62, a display module 63, and a second wireless transceiver module 64 arranged on the shell. The second wireless transceiver module 64 is respectively connected to the handle 61, the function buttons 62 and the display module 63 for communication. The second wireless transceiver module 64 is used to communicate with the first wireless transceiver module 81.

[0037] The operating arm 82 of this embodiment includes 7 support arms 84, that is, the operating arm 82 is provided with at least 7 degrees of freedom, so that the operating arm 82 can reach any position within the working space. The end of the operating arm 82 is connected to a dexterous hand 39, and the dexterous hand 39 is provided with 5 indexes, so as to improve the flexibility when carrying the target object; there is also an operating handle 60, and the operating handle 60 is provided with a handle 61, a function button 62 and a reality module 63, which is convenient for the operator to control the humanoid robot body through the handle to carry the target object to different positions, thereby increasing the operator's sense of presence and saving labor; this embodiment can be used in a variety of scenarios and has a wide range of applications.

[0038] In a possible embodiment, Figure 2 As shown, the operating handle 60 includes a posture sensing module 66, a second wireless transceiver module 64, a handle, a function button 62 and a display module 63, and the posture sensing module 66, the function button 62 and the display module 63 are respectively connected to the second wireless transceiver module 64. Among them, the posture sensing module 66 is used to detect the speed, acceleration, angle and other information of the handle 61, and the handle control module 65 includes any one or a combination of the following: a gyroscope, an accelerometer, a posture sensor and an electronic compass. Among them, the gyroscope can detect the angle of the handle 61 when it is rotated, the accelerometer can detect the acceleration of the handle 61 when it is operated, the posture sensor can detect the posture (position and direction) of the handle 61, and the electronic compass can detect the direction of the handle 61. Among them, the display module 63 includes a touch display screen. The function button 62 may include a forward key, a backward key, a left shift key, a right shift key, a leg squat key, an operating arm 82

[0039] The second wireless transceiver module 64 collects the operation information sent by the posture sensing module 66, the function button 62 and the touch display screen, and sends the operation information to the first wireless transceiver module 61. The first wireless transceiver module 61 sends action information to the operating arm 82 according to the operation information and the image information collected by the image acquisition device 70 to control the operating arm 82 to carry the target object. Since the operating handle 60 of this embodiment is provided with the handle 61 and the touch display screen, it is convenient for the operator to operate the handle 61 to control the action of the humanoid robot body, and at the same time observe the RGB image taken by the humanoid robot body through the depth camera through the touch display screen, thereby enhancing the operator's sense of presence.

[0040] like Figure 3As shown, in another possible embodiment, the operating handle 60 also includes a handle control module 65, and the handle control module 65 is respectively connected to the second wireless transceiver module 64, the posture sensing module 66, and the display module 63. Among them, the handle control module 65 can adopt Xbox or a control module with the same principle as Xbox. In addition, the first wireless transceiver module 81 and the second wireless transceiver module 64 can also include any one or a combination of the following: Wi-Fi transceiver, Z-Wave transceiver, infrared transceiver, ZigBee transceiver, Bluetooth component and global mobile communication system terminal. Among them, the Z-Wave transceiver uses Z-Wave wireless communication technology, which is a dedicated, low-power, two-way protocol that provides reliable, low-bitrate wireless control for home and small business applications; the ZigBee transceiver uses ZigBee wireless communication technology, which is based on the IEEE 802.15.4 standard and has the characteristics of high performance, reliability, low energy consumption and low cost, supports wireless communication between multiple nodes, and adopts self-organizing network technology, which can automatically realize the establishment, maintenance and management of wireless networks. Preferably, Bluetooth communication or TCP / IP communication is adopted between the first wireless transceiver module 81 and the second wireless transceiver module 64 of this embodiment.

[0041] Similarly, the image acquisition module 70 can also communicate with the first wireless transceiver module 81 through wireless communication. Figure 3 As shown, in a possible embodiment, the image acquisition device includes a depth camera 71, an image acquisition control module 72 and a third wireless transceiver module 73. The depth camera 71 is used to obtain an RGB image with depth information and send the RGB image to the image acquisition control module 72. The image acquisition control module 72 can adopt an Ubuntu-based host computer, and the operating environment of the host computer can be Ubuntu20.04+ROS+openCV+pyrealsense2. The host computer processes the RGB image to obtain posture information, and sends the posture information to the first wireless transceiver module 81 through the third wireless transceiver module 73. The first wireless transceiver module 81 then sends the control information to the operating arm 30 to enable the operating arm 30 to perform corresponding actions. In the specific implementation, refer to Figure 1 The trunk 20 is provided with a receiving cavity for receiving the image acquisition device, and the image acquisition device is rotatably installed in the receiving cavity. Both ends of the depth camera 71 are rotatably connected to the receiving cavity through a rotating shaft (to achieve pitch shooting), and / or the depth camera 71 is installed in the receiving cavity through a telescopic mechanism.

[0042] Since the humanoid robot body 80 is in human form, in some possible embodiments, the humanoid robot body 80 also includes lower limbs 40 and feet 50, the lower limbs 40 are arranged at the lower part of the trunk 20, and the feet 50 are arranged at the bottom of the lower limbs 40, and the trunk 20 is driven to move by the lower limbs 40 and / or the feet 50.

[0043] In some possible embodiments, Figure 4 As shown, the humanoid robot body 80 includes a robot control module 83, an EtherCAT master module 86, an EtherCAT slave module 87 and a first wireless transceiver module 81. The humanoid robot body 80 also includes a state sensor 88 (for detecting the joint state) arranged in each joint (the joints here include at least the respective arms 84 of the operating arm 30, the respective joints of the foot 50 and the respective joints of the lower limbs 40), a driving module 85 and a state sensor 88. The driving module 85, the joint 84, the state sensor 88 and the EtherCAT slave module 87 are connected in sequence, and the driving module 85 is also connected to the EtherCAT master module 86. The EtherCAT master module 86 sends the action information (of the joint) to the driving module 85 to drive the joint module to move, the state sensor 88 detects the state information of the joint, and sends the state information to the EtherCAT master module 86 through the EtherCAT slave module 87, and the EtherCAT master module 86 sends the state information to the robot control module 83. The robot control module 83 may adopt a high-performance EtherCAT controller, such as an SMC-600 series controller.

[0044] In an optional embodiment, the arms 84 are all rotating arms, such as Figure 1 As shown, the support arm 84 includes two sequentially connected lateral support arms (i.e., the first support arm 31 and the second support arm 32) and five vertical support arms (the third support arm 33, the fourth support arm 34, the fifth support arm 35, the sixth support arm 36 and the seventh support arm 37); one end of the first support arm 31 is rotationally connected to the trunk 20, and the other end of the first support arm 31 is connected to one end of the second support arm 32; the other end of the second support arm 32 is rotationally connected to the upper ends of the five sequentially connected vertical support arms.

[0045] In a possible embodiment, the rotation plane at one end of each support arm 84 is perpendicular to the axial direction of the support arm 84, and the rotation plane at the other end of each support arm 84 is parallel to the axial direction of the support arm 84. Figure 4 and Figure 5 As shown, the vertical support arm includes a first vertical support arm ( Figure 4 The arm shown in ) and the second vertical arm ( Figure 5 The first vertical arm is L Type, the first vertical arm is LThe upper end 301 of the first vertical support arm is matched with the lower end 304 of the second vertical support arm, and the lower end 302 of the first vertical support arm is matched with the upper end 303 of the second vertical support arm; wherein, the rotation surface of the upper end 301 of the first vertical support arm and the rotation surface of the lower end 304 of the second vertical support arm are respectively perpendicular to the axial direction of the support arm 84, which can make the first vertical support arm rotate 360° around the axial direction, so as to facilitate the adjustment of the angles of the two operating arms 30 according to the position of the target object; the rotation surface of the lower end 302 of the first vertical support arm and the rotation surface of the upper end 304 of the second vertical support arm are respectively parallel to the axial direction of the support arm 84, which can realize the 360° rotation of the second vertical support arm around the horizontal axis, so as to facilitate the action of lifting the operating arm 30 to lift the target object. wherein, the position of the driving module 85 in the first vertical support arm and the second vertical support arm is as shown in FIG. Figure 4 and Figure 5 Shown by the dashed line.

[0046] Similarly, combined Figure 1 It can be seen that the connection mode between the first support arm 31 and the second support arm 32 is the same as the connection mode between the upper end 301 of the first vertical support arm and the lower end 304 of the second vertical support arm, and the second support arm 32 can rotate 360° around the axis, so as to realize the action of lifting the operating arm 30. In this embodiment, a plurality of support arms 84 are provided, thereby greatly increasing the degree of freedom and flexibility of the operating arm 30, so as to be applicable to target objects of various shapes.

[0047] The upper end of each dexterous hand 39 is rotatably connected to the lower ends of five vertical arms rotatably connected in sequence on the same side. In some embodiments, the dexterous hand 39 can be connected to the lower ends of the five vertical arms rotatably connected in sequence by a connecting rod 38. Each dexterous hand 39 is provided with five fingers 391. Each finger 391 includes at least two joints, that is, each finger 391 has at least one degree of freedom. The joints of each finger 391 are rotatably connected to achieve the bending of each finger 391 and the purpose of grasping objects. In specific implementation, the joints and the degrees of freedom of the joints of the dexterous hand 39 can be set to imitate the shape of a human hand to improve the flexibility during the carrying operation.

[0048] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the embodiments disclosed above are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

Claims

1. A humanoid robot based on vision and teleoperation, characterized in that: It includes a humanoid robot body and an operating handle; The humanoid robot body comprises: head; A trunk portion, arranged at the lower end of the head portion, wherein the trunk portion is provided with an image acquisition device; An operating arm is arranged on the trunk, and the operating arm comprises a plurality of supporting arms connected in sequence and rotationally; wherein each supporting arm is respectively provided with an independent driving module, and the number of the supporting arms is greater than or equal to 4; A dexterous hand, disposed at the lower end of the operating arm, the index of the dexterous hand being greater than or equal to 3; A first wireless transceiver module is respectively connected to the operating arm and the image acquisition device for communication; The operating handle comprises: shell; A handle and / or a function button, arranged on the housing; A display module, arranged on the housing; The second wireless transceiver module is respectively connected to the handle and / or the function button and the display module for communication, and the second wireless transceiver module is used to connect to the first wireless transceiver module for communication.

2. The humanoid robot based on vision and teleoperation according to claim 1, characterized in that: The operating handle also includes a posture sensing module, which includes any one or a combination of the following: a gyroscope, an accelerometer, a posture sensor and an electronic compass.

3. The humanoid robot based on vision and teleoperation according to claim 1, characterized in that: The image acquisition device comprises a depth camera. The torso is provided with a receiving cavity for receiving the image acquisition device. The image acquisition device is rotatably mounted in the receiving cavity.

4. The humanoid robot based on vision and teleoperation according to claim 1, characterized in that: The plurality of arms are connected in rotation in sequence, wherein a rotation plane at one end of each arm is perpendicular to the axial direction of the arm, and a rotation plane at the other end of each arm is parallel to the axial direction of the arm.

5. The humanoid robot based on vision and teleoperation according to claim 4, characterized in that: The multiple arms include at least one transverse arm and at least five vertical arms; one end of the transverse arm is rotationally connected to the trunk, and the other end of the transverse arm is rotationally connected to the upper ends of at least five vertical arms rotationally connected in sequence.

6. The humanoid robot based on vision and teleoperation according to claim 5, characterized in that: The vertical support arm includes a first vertical support arm and a second vertical support arm, the upper end of the first vertical support arm is matched with the lower end of the second vertical support arm, and the lower end of the first vertical support arm is matched with the upper end of the second vertical support arm; wherein the rotation surface of the upper end of the first vertical support arm and the rotation surface of the lower end of the second vertical support arm are respectively perpendicular to the axial direction of the support arm, and the rotation surface of the lower end of the first vertical support arm and the rotation surface of the upper end of the second vertical support arm are respectively parallel to the axial direction of the support arm.

7. The humanoid robot based on vision and teleoperation according to claim 1, characterized in that: The number of the operating arms is two, and the two operating arms are respectively arranged on both sides of the trunk.

8. The humanoid robot based on vision and teleoperation according to claim 1, characterized in that: The first wireless transceiver module and the second wireless transceiver module include any one or a combination of the following: a Wi-Fi transceiver, a Z-Wave transceiver, an infrared transceiver, a ZigBee transceiver, a Bluetooth component, and a global mobile communication system terminal.

9. The humanoid robot based on vision and teleoperation according to claim 1, characterized in that: The display module includes a touch display screen.

10. The humanoid robot based on vision and teleoperation according to claim 1, characterized in that: The humanoid robot body also includes: Lower limbs, arranged at the lower part of the trunk; a foot, disposed at the bottom of the lower limb; The lower limbs and / or the feet are used to drive the trunk to move.

Images