Bionic facial structure and high-fidelity facial expression interaction robot system

Abstract

The invention discloses a bionic facial structure and a high-fidelity facial expression interaction robot system. The system comprises an eyebrow control part, a lip and cheek control mechanism, an eye control part and a lower jaw control part, according to the invention, abundant multi-degree-of-freedom facial movement forms are realized, so that facial expressions are more natural; a transmission system for controlling facial muscles is simplified, the number of overall degrees of freedom of the mechanism is reduced, and the naturalness of expressions is improved through elastic restraint of the outer skin; the method can be applied to a deep man-machine interaction scene of the humanoid robot, and has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the fields of facial expression interaction and humanoid robots, and particularly to a bionic facial structure and a high-fidelity human face expression interaction robot system. Background Art

[0002] Humanoid robot technology can achieve precise simulation and reproduction of human facial expressions, and can be used for delicate human-machine interaction with emotional transmission functions, and can be applied to fields such as entertainment, medical treatment, and education.

[0003] In the prior art, humanoid robot technology has at least the following technical problems:

[0004] 1) The mechanical degrees of freedom of existing facial robots are generally insufficient, and it is difficult to achieve precise imitation of expressions by mechanical human faces;

[0005] 2) The motion forms of existing facial robots are relatively single, and the movement of the facial skin does not conform to the natural movement of the human face and lacks a natural sense;

[0006] 3) To improve this point, it is usually necessary to increase the mechanical degrees of freedom, thereby increasing the complexity of the mechanism.

[0007] Therefore, in order to solve the above technical problems and address the problems of insufficient degrees of freedom, single motion form, and lack of natural sense in the expression imitation of existing humanoid robots, the present invention proposes a new multi-degree-of-freedom bionic facial mechanical structure and a high-fidelity human face expression interaction system. Summary of the Invention

[0008] In view of this, the present invention provides a bionic facial structure and a high-fidelity human face expression interaction robot system.

[0009] To solve the above technical problems, the present invention adopts the following technical solutions:

[0010] A bionic facial structure and a high-fidelity human face expression interaction robot system, comprising: an eyebrow control part, a lip and cheek control mechanism, an eye control part, and a mandible control part; wherein,

[0011] The eyebrow control part includes two groups of left eyebrow head control mechanisms and left eyebrow tail control mechanisms, a left eyebrow head control servo, and a left eyebrow tail control servo. The inner frame joints of each group of the left eyebrow head control mechanism and the left eyebrow tail control mechanism are fixedly connected to the frame and the ends of the left eyebrow head control servo and the left eyebrow tail control servo;

[0012] Lip and cheek control mechanism, including a frame part, two groups of left cheek control mechanisms and a left cheek control mechanism, the connection between the mechanism and the skin tissue, and a fixing mechanism in the middle of the lower lip; the connection between the mechanism and the skin tissue, the two groups of left cheek control mechanisms, and the left cheek control mechanism are symmetrically distributed, the fixing mechanism in the middle of the lower lip is located on the central axis of the entire mechanism, and the fixing mechanism in the middle of the lower lip is connected to the frame through a shaft;

[0013] Eye control part, including two groups of eyelid mechanisms and two groups of eyeball mechanisms, where the eyelid mechanism includes: lower left eyelid, upper left eyelid, lower left eyelid control mechanism and upper left eyelid control mechanism; the eyeball mechanism includes a camera fixing slot, an eyeball rotation center ball, a right eyeball horizontal rotation control mechanism and a control link mechanism for eyeball vertical rotation, and the camera fixing slot is fixed to the frame through the eyeball rotation center ball and is also connected to the two groups of right eyeball horizontal rotation control mechanisms and the control link mechanism for eyeball vertical rotation fixed on its side;

[0014] Mandible control part, including the part below the teeth and the mandible control mechanism.

[0015] Preferably, the left eyebrow control mechanism is connected to the left eyebrow control servo, which is located on the lower side of the left eyebrow control servo and is connected to the shaft on the same side of the left eyebrow control servo on the frame.

[0016] Preferably, the left eyebrow tail control mechanism is connected to the left eyebrow tail control servo, which is located on the lower side of the left eyebrow tail control servo and is connected to the shaft on the same side of the left eyebrow tail control servo on the frame.

[0017] Preferably, the fixing mechanism in the middle of the lower lip is at the symmetry plane of the mechanism and in the middle of the two underactuated drive mechanisms.

[0018] Preferably, the left cheek control mechanism and the left cheek control mechanism are connected together by a shaft in the middle to form the connection between the mechanism and the skin tissue, and one end is respectively connected to the left eyebrow control servo and the left eyebrow tail control servo.

[0019] Preferably, the left eyebrow control mechanism and the left eyebrow tail control mechanism adopt a parallel link drive mechanism, and the ends of the left lip control mechanism and the fixing mechanism in the middle of the lower lip are coplanar in the initial position.

[0020] Preferably, the eyelid mechanism adopts link transmission, the two groups of eyeball mechanisms share a set of vertical transmission links, and the two groups of eyeball mechanisms are respectively connected to different horizontal transmission links.

[0021] Preferably, the vertical rotation control servo of the eyeballs is used to control the vertical rotation of the two eyeballs. The linkage control mechanism for the vertical movement of the two eyeballs drives the two eyeballs to perform the same vertical movement, and the horizontal rotation of the left and right eyeballs is independent of each other.

[0022] Preferably, both the lip movement and the cheek movement are realized by the pulling movement of the connecting rod.

[0023] Preferably, the lip and cheek control mechanism further includes a left part control mechanism of the lower lip and a left lip control servo. The left lip control servo pulls the left part control mechanism of the lower lip to drive the left lip to move, realizing the pulling movement of the lips.

[0024] The present invention has achieved the following technical effects compared with the prior art:

[0025] (1) The present invention is composed of a brow control part, a lip and cheek control mechanism, an eye control part and a mandible control part in combination, and can realize various movements. The connecting rod mechanism can realize that the controlled facial key points move approximately along the facial plane;

[0026] (2) The transmission system of the present invention is simple;

[0027] (3) The present invention reduces the overall degree of freedom of the mechanism by omitting the degree of freedom at appropriate positions;

[0028] (4) By using the left lip control mechanism and the middle fixed mechanism of the lower lip, the movement law is elastically constrained by the outer skin, and the expression is more natural. Description of the Drawings

[0029] Figure 1 is a cross-sectional view of the robot brow control mechanism of the present invention;

[0030] Figure 2 is the overall front view of the robot brow control mechanism of the present invention;

[0031] Figure 3 is the overall left view of the robot brow control mechanism of the present invention;

[0032] Figure 4 is a cross-sectional view of the robot brow end control mechanism of the present invention;

[0033] Figure 5 is the overall structure diagram of the robot lip and cheek control mechanism of the present invention;

[0034] Figure 6 is the view in the D direction of the robot lip and cheek control mechanism of the present invention;

[0035] Figure 7 is the detailed view of the robot lip control mechanism of the present invention;

[0036] Figure 8 It is a detailed view of the control mechanism of the robot's gills (hidden lips) of the present invention;

[0037] Figure 9 It is the overall front view of the control mechanism of the robot's eyes of the present invention;

[0038] Figure 10 It is the overall left view of the control mechanism of the robot's eyes of the present invention;

[0039] Figure 11 It is the overall top view of the control mechanism of the robot's eyes of the present invention;

[0040] Figure 12 It is a detailed view of the control mechanism of the robot's eyelids (hidden eyeballs) of the present invention;

[0041] Figure 13 It is the detailed view in the H direction of the control mechanism of the robot's eyelids (hidden eyeballs) of the present invention;

[0042] Figure 14 It is a detailed view of the control mechanism of the robot's eyeballs (hidden eyelids) of the present invention;

[0043] Figure 15 It is the detailed view in the G direction of the control mechanism of the robot's eyeballs (hidden eyelids) of the present invention;

[0044] Figure 16 It is the overall front view of the control mechanism of the robot's mandible of the present invention;

[0045] Figure 17 It is the overall left view of the control mechanism of the robot's mandible (hidden housing) of the present invention;

[0046] Figure 18 It is the overall top view of the control mechanism of the robot's mandible of the present invention;

[0047] Figure 19 It is the overall front view of the control mechanism of the robot's mandible (hidden housing) of the present invention;

[0048] Figure 20 It is the overall design structure diagram of the facial robot of the present invention.

[0049] In the figure, 1. left eyebrow head control mechanism; 2. left eyebrow head control servo; 3. left eyebrow tail control mechanism; 4. left eyebrow tail control servo; 5. connection between the mechanism and the skin tissue; 6. left cheek control mechanism; 7. left cheek control mechanism; 8. lower lip middle fixing mechanism; 9. left part of the lower lip control mechanism; 10. left lip control servo; 11. lower left eyelid; 12. upper left eyelid; 13. lower left eyelid control mechanism; 14. upper left eyelid control mechanism; 15. camera fixing slot; 16. eyeball rotation center ball; 17. linkage control mechanism for the vertical movement of the two eyeballs; 18. right eyeball horizontal rotation control mechanism; 19. left eyeball horizontal rotation control mechanism; 20. eyeball vertical rotation control servo; 21. control linkage mechanism for eyeball vertical rotation. 22. part below the teeth; 23. mandible control mechanism; 24. left part of the lip control mechanism. Specific implementation mode

[0050] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present invention.

[0051] The present invention discloses a bionic facial structure and a high-fidelity human face expression interaction robot system, including: an eyebrow control part, a lip and cheek control mechanism, an eye control part and a mandible control part; among them,

[0052] The eyebrow control part includes two groups of left eyebrow head control mechanisms 1 and left eyebrow tail control mechanisms 3, a left eyebrow head control servo 2, and a left eyebrow tail control servo 4. The inner frame joints of each group of left eyebrow head control mechanisms 1 and left eyebrow tail control mechanisms 3 are fixedly connected to the frame and the ends of the left eyebrow head control servo 2 and the left eyebrow tail control servo 4;

[0053] The lip and cheek control mechanism includes a frame part, two groups of left cheek control mechanisms 6 and a left cheek control mechanism 7, a connection 5 between the mechanism and the skin tissue, and a lower lip middle fixing mechanism 8; the connection 5 between the mechanism and the skin tissue, the two groups of left cheek control mechanisms 6, and the left cheek control mechanism 7 are symmetrically distributed. The lower lip middle fixing mechanism 8 is located on the central axis of the entire mechanism, and the lower lip middle fixing mechanism 8 is connected to the frame through an axis;

[0054] The eye control part includes two sets of eyelid mechanisms and two sets of eyeball mechanisms. The eyelid mechanism includes: the lower left eyelid 11, the upper left eyelid 12, the lower left eyelid control mechanism 13, and the upper left eyelid control mechanism 14. The eyeball mechanism includes the camera fixing slot 15, the eyeball rotation center ball 16, the right eyeball horizontal rotation control mechanism 18, and the control link mechanism 21 for controlling the vertical rotation of the eyeball. The camera fixing slot 15 is fixed to the frame through the eyeball rotation center ball 16, and is also connected to the two sets of right eyeball horizontal rotation control mechanisms 18 and the control link mechanism 21 for controlling the vertical rotation of the eyeball fixed on its side.

[0055] The mandible control part includes the part below the teeth 22 and the mandible control mechanism 23.

[0056] The left brow control mechanism 1 is connected to the left brow control servo 2. It is located on the lower side of the left brow control servo 2 and is connected to the shaft on the same side of the left brow control servo 2 on the frame.

[0057] The left brow tail control mechanism 3 is connected to the left brow tail control servo 4. It is located on the lower side of the left brow tail control servo 4 and is connected to the shaft on the same side of the left brow tail control servo 4 on the frame.

[0058] The middle fixing mechanism 8 of the lower lip is at the symmetry plane of the mechanism and in the middle of the two sets of underactuated drive mechanisms.

[0059] The left cheek control mechanism 6 and the left cheek control mechanism 7 are connected together by a shaft in the middle to form the connection 5 between the mechanism and the skin tissue, and one end is respectively connected to the left brow control servo 2 and the left brow tail control servo 4.

[0060] The left brow control mechanism 1 and the left brow tail control mechanism 3 adopt a parallel link drive mechanism. The end of the left lip control mechanism 24 and the end of the middle fixing mechanism 8 of the lower lip are coplanar in the initial position.

[0061] The eyelid mechanism uses link transmission. The two sets of eyeball mechanisms share a set of vertical transmission links, and each of the two sets of eyeball mechanisms is connected to a different horizontal transmission link.

[0062] The eyeball vertical rotation control servo 20 is used to control the vertical rotation of the two eyeballs. The interlocking control mechanism 17 for the vertical movement of the two eyeballs drives the two eyeballs to perform the same vertical movement, and the horizontal rotation of the left and right eyeballs is independent of each other.

[0063] The movement of the lips and the movement of the cheeks are both realized by the pulling movement of the links. The mandible control part realizes the opening and closing of the mandible through the pulling movement of the link mechanism.

[0064] The lip and cheek control mechanism further includes a left lower lip control mechanism 9 and a left lip control servo 10. The left lip control servo 10 pulls the left lower lip control mechanism 9 to drive the left lip to move, realizing the pulling movement of the lips.

[0065] As Figure 1 , Figure 2 , Figure 3 , Figure 4 shown, the structure diagram of the robot eyebrow control mechanism is displayed. The movement of the eyebrow part can be simplified as being carried out by the pulling movement of three nodes. For the left and right eyebrows respectively, the left brow control mechanism 1 and the left brow tail control servo 4 drive the connecting rod to pull the left brow control mechanism 1 and the left brow tail control mechanism 3 to precisely control the expressions of the eyebrow part.

[0066] Taking the left eyebrow as an example, the left brow control servo 2 drives the left brow control mechanism 1 to pull the left brow to move in multiple directions, and the left brow tail control servo 4 drives the left brow tail control mechanism 3 to pull the left brow tail to move coordinately with the brow. The combined movement of the left brow control mechanism 1 and the left brow control mechanism 1 can make the robot's eyebrows imitate a person to present various forms such as raising, frowning, and stretching.

[0067] As Figure 5 , Figure 6 , Figure 7 , Figure 8 shown, the structure diagram of the robot lip and cheek movement mechanism is displayed. Similar to the eyebrow part, the lip movement and the cheek movement are also realized by the pulling movement of the connecting rod. Taking the left cheek as an example, the connection point 5 between the mechanism and the skin tissue is connected to the external skin through a magnet. Two left cheek control servos drive the left cheek control mechanism 6 and the left cheek control mechanism 7 respectively to pull the skin at the connection point 5 between the mechanism and the skin tissue, thereby pulling the movement of the left side of the cheek. The right side is the same.

[0068] A total of 4 servos are used to precisely control the movement of the cheek part.

[0069] The lip movement is similar to the eyebrow movement. The middle fixed mechanism 8 of the lower lip is connected to the external skin through a magnet. After considering the actual expression of the human face, this point is simplified as a fixed point, and the left lip control servo 10 is used to pull the left lower lip control mechanism 9 to drive the left lip to move. The right side is the same. Finally, the pulling movement of the lips is realized. By adjusting the different angles of the left and right lips, the lips can present various forms such as different degrees of upward curling, downward flattening, and horizontal pulling.

[0070] As Figure 12 , Figure 13As shown, a simplified structural diagram of the eyelid device of the robot with the eyeball part hidden is presented. Similar to the horizontal rotation of the eyeball, the movements of the left and right eyelids are also independent of each other: the upper and lower eyelids of each eyeball are controlled by a single servo motor respectively.

[0071] Taking the left eyelid as an example, one control servo motor controls the opening and closing of the lower left eyelid 11 through the lower left eyelid control mechanism 13, and another control servo motor controls the opening and closing of the upper left eyelid 12 through the upper left eyelid control mechanism 14. The control of the upper and lower eyelids of the right eye is the same.

[0072] A total of four control servo motors are used for the eyelid mechanism, simplifying the movement of the eyelids to simple opening and closing. In the remaining degrees of freedom, the eyelids will automatically conform to the movement of the eyeball, which greatly simplifies the complexity of the mechanism without affecting the flexibility of the robot's eyes.

[0073] As Figure 14 、 Figure 15 shown, a simplified structural diagram of the eyeball device of the robot with the eyelid part hidden is presented, as well as the eyeball device finally installed on the skull. The movement of the eyeball can be regarded as a two-degree-of-freedom rotation around the eyeball rotation center sphere 16. Therefore, a two-axis gimbal is used to fix the eyeball, and a linkage mechanism is adopted at both ends of the eyeball to make it rotate around the gimbal.

[0074] The eyeball vertical rotation control servo motor 20 is used to control the vertical rotation of both eyeballs. By using the control eyeball vertical rotation control linkage mechanism 21 to drive both eyeballs to perform the same vertical movement, the complexity of the eyeball movement is greatly simplified.

[0075] The horizontal rotations of the left and right eyeballs are independent of each other. One servo motor controls the horizontal rotation of the left eyeball through the left eyeball horizontal rotation control mechanism 19, and another servo motor controls the horizontal rotation of the right eyeball through the right eyeball horizontal rotation control mechanism 18.

[0076] A groove is designed at the center of the eyeball to place the camera module for real-time visual recognition perception and receiving sound.

[0077] As Figure 16 、 Figure 17 、 Figure 18 、 Figure 19 shown, a structural diagram of the robot's jaw control part is presented.

[0078] Taking the left side as an example, the opening and closing of the left jaw part of the robot are achieved through the pulling movement of the jaw control mechanism 23. The same applies to the right side.

[0079] Through the joint cooperation of the left and right sides, the opening and closing actions of the robot's mouth can be achieved.

[0080] As Figure 20The figure is a diagram of the overall structure model of the present invention, wherein the solid shell part, namely the skull, chin, teeth, eyeballs, eyelids and connecting rod connectors are all manufactured by 3D printing according to the proportions of the human face; the connection between the mechanism and the external skin (such as components 5 and 8) is magnetically bonded to the artificial skin by magnets, and the traction between the skins drives the natural presentation of the entire skin expression; at the same time, since the human body has fewer muscle groups in the nose, the movement of the nose is approximated to be only affected by the natural traction of other facial parts, and no spontaneous movement will occur, so the nose is only designed as a fixed structure, and this part is hidden in the figure.

[0081] The present invention realizes the precise mobilization and control of the facial muscles of the facial robot based on the pulling movement of the connecting rod mechanism. The overall changes of the human face can express emotions. According to the layout of the facial muscles of the human body, the present invention installs a magnetic suction sheet under the skin and connects it to the internal actuator through the shell. For the overall face, the facial movement is controlled by the connecting rod drawn by the servo, and the skin, after reasonable design, will shrink or expand in the tangential direction while being pulled and compressed by the connecting rod, and has a muscle-like movement effect. In the remaining directions, the skin will slide on the shell. The human face has a large number of muscles, but when moving expressions, each muscle shows correlation. If the driving structure and the overall face are designed to operate in coordination in the mechanism, the actual degree of freedom of driving the face can be greatly reduced, thereby reducing the number of corresponding actuators and reducing costs.

[0082] The above description is only a preferred embodiment of the present invention and does not limit the technical scope of the present invention. Therefore, any slight modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A bionic facial structure and a high-fidelity human face expression interaction robot system, characterized in that, Comprising: An eyebrow control part, a lip and cheek control mechanism, an eye control part, and a mandible control part; wherein, The eyebrow control part includes two groups of left eyebrow head control mechanisms (1) and left eyebrow tail control mechanisms (3), a left eyebrow head control servo (2), and a left eyebrow tail control servo (4). The inner frame joints of each group of the left eyebrow head control mechanism (1) and the left eyebrow tail control mechanism (3) are fixedly connected to the frame and the ends of the left eyebrow head control servo (2) and the left eyebrow tail control servo (4); The lip and cheek control mechanism includes a frame part, two groups of left cheek control mechanisms (6) and a left cheek control mechanism (7), a connection part between the mechanism and the skin tissue (5), and a fixing mechanism (8) at the middle of the lower lip; the connection part between the mechanism and the skin tissue (5), the two groups of left cheek control mechanisms (6), and the left cheek control mechanism (7) are symmetrically distributed. The fixing mechanism (8) at the middle of the lower lip is located on the central axis of the entire mechanism, and the fixing mechanism (8) at the middle of the lower lip is connected to the frame through a shaft; The eye control part includes two groups of eyelid mechanisms and two groups of eyeball mechanisms. The eyelid mechanism includes: a lower left eyelid (11), an upper left eyelid (12), a lower left eyelid control mechanism (13), and an upper left eyelid control mechanism (14); the eyeball mechanism includes a camera fixing slot (15), an eyeball rotation center ball (16), a right eyeball horizontal rotation control mechanism (18), and a control link mechanism for controlling the vertical rotation of the eyeball (21). The camera fixing slot (15) is fixed to the frame through the eyeball rotation center ball (16), and is also connected to the two groups of right eyeball horizontal rotation control mechanisms (18) and the control link mechanism for controlling the vertical rotation of the eyeball (21) fixed on its side; The mandible control part includes a part below the teeth (22) and a mandible control mechanism (23).

2. The bionic facial structure and high-fidelity human face expression interaction robot system according to claim 1, wherein The left eyebrow head control mechanism (1) is connected to the left eyebrow head control servo (2), is located below the left eyebrow head control servo (2), and is connected to the frame through a shaft on the same side as the left eyebrow head control servo (2).

3. A bionic facial structure and a high-fidelity human face expression interaction robot system according to claim 1, characterized in that, The left eyebrow tail control mechanism (3) is connected to the left eyebrow tail control servo (4), is located below the left eyebrow tail control servo (4), and is connected to the frame through a shaft on the same side as the left eyebrow tail control servo (4).

4. A bionic facial structure and a high-fidelity human face expression interaction robot system according to claim 1, characterized in that The fixing mechanism (8) at the middle of the lower lip is at the symmetry plane of the mechanism and in the middle of the two groups of underactuated drive mechanisms.

5. The bionic facial structure and high-fidelity human face expression interaction robot system according to claim 1, characterized in that, The left cheek control mechanism (6) and the left cheek control mechanism (7) are connected together through a shaft in the middle to form the connection part between the mechanism and the skin tissue (5), and one end is respectively connected to the left eyebrow head control servo (2) and the left eyebrow tail control servo (4).

6. The bionic facial structure and high-fidelity human face expression interaction robot system according to claim 1, wherein The left eyebrow head control mechanism (1) and the left eyebrow tail control mechanism (3) adopt a parallel link drive mechanism, and the ends of the left lip control mechanism (24) and the ends of the fixing mechanism (8) at the middle of the lower lip are coplanar in the initial position.

7. The bionic facial structure and high-fidelity human face expression interaction robot system according to claim 1, wherein The eyelid mechanism adopts link transmission, the two groups of eyeball mechanisms share a set of vertical transmission links, and the two groups of eyeball mechanisms are respectively connected to different horizontal transmission links.

8. The bionic facial structure and high-fidelity human face expression interaction robot system according to claim 1, characterized in that The vertical rotation control servo (20) of the eyeballs is used to control the vertical rotation of the two eyeballs. The linkage control mechanism (17) for the vertical movement of the two eyeballs drives the two eyeballs to perform the same vertical movement, and the horizontal rotation of the left and right eyeballs is independent of each other.

9. The bionic facial structure and high-fidelity human face expression interaction robot system according to claim 1, wherein The movements of the lips and the cheeks are both realized by the pulling movement of the connecting rods. The lower jaw control part realizes the opening and closing of the lower jaw through the pulling movement of the connecting rod mechanism.

10. The bionic facial structure and high-fidelity human face expression interaction robot system according to claim 1, characterized in that, The lip and cheek control mechanism further includes a left lower lip control mechanism (9) and a left lip control servo (10). The left lip control servo (10) pulls the left lower lip control mechanism (9) to drive the left lip to move, realizing the pulling movement of the lips.

Images