Robotic humanoid playing an accordion and its human-robot interaction method of playing

Abstract

The application provides a body intelligent humanoid robot playing a mouth organ and a man-machine interactive playing method thereof, and comprises a body intelligent robot and a mouth organ. The body intelligent humanoid robot comprises a wheeled chassis walking mechanism, a lifting column leg mechanism is arranged above the wheeled chassis walking mechanism, and a robot trunk structure of a bionic upper body of a human is arranged on the lifting column leg mechanism. A touch display screen for man-machine interaction is arranged on the front of the trunk structure, and a flow valve, a gas pump, a driver and a gas path control circuit are arranged in the trunk structure. A robot head mechanism is arranged at the top of the robot trunk mechanism, the head mechanism is provided with a gas path pipeline connected with the chest, a camera and a sound box. Robot arm mechanisms are arranged on both sides of the robot trunk mechanism, and a five-fingered dexterous hand is arranged at the end of the mechanical arm of each robot arm mechanism. The mouth organ is provided with a double-row key structure, a gas path control valve, two gas pipes connected with the robot and another gas pipe for a user.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent humanoid robots, in particular, to a body-possessing intelligent humanoid robot capable of playing a melodeon and a man-machine interactive playing method thereof. BACKGROUND

[0002] With the rapid development of the field of body-possessing intelligent robots, a large number of robots have appeared in many practical application scenarios, such as intelligent education robots, emotional communication robots, cooking robots, and performance robots. With the increasing progress of robot precision, the degree of humanization that robots can achieve is getting higher and higher, and some robots with higher humanization needs have emerged, such as a piano-playing robot capable of playing the piano.

[0003] Patent document CN117207204A (application number: 202311489049.3) discloses a control method and control device for a piano-playing robot. After obtaining the key area image through the camera, the coordinates of each key in the camera coordinate system are determined according to the image feature points in the image, and the coordinates of each key in the camera coordinate system in the key template are determined according to the coordinates of the image feature points in the camera coordinate system. The relative position between the piano-playing robot and the keys is determined by the pose of the piano-playing robot during playing and the coordinates of each key in the camera coordinate system. The relative position between the piano-playing robot and the keys is compensated for error through a pre-determined error fitting function, and the mechanical arm of the piano-playing robot plays the compensated relative position. The relative position between the piano-playing robot and the keys is compensated for error through a pre-determined error fitting function, reducing the error of real-time positioning of the piano-playing robot to the keys.

[0004] Patent document CN116394277A (application number: 202310676079.9) discloses a human-like piano playing robot. The human-like piano playing robot can include a hand claw, an arm, a waist, a neck, a leg, a control system, and a base. The neck corresponds to two degrees of freedom to drive the head to perform rotation and pitching movements. The waist corresponds to two degrees of freedom to drive the upper body to perform rotation and pitching movements. The human-like piano playing robot can accurately locate the position of the keys through a visual perception unit, and can intelligently identify the content of the music score and automatically play the piano according to the music score. Through the two degrees of freedom of the waist and the head, and the control of the two mechanical arm poses, the piano playing robot in the specification can play the piano skillfully and intelligently.

[0005] Patent document CN113183166A (application number: 202110600930.0) discloses an electronic keyboard playing robot, including a human model torso, a first mechanism installed inside the human model torso for swinging arms, a second mechanism for controlling the left and right fingers to press the electronic keyboard, and a drive device for driving the first mechanism; the first mechanism includes a rotating shaft mounted on the right shoulder of the human model torso.

[0006] However, existing harmonica-playing robots cannot achieve human-computer interaction during anthropomorphic playing because their operating modes are only set according to human models, lacking a human-computer interaction mode. Therefore, how to invent a device and method for human-computer interaction is an urgent problem to be solved. This invention provides an embodied intelligent humanoid robot for playing the harmonica and its human-computer interaction playing method. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide an embodied intelligent humanoid robot that plays the harmonica and its human-computer interaction method for playing the harmonica.

[0008] According to the present invention, an embodied intelligent humanoid robot for playing a melodica includes: an embodied intelligent wheeled humanoid robot and a melodica 6 for human-computer interaction;

[0009] The embodied intelligent wheeled humanoid robot includes: a wheeled chassis walking mechanism 1, a lifting column leg mechanism 2, a robot torso mechanism 3, a robot head mechanism 4, a robot arm mechanism 5, and a five-fingered bionic dexterous hand 56.

[0010] The lifting column leg mechanism 2 is arranged above the wheeled chassis walking mechanism 1; the robot torso mechanism 3 is arranged above the lifting column leg mechanism 2; the robot head mechanism 4 is arranged at the top of the robot torso mechanism 3; the robot arm mechanism 5 is arranged on both sides of the robot torso mechanism 3; the five-fingered bionic dexterous hand 56 is installed at the end of the mechanical arm of each robot arm mechanism 5; the five-fingered bionic dexterous hand 56 is used to play the melodica;

[0011] An airflow generating module 36 is provided inside the robot's torso mechanism 3; the airflow generating module 36 is used to simulate the human body to generate and output airflow.

[0012] An air passage is provided inside the robot head mechanism 4. One end of the air passage is connected to the airflow generating device module 36, and the other end is connected to the lip trachea 47.

[0013] The air path control valve 61 is arranged at one end of the human-computer interaction played accordion 6, and the air path control valve 61 is connected with two air pipes, including a robot air pipe 7 and a user air pipe 8; wherein the robot air pipe 7 is connected with the lip air pipe 47, and is used for providing the performance of the intelligent wheeled humanoid robot; the user air pipe 8 is used for providing the performance of the user.

[0014] The air path control valve 61 is used for controlling the performance mode of the intelligent wheeled humanoid robot, including the autonomous performance of the accordion by the intelligent wheeled humanoid robot or the interactive performance with a person.

[0015] Preferably, the wheeled chassis walking mechanism 1 includes a bottom plate 11 and a chassis shell 12.

[0016] The bottom plate 11 includes an industrial computer 114, an embedded control board 111, a first depth camera 113, a laser radar 112, a driving wheel 115, a universal following wheel 118, and a driving motor 116.

[0017] The industrial computer 114 is connected with the embedded control board 111 through a communication interface; the embedded control board 111 is electrically connected with the driving wheel 115; and two driving wheels 115 are respectively connected with two driving motors 116.

[0018] The universal following wheel 118 is the third wheel of the wheeled chassis walking mechanism 1, and has no active driving capability.

[0019] The chassis shell 12 includes an emergency stop button 122, a start-stop button 121, and an ultrasonic sensor 123; wherein the ultrasonic sensor 123 is distributed on four surrounding surfaces of the chassis shell 12, and is electrically connected with the industrial computer 114.

[0020] The first depth camera 113, the laser radar 112, and the ultrasonic sensor 123 are all used for capturing environmental information.

[0021] Preferably, the industrial computer 114 collects the data of the first depth camera 113 and the data of the laser radar 112, and establishes an environmental model and a navigation model.

[0022] In the navigation motion control, the industrial computer 114 collects the data of the ultrasonic sensor 123, the data of the first depth camera 113, and the data of the laser radar 112, and controls the autonomous navigation of the robot, so as to avoid the collision of the robot with a moving object.

[0023] Preferably, the lifting column leg mechanism 2 includes a column outer square 21, a column inner square 22, a motor 23, a motor gear 231, a driving rod 24, and a sliding block 25.

[0024] The lifting column leg mechanism 2 is a square nested structure, the outer column square 21 is fixedly connected with the column butt joint plate 117 in the center area of the bottom plate, the inner column square 22 is nested in the outer column square 21, the motor 23 is installed in the outer column square 21, the motor gear 231 is connected with the driving rod 24, the sliding block 25 on the driving rod 24 is connected with the inner column square 22, under the driving of the motor 23, the sliding block 25 drives the inner column square 22 to lift, and the inner column square 22 is connected with the robot trunk mechanism 3 at the top end.

[0025] Preferably, the robot trunk mechanism 3 comprises a trunk shell 31, a trunk skeleton 32, a bending motor module 33, and a neck rotation mechanism 34.

[0026] The trunk skeleton 32 is fixedly arranged in the trunk shell 31, the bending motor module 33 is arranged at the bottom end of the trunk skeleton 32, and the neck rotation mechanism 34 is arranged at the upper end of the trunk skeleton 32; the robot head mechanism 4 is arranged at the upper end of the neck rotation mechanism 34; and the robot arm mechanism 5 is arranged on both sides of the trunk skeleton 32.

[0027] The trunk skeleton 32 is internally provided with an airflow generating device module 36; under the control of an air path control circuit and a driver, the airflow generating device module 36 generates airflow by means of an air pump, and the flow valve controls the airflow output, thereby simulating the required gas flow when a human plays an organ.

[0028] The bending motor module 33 comprises a bending motor 331, a lower groove 332, an upper groove 333, and a bearing 334; one end of the lower groove 332 is provided with the bearing 334 rotating passively, the bearing 334 rotating passively is connected with one end of the upper groove 333, the other end of the lower groove 332 is provided with the bending motor 331, and the bending motor 331 is connected with the other end of the upper groove 333; when the bending motor 331 rotates, the upper groove 333 rotates, and the rotation range is -90° to 90°.

[0029] A touch display screen 35 is further arranged in front of the trunk shell 31, and the touch display screen 35 is used for human-computer interaction, including setting a robot working mode.

[0030] The neck rotation mechanism 34 comprises a neck rotation motor 341, a shoulder connecting piece 342, and a neck rotation connecting piece 343; the neck rotation motor 341 is arranged on the shoulder connecting piece 342, and a convex end thereof is connected with the neck rotation connecting piece 343; when the neck rotation motor 341 rotates, the neck rotation connecting piece 343 rotates, and the shoulder connecting piece 342 is arranged on the trunk skeleton 32.

[0031] Preferably, the robot head mechanism 4 comprises a neck connecting piece 41, a nodding motor 42, a head fixing piece 43, a second depth camera 44, a head shell 45, a sound 46, a lip trachea 47;

[0032] The neck connecting piece 41 is arranged in the torso shell 31 and detachably connected with the torso framework 32. The nodding motor 42 is installed on the neck rotating connecting piece 343. The head fixing piece 43 is outside the nodding motor 42. The head shell 45 is arranged on the head fixing piece 43. The second depth camera 44 and the sound 46 are fixedly arranged on the head shell 45.

[0033] The second depth camera 44 is used to identify the key button and observe the situation of the robot playing the keyboard.

[0034] The sound 46 is used for music playing, voice playing of human-computer interaction, and abnormal prompt.

[0035] One end of the lip trachea 47 is connected with the airflow generating device module 36 through an air path pipeline, and the other end is connected with the robot trachea 7.

[0036] Preferably, the robot arm mechanism 5 comprises a mechanical arm, a six-dimensional force sensor 55, and a five-finger bionic dexterous hand 56.

[0037] One end of the mechanical arm is connected with the robot torso mechanism 3, and the other end is connected with the six-dimensional force sensor 55. The six-dimensional force sensor 55 is connected with the five-finger bionic dexterous hand 56.

[0038] The mechanical arm is composed of a first section mechanical arm 51, a second section mechanical arm 52, a third section mechanical arm 53, and a fourth section mechanical arm 54. The first section mechanical arm 51, the second section mechanical arm 52, the third section mechanical arm 53, and the fourth section mechanical arm 54 are connected in sequence, and each mechanical arm connection has one rotation degree of freedom. Each mechanical arm has a rotation degree of freedom, so that the mechanical arm realizes six degrees of freedom or seven degrees of freedom, and can simulate the free movement of human arm.

[0039] Each finger of the five-finger bionic dexterous hand 56 has at least two degrees of freedom, which can bend and straighten.

[0040] Preferably, the human-computer interaction playing accordion 6 is a double-row key structure, wherein the inner keyboard is used for robot playing, and the outer keyboard is used for user playing.

[0041] The gas path control valve 61 comprises: the outer channels of the first control valve 611 and the second control valve 612 are connected to the robot gas pipe 7 through a Y-shaped bifurcated pipe, and the outer channels of the third control valve 613 and the fourth control valve 614 are connected to the user gas pipe 8 through a Y-shaped bifurcated pipe; the inner channels of the second control valve 612 and the fourth control valve 614 are connected to the outer keyboard through a Y-shaped pipe, and the inner channels of the third control valve 613 and the first control valve 611 are connected to the inner keyboard through a Y-shaped pipe.

[0042] According to the present application, a method for human-robot interaction performance of a playing mouth organ embodied intelligent humanoid robot is provided, which comprises the following steps:

[0043] The robot performance mode is selected through the touch display screen 35, which comprises:

[0044] The control valve is used to realize various human-robot interaction modes, which comprises: when the first control valve 611 is opened, and the second control valve 612, the third control valve 613 and the fourth control valve 614 are closed, the robot gas path is connected to the inner keyboard gas path 616, and the robot performs alone;

[0045] When the first control valve 611, the third control valve 613 and the fourth control valve 614 are closed, and the second control valve 612 is opened, the robot gas path is connected to the outer keyboard gas path 615, and the robot provides the gas source, and the user plays together with the robot;

[0046] When the first control valve 611, the second control valve 612 and the fourth control valve 614 are closed, and the third control valve 613 is opened, the user gas path is connected to the inner keyboard gas path 616, and the robot plays;

[0047] When the first control valve 611, the second control valve 612 and the third control valve 613 are closed, and the fourth control valve 614 is opened, the user gas path is connected to the outer keyboard gas path 615, and the user provides the gas source, and the user plays alone;

[0048] When the second control valve 612 and the third control valve 613 are closed, and the first control valve 611 and the fourth control valve 614 are opened, the user gas path is connected to the outer keyboard gas path 615, and the robot gas path is connected to the inner keyboard gas path 616, and the user and the robot perform simultaneously.

[0049] Preferably, when the robot provides the gas source, the robot generates the gas source through the air pump of the air flow generating device module 36, and the flow valve controls the air flow output according to the human body exhalation amount simulated by the rhythm set by the music;

[0050] When the robot plays, the robot drives the five-finger bionic dexterous hand 56 above the keyboard through the robot arm mechanism 5, and presses the keyboard according to the rhythm simulated by the human finger bending.

[0051] Compared with the prior art, the present application has the following beneficial effects:

[0052] 1、The body intelligent wheeled humanoid robot adopts a wheeled chassis walking structure to realize autonomous motion control, increases a lifting column leg mechanism to realize height free adjustment, and increases a bending structure module to expand the motion space of the robot bionic human body motion performance;

[0053] 2、The mouth organ adopts a double-row key structure, sets an air path switching control valve, and realizes the performance of various human-computer interaction modes;

[0054] 3、The robot integrates an air flow generating device module capable of simulating human body air flow output, generates the required air flow and pressure of the mouth organ driving, and realizes the blowing of the mouth organ by the robot;

[0055] 4、The robot bionic human body upper body structure design, the arm structure can imitate human arm movement, the mechanical arm end integrates five-fingered dexterous hand, the dexterous hand imitates human hand to strike the mouth organ keyboard, and the mouth organ key performance can be realized;

[0056] 5、The robot can autonomously perform the mouth organ and interact with the person to perform through the cooperation between the structures. BRIEF DESCRIPTION OF DRAWINGS

[0057] Other characteristics, objects and advantages of the present application will become more apparent through reading the detailed description of the non-limiting embodiments made with reference to the following drawings:

[0058] Figure 1 It is a kind of body intelligent humanoid robot device schematic diagram of playing mouth organ.

[0059] Figure 2 It is a wheeled chassis walking mechanism schematic diagram.

[0060] Figure 3 It is a lifting column leg structure schematic diagram.

[0061] Figure 4 It is a robot trunk structure schematic diagram.

[0062] Figure 5 It is a head structure schematic diagram.

[0063] Figure 6 It is an arm structure schematic diagram.

[0064] Figure 7 It is a mouth organ component structure schematic diagram.

[0065] Figure 8 It is an air path control valve control schematic diagram.

[0066] Wherein, 1-wheel chassis walking mechanism, 2-lifting column leg mechanism, 3-robot torso mechanism, 4-robot head mechanism, 5-robot arm mechanism, 6-human-computer interaction playing accordion, 7-robot air pipe, 8-user air pipe, 11-bottom plate, 111-embedded control board, 112-laser radar, 113-first depth camera, 114-industrial computer, 115-driving wheel, 116-driving motor, 117-column butt joint plate, 118-omni-directional following wheel, 119-battery, 12-chassis shell, 121-start-stop button, 122-emergency stop button, 123-ultrasonic sensor, 21-column outer square, 22-column inner square, 23-motor, 231-motor gear, 24-driving rod, 25-sliding block, 31-torso shell, 32-torso framework, 33-bending motor module, 331-bending motor, 332-lower groove, 333-upper groove, 334-bearing, 34-neck rotation mechanism, 341-neck rotation motor, 342-shoulder connecting piece, 343-neck rotation connecting piece; 35-touch display screen, 36-air flow generating device module; 41-neck connecting piece, 42-nodding motor, 43-head fixing piece, 44-second depth camera, 45-head shell, 46-sound, 47-lip air pipe, 51-first section of mechanical arm, 511-first rotation degree of freedom, 52-second section of mechanical arm, 521-second rotation degree of freedom, 522-third rotation degree of freedom, 53-third section of mechanical arm, 531-fourth rotation degree of freedom, 532-fifth rotation degree of freedom, 54-fourth section of mechanical arm, 541-sixth rotation degree of freedom, 542-seventh rotation degree of freedom, 55-six-dimensional force sensor, 56-five-finger bionic dexterous hand, 561-first degree of freedom finger, 562-second degree of freedom finger, 563-thumb, 61-air path control valve, 62-outer keyboard, 63-inner keyboard, 611-first control valve, 612-second control valve, 613-third control valve, 614-fourth control valve, 615-outer keyboard air path, 616-inner keyboard air path. DETAILED DESCRIPTION

[0067] The application will be described in detail below with specific embodiments. The following examples will help those skilled in the art to further understand the application, but do not limit the application in any form. It should be pointed out that, for those skilled in the art, without departing from the concept of the application, a number of changes and improvements can be made. These all belong to the protection scope of the application.

[0068] Example 1

[0069] According to the present application, a playing accordion with a body intelligent humanoid robot is provided, as shown in the figure, including: a body intelligent wheeled humanoid robot and a human-computer interaction playing accordion; Figures 1 to 8 ​

[0070] The embodied intelligent wheeled humanoid robot comprises a wheeled chassis walking mechanism 1, a lifting column leg mechanism 2 is arranged above the wheeled chassis walking mechanism 1, a robot trunk mechanism 3 of a bionic human upper body is installed on the lifting column leg mechanism 2, a touch display screen 35 for human-computer interaction is installed on the front of the robot trunk mechanism 3, a flow valve, a gas pump, a driver and a gas path control circuit and other gas flow generating device modules 36 that can simulate human body to generate air flow output are arranged inside, a robot head mechanism 4 is arranged at the top of the robot trunk mechanism 3, the robot head mechanism 4 has a gas path pipeline connected with the gas flow generating device module 36 in the chest, a camera and a sound, robot arm mechanisms 5 are arranged on both sides of the robot trunk mechanism 3, and five-finger bionic dexterous hands 56 capable of playing a melodeon are arranged at the mechanical arm ends of each robot arm mechanism 5.

[0071] The melodeon 6 played by human-computer interaction is a double-row key structure, is provided with a gas path control valve 61 at one end and leads out two gas pipes, one of the gas pipes is connected with the robot, and the other gas pipe can be used by a user; the robot can play the melodeon autonomously or interactively play the melodeon with a person in various modes through the arrangement of the gas path control valve 61.

[0072] Specifically, the wheeled chassis walking mechanism 1 comprises a bottom plate 11, the upper end of which is provided with a chassis shell 12, the front end of which is provided with a window for a first depth camera 113 and a laser radar 112 to capture environmental information, for controlling the autonomous movement of the robot, and the top of which is provided with a window for a lifting column; the bottom plate 11 is provided with a battery 119, an industrial computer 114, an embedded control board 111, two drive wheels 115, the first depth camera 113 and the laser radar 112 for environmental modeling and autonomous navigation, the industrial computer 114 is connected to the embedded control board 111 through a communication interface, the embedded control board 111 is electrically connected to the drive wheels 115, the two drive wheels 115 are respectively connected to left and right drive motors 116, and the third wheel is a universal following wheel 118 without active driving capability. The chassis shell 12 is provided with an emergency stop button 122, a start-stop button 121 and ultrasonic sensors 123, the ultrasonic sensors 123 are distributed on the four surrounding surfaces of the shell and are electrically connected to the industrial computer, in the process of navigation modeling, the industrial computer 114 collects first depth camera data and laser radar data, and establishes an environmental model and a navigation model through a first fusion algorithm, in the process of navigation movement control, the industrial computer 114 collects ultrasonic sensor data, first depth camera data and laser radar data, and controls the autonomous navigation of the robot and avoids collision with moving objects through a second fusion algorithm; wherein the first fusion algorithm is Kalman filtering, Bayesian estimation, weighted average method, fuzzy logic, neural network, etc., and the second fusion algorithm is convolutional neural network, transform network, etc.

[0073] Specifically, the lifting column leg mechanism 2 is a square nested structure, the outer square 21 of the column is installed at the center of the bottom plate and is fixedly connected to the column butt joint plate 117; the inner square 22 of the column is nested in the outer square 21 of the column; the motor 23 is installed in the outer square 21 of the column, the motor gear 231 is connected to the drive rod 24, and the slider 25 on the drive rod 24 is connected to the inner square 22 of the column; under the driving of the motor 23, the slider 25 drives the inner square 22 of the column to realize lifting; the top end of the inner square 22 of the column is connected to the robot torso mechanism 3.

[0074] The robot trunk mechanism 3 comprises a trunk shell 31, a trunk skeleton 32, an airflow generating device module 36 and a touch display screen 35. The touch display screen 35 for human-computer interaction is installed in front of the trunk structure so that the user can perform human-computer interaction and set the robot working mode. The trunk skeleton 32 is fixedly arranged in the trunk shell 31. A bending motor module 33 is fixedly arranged at the bottom end of the trunk skeleton. The bending motor module 33 comprises a bending motor 331, a lower groove 332, an upper groove 333 and a bearing 334. The passive rotating bearing 334 is arranged on the left side of the lower groove 332. The passive rotating bearing 334 is connected with the left end of the upper groove 333. The bending motor 331 is arranged on the right side of the lower groove 332. One end of the bending motor 331 is connected with the upper groove 333. When the bending motor 331 rotates, the upper groove 333 rotates with a rotation range of -90° to 90°. The airflow generating device module 36 is arranged in the trunk skeleton 32. The air pump generates airflow under the control of the air path control circuit and the driver. The flow valve controls the airflow output to simulate the required gas flow when a person plays an organ. One air path pipeline of the airflow generating device module 36 is connected to the lips of the robot head.

[0075] Specifically, the robot head mechanism 4 comprises a neck connecting piece 41 which is arranged in the trunk shell 31 and detachably connected with the trunk skeleton 32. A nodding motor 42 is arranged on the neck rotating connecting piece 343. The nodding motor 42 is surrounded by a head fixing piece 43. A head shell 45 is arranged on the head fixing piece 43. A second depth camera 44 and a sound 46 are fixedly arranged on the head shell 45. The second depth camera 44 can be used to identify the piano key button and observe the situation of the robot playing the keyboard. The sound 46 can be used for music playing, voice playing for human-computer interaction, abnormality prompt and the like. One end of a lip air pipe 47 is directly connected to the airflow generating device module 36 in the trunk skeleton 32. The other end of the lip air pipe 47 is connected to the first air inlet of the organ through an airflow pipe.

[0076] Specifically, the robot arm mechanism comprises a mechanical arm, a six-dimensional force sensor 55 and a five-finger bionic dexterous hand 56; the mechanical arm is connected to the torso skeleton 32 at one end and connected to the six-dimensional force sensor 55 at the other end; the mechanical arm is composed of four sections of arms, specifically, a first section of mechanical arm 51, a second section of mechanical arm 52, a third section of mechanical arm 53 and a fourth section of mechanical arm 54; the first section of mechanical arm 51 and the second section of mechanical arm 52, the second section of mechanical arm 52 and the third section of mechanical arm 53, and the third section of mechanical arm 53 and the fourth section of mechanical arm 54 each contain one rotational degree of freedom, specifically, a second rotational degree of freedom 521, a fourth rotational degree of freedom 531 and a sixth rotational degree of freedom 541; each section has one rotational degree of freedom, specifically, a first rotational degree of freedom 511, a third rotational degree of freedom 522, a fifth rotational degree of freedom 532 and a seventh rotational degree of freedom 542; the fifth rotational degree of freedom 532 of the third section of mechanical arm is usually redundant and can be added or removed, so that the mechanical arm has six degrees of freedom or seven degrees of freedom and can simulate the free movement of a human arm; the end of the six-dimensional force sensor 55 is installed with the five-finger bionic dexterous hand 56 capable of playing the accordion, the five-finger dexterous hand is bionic to a human hand structure, and each finger has at least two degrees of freedom, including a first degree of freedom finger 561 and a second degree of freedom finger 562, capable of bending and straightening.

[0077] Specifically, the accordion has a double-row key structure, the inner keyboard 63 is mainly for the robot to play, and the outer keyboard 62 is mainly for the user to play; the key structure in each row is a commonly used key; one end of the accordion is provided with an air path control valve 61 and two air pipes are led out, one of which is connected to the robot, and the other of which is available for the user; the outer passages of the first control valve 611 and the second control valve 612 are connected to the robot air pipe 7 through a Y-shaped bifurcated pipe, and the outer passages of the third control valve 613 and the fourth control valve 614 are connected to the user air pipe 8 through a Y-shaped bifurcated pipe; the inner passages of the second control valve and the fourth control valve are connected to the outer keyboard 62 through a Y-shaped pipe, and the inner passages of the third control valve and the first control valve are connected to the inner keyboard 63 through a Y-shaped pipe.

[0078] Embodiment 2

[0079] According to the embodiment of the present application, a method for human-robot interaction and performance of a body-equipped intelligent humanoid robot playing an accordion is provided, and the method comprises the following steps:

[0080] The human-computer interaction performance method can realize various human-computer interaction modes through control of the valves. When the first control valve 611 is opened and the second control valve 612, the third control valve 613 and the fourth control valve 614 are closed, the gas circuit of the robot is connected with the internal keyboard, and the robot can perform alone. When the first control valve 611, the third control valve 613 and the fourth control valve 614 are closed and the second control valve 612 is opened, the gas circuit of the robot is connected with the external keyboard, and the robot provides the gas source, so that the user can play together with the robot. When the first control valve 611, the second control valve 612 and the fourth control valve 614 are closed and the third control valve 613 is opened, the gas circuit of the user is connected with the internal keyboard, and the user provides the gas source, so that the robot can play. When the first control valve 611, the second control valve 612 and the third control valve 613 are closed and the fourth control valve 614 is opened, the gas circuit of the user is connected with the external keyboard, and the user provides the gas source, so that the user can play alone. When the second control valve 612 and the third control valve 613 are closed and the first control valve 611 and the fourth control valve 614 are opened, the gas circuit of the user is connected with the external keyboard and the gas circuit of the robot is connected with the internal keyboard, so that the user and the robot can play together.

[0081] The working mode of the robot is as follows: the robot is started, and the robot starts a self-checking mode. The self-checking mode includes whether a position is appropriate, for example, whether the accordion center position in front of the robot is at a height of 1 meter and a distance of 0.5 meters. If the distance between the accordion and the robot is not appropriate, the robot controls the wheel chassis walking mechanism 1 to be fine-tuned in front, back, left and right and height. The user can select a robot performance mode through the touch display screen 35. The touch display screen 35 is integrated with various music performance cases. A music button that needs to be performed is clicked. For example, the robot performs alone. The instruction on the touch display screen 35 is transmitted to the industrial computer 114. The industrial computer 114 controls the first control valve 611 of the gas circuit control valve 61 to be opened and the second control valve 612, the third control valve 613 and the fourth control valve 614 to be closed. Then, the airflow generating device module 36 is started. The air pump of the airflow generating device module 36 generates the gas source. The driver and the gas circuit control circuit control the flow valve to output the airflow according to the rhythm set by the music to simulate the amount of human exhalation. At the same time, the robot starts the robot arm mechanism 5 with the five-finger bionic dexterous hand 56 to be above the keys of the internal keyboard 63. According to the rhythm, the keys are pressed by simulating the bending of the human fingers. At the same time, the multi-modal sensor on the finger records the force information. According to the feedback information of the force, the time and the force of pressing can be further controlled, so that the robot bionic human performance accordion paradigm is simulated more realistically.

[0082] In the description of the present application, it needs to be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0083] Those skilled in the art know that, in addition to implementing the system, device and each module thereof provided by the present application in the form of pure computer readable program code, the same program can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers by logically programming the method steps. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing methods and structures within hardware components.

[0084] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be arbitrarily combined with each other without conflict.

Claims

1. An embodied intelligent humanoid robot that plays a melodic wind instrument, characterized by, The application relates to a body-possessed intelligent wheeled humanoid robot and a human-machine interactive mouth organ (6) played by the robot. The body-possessed intelligent wheeled humanoid robot comprises a lifting column leg mechanism (2) arranged above a wheeled chassis walking mechanism (1), a robot trunk mechanism (3) arranged above the lifting column leg mechanism (2), a robot head mechanism (4) arranged at the top of the robot trunk mechanism (3), and robot arm mechanisms (5) arranged at the two sides of the robot trunk mechanism (3); a five-finger bionic dexterous hand (56) is arranged at the end of each robot arm mechanism (5) and used for playing the mouth organ. An airflow generating device module (36) is arranged in the robot trunk mechanism (3) and used for simulating human body airflow generation and output. The robot head mechanism (4) comprises a mouth-lip air tube (47); an air path pipeline is arranged in the robot head mechanism (4) and connected with the airflow generating device module (36) at one end and with the mouth-lip air tube (47) at the other end. An air path control valve (61) is arranged at one end of the human-machine interactive mouth organ (6) and connected with two air tubes, namely a robot air tube (7) and a user air tube (8); the robot air tube (7) is connected with the mouth-lip air tube (47) and used for providing the body-possessed intelligent humanoid robot with playing; and the user air tube (8) is used for providing a user with playing. The air path control valve (61) is used for controlling the playing mode of the body-possessed intelligent wheeled humanoid robot, including autonomous playing of the mouth organ by the body-possessed intelligent wheeled humanoid robot or interactive playing of the mouth organ by the body-possessed intelligent wheeled humanoid robot and a human. The human-machine interactive mouth organ (6) has a double-row key structure, wherein an inner keyboard is used for robot playing, and an outer keyboard is used for user playing. The air path control valve (61) comprises a first control valve (611) and a second control valve (612) whose outer channels are connected with the robot air tube (7) through a Y-shaped branch pipeline, a third control valve (613) and a fourth control valve (614) whose outer channels are connected with the user air tube (8) through a Y-shaped branch pipeline, the inner channels of the second control valve (612) and the fourth control valve (614) are connected with the outer keyboard through a Y-shaped pipeline, and the inner channels of the third control valve (613) and the first control valve (611) are connected with the inner keyboard through a Y-shaped pipeline. The wheeled chassis walking mechanism (1) comprises a bottom plate (11) and a chassis shell (12).

2. The animatronic humanoid robot that plays a melodeon according to claim 1, characterized in that, The bottom plate (11) comprises an industrial computer (114), an embedded control board (111), a first depth camera (113), a laser radar (112), a driving wheel (115), a universal following wheel (118) and a driving motor (116). The industrial computer (114) is connected with the embedded control board (111) through a communication interface; the embedded control board (111) is electrically connected with the driving wheel (115); and two driving wheels (115) are connected with two driving motors (116) respectively. ​ The universal follow-up wheel (118) is the third wheel of the wheeled chassis walking mechanism (1) and has no active driving capability; The chassis shell (12) comprises an emergency stop button (122), a start-stop button (121) and ultrasonic sensors (123); the ultrasonic sensors (123) are distributed on the four peripheral surfaces of the chassis shell (12) and are electrically connected with the industrial computer (114); The first depth camera (113), the laser radar (112) and the ultrasonic sensors (123) are used for capturing environmental information.

3. The physically intelligent humanoid robot playing a melodeon, according to claim 2, characterized in that, The industrial computer (114) collects data of the first depth camera (113) and the laser radar (112), establishes an environmental model and a navigation model; In the navigation motion control, the industrial computer (114) collects data of the ultrasonic sensors (123), the first depth camera (113) and the laser radar (112), controls the autonomous navigation of the robot, so that the robot avoids collision with moving objects.

4. The animatronic humanoid robot that plays a melodeon according to claim 2, characterized in that, The lifting column leg mechanism (2) comprises a column outer square (21), a column inner square (22), a motor (23), a motor gear (231), a driving rod (24) and a sliding block (25); The lifting column leg mechanism (2) is a square nested structure, the column outer square (21) is installed at the center region of the bottom plate (11) and is fixedly connected with the column butt joint plate (117) on the bottom plate (11); the column inner square (22) is nested in the column outer square (21); the motor (23) is installed in the column outer square (21); the motor gear (231) is connected with the driving rod (24), the sliding block (25) on the driving rod (24) is connected with the column inner square (22); under the driving of the motor (23), the sliding block (25) drives the column inner square (22) to lift; the top end of the column inner square (22) is connected with the robot torso mechanism (3).

5. The animatronic human form robot that plays a melodeon according to claim 1, characterized in that, The robot torso mechanism (3) comprises a torso shell (31), a torso skeleton (32), a bending motor module (33) and a neck rotation mechanism (34); The torso skeleton (32) is fixedly arranged in the torso shell (31); the bending motor module (33) is fixedly arranged at the bottom end of the torso skeleton (32), and the neck rotation mechanism (34) is arranged at the upper end of the torso skeleton (32); the robot head mechanism (4) is installed at the upper end of the neck rotation mechanism (34); the robot arm mechanism (5) is arranged at the two sides of the torso skeleton (32); An airflow generating device module (36) is arranged in the torso skeleton (32); the airflow generating device module (36) generates airflow under the control of an air pump, and the airflow output is controlled by a flow valve, so as to simulate the gas flow required when a human body plays an organ; The crouching motor module (33) comprises a crouching motor (331), a lower groove (332), an upper groove (333), and a bearing (334). One end of the lower groove (332) is provided with the bearing (334) rotating passively, and the bearing (334) rotating passively is connected with one end of the upper groove (333). The other end of the lower groove (332) is provided with the crouching motor (331), and the crouching motor (331) is connected with the other end of the upper groove (333). When the crouching motor (331) rotates, the upper groove (333) rotates, and the rotating range is -90° to 90°. A touch display screen (35) is further arranged on the front of the trunk shell (31), and the touch display screen (35) is used for human-computer interaction, including setting a robot working mode. The neck rotating mechanism (34) comprises a neck rotating motor (341), a shoulder connecting piece (342), and a neck rotating connecting piece (343). The neck rotating motor (341) is installed on the shoulder connecting piece (342), and a protruding end is connected with the neck rotating connecting piece (343). When the neck rotating motor (341) rotates, the neck rotating connecting piece (343) rotates. The shoulder connecting piece (342) is installed on the trunk skeleton (32).

6. The physically intelligent humanoid robot playing a melodeon, according to claim 5, characterized in that, The robot head mechanism (4) comprises a neck connecting piece (41), a nodding motor (42), a head fixing piece (43), a second depth camera (44), a head shell (45), and a sound (46). The neck connecting piece (41) is arranged in the trunk shell (31) and is detachably connected with the trunk skeleton (32). The nodding motor (42) is installed on the neck rotating connecting piece (343). The nodding motor (42) is surrounded by the head fixing piece (43). The head fixing piece (43) is provided with the head shell (45), and the head shell (45) is fixedly provided with the second depth camera (44) and the sound (46). The second depth camera (44) is used for identifying piano key buttons and observing the situation of the robot playing a keyboard. The sound (46) is used for music playing, voice playing for human-computer interaction, and abnormality prompting.

7. The physically intelligent humanoid robot playing a melodeon, according to claim 5, characterized in that, The robot arm mechanism (5) comprises a mechanical arm, a six-dimensional force sensor (55), and a five-finger bionic dexterous hand (56). One end of the mechanical arm is connected with the robot trunk mechanism (3), and the other end is connected with the six-dimensional force sensor (55). The six-dimensional force sensor (55) is connected with the five-finger bionic dexterous hand (56). The mechanical arm is composed of a first section mechanical arm (51), a second section mechanical arm (52), a third section mechanical arm (53) and a fourth section mechanical arm (54); the first section mechanical arm (51), the second section mechanical arm (52), the third section mechanical arm (53) and the fourth section mechanical arm (54) are connected in sequence, and each mechanical arm connection has one rotation degree of freedom; each mechanical arm has a rotation degree of freedom, so that the mechanical arm realizes six degrees of freedom or seven degrees of freedom, and can imitate the free movement of a human arm; Each finger of the five-finger bionic dexterous hand (56) has at least two degrees of freedom, and can bend and straighten.

8. A method of human-robot interaction for playing a concertina by a humanoid robot having a body intelligence, characterized in that, The method comprises the following steps: Selecting a robot performance mode through the touch display screen (35), including: A variety of human-computer interaction modes are realized through the control valve, including: when the first control valve (611) is opened, and the second control valve (612), the third control valve (613) and the fourth control valve (614) are closed, the robot gas circuit is connected with the internal keyboard gas circuit (616), and the robot performs alone; When the first control valve (611), the third control valve (613) and the fourth control valve (614) are closed, and the second control valve (612) is opened, the robot gas circuit is connected with the external keyboard gas circuit (615), the robot provides the air source, and the user plays together with the robot; When the first control valve (611), the second control valve (612) and the fourth control valve (614) are closed, and the third control valve (613) is opened, the user's gas circuit is connected with the internal keyboard gas circuit (616), the user provides the air source, and the robot plays; When the first control valve (611), the second control valve (612) and the third control valve (613) are closed, and the fourth control valve (614) is opened, the user's gas circuit is connected with the external keyboard gas circuit (615), the user provides the air source, and the user plays alone; When the second control valve (612) and the third control valve (613) are closed, and the first control valve (611) and the fourth control valve (614) are opened, the user's gas circuit is connected with the external keyboard gas circuit (615), and the robot's gas circuit is connected with the internal keyboard gas circuit (616), so that the user and the robot play together.

9. The method of human-machine interactive performance of a wind-accordion playing humanoid robot according to claim 8, wherein, When the robot provides the air source, the robot generates the air source through the air pump of the airflow generating device module (36), and the flow valve controls the airflow output according to the breath amount of the human body simulated by the rhythm set by the music; When the robot plays, the robot drives the five-finger bionic dexterous hand (56) above the keys through the robot arm mechanism (5), and simulates the bending of the human fingers to press the keys according to the rhythm.

Images