Performance control device, performance control method, and performance control program

Abstract

To provide a musical performance control device which enables a robot to play a musical instrument by simple operation input of a user.SOLUTION: A musical performance control device includes: a display control part for displaying graphics (50a to 54) indicating a musical performance motion including a position of a hand part of a robot and a timing at which the hand part of the robot operates a musical instrument on a display device; an edition part for editing the musical performance motion according to operation input of the graphics (50a to 54) by a user; and a job generation part for generating a job program of operating the robot on the basis of the information on the musical performance motion edited by the edition part.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a performance control device, a performance control method, and a performance control program for causing a robot to play a musical instrument. [Background technology]

[0002] Patent Document 1 discloses a performance control device for making a robot play a musical instrument (see Patent Document 1). The performance control device is made up of a computer with a performance control program installed.

[0003] The performance control device executes a performance control program to sequentially transmit a large number of motion (posture) commands to the robot according to the music. The robot sequentially moves its hands and fingers based on the large number of motion (posture) commands transmitted from the performance control device. Specifically, the robot moves its hands to a predetermined position on the musical instrument (piano keys) and presses the musical instrument (piano keys) with its fingers. This causes the robot to play the musical instrument. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 8-1562 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional performance control devices, the user must pre-register robot motions (postures) corresponding to each musical scale of the instrument, and create a large number of motion (posture) commands according to the music.

[0006] For this reason, for example, to play a musical instrument with a two-octave (24-note) scale using a three-fingered robot, 72 types of motions (postures) must be registered, and managing all of the postures is a major challenge. Another issue is that only users who are familiar with robot operation inputs can create motion (posture) commands.

[0007] The present invention has been made in consideration of the above problems, and has an object to provide a performance control device that can cause a robot to play a musical instrument through simple operational input by a user. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, one aspect of the present invention is a performance control device for causing a robot to play a musical instrument, the device comprising: a display control unit that displays, on a display device, a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's input of an operation on the graphic; and a job generation unit that generates a job program for operating the robot based on information on the performance motion edited by the editing unit. The timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the performance motion includes selection of at least one finger that presses the musical instrument among a plurality of fingers of the robot. It is a performance control device. Another aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for operating the robot based on information about the performance motion edited by the editing unit, wherein the timing at which the robot's hands operate the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the performance motion includes a pre-processing time from when at least one finger of the robot starts to press the musical instrument to when the at least one finger of the robot actually presses the musical instrument, before the timing at which the at least one finger of the robot presses the musical instrument. Another aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for operating the robot based on information about the performance motion edited by the editing unit, wherein the performance motion includes the timing at which the robot's hands move. Another aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for moving the robot based on information about the performance motion edited by the editing unit, wherein the performance motion includes a choreography motion for moving a part of the robot that is unrelated to the performance. Another aspect of the present invention is a performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including the position of the robot's hands and the timing at which the robot's hands operate the musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the graphic; and a job generation unit that generates a job program for operating the robot based on information about the performance motion edited by the editing unit, wherein the timing at which the robot's hands operate the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the display control unit displays on the display device the scale of the finger currently being pressed based on the information about the performance motion edited by the editing unit.

[0009] Another aspect of the present invention is a performance control method for causing a robot to play a musical instrument, the method comprising: a display step of displaying on a display device a figure representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate the musical instrument; an editing step of editing the performance motion in response to an operation input of the figure by a user; Collection Steps and a job generating step of generating a job program for operating the robot based on the information of the performance motion edited by the user. The timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the performance motion includes selection of at least one finger that presses the musical instrument among a plurality of fingers of the robot. A performance control method.

[0010] Yet another aspect of the present invention is a performance control program for causing a robot to play a musical instrument, the program including a display function for displaying on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate the musical instrument, an editing function for editing the performance motion in response to an operation input of the graphic by a user, and a program for editing the editing function. Collection function A job generation function that generates a job program for operating the robot based on the information of the performance motion edited by the The timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument, and the performance motion includes selection of at least one finger that presses the musical instrument among a plurality of fingers of the robot. It is a performance control program. Effect of the Invention

[0011] According to the present invention, a performance motion is created using a user interface that focuses on the robot's hands, making it easy to understand the position of the robot's hands and the timing of the hands' operation on the instrument, and allowing the user to make the robot play an instrument through simple operational input. [Brief description of the drawings]

[0012] [Figure 1] 1 is a perspective view of a robot controlled by the performance control device of the present embodiment. [Diagram 2] FIG. 2 is a detailed view of the hand of the robot. [Diagram 3] 1 is a block diagram showing an example of the hardware configuration of a performance control device according to an embodiment of the present invention; [Figure 4] 2 is a block diagram showing an example of the functional configuration of the performance control device of the present embodiment. FIG. [Diagram 5] FIG. 13 is a diagram showing an example of a graphic representation of a performance motion. [Figure 6] FIG. 13 is a diagram showing an example of a graphic representation of a performance motion. [Figure 7] FIG. 7 is a diagram showing the correspondence between a graphic representing a performance motion and the movements of the robot's fingers (FIG. 7(a) shows a graphic representing a performance motion, and FIGS. 7(b1) to (b3) show the movements of the robot's fingers). [Figure 8]FIG. 8 shows the correspondence between a figure representing a performance motion and the movements of the robot's fingers (FIG. 8(a) shows a figure representing a performance motion, and FIGS. 8(b1) and (b2) show the movements of the robot's fingers). [Figure 9] FIG. 9 shows the correspondence between figures representing performance motions and robot movements (FIGS. 9(a) and (b) show figures representing performance motions, and FIGS. 9(c1) and (c2) show the movements of the robot's fingers). [Figure 10] FIG. 10(a) shows an example of a graphic representing a musical score, and FIG. 10(b) shows an example of a graphic representing a performance motion. [Figure 11] FIG. 13 is a diagram showing an example of an initial setting screen. [Figure 12] FIG. 2 is a diagram showing a moving range of a hand of a robot. [Figure 13] FIG. 13(a) shows an example in which the robot's left hand holds a fingerboard, and FIG. 13(b) shows an example in which the robot's right hand holds a fingerboard. [Figure 14] FIG. 13 is a diagram showing another example of the initial setting screen. [Figure 15] FIG. 2 is a diagram showing a moving range of a hand of a robot. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, a performance control device, a performance control method, and a performance control program according to an embodiment of the present invention will be described with reference to the accompanying drawings. However, the present invention can be embodied in various forms and is not limited to the embodiments described in this specification. The present embodiment is provided with the intention of enabling those skilled in the art to fully understand the invention by fully disclosing the specification. <Robot configuration>

[0014] FIG. 1 is a perspective view of a robot 1 controlled by a performance control device 21 of this embodiment. The robot 1 is, for example, a humanoid robot, and includes a waist 2, a torso 4 connected to the waist 2 via a waist joint 3, two arms 6 (left and right as seen from the robot) connected to the torso 4 via shoulder joints 5, and a head 11 connected to the torso 4 via a neck joint 10. Each arm 6 includes an upper arm 7 and a lower arm 8 connected to the upper arm 7 via an elbow joint 9. A hand 14 is connected to the lower arm 8 via a wrist joint 12. The hand 14 has a palm 13 (see FIG. 2) and fingers 15 connected to the palm 13 via finger joints 16. Each joint of the robot 1 is driven by a servo motor. Note that the robot is not limited to a humanoid robot as long as it has the hands 14 and can play a musical instrument.

[0015] FIG. 2 is a detailed view of the hand 14 of the robot 1. A servo motor 18 that rotates a disk 17 is provided on the palm 13. Each disk 17 and each finger 15 are connected via a rod 19. By driving the servo motor 18, the finger 15 rotates about the knuckle 16, and the finger 15 presses down the instrument 20 or separates the finger 15 from the instrument 20. 14b is a holder for the instrument. Note that, although multiple (three) finger 15 are configured to be operable in FIG. 2, only one finger 15 may be operable. The hand 14 may be configured appropriately to suit the instrument.

[0016] As shown in Fig. 1, the musical instrument 20 has a fingerboard 20a. The musical instrument 20 is, for example, an electronic musical instrument. With this musical instrument 20, sound is produced simply by pressing the fingerboard 20a with the finger portions 15 of the hand portion 14 of the robot 1, and the scale changes by changing the position where the finger portions 15 press the fingerboard 20a. Note that the musical instrument is not limited to an electronic musical instrument, and may be a keyboard instrument such as a piano, a wind instrument such as a recorder, a string instrument such as a guitar, or a percussion instrument such as a drum.

[0017] The robot 1 is controlled by a performance control device 21. The performance control device 21 controls the servo motors 18 of each joint of the robot 1 so that the robot 1 plays the musical instrument in accordance with music in the same way as a human being. That is, the performance control device 21 controls the servo motors 18 of each joint of the robot 1 so that the hand 14 of the robot 1 moves along the musical instrument 20 (in this embodiment, the hand 14 moves along the fingerboard 20a) and operates the musical instrument 20 (in this embodiment, the finger 15 presses the fingerboard 20a). Note that the operation of the musical instrument 20 by the hand 14 of the robot 1 is not limited to the above embodiment. For example, in the case of a keyboard instrument, the finger 15 of the robot 1 may press the keyboard, in the case of a wind instrument, the finger 15 of the robot 1 may cover the tone hole, in the case of a string instrument, the hand 14 of the robot 1 may pluck the strings directly or using a bow, etc., and in the case of a percussion instrument, the hand 14 of the robot 1 may strike the percussion instrument directly or using a drumstick, etc. <Hardware configuration of the performance control device>

[0018] 3 is a block diagram showing an example of the hardware configuration of the performance control device 21 of this embodiment. The hardware of the performance control device 21 is, for example, a general-purpose information processing device such as a personal computer.

[0019] 3, the performance control device 21 includes a CPU 22, a ROM 23, a RAM 24, a display device 25, a keyboard 26, a mouse 27, a storage device 28, a communication interface 29, a VRAM 30, a key scan circuit 31, a mouse scan circuit 32, a sound source unit 33, a sound system 34, and a bus 35. The display device 25 is connected to the bus 35 via the VRAM 30. The keyboard 26 is connected to the bus 35 via the key scan circuit 31. The mouse 27 is connected to the bus 35 via the mouse scan circuit 32. The mouse 27 may be a tablet, a touch screen, a touch panel, or the like.

[0020] The display device 25 is a liquid crystal display, an organic electroluminescence display, or the like. Graphics representing performance motions, which will be described later, are displayed on the display device 25. A user can move a cursor displayed on the display device 25 and give instructions by operating a mouse 27. A mouse scan circuit 32 detects the movement and operation of the mouse 27, and outputs a detection signal to the CPU 22 via a bus 35. The keyboard 26 has keys for inputting letters, numbers, symbols, and the like. A key scan circuit 31 detects whether the keyboard 26 is on or off, and outputs a detection signal to the CPU 22.

[0021] The storage device 28 is, for example, a hard disk, a CD-ROM, a DVD, or a memory card, etc. The storage device 28 stores a performance control program (to be described later), drawing information of a figure representing a performance motion (to be described later), information on the performance motion edited by an editing unit (to be described later), information on the posture of the robot 1 (to be described later), etc.

[0022] The CPU 22 loads the performance control program stored in the storage device 28 into the RAM 24, and executes the performance control program loaded into the RAM 24, thereby controlling the entire performance control device 21. The RAM 24 temporarily stores information used in the processing of the CPU 22.

[0023] The communication interface 29 is an interface connected to a communication network such as a USB, a LAN, or the Internet. The robot 1 is connected to the communication interface 29 by wire or wirelessly. The communication interface 29 transmits signals to the robot 1 and receives signals from the robot 1.

[0024] The sound source unit 33 includes software as a sound source and a DSP for adding effects. The sound source unit 33 generates a sound signal based on information about a performance motion edited by an editing unit (to be described later), and outputs the sound signal to a sound system 34. The sound system 34 has a D / A converter, an amplifier, and a speaker, and produces a sound for simulation based on the sound signal. <Functional configuration of the performance control device>

[0025] 4 is a block diagram showing an example of the functional configuration of the performance control device 21 of this embodiment. The performance control device 21 includes a display control unit 41, an editing unit 42, a job generating unit 43, and a sound control unit 44. These functional configurations are realized by the CPU 22 executing a performance control program.

[0026] The display control unit 41 displays on the display device 25 a graphic representing a performance motion including the position of the hands 14 of the robot 1 and the timing at which the hands 14 of the robot 1 operate the musical instrument 20. The editing unit 42 edits the information of the performance motion in response to the user's operation input of the graphic. The job generation unit 43 generates a job program for operating the robot 1 based on the information of the performance motion edited by the editing unit 42. The job program generated by the job generation unit 43 is transmitted to the robot 1 via the communication interface 29. The sound control unit 44 controls the sound source unit 33 and the sound system 34 to play sounds for simulation based on the information of the performance motion edited by the editing unit 42. <Specific processing of the performance control device>

[0027] Specific processing in the performance control device 21 configured as above and user operation inputs will be described with reference to FIG. 5 to FIG.

[0028] First, the display control unit 41 of the performance control device 21 displays figures representing performance motions on the display device 25. Fig. 5 shows examples of figures 50a-54 representing performance motions displayed on the display device 25. As shown in Fig. 6, the figures 50a-54 representing performance motions are displayed in a performance motion display area 101 of the display device 25.

[0029] The figure 50a in FIG. 5 represents the fingerboard 20a of the musical instrument 20. The three circles 51a to 51c represent the three finger portions 15 of the robot 1. The vertical position of the top circle 51a, which represents the three circles 51a to 51c, represents the position of the hand 14 of the robot 1 on the fingerboard 20a. ● 51c represents the finger portion 15 (the finger portion 15 that presses the musical instrument 20) selected by the user's operation input among the three finger portions 15. ● 51c represents the timing at which the finger portion 15 of the robot 1 presses the musical instrument 20. ● The diagonal line band 52 extending from 51c to the right represents the duration of the state in which the finger portion 15 of the robot 1 presses the musical instrument 20. ● The diagonal line band 53 extending from 51c to the left represents the pre-processing time from when the finger portion 15 of the robot 1 starts the action of pressing the musical instrument 20 to when it actually presses the musical instrument 20. The dotted rectangular zones 54 indicate the timing for moving the hand 14 of the robot 1 (moving the arm 6 of the robot 1). The horizontal axis of Fig. 5 indicates time, and the vertical axis of Fig. 5 indicates the position of the hand 14 on the fingerboard 20a. The graphic 55 is a timing indicator that matches the tempo of the music.

[0030] Next, the editing unit 42 edits the performance motion according to the user's operation input of the figures 50a, 51 to 54. For example, when the user clicks one of the circles 51a, 51b, and 51c with the mouse 27, the editing unit 42 selects the bottommost finger 15 of the robot 1 represented by the circle 51c as the finger 15 that presses the instrument 20. The display control unit 41 changes the circle 51c selected by the user to a black circle 51c.

[0031] Also, for example, when the user drags and drops the figure 50a and ●51c with the mouse 27, the editing unit 42 determines the timing at which the finger unit 15 of the robot 1, represented by ●51c, presses the instrument 20. The display control unit 41 changes the left-right positions of the figure 50a and ●51c and displays them.

[0032] Furthermore, for example, when the user drags and drops the right end of band 52 with mouse 27, editing unit 42 determines the duration of the state in which finger unit 15 of robot 1 presses instrument 20, which is represented by band 52. Display control unit 41 changes the length of band 52 and displays it.

[0033] Furthermore, for example, when the user drags and drops the left end of band 53 with mouse 27, editing unit 42 determines the pre-processing time represented by the length of band 53, from when finger unit 15 of robot 1 starts the action of pressing musical instrument 20 to when the musical instrument 20 is actually pressed. Display control unit 41 changes the length of band 53 and displays it.

[0034] Furthermore, for example, when the user drags and drops the rectangular zone 54 with the mouse 27, the editing unit 42 determines the timing at which the hand 14 of the robot 1, represented by the rectangular zone 54, moves. The display control unit 41 changes and displays the rectangular zone 54.

[0035] FIG. 7 is a diagram showing the correspondence between ●51b, band 52, band 53, which are figures showing playing motions, and the movements of the fingers 15 of the robot 1. The band 53 in FIG. 7(a) represents the pre-processing time (movement time of the fingers 15) from when the fingers 15 of the robot 1 start to press the musical instrument 20 until they actually press the musical instrument 20, as shown in FIG. 7(b1)(b2). By adjusting the length of the band 53, it is possible to reliably sound at the point of ●51b. Also, by adjusting the length of the band 53, the speed of the fingers 15 can be adjusted (for example, when the length of the band 53 is short, the fingers 15 move quickly), so the strength of the sound can be adjusted in keyboard instruments and percussion instruments.

[0036] ●51b in Fig. 7(a) represents the timing at which the fingers 15 of the robot 1 press the instrument 20, as shown in Fig. 7(b2). Band 52 in Fig. 7(a) represents the duration during which the fingers 15 of the robot 1 press the instrument 20. Area 102 in Fig. 7(a) (a section without band 52) represents the time during which the fingers 15 are off the instrument 20, as shown in Fig. 7(b3). The user can adjust the position of ●51b and the length of band 52 by inputting an operation to adjust the timing at which the sound is produced and the duration of the sound.

[0037] The vertical position of ●51b indicates the position of fingering 15 on fingerboard 20a of instrument 20, so the user can adjust the scale by adjusting the vertical position of ●51b through operational input.

[0038] Fig. 8 is a diagram showing the correspondence between a rectangular zone 54, which is a graphic representing a performance motion, and the movement of the fingers 15 of the robot 1. The rectangular zone 54 in Fig. 8(a) represents the timing for moving the hand 14 of the robot 1. If the width of the rectangular zone 54 is wide, the hand 14 of the robot 1 moves slowly, and if the width of the rectangular zone 54 is narrow, the hand 14 of the robot 1 moves quickly.

[0039] As shown in Fig. 8(a), when the band 52 and the rectangular zone 54 overlap, the fingers 15 of the robot 1 can move the hand 14 of the robot 1 while pressing the instrument 20, as shown in Fig. 8(b1)(b2). This enables slide performance, which smoothly changes the scale while producing sound. As shown in area 103 in Fig. 8(a), when the length of the band 53 on the left side of ●51b is set to zero and the band 52 on the right side of the previous ●51b is brought closer to ●51b, the finger 15 represented by ●51b can continue to press the instrument 20.

[0040] 9 is a diagram showing the correspondence between the figures ●51b, ●51c, band 52, and band 53 that represent playing motions, and the movements of the robot 1. As shown in Fig. 9(a), if adjacent ●51b are the same finger 15, band 52 and band 53 cannot be arranged so as to overlap. This is because the same finger 15 cannot simultaneously press the instrument 20 and be released from the instrument 20.

[0041] As shown in Fig. 9(b), if adjacent ●51b and ●51c are different finger parts 15, the belt 52 and the belt 53 can be arranged so that they overlap. In this case, as shown in Fig. 9(c1), while the finger part 15 (finger 2) represented by ●51b is pressing down, the finger part 15 (finger 3) represented by ●51c prepares to press down, and as shown in Fig. 9(c2), finger 3 presses the instrument 20 and at the same time finger 2 leaves the instrument. In this way, by arranging the belt 52 and the belt 53 so that they overlap, quick switching of the finger parts 15 becomes possible.

[0042] Next, the job generating unit 43 generates a job program for operating the robot 1 based on the information of the performance motion edited by the editing unit 42. The performance motion includes the position of the hand 14 of the robot 1 and the timing at which the finger 15 of the robot 1 presses the musical instrument 20, and so on, so that the robot 1 can be made to play the musical instrument 20. <Displaying musical scales, generating sounds for simulation>

[0043] As shown in area 104 of FIG. 6, based on the information of the playing motion edited by the editing unit 42 (the vertical position of the hand 14, the distance between the fingers described later, and the relationship between the position of the hand 14 and the scale), the display control unit 41 displays on the display device 25 the scale of the finger 15 that the robot 1 is pressing down, for example, C4, and also displays on the display device 25 the position of the hand 14 on the fingerboard 20a, for example, 0.110.

[0044] The sound control unit 44 controls the sound source unit 33 and the sound system 34 to generate sounds for the simulation based on the information of the performance motion edited by the editing unit 42. This allows the edited performance motion to be played back and checked in the simulation without moving the actual robot 1. When the playback process starts, the display control unit 41 moves the figure 55 representing the timing indicator to the right. <Choreography Motion>

[0045] The performance motion includes a choreography motion that moves a part of the robot 1 (head 11, torso 4, etc.) that is not related to the performance. As shown in Fig. 6, the display control unit 41 displays figures 56a and 56b representing the choreography motion on the display device 25. The figures 56a and 56b representing the choreography motion are displayed in the choreography motion display area 105.

[0046] For example, the figure 56a represents an action of rotating the torso 4 of the robot 1 to the left, and the figure 56b represents an action of rotating the torso 4 of the robot 1 to the right. The choreography motion is registered in advance, i.e., stored in the storage device 28.

[0047] The editing unit 42 edits the choreography motion in response to user operation input, for example dragging and dropping the figures 56a, 56b with the mouse 27. The editing unit 42 edits the performance motion and then adds and edits the choreography motion. The movement choreography motion of the arm 6 of the robot 1 is overwritten by the performance motion of the arm 6 of the robot 1. The editing unit 42 can match the performance motion even if the choreography motion is time-series data of the joint angles of the robot 1.

[0048] As a choreography motion, the hand 14 on the opposite side to the hand 14 holding the fingerboard 20a can also be moved. In this case, the editing unit 42 creates a choreography motion in which the fingers 15 of the hand 14 on the opposite side close when the graphic 50 appears, by inputting a user operation, for example, when the user uses the mouse 27 to click on the graphic 50 representing the holding part 20b (see FIG. 1) of the musical instrument 20.

[0049] The job generating unit 43 operates the robot 1 based on the information of the choreography motion edited by the editing unit 42. This allows the robot 1 to have a choreography motion that matches the music performance. <User interface showing musical score>

[0050] 10(a), the display control unit 41 may display a graphic 61 representing a musical score with notes arranged on a five-line staff on the display device 25. The editing unit 42 edits the musical score through user input operations. Since the graphic 61 representing the musical score corresponds to a general musical score, the user can create music more easily.

[0051] The display control unit 41 converts the figure 61 representing the musical score shown in Fig. 10(a) into a figure representing a performance motion shown in Fig. 10(b) based on the information of the musical score edited by the editing unit 42. Fine adjustment of the position of the hand 14, the timing of moving the hand 14, and the selection of the finger 15 pressing the instrument 20 are performed using the figure representing the performance motion shown in Fig. 10(b). <Initial settings>

[0052] 11 shows an example of an initial setting screen displayed by the display control unit 41 on the display device 25. A dialogue 62 for setting the reference posture of the robot 1 is displayed on the display device 25. The user inputs a posture of the robot 1 registered in advance into the dialogue 62 as the reference posture. The editing unit 42 takes into account the set reference posture of the robot 1 when creating a performance motion. This is the only reference posture of the robot 1 that the user sets, and there is no need to set multiple postures according to music.

[0053] A dialogue 63 for setting the length of the fingerboard 20a is displayed on the display device 25. As shown in FIG. 12, the length of the fingerboard 20a represents the range of movement of the hand 14 of the robot 1 (the range of movement of the arm 6). The user inputs the range of movement of the hand 14 in this dialogue 63. When creating a performance motion for moving the hand 14 of the robot 1, the editing unit 42 creates the performance motion so that the hand 14 holding the fingerboard 20a of the robot 1 moves parallel to the back of the hand. The editing unit 42 may create a performance motion so that the hand 14 holding the fingerboard 20a moves closer to the hand 14 that fixes the musical instrument 20.

[0054] The display device 25 displays dialogues 66a and 66b for setting the hand 14 holding the fingerboard 20a of the musical instrument 20 and the hand 14 to which the musical instrument 20 is fixed. When the user selects the dialogue 66a, the display control unit 41 displays a choice of the hand 14 holding the fingerboard 20a (hand 14a or hand 14b (see Figs. 13(a) and 13(b))) in the dialogue 66a. When the user selects the dialogue 66b, the display control unit 41 displays a choice of the hand 14 to which the fingerboard 20a is fixed (hand 14a or hand 14b) in the dialogue 66a. In response to the user's operation input, the editing unit 42 creates a performance motion so that the left hand 14a of the robot 1 holds the fingerboard 20a and the right hand 14b fixes the musical instrument 20, as shown in Fig. 13(a), or creates a performance motion so that the robot 1 moves in the opposite direction, as shown in Fig. 13(b). This allows the robot 1 to set the hand 14 that plays the instrument 20 and the hand 14 that is fixed. Even after creating a performance motion, the robot 1 can be made to play the instrument 20 with the opposite hand 14 by switching to the opposite hand 14 in the dialogues 66a and 66b and then switching the reference posture.

[0055] 11, a dialogue 67 for setting the distance between the fingers 15 of the robot 1 is displayed on the display device 25. Also, a dialogue 68 for setting the relationship between the position of the hand 14 (hand position) and the musical scale (the note name of the top finger) is displayed on the display device 25. C3, D3, E3, etc. in the dialogue 68 represent musical scales such as Do, Re, Mi, etc. Based on this setting, the display control unit 41 displays on the display device 25 the musical scale of the finger 15 that is being pressed.

[0056] As shown in Fig. 14, instead of setting the length of the fingerboard 20a on the initial setting screen, a reference posture 1 (reference symbol 64) and a reference posture 2 (reference symbol 65) of the robot 1 may be set. As shown in Fig. 15, the reference posture 1 represents the upper end position of the hand 14 of the robot 1, and the reference posture 2 represents the lower end position of the hand 14 of the robot 1. In this case, the editing unit 42 creates a performance motion that moves the hand 14 by apportioning between the reference posture 1 and the reference posture 2. <Effects of the performance control device of this embodiment>

[0057] The above describes an example of the configuration, functions, and specific processing of the performance control device 21 of this embodiment. The performance control device 21 of this embodiment has the following advantages.

[0058] Since a performance motion is created using a user interface focused on the hands 14 of the robot 1, the position of the hands 14 of the robot 1 and the timing of the hand 14 operating the instrument are easy to understand, and the user can make the robot 1 play the instrument 20 by simply inputting operations.

[0059] Since there is no need to register a large number of postures of the robot 1, the postures of the robot 1 can be easily managed.

[0060] The timing at which the fingers 15 of the robot 1 press the musical instrument 20 can be set.

[0061] In the robot 1 having a plurality of fingers 15, it is possible to select which finger 15 to use to press the fingerboard 20a, and quick performance is possible by switching between the fingers 15.

[0062] Since the graphic representing the performance motion includes bands 52, the duration for which the fingers 15 of the robot 1 press the musical instrument 20 can be set.

[0063] Since the figure representing the performance motion includes a band 53, the pre-processing time can be set, and it is possible to play sounds at optimal timing in accordance with the musical score.

[0064] Since the figure representing the performance motion includes the rectangular zone 54, performance that makes use of the characteristics of the musical instrument 20 having the fingerboard 20a, such as slide performance, is possible.

[0065] It is also possible to combine choreography motions with performance motions.

[0066] Since the display device 25 displays the musical scale of the finger 15 of the robot 1 while it is being pressed, it is possible to check the performance without actually moving the robot 1.

[0067] Since the sound control unit 44 generates simulation sounds, it is possible to check the performance without actually moving the robot 1. [Explanation of symbols]

[0068] 1... robot, 14... hand, 15... finger, 20... musical instrument, 21... performance control device, 34... sound system, 41... display control unit, 42... editing unit, 43... job generation unit, 44... sound control unit, 50a to 54... figures representing performance motion

Claims

1. A performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the figure; a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit, the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument; The performance control device, wherein the performance motion includes selection of at least one finger portion of the robot to press the musical instrument.

2. 2. The performance control device according to claim 1, wherein the performance motion includes a duration during which at least one finger of the robot presses the musical instrument.

3. A performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the figure; a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit, the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument; The performance control device, wherein the performance motion includes a pre-processing time from when at least one finger of the robot starts to press the instrument until when the at least one finger of the robot actually presses the instrument, before the timing when the at least one finger of the robot presses the instrument.

4. A performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the figure; a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit, The performance control device, wherein the performance motion includes a timing for moving the robot's hands.

5. A performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the figure; a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit, The performance control device, wherein the performance motion includes a choreography motion that moves a part of the robot that is unrelated to the performance.

6. A performance control device for causing a robot to play a musical instrument, comprising: a display control unit that displays on a display device a graphic representing a performance motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument; an editing unit that edits the performance motion in response to a user's operation input of the figure; a job generating unit that generates a job program for operating the robot based on the information of the performance motion edited by the editing unit, the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument; The display control unit is a performance control device that displays the scale of the finger currently being pressed on the display device based on the information of the performance motion edited by the editing unit.

7. The performance control device according to any one of claims 1 to 6, characterized in that the performance control device further comprises a sound control unit that controls a sound system to produce sounds for simulation based on the information of the performance motion edited by the editing unit.

8. A performance control method for causing a robot to play a musical instrument, comprising the steps of: a display step of displaying on a display device a graphic representing a playing motion including a position of the robot's hands and a timing at which the robot's hands operate a musical instrument; an editing step of editing the performance motion in response to an operation input of the figure by a user; a job generating step of generating a job program for operating the robot based on the information of the performance motion edited by the editing step, the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument; The performance control method, wherein the performance motion includes selecting at least one finger portion of the robot to press the musical instrument.

9. A performance control program for causing a robot to play a musical instrument, On the computer, a display function for displaying on a display device a graphic representing a playing motion including the position of the robot's hands and the timing at which the robot's hands operate a musical instrument; an editing function for editing the performance motion in response to a user's operation input of the figure; a job generation function that generates a job program for operating the robot based on the information of the performance motion edited by the editing function; the timing at which the hand of the robot operates the musical instrument is the timing at which at least one finger of the robot presses the musical instrument; The performance control program, in which the performance motion includes selection of at least one finger portion of the robot to press the musical instrument.

Images