Program, musical performance system, sound production control method and information processing device

Abstract

To make it easy to enjoy musical performance.SOLUTION: A control unit 21 of a terminal device 2 acquires operation information of an operator of a musical performance apparatus 1 to which the terminal device 2 is attached and a vertical tilt angle of the entire musical performance apparatus 1, and causes a sound producing unit 28 to produce a musical tone of a pitch corresponding to the operation of the operator of the musical performance apparatus 1 and the vertical tilt of the entire musical performance apparatus 1.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a program, a performance system, a sound generation control method, and an information processing device. [Background technology]

[0002] Conventionally, a performance device using a terminal device such as a mobile terminal is known. For example, Patent Document 1 describes a method of attaching a cover member imitating an operation unit of a performance device to a mobile terminal, and outputting a performance sound in response to the operation of the cover member. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-150215 A Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 allows the user to easily perform operations on a terminal device similar to those of a musical instrument, but does not allow users to easily enjoy musical performance by incorporating other movements.

[0005] An object of the present invention is to make it possible to easily enjoy playing music. [Means for solving the problem]

[0006] In order to solve the above problems, a program according to one aspect of the present invention comprises: Computer, acquiring operation information of an operator provided on a performance device to which the terminal device is attached and information indicating a movement of the performance device as a whole; causing a sound generator to generate musical tones of a pitch corresponding to a combination of the operation of the control and the movement of the entire performance device; Function as a control unit. Effect of the Invention

[0007] According to the present invention, it becomes possible to easily enjoy playing music. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of the overall configuration of a performance system according to the present invention; [Diagram 2] FIG. 2A is a perspective view showing the performance device as seen diagonally from behind, and FIG. 2B is a perspective view showing the performance device as seen diagonally from the front. [Diagram 3] FIG. 2 is a block diagram showing the functional configuration of the performance device. [Figure 4] FIG. 2 is a block diagram showing a functional configuration of a terminal device. [Diagram 5] FIG. 2 is a schematic diagram showing the appearance of a terminal device. [Figure 6] 5 is a flowchart showing the flow of a performance process executed by the control unit in FIG. [Figure 7] 11A and 11B are diagrams showing examples of performance guide information and avatars displayed on a terminal device. [Figure 8] FIG. 13 is a diagram showing an example of an avatar whose cheeks are puffed out in response to a performer blowing into a performance device. [Figure 9] FIG. 13 is a diagram showing an example of an avatar when a performer points a performance device upward. [Figure 10] FIG. 13 is a diagram showing an example of an avatar when a performer winks. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are subject to various limitations that are technically preferable for implementing the present invention. Therefore, the technical scope of the present invention is not limited to the following embodiments and illustrated examples.

[0010] [Configuration of performance system 100] FIG. 1 is a diagram showing an example of the overall configuration of a performance system 100 according to the present invention. As shown in Fig. 1, the performance system 100 is configured to include a performance device 1 and a terminal device 2 (information processing device). The performance device 1 is a trumpet-type performance device, and includes a mounting section 15 for mounting the terminal device 2. In the performance system 100, when the performer performs a predetermined performance action on the performance device 1 with the terminal device 2 mounted on the mounting section 15 of the performance device 1, the terminal device 2 produces musical tones corresponding to the performance action.

[0011] [Configuration of Performance Device 1] 2(a) and (b) are perspective views of the performance device 1. Fig. 2(a) shows the performance device 1 as seen obliquely from behind. Fig. 2(b) shows the performance device 1 as seen obliquely from the front. Fig. 3 is a block diagram showing the functional configuration of the performance device 1. As shown in Figures 2(a), (b), and 3, the playing device 1 is configured to include a mouthpiece 11, a mouth pipe 12, a housing 14 having an operation unit 13, a loudspeaker unit 16 having an attachment unit 15, a control unit 17, and a communication unit 18.

[0012] The mouthpiece 11 is an opening through which the performer blows breath, and is connected to a mouth pipe 12. The breath blown into the mouthpiece 11 is transmitted to a microphone 26 of the terminal device 2 through the mouth pipe 12 and the housing 14.

[0013] Operation unit 13 is configured to include first valve 13a, second valve 13b, and third valve 13c as operators, and finger hook 13d, on the upper part of housing 14. Operation unit 13 outputs operation signals of first valve 13a, second valve 13b, and third valve 13c to control unit 17. Housing 14 includes a board (not shown) on which control unit 17, communication unit 18, etc. are arranged.

[0014] The mounting section 15 is provided on the back of the loudspeaker section 16, and the terminal device 2 can be mounted thereon. The mounting section 15 is provided with a terminal 18a that is connected to a terminal 2a (see FIG. 5) of the terminal device 2 when the terminal device 2 is mounted on the mounting section 15. The performance device 1 can communicate with the terminal device 2 and receive power from the terminal device 2 via the terminal 18a.

[0015] The loudspeaker unit 16 is a circular enclosure, and amplifies the sound (musical sound) generated by the terminal device 2.

[0016] The control unit 17 is a computer that controls each part of the performance device 1. Specifically, the control unit 17 is configured with at least one CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The control unit 17 reads out a specified program from among the programs stored in the ROM, loads it in the RAM, and executes various processes in cooperation with the loaded program. For example, the control unit 17 transmits operation information of the first valve 13a to the third valve 13c based on an operation signal from the operation unit 13 to the terminal device 2 via the communication unit 18. The communication unit 18 transmits and receives data to and from the terminal device 2 via a terminal 18a.

[0017] [Configuration of terminal device 2] The terminal device 2 is, for example, a smartphone, a tablet terminal, or the like. FIG. 4 is a block diagram showing the functional configuration of the terminal device 2. As shown in FIG. As shown in FIG. 4, the terminal device 2 is configured with a control unit 21, an operation unit 22, a display unit 23, a communication unit 24, a memory unit 25, a microphone 26, an acceleration sensor 27, a sound output unit 28, an imaging unit 29, etc., and each unit is connected by a bus.

[0018] The control unit 21 is a computer that controls each part of the terminal device 2. Specifically, the control unit 21 includes at least one CPU (Central Processing Unit), RAM (Random Access Memory), The control unit 21 reads out a specified program from among the programs stored in the storage unit 25, loads it in the RAM, and executes various processes including a performance process described later in cooperation with the loaded program.

[0019] The operation unit 22 is composed of push button switches, a touch panel attached to the display unit 23, etc. The operation unit 22 outputs to the control unit 21 an operation signal of the push button switch by the user or an operation signal on the screen of the display unit 23 by a finger or the like.

[0020] The display unit 23 is configured with a display device such as a liquid crystal display, an organic EL (Electro Luminescence) display, or the like, and performs display based on the control operation of the control unit 21.

[0021] The communication unit 24 controls communication for transmitting and receiving data to and from an external device. In this embodiment, the communication unit 24 is connected to the performance device 1 via the terminal 2a to transmit and receive data.

[0022] The storage unit 25 is configured with a non-volatile semiconductor memory, a HDD (Hard Disk Drive), etc. The storage unit 25 stores various programs and data required for executing the programs. The storage unit 25 is not limited to being built into the terminal device 2, and may include an external recording medium that is detachable from the terminal device 2.

[0023] For example, the storage unit 25 stores a performance application 251. The performance application 251 is an application program that is executed when the terminal device 2 is attached to the performance device 1. The performance application 251 includes a program for executing a performance process, which will be described later.

[0024] Furthermore, the storage unit 25 stores correct performance information for each piece of music that the performer can play using the performance application 251. The performance information includes information indicating the timing of blowing into the music, information indicating the timing of pressing / releasing each of the first valve 13a to the third valve 13c, and information indicating the timing and angle of tilting the performance device 1. The storage unit 25 also stores accompaniment data for each piece of music that can be played (selected) by a performer using the performance application 251. The storage unit 25 also stores image data of an avatar 232 to be displayed on the display unit 23 during performance processing, which will be described later.

[0025] 5, the microphone 26 is provided at the bottom of the terminal device 2. The microphone 26 detects vibrations in the air (voice and breath), converts them into an electrical signal, and outputs it to the control unit 21. In this embodiment, the microphone 26 functions as a breath detection unit that detects the breath blown into the mouthpiece 11 of the performance device 1. The microphone 26 is preferably a waterproof microphone.

[0026] The acceleration sensor 27 is configured, for example, by a three-axis acceleration sensor, detects acceleration in three axial directions, and outputs the detected acceleration data to the control unit 21. The control unit 21 obtains information indicating the inclination (included in information indicating the operation of the entire performance device 1) based on the acceleration data.

[0027] The sound generating unit 28 includes a sound source, a D / A converter, and a speaker, and generates musical tones according to instructions from the control unit 21.

[0028] The photographing unit 29 is a camera equipped with an imaging lens, an imaging element such as an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-oxide Semiconductor), an A / D conversion circuit, etc. The photographing unit 29 converts an optical image that has passed through the imaging lens into a two-dimensional image signal by the imaging element to obtain image data (2D image).

[0029] [Operation of the performance system 100] Next, the operation of the performance system 100 will be described. When the terminal device 2 is attached to the attachment section 15 of the performance device 1 and the terminal device 2 and the performance device 1 are connected via the terminal 2a of the terminal device 2 and the terminal 18a of the performance device 1, the control section 21 starts up the performance application 251 and causes the display section 23 to display a selection screen (not shown) for selecting a piece of music to be played. When a piece of music to be played is selected from the selection screen, the control section 21 causes the display section 23 to display a performance start button (not shown). When the performance start button is tapped, the control section 21 starts the performance process.

[0030] In a trumpet, the pitch of the sound produced is determined by a combination of the operation (pressing / releasing) of the first valve 13a to the third valve 13c and the way the breath is blown. However, it is difficult for people who are not familiar with the trumpet to distinguish between different ways of blowing, and it is not easy to enjoy playing the trumpet. Therefore, in the performance system 100 of this embodiment, instead of blowing air, the movement of the entire performance device 1 (in this embodiment, the movement of tilting the entire performance device 1 so that the tip (loudspeaker 16) of the performance device 1 faces upward or downward) is used to cause the sound generator 28 to produce musical tones of a pitch that corresponds to the combination of pressing / releasing each of the first to third valves 13a to 13c and the upward or downward tilt of the performance device 1. This allows even people who are unfamiliar with the trumpet to easily enjoy playing the trumpet.

[0031] 6 is a flow chart showing the flow of the performance processing. The performance processing is executed by the control unit 21 and the performance application 251 in cooperation with each other.

[0032] First, the control unit 21 starts displaying the performance guide information 231 of the song to be performed and the performer's avatar 232 on the screen of the display unit 23, and also causes the sound generation unit 28 to start outputting (sound generation) the accompaniment of the song to be performed (step S1). The control unit 21 generates performance guide information 231 based on the performance information of the piece of music to be performed stored in the storage unit 25, and sequentially displays the information on the display unit 23. The control unit 21 also displays a preregistered avatar 232 on the display unit 23. The control unit 21 also causes the sound output unit 28 to output the accompaniment of the piece of music to be performed, based on the accompaniment data of the piece of music to be performed stored in the storage unit 25.

[0033] FIG. 7 is a diagram showing an example of how the performance guide information 231 and the avatar 232 are displayed. The performance guide information 231 is information that indicates which of the first to third valves 13a to 13c should be pressed and how to move the performance device 1 (how to tilt the entire performance device 1 in the vertical direction in this embodiment). As shown in FIG. 7, the performance guide information 231 is displayed at the top of the screen of the display unit 23. The performance guide information 231 is composed of three bars 231a to 231c aligned vertically and a position adjustment display 231d fixedly displayed on the right side of the screen. The three vertically aligned bars 231a to 231c are displayed in one or more sets based on performance information from the present to a predetermined time after the lapse of time. In FIG. 7, three sets of bars 231a1 to 231c1, bars 231a2 to 231c2, and bars 231a3 to 231c3 are displayed from the right side of the screen. The horizontal direction of the screen of the display unit 23 indicates the time axis, and the length of the bars 231a to 231c corresponds to the length of time that the breath should be blown into. The bars 231a to 231c indicate the progress of the performance from the start of the performance. The bars 231a-231c move from left to right according to the elapsed time, and the timing when the horizontal position of the bars 231a-231c overlaps with the alignment display 231d indicates that it is the timing to blow into the valve. The bar 231a corresponds to the first valve 13a, and when the bar 231a is displayed (highlighted) in a predetermined color (for example, the same color as the first valve 13a), the timing when the horizontal position of the bar 231a overlaps with the alignment display 231d indicates that it is the timing to press the first valve 13a. The bar 231b corresponds to the second valve 13b, and when the bar 231b is displayed (highlighted) in a predetermined color (for example, the same color as the second valve 13b), the timing when the horizontal position of the bar 231b overlaps with the alignment display 231d indicates that it is the timing to press the second valve 13b. The bar 231c corresponds to the third valve 13c, and when the bar 231c is displayed (highlighted) in a predetermined color (for example, the same color as the third valve 13c), the timing when the horizontal position of the bar 231c overlaps with the alignment display 231d indicates that it is the timing to press the third valve 13c.

[0034] In the state of FIG. 7, the positioning display 231d displayed fixedly on the right side of the screen and the three vertically aligned bars 231a1 to 231c1 are aligned in the vertical direction and in the display unit 23. This indicates that the way in which the performer is currently tilting the performance device 1 in the vertical direction is correct. Also, the three vertically aligned bars 231a1 to 231c1 are not highlighted (they are not hatched for the sake of illustration). This indicates that the correct answer is to blow into the instrument without pressing down any of the first to third valves 13a to 13c. Furthermore, the three vertically aligned bars 231a2-231c2 on the display unit 23 are not highlighted (not hatched for the sake of illustration), and are displayed lower than the three vertically aligned bars 231a1-231c1. This indicates that in the next performance action, the performer should tilt the performance device 1 downward (the correct answer may be to tilt the performance device 1 upward, depending on the settings) without pressing any of the first to third valves 13a-13c, and then blow into the device. Furthermore, of the three bars 231a3 to 231c3 aligned vertically on the display unit 23, two bars 231a3 and 231c3 are not highlighted (not hatched for illustration purposes), and only the middle bar 231b3 is highlighted (hatched for illustration purposes). This indicates that in the next performance action, the performer should blow into the performance device 1 while holding down the second valve 13b and tilting it downward (depending on the settings, the correct state may be when the performance device 1 is tilted upward).

[0035] In other words, the vertical positions of the bars 231a to 231c correspond to the angle at which the performance device 1 is tilted, and indicate that the entire performance device 1 should be moved (tilted) up or down so that, when the bars 231a to 231c overlap the alignment display 231d, the bars 231a to 231c each overlap the marks M1, M2, and M3 of the alignment display 231d. The avatar 232 is a character image representing a player, and is displayed in an appearance of playing the trumpet.

[0036] Next, the control unit 21 judges, based on the electrical signal input from the microphone 26, whether or not the player's blowing into the performance device 1 has been detected (step S2). For example, it is determined whether or not the blowing of a breath into the performance device 1 has been detected based on whether or not the value of the electrical signal input from the microphone 26 is equal to or greater than a predetermined threshold value. When it is determined that no breath is being blown into the performance device 1 (step S2; NO), the control section 21 proceeds to step S12.

[0037] When it is determined that the blowing of breath into the performance device 1 is detected (step S2; YES), Control unit 21 displays avatar 232 with puffed cheeks in accordance with the value of the electrical signal input from microphone 26 (step S3). FIG. 8 shows an example of how the avatar 232 is displayed when blowing into the performance device 1 is detected.

[0038] Next, the control unit 21 obtains the vertical tilt angle of the performance apparatus 1 based on the output value of the acceleration sensor 27 (step S4). For example, the control unit 21 sets the tilt angle to 0° when the longitudinal direction of the mouth pipe 12 of the performance device 1 is horizontal (when the terminal device 2 faces vertically = when the output value of the acceleration sensor is the value of gravitational acceleration), and calculates (obtains) the vertical tilt angle of the entire performance device 1 based on the output value of the acceleration sensor 27. When the tip of the performance device 1 faces upward, the tilt angle is positive, and when it faces downward, the tilt angle is negative. Note that + and - may be set inversely. The vertical tilt angle of the entire performance device 1 is information indicating the movement of the entire performance device 1.

[0039] Next, the control unit 21 moves the bars 231a to 231c upward or downward (vertical direction) according to the acquired tilt angle (step S5). Here, as shown in FIG. 9, when the bars 231a to 231c overlap the alignment display 231d, if the vertical positions of the bars 231a to 231c match M1, M2, and M3 of the alignment display 231d, this indicates that the performer is moving the performance device 1 correctly to perform.

[0040] Next, control unit 21 tilts avatar 232 according to the acquired tilt angle (step S6). For example, when the tilt angle is +n° (0 < n ≤ 90), as shown in FIG. 9, the control unit 21 tilts the avatar 232 so that the avatar 232 faces upward. The higher the tilt angle is in the positive direction, the higher the pitch becomes. Conversely, the lower the tilt angle is in the negative direction, the lower the pitch becomes.

[0041] Next, the control unit 21 determines whether or not an operator of the performance device 1 has been pressed (step S7). The control unit 21 acquires the operation information from the performance device 1 received by the communication unit 24, and determines whether or not an operator of the performance device 1 has been pressed based on the acquired operation information.

[0042] When it is determined that an operator of the performance device 1 has been pressed (step S7; YES), the control unit 21 causes a display indicating the pressed operator to be displayed on the screen of the display unit 23 (step S8), and proceeds to step S9. For example, as shown in FIG. 9, the control unit 21 causes the marks at the positions corresponding to the pressed first valve 13a to third valve 13c among the marks M1 to M3 of the alignment display 231d to be displayed in a color corresponding to the pressed valve. In FIG. 9, it shows that the second valve 13b has been pressed. When it is determined that an operator of the performance device 1 has not been pressed (step S7; NO), the control unit 21 proceeds to step S9.

[0043] In step S9, the control unit 21 determines whether or not the combination of the operation of the operator of the performance device 1 and the tilt angle of the performance device 1 matches the correct performance information stored in the storage unit 25 (step S9). Based on the operation information received from the performance device 1 by the communication unit 24 and the tilt angle acquired in step S4, the control unit 21 determines whether the combination of the operation of the operator of the performance device 1 (pressing / releasing of each of the first valve 13a to third valve 13c) and the tilt angle of the performance device 1 matches the correct performance information stored in the storage unit 25.

[0044] If it is determined that the combination of the operation of the controls of the performance device 1 and the tilt angle of the performance device 1 matches the correct performance information stored in the memory unit 25 (step S9; YES), the control unit 21 causes the sound generation unit 28 to produce the correct musical tone (step S10) and proceeds to step S12. In step S10, the control unit 21 causes the sound generation unit 28 to generate a trumpet sound at a pitch that corresponds to a combination of the operation of the control of the performance device 1 and the tilt angle of the performance device 1.

[0045] If it is determined that the combination of the operation of the controls of the performance device 1 and the tilt angle of the performance device 1 does not match the correct performance information stored in the memory unit 25 (step S9; NO), the control unit 21 causes the sound generation unit 28 to produce an incorrect musical tone (step S11) and proceeds to step S12. In step S11, the control unit 21 causes the sound generation unit 28 to generate a trumpet sound at a pitch that corresponds to a combination of the operation of the control of the performance device 1 and the tilt angle of the performance device 1. In order to enable the player to recognize that the answer is incorrect, a beep may also be generated by the sound generating unit 28 in step S11.

[0046] In step S12, the control unit 21 determines whether or not the performance has ended (step S12).

[0047] For example, when the display of the performance guide information is completed up to the end of the piece of music to be played, it is determined that the performance is completed. When it is determined that the performance has not ended (step S12; NO), the control unit 21 returns to step S2.

[0048] When it is determined that the performance has ended (step S12; YES), the control unit 21 scores the performance, outputs the score result (step S13), and ends the performance processing. For example, the control unit 21 counts the number of times that the combination of the tilt angle of the performance device 1 and the pressing / releasing of the operator is judged to match the performance information stored in the storage unit 25 (the number of correct answers) and the number of times that the combination is judged to not match (the number of incorrect answers), and scores the performance based on the number of correct answers and the number of incorrect answers. For example, if the number of correct answers relative to the total number of correct answers and the number of incorrect answers is equal to or greater than a predetermined threshold, the display unit 23 judges the performance as "GREAT!" and displays it, whereas if the number of correct answers relative to the total number of correct answers and the number of incorrect answers is less than the predetermined threshold, the display unit 23 judges the performance as "BAD!" and displays it. In the case of "GREAT!", the sound unit 28 may sound a sound effect such as a cheer, and in the case of "BAD!", a beep sound or the like may be sounded. Furthermore, in the case of "BAD!", the display unit 23 may display a monochrome image.

[0049] When the performance processing is started, the control unit 21 causes the image capturing unit 29 to start capturing a moving image in parallel with the performance processing, analyzes each frame image acquired by the image capturing unit 29, and judges whether or not the performer has winked. If it is judged that the performer has winked, the control unit 21 causes the display unit 23 to display an image of the avatar 232 winking, as shown in Fig. 10, and causes the sound generating unit 28 to emit a sound effect corresponding to the wink. Note that the facial expression reflected on the avatar 232 is not limited to a wink.

[0050] During the performance process, the control unit 21 may obtain an output value from the acceleration sensor 27, and detect left-right movement of the performance device 1 based on the obtained output value. When left-right movement is detected, the control unit 21 may cause the sound output unit 28 to emit a predetermined sound effect. The manner in which the performer moves the performance device 1 left-right may also be reflected in the avatar 232. This allows the performer to perform an enjoyable performance operation by moving the performance device 1 up, down, left, and right. Performance The user may perform a performance operation by moving the performance device 1 diagonally.

[0051] As described above, the control unit 21 of the terminal device 2 acquires operation information of the operator provided on the performance device 1 to which the terminal device 2 is attached and the vertical tilt angle of the entire performance device 1, and causes the sound generation unit 28 to generate musical sounds of a pitch that corresponds to the operation of the operator and the vertical tilt of the entire performance device 1. Therefore, one of the playing actions required for a conventional musical instrument (for example, blowing air) is replaced with moving the entire performance device 1 up and down, and musical tones are generated at a pitch that corresponds to the operation of the controls and the moving the entire performance device 1 up and down, allowing you to enjoy playing easily.

[0052] In addition, the control unit 21 causes the display unit 23 to display performance guide information 231 which indicates which of the operators should be operated and how to move the performance apparatus 1 as a whole. Therefore, the performer can easily enjoy playing while looking at the playing guide information 231 displayed on the terminal device 2.

[0053] In addition, when the microphone 26, which detects the performer blowing into the performance device 1, detects the performer blowing into the performance device 1, the control unit 21 causes the sound generation unit 28 to generate a trumpet sound of a pitch that corresponds to the combination of the operation of the controls and the movement of the entire performance device 1. Therefore, the player can easily enjoy playing the trumpet.

[0054] Furthermore, control unit 21 causes avatar 232 of the performer to be displayed on display unit 23, obtains information on the facial expression of the performer during performance from image capture unit 29, and reflects the information in the facial expression of avatar 232. Therefore, since the performer can have his / her own facial expression reflected in avatar 232, he / she can enjoy changing his / her facial expression during performance. Furthermore, the control unit 21 displays the performer's avatar 231 on the display unit 23, and moves the avatar 231 based on the tilt angle acquired from the acceleration sensor 27. Therefore, the performer can have the manner in which he or she is moving the performance device 1 reflected in the avatar 232, and can enjoy incorporating movement into the performance.

[0055] The contents described in the above embodiment are preferred examples of the program, performance system, sound generation control method, and information processing device according to the present invention, and are not limited to these. For example, in the above embodiment, the microphone 26 provided in the terminal device 2 is used as the microphone for detecting the performer's blowing into the performance device 1, but a microphone for detecting the blowing of breath into the performance device 1 may be provided on the performance device 1 side. For example, a microphone may be provided near the boundary between the housing 14 and the mouth pipe 12. An electrical signal obtained by the microphone may be transmitted to the terminal device 2 by the communication unit 18.

[0056] In addition, in the above embodiment, the value of the electrical signal from microphone 26 (a value corresponding to the amount of breath blown into the instrument) and the tilt angle of performance device 1 are obtained as information relating to the movements of the performer and are reflected in the movements of avatar 232 (such as puffing out cheeks and leaning the body). However, information relating to the movements of the performer may also be obtained based on images captured by image capture unit 29 during performance processing and reflected in avatar 232.

[0057] In the above embodiment, the performance device 1 and the terminal device 2 communicate with each other via the terminal 18a and the terminal 2a. However, wireless communication (eg, Bluetooth (registered trademark)) may be used.

[0058] In the above embodiment, the operation information of the operation unit 13 is transmitted to the terminal device 2 by communication. However, the method of transmitting the operation information from the performance device 1 to the terminal device 2 is not limited to communication. For example, the performance device 1 may be provided with a structure that, when the first valve 13a to the third valve 13c are pressed, a predetermined position (a different position depending on the first valve 13a, the second valve 13b, and the third valve 13c) of the display unit 23 of the terminal device 2 mounted on the mounting part 15 is touched. Then, the operation information of the operation unit 13 and the like may be transmitted to the terminal device 2 by touching a predetermined position of the display unit 23 in response to pressing of the first valve 13a to the third valve 13c.

[0059] In the above embodiment, the control unit 21 acquires the vertical tilt angle of the performance device 1 as information indicating the overall movement of the performance device 1, but is not limited thereto, and may acquire, for example, the vertical or horizontal movement distance of the performance device 1, the horizontal rotation angle of the performance device 1, etc. as information indicating the overall movement of the performance device 1. Then, the sound generation unit 28 may generate musical tones of a pitch according to a combination of the player's operation of the controls of the performance device 1 and the overall movement of the performance device 1.

[0060] In addition, the control unit 21 may store data of the screen displayed on the display unit 23 during performance (the screen displaying the performance guide information 231 and the avatar 232) in the memory unit 25 so that the data can be posted (uploaded) later to a social networking site or the like.

[0061] In the above embodiment, the performance device 1 is a trumpet-type performance device, and the terminal device 2 causes the sound generator 28 to generate trumpet sounds in response to the performer blowing into the performance device 1. However, the performance device 1 may be of another instrument type (for example, another wind instrument type). In this case, the terminal device 2 causes the sound generator 28 to generate instrument sounds corresponding to the instrument type of the performance device 1.

[0062] Also, the performance device 1 may be configured to include a performance application, a display unit, a microphone, an acceleration sensor, a sound generation unit, a photographing unit, and the like for executing the performance process, and the performance process shown in Fig. 6 may be executed by cooperation between the control unit 17 of the performance device 1 and the performance application, thereby executing the functions of the control unit of the information processing device of the present invention. In other words, the performance device 1 may be configured to include the information processing device of the present invention. In this case, the control unit 17 can obtain operation information of the operators from the operation unit 13 without going through the communication unit 18.

[0063] Also, the performance device 1 may be equipped with a microphone and an acceleration sensor, and the performance device 1 and the terminal device 2 may be configured to be connectable by wireless communication such as Bluetooth, and the control unit 21 of the terminal device 2 may receive operation information of the controls from the performance device 1 by wireless communication, receive an electric signal from the microphone to detect the blowing of breath, and receive an output value from the acceleration sensor to obtain the tilt angle of the performance device 1, when executing the above-mentioned performance process in cooperation with the performance application 251. In this way, even if the terminal device 2 is not attached to the performance device 1, it is possible for the terminal device 2 to produce sounds in response to the operation of the controls of the performance device 1 and the movement of the entire performance device 1, and to display the performance guide information 231 and the avatar 232.

[0064] In the above embodiment, a semiconductor memory such as a ROM or a hard disk is used as a computer-readable medium for the program according to the present invention, but the present invention is not limited to this example. As other computer-readable media, a portable recording medium such as a CD-ROM can be applied. Furthermore, a carrier wave can be applied as a medium for providing data of the program according to the present invention via a communication line.

[0065] In addition, the detailed configurations and operations of the performance device 1 and the terminal device 2 may be modified as appropriate without departing from the spirit of the invention.

[0066] Although the embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the above-described embodiments. The scope of the present invention is not limited to the above, but is defined based on the claims. Furthermore, the scope of the present invention also includes equivalent modifications made from the claims that are not related to the essence of the present invention. [Explanation of symbols]

[0067] 100 Performance System 1 Performance device 11 Mouthpiece 12 Mouse pipe 13 Control section 13a First valve 13b Second valve 13c 3rd valve 13d Finger loop 14. Chassis 15 Mounting part 16. Public address section 17 Control Unit 18 Communications Department 18a terminal 2 Terminal Equipment 2a terminal 21 Control section 22 Control section 23 Display section 24 Communications Department 25 Memory section 251 Performance App 26. Mike 27 Acceleration Sensor 28 Pronunciation Section 29 Photography Department

Claims

1. Computer, acquiring operation information of an operator provided on a performance device to which the terminal device is attached and information indicating a movement of the performance device as a whole; causing a sound generator to generate musical tones of a pitch corresponding to a combination of the operation of the control and the movement of the entire performance device; A program to function as a control unit.

2. The control unit is displaying on a display unit of the terminal device performance guide information indicating the operators to be operated and how to move the entire performance device; The program according to claim 1.

3. The movement of the entire performance device is a movement of the performance device tilting in an up-down direction. The program according to claim 1.

4. The information indicating the movement of the entire performance device is acquired based on a sensor built into the terminal device. The program according to claim 1.

5. the musical performance device is a wind instrument type musical performance device, The control unit is when a detection unit for detecting the blowing of a breath into the performance device by a performer detects the blowing of a breath into the performance device, the sound generation unit generates the musical tone. The program according to claim 1.

6. The control unit is Display the avatar on the display. moving the avatar based on the acquired information indicating the movement of the entire performance device; The program according to claim 1.

7. The control unit is Display the avatar on the display. Acquiring information regarding the facial expression of the performer during the performance from a camera built into the terminal device; Reflecting the facial expression of the avatar, The program according to claim 1.

8. A terminal device; a performance device to which the terminal device is detachably attached, the performance device having an operator; The terminal device includes: acquiring operation information of the operator and information indicating a movement of the entire performance device; causing a sound generator to generate musical tones of a pitch corresponding to a combination of the operation of the control and the movement of the entire performance device; Playing system.

9. The computer Acquiring operation information of an operator provided on a detachably mounted performance device and information indicating a movement of the performance device as a whole; A sound generating section generates musical tones with pitches corresponding to a combination of the operation of the control and the movement of the entire performance device. to pronounce, Sound control method.

10. Acquiring operation information of an operator provided on a detachably mounted performance device and information indicating a movement of the performance device as a whole; causing a sound generator to generate musical tones of a pitch corresponding to a combination of the operation of the control and the movement of the entire performance device; An information processing device having a control unit.

Images