Information processing system, method for processing information, and program

Abstract

To confirm whether a performer is practicing a performance in an effective manner.SOLUTION: An information processing system includes: a performance analysis unit 52 for determining whether the performance using a musical instrument of a performer corresponds to a task; and a display control unit 53 for displaying the result of determination by the performance analysis unit 52 in the display device.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a technique for analyzing a musical instrument performance by a performer. [Background technology]

[0002] Various techniques for analyzing musical instrument performances by musicians have been proposed. For example, Patent Document 1 discloses a technique for identifying a reference piece of music played by a musician from among multiple candidates by calculating the degree of similarity between a sequence of notes specified by the musician in his / her performance and a sequence of notes in a reference piece of music for each reference piece of music. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-038308 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a demand for a performer's manager (e.g., an instructor or guardian) to be able to confirm whether the performer is practicing effectively. In consideration of the above circumstances, one aspect of the present disclosure aims to make it possible to confirm whether the performer is practicing effectively. [Means for solving the problem]

[0005] In order to solve the above problems, an information processing system according to one aspect of the present disclosure includes a performance analysis unit that determines whether a performer's performance of an instrument corresponds to a task, and a display control unit that displays the results of the determination made by the performance analysis unit on a display device.

[0006] An information processing method according to one aspect of the present disclosure determines whether a performer's performance of a musical instrument corresponds to a task, and displays the result of the determination on a display device.

[0007] A program according to one aspect of the present disclosure causes a computer system to function as a performance analysis unit that determines whether a performer's performance of an instrument corresponds to a task, and a display control unit that displays the results of the determination by the performance analysis unit on a display device. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating a configuration of an information processing system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram illustrating a configuration of a terminal device. [Figure 3] FIG. 1 is a block diagram illustrating the configuration of a keyboard instrument. [Figure 4] FIG. 1 is a schematic diagram of a keyboard animation. [Figure 5] FIG. 1 is a schematic diagram of a performance video. [Figure 6] FIG. 1 is a schematic diagram of a performance video. [Figure 7] Schematic diagram of pedal animation. [Figure 8] FIG. 10 is a schematic diagram of a setting screen. [Figure 9] FIG. 2 is a schematic diagram of performance data. [Figure 10] FIG. 2 is a block diagram illustrating a functional configuration of a terminal device. [Figure 11] FIG. 10 is a schematic diagram of a reference screen that displays practice status in a calendar format. [Figure 12] FIG. 10 is a schematic diagram of a reference screen G1. [Figure 13] FIG. 10 is a schematic diagram of a reference screen G2. [Figure 14] FIG. 10 is a schematic diagram of a reference screen G3. [Figure 15] 10 is a flowchart of a performance management process. [Figure 16] FIG. 10 is a schematic diagram of a reference screen G4. [Figure 17] FIG. 10 is a schematic diagram of a reference screen G5. [Figure 18] FIG. 10 is a schematic diagram of a reference screen G6. [Figure 19]FIG. 13 is a schematic diagram of an edited video in the fourth embodiment. [Figure 20] FIG. 10 is a block diagram illustrating a functional configuration of a terminal device according to a fourth embodiment. [Figure 21] FIG. 10 is a schematic diagram showing how to upload a video of a practice session. [Figure 22] FIG. 10 is an explanatory diagram regarding storage of performance data D in a modified example. [Figure 23] FIG. 10 is a schematic diagram of a music selection screen. DETAILED DESCRIPTION OF THE INVENTION

[0009] A: First embodiment 1 is a block diagram illustrating the configuration of an information processing system 100 according to a first embodiment. The information processing system 100 is a computer system (i.e., a performance recording system or a performance management system) for recording and managing a performance of a keyboard instrument 10 by a performer Ua. The information processing system 100 includes the keyboard instrument 10, a recording system 20, multiple terminal devices 30 (30a, 30b, 30c), and a measurement system 40.

[0010] Each terminal device 30 is an information device such as a smartphone, a tablet terminal, or a personal computer. Of the multiple terminal devices 30, terminal device 30a is used by a performer Ua who plays the keyboard instrument 10. The performer Ua is, for example, a person who practices playing the keyboard instrument 10. The performer Ua is, for example, a student belonging to a music school. The terminal device 30a is an example of a "control system."

[0011] Of the multiple terminal devices 30, terminal device 30b is used by instructor Ub. Instructor Ub instructs performer Ua in a music classroom. That is, instructor Ub uses terminal device 30 to manage the performance of the keyboard instrument 10 by performer Ua. Terminal device 30c is used by guardian Uc (e.g., parent) of performer Ua. Like instructor Ub, guardian Uc uses terminal device 30c to manage the performance of the keyboard instrument 10 by performer Ua. In the following description, when there is no need to particularly distinguish between performer Ua, instructor Ub, and guardian Uc, they will be collectively referred to as "user U."

[0012] 2 is a block diagram illustrating the configuration of a terminal device 30 (30a, 30b, 30c). The terminal device 30 includes a control device 31, a storage device 32, a communication device 33, a display device 34, an operation device 35, a sound emitting device 36, and a sound collecting device 37. The terminal device 30 may be realized as a single device, or may be realized as a plurality of devices configured separately from each other.

[0013] The control device 31 is composed of one or more processors that control each element of the terminal device 30. For example, the control device 31 is composed of one or more types of processors such as a central processing unit (CPU), a graphics processing unit (GPU), a sound processing unit (SPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0014] The storage device 32 is one or more memories that store programs executed by the control device 31 and various data used by the control device 31. The storage device 32 is configured with a known storage medium such as a magnetic storage medium or a semiconductor storage medium. The storage device 32 may be configured with a combination of multiple types of storage media. Furthermore, the storage device 32 may be a portable storage medium that is detachable from the terminal device 30, or a storage medium (e.g., cloud storage) that the control device 31 can write to or read from via a communication network.

[0015] The communication device 33 communicates with peripheral devices (e.g., the keyboard instrument 10, the recording system 20, other terminal devices 30, and the measurement system 40). The communication device 33 communicates with the peripheral devices via wireless communication, such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). The communication between the communication device 33 and the peripheral devices may be wired communication. The communication between the communication device 33 and the peripheral devices may be communication via a communication network, such as the Internet.

[0016] The display device 34 displays an image under the control of the control device 31. The display device 34 is configured with a display panel such as a liquid crystal panel or an organic EL panel. The operation device 35 is an input device that accepts operations by the user U (Ua, Ub, Uc). For example, an operator operated by the user U or a touch panel that detects contact by the user U is used as the operation device 35. Note that the display device 34 or the operation device 35, which are separate from the terminal device 30, may be connected to the terminal device 30 by wire or wirelessly.

[0017] The sound emitting device 36 emits sound waves under the control of the control device 31. The sound emitting device 36 is an output device such as a speaker or headphones. The sound emitting device 36 may include a D / A converter that converts the acoustic signal of the sound emitting target from digital to analog, and an amplifier that amplifies the acoustic signal. Note that the sound emitting device 36, which is separate from the terminal device 30, may be connected to the terminal device 30 by wire or wirelessly.

[0018] The sound collection device 37 collects ambient sounds to generate an acoustic signal (hereinafter referred to as a "collected sound signal"). Specifically, the sound collection device 37 collects the performance sound emitted from the keyboard instrument 10 by the performer Ua. In other words, the collected sound signal is a signal representing the waveform of the performance sound of the keyboard instrument 10. For example, the sound collection device 37 may be a single microphone or a microphone array in which multiple microphones are arranged. For convenience, an A / D converter that converts the collected sound signal from analog to digital and an amplifier that amplifies the collected sound signal are not shown in the figure. The sound collection device 37 may be separate from the terminal device 30 and connected to the terminal device 30 by wire or wirelessly. The control device 31 stores the collected sound signal generated by the sound collection device 37 in the storage device 32.

[0019] The terminal device 30 can establish communication with the keyboard instrument 10 by logging in to the keyboard instrument 10. Logging in to the keyboard instrument 10 requires authentication processing using, for example, the identification information of the keyboard instrument 10, and the identification information (user ID) and password of the user U. The terminal device 30 can obtain the identification information by, for example, reading an image code attached to the keyboard instrument 10. The image code is a code represented by an optically readable image (for example, a QR code (registered trademark)). The identification information of the keyboard instrument 10 may be input by the user U by operating the operation device 35, for example.

[0020] 3 is a block diagram illustrating the configuration of a keyboard instrument 10. The keyboard instrument 10 is an instrument that accepts a performance by a performer Ua, and includes a control device 11, a storage device 12, a communication device 13, a display device 14, an operation device 15, a sound source device 16, a sound output device 17, and a performance mechanism 18. The keyboard instrument 10 is an example of a performance device that the performer Ua uses for performance. Note that some or all of the functions of the terminal device 30 may be incorporated into the keyboard instrument 10.

[0021] The performance mechanism 18 includes a keyboard 181, a sound generation mechanism 183, pedals 184, and a detection device 185. The keyboard 181 is a mechanism in which a plurality of keys 182, including white keys and black keys, are arranged. Each of the plurality of keys 182 is a performance operator that receives operation (e.g., key depression and key release) by the performer Ua. The sound generation mechanism 183 is an action mechanism that generates performance sounds by vibrating strings in response to operation of each key 182 on the keyboard 181. Specifically, the sound generation mechanism 183 includes strings, which are an example of a sound source, hammers that rotate to strike the strings, and a transmission mechanism (e.g., a wippen, jack, or repetition lever) that rotates the hammer in response to the displacement of each key 182. The sound generation mechanism 183 also includes a mute mechanism that prevents the strings from being struck in mute mode.

[0022] The pedal 184 is a performance operator that the performer Ua operates with his / her foot. For example, the pedal 184 may be a damper pedal (sustain pedal) for instructing the extension of a performance sound, or a soft pedal for reducing the volume of a performance sound.

[0023] The detection device 185 is a sensor that detects the operation of each key 182 and pedal 184. Specifically, the detection device 185 detects the amount of displacement of each key 182 and pedal 184. The detection device 185 is configured, for example, by an optical, mechanical, or magnetic sensor.

[0024] The control device 11 is composed of one or more processors that control each element of the keyboard instrument 10. For example, the control device 11 is composed of one or more types of processors such as a CPU, a GPU, an SPU, a DSP, an FPGA, or an ASIC.

[0025] The control device 11 generates music data according to the detection result of the detection device 185. The music data is data representing a musical piece performed by the performer Ua. The musical piece to be performed includes a right-hand part that the performer Ua should play with his / her right hand HR and a left-hand part that the performer Ua should play with his / her left hand HL.

[0026] Specifically, the music data is a time series of event data conforming to, for example, the MIDI (Musical Instrument Digital Interface) standard. That is, the music data specifies, in time series, the pitches (note numbers) corresponding to the keys 182 operated by the performer Ua and the intensity (velocity) of the operation. The music data also represents the operation of the pedal 184 by the performer Ua. As described above, the control device functions as a music data generation unit that generates music data in accordance with the results of detection by the detection device 185.

[0027] The display device 14 displays an image under the control of the control device 31. The display device 14 is configured with a display panel such as a liquid crystal panel or an organic EL (Electroluminescence) panel. The operation device 15 is an input device that accepts operations by the performer Ua. For example, an operator operated by the performer Ua or a touch panel that detects contact by the performer Ua is used as the operation device 15. Note that the display device 14 or the operation device 15, which are separate from the keyboard instrument 10, may be connected to the keyboard instrument 10 by wire or wirelessly.

[0028] The sound source device 16 generates an audio signal representing a performance sound according to the result of detection by the detection device 185. Specifically, the sound source device 16 generates an audio signal according to the performance data generated by the control device 11. For example, an audio signal is generated having a pitch corresponding to the key 182 whose displacement has been detected by the detection device 185. Note that the control device 11 may realize the function of the sound source device 16 by executing a program stored in the storage device 12. As described above, the keyboard instrument 10 is an electronic musical instrument that can selectively generate natural sounds using the sound generation mechanism 183 and electronic sounds using the sound source device 16. Note that one of the sound generation mechanism 183 and the sound source device 16 may be omitted.

[0029] The sound emitting device 17 emits sound waves under the control of the control device 11. The sound emitting device 17 is an output device such as a speaker or headphones. Specifically, the sound emitting device 17 emits a performance sound represented by an acoustic signal generated by the sound source device 16. As described above, the keyboard instrument 10 of the first embodiment can selectively generate a performance sound by the sound generation mechanism 183 or by the sound source device 16. The sound emitting device 17 may include a D / A converter that converts the acoustic signal from digital to analog, and an amplifier that amplifies the acoustic signal. The sound emitting device 17 may be separate from the keyboard instrument 10 and connected to the keyboard instrument 10 by wire or wirelessly.

[0030] The storage device 12 is one or more memories that store programs executed by the control device 11 and various data used by the control device 11. The storage device 12 is configured with a known storage medium, such as a magnetic storage medium or a semiconductor storage medium. The storage device 12 may also be configured with a combination of multiple types of storage medium. The storage device 12 may also be a portable storage medium that is detachable from the keyboard instrument 10, or a storage medium to which the control device 11 can write or read via a communication network (e.g., cloud storage). The storage device 12 stores the operating characteristic Xa, tuning information Xb, and history information Xc.

[0031] The operating characteristics Xa are characteristics (instrument profile) specific to the instrument regarding the operation of the keyboard instrument 10. The operating characteristics Xa include a dynamic range, a velocity range, a frequency range, a repeated key characteristic, and a key depression sensitivity.

[0032] The dynamic range is the range of volume (the ratio between maximum and minimum volume) of the performance sounds that the keyboard instrument 10 can generate through the sound generation mechanism 183 or the sound source device 16. The velocity range is the range of key pressing speed (key pressing strength). Since the volume of the performance sounds generated by the sound source device 16 corresponds to the key pressing speed, the velocity range is also expressed as the range of the performance sound volume. The frequency range is the range of frequencies of the performance sounds that the keyboard instrument 10 can generate through the sound generation mechanism 183 or the sound source device 16. The repeated keying characteristic is the maximum number of key presses per unit time. The key pressing sensitivity is the relationship (touch sensitivity) between the key pressing speed or displacement amount and the performance volume of each key 182. The key pressing sensitivity includes, for example, the key depth (mm), half touch depth (mm), aftertouch depth (mm), resonance (Hz), the downweight (g) of each key 182, and the return weight (g) of each key 182.

[0033] The information included in the operating characteristics Xa is not limited to the above examples. For example, the operating characteristics Xa may include pedal settings (differences in damper, sostenuto, muffler, shift, softness, etc.), location information of the installation location of the keyboard instrument 10, the type of building where the installation location is located (recital hall, concert hall, salon, cathedral, club, plate, soundproof room, setting value), and reverberation measurement results (dB-Time) of the installation location.

[0034] The tuning information Xb is information relating to the tuning of the keyboard instrument 10. The tuning information Xb includes, for example, a reference pitch or temperament. The reference pitch is a reference pitch (e.g., 440 Hz, 441 Hz, 442 Hz, etc.) during tuning. The temperament is set to, for example, equal temperament, just intonation, Pythagorean temperament, or middle temperament. The tuning information Xb is input by a tuner when tuning the keyboard instrument 10. Note that the initial values of each piece of information, including the tuning information Xb, are set, for example, before the keyboard instrument 10 is shipped.

[0035] The information included in the tuning information Xb is not limited to the above examples. For example, the tuning information Xb also includes information such as rapid-fire performance, key depth, aftertouch, velocity range, dynamic range, resonance, downweight, upweight, and comments. While some of the tuning information Xb overlaps with the operating characteristics Xa, the results of changes made during tuning or over time are saved. Even for acoustic pianos that do not have a detection device 185, tuning information Xb is input and saved during tuning and used for comparison with other instruments. During tuning, the terminal device 30 is placed in a specific position, such as a music stand, and the volume or frequency measured by the terminal device 30 can be saved as tuning information Xb. The measured values can also be converted using a certain ratio and saved as tuning information Xb.

[0036] The historical information Xc is information relating to the performance by the performer Ua, and is temporarily stored in the keyboard instrument 10 until it is transferred to the terminal device 30. That is, after being transferred to the terminal device 30, the historical information Xc is deleted from the storage device 12. The historical information Xc is also expressed as information (performance log) that represents the history of the performance by the performer Ua.

[0037] Specifically, the historical information Xc includes the identification information of the performer Ua, the sound source used, the mode used, the performance date and time (start time and end time), the performance location, the performance scene, the identification information of the keyboard instrument 10, the model, the training text used by the performer Ua, and the sheet music information of the music played by the performer Ua.

[0038] The communication device 13 communicates with the terminal device 30. The communication device 13 communicates with the terminal device 30 via wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark). For example, the communication device 13 transmits the operation characteristic Xa, tuning information Xb, and history information Xc stored in the storage device 12 to the terminal device 30. Note that the communication between the communication device 13 and the terminal device 30 may be wired communication. The communication device 13, which is separate from the keyboard instrument 10, may be connected to the keyboard instrument 10 by wire or wirelessly.

[0039] 1 records a performance by a performer Ua. Specifically, the recording system 20 includes multiple image capture devices 21 and one sound collection device 22. Note that some or all of the multiple image capture devices 21 may be mounted on the terminal device 30 or the keyboard instrument 10. Similarly, the sound collection device 22 may be mounted on the terminal device 30 or the keyboard instrument 10. Note that the sound collection device 22 may also be mounted on any of the multiple image capture devices 21.

[0040] Each imaging device 21 is a video camera that generates a video of the performer Ua and the keyboard instrument 10 (hereinafter referred to as a "recorded video Ya"). That is, the recording system 20 generates a plurality of recorded videos Ya by capturing images of the performance by the performer Ua. Each imaging device 21 includes, for example, an optical system such as a photographing lens, an imaging element that receives incident light from the optical system, and a processing circuit that generates data of the recorded video Ya according to the amount of light received by the imaging element. Note that in addition to video equipment dedicated to imaging, a general-purpose information device (e.g., a smartphone or tablet terminal) equipped with an imaging function may also be used as the imaging device 21. Furthermore, the format of the data representing the recorded video Ya is arbitrary.

[0041] The multiple imaging devices 21 are installed at different positions and angles relative to the performer Ua and the keyboard instrument 10. Furthermore, the angle of view of each imaging device 21 is different. That is, each imaging device 21 captures a different part of the performer Ua or the keyboard instrument 10. The multiple imaging devices 21 generate recorded videos Ya in parallel with each other in parallel with the performance by the performer Ua. That is, the recording system 20 generates multiple recorded videos Ya that are recorded in parallel at different positions or angles.

[0042] 4 to 7 are schematic diagrams of recorded videos Ya. The recorded videos Ya generated by the recording system 20 include a keyboard video Ya1, a performance video Ya2, a performance video Ya3, and a pedal video Ya4.

[0043] 4, the keyboard video Ya1 is a video captured from above the keyboard instrument 10 of the keys 181 and the right hand HR and left hand HL of the performer Ua. In other words, the keyboard video Ya1 includes the keys 181 used to play the musical piece and both hands of the performer Ua. The keyboard video Ya1 does not include the face of the performer Ua.

[0044] 5, the performance video Ya2 is a video captured from the side of the performer Ua. The performance video Ya2 includes the performer Ua's face (e.g., a profile), the performer Ua's right hand HR and left hand HL, and the side of the keyboard instrument 10. In other words, the performance video Ya2 shows the performer Ua's facial expressions while performing the musical piece.

[0045] 6, the performance video Ya3 is a video captured from the right rear of the performer Ua. The performance video Ya3 includes the back and back of the head of the performer Ua, the right hand HR and both feet of the performer Ua, and the front of the keyboard instrument 10. The performance video Ya3 does not include the face of the performer Ua.

[0046] 7, the pedal video Ya4 is a video capturing images of the pedals 184 of the keyboard instrument 10 and the feet of the performer Ua. In other words, the pedal video Ya4 includes both feet of the performer Ua and the pedals 184 of the keyboard instrument 10. The pedal video Ya4 does not include the face of the performer Ua.

[0047] 1 is one or more microphones that record sounds around the performer Ua (hereinafter referred to as "recorded sounds Yb"). Specifically, the sound collection device 22 is installed facing the keyboard instrument 10, and records the performance sounds emitted from the keyboard instrument 10 by the performance of the performer Ua as recorded sounds Yb. In addition to devices dedicated to sound collection, a general-purpose information device (e.g., a smartphone or tablet terminal) equipped with a sound collection function may also be used as the sound collection device 22. Furthermore, the format of the data representing the recorded sounds Yb is arbitrary.

[0048] Each imaging device 21 is connected to the terminal device 30 by wire or wirelessly. The recorded video Ya recorded by each imaging device 21 is transmitted to the terminal device 30. Similarly, the sound collection device 22 is connected to the terminal device 30 by wire or wirelessly. The recorded sound Yb by the sound collection device 22 is transmitted to the terminal device 30. As described above, the recording data Y representing the multiple recorded videos Ya and recorded sounds Yb is transmitted from the recording system 20 to the terminal device 30.

[0049] The terminal device 30 manages the temporal correspondence between each recorded video Ya and recorded sound Yb generated by the recording system 20 and the performance data generated by the keyboard instrument 10. Specifically, the terminal device 30 transmits synchronization data to the recording system 20 and the keyboard instrument 10. Each imaging device 21 adds synchronization data to the recorded video Ya, and each sound collection device 22 adds synchronization data to the recorded sound Yb. The keyboard instrument 10 also adds synchronization data to the performance data. Therefore, by referring to the synchronization data, the control device 31 can identify the corresponding points in time between each recorded video Ya, recorded sound Yb, and performance data. In other words, it is possible to synchronize each recorded video Ya, recorded sound Yb, and performance data D with each other. The synchronization data is, for example, LTC (Longitudinal Time Code) or MIDI time code.

[0050] 8 is a schematic diagram of a screen (hereinafter referred to as a "setting screen 60") displayed on the display device 34 for managing the connection between the recording system 20 and the terminal device 30. The setting screen 60 is displayed on the display device 34 in response to an operation by the user U on the operation device 35. After completing the settings for the recording system 20 using the setting screen 60, the performer Ua starts playing using the keyboard instrument 10.

[0051] The setting screen 60 includes a guide screen 61. The guide screen 61 is an image that guides the user U (Ua, Ub, Uc) to the position where each device of the recording system 20 should be installed. Specifically, on the guide screen 61, a plurality of device icons 62 representing each device are arranged at the position where each device should be installed relative to the keyboard instrument 10.

[0052] The multiple device icons 62 include an imaging icon 62a and a sound collection icon 62b. The imaging icon 62a is an image representing the imaging device 21 connected to the terminal device 30. The sound collection icon 62b is an image representing the sound collection device 22 connected to the terminal device 30. By operating the operation device 35, the user U can move the imaging icon 62a to a position corresponding to the actual position of each imaging device 21 relative to the keyboard instrument 10. Similarly, the user U can move the sound collection icon 62b to a position corresponding to the actual position of the sound collection device 22 relative to the keyboard instrument 10.

[0053] While referring to the guide screen 61 , the user U sets up the recording system 20 around the keyboard instrument 10 and connects each of the image capturing devices 21 and sound collecting devices 22 to the terminal device 30 .

[0054] As illustrated in FIG. 8, the setting screen 60 includes a scene selection window 63 and an equipment setting window 64. The scene selection window 63 is an image that allows the user U to select a performance scene by the performer Ua. Specifically, a plurality of performance scenes (normal performance, performance lesson, recital, street performance) with different performance purposes or situations are displayed in the scene selection window 63 as selection candidates for the user U. The user U can select one of the plurality of performance scenes by operating the operation device 35. The performance scene selected by the user U is shared between the terminal device 30 and the keyboard instrument 10.

[0055] The location where each device in the recording system 20 should be installed varies for each performance scene. Therefore, the guidance screen 61 changes depending on the performance scene selected by the user U from the scene selection window 63. In other words, the guidance screen 61 guides the user U to the appropriate location of each device in each performance scene. Figure 8 shows the setting screen 60 when "performance training" is selected as the performance scene.

[0056] The measurement system 40 in FIG. 1 measures biometric information Z related to the player Ua. Specifically, the measurement system 40 includes a biometric sensor 41 and a center-of-gravity sensor 42. The biometric sensor 41 is a portable sensor worn on the body of the player Ua. The biometric sensor 41 is, for example, a heart rate sensor that detects the heart rate of the player Ua. The center-of-gravity sensor 42 is a sensor installed on the chair on which the player Ua sits while playing the keyboard instrument 10, and detects movement of the center of gravity of the seat. The biometric information Z of the player Ua is data including, for example, the playing posture, tension (strain), heart rate, and concentration of the player Ua.

[0057] The playing posture is an index relating to the posture of the performer Ua playing the keyboard instrument 10. The playing posture is determined by analyzing the detection results of the center of gravity sensor 42. Note that the playing posture may also be determined by analyzing a recorded video Ya of the performer Ua captured by the imaging device 21.

[0058] The heart rate is detected, for example, by a heart rate sensor. The tension level is an index of the player Ua's level of tension (strain) and is calculated, for example, based on the player Ua's heart rate and the stability of his / her playing posture. The concentration level is an index of the player Ua's level of concentration on playing and is calculated, for example, based on the player Ua's heart rate and the stability of his / her playing posture. Note that the concentration level may be set to a larger value the more times a key is pressed within a specified time. Note that the configuration and type of the measurement system 40 are not limited to the above examples.

[0059] In the configuration described above, the terminal device 30 acquires various information from the peripheral devices (the keyboard instrument 10, the recording system 20, and the measurement system 40). For example, the control device 31 transmits an information request to the peripheral devices from the communication device 33, and receives, via the communication device 33, information transmitted from the peripheral devices in response to the information request. Specifically, the terminal device 30 acquires the operation characteristics Xa, tuning information Xb, and history information Xc from the keyboard instrument 10, acquires recorded data Y representing a plurality of recorded videos Ya and recorded sounds Yb from the recording system, and acquires biometric information Z of the performer Ua from the measurement system 40.

[0060] As illustrated in FIG. 9, performance data D is stored in the storage device 32 of the terminal device 30. The performance data D is comprehensive data related to the performance by the performer Ua. Specifically, the performance data D includes music data corresponding to the results of detection by the detection device 185, motion characteristics Xa, tuning information Xb, history information Xc, recorded data Y, and biometric information Z. The performance data D is stored in the storage device 32 for each performance by the performer Ua (each time the performer logs in to the keyboard instrument 10). The terminal device 30 executes various processes using the performance data D.

[0061] 1 is a computer system (cloud storage) that records various types of information. Some or all of the information stored in the storage device 32 of each terminal device 30 or the storage device 12 of the keyboard instrument 10 may be stored in the recording system 90. In other words, some or all of the storage device 32 or the storage device 12 may be replaced by the recording system 90. For example, some or all of the performance data D may be transferred from peripheral devices (the keyboard instrument 10, the recording system 20, the measurement system 40) to the recording system 90, and then transmitted from the recording system 90 to the terminal device 30 in response to a request from the terminal device 30.

[0062] 10 is a block diagram illustrating an example of the functional configuration of the terminal device 30. The storage device 32 stores musical score data Dm of a piece of music that the performer Ua should practice (hereinafter referred to as the "challenge piece Ma") and performance items that the performer Ua should practice (hereinafter referred to as the "challenge items Na").

[0063] The assignment piece Ma is a piece of music that the performer Ua should prioritize as a challenge, and is determined in advance based on the performer Ua's level of performance proficiency. The score data Dm is data representing the score of the assignment piece Ma. Specifically, the score data Dm specifies the pitch, duration, and intensity of each of the multiple notes that make up the assignment piece Ma. For example, the score data Dm is data that conforms to the MIDI standard. Note that the score data Dm may also specify information such as performance symbols that express musical expression.

[0064] The instructor Ub or guardian Uc determines the level of proficiency of the performer Ua's performance (hereinafter referred to as "proficiency level") based on the performance data D representing the performer Ua's performance. The proficiency level may also be determined automatically based on the degree of improvement in the assignment items Na. For example, the control device 31 determines that the performer Ua has reached a specific level of proficiency when the performer Ua is able to perform more than a predetermined percentage of the multiple performance items. The performer Ua's proficiency level may be included in the performance data D, for example, together with the performer Ua's identification information.

[0065] The score data Dm may be included in the performance data D stored in the storage device 32, or may be data representing a performance of the keyboard instrument 10 by the instructor Ub or guardian Uc. The score data Dm may also be distributed to the terminal device 30 from a music distribution system (e.g., a cloud server). The score data Dm may include not only the score of the assigned piece Ma but also the assignment items Na. By storing the score data Dm in the music distribution system, the score data Dm can be shared by each terminal device 30.

[0066] The performance items are various items related to playing the keyboard instrument 10. Specifically, performance techniques that are targets to be acquired through practice of the keyboard instrument 10 are exemplified as performance items. For example, various performance techniques or performance expressions related to playing an instrument, such as playing with stackers, playing chords, playing finger crossings, and playing broken octaves, are exemplified as performance items. Of the multiple performance items, one or more performance items that the performer Ua should prioritize as a challenge are stored in the storage device 32 as challenge items Na. As can be understood from the above explanation, the challenge piece Ma and challenge item Na are challenges that the performer Ua should prioritize when practicing the keyboard instrument 10.

[0067] The assignment piece Ma and assignment items Na may be set by, for example, the performer Ua, the instructor Ub, or the guardian Uc. The assignment piece Ma and assignment items Na are updated in each terminal device 30 in response to instructions from the user U (Ua, Ub, Uc).

[0068] The terminal device 30 analyzes the performance of the keyboard instrument 10 by the performer Ua using the performance data D stored in the storage device 32. The results of the analysis of the performance by the performer Ua are displayed on the terminal device 30b of the instructor Ub and the terminal device 30c of the guardian Uc. Specifically, the control device 31 executes the program stored in the storage device 32 to realize multiple functions (an information acquisition unit 51, a performance analysis unit 52, and a display control unit 53) for generating the reference screen G.

[0069] The information acquisition unit 51 acquires various types of information from peripheral devices (e.g., the keyboard instrument 10, the recording system 20, other terminal devices 30, and the measurement system 40) via the communication device 33, and stores performance data D including the information in the storage device 32. Specifically, the information acquisition unit 51 acquires music data corresponding to the results of detection by the detection device 185, motion characteristics Xa, tuning information Xb, and history information Xc from the keyboard instrument 10, acquires recording data Y from the recording system 20, and acquires biometric information Z from the measurement system 40. The information acquisition unit 51 also acquires score data Dm of the assigned piece Ma and assignment items Na from the terminal device 30b of the instructor Ub.

[0070] For example, the information acquisition unit 51 of the terminal device 30a of the performer Ua transmits an information request including the identification information of the performer Ua from the communication device 33 to the terminal device 30b of the instructor Ub. The display device 34 of the terminal device 30b displays the identification information of the performer Ua included in the information request on the display device 34. The terminal device 30b receives from the instructor Ub a selection of the set piece Ma corresponding to the performer Ua and a selection of the task item Na corresponding to the performer Ua. The terminal device 30b transmits the score data Dm of the set piece Ma selected by the instructor Ub and the task item Na selected by the instructor Ub to the terminal device 30a that issued the request. The information acquisition unit 51 of the terminal device 30a receives the score data Dm and the task item Na transmitted from the terminal device 30b via the communication device 33. The information received from the terminal device 30b (the score data Dm, the task item Na) is stored in the storage device 32. The information acquisition unit 51 of the terminal device 30a may acquire the musical score data Dm and the assignment items Na from the terminal device 30c of the guardian Uc.

[0071] The performance analysis unit 52 analyzes the performance of the keyboard instrument 10 by the performer Ua using the performance data D acquired by the information acquisition unit 51. The performance analysis unit 52 determines whether or not the performance by the performer Ua corresponds to a task that should be prioritized. In other words, it determines whether or not the performer Ua is working on the task.

[0072] Specifically, the performance analysis unit 52 determines whether or not the performer Ua has performed the set piece Ma. For example, the performance analysis unit 52 compares the score data Dm of the set piece Ma with music data representing the performance by the performer Ua to determine whether or not the piece of music performed by the performer Ua (hereinafter referred to as the "performed piece") is the set piece Ma. The performance analysis unit 52 stores the time, place, equipment used, and recording data Y when the performer Ua performed the set piece Ma, separate from other information.

[0073] If the piece played by the performer Ua differs from the assignment piece Ma, the performance analysis unit 52 identifies the piece played by the performer Ua (hereinafter referred to as the "etudes Mb"). The etudes Mb are pieces that the performer Ua has independently practiced, separate from the assignment piece Ma. For example, the performance analysis unit 52 estimates the etudes Mb played by the performer Ua by comparing music data distributed from a music distribution system with music data of the performance by the performer Ua. The performance analysis unit 52 stores the time, place, equipment used, and recording data Y of the performance of the etudes Mb by the performer Ua, separate from other information.

[0074] The performance analysis unit 52 is capable of detecting performance items performed by the performer Ua. Specifically, performance items such as repeated hits, trills, staccatos, slurs, arpeggios, and ornaments are detected from the performance of the performer Ua. For example, the score data Dm compared with the performance of the performer Ua includes not only the musical score of the piece of music but also the performance items within the piece of music. The performance analysis unit 52 determines the performance items performed by the performer Ua by comparing the score data Dm with music data representing the performance by the performer Ua. Specifically, the performance analysis unit 52 determines whether the performer Ua performed the assignment item Na. The performance analysis unit 52 determines whether the performance items detected from the performance of the performer Ua match the assignment item Na stored in the storage device 32.

[0075] If the performance item by the performer Ua does not correspond to the assignment item Na, the performance analysis unit 52 identifies the performance item by the performer Ua (hereinafter referred to as "practice item Nb"). That is, the practice item Nb is distinguished from the assignment item Na. For example, even if "Turkish March" is instructed as the assignment piece Ma and "split octaves" is instructed as the assignment item Na by the instructor Ub, the performer Ua may be practicing the assignment item Na "split octaves" by playing the etude Mb "Hanon," which is easier in difficulty than the assignment piece Ma. The performance analysis unit 52 distinguishes the performance of the assignment item Na according to the piece being performed (etude Ma / practice piece Mb).

[0076] The performance analysis unit 52 can evaluate not only the music data representing the performance by the performer Ua, but also the posture information of the performer Ua detected by the center of gravity sensor 42. If the center of gravity of the performer Ua while playing is offset forward or backward, it is determined that the performer Ua is performing with poor posture. Furthermore, by analyzing the information detected by the detection device 185, it can be determined that the performer Ua is playing with more force than necessary, or that the key-pressing speed of the keyboard 181 is too slow to utilize the dynamic range of the keyboard instrument 10.

[0077] By analyzing the results of recording by the recording system 20 (recorded data Y), it is possible to determine whether the performer Ua is looking at the keyboard 181 or the musical score, or whether he is looking away. Using known emotion evaluation technology, it is possible to detect the performer Ua's level of concentration from his facial expression, or to calculate the number of keystrokes per unit time, which can be used as an index of concentration to determine whether he is concentrating on practicing. Note that the data used for performance evaluation is not limited to music data. For example, the performance of the performer Ua can be evaluated by comparing the recorded sound Yb of a performance by the instructor Ub with the recorded sound Yb of a performance by the performer Ua.

[0078] The performance analysis unit 52 can save the performance data D by distinguishing between locations such as a music school, home, and a recital venue. The saved performance data D can be classified not only as MIDI data, which is performance information, but also as image data (video) of the performer Ua performing, along with evaluation results. Generally, a performance video at a recital where the performer Ua shows off their performance skills is an important record for the performer Ua. Furthermore, since the performer Ua has accumulated practice in the classroom and at home up until the recital, the performance data D can be saved as practice videos of the classroom and home as information related to the recital. By using the performance analysis unit, it is possible to add evaluated tag information to the vast amount of data such as audio, MIDI, and images generated by the performance data generation unit and save it.

[0079] The display control unit 53 displays the results of the analysis by the performance analysis unit 52 on the display device 34 of each terminal device 30. Specifically, the display control unit 53 generates image data representing the reference screen G and transmits the image data to the display device 34 of each terminal device 30, thereby causing the display device 34 to display the reference screen G (G0, G1, G2).

[0080] FIG. 11 shows a reference screen G0 that displays practice status in a calendar format. The reference screen G0 is an image displayed on the display device 34 of the terminal device 30b of the instructor Ub or the terminal device 30c of the parent / guardian Uc. The reference screen G0 displays daily practice time related to playing the keyboard instrument 10 in a calendar format. When the performer Ua logs in to the terminal device 30 and the keyboard device 10 installed in the music studio, at home, or at the recital venue using the terminal device 30 and his / her own ID, the performance history of the performer Ua on each keyboard device is displayed on one screen. Clicking on the time of a specific day on the reference screen G1 displays details of the practice status, which will be described later.

[0081] 12 is a schematic diagram of a reference screen G1. The reference screen G1 is an image displayed on the display device 34 of the terminal device 30b of the instructor Ub or the terminal device 30c of the guardian Uc. The reference screen G1 is an image that shows the proportion of time spent by the player Ua practicing each of a plurality of items related to playing the keyboard instrument 10. In other words, the reference screen G1 is an image that allows the instructor Ub or the guardian Uc to check the status of the player Ua's practice of the keyboard instrument 10.

[0082] Specifically, the reference screen G1 includes a count image R (R1, R2, ...) for each of a plurality of counting periods τ (τ1, τ2, ...). The counting period τ is a period during which the performance results of the performer Ua are to be counted. For example, each day of the week (Monday, Tuesday, ...) is exemplified as the counting period τ.

[0083] The summary image R for each collection period τ is a bar graph that visualizes the content of the player Ua's practice during that collection period τ. Specifically, the summary image R is divided into multiple regions r (r1 to r4) corresponding to different items practiced by the player Ua. The display length of each region r in the vertical direction is set according to the proportion of the time the player Ua spent practicing the item corresponding to that region r out of the total time the player Ua spent practicing during the collection period τ.

[0084] Areas r1 and r2 of the tally image R correspond to the time during the tally period τ that the performer Ua played the set piece Ma. Specifically, area r1 represents the proportion of the length of time that the performer Ua played the set item Na to the total time that the performer Ua played the set piece Ma. On the other hand, area r2 represents the proportion of the length of time that the performer Ua did not play the set item Na to the total time that the performer Ua played the set piece Ma. As explained above, areas r1 and r2 of the reference screen G1 represent the time that the performer Ua played the set piece Ma.

[0085] Regions r3 and r4 of the tally image R correspond to the time during the tally period τ during which the performer Ua played the etude Mb other than the task piece Ma. Specifically, region r3 represents the proportion of the length of time during which the performer Ua played the task item Na to the total time during which the performer Ua played the etude Mb. That is, regions r1 and r3 represent the time during which the performer Ua played the task item Na. On the other hand, region r4 represents the proportion of the length of time during which the performer Ua played the etude Mb to the total time during which the performer Ua did not play the task item Na.

[0086] As explained above, the reference screen G1 displays the time ratio during which the performer Ua played the assignment piece Ma and the time ratio during which the performer Ua played the assignment item Na during the aggregation period τ during which the performer Ua played.

[0087] As described above, in the first embodiment, whether the performance by the performer Ua corresponds to the assignment is determined and displayed. Therefore, the instructor Ub and the guardian Uc can confirm whether the performer Ua is practicing in a way that corresponds to the assignment. In other words, the instructor Ub and the guardian Uc can confirm whether the performer Ua is practicing effectively to solve the assignment related to the performance.

[0088] Specifically, it is determined whether the performer Ua has performed the task Na. Therefore, the instructor Ub and the guardian Uc can confirm whether the performer Ua is actually practicing the task Na. In the first embodiment, the time taken for the performer Ua to perform the task Na is particularly displayed on the reference screen G1. Therefore, the instructor Ub and the guardian Uc can confirm the amount of practice of the task Na by the performer Ua.

[0089] It is also determined whether the performer Ua has performed the set piece Ma. Therefore, the instructor Ub and guardian Uc can confirm whether the performer Ua is actually practicing the set piece Ma. In other words, the instructor Ub and guardian Uc can confirm whether the performer Ua is performing effective practice to master the set piece Ma. In the first embodiment, in particular, the time the performer Ua has spent playing the set piece Ma is displayed on the reference screen G1. Therefore, the instructor Ub and guardian Uc can confirm the amount of practice the performer Ua has performed of the set piece Ma.

[0090] Also, a reference screen G1 is displayed that shows the proportion of time spent on the assignment item Na and the proportion of time spent on the assignment piece Ma in the performance by the performer Ua. Therefore, for example, the performer Ua's instructor Ub and guardian Uc can visually and intuitively grasp the amount of practice the performer Ua has actually done on the assignment item Na and the assignment piece Ma.

[0091] The instructor Ub or the guardian Uc can operate (e.g., tap) areas r1 and r2 corresponding to the assignment piece Ma on the reference screen G1 displayed on the display device 34. When area r1 or area r2 on the reference screen G1 is operated, the display control unit 53 displays the reference screen G2 of Fig. 13 on the display device 34. That is, the reference screen G1 and the reference screen G2 are displayed on the terminal device 30b of the instructor Ub or the terminal device 30c of the guardian Uc.

[0092] The reference screen G2 includes a score 711 of the challenge piece Ma represented by the musical score data Dm, and performance comments 712 regarding the performance of the challenge piece Ma. The performance comments 712 are character strings that are associated with the musical score data Dm of the challenge piece Ma and stored in the storage device 32. Note that the display of the performance comments 712 may be omitted.

[0093] The reference screen G2 includes a display according to the results of the analysis by the performance analysis unit 52. Specifically, the display control unit 53 displays portions 72 of the notes played by the performer Ua that differ from the notes of the assigned piece Ma represented by the musical score data Dm in a manner different from that of the normal musical score 711. In other words, the notes that correspond to the performance mistakes made by the performer Ua are highlighted.

[0094] Furthermore, each section of the score 711 on the reference screen G2 is displayed in a manner corresponding to the number of times the performer Ua has performed that section. Specifically, the display control unit 53 displays a section of the score 711 with a higher density gradation the more times the performer Ua has performed that section. The term "display manner" refers to image characteristics that can be visually distinguished by an observer. For example, the display color, pattern (design), size, or shape of the image are encompassed within the concept of "display manner." The display color is defined by hue (tone), saturation, or brightness (gradation). As described above, the reference screen G2 displays the score 711 of the assigned piece Ma along with the results of the analysis by the performance analysis unit 52. Therefore, the instructor Ub and guardian Uc can visually and intuitively grasp the points that the performer Ua should pay attention to when performing the assigned piece Ma.

[0095] 14 on the display device 34 of the terminal device 30a used by the performer Ua. The reference screen G3 is an image corresponding to the results of the analysis by the performance analysis unit 52. Specifically, the reference screen G3 includes not only the musical score 711 and performance comments 712 similar to those on the reference screen G2, but also instruction comments 713 corresponding to the results of the analysis by the performance analysis unit 52.

[0096] The instruction comment 713 is a character string according to the result of the analysis by the performance analysis unit 52. Specifically, the instruction comment 713 according to the length of time that the performer Ua played the set piece Ma and the length of time that the performer Ua played the set item Na is displayed on the display device 34. For example, if the ratio of time that the performer Ua played the set piece Ma or the set item Na is below a predetermined value, the instruction comment 713 encouraging the performer Ua to practice the set piece Ma and the set item Na instructed by the instructor Ub is displayed on the display device 34, as shown in the example of FIG.

[0097] As explained above, the reference screens G1 and G2 are displayed on the terminal device 30b of the instructor Ub and the terminal device 30c of the guardian Uc, and the reference screen G3 is displayed on the terminal device 30a of the performer Ua. That is, the reference screens G1 and G2 viewed by the instructor Ub or guardian Uc are different from the reference screen G3 viewed by the performer Ua. Specifically, the reference screens G1 and G2, which directly and in detail show the results of the performance analysis unit 52 analyzing the performance of the performer Ua, are provided to the instructor Ub or guardian Uc. Therefore, the instructor Ub and guardian Uc can check the performance status of the performer Ua (for example, whether the performer Ua is properly practicing the assigned piece Ma and the assigned item Na).

[0098] In contrast to the reference screens G1 and G2, the reference screen G3 visible to the performer Ua is an image that indirectly and ambiguously displays the results of the analysis by the performance analysis unit 52. In other words, the performer Ua is not directly or explicitly provided with all of the analysis results related to the assignment practice, but is indirectly provided with only information (instructional comments 713) that serves as a hint for the performer's own practice. Therefore, the performer Ua can be effectively provided with insights that allow the performer Ua to consider, for example, the appropriateness of their performance practice or ways to improve their practice. As described above, by presenting only the assignment in a music class and having the performer Ua independently consider the assignment during practice at home, efficient improvement in the performance of the keyboard instrument 10 can be expected.

[0099] 15 is a flowchart of a process (hereinafter referred to as a "performance management process") executed by the control device 31 of the terminal device 30a to manage the practice of the performer Ua. For example, the performance management process is executed in response to an operation by the performer Ua on the operation device 35.

[0100] When the performance management process is started, the control device 31 (information acquisition unit 51) acquires the score data Dm of the assignment piece Ma and the assignment items Na from the terminal device 30b of the instructor Ub (Sa1). In addition, the control device 31 (display control unit 53) displays the reference screen G4 on the display device 34 (Sa2).

[0101] 16 is a schematic diagram of the reference screen G4. The reference screen G4 is an image for the performer Ua to memorize the set piece Ma, and like the reference screens G2 and G3, it includes a musical score 711 and performance comments 712. The performer Ua memorizes the set piece Ma by visually checking the reference screen G4. When a predetermined time has elapsed since the start of display of the reference screen G4, the display control unit 53 erases the reference screen G4 (Sa3). Note that the reference screen G4 may also be erased by the performer Ua operating the operation device 35.

[0102] When the reference screen G4 is cleared, the performer Ua begins playing using the keyboard instrument 10. The performer Ua has been instructed by the instructor Ub to practice the assignment piece Ma. Therefore, the performer Ua will basically play the assignment piece Ma. However, the performer Ua may also play a piece other than the assignment piece Ma (etudes Mb).

[0103] When the performer Ua finishes playing, he operates the operating device 35. The control device 31 (information acquisition unit 51) waits until the performer Ua instructs the performance to end (Sa4: NO). If the performance is instructed to end (Sa4: YES), the control device 31 (information acquisition unit 51) acquires the performance data D from the keyboard instrument 10 (Sa5).

[0104] The control device 31 (performance analysis unit 52) analyzes the performance by the performer Ua by analyzing the performance data D (Sa6). Specifically, the control device 31 determines whether the performer Ua has performed the set piece Ma and whether the performer Ua has performed the set item Na. The performance analysis unit 52 also calculates the time ratio (r1) during which the performer Ua performed the set item Na and the time ratio (r2) during which the performer Ua did not perform the set item Na, among the time during which the performer Ua performed the set piece Ma, and the time ratio (r3) during which the performer Ua performed the set item Na and the time ratio (r4) during which the performer Ua did not perform the set item Na, among the time during which the performer Ua performed an etude Mb other than the set piece Ma.

[0105] The control device 31 (display control unit 53) displays the reference screen G3 of Fig. 14, which shows the results of the analysis by the performance analysis unit 52, on the display device 34 (Sa7). The control device 31 (display control unit 53) also generates image data for the reference screen G1 and the reference screen G2, and transmits the image data to the terminal device 30b or the terminal device 30c via the communication device 33 (Sa8). Therefore, the reference screen G1 is displayed on the display device 34 of the terminal device 30b and the terminal device 30c, and the reference screen G2 is displayed in response to an operation on the reference screen G2 (for example, a tap on the area r1 or the area r2).

[0106] B: Second embodiment A second embodiment will be described. Note that, for elements in the following exemplary aspects that have the same functions as those in the first embodiment, the same reference numerals as those in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.

[0107] In the second embodiment, the reference screen G displayed on the display device 34 of the terminal device 30b or 30c includes the reference screen G5 shown in FIG. 17 in addition to the reference screen G1 and reference screen G2 similar to those in the first embodiment. The reference screen G5 is displayed, for example, in parallel with the reference screen G1 or the reference screen G2. Note that the reference screen G1 or the reference screen G2 and the reference screen G5 may be selectively displayed in response to operations by the instructor Ub and the guardian Uc. The reference screen G5 may also be displayed on the terminal device 30a of the performer Ua.

[0108] The reference screen G5 includes a keyboard area 731 and a performance area 732. The keyboard area 731 displays an image of a keyboard 181 made up of a plurality of keys 182. The performance area 732 displays the contents of the assignment piece Ma indicated by the musical score data Dm and the performance data D, respectively.

[0109] The performance area 732 is divided into a plurality of areas (hereinafter referred to as "unit areas 733") corresponding to different pitches. The unit areas 733 are arranged in the horizontal direction along the arrangement of the keys 182 in the keyboard area 731. Each unit area 733 is a vertically elongated area.

[0110] In the performance area 732, a score note image 734 is displayed for each note specified by the score data Dm. That is, each score note image 734 represents a note that makes up the score of the assigned piece Ma. The score note image 734 for each note is arranged in the unit area 733 among the multiple unit areas 733 that corresponds to the pitch of that note. The vertical display length of each score note image 734 is set according to the duration (note value) of that note specified by the score data Dm. Furthermore, each score note image 734 moves from the top to the bottom of the unit area 733. At the time that each note specified by the score data Dm is sounded, the score note image 734 for that note reaches the bottom of the unit area 733.

[0111] Additionally, in the performance area 732, a performed note image 735 (735a, 735b) is displayed for each note specified by the performance data D. That is, each performed note image 735 represents a note played by the performer Ua. The musical score note image 734 and the performed note image 735 are displayed in different ways. For example, the width of the performed note image 735 is smaller than the width of the musical score note image 734, and the musical score note image 734 and the performed note image 735 are displayed in different colors. Each musical score note image 734 and each performed note image 735 are displayed overlapping each other.

[0112] The played note image 735 of each note is arranged in the unit area 733 corresponding to the pitch of the note among the multiple unit areas 733. The vertical display length of each played note image 735 is set according to the duration of the note specified by the performance data D. In other words, the vertical display length of each played note image 735 represents the length of time the performer Ua continues to play the note. Furthermore, each played note image 735 moves from the top to the bottom of the unit area 733. At the time when each note specified by the performance data D is sounded, the played note image 735 of that note reaches the bottom of the unit area 733. As explained above, the reference screen G5 is an image that displays the performance by the performer Ua (played note image 735) and the musical score represented by the musical score data Dm (musical score note image 734) in a contrastive manner.

[0113] The multiple performed note images 735 are divided into performed note images 735a and performed note images 735b. The performed note images 735a correspond to notes that the performer Ua was able to play appropriately according to the musical score. In other words, notes among the multiple notes played by the performer Ua that are similar to or match notes in the set piece Ma are represented by the performed note images 735a. For example, among the multiple notes in the set piece Ma, notes whose durations played by the performer Ua are similar to or match the durations specified by the musical score data Dm are represented by the performed note images 735a.

[0114] On the other hand, the performed note image 735b corresponds to a note whose performance by the performer Ua differs from that of the musical score. That is, among the multiple notes performed by the performer Ua, a note that differs from the notes in the target piece Ma is represented by the performed note image 735b. For example, as shown in the note in the upper left of FIG. 17, among the multiple notes in the target piece Ma, a note whose duration performed by the performer Ua differs from the duration specified by the musical score data Dm is represented by the performed note image 735b. Furthermore, as shown in the note in the upper right of FIG. 17, among the multiple notes in the target piece Ma, a note whose pitch performed by the performer Ua differs from the pitch specified by the musical score data Dm is represented by the performed note image 735b.

[0115] The played note image 735a and the played note image 735b are displayed in different modes. For example, the played note image 735a is displayed in a first display color, and the played note image 735b is displayed in a second display color that is different from the first display color. In other words, the played note image 735a corresponding to notes that the performer Ua was able to perform appropriately and the played note image 735b corresponding to notes that the performer Ua was unable to perform appropriately are displayed in different modes.

[0116] The second embodiment also achieves the same effects as the first embodiment. Furthermore, in the second embodiment, each note (musical score note image 734) constituting the score of the set piece Ma and each note (performed note image 735) played by the performer Ua are displayed in contrast to each other. Therefore, the instructor Ub and guardian Uc can visually and intuitively grasp the differences and similarities between the score of the set piece Ma and the performance by the performer Ua. Furthermore, performed note images 735a of notes that the performer Ua was able to perform appropriately and performed note images 735b of notes that the performer Ua was unable to perform appropriately are displayed in different ways. Therefore, the instructor Ub and guardian Uc can visually and intuitively grasp the notes that the performer Ua was unable to perform appropriately.

[0117] C: Third embodiment In the third embodiment, the reference screen G displayed on the display device 34 of the terminal device 30b or 30c includes the reference screen G6 shown in FIG. 18 in addition to the reference screen G1 and reference screen G2 similar to those in the first embodiment. The reference screen G6 is displayed, for example, in parallel with the reference screen G1 or the reference screen G2. Note that the reference screen G1 or the reference screen G2 and the reference screen G6 may be selectively displayed in response to operations by the instructor Ub and the guardian Uc. The reference screen G6 may also be displayed on the terminal device 30a of the performer Ua.

[0118] The reference screen G6 is a graph showing the evaluation value of the performance for each of a plurality of items. Specifically, the reference screen G6 is a polygonal radar chart in which the evaluation value of each item is expressed by a plurality of number lines extending radially. The plurality of items shown on the reference screen G6 include, for example, pitch accuracy, dynamic accuracy, tempo accuracy, expressiveness, continuity of notes, and concentration.

[0119] Pitch accuracy is an index of the accuracy of pitch in the performance of the performer Ua. Specifically, the smaller the difference (i.e., performance error) between the pitch of each note represented by the performance data D and the pitch of each note represented by the score data Dm, the greater the numerical value of pitch accuracy. Dynamic accuracy is an index of the accuracy of the dynamics of the performance by the performer Ua. Specifically, the smaller the difference between the intensity of each note represented by the performance data D and the intensity of each note represented by the score data Dm, the greater the numerical value of dynamic accuracy. Tempo accuracy is an index of the accuracy of the tempo of the performance by the performer Ua. Specifically, the smaller the difference between the tempo of the performance represented by the performance data D and the tempo specified by the score data Dm, the greater the numerical value of tempo accuracy.

[0120] Expressiveness is an index of the appropriateness of musical expression. Specifically, the smaller the difference between the musical expression of the performance represented by the performance data D and the musical expression represented by the score data Dm, the greater the expressiveness. The continuity of the notes is an index of the continuity of each note in the performance represented by the performance data D.

[0121] Concentration is an index of the degree to which the performer Ua is concentrating on playing. Specifically, the greater the number of keystrokes within a predetermined time, the greater the concentration is set to a numerical value. Note that in a configuration in which a center-of-gravity sensor 42 is installed on the chair on which the performer Ua sits, the concentration may be calculated based on the output result of the center-of-gravity sensor 42. For example, the smaller the movement of the center of gravity detected by the center-of-gravity sensor 42, the greater the concentration is set to a numerical value.

[0122] The performance analysis unit 52 calculates an evaluation value for each of the above-mentioned items by analyzing the performance data D. The display control unit 53 uses the results of the analysis by the performance analysis unit 52 to generate image data representing the reference screen G6 of FIG.

[0123] The third embodiment also achieves the same effects as the first embodiment. Furthermore, in the third embodiment, the evaluation values of multiple items for the performance by the performer Ua are displayed, so that the instructor Ub and the guardian Uc can grasp the degree of proficiency of the performance by the performer Ua from multiple angles.

[0124] The display items in Fig. 18 can be specified by the user U. The item specification screen allows selection and setting for each item, such as performance characteristics, performance expression, and biometric information. In addition, by specifying a piece of music to be played, data can be collected during the period in which a specific piece of music is being played and the evaluation results can be displayed. Information collected by the terminal device 30, such as location and equipment selection, can be analyzed, sorted, and displayed to convey the items that the user U wants to know.

[0125] D: Fourth embodiment The terminal device 30 (30a, 30b, 30c) of the fourth embodiment generates a moving image (hereinafter referred to as an "edited moving image V") representing a performance by a performer Ua, using performance data D stored in a storage device 32. Specifically, the edited moving image V is generated using a plurality of recorded moving images Ya and recorded sounds Yb included in the recording data Y.

[0126] FIG. 19 is an explanatory diagram of edited video V. Edited video V is composed of video Va and audio Vb. Video Va and audio Vb are played back in parallel with each other. Audio Vb is recorded sound Yb (i.e., the performance sound of the keyboard instrument 10) recorded by the sound collection device 22. Audio Vb may be generated by performing various types of acoustic processing on recorded sound Yb (for example, adding effects or removing noise).

[0127] The video Va is a video in which multiple partial videos Vq (Vq1, Vq2, ...) corresponding to different phrase periods Q (Q1, Q2, ...) on the time axis are arranged in chronological order. Each phrase period Q is a section (musical phrase) that makes up a musical unit in a song. The partial video Vq corresponding to each phrase period Q is part or all of any of the multiple recorded videos Ya (Ya1 to Ya4) recorded by the recording system 20 in that phrase period Q. As can be understood from the above explanation, the terminal device 30 generates an edited video V in which the recorded videos Ya are switched sequentially for each phrase period Q of the song. That is, in the edited video V, the capturing position and direction change at the boundaries of the phrase periods Q of the song.

[0128] The display device 34 displays a moving image Va of the edited moving image V. The sound emitting device 36 reproduces an audio Vb of the edited moving image V. That is, the display device 34 and the sound emitting device 36 function as a reproduction device that reproduces the edited moving image V.

[0129] 20 is a block diagram illustrating an example of the functional configuration of each terminal device 30. The control device 31 executes a program stored in the storage device 32 to realize multiple functions (music analysis unit 55, video editing unit 56, and playback control unit 57) for generating an edited video V.

[0130] The music analysis unit 55 identifies the phrase period Q of the music piece by analyzing the performance data D. Specifically, the music analysis unit 55 identifies the phrase period Q by using the performance data D and reference data. The reference data is data representing the musical score of the music piece and is stored in the storage device 32. The reference data is, for example, a time series of event data conforming to the MIDI standard. Specifically, the reference data specifies the pitch (note number) and intensity (velocity) for each of the multiple notes that make up the music piece. Note that any known technology may be used to identify the phrase period Q by the music analysis unit 55.

[0131] The video editing unit 56 generates an edited video V by selecting a plurality of recorded videos Ya according to a phrase period Q. Specifically, as described above with reference to FIG. 19 , the video editing unit 56 generates a video Va in which a partial video Vq is sequentially switched to each of a plurality of recorded videos Ya for each phrase period Q of a song. That is, in the video Va, the partial videos Vq are switched at the times specified by each period information T (i.e., the start and end points of each phrase period Q). That is, editing by the video editing unit 56 includes a process of selecting different recorded videos Ya from the plurality of recorded videos Ya as the partial videos Vq in one phrase period Q and the immediately following phrase period Q. For example, as illustrated in FIG. 19 , different recorded videos Ya are selected as the partial videos Vq in phrase period Q1 and phrase period Q2. The video editing unit 56 generates an edited video V by adding recorded sound Yb as audio Vb to the video Va.

[0132] 20 plays back the edited video V generated by the video editing unit 56. Specifically, the playback control unit 57 displays a video Va of the edited video V on the display device 34, and emits audio Vb of the edited video V from the sound emitting device 36.

[0133] As explained above, in the fourth embodiment, the edited video V is generated in accordance with the phrase period Q identified by analyzing the performance data D, so that an edited video V suitable for a piece of music can be generated without placing an excessive burden on the user U. In particular, in the fourth embodiment, different recorded videos Ya are selected as partial videos Vq in two consecutive phrase periods Q, so that an appropriate edited video V can be generated in which the partial videos Vq switch in synchronization with musical divisions in the piece of music (i.e., the start or end point of the phrase period Q).

[0134] E: Fifth embodiment As described above, performance data D is stored in the storage device 32 for each performance by the performer Ua. That is, as the performer Ua practices playing the keyboard instrument 10, multiple pieces of performance data D are accumulated in chronological order in the storage device 32. For example, multiple pieces of performance data D representing past performances by the performer Ua, such as practice at a music studio, practice at the performer Ua's home, and performances at recitals, are stored in the storage device 32. The video editing unit 56 may generate an edited video V using the multiple pieces of performance data D representing past performances by the performer Ua.

[0135] The video editing unit 56 of the fifth embodiment selects from the storage device 32 a plurality of pieces of performance data D corresponding to a performance of a specific piece of music (hereinafter referred to as a "target piece of music") from the plurality of pieces of performance data D stored in the storage device 32. The target piece of music is, for example, a piece of music that the performer Ua will play at a recital.

[0136] The video editing unit 56 generates an edited video V by editing the recorded videos Ya of the multiple performance data D corresponding to the target song. Specifically, the video editing unit 56 generates the edited video V by selecting one of the multiple recorded videos Ya for each phrase period Q of the target song. For example, the multiple recorded videos Ya are selected sequentially in chronological order. Therefore, an edited video V is generated in which scenes of the performer Ua performing the target song are switched sequentially. Note that the conditions for switching between the recorded videos Ya are the same as those in the above-mentioned embodiments.

[0137] The fifth embodiment also achieves the same effects as the fourth embodiment. In the fifth embodiment, an edited video V is generated from a plurality of recorded videos Ya related to a target piece of music. Therefore, an edited video V suitable for recording, for example, a child's growth, can be automatically generated, showing the progress of the performer Ua's improvement in playing the target piece of music, or scenes of the performer Ua's past performances.

[0138] The video editing unit 56 recognizes data entered by the performer Ua, such as the target song, performance date, and piano at the venue of the recital, and can determine the importance of a particular performance on a particular day to generate an edited video V. Once the edited video V is generated, the video editing unit 56 sends a notification to the terminal device 30 of the performer Ua and the guardian Uc that a video has been created. The performer Ua and the guardian Uc can download the edited video V to their own terminal device 30 and enjoy watching the video of their own performance without requiring any special training in video editing or the like.

[0139] As illustrated in FIG. 21 , the video editing unit 56 works in conjunction with the performance analysis unit 52 to upload and distribute videos of the performer Ua's practice to an SNS (social networking service) server 300 or the like. Distribution to SNS can be either live or edited. The performance analysis unit 52 detects that the performer Ua is performing the target song and prompts the terminal device 30 on whether or not to distribute the video to the SNS. If the performer Ua requests distribution of an edited video, the video editing unit 56 edits the performance that has received high evaluations and videos accompanying the performance and uploads them to the SNS. Even in the case of live distribution, distribution begins simultaneously with the start of the performance of the target song, and prompts the performer Ua to end distribution upon detecting the completion of the performance of the target song or a certain period of pause in the performance. If the performer Ua does not want their face to be exposed, the performance analysis unit 52 can compare the performer's facial data with the recorded video Ya and edit and upload a video that does not show their face.

[0140] Manufacturers of keyboard devices 10 and music schools that install them can provide the above services via the terminal device 30 and associated applications, in addition to providing peripheral devices and analysis devices associated with the keyboard device 10. Such services can be managed so that they can be received by using the keyboard device 10 manufactured by the manufacturer, by using a music school that installs the keyboard device 10, or by setting a fee for the service. Furthermore, the manufacturer can manage the display and copyright of the practice history information and edited videos. They can also be deleted after a certain period of time has passed. Management information and videos can be provided on the administrator's cloud, and can be viewed and watched via an application.

[0141] The performance data stored in the terminal device 30 and recording system 90 can be deleted automatically or manually when the performance analysis results are displayed or the video editing results are played back. The terminal device 30 or recording system 90 will inquire with the user about deleting the performance data after the performance analysis results are displayed or the video editing results are played back. In addition, the application settings can also be configured to delete the performance data when the performance analysis results are displayed or the video editing results are played back.

[0142] F: Variation Specific modified embodiments that can be added to each of the embodiments exemplified above are exemplified below. Two or more embodiments arbitrarily selected from the following examples may be combined as appropriate within the scope of not being mutually contradictory.

[0143] (1) In each of the above-described embodiments, it is assumed that the performer Ua uses one keyboard instrument 10. However, it is also assumed that the performer Ua uses multiple keyboard instruments 10 installed in different locations. For example, the performer Ua may play a keyboard instrument 10 installed in a music studio or a recital venue in addition to a keyboard instrument 10 installed at home.

[0144] Assuming that a performer Ua plays multiple keyboard instruments 10, it is preferable that the performance data D of the performer Ua be stored in the storage device 32 in association with identification information α (α1, α2, ...) of each keyboard instrument 10, as illustrated in Fig. 22. The identification information α is a code string for identifying the keyboard instrument 10.

[0145] The terminal device 30 can acquire the identification information α of the keyboard instrument 10 by, for example, reading an image code attached to the keyboard instrument 10. The image code is a code expressed by an optically readable image (e.g., QR Code (registered trademark)). The identification information α of the keyboard instrument 10 may be input by operating the operating device 35, for example.

[0146] With the above configuration, the control device 31 can selectively extract performance data D from any of a plurality of keyboard instruments 10 installed in different locations. For example, the performance analysis unit 52 can analyze a plurality of performance data D representing the performance of a keyboard instrument 10 installed in a specific location from among the plurality of performance data D stored in the storage device 32.

[0147] (2) The instructor Ub and guardian Uc may select a challenge piece Ma by any method. For example, the song selection screen 74 of FIG. 23 may be displayed on the display device 34 of the terminal device 30b or 30c. The song selection screen 74 includes a plurality of challenge pieces Ma arranged in a matrix along mutually orthogonal horizontal and vertical axes. Each challenge piece Ma is represented by a polygonal image. The instructor Ub or guardian Uc may manually select a challenge piece Ma that corresponds to the level of the performer Ua. Alternatively, a challenge piece Ma that corresponds to the level of the performer Ua may be selected automatically.

[0148] The horizontal axis of the song selection screen 74 represents the proficiency level (hereinafter referred to as "grade") of the performance by the performer Ua, and the vertical axis represents the difficulty level of the challenge song Ma corresponding to one grade. In other words, one challenge song Ma is specified by a combination of grade and difficulty level. A challenge item Na is set for each of the multiple challenge songs Ma.

[0149] The instructor Ub and guardian Uc select multiple challenge pieces Ma corresponding to the grade to which the performer Ua belongs in ascending order of difficulty (i.e., in the positive direction of the vertical axis). For example, in FIG. 23, the assignment pieces Ma(1-1) → assignment piece Ma(1-2) → assignment piece Ma(1-3) are selected in this order. Furthermore, when the performer Ua becomes proficient with the assignment piece Ma of the highest difficulty level in one grade, the instructor Ub and guardian Uc select the assignment piece Ma of the lowest difficulty level in the grade immediately following the current grade. For example, in FIG. 23, the assignment pieces Ma(1-2) → assignment piece Ma(1-3) → assignment piece Ma(2-1) are selected in this order. The terminal device 30b and the terminal device 30c transmit the score data Dm of the assignment piece Ma selected by the instructor Ub and guardian Uc and the assignment items Na corresponding to the assignment piece Ma to the terminal device 30a.

[0150] Furthermore, if the performer Ua has sufficient practice or skill, a set piece Ma of a higher grade may be selected, skipping the grade immediately following the performer Ua's current grade, as shown by the dashed arrow in Figure 23.

[0151] For example, assume that the assignment piece Ma(1-2), which corresponds to the combination of Grade 1 and Difficulty Level 2, corresponds to the current level of the performer Ua. The assignment piece Ma, whose assignment item Na is a dispersed octave, is assigned to the assignment piece Ma(1-2). If the instructor Ub and guardian Uc determine that the performer Ua has sufficient proficiency in the dispersed octave, they select the assignment piece Ma(3-3) from among the multiple assignment pieces Ma of Grade 3, which share the same assignment item Na (dispersed octave). As described above, the song selection screen 74 in FIG. 23 allows the instructor Ub and guardian Uc to visually and intuitively select an assignment piece Ma suitable for the performer Ua.

[0152] (3) In the above-described embodiments, the terminal devices 30b and 30c of the instructor Ub and guardian Uc transmit the score data Dm of the set piece Ma to the terminal device 30a, respectively. However, the configuration for providing the score data Dm of the set piece Ma to the terminal device 30a is not limited to the above examples. For example, the score data Dm of the set piece Ma may be transmitted to the terminal device 30a from a music distribution system.

[0153] The terminal device 30a transmits performance data D representing a performance by the performer Ua to the music distribution system. The music distribution system selects an appropriate challenge piece Ma from a plurality of candidate songs for distribution based on the performance history of the performer Ua, and transmits sheet music data Dm for the challenge piece Ma to the terminal device 30a. For example, the music distribution system estimates the performance level of the performer Ua from the performance data D, and selects a challenge piece Ma from a plurality of songs that is appropriate for the level of the performer Ua.

[0154] The challenge piece Ma may be selected based on multiple factors related to the performer Ua's performance history, such as the genres of pieces the performer Ua has played in the past, the instrument used for the performance, the musical genre and difficulty level, etc. Alternatively, the music distribution system may transmit a list of multiple candidate pieces to the terminal device 30a. The music distribution system determines the candidate piece selected by the performer Ua from the list as the challenge piece Ma and transmits the sheet music data Dm of the challenge piece Ma to the terminal device 30a.

[0155] (4) An edited video V recording a performance by performer Ua may be distributed from a video distribution system to an information device. The information device may be, for example, a smartphone, a tablet terminal, or a personal computer. The video distribution system distributes videos created by many users to the information device. For example, a server system that provides a social networking service (SNS) is an example of a video distribution system.

[0156] The performance analysis unit 52 calculates an evaluation value of the performance by the performer Ua by analyzing the performance data D. If the evaluation value exceeds a predetermined threshold, the display control unit 53 transmits the edited video V recorded by the recording system 20 to the video distribution system. If the evaluation value is below the threshold, the edited video V is not transmitted to the video distribution system. The video distribution system distributes the edited video V to multiple information devices. Note that distribution of the edited video V may begin after the performance by the performer Ua has finished, or distribution of the edited video V may be performed in parallel with the performance by the performer Ua (i.e., live distribution).

[0157] (5) In the above-described embodiments, the performance data D generated by the keyboard instrument 10 was analyzed, but the method by which the performance analysis unit 52 analyzes the performance by the performer Ua is not limited to the above examples. For example, consider a configuration in which performance sounds produced by the sound generation mechanism 183 of the keyboard instrument 10 are collected by the sound collection device 37. The sound collection device 37 is composed of, for example, one or more microphones. The performance analysis unit 52 analyzes the performance by the performer Ua by analyzing the performance sounds collected by the sound collection device 37. Furthermore, in a configuration in which an edited video V is generated by recording the performance by the performer Ua, the performance analysis unit 52 analyzes the performance by the performer Ua by analyzing the edited video V. As can be understood from the above explanation, the performance analysis unit 52 is comprehensively expressed as an element that analyzes the performance by the performer Ua, and the information and analysis method used for the analysis are arbitrary.

[0158] (6) In the above-described embodiments, the time (specifically, the time ratio relative to the counting period τ) that the performer Ua has performed the task Na is displayed on the reference screen G1, but the display manner of the task Na is not limited to the above examples. For example, the number of times that the performer Ua has performed the task Na may be displayed on the reference screen G1.

[0159] Furthermore, in the above-described embodiments, the time that the performer Ua has played the challenge piece Ma (specifically, the time ratio relative to the counting period τ) is displayed on the reference screen G1, but the display mode for the challenge piece Ma is not limited to the above examples. For example, the number of times that the performer Ua has played the challenge piece Ma may be displayed on the reference screen G1.

[0160] (7) Comments corresponding to the results of the analysis of the performance of the performer Ua by the performance analysis unit 52 may be displayed on the display device 34 of the terminal device 30a. For example, multiple comments corresponding to different analysis results are stored in the storage device 32. The display control unit 53 displays, on the display device 34, the comment corresponding to the analysis result by the performance analysis unit 52 from the multiple comments stored in the storage device 32. For example, if the length of time that the performer Ua has played the assigned piece Ma or the assigned item Na during the calculation period τ falls below a predetermined value, a comment such as "Is that practice okay? Let's check your lesson notes" is displayed on the display device 34.

[0161] (8) In the above embodiments, the performer Ua plays the keyboard instrument 10, but the instrument played by the performer Ua is not limited to the keyboard instrument 10. In other words, the present disclosure can be applied to managing the performance of any type of instrument.

[0162] (9) In each of the above-described embodiments, the information processing system 100 includes a keyboard instrument 10. However, the keyboard instrument 10 may be omitted from the information processing system 100. For example, the keyboard instrument 10 may be installed in an acoustic space located remotely from the information processing system 100 (terminal device 30a). The performance data D generated by the keyboard instrument 10 is transmitted to the information processing system 100 via a communication network such as the Internet. The operation of the terminal device 30a is the same as in each of the above-described embodiments. As can be understood from the above explanation, the "information processing system" in this disclosure may or may not include a musical instrument. Therefore, the terminal device 30a alone in each of the above-described embodiments is also included in the "information processing system."

[0163] (10) The functions of the terminal device 30 exemplified in each of the above embodiments may be realized by a server device (for example, a cloud server) that communicates with an information device such as a smartphone, a tablet terminal, or a personal computer.

[0164] (11) As mentioned above, the functions of the terminal device 30 exemplified above are realized by cooperation between one or more processors constituting the control device 31 and a program stored in the storage device 32. The program according to the present disclosure may be provided in a form stored on a computer-readable recording medium and installed on a computer. The recording medium may be, for example, a non-transitory recording medium, such as an optical recording medium (optical disk) such as a CD-ROM, but may also include any known type of recording medium, such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium other than a transitory, propagating signal, and does not exclude volatile recording media. Furthermore, in a configuration in which a distribution device distributes a program via a communication network, the storage medium that stores the program in the distribution device corresponds to the non-transitory recording medium described above.

[0165] G: Notes From the above-described exemplary embodiments, the following configurations can be understood, for example.

[0166] An information processing system according to one aspect (aspect 1) of the present disclosure includes a performance analysis unit that determines whether a performer's performance on a musical instrument corresponds to a task, and a display control unit that displays the result of the determination by the performance analysis unit on a display device. According to the above aspect, it is determined whether a performer's performance corresponds to a task. Therefore, for example, a manager of the performer (e.g., an instructor or parent) can confirm whether the performer is practicing in accordance with the task. In other words, the manager can confirm whether the performer is practicing effectively to solve the task.

[0167] In a specific example (Aspect 2) of Aspect 1, the performance analysis unit determines whether the performer has performed the assigned items. According to the above aspect, for example, a manager of the performers can confirm whether the performers are actually practicing the assigned items.

[0168] "Performance matters" are various matters related to playing an instrument. Specifically, examples of "performance matters" include performance techniques or methods that are the goal to be acquired through practicing an instrument. For example, various matters related to playing an instrument, such as staccato playing, chord playing, finger crossing on a keyboard instrument, and playing split octaves, are examples of "performance matters." As can be understood from the above examples, "performance matters" can also be referred to as performance expression.

[0169] The "task items" are performance items that are determined in advance. Specifically, performance items that are determined as tasks related to the performance of a performer are exemplified as "task items." For example, the task items may be determined by a trainer who instructs the performer, or the task items may be automatically determined depending on the performance level of the performer.

[0170] The "result of the determination by the performance analysis unit" includes not only the result of determining whether the performer performed the assigned item, but also any information generated in response to the result. That is, the display control unit may display the result of determining whether the performer performed the assigned item, or may display information in response to the result of the determination. Examples of information in response to the result of the determination include the time or number of times the performer performed the assigned item.

[0171] In a specific example of Aspect 2 (Aspect 3), the display control unit displays information indicating the time or number of times the performer performed the task. According to the above aspect, the extent to which the performer has practiced the task can be confirmed by, for example, the performer's manager.

[0172] "Information representing time" is, for example, the length of time the task item was played, the proportion of time the task item was played out of the performer's performance, or any of a plurality of levels (e.g., long / medium / short) dividing the range of values for the length of time. "Information representing the number of times" is, for example, the number of times the task item was played, the proportion of the number of times the performer played the piece of music, or any of a plurality of levels (e.g., many / medium / little) dividing the range of values for the number of times.

[0173] In a specific example (Aspect 4) of any of Aspects 1 to 3, the performance analysis unit determines whether the performer has performed the set piece. According to the above aspects, for example, the performer's manager (e.g., an instructor or guardian) can confirm whether the performer is actually practicing the set piece. In other words, the manager can confirm whether the performer is performing effective practice to master the set piece.

[0174] A "performance piece" is a piece of music being played by a performer. A "set piece" is a piece of music that has been determined in advance. Specifically, a piece of music determined according to the performer's level of proficiency in playing is an example of a "set piece." For example, a set piece may be determined by the instructor who instructs the performer, or the set piece may be determined automatically according to the performer's level of playing.

[0175] In a specific example of Aspect 4 (Aspect 5), the display control unit displays information indicating the time or number of times the performer has performed the set piece. According to the above aspect, the extent to which the performer has actually practiced the set piece can be confirmed by, for example, the performer's manager.

[0176] In a specific example of Aspect 5 (Aspect 6), the display control unit displays an image representing the proportion of time that the performer performed the assignment and the proportion of time that the performer performed the assigned piece. According to the above aspect, for example, a performer's manager can visually and intuitively grasp the extent to which the performer actually practiced the assignment and the assigned piece.

[0177] In a specific example (Aspect 7) of any of Aspects 4 to 6, the display control unit displays the score of the set piece on a display device used by the performer, and displays the results of the judgment by the performance analysis unit on a terminal device used by the performer's manager. According to the above aspects, the performer can check the score of the set piece on his / her own display device, while the manager who manages the performer can visually check the performer's practice status on his / her own terminal device. Note that it is envisioned that the results of the judgment by the performance analysis unit (i.e., the performer's practice status) will not be displayed on the performer's terminal device.

[0178] An information processing method according to one aspect (aspect 8) of the present disclosure determines whether or not a performance of a musical instrument by a performer corresponds to a task, and displays the result of the determination on a display device.

[0179] A program according to one aspect (aspect 9) of the present disclosure causes a computer system to function as a performance analysis unit that determines whether a performer's performance of an instrument corresponds to a task, and a display control unit that displays the results of the determination by the performance analysis unit on a display device. [Explanation of symbols]

[0180] 100...information processing system, 10...keyboard instrument, 11...control device, 12...storage device, 13...communication device, 14...display device, 15...operation device, 16...sound source device, 17...sound emission device, 18...performance mechanism, 181...keyboard, 182...key, 183...sound generation mechanism, 184...pedal, 185...detection device, 20...recording system, 21...imaging device, 22...sound collection device, 30 (30a, 30b, 30c)...terminal device, 31...control device, 32...storage device, 33...communication device, 3 4...display device, 35...operation device, 36...sound emission device, 37...sound collection device, 40...measurement system, 41...biometric sensor, 42...center of gravity sensor, 51...information acquisition unit, 52...performance analysis unit, 53...display control unit, 55...music analysis unit, 56...video editing unit, 57...playback control unit, 60...setting screen, 61...guidance screen, 62...device icon, 62a...imaging icon, 62b...sound collection icon, 63...scene selection window, 64...device setting window, 90...recording system.

Claims

1. a performance analysis unit that determines whether the performer's performance of the instrument corresponds to the task; a display control unit that displays the result of the determination by the performance analysis unit on a display device; An information processing system comprising:

2. The performance analysis unit determines whether the performer has performed the task. The information processing system of claim 1.

3. The display control unit displays information indicating the time or number of times the performer has performed the task.

3. The information processing system of claim 2.

4. The performance analysis unit determines whether the performer has performed the assigned piece. The information processing system of claim 1.

5. The display control unit displays information indicating the time or number of times the performer has performed the assigned piece.

5. The information processing system of claim 4.

6. The display control unit displays an image representing the time ratio during which the performer performed the assignment and the time ratio during which the performer performed the assignment piece. The information processing system according to claim 5.

7. The display control unit displaying the score of the assigned piece on a display device used by the performer; The result of the judgment by the performance analysis unit is displayed on a terminal device used by the manager of the performers.

5. The information processing system of claim 4.

8. determining whether the performer's performance of the instrument corresponds to the task; The result of the determination is displayed on a display device. An information processing method implemented by a computer system.

9. a performance analysis unit that determines whether the performer's performance of the instrument corresponds to the task; and a display control unit that displays the result of the determination made by the performance analysis unit on a display device; A program that makes a computer system function as a

Images