Information processing method, information processing system, and program

Abstract

To provide a technique capable of specifying an image of a player necessary for teaching a performance using a musical instrument.SOLUTION: An information processing method executed by a computer includes: determining a point of interest in a body of a player who plays a musical instrument based on information on a person who confirms a performance of the player; and acquiring image information representing an image of the determined point of interest.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing method and an information processing system. [Background technology]

[0002] Patent Document 1 discloses a performance evaluation device that automatically evaluates a performance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-63175 Summary of the Invention [Problem to be solved by the invention]

[0004] When lessons on playing a musical instrument are given using images, it is important to specify an image of the player required for the lessons. The present disclosure aims to provide a technique for identifying an image of a player required for training. [Means for solving the problem]

[0005] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, which determines a point of interest on the body of a player playing the instrument indicated by the instrument information, based on instrument information indicating the instrument, and obtains image information representing an image of the determined point of interest.

[0006] An information processing method according to another aspect of the present disclosure is an information processing method executed by a computer, which determines a point of interest on the body of a player playing an instrument based on sound information indicating a sound output from the instrument, and obtains image information indicating an image of the determined point of interest.

[0007] An information processing system according to yet another aspect of the present disclosure includes a determination unit that determines a point of interest from the body of a player playing an instrument indicated by instrument information indicating the instrument, based on the instrument information indicating the instrument, and an acquisition unit that acquires image information representing an image of the point of interest determined by the determination unit.

[0008] An information processing system according to yet another aspect of the present disclosure includes a determination unit that determines a point of interest from the body of a player playing an instrument based on sound information indicating a sound output from the instrument, and an acquisition unit that acquires image information representing an image of the point of interest determined by the determination unit. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of an information providing system 1. [Diagram 2] FIG. 1 illustrates an example of a student training system 100. [Diagram 3] FIG. 2 is a diagram showing an example of a correspondence table Ta. [Figure 4] 2 is a diagram for explaining the operation of the student training system 100. FIG. [Diagram 5] FIG. 13 shows a student image G3. [Figure 6] 2 is a diagram for explaining the operation of the student training system 100. FIG. [Figure 7] FIG. 13 is a diagram showing an example of a correspondence table Ta1. [Figure 8] FIG. 1 illustrates a student training system 101. [Figure 9] FIG. 13 is a diagram for explaining cutting out an image showing a part of a player's body. [Figure 10] FIG. 1 illustrates a student training system 102. [Figure 11] FIG. 1 is a diagram showing an example of a tablature. [Figure 12] FIG. 2 is a diagram showing an example of a guitar chord score. [Figure 13] FIG. 1 is a diagram showing an example of a drum score. [Figure 14] FIG. 1 is a diagram showing an example of a duet score. [Figure 15] FIG. 1 is a diagram illustrating an example of a musical note indicating simultaneous production of multiple sounds. [Figure 16] FIG. 4 is a diagram showing an example of a schedule indicated by schedule information. [Figure 17] FIG. 11 is a diagram showing another example of a schedule indicated by the schedule information. [Figure 18] FIG. 1 illustrates a student training system 103. [Figure 19] FIG. 1 illustrates a student training system 104. [Figure 20] FIG. 4 illustrates an example of a user interface. [Figure 21] FIG. 1 illustrates a student training system 105. [Figure 22] FIG. 13 illustrates an example of a learning processing unit 191. [Diagram 23] FIG. 11 illustrates an example of a learning process. [Figure 24] FIG. 13 illustrates another example of the processing device 180. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] A: First embodiment A1: Information provision system 1 FIG. 1 is a diagram showing an example of an information providing system 1 of the present disclosure. The information providing system 1 is an example of an information processing system. The information providing system 1 includes a student training system 100 and a teacher training system 200. The student training system 100 and the teacher training system 200 can communicate with each other via a network NW. The configuration of the teacher training system 200 is the same as the configuration of the student training system 100.

[0011] The student training system 100 is used by a student 100B who is learning to play a piece of music using a musical instrument 100A. The student training system 100 is placed in a student room in a music classroom. The student training system 100 may be placed in a location different from the student room in the music classroom, for example, in the home of the student 100B.

[0012] The musical instrument 100A is a piano or a flute. The piano and the flute are an example of a type of musical instrument and an example of a musical instrument, respectively. Hereinafter, the term "type of musical instrument" can be replaced with the term "musical instrument". The student 100B is an example of a performer. The location where the student 100B plays the musical instrument 100A is predetermined in the room where the student training system 100 is placed. For this reason, the student 100B while playing, the student 100B just before playing, and the student 100B just after playing can be imaged by a fixed camera.

[0013] The teacher training system 200 is used by the teacher 200B who teaches the performance of a piece of music using the musical instrument 200A. The type of the musical instrument 200A is the same as the type of the musical instrument 100A. For example, if the musical instrument 100A is a piano, the musical instrument 200A is also a piano. The teacher training system 200 is placed in a teacher's room provided in a music classroom. The teacher training system 200 may be placed in a location different from the teacher's room provided in the music classroom, for example, in the home of the teacher 200B.

[0014] The teacher 200B is an example of a performer. The place where the teacher 200B plays the musical instrument 200A is determined in advance in a room where the teacher training system 200 is placed. Therefore, the teacher 200B during the performance, the teacher 200B just before the performance, and the teacher 200B just after the performance can be imaged by a fixed camera.

[0015] The student training system 100 transmits the student performance information a to the teacher training system 200. The student performance information a indicates the situation in which the student 100B plays the musical instrument 100A. The student performance information a includes student image information a1 and student sound information a2.

[0016] The student image information a1 indicates an image (hereinafter referred to as a "student image") showing a situation in which the student 100B is playing the musical instrument 100A. The student sound information a2 indicates a sound (hereinafter referred to as a "student performance sound") output from the musical instrument 100A in the situation in which the student 100B is playing the musical instrument 100A.

[0017] The teacher training system 200 receives the student performance information a from the student training system 100. The teacher training system 200 displays a student image based on the student image information a1 included in the student performance information a. The teacher training system 200 outputs a student performance sound based on the student sound information a2 included in the student performance information a.

[0018] The teacher training system 200 transmits teacher performance information b to the student training system 100. The teacher performance information b shows the situation in which the teacher 200B plays the musical instrument 200A. The teacher performance information b includes teacher image information b1 and teacher sound information b2.

[0019] The teacher image information b1 shows an image (hereinafter referred to as "teacher image") showing a situation where the teacher 200B plays the musical instrument 200A. The teacher sound information b2 shows the sound of a piece of music (hereinafter referred to as "teacher performance sound") output from the musical instrument 200A in a situation where the teacher 200B plays the musical instrument 200A.

[0020] The student training system 100 receives teacher performance information b from the teacher training system 200. The student training system 100 displays a teacher image based on teacher image information b1 included in the teacher performance information b. The student training system 100 outputs a teacher performance sound based on teacher sound information b2 included in the teacher performance information b.

[0021] A2: Student Training System 100 2 is a diagram showing an example of the student training system 100. The student training system 100 includes cameras 111-115, a microphone 120, a display unit 130, a speaker 140, an operation unit 150, a communication unit 160, a storage device 170, and a processing device 180.

[0022] Each of the cameras 111 to 115 includes an image sensor that converts light into an electrical signal. The image sensor is, for example, a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor.

[0023] The camera 111 captures an image of each finger of the hand of the student 100B who is operating the musical instrument 100A, thereby generating student finger information a11. The student finger information a11 represents, in the form of an image, each finger of the hand of the student 100B who is operating the musical instrument 100A, and the musical instrument 100A.

[0024] The camera 112 captures an image of both feet of the student 100B who is operating the musical instrument 100A, thereby generating student foot information a12. The student foot information a12 represents, in the form of an image, both feet of the student 100B who is operating the musical instrument 100A, and the musical instrument 100A.

[0025] The camera 113 captures an image of the entire body of the student 100B operating the musical instrument 100A, thereby generating student whole-body information a13. The student whole-body information a13 represents, in the form of an image, the entire body of the student 100B operating the musical instrument 100A, and the musical instrument 100A.

[0026] The camera 114 captures an image of the mouth of the student 100B who is operating the musical instrument 100A, thereby generating student mouth information a14. The student mouth information a14 represents, in the form of an image, the mouth of the student 100B who is operating the musical instrument 100A, and the musical instrument 100A.

[0027] Camera 115 captures an image of the upper body of student 100B operating musical instrument 100A to generate student upper body information a15. Student upper body information a15 represents, in the form of an image, the upper body of student 100B operating musical instrument 100A and musical instrument 100A.

[0028] At least one of the student finger information a11, student foot information a12, student whole body information a13, student mouth information a14, and student upper body information a15 is included in the student image information a1. The direction and attitude of the cameras 111 to 115 are adjustable. Each of the cameras 111 to 115 is also referred to as an imaging unit.

[0029] The microphone 120 collects the sounds of the students' performances. The microphone 120 generates student sound information A2 based on the sounds of the students' performances. The microphone 120 is also referred to as a sound collection unit.

[0030] The display unit 130 is a liquid crystal display. The display unit 130 is not limited to a liquid crystal display, and may be, for example, an OLED (Organic Light Emitting Diode) display. The display unit 130 may be a touch panel. The display unit 130 displays various information. For example, the display unit 130 displays a teacher image based on teacher image information b1. The display unit 130 may display a student image based on student image information a1.

[0031] The speaker 140 outputs various sounds. For example, the speaker 140 outputs a teacher's performance sound based on the teacher sound information b2. The speaker 140 may output a student's performance sound based on the student sound information a2.

[0032] The operation unit 150 is a touch panel. The operation unit 150 is not limited to a touch panel, and may be, for example, various operation buttons. The operation unit 150 receives various information from a user such as the student 100B. The operation unit 150 receives, for example, student instrument information c1 from the user. The student instrument information c1 indicates the type of instrument 100A. The student instrument information c1 is an example of instrument information indicating the type of instrument.

[0033] The communication unit 160 communicates with the teacher training system 200 by wired or wireless communication via the network NW. The communication unit 160 may also communicate with the teacher training system 200 by wired or wireless communication without via the network NW. The communication unit 160 transmits student performance information a to the teacher training system 200. The communication unit 160 receives teacher performance information b from the teacher training system 200.

[0034] The storage device 170 is a computer-readable recording medium (e.g., a non-transitive recording medium readable by a computer). The storage device 170 includes one or more memories. The storage device 170 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), and an EEPROM (Electrically Erasable Programmable Read Only Memory). The volatile memory is, for example, a RAM (Random Access Memory).

[0035] The storage device 170 stores a processing program, an arithmetic program, and various data. The processing program defines the operation of the student training system 100. The arithmetic program defines an operation for determining an output Y1 from an input X1.

[0036] The storage device 170 may store a processing program and an arithmetic program read from a storage device in a server (not shown). In this case, the storage device in the server is an example of a computer-readable recording medium (e.g., a non-transitive recording medium readable by a computer). The various data includes a plurality of variables K1 described below.

[0037] The processing device 180 includes one or more central processing units (CPUs). The one or more CPUs are an example of one or more processors. Each of the processing device, processor, and CPU is an example of a computer. Some or all of the functions of the processing device 180 may be realized by a circuit such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA).

[0038] The processing device 180 reads the processing program and the arithmetic program from the storage device 170. The processing device 180 executes the processing program to function as an identifying unit 181, a deciding unit 183, an acquiring unit 184, a transmitting unit 185, and an output control unit 186. The processing device 180 executes the arithmetic program and uses a plurality of variables K1 to function as a trained model 182. The processing device 180 is an example of an information processing device.

[0039] The identification unit 181 uses the student sound information a2 to identify the student instrument information c2. The student instrument information c2 indicates the type of the instrument 100A. The student instrument information c2 is an example of instrument information indicating the type of instrument. The instrument information indicating the type of instrument (e.g., piano) is an example of instrument information indicating an instrument (e.g., piano). The student sound information a2 is an example of related information related to the type of instrument. The related information related to the type of instrument (e.g., piano) is an example of related information about the instrument (e.g., piano). When the student sound information a2 indicates the sound of a piano, the identification unit 181 identifies the student instrument information c2 indicating piano as the type of the instrument 100A. The identification unit 181 identifies the student instrument information c2 by using, for example, the trained model 182.

[0040] The trained model 182 is configured with a neural network. For example, the trained model 182 is configured with a deep neural network (DNN). The trained model 182 may be configured with a convolutional neural network (CNN). Each of the deep neural network and the convolutional neural network is an example of a neural network. The trained model 182 may be configured with a combination of multiple types of neural networks. The trained model 182 may have an additional element such as self-attention. The trained model 182 may not be configured with a neural network, but may be configured with a hidden Markov model (HMM) or a support vector machine (SVM).

[0041] The trained model 182 has learned the relationship between first information related to the type of musical instrument and second information indicating the type of musical instrument to which the first information relates. The first information is an example of relevant information for learning related to musical instruments. The second information is an example of musical instrument information for learning indicating an instrument identified from the relevant information for learning. The trained model 182 uses output sound information indicating the sound output by the musical instrument as the first information. The trained model 182 uses information indicating the type of musical instrument that outputs the sound indicated by the output sound information as the second information. The trained model 182 is an example of a first trained model.

[0042] The multiple variables K1 used to realize the trained model 182 are identified by machine learning using multiple training data T1. The training data T1 includes a combination of training input data and training output data. The training data T1 includes first information as training input data. The training data T1 includes second information as training output data. One example of the training data T1 is a combination of output sound information (first information) indicating a sound output by a musical instrument and information (second information) indicating the type of musical instrument that outputs the sound indicated by the output sound information.

[0043] The trained model 182 generates an output Y1 according to the input X1. The trained model 182 uses "related information related to the type of musical instrument (e.g., student sound information a2)" as the input X1, and uses "information indicating the type of musical instrument that outputs the sound indicated by the related information" as the output Y1.

[0044] The multiple training data T1 may have only training input data (first information) without having training output data (second information). In this case, multiple variables K1 are identified by machine learning so that the multiple training data T1 are divided into multiple clusters based on the similarity of the multiple training data T1. Then, in the trained model 182, second information suitable for each cluster is manually associated with the cluster. The trained model 182 identifies a cluster according to the input X1, and generates the second information corresponding to the identified cluster as an output Y1.

[0045] The determination unit 183 determines a focus area from the body of a player (e.g., student 100B) who uses an instrument of the type indicated by the instrument information (student instrument information c1 or c2) based on the instrument information. A player who uses an instrument of the type indicated by the instrument information is an example of a player who plays the instrument indicated by the instrument information. The focus area is a body area that is focused on by a teacher of the instrument of the type indicated by the instrument information. The determination unit 183 determines the focus area by referring to a correspondence table Ta indicating the correspondence between the type of instrument and a body area (focus area). The focus area is, for example, at least one of each finger of the student 100B's hand, both feet of the student 100B, the whole body of the student 100B, the mouth of the student 100B, and the upper body of the student 100B. The correspondence table Ta is stored in the storage device 170.

[0046] The acquisition unit 184 acquires various information. For example, the acquisition unit 184 acquires image information representing an image of the attention area determined by the determination unit 183. The acquisition unit 184 acquires information representing an image of the attention area determined by the determination unit 183 from among the student finger information a11, the student foot information a12, the student whole body information a13, the student mouth information a14, and the student upper body information a15 as target image information. The target image information is an example of image information. The acquisition unit 184 generates student image information a1 by using the target image information. For example, the acquisition unit 184 generates student image information a1 including the target image information.

[0047] The sending unit 185 sends the student image information a1 generated by the acquiring unit 184 from the communication unit 160 to the teacher training system 200. The teacher training system 200 is an example of a destination. The destination is an example of an external device.

[0048] The output control unit 186 controls the display unit 130 and the speaker 140. For example, the output control unit 186 causes the display unit 130 to display a teacher image based on teacher image information b1. In this case, first, the acquisition unit 184 acquires the teacher image information b1 from the communication unit 160. The acquisition unit 184 provides the teacher image information b1 to the output control unit 186. The output control unit 186 causes the display unit 130 to display the teacher image using the teacher image information b1.

[0049] The output control unit 186 may display the student image on the display unit 130 based on the student image information a1. In this case, the acquisition unit 184 provides the student image information a1 to the output control unit 186. The output control unit 186 displays the student image on the display unit 130 using the student image information a1. In this case, even if the teacher 200B is not present, the student 100B can learn to play the musical instrument 100A by himself while looking at the student image (image of the attention point) indicated by the student image information a1. Also, in a situation where the teacher training system 200 does not exist but at least the student training system 100 exists, the student 100B can learn to play the musical instrument 100A by himself while looking at the student image (image of the attention point) indicated by the student image information a1.

[0050] The output control unit 186 may display the teacher image and the student image side by side on the display unit 130 based on the teacher image information b1 and the student image information a1. In this case, the acquisition unit 184 acquires the teacher image information b1 and the student image information a1 as described above. The acquisition unit 184 provides the teacher image information b1 and the student image information a1 to the output control unit 186. The output control unit 186 displays the teacher image and the student image side by side on the display unit 130 based on the teacher image information b1 and the student image information a1.

[0051] The output control unit 186 causes the speaker 140 to output the teacher performance sound based on the teacher sound information b2. In this case, first, the acquisition unit 184 acquires the teacher sound information b2 from the communication unit 160. The acquisition unit 184 provides the teacher sound information b2 to the output control unit 186. The output control unit 186 causes the speaker 140 to output the teacher performance sound using the teacher sound information b2.

[0052] The output control unit 186 may output the student performance sound to the speaker 140 based on the student sound information a2. In this case, first, the acquisition unit 184 acquires the student sound information a2 from the microphone 120. The acquisition unit 184 provides the student sound information a2 to the output control unit 186. The output control unit 186 uses the student sound information a2 to output the student performance sound to the speaker 140.

[0053] The output control unit 186 may alternately output the teacher performance sound and the student performance sound to the speaker 140 based on the teacher sound information b2 and the student sound information a2. In this case, the acquisition unit 184 acquires the teacher sound information b2 and the student sound information a2 as described above. The acquisition unit 184 provides the teacher sound information b2 and the student sound information a2 to the output control unit 186. The output control unit 186 alternately outputs the teacher performance sound and the student performance sound to the speaker 140 based on the teacher sound information b2 and the student sound information a2.

[0054] A3: Teacher Guidance System 200 The teacher training system 200 differs from the student training system 100 in that it is used by a teacher 200B instead of a student 100B. The configuration of the teacher training system 200 is the same as the configuration of the student training system 100, as described above.

[0055] The main configuration of the teacher training system 200 can be explained by making the following substitutions in the above explanation of the student training system 100. "Instrument 100A" is replaced with "Instrument 200A". "Student 100B" is replaced with "Teacher 200B". "Student performance information a" is replaced with "Teacher performance information b". "Student image information a1" is replaced with "Teacher image information b1". "Student finger information a11" is replaced with "Teacher finger information b11". "Student foot information a12" is replaced with "Teacher foot information b12". "Student whole body information a13" is replaced with "Teacher whole body information b13". "Student mouth information a14" is replaced with "Teacher mouth information b14". "Student upper body information a15" is replaced with "Teacher upper body information b15". "Student sound information a2" is replaced with "Teacher sound information b2". "Student instrument information c1, c2" is read as "teacher instrument information d1, d2". "Teacher performance information b" is read as "student performance information a". "Teacher image information b1" is read as "student image information a1". "Teacher sound information b2" is read as "student sound information a2". For this reason, a detailed explanation of the configuration of teacher guidance system 200 will be omitted.

[0056] A4: Corresponding table Ta 3 is a diagram showing an example of the correspondence table Ta. The correspondence table Ta shows the correspondence relationship between types of musical instruments and parts of the body (parts of interest). The column for instrument type in the correspondence table Ta shows the type of musical instrument that is the subject of training. The correspondence table Ta shows "piano" and "flute" as types of musical instruments. The column for body part (parts of interest) in the correspondence table Ta shows parts of the player's body that are required as images for training on the instrument shown in the column for instrument type.

[0057] In piano lessons, a student faces the piano in a posture that the student prefers, presses the piano keys with each finger of the student's hand, and operates the piano's damper pedal with the student's foot. In order to instruct the student, the teacher focuses on each finger of the student's hand, both feet, and the student's whole body (e.g., posture). For example, the teacher focuses on each finger of the student's hand in order to instruct the movement of each finger of the hand in a passage of a piece of music. The teacher focuses on the student's feet in order to instruct the operation of the damper pedal. The teacher focuses on the positional relationship between each finger of the student's hand and the keyboard in order to instruct the correct keystroke. The teacher focuses on the student's whole body in order to instruct the student's posture during performance. The teacher instructs the student by showing at least one of the fingers of the teacher's hand, both feet, and the teacher's whole body (posture, etc.). For this reason, in the correspondence table Ta, the type of instrument "piano" is associated with the body part "each finger of the hand, both feet, and whole body."

[0058] In flute lessons, a student holds the flute near the student's upper body, blows into the flute through the student's mouth, and operates the keys of the flute with the student's fingers. In order to instruct the student, the teacher focuses on the student's mouth and upper body (e.g., the student's posture, the angle between the student and the flute, and the student's fingering). For example, the teacher focuses on the student's mouth in order to instruct the shape of the student's lips when playing. The teacher focuses on the student's upper body in order to instruct the student on the positional relationship between the student and the flute. The teacher instructs the student by showing at least one of the teacher's mouth and the teacher's upper body. For this reason, in the correspondence table Ta, the type of instrument "flute" is associated with the body part "mouth and upper body".

[0059] A5: Operation of student training system 100 4 is a diagram for explaining the operation of transmitting student performance information a by student training system 100. Note that storage device 170 stores imaging target information indicating imaging targets of each of cameras 111-115.

[0060] The student 100B plays the musical instrument 100A in order to allow the student training system 100 to identify the type of the musical instrument 100A. In step S101, the microphone 120 generates student sound information a2 based on the sound output from the musical instrument 100A.

[0061] Next, in step S102, the identifying unit 181 identifies the student instrument information c2 indicating the type of the instrument 100A, using the student sound information a2.

[0062] In step S102, the identification unit 181 first inputs the student sound information a2 to the trained model 182. Next, the identification unit 181 identifies information output by the trained model 182 in response to the input of the student sound information a2 as student instrument information c2.

[0063] Next, in step S103, the determination unit 183 determines a target part on the body of the student 100B who is the player, based on the student instrument information c2.

[0064] In step S103, the determination unit 183 determines, as the attention points, the body parts corresponding to the type of instrument indicated by the student instrument information c2 in the correspondence table Ta. For example, when the student instrument information c2 indicates a piano, the determination unit 183 determines, as the attention points of the student 100B, each finger of the student 100B's hand, both feet of the student 100B, and the whole body of the student 100B.

[0065] In addition, when the operation unit 150 receives student instrument information c1 indicating the type of instrument 100A from a user such as student 100B, in step S103, the determination unit 183 may determine a focus area of ​​the body of student 100B based on the student instrument information c1.

[0066] Next, in step S104, the acquisition unit 184 determines, from among the cameras 111-115, a camera to be used to capture an image of the student 100B (hereinafter, referred to as the "camera to be used") based on the location of interest.

[0067] In step S104, the acquisition unit 184 refers to the imaging target information indicating the imaging target of each of the cameras 111-115, and determines the camera that captures the target location among the cameras 111-115 as the camera to be used.

[0068] Next, in step S105, the acquisition unit 184 acquires information generated by the camera in use as target image information.

[0069] Next, in step S106, the acquisition unit 184 generates student image information a1 by using the target image information.

[0070] For example, when cameras 114 and 115 are both used cameras, acquisition unit 184 generates student image information a1 including student mouth information a14 generated by camera 114 and student upper body information a15 generated by camera 115. Fig. 5 is a diagram showing an example of a student image G3 indicated by student image information a1. Student image G3 includes image G1 indicated by student mouth information a14 and image G2 indicated by student upper body information a15.

[0071] Next, in step S107 of FIG. 4, the transmitting unit 185 transmits the student performance information a, which includes the student image information a1 and the student sound information a2, from the communication unit 160 to the teacher training system 200.

[0072] The teacher training system 200 also operates in the same manner as the student training system 100 to transmit the teacher performance information b to the student training system 100.

[0073] FIG. 6 is a diagram for explaining the operation of the student training system 100 for outputting a teacher image and a teacher performance sound based on the teacher performance information b.

[0074] In step S201, the communication unit 160 receives the teacher performance information b. The teacher performance information b includes teacher image information b1 and teacher sound information b2.

[0075] Next, in step S202, the output control unit 186 displays on the display unit 130 a teacher image based on the teacher image information b1.

[0076] Next, in step S203, the output control unit 186 outputs the teacher performance sound based on the teacher sound information b2 from the speaker 140. Note that the timing at which step S203 is executed may be before the timing at which step S202 is executed.

[0077] The teacher training system 200 also operates in the same manner as the student training system 100, thereby displaying a student image based on the student image information a1 and outputting a student performance sound based on the student sound information a2.

[0078] According to this embodiment, an image of a player (student or teacher) required for training in playing an instrument can be identified according to the type of instrument (according to the instrument). Furthermore, this embodiment can transmit an image of a player required for training to a destination. Therefore, even if the teacher 200B is in a room different from the room in which the student 100B plays the instrument 100A, the teacher 200B can observe an image of the student 100B required for guidance in playing the instrument 100A. Even if the student 100B is in a room different from the room in which the teacher 200B plays the instrument 200A, the student 100B can view an image of the performance by the teacher 200B, which is a model for playing the instrument 200A.

[0079] The determining unit 183 of the student training system 100 may determine the attention portion using the teacher instrument information d1 or d2 instead of the student instrument information c1 or c2. For example, the communication unit 160 of the teacher training system 200 transmits the teacher instrument information d1 or d2 to the student training system 100. The determining unit 183 of the student training system 100 obtains the teacher instrument information d1 or d2 via the communication unit 160 of the student training system 100. In this case, the identifying unit 181 and the trained model 182 can be omitted in the student training system 100.

[0080] The determining unit 183 of the teacher training system 200 may determine the attention portion using the student instrument information c1 or c2 instead of the teacher instrument information d1 or d2. For example, the communication unit 160 of the student training system 100 transmits the student instrument information c1 or c2 to the teacher training system 200. The determining unit 183 of the teacher training system 200 obtains the student instrument information c1 or c2 via the communication unit 160 of the teacher training system 200. In this case, the identifying unit 181 and the trained model 182 can be omitted in the teacher training system 200.

[0081] B: Variation The following are examples of modified aspects of the above-described embodiment. Two or more aspects selected from the following aspects may be combined as appropriate within the scope of not being mutually inconsistent.

[0082] B1: First modified example In the above-described embodiment, the types of musical instruments are not limited to piano and flute, but may be two or more types. For example, the types of musical instruments may be two or more types of piano, flute, Electone (registered trademark), violin, guitar, saxophone, and drums. Each of piano, flute, Electone, violin, guitar, saxophone, and drums is an example of a musical instrument.

[0083] FIG. 7 is a diagram showing an example of a correspondence table Ta1 used when the types of musical instruments are piano, flute, electone, violin, guitar, saxophone, and drums.

[0084] For example, in an Electone lesson, a student operates the Electone as follows: The student faces the Electone in a posture that the student prefers. The student operates the upper and lower keys of the Electone with each finger of the student's hand. The student operates the pedal keys of the Electone with the student's feet (toes and heels). The student operates the expression pedal of the Electone with the student's right foot.

[0085] In the lesson of the electric organ, the teacher pays attention to each finger of the student's hand, both feet of the student (particularly the right foot), and the whole body of the student (e.g., posture) in order to instruct the student. The teacher instructs the student by showing at least one of each finger of the teacher's hand, both feet of the teacher (particularly the right foot), and the whole body of the teacher (posture, etc.).

[0086] For this reason, in the correspondence table Ta1, the type of musical instrument "Electone" is associated with the body parts "each finger, both feet, right foot, and whole body."

[0087] In violin lessons, a student operates the violin as follows: the student supports the violin with the student's chin, shoulder, and left hand, and holds the bow with the student's right hand. The student presses the violin strings with the fingers of the student's left hand. The student plays the violin while changing the angle of the violin relative to the student, the angle of the bow relative to the violin, and the position of the fingers of the student's left hand relative to the violin strings.

[0088] In violin lessons, the teacher focuses on the upper body of the student (the positional relationship between the student and the violin) and the left hand of the student to instruct the student. The teacher instructs the student by showing at least one of the upper body of the student (the positional relationship between the student and the violin) and the left hand of the teacher.

[0089] Therefore, in the correspondence table Ta1, the type of instrument "violin" is associated with the body part "upper body and left hand."

[0090] In guitar lessons, a student holds down the guitar strings with his / her left hand and plucks the strings with his / her right hand. The teacher focuses on the student's right hand and the student's left hand to instruct the student. The teacher instructs the student by showing at least one of the teacher's right hand and the teacher's left hand to the student.

[0091] Therefore, in the correspondence table Ta1, the type of musical instrument "guitar" is associated with the body parts "left hand and right hand."

[0092] In saxophone lessons, a student holds the saxophone close to his or her upper body, holds the saxophone reed in the student's mouth, and operates the keys and levers with the fingers of the student's hand. To instruct the student, the teacher focuses on the student's mouth and upper body (e.g., how the saxophone reed is held in the student's mouth, how the mouth fits over the saxophone mouthpiece, the student's posture, the angle between the student and the saxophone, and the student's fingering). The teacher instructs the student by showing at least one of the teacher's mouth and the teacher's upper body.

[0093] Therefore, in the correspondence table Ta1, the type of instrument "saxophone" is associated with the body part "mouth and upper body."

[0094] In drum lessons, students use their hands and feet to play the drums. The teacher focuses on the hands and feet and the whole body of the student to instruct the student (e.g., instructing the timing of the hand and foot movements). The teacher instructs the student by showing the movements of the hands and feet and the teacher's whole body to the student.

[0095] Therefore, in the correspondence table Ta1, the type of musical instrument "drum" is associated with the body part "hands, feet and whole body."

[0096] Each of the student training system 100 and the teacher training system 200 has a camera for capturing images of the body parts shown in the correspondence table Ta1.

[0097] According to the first variant, the image of the player required for training on playing an instrument can be switched according to the type of instrument other than either the piano or the flute, and the image can be transmitted to the destination.

[0098] B2: Second variant In the above-described embodiment and the first modified example, the determination unit 183 may determine the attention part on the player's body without using either the correspondence table Ta or Ta1. For example, the determination unit 183 may determine the attention part on the player's body by using a trained model that has trained the relationship between the type of musical instrument and the part of the body.

[0099] FIG. 8 is a diagram showing a student training system 101 including a trained model 187 that has learned the relationship between types of musical instruments and parts of the body.

[0100] The trained model 187 is configured with a neural network. For example, the trained model 187 is configured with a deep neural network. The trained model 187 may be configured with, for example, a convolutional neural network. The trained model 187 may be configured with a combination of multiple types of neural networks. The trained model 187 may have additional elements such as self-attention. The trained model 187 may not be configured with a neural network, but may be configured with a hidden Markov model or a support vector machine.

[0101] The processing device 180 functions as a trained model 187 based on a combination of a calculation program that specifies a calculation for determining an output Y1 from an input X1 and a plurality of variables K2. The plurality of variables K2 are determined by machine learning using a plurality of training data T2. The training data T2 includes a combination of information indicating the type of musical instrument (training input data) and information indicating a body part (training output data). The information indicating the type of musical instrument in the training data T2 indicates, for example, the type of musical instrument shown in FIG. 7. The information indicating the body part in the training data T2 indicates, for example, the body part shown in FIG. 7. In the training data T2, the combination of the information indicating the type of musical instrument and the information indicating the body part corresponds to the combination of the type of musical instrument and the body part shown in FIG. 7. Therefore, the information indicating the body part in the training data T2 indicates a part (attention part) that is focused on by a teacher of the musical instrument on the body of a player who uses the type of musical instrument indicated by the training input data of the training data T2.

[0102] The determination unit 183 inputs the student instrument information c1 or c2 to the trained model 187. Next, the determination unit 183 determines the location indicated by the information output by the trained model 187 in response to the input of the student instrument information c1 or c2 as the attention location on the player's body.

[0103] In addition, the multiple training data T2 may have only training input data without having training output data. In this case, multiple variables K2 are identified by machine learning so that the multiple training data T2 are divided into multiple clusters based on the similarity of the multiple training data T2. Then, in the trained model 187, information indicating a body part (attention part) suitable for each cluster is manually associated with the cluster. The trained model 187 identifies a cluster according to the input X1, and generates information corresponding to the identified cluster as an output Y1.

[0104] According to the second modified example, the determining unit 183 can determine the part of the player's body without using either the correspondence table Ta or Ta1.

[0105] B3: Third variant In the above-described embodiment and the first and second modified examples, when the attention area is a part of the body (for example, both feet), the acquisition unit 184 may acquire image information indicating the attention area from whole-body image information indicating the entire body of the player.

[0106] 9 is a diagram showing an example of the relationship between an image G11 shown by the whole-body image information and an image G12 showing a part of the player's body. Image G12 shows both feet of the player as a part of the player's body. Image G12 may show a part of the player's body other than both feet.

[0107] The position of the image G12 in the image G11 is preset in pixel units for each type of musical instrument. Therefore, the position of the image G12 in the image G11 can be changed according to the type of musical instrument. The acquisition unit 184 acquires, from the whole-body image information representing the image G11, a part preset according to the type indicated by the student musical instrument information c1 or c2, as image information representing the image G12.

[0108] The position of the image G12 in the image G11 does not have to be preset for each type of musical instrument. For example, the acquisition unit 184 first identifies a portion indicating a point of interest from the image G1 by using an image recognition technique. Then, the acquisition unit 184 acquires the portion indicating the point of interest from the whole-body image information.

[0109] The acquisition unit 184 may use image recognition technology to specify the position of the image G12 in the image G11 only for instruments whose positional relationship between the performer and the instrument is likely to change, such as the flute, violin, guitar, and saxophone. In this case, it is easier to acquire image information indicating the point of interest, compared to a configuration in which the position of the image G12 in the image G11 is fixed.

[0110] For instruments such as piano, electone, and drums, in which the positional relationship between the performer and the instrument is unlikely to change, the acquisition unit 184 acquires a part of the whole-body image information that is preset according to the type indicated by the student instrument information c1 or c2 as image information representing the image G12. In this case, the acquisition unit 184 can easily identify the position of the image G12 without using image recognition technology.

[0111] According to the third modified example, the number of cameras can be reduced compared to a configuration in which a plurality of cameras are placed in one-to-one correspondence with a plurality of body parts (parts of interest).

[0112] B4: Fourth variant In the above-described embodiment and the first to third modified examples, the destination of the teacher performance information b is not limited to the student training system 100, and may be, for example, an electronic device used by the guardian of the student 100B (for example, the parent of the student 100B). The electronic device may be, for example, a smartphone, a tablet, or a notebook personal computer. The destination of the teacher performance information b may be both the student training system 100 and the electronic device used by the guardian of the student 100B.

[0113] According to the fourth modified example, the guardian of student 100B can instruct student 100B while watching the video of the teacher.

[0114] B5: Fifth variant In the above-described embodiment and the first to fourth modified examples, the related information related to the type of musical instrument (related information related to the musical instrument) is not limited to the student sound information a2. The related information may be image information showing the musical instrument 100A (image information showing an image representing the musical instrument 100A).

[0115] In a configuration in which image information representing the musical instrument 100A is used as related information, the identification unit 181 identifies the musical instrument information (student musical instrument information c2) by using a trained model that has learned the relationship between information representing an image of the musical instrument and information indicating the type of instrument that the image represents.

[0116] 10 is a diagram showing a student training system 102 including a trained model 188 that has learned the relationship between information showing an image of a musical instrument and information showing the type of the musical instrument. The trained model 188 is an example of a first trained model.

[0117] The trained model 188 is composed of a neural network. For example, the trained model 188 is composed of a deep neural network. The trained model 188 may be composed of, for example, a convolutional neural network. The trained model 188 may be composed of a combination of multiple types of neural networks. The trained model 188 may have additional elements such as self-attention. The trained model 188 may not be composed of a neural network, but may be composed of a hidden Markov model or a support vector machine.

[0118] The processing device 180 functions as a trained model 188 based on a combination of a calculation program that specifies a calculation for determining an output Y1 from an input X1 and a plurality of variables K3. The plurality of variables K3 are determined by machine learning using a plurality of training data T3. The training data T3 includes a combination of information (training input data) showing an image of a musical instrument and information (training output data) showing the type of musical instrument shown by the image in the training input data.

[0119] The identification unit 181 inputs image information representing the musical instrument 100A to the trained model 188. Next, the identification unit 181 identifies information output by the trained model 188 in response to the input of the image information representing the musical instrument 100A as student musical instrument information c2.

[0120] The multiple training data T3 may have only training input data without having training output data. In this case, multiple variables K3 are identified by machine learning so that the multiple training data T3 are divided into multiple clusters based on the similarity of the multiple training data T3. Then, in the trained model 188, "information indicating the type of musical instrument" suitable for each cluster is manually associated with the cluster. The trained model 188 identifies a cluster according to the input X1, and generates information corresponding to the identified cluster as an output Y1.

[0121] According to the fifth modification, the image information representing the musical instrument 100A can be used as related information representing the musical instrument.

[0122] B6: 6th variant In the fifth modified example, the identification section 181 may use information generated by any of the cameras 111 to 115 (hereinafter referred to as "camera image information") as image information showing the musical instrument 100A.

[0123] In addition to the musical instrument 100A and the student 100B, the camera image information may show a type of musical instrument different from the musical instrument 100A. When camera image information showing a plurality of types of musical instruments is input to the trained model 188, there is a possibility that the information output from the trained model 188 does not show the type of the musical instrument 100A. For this reason, the identification unit 181 first extracts partial image information showing only the musical instrument 100A from the camera image information. The identification unit 181 then inputs the partial image information to the trained model 188.

[0124] For example, the identification unit 181 first identifies a human (student 100B) from the image indicated by the camera image information. Humans are easier to recognize than musical instruments. Next, the identification unit 181 identifies the object that is closest to the human (student 100B) in the image indicated by the camera image information as the musical instrument 100A. Next, the identification unit 181 extracts partial image information indicating only the object identified as the musical instrument 100A from the camera image information. Next, the identification unit 181 inputs the partial image information to the trained model 188.

[0125] According to the sixth modification, the camera image information generated by any of the cameras 111 to 115 can be used as related information relating to the type of musical instrument. Therefore, any of the cameras 111 to 115 can also be used as a device for generating related information.

[0126] B7: 7th variant In the above-described embodiment and the first to sixth modified examples, the related information related to the type of instrument may be score information indicating a score according to the type of instrument. A score according to a type of instrument (e.g., guitar) is an example of a score according to an instrument (e.g., guitar). A score is also called a musical score. The score information is generated, for example, by a camera that captures an image of the score. When any of the cameras 111 to 115 generates the score information, any of the cameras 111 to 115 can also be used as a device that generates the score information.

[0127] The identifying unit 181 identifies the student instrument information c2 based on the musical score indicated by the musical score information. For example, the identifying unit 181 identifies the student instrument information c2 based on the type of musical score.

[0128] When the score indicated by the score information is a tablature, the identifying unit 181 identifies student instrument information c2 that indicates guitar as the type of instrument. As shown in Fig. 11, a tablature indicates guitar strings with six parallel lines. Therefore, when the score indicated by the score information is composed of six parallel lines, the identifying unit 181 determines that the score indicated by the score information is a tablature.

[0129] When the score indicated by the score information is a guitar chord score, the identifying unit 181 identifies the student instrument information c2 that indicates guitar as the type of instrument. The guitar chord score indicates guitar chords that follow the lyrics, as shown in Fig. 12. Therefore, when the score indicated by the score information indicates guitar chords, the identifying unit 181 determines that the score indicated by the score information is a guitar chord score.

[0130] When the score indicated by the score information is a drum score, the identification unit 181 identifies the student instrument information c2 that indicates drum as the type of instrument. As shown in Fig. 13, the drum score indicates symbols corresponding to each instrument included in the drum set. Therefore, when the score indicated by the score information indicates symbols corresponding to each instrument of the drum set, the identification unit 181 determines that the score indicated by the score information is a drum score.

[0131] When the score indicated by the score information is a duet score, the identifying unit 181 identifies the student instrument information c2 that indicates piano as the type of instrument. A duet score indicates the symbol 14a indicating a duet score, as shown in Fig. 14. Therefore, when the score indicated by the score information indicates the symbol 14a indicating a duet score, the identifying unit 181 determines that the score indicated by the score information is a duet score.

[0132] The identification unit 181 may identify the student instrument information c2 based on the arrangement of notes in the score indicated by the score information. As shown in Fig. 15, when the score indicated by the score information represents note 15a indicating simultaneous pronunciation of multiple sounds, the identification unit 181 identifies the score indicated by the score information as being for a keyboard instrument (for example, piano or electone). In this case, the identification unit 181 identifies the student instrument information c2 indicating piano or electone as the type of instrument.

[0133] When the score indicated by the score information indicates a symbol (for example, a character string representing an instrument name or a code relating to an instrument type) that identifies the type of instrument, the identification unit 181 may identify the information indicating the type of instrument identified by the symbol as the student instrument information c2. For example, when the storage device 170 stores an instrument table indicating the correspondence between information indicating the type of instrument and a code relating to the type of instrument, the identification unit 181 refers to the instrument table to identify information corresponding to the code indicated on the score (information indicating the type of instrument) as the student instrument information c2. In this case, the code relating to the type of instrument is an example of related information. The instrument table is an example of a table indicating the correspondence between information related to the type of instrument and information indicating the type of instrument. The information related to the type of instrument is an example of reference related information related to the instrument. The information indicating the type of instrument is an example of reference instrument information indicating the instrument.

[0134] The score information is not limited to information generated by a camera that captures the score, and may be a so-called electronic score. If the electronic score has type data indicating the type of instrument, the identification unit 181 may identify the type data as the student instrument information c2.

[0135] According to the seventh modification, the musical score information can be used as related information relating to the type of musical instrument.

[0136] B8: 8th variant In the above-described embodiment and the first to seventh modified examples, when the schedule information showing the schedule of the student 100B shows the type of musical instrument, the schedule information may be used as related information related to the type of musical instrument. The schedule information may show the schedule of any of the student 100B, the teacher 200B, the student's room in the music classroom, and the teacher's room in the music classroom, as long as it shows a combination of the type of musical instrument and the lesson schedule of that type of musical instrument. The combination of the type of musical instrument (e.g., piano) and the lesson schedule of that type of musical instrument (e.g., piano) is an example of a combination of a musical instrument (e.g., piano) and the lesson schedule of that musical instrument (e.g., piano).

[0137] Fig. 16 is a diagram showing an example of a schedule indicated by the schedule information. In Fig. 16, the type of musical instrument (piano, flute, or violin) of the learning target is indicated for each lesson time period. The identification unit 181 first identifies the lesson time period that includes the current time, using the schedule information. Then, the identification unit 181 identifies the type of musical instrument of the learning target that corresponds to the identified time period. Then, the identification unit 181 identifies information indicating the identified type of musical instrument of the learning target as student musical instrument information c2.

[0138] Fig. 17 is a diagram showing another example of a schedule indicated by the schedule information. In Fig. 17, the type of musical instrument to be taught is indicated for each lesson date. The identification unit 181 first identifies the type of musical instrument to be taught corresponding to the current date using the schedule information. Next, the identification unit 181 identifies information indicating the identified type of musical instrument to be taught as student musical instrument information c2.

[0139] According to the eighth modified example, the schedule information can also be used as related information relating to the type of musical instrument.

[0140] B9: 9th variant In the above-described embodiment and the first to eighth modified examples, the determination unit 183 may determine the attention portion based on the student instrument information c1 or c2 and the student sound information a2.

[0141] In piano lessons, teacher 200B often pays attention to the movement of each finger of student 100B's hand in the fast part of the music piece used for teaching. Therefore, in piano lessons, when the student's performance sound indicated by student sound information a2 indicates the part immediately before the fast part of the music piece, determination unit 183 determines only each finger of the hand as the part of interest. After that, when the student's performance sound indicated by student sound information a2 indicates the part immediately after the fast part of the music piece, determination unit 183 determines each finger of the player's hand, both feet of the player, and the whole body of the player as the part of interest.

[0142] In this case, the storage device 170 stores the musical score data indicating the part immediately before the part with a fast melody and the part immediately after the part with a fast melody. The determination unit 183 generates the musical note data indicating the student's performance sound based on the student sound information a2. When the musical note data matches the part immediately before the part with a fast melody in the musical score data, the determination unit 183 determines that the student's performance sound indicates the part immediately before the part with a fast melody. The determination unit 183 may determine that the student's performance sound indicates the part immediately before the part with a fast melody when the degree of agreement between the musical note data and the part immediately before is equal to or greater than a first threshold value (e.g., 90%). The first threshold value is not limited to 90% and can be changed as appropriate. When the musical note data matches the part immediately after the part with a fast melody in the musical score data, the determination unit 183 determines that the student's performance sound indicates the part immediately after the part with a fast melody. The determination unit 183 may determine that the student's performance sound indicates the part immediately after the part with a fast melody when the degree of agreement between the musical note data and the part immediately after is equal to or greater than a second threshold value (e.g., 90%). The second threshold is not limited to 90% and can be changed as appropriate.

[0143] Regarding the piano, the timing at which the focus point is changed is not limited to the timing when the student's playing sound indicates the part immediately before the fast part of the melody and the timing when the student's playing sound indicates the part immediately after the fast part of the melody, and can be changed as appropriate. Regarding the piano, the transition of the focus point is not limited to the transition described above and can be changed as appropriate.

[0144] For instruments of a type other than the piano, the determination unit 183 may determine the attention portion based on the student instrument information c1 or c2 and the student sound information a2.

[0145] For example, in flute lessons, teacher 200B often focuses on the shape of student 100B's mouth at the beginning of a piece of music. For this reason, in flute lessons, when the student's performance sound indicated by student sound information a2 indicates the beginning of a piece of music, determination unit 183 determines only the mouth as the part of interest. After that, when the student's performance sound indicated by student sound information a2 indicates the part immediately following the beginning of the piece of music, determination unit 183 determines the player's mouth and the player's upper body as the parts of interest.

[0146] In this case, the storage device 170 stores the musical score data indicating the beginning part of the music piece and the part immediately after the beginning part of the music piece. The determination unit 183 generates note data indicating the student's performance sound based on the student sound information a2. When the note data matches the beginning part of the music piece in the musical score data, the determination unit 183 determines that the student's performance sound indicates the beginning part of the music piece. The determination unit 183 may determine that the student's performance sound indicates the beginning part of the music piece when the degree of agreement between the note data and the beginning part is equal to or greater than a third threshold value (e.g., 90%). The third threshold value is not limited to 90% and can be changed as appropriate. When the note data matches the part immediately after the beginning part of the music piece in the musical score data, the determination unit 183 determines that the student's performance sound indicates the part immediately after the beginning part of the music piece. The determination unit 183 may determine that the student's performance sound indicates the part immediately after the beginning part of the music piece when the degree of agreement between the note data and the part immediately after the beginning part is equal to or greater than a fourth threshold value (e.g., 90%). The fourth threshold value is not limited to 90% and can be changed as appropriate.

[0147] Regarding the flute, the timing of changing the focus point is not limited to the timing when the student's playing sound indicates the beginning of the song, and the timing when the student's playing sound indicates the part immediately following the beginning of the song, and can be changed as appropriate. Regarding the flute, the transition of the focus point is not limited to the transition described above, and can be changed as appropriate.

[0148] The determination unit 183 may determine the attention location using a trained model that has learned the relationship between information including instrument type information indicating the type of instrument and instrument sound information indicating the sound output from the instrument of the type indicated by the instrument type information, and information indicating the attention location on the player's body. The instrument type information is an example of learning instrument information indicating an instrument. The instrument sound information is an example of learning sound information indicating the sound output from the instrument indicated by the learning instrument information. Information including the instrument type information and the instrument sound information is an example of learning input information. The information indicating the attention location on the player's body indicates, as the attention location, a location that is noticed by a teacher of the instrument on the body of the player who outputs the sound indicated by the instrument sound information from the instrument of the type indicated by the instrument type information. The information indicating the attention location on the player's body is an example of learning output information indicating a location that is noticed on the body of the player who plays the instrument indicated by the learning instrument information and that outputs the sound indicated by the learning sound information.

[0149] 18 is a diagram showing a student training system 103 including a trained model 189 that has learned the correspondence between a combination of musical instrument type information and musical instrument sound information, and information indicating a point of interest. The trained model 189 is an example of a second trained model.

[0150] The trained model 189 is composed of a neural network. For example, the trained model 189 is composed of a deep neural network. The trained model 189 may be composed of, for example, a convolutional neural network. The trained model 189 may be composed of a combination of multiple types of neural networks. The trained model 189 may have additional elements such as self-attention. The trained model 189 may not be composed of a neural network, but may be composed of a hidden Markov model or a support vector machine.

[0151] The processing device 180 functions as a trained model 189 based on a combination of a calculation program that specifies a calculation for determining an output Y1 from an input X1 and a plurality of variables K4. The plurality of variables K4 are determined by machine learning using a plurality of training data T4. The training data T4 includes a combination of a set of musical instrument type information and musical instrument sound information (training input data), and attention point information (training output data) indicating an attention point on the body. The attention point information indicates, as an attention point, a part of the body of a player that outputs a sound indicated by the musical instrument sound information from a type of musical instrument indicated by the musical instrument type information, which is to be paid attention to by a teacher of the musical instrument.

[0152] The instrument sound information is used for each bar of the music piece to be performed. The instrument sound information is not limited to being used for each bar, but may be used for every four bars, for example. The attention point information (training output data) indicates an attention point on the body of a player who uses an instrument indicated by the instrument type information during performance of a bar immediately following a bar indicated by the instrument sound information in the training input data.

[0153] The determination unit 183 inputs a pair of the student instrument information c1 or c2 and the student sound information a2 to the trained model 189 for each measure. The determination unit 183 generates note data indicating the student's performance sound based on the student sound information a2, and identifies one measure in the student sound information a2 based on the arrangement of the note data. Next, the determination unit 183 determines the location indicated by the information output by the trained model 189 in response to the input of the pair of the student instrument information c1 or c2 and the student sound information a2 as the attention location.

[0154] In addition, the multiple training data T4 may have only training input data without having training output data. In this case, multiple variables K4 are identified by machine learning so that the multiple training data T4 are divided into multiple clusters based on the similarity of the multiple training data T4. Then, in the trained model 189, "information indicating a body part (attention part)" suitable for each cluster is associated by a human. The trained model 189 identifies a cluster according to the input X1, and generates information corresponding to the identified cluster as an output Y1.

[0155] According to the ninth modification, an image required for teaching the type of instrument indicated by the student instrument information c1 or c2 can be specified based on the played sound.

[0156] B10: 10th variant In the ninth modified example, the student training system 100 and the teacher training system 200 may be used for training on playing one type of musical instrument (for example, piano). The one type of musical instrument is not limited to the piano and can be changed as appropriate. In this case, the determination unit 183 determines the attention location on the player's body based on the student sound information a2. For example, the determination unit 183 inputs the student sound information a2 for each measure to a model (trained model) that has learned training data that is a combination of musical instrument sound information (training input data) and attention location information (training output data) indicating the attention location on the body. In this case, the attention location information (training output data) indicating the attention location on the body indicates a location (attention location) that is noticed by a teacher of the instrument on the body of a player who uses an instrument that outputs the sound indicated by the musical instrument sound information (training input data). Next, the determination unit 183 determines the location indicated by the information output by the trained model in response to the input of the student sound information a2 as the attention location. According to the tenth modified example, an image required for musical instrument instruction can be identified based on the playing sound.

[0157] B11: 11th variant In the above-described embodiment and the first to tenth modified examples, the determination unit 183 may determine the body attention part based on the correspondence between the student sound information a2 and the musical score information indicating the musical score of the music piece. The correspondence between the student sound information a2 and the musical score information is an example of the relationship between the student sound information a2 and the musical score information.

[0158] The determination unit 183 determines the degree of agreement between the sound indicated by the student sound information a2 and the sound represented in the musical score indicated by the musical score information.

[0159] For example, in piano lessons, when the sound of the student's performance is distorted, the teacher 200B often focuses on the movement of each finger of the student 100B's hand. In piano lessons, when the degree of agreement is less than a threshold, the determination unit 183 determines only each finger of the hand as the focus area. When the degree of agreement is equal to or greater than a threshold, the determination unit 183 determines each finger of the player's hand, both feet of the player, and the whole body of the player as the focus area.

[0160] In flute lessons, when the student's playing sound is distorted, the teacher 200B often focuses on the mouth and upper body of the student 100B. In flute lessons, when the degree of agreement is less than a threshold, the determination unit 183 determines the mouth and upper body as the focus area. When the degree of agreement is equal to or greater than a threshold, the determination unit 183 determines the upper body of the performer as the focus area.

[0161] The determination unit 183 may determine the attention location using a trained model that has learned the relationship between information including output sound information indicating a sound output from an instrument and score-related information indicating a score, and information indicating a body part of a player. The output sound information is an example of learning sound information indicating a sound output from an instrument. The score-related information is an example of learning score information indicating a score. Information including the output sound information and the score-related information is an example of learning input information. The information indicating the body part of a player indicates a focused part (focused part) of the body of a player who outputs the sound indicated by the output sound information from the instrument according to the score indicated by the score-related information. The information indicating the body part of a player is an example of learning output information indicating a focused part of the body of a player who plays an instrument that outputs the sound indicated by the learning sound information according to the score indicated by the learning score information.

[0162] 19 is a diagram showing a student training system 104 including a trained model 190 that has learned the relationship between a set of output sound information and score-related information, and information indicating a body part of a performer that is of interest. The trained model 190 is an example of a third trained model.

[0163] The trained model 190 is composed of a neural network. For example, the trained model 187 is composed of a deep neural network. The trained model 190 may be composed of, for example, a convolutional neural network. The trained model 190 may be composed of a combination of multiple types of neural networks. The trained model 190 may have additional elements such as self-attention. The trained model 190 may not be composed of a neural network, but may be composed of a hidden Markov model or a support vector machine.

[0164] The processing device 180 functions as a trained model 190 based on a combination of a calculation program that specifies a calculation for determining an output Y1 from an input X1 and a plurality of variables K5. The plurality of variables K5 are determined by machine learning using a plurality of training data T5. The training data T5 is a combination of a set of output sound information and score-related information (training input data), and attention point information (training output data) indicating an attention point on the body. The attention point information (training output data) indicates a part of the body of a player who outputs the sound indicated by the output sound information from the instrument according to the score indicated by the score-related information, which is to be paid attention to by a teacher of the instrument.

[0165] The output sound information is used for each bar of the music piece to be performed. The output sound information is not limited to being used for each bar, but may be used for every four bars, for example. The attention point information (training output data) indicates the attention point in the bar immediately following the bar indicated by the output sound information in the training input data.

[0166] The determination unit 183 inputs a pair of student sound information a2 and score information to the trained model 190 for each measure. The pair of student sound information a2 and score information is an example of input information including sound information and score information. The determination unit 183 generates note data indicating the student's performance sound based on the student sound information a2, and identifies one measure in the student sound information a2 based on the arrangement of the note data. Next, the determination unit 183 determines the location indicated by the information output by the trained model 190 in response to the input of the pair of student sound information a2 and score information as the attention location.

[0167] In addition, the multiple training data T5 may have only training input data without having training output data. In this case, multiple variables K5 are identified by machine learning so that the multiple training data T5 are divided into multiple clusters based on the similarity of the multiple training data T5. Then, in the trained model 190, "information indicating a body part (attention part)" suitable for each cluster is associated by a human. The trained model 190 identifies a cluster according to the input X1, and generates information corresponding to the identified cluster as an output Y1.

[0168] According to the eleventh modified example, the images necessary for instruction can be switched according to the correspondence between the student's performance sound and the musical score.

[0169] B12: 12th variant In the above-mentioned embodiment and the first to eleventh modifications, the determination unit 183 of the student training system 100 may further determine the attention part on the body based on written information. The written information indicates notes written about the performance. The notes may be indicated by characters or symbols. The written information is an example of the attention information indicating the notes about the performance.

[0170] For example, the determination unit 183 of the student training system 100 determines the attention location based on the teacher's writing information. The teacher's writing information indicates the notes written on the musical score by the teacher 200B. The teacher's writing information is generated by one of the cameras 111-115 in the teacher training system 200 that captures the musical score on which the notes are written. The communication unit 160 of the teacher training system 200 transmits the teacher's writing information to the student training system 100. The determination unit 183 of the student training system 100 receives the teacher's writing information via the communication unit 160 of the student training system 100. The storage device 170 of the student training system 100 pre-stores a note table indicating the correspondence between the notes and the body parts. The determination unit 183 of the student training system 100 further determines, in the note table, the body part corresponding to the note indicated by the teacher's writing information as the attention location.

[0171] The determining unit 183 of the student training system 100 may determine the attention part based on the position of the note in the musical score. In this case, the storage device 170 of the student training system 100 stores in advance a position table indicating the correspondence between the position in the musical score and the part of the body. The determining unit 183 of the student training system 100 further determines, as the attention part, the part of the body corresponding to the position of the note in the musical score in the position table.

[0172] The notes may be written on an object separate from the musical score (eg, a notepad, notebook, or whiteboard).

[0173] According to the twelfth modified example, points of interest can be added based on notes written about the performance.

[0174] B13: 13th variant In the above-described embodiment and the first to twelfth modifications, the determining unit 183 of the student training system 100 may determine the body part of interest based on player information related to the player. The player information is, for example, identification information of the teacher 200B.

[0175] In musical instrument lessons, the focus may be different for each teacher 200B. For example, in piano lessons, the teacher 200B1 may focus on the right arm of the student 100B in addition to each finger on the hand, both feet, and the whole body of the student 100B, and the teacher 200B1 may focus on the left arm of the student 100B in addition to each finger on the hand, both feet, and the whole body of the student 100B. Therefore, the determination unit 183 of the student training system 100 further determines the focus based on the identification information (e.g., an identification code) of the teacher 200B.

[0176] The identification information of the teacher 200B is input from the operation unit 150 by a user such as the student 100B. The identification information of the teacher 200B may be transmitted from the teacher training system 200 to the student training system 100. The storage device 170 of the student training system 100 stores in advance an identification information table indicating the correspondence between the identification information of the teacher 200B and parts of the body. The determination unit 183 of the student training system 100 further determines, as a part of interest, the part of the body corresponding to the identification information of the teacher 200B in the identification information table.

[0177] The performer information is not limited to the identification information of the teacher 200B, and may be, for example, motion information indicating the motion of the teacher 200B. For example, one of the cameras 111-115 in the teacher training system 200 captures an image of the teacher 200B to generate motion information. The communication unit 160 of the teacher training system 200 transmits the motion information to the student training system 100. The decision unit 183 of the student training system 100 receives the motion information via the communication unit 160 of the student training system 100. The storage device 170 of the student training system 100 prestores a motion table indicating the correspondence between a person's motion and a body part. The decision unit 183 of the student training system 100 further decides, in the motion table, a body part corresponding to the motion indicated by the motion information as a focus part. Therefore, the teacher 200B can specify a focus part according to the motion of the teacher 200B. The performer information may be identification information of the student 100B, or movement information indicating the movement of the student 100B. In this case, the determination unit 183 can determine the attention portion according to the student 100B.

[0178] According to the thirteenth modified example, body parts of the performer can be added based on performer information about the performer.

[0179] B14: 14th variant In the above-described embodiment and the first to thirteenth modified examples, the operation unit 150 which is a touch panel may have a user interface as shown in FIG. 20 as a user interface for receiving the student instrument information c1. Touching the piano button 151 means inputting the student instrument information c1 indicating piano as the type of instrument. Touching the flute button 152 means inputting the student instrument information c1 indicating flute as the type of instrument. The user interface for receiving the student instrument information c1 is not limited to the user interface shown in FIG. 20. According to the fourteenth modified example, the user can intuitively input the student instrument information c1.

[0180] B15: 15th variant In the above-mentioned embodiment and the first to fourteenth modified examples, the communication unit 160 of the teacher training system 200 may transmit the teacher instrument information d1 or d2 to the student training system, and the determination unit 183 of the student training system may determine the attention point based on the teacher instrument information d1 or d2. Also, the communication unit 160 of the student training system may transmit the student instrument information c1 or c2 to the teacher training system, and the determination unit 183 of the teacher training system may determine the attention point based on the student instrument information c1 or c2. Also, the configuration of the teacher training system 200 may be the same as the configuration of any one of the student training systems 101 to 105.

[0181] B16: 16th variant In the above-described embodiment and the first to fifteenth modifications, the processing device 180 may generate a trained model 182.

[0182] Fig. 21 is a diagram showing a student training system 105 according to a sixteenth modified example. The student training system 105 differs from the student training system 104 shown in Fig. 19 in that it has a learning processing unit 191. The learning processing unit 191 is realized by a processing device 180 that executes a machine learning program. The machine learning program is stored in the storage device 170.

[0183] 22 is a diagram illustrating an example of the learning processing unit 191. The learning processing unit 191 includes a data acquiring unit 192 and a training unit 193. The data acquiring unit 192 acquires a plurality of training data T1. For example, the data acquiring unit 192 acquires the plurality of training data T1 via the operation unit 150 or the communication unit 160. When the storage device 170 stores a plurality of training data T1, the data acquiring unit 192 acquires the plurality of training data T1 from the storage device 170.

[0184] The training unit 193 generates the trained model 182 by executing a process (hereinafter referred to as a "learning process") using a plurality of training data T1. The learning process is supervised machine learning using a plurality of training data T1. The training unit 193 changes the trained model 182a to the trained model 182 by training the trained model 182a using the plurality of training data T1.

[0185] The learning object model 182a is generated by a processing device 180 that uses a plurality of provisional variables K1 and a calculation program. The plurality of provisional variables K1 are stored in the storage device 170. The learning object model 182a differs from the trained model 182 in that the learning object model 182a uses a plurality of provisional variables K1. The learning object model 182a generates information (output data) according to input information (input data).

[0186] The training unit 193 identifies a value of a loss function L that represents an error between output data generated by the learning model 182a when input data in the training data T1 is input to the learning model 182a and the output data in the training data T1. The training unit 193 updates a plurality of provisional variables K1 so as to reduce the value of the loss function L. The training unit 193 executes a process of updating the plurality of provisional variables K1 for each of the plurality of training data T1. Upon completion of training by the training unit 193, the plurality of variables K1 are finalized. The learning model 182a after training by the training unit 193, i.e., the learned model 182, outputs statistically valid output data for unknown input data.

[0187] 23 is a diagram showing an example of the learning process, which is started in response to, for example, an instruction from a user.

[0188] In step S301, the data acquisition unit 192 acquires unacquired training data T1 from among multiple training data T1. Then, in step S302, the training unit 193 trains the learning target model 182a using the training data T1. In step S302, the training unit 193 updates multiple provisional variables K1 so that the value of the loss function L specified using the training data T1 is reduced. For example, the backpropagation method is used in the process of updating the multiple provisional variables K1 according to the value of the loss function L.

[0189] Next, in step S303, the training unit 193 determines whether or not a termination condition for the learning process is satisfied. The termination condition is, for example, that the value of the loss function L falls below a predetermined threshold, or that the amount of change in the value of the loss function L falls below a predetermined threshold. If the termination condition is not satisfied, the process returns to step S301. Therefore, until the termination condition is satisfied, the acquisition of training data T1 and the update of multiple provisional variables K1 using the training data T1 are repeated. If the termination condition is satisfied, the learning process is terminated.

[0190] The learning processing unit 191 may be realized in a processing device different from the processing device 180. The processing device different from the processing device 180 includes at least one computer.

[0191] The data acquisition unit 192 may acquire multiple training data different from the multiple training data T1, for example, one or more types of multiple training data among four types of multiple training data T2, T3, T4, and T5. The training unit 193 trains a learning target model according to the type of the multiple training data acquired by the data acquisition unit 192. The learning target model according to the multiple training data T2 is a learning target model generated by the processing device 180 using a provisional multiple variables K2 and a calculation program. The learning target model according to the multiple training data T3 is a learning target model generated by the processing device 180 using a provisional multiple variables K3 and a calculation program. The learning target model according to the multiple training data T4 is a learning target model generated by the processing device 180 using a provisional multiple variables K4 and a calculation program. The learning target model according to the multiple training data T5 is a learning target model generated by the processing device 180 using a provisional multiple variables K5 and a calculation program.

[0192] The data acquisition unit 192 may be provided for each of a plurality of types of training data. In this case, each data acquisition unit 192 acquires a corresponding plurality of training data.

[0193] The training unit 193 may be provided for each of a plurality of types of training data. In this case, each training unit 193 uses the corresponding plurality of training data to train a learning object model according to the corresponding plurality of training data.

[0194] According to the sixteenth modification, the learning processing unit 241 can generate at least one learned model.

[0195] B17: 17th variant In the above-described embodiment and the first to sixteenth modifications, the processing device 180 may function only as the determining unit 183 and the acquiring unit 184, as shown in Fig. 24. The determining unit 183 shown in Fig. 24 determines a point of interest from the body of a player who uses an instrument of the type indicated by the instrument information, based on the instrument type. The acquiring unit 184 shown in Fig. 24 acquires image information representing an image of the point of interest determined by the determining unit 183. According to the seventeenth modification, it is possible to identify an image of a player required for training in playing a musical instrument, depending on the type of the instrument.

[0196] B18: 18th variant In the seventeenth modification, the determination unit 183 shown in Fig. 24 may determine a point of interest from the body of a player who uses an instrument based on sound information indicating a sound output from the instrument, not based on instrument information indicating the type of the instrument. Furthermore, in the seventeenth modification, the acquisition unit 184 shown in Fig. 24 may acquire image information representing an image of the point of interest determined by the determination unit 183 based on sound information indicating a sound output from the instrument. According to the eighteenth modification, it is possible to identify an image of a player required for training in playing a musical instrument, according to the sound output from the instrument.

[0197] C: Aspects understood from the above-mentioned forms The following aspects can be understood from at least one of the above-described aspects.

[0198] C1: First aspect An information processing method according to a first aspect of the present disclosure is an information processing device executed by a computer, which determines a focus point on the body of a player playing the instrument indicated by the instrument information based on the instrument, and obtains image information representing an image of the determined focus point. According to this aspect, an image of a player required for training on playing the instrument can be identified according to the instrument.

[0199] C2: Second mode In the example of the first aspect (second aspect), the acquired image information is further transmitted to an external device. According to this aspect, an image of a player required for training on playing a musical instrument can be transmitted to the external device.

[0200] C3: Third mode In the example of the first or second aspect (third aspect), specifying the musical instrument information using related information about the musical instrument and determining the attention point further includes determining the attention point based on the specified musical instrument information. According to this aspect, an image of a player required for training on playing a musical instrument can be specified based on related information about the musical instrument.

[0201] C4: Fourth mode In an example of the third aspect (fourth aspect), the related information is information showing a sound output by the musical instrument, information showing an image of the musical instrument, information showing a musical score corresponding to the musical instrument, or information showing a combination of the musical instrument and a lesson schedule for the musical instrument. According to this aspect, various information can be used as the related information.

[0202] C5: Fifth mode In an example of the third or fourth aspect (fifth aspect), identifying the musical instrument information includes inputting the related information to a first trained model that has learned the relationship between related information for learning about the musical instrument and musical instrument information for learning that indicates the musical instrument identified from the related information for learning, and identifying information output by the first trained model in accordance with the related information as the musical instrument information. According to this aspect, the musical instrument information is identified using a trained model, so that the musical instrument information can indicate the instrument played by the performer with high accuracy.

[0203] C6: Sixth mode In an example of the fifth aspect (sixth aspect), the related information and the learning related information indicate a sound output by the musical instrument, and the learning musical instrument information indicates an instrument that outputs the sound indicated by the learning related information. According to this aspect, the musical instrument can be identified based on the sound output by the musical instrument.

[0204] C7: Seventh mode In an example of the fifth aspect (seventh aspect), the related information and the learning related information indicate an image representing the musical instrument, and the learning musical instrument information indicates the musical instrument represented by the image indicated by the learning related information. According to this aspect, the musical instrument can be identified based on the image representing the musical instrument.

[0205] C8: Eighth mode In an example of the third aspect (eighth aspect), the identifying the musical instrument information includes identifying, as the musical instrument information, reference musical instrument information corresponding to the related information by referring to a table showing a correspondence relationship between reference related information about the musical instrument and reference musical instrument information indicating the musical instrument. According to this aspect, the musical instrument information can be identified without using a trained model.

[0206] C9: 9th mode In any one of the first to eighth aspects (ninth aspect), determining the focus point includes determining the focus point based on the instrument information and sound information indicating a sound output from the instrument indicated by the instrument information. According to this aspect, an image of a player required for training on playing the instrument can be identified based on the sound output from the instrument.

[0207] C10: 10th aspect In an example of the ninth aspect (tenth aspect), determining the attention point includes inputting input information including the musical instrument information for learning, which indicates the musical instrument, and learning sound information indicating the sound output from the musical instrument indicated by the musical instrument information for learning, to a second trained model that has learned the relationship between learning output information indicating the attention point on the body of a player who plays the musical instrument indicated by the musical instrument information for learning and that outputs the sound indicated by the learning sound information, and determining the attention point based on output information output by the second trained model in response to the input information. According to this aspect, the attention point is identified using a trained model, so that an image of a player required for training to play a musical instrument can be identified with high accuracy based on the sound output from the musical instrument.

[0208] C11: 11th aspect An information processing method according to an eleventh aspect of the present disclosure is an information processing method executed by a computer, which determines a point of interest on the body of a player playing an instrument based on sound information indicating a sound output from the instrument, and obtains image information indicating an image of the determined point of interest. According to this aspect, an image of the player required for training in playing the instrument can be identified according to the sound output from the instrument.

[0209] C12: 12th mode In an example of the ninth or eleventh aspect (twelfth aspect), determining the attention location includes determining the attention location based on a relationship between score information indicating a musical score and the sound information. According to this aspect, an image of a player required for training on playing a musical instrument can be identified based on the relationship between the score information and the sound information.

[0210] C13: 13th aspect In an example of the eleventh aspect (thirteenth aspect), determining the attention point includes inputting input information including learning sound information indicating the sound output from the musical instrument and learning score information indicating the score to a third trained model that has learned the relationship between learning input information including learning sound information indicating the sound output from the musical instrument and learning score information indicating the score, and learning output information indicating the attention point on the body of a player who plays an instrument that outputs the sound indicated by the learning sound information according to the score indicated by the learning score information, and determining the attention point based on output information output by the third trained model in response to the input information. According to this aspect, the attention point is identified using a trained model, so that an image of a player required for training to play an instrument can be identified with high accuracy.

[0211] C14: 14th aspect In any one of the first to thirteenth aspects (fourteenth aspect), determining the attention point includes determining the attention point based on caution information indicating cautions about playing. According to this aspect, it is possible to switch an image of a player required for playing a musical instrument in accordance with the cautions about playing.

[0212] C15: 15th aspect In any one of the first to fourteenth aspects (fifteenth aspect), determining the attention point includes determining the attention point based on player information about the player. According to this aspect, it is possible to switch an image of a player required for training on playing a musical instrument in accordance with the player information about the player.

[0213] C16: 16th aspect An information processing system according to a sixteenth aspect of the present disclosure includes a determination unit that determines, based on musical instrument information indicating an instrument, a point of interest from the body of a player playing the musical instrument indicated by the musical instrument information, and an acquisition unit that acquires image information representing an image of the point of interest determined by the determination unit. According to this aspect, an image of a player required for training in playing a musical instrument can be identified according to the instrument.

[0214] C17: 17th mode An information processing system according to a seventeenth aspect of the present disclosure includes a determination unit that determines a focus point from the body of a player who uses a musical instrument based on sound information indicating a sound output from the musical instrument, and an acquisition unit that acquires image information representing an image of the focus point determined by the determination unit. According to this aspect, an image of the player required for training in playing the musical instrument can be identified according to the sound output from the musical instrument. [Explanation of symbols]

[0215] 1...information providing system, 100...student training system, 100A...musical instrument, 100B...student, 111-115...camera, 120...microphone, 130...display unit, 140...speaker, 150...operation unit, 160...communication unit, 170...storage device, 180...processing device, 181...identification unit, 182...trained model, 182a...learning target model, 183...determination unit, 184...acquisition unit, 185...transmission unit, 186...output control unit, 187-190...trained model, 191...learning processing unit, 192...data acquisition unit, 193...training unit, 200...teacher guidance system, 200A...musical instrument, 200B...teacher.

Claims

1. 1. A computer-implemented information processing method, comprising: determining a focus point on the body of a player of a musical instrument based on information about a person who is to check the performance of the player; acquiring image information representing an image of the determined point of interest; Information processing methods.

2. The information about the person includes an identification of the person. The information processing method according to claim 1 .

3. The information about the person includes movement information indicating a movement of the person. The information processing method according to claim 1 .

4. The information about the person includes caution information indicating cautions about the performance presented by the person. The information processing method according to claim 1 .

5. the type of instrument played by the performer is the same as the type of instrument played by the person; The information about the person includes information indicative of the type of instrument played by the person. The information processing method according to claim 1 .

6. 1. A computer-implemented information processing method, comprising: determining a focus area on a body of a player of a musical instrument based on information about the player; acquiring image information representing an image of the determined point of interest; Information processing methods.

7. The information about the performer includes identification information of the performer. The information processing method according to claim 6.

8. The information about the performer includes movement information indicating the movement of the performer. The information processing method according to claim 6.

9. Determining the attention portion includes determining the attention portion based on instrument information indicating a type of instrument played by the performer. The information processing method according to any one of claims 1 to 8.

10. and identifying the instrument information using related information regarding a type of instrument played by the performer; The determining of the attention portion determines the attention portion based on the identified musical instrument information. The information processing method according to claim 9.

11. The related information is information indicative of a sound output by an instrument played by said player; information indicative of an image representing a musical instrument played by the performer; information indicative of a musical score according to the type of instrument played by said player; or The information includes a combination of the type of musical instrument played by the player and a lesson schedule for the musical instrument. The information processing method according to claim 10.

12. Determining the point of interest includes: determining the attention point based on sound information indicating a sound output from the type of instrument indicated by the instrument information and the instrument information. The information processing method according to any one of claims 9 to 11.

13. Determining the attention point includes determining the attention point based on sound information indicating a sound output from an instrument played by the player. The information processing method according to any one of claims 1 to 11.

14. Determining the attention point includes determining the attention point based on a relationship between score information indicating a score representing a sound and the sound information.

14. The information processing method according to claim 12 or 13.

15. Determining the attention portion includes determining the attention portion based on attention information indicating attention points for playing. The information processing method according to any one of claims 6 to 8.

16. A determination unit that determines a focus area on a body of a player who is playing a musical instrument based on information about a person who is checking the performance of the player; an acquisition unit that acquires image information representing an image of the point of interest determined by the determination unit; An information processing system comprising:

17. A determination unit that determines a focus area on a body of a player who plays an instrument based on information about the player; an acquisition unit that acquires image information representing an image of the point of interest determined by the determination unit; An information processing system comprising:

18. On the computer, determining a focus point on the body of a player of a musical instrument based on information about a person who is to check the performance of the player; acquiring image information representing an image of the determined point of interest; A program that executes a process.

19. determining a focus area on a body of a player of a musical instrument based on information about the player; acquiring image information representing an image of the determined point of interest; A program that executes a process.

Images