Information processing system, electronic musical instrument, information processing method, and machine learning system

Abstract

An information processing system has: a performance data acquisition section that acquires performance data indicating performance of a musical piece by a user; a tendency determination section that generates tendency data indicating a tendency of performance by the user by inputting the performance data acquired by the performance data acquisition section to a trained model that has learned a relationship between learning performance data indicating performance of a musical piece and learning tendency data indicating a tendency of performance indicated by the learning performance data; and an exercise phrase determination section that determines an exercise phrase corresponding to the tendency data generated by the tendency determination section.

Description

TECHNICAL FIELD

[0001] The present application relates to a technology of assisting performance of a musical instrument such as an electronic musical instrument. BACKGROUND

[0002] Various technologies of assisting performance of a musical instrument such as an electronic musical instrument have been proposed in the past. For example, a technology is disclosed in Patent Literature 1 in which a statistical value such as a standard deviation is calculated from a difference between a parameter of musical piece data prepared in advance and a parameter of performance data representing performance by a user, and the statistical value is aggregated by a method corresponding to the kind of the parameter.

[0003] Patent Literature 1: Japanese Patent Application Laid-Open No. 2005-55635 SUMMARY

[0004] However, it is actually difficult to effectively practice performance while taking into account tendencies related to performance by each user (e.g., tendencies of performance errors, etc.) only by giving the user an evaluation value that evaluates the result of performance. In view of the above, one of the objects of one embodiment of the present application is to achieve effective practice of performance corresponding to tendencies of performance by a user.

[0005] To solve the above problem, one embodiment of the present application relates to an information processing system including: a performance data acquisition unit that acquires performance data representing performance of a musical piece by a user; a tendency determination unit that generates tendency data representing a tendency of performance by the user by inputting the performance data acquired by the performance data acquisition unit to a first trained model that has learned a relationship between learning performance data representing performance of a musical piece and learning tendency data representing a tendency of performance represented by the learning performance data; and a practice phrase determination unit that determines a practice phrase corresponding to the tendency data generated by the tendency determination unit.

[0006] One embodiment of the present application relates to an electronic musical instrument including: a performance acceptance unit that accepts performance of a musical piece by a user; a performance data acquisition unit that acquires performance data representing performance accepted by the performance acceptance unit; a tendency determination unit that outputs tendency data representing a tendency of performance by the user from a first trained model by inputting the performance data acquired by the performance data acquisition unit to the first trained model that has learned a relationship between learning performance data representing performance of a musical piece and learning tendency data representing a tendency of performance represented by the learning performance data; a practice phrase determination unit that determines a practice phrase corresponding to the tendency of performance by the user using the tendency data output by the tendency determination unit; and

[0007] a prompt processing section that prompts the user with the practice phrase.

[0008] One embodiment of the information processing method according to the present application acquires performance data indicating performance of a musical piece by a user, generates tendency data indicating a tendency of performance by the user by inputting the acquired performance data to a first trained model that has learned a relationship between learning performance data indicating performance of a musical piece and learning tendency data indicating a tendency of performance indicated by the learning performance data, and determines a practice phrase corresponding to the tendency data.

[0009] One embodiment of the machine learning system according to the present application includes a first learning data acquisition section that acquires performance data indicating performance of a musical piece by a user and pointing data indicating a time point within the musical piece and a tendency of performance at the time point, and a first learning processing section that creates a first trained model that has learned a relationship between learning performance data and learning tendency data by machine learning using first learning data indicating a combination of the learning performance data and the learning tendency data, wherein the learning performance data indicates performance within an interval containing the time point indicated by the pointing data among the performance data, and the learning tendency data indicates a tendency of performance indicated by the pointing data. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 is a block diagram illustrating a structure of a performance system according to a first embodiment.

[0011] Figure 2 is a block diagram illustrating a structure of an electronic musical instrument.

[0012] Figure 3 is a block diagram illustrating a structure of an information processing system.

[0013] Figure 4 is a block diagram illustrating a functional structure of an information processing system.

[0014] Figure 5 is a flowchart illustrating a specific flow of a determination process.

[0015] Figure 6 is a block diagram illustrating a structure of a machine learning system.

[0016] Figure 7 is a block diagram illustrating a functional structure of a machine learning system.

[0017] Figure 8 is a block diagram illustrating a structure of an information device used by an instructor.

[0018] Figure 9 is a diagram schematically showing the data.

[0019] Figure 10 is a flowchart illustrating a specific flow of the preparation process.

[0020] Figure 11 is a flowchart illustrating a specific flow of the learning process.

[0021] Figure 12 is a block diagram illustrating a functional structure of the information processing system of the second embodiment.

[0022] Figure 13 is a flowchart illustrating a flow of the determination process of the second embodiment.

[0023] Figure 14 is a block diagram illustrating a functional structure of the information processing system of the third embodiment.

[0024] Figure 15 is a flowchart illustrating a flow of the determination process of the third embodiment.

[0025] Figure 16 is a block diagram illustrating a functional structure of the machine learning system of the third embodiment.

[0026] Figure 17 is a flowchart illustrating a flow of the learning process of the third embodiment.

[0027] Figure 18 is a block diagram illustrating a functional structure of the electronic musical instrument of the fourth embodiment.

[0028] Figure 19 is a block diagram illustrating a functional structure of the information device of the fifth embodiment. DETAILED DESCRIPTION

[0029] A: First Embodiment

[0030] Figure 1 is a block diagram illustrating a structure of a performance system 100 involved in the first embodiment. The performance system 100 is a computer system for practicing performance of an electronic musical instrument 10 by a user U of the electronic musical instrument 10, and has the electronic musical instrument 10, an information processing system 20, and a machine learning system 30. Each element constituting the performance system 100 communicates with each other, for example, via a communication network 200 such as the Internet. Further, the performance system 100 actually includes a plurality of electronic musical instruments 10, but in the following description, attention is focused on an arbitrary one of the electronic musical instruments 10 for convenience.

[0031] Figure 2is a block diagram illustrating the structure of the electronic musical instrument 10. The electronic musical instrument 10 is a performance device used by the user U for playing a musical piece. The electronic musical instrument 10 of the first embodiment is an electronic keyboard instrument having a plurality of keys operated by the user U. The electronic musical instrument 10 is implemented by a computer system having a control device 11, a storage device 12, a communication device 13, a performance device 14, a display device 15, a sound source device 16, and a sound output device 17. Further, the electronic musical instrument 10 can be implemented by a plurality of devices configured separately from each other, in addition to being implemented as a single device.

[0032] The control device 11 is constituted by a single or a plurality of processors that control each element of the electronic musical instrument 10. For example, the control device 11 is constituted by one or more processors such as a CPU (Central Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

[0033] The storage device 12 is a single or a plurality of memories that store programs executed by the control device 11 and various data used by the control device 11. The storage device 12 is constituted by, for example, a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of recording media. Further, a removable type recording medium that is detachable with respect to the electronic musical instrument 10, or a recording medium (for example, cloud storage) that can be written or read by the control device 11 via, for example, the communication network 200 can be utilized as the storage device 12.

[0034] The storage device 12 of the first embodiment stores a plurality of musical piece data X representing different musical pieces. The musical piece data X of each musical piece specifies a time series of a plurality of notes that constitute a part or all of the musical piece. Specifically, the musical piece data X specifies a pitch and a sound emission period for each note within the musical piece. The musical piece data X is, for example, data in a format in accordance with the MIDI (Musical Instrument Digital Interface) standard.

[0035] The communication device 13 communicates with the information processing system 20 via the communication network 200. Furthermore, the communication between the communication device 13 and the communication network 200 can be any of wired communication and wireless communication. In addition, the communication device 13 that is separate from the electronic musical instrument 10 can be connected to the electronic musical instrument 10 in a wired or wireless manner. As the communication device 13 that is separate from the electronic musical instrument 10, an information terminal such as a smartphone or a tablet terminal is exemplified.

[0036] The display device 15 displays an image under the control of the control device 11. For example, various display panels such as a liquid crystal display panel or an organic EL (Electroluminescence) panel are utilized as the display device 15. The display device 15 displays, for example, a musical score of a piece of music played by the user U using piece data X of the piece of music.

[0037] The performance device 14 is an input device that accepts a performance by the user U. For example, the performance device 14 has a keyboard in which a plurality of keys corresponding to different pitches are arranged. The user U performs a piece of music by sequentially operating desired keys of the performance device 14. The performance device 14 is an example of a "performance accepting section".

[0038] The control device 11 generates performance data Y that indicates a performance of a piece of music by the user U. Specifically, the performance data Y specifies a pitch and a sound emission period for each of a plurality of notes instructed by the user U through an operation of the performance device 14. The performance data Y is, like the piece data X, time series data in a format according to, for example, the MIDI standard. The communication device 13 transmits the performance data Y that indicates a performance of a piece of music by the user U and the piece data X of the piece of music to the information processing system 20. The piece data X is data that indicates a demonstrative or standard performance related to the piece of music, and the performance data Y is data that indicates an actual performance of the piece of music by the user U. Therefore, each note specified by the piece data X and each note specified by the performance data Y are related to each other but not completely identical. In particular, at a site in the piece of music where a performance error by the user U is likely to occur, or a site that is not good at a performance by the user U, a difference between the piece data X and the performance data Y is noticeable.

[0039] The sound source device 16 generates a sound signal A corresponding to the performance of the performance device 14. The sound signal A is a signal indicating a waveform of a tone instructed by the performance of the performance device 14. Specifically, the sound source device 16 is a MIDI sound source that generates a sound signal A indicating a tone of each note specified in a time series by the performance data Y. That is, the sound source device 16 generates a sound signal A indicating a tone of a pitch corresponding to a key pressed by the user U among the plurality of keys of the performance device 14. Furthermore, the function of the sound source device 16 can be implemented by the control device 11 by executing a program stored in the storage device 12. That is, the sound source device 16 dedicated for the generation of the sound signal A is omitted.

[0040] The sound emitting device 17 emits the performance tone indicated by the sound signal A. For example, a speaker or a headphone is utilized as the sound emitting device 17. As understood from the above description, the sound source device 16 and the sound emitting device 17 of the first embodiment function as a playback system 18 that plays back a tone corresponding to the performance of the user U.

[0041] Figure 3 is a block diagram illustrating the structure of the information processing system 20. The information processing system 20 provides a musical phrase (hereinafter, referred to as "practice phrase") Z suitable for the practice of the performance of the user U to the user U. The information processing system 20 is implemented by a computer system having a control device 21, a storage device 22, and a communication device 23. Furthermore, the information processing system 20 can be implemented by a plurality of devices configured to be separate from each other, in addition to being implemented as a single device.

[0042] The control device 21 is constituted by a single or a plurality of processors that control each element of the information processing system 20. For example, the control device 21 is constituted by one or more processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC. The communication device 23 communicates with each of the electronic musical instrument 10 and the machine learning system 30 via the communication network 200. Furthermore, the communication between the communication device 23 and the communication network 200 can be any of wired communication and wireless communication.

[0043] The storage device 22 is a single or a plurality of memories that store programs executed by the control device 21 and various data used by the control device 21. The storage device 22 is constituted by, for example, a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of recording media. Furthermore, a removable type recording medium detachable with respect to the information processing system 20, or a recording medium (for example, cloud storage) capable of being written or read by the control device 21 via, for example, the communication network 200 can be utilized as the storage device 22.

[0044] Figure 4is a block diagram illustrating a functional configuration of the information processing system 20. The storage device 22 stores a plurality of practice phrases Z corresponding to different tendency data D. In other words, a table in which the plurality of tendency data D and the plurality of practice phrases Z are each associated with each other is stored in the storage device 22.

[0045] The tendency data D is arbitrary form data indicating a tendency of the performance by the performer (hereinafter, referred to as "performance tendency"). The performance tendency is, for example, a tendency of a performance error of the performer or a tendency of a performance method that the performer is not good at. For example, any of a plurality of performance tendencies of "time point shift of keying", "keying of other keys adjacent to a target key", "mistake in pitch", "poor performance of a leap", "poor performance of a chord (harmony)", "poor performance of a finger stretch", and the like is specified by the tendency data D. Further, the leap is a section in which two notes with a pitch difference exceeding a prescribed value (for example, 3 degrees) are performed in succession. In addition, the finger stretch is a performance method in which other fingers are moved in a manner of passing under the fingers that key the keys corresponding to one note.

[0046] The practice phrase Z is time series data indicating a piece of music composed of a plurality of notes, and specifically, is a melody (for example, a part or all of a practice piece) suitable for practice of the electronic musical instrument 10. The practice phrase Z is composed of a time series of single notes or chords. The practice phrase Z corresponding to each tendency data D indicates a piece of music suitable for improvement of the performance tendency specified by the tendency data D. For example, for the tendency data D of the performance tendency of "poor performance of a leap", a large number of practice phrases Z containing a leap are registered. In addition, for the tendency data D of the performance tendency of "poor performance of a chord", a large number of practice phrases Z containing a chord are registered. The practice phrase Z is, for example, MIDI form data in which a pitch and a sound emission period are specified for each of a plurality of notes.

[0047] The control device 21 of the information processing system 20 realizes a plurality of elements (the performance data acquisition section 71, the tendency determination section 72, and the practice phrase determination section 73) for determining the practice phrase Z from the piece data X and the performance data Y by executing a program stored in the storage device 22.

[0048] The performance data acquisition section 71 acquires the performance data Y indicating the performance of the piece by the user U. Specifically, the performance data acquisition section 71 receives the piece data X and the performance data Y transmitted from the electronic musical instrument 10 through the communication device 23. The control data C including the piece data X and the performance data Y is generated by the performance data acquisition section 71.

[0049] The tendency determination section 72 generates tendency data D indicating the performance tendency of the user U in correspondence with the control data C. For the generation of the tendency data D by the tendency determination section 72, a trained model Ma is utilized. The trained model Ma is one example of the "first trained model".

[0050] There is a correlation between the difference between the musical score of the piece played by the performer (the piece data X) and the actual performance (the performance data Y) by the performer and the performance tendency (the tendency data D) of the performer. For example, in a case where the time point of the sound of each note differs between the piece data X and the performance data Y, the performance tendency of "shift of the time point of the key pressing" is presumed. In addition, in a case where another note close to the note indicated by the piece data X is designated by the performance data Y, the performance tendency of "pressing of another key adjacent to the target key" is presumed. In addition, in a case where the difference between the piece data X and the performance data Y is obvious at a site where there is a leap in the piece, the performance tendency of "poor at the leap" is presumed. The trained model Ma is a statistical presumption model that has learned the above such tendencies. That is, the trained model Ma is a statistical presumption model that has learned the relationship between the combination of the piece data X and the performance data Y (that is, the control data C) and the tendency data D indicating the performance tendency of the performer. The tendency determination section 72 outputs the tendency data D indicating the performance tendency of the user U from the trained model Ma by inputting the control data C including the piece data X and the performance data Y to the trained model Ma.

[0051] The trained model Ma is constituted by, for example, a deep neural network (DNN). For example, any form of neural network such as a recurrent neural network (RNN) or a convolutional neural network (CNN) is utilized as the trained model Ma. The trained model Ma can be constituted by a combination of a plurality of deep neural networks. In addition, an additional element such as a long short-term memory (LSTM) can be mounted on the trained model Ma.

[0052] The trained model Ma is realized by a combination of a program that causes the control device 21 to execute an operation of generating the tendency data D in accordance with the control data C and a plurality of variables (specifically, weighting values and biases) applied to the operation. The program and the plurality of variables realizing the trained model Ma are stored in the storage device 22. The values of the respective plurality of variables of the trained model Ma are set in advance by machine learning.

[0053] The exercise piece determining section 73 determines the exercise piece Z corresponding to the performance tendency of the user U, using the tendency data D determined by the tendency determining section 72. Specifically, the exercise piece determining section 73 searches for the exercise piece Z corresponding to the tendency data D determined by the tendency determining section 72, from among the plurality of exercise pieces Z stored in the storage device 22. That is, the exercise piece determining section 73 determines the exercise piece Z appropriate for improving the performance tendency of the user U shown by the tendency data D.

[0054] The exercise piece Z determined by the exercise piece determining section 73 is transmitted from the communication device 23 to the electronic musical instrument 10. The communication device 13 of the electronic musical instrument 10 receives the exercise piece Z transmitted from the information processing system 20. The control device 11 causes the score of the exercise piece Z to be displayed on the display device 15. The user U performs the exercise piece Z while checking the score displayed on the display device 15.

[0055] Figure 5 is a flowchart illustrating a specific flow of processing (hereinafter, referred to as "determination processing") Sa performed by the control device 21 of the information processing system 20.

[0056] If the determination processing Sa is started, the performance data acquiring section 71 waits until the musical piece data X and the performance data Y transmitted from the electronic musical instrument 10 are received by the communication device 23 (Sa1: NO). If the performance data acquiring section 71 acquires the musical piece data X and the performance data Y (Sa1: YES), the tendency determining section 72 inputs the control data C including the musical piece data X and the performance data Y to the trained model Ma, thereby outputting the tendency data D from the trained model Ma (Sa2). The exercise piece determining section 73 determines the exercise piece Z corresponding to the tendency data D, from among the plurality of exercise pieces Z stored in the storage device 22 (Sa3). The exercise piece determining section 73 transmits the exercise piece Z from the communication device 23 to the electronic musical instrument 10 (Sa4).

[0057] As described above, in the first embodiment, the tendency data D indicating the performance tendency of the user U is generated by inputting the performance data Y indicating the performance of the musical piece by the user U to the trained model Ma, and the exercise piece Z corresponding to the tendency data D is determined. Therefore, by the user U performing the exercise piece Z, effective practice corresponding to the performance tendency of the user U is realized.

[0058] In the first embodiment, the exercise piece Z corresponding to the performance tendency of the user U is determined from among the plurality of exercise pieces Z corresponding to different performance tendencies (tendency data D). Therefore, the load of processing for determining the exercise piece Z corresponding to the performance tendency of the user U is reduced.

[0059] Figure 1 The machine learning system 30 generates the above exemplified trained model Ma. Figure 6 is a block diagram exemplifying a structure of the machine learning system 30. The machine learning system 30 has a control device 31, a storage device 32, and a communication device 33. Further, the machine learning system 30 can be realized by a plurality of devices configured to be separated from each other, in addition to being realized as a single device.

[0060] The control device 31 is constituted by a single or a plurality of processors that control each element of the machine learning system 30. For example, the control device 31 is constituted by one or more processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC. The communication device 33 communicates with the information processing system 20 via the communication network 200. Further, the communication between the communication device 33 and the communication network 200 can be any of wired communication and wireless communication.

[0061] The storage device 32 is a single or a plurality of memories that store programs executed by the control device 31 and various data used by the control device 31. The storage device 32 is constituted by, for example, a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of recording media. In addition, a removable type recording medium that is attachable to and detachable from the machine learning system 30, or a recording medium (for example, cloud storage) that can be written or read by the control device 31 via, for example, the communication network 200 can be utilized as the storage device 32.

[0062] Figure 7 is a block diagram exemplifying a functional structure of the machine learning system 30. The control device 31 functions as a plurality of elements (a learning data acquisition unit 81a and a learning processing unit 82a) for creating a trained model Ma by machine learning by executing programs stored in the storage device 32.

[0063] The learning processing section 82a creates the trained model Ma by teacher-attended machine learning (learning processing Sc described later) using the plurality of learning data Ta. The learning data acquisition section 81a acquires the plurality of learning data Ta. The plurality of learning data Ta acquired by the learning data acquisition section 81a is stored in the storage device 32. The plurality of learning data Ta is each constituted by a combination of the control data Ct for learning and the tendency data Dt for learning. The control data Ct includes the musical piece data Xt for learning and the performance data Yt for learning. The musical piece data Xt is an example of "musical piece data for learning", the performance data Yt is an example of "performance data for learning", and the tendency data Dt is an example of "tendency data for learning". In addition, the musical piece represented by the musical piece data Xt is an example of "reference musical piece". The learning data acquisition section 81a is an example of "first learning data acquisition section", and the learning processing section 82a is an example of "first learning processing section". In addition, the learning data Ta is an example of "first learning data".

[0064] As illustrated as Figure 7 The learning data Ta is generated as a result of the performance of the musical piece by the learner U1 and the guidance of the performance by the instructor U2, as illustrated. The learner U1 performs the musical piece using the electronic musical instrument 10. The instructor U2 evaluates and guides the performance of the learner U1 using the information device 40. The information device 40 is, for example, an information terminal such as a smartphone or a tablet terminal. The learner U1 and the instructor U2 are, for example, remote. However, the learner U1 and the instructor U2 can be in the same place.

[0065] The electronic musical instrument 10 transmits the musical piece data X0 representing the musical piece and the performance data Y0 representing the performance of the musical piece by the learner U1 to the information device 40 and the machine learning system 30. The musical piece data X0 specifies the time series of the plurality of notes constituting the musical piece, as with the aforementioned musical piece data X. The performance data Y0 specifies the time series of the plurality of notes instructed by the learner U1 by the operation of the performance device 14, as with the aforementioned performance data Y.

[0066] Figure 8 is a block diagram illustrating the structure of the information device 40. The information device 40 is a computer system for evaluating and guiding the performance of the electronic musical instrument 10 by the learner U1 by the instructor U2, and has a control device 41, a storage device 42, a communication device 43, an operation device 44, a display device 45, and a playback system 46. In addition, the information device 40 can be realized by a plurality of devices constituted separately from each other, in addition to being realized as a single device.

[0067] The control device 41 is constituted by a single or a plurality of processors that control each element of the information device 40. For example, the control device 41 is constituted by one or more processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC.

[0068] The storage device 42 is a single or a plurality of memories that store programs executed by the control device 41 and various data used by the control device 41. The storage device 42 is constituted by, for example, a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of recording media. Further, a removable type recording medium that is detachable with respect to the information device 40, or a recording medium (for example, cloud storage) that can be written or read by the control device 41 via, for example, the communication network 200 can be utilized as the storage device 42.

[0069] The communication device 43 communicates with each of the electronic musical instrument 10 and the machine learning system 30 via the communication network 200. Further, the communication between the communication device 43 and the communication network 200 can be any of wired communication and wireless communication. The communication device 43 receives, for example, the musical piece data X0 and the performance data Y0 transmitted from the electronic musical instrument 10.

[0070] The operation device 44 is an input device that accepts an instruction from the instructor U2. The operation device 44 is, for example, a plurality of operation pieces operated by the instructor U2, or a touch panel that detects a contact of the instructor U2. The display device 45 displays an image under the control of the control device 41. Specifically, the display device 45 displays a time series of notes specified by the performance data Y received by the communication device 43. That is, an image representing the performance of the trainee U1 is displayed on the display device 45. Further, a time series of notes specified by the musical piece data X can be displayed in parallel with the notes of the performance data Y. The playback system 46 plays the tone of each note specified by the performance data Y, similarly to the playback system 18 of the electronic musical instrument 10. That is, the tone of the performance of the trainee U1 is played by the playback system 46.

[0071] The instructor U2 can confirm the performance of the musical piece by the trainee U1 by visually observing the image displayed on the display device 45 while listening to the playback sound of the playback system 46. The instructor U2 inputs the performance tendency that should be pointed out with respect to the performance of the musical piece by the trainee U1 by operating the operation device 44. The instructor U2 specifies the performance tendency related to the performance of the musical piece by the trainee U1 and the time point at which the performance tendency is observed within the musical piece. The performance tendency is, for example, selected from a plurality of options by the instructor U2 by operation of the operation device 44. For example, any of a plurality of performance tendencies such as "time point shift of key", "keying of other keys adjacent to the target key", "mistake in pitch", "poor in jump progress", "poor in performance of chord", "poor in fast performance of short notes such as 16th note", and the like is selected as the matter to be pointed out with respect to the performance of the trainee U1.

[0072] The control device 41 generates the pointing-out data P corresponding to the instruction from the instructor U2. Figure 9 is a schematic view of the pointing-out data P. The pointing-out data P includes the tendency data Dt and the time point data τ for each pointing-out by the instructor U2. The tendency data Dt is data indicating the performance tendency pointed out by the instructor U2. The time point data τ is data indicating the time point at which the performance tendency is observed within the musical piece. As understood from the above description, the pointing-out data P is data indicating the time point within the musical piece and the performance tendency at the time point.

[0073] The communication device 43 transmits the pointing-out data P generated by the control device 41 to the electronic musical instrument 10 and the machine learning system 30. The communication device 13 of the electronic musical instrument 10 receives the pointing-out data P transmitted from the information device 40. The control device 11 displays the performance tendency indicated by the pointing-out data P on the display device 15. The trainee U1 can confirm the pointing-out (performance tendency) by the instructor U2 by visually observing the image of the display device 15.

[0074] As exemplified in Figure 7 , the learning data acquisition section 81a of the machine learning system 30 receives the musical piece data X0 and the performance data Y0 transmitted from the electronic musical instrument 10 and the pointing-out data P transmitted from the information device 40 through the communication device 33. The learning data acquisition section 81a generates the learning data Ta using the musical piece data X0, the performance data Y0, and the pointing-out data P. Further, the electronic musical instrument 10 is an example of the "first device", and the information device 40 is an example of the "second device".

[0075] Figure 10is a flowchart illustrating a concrete flow of a process (hereinafter, referred to as "preparation process") Sb in which the learning data acquisition section 81a generates learning data Ta. For example, the preparation process Sb is started as an opportunity of the communication device 33 receiving the musical piece data X0, the performance data Y0, and the pointing data P. If the preparation process Sb is started, the learning data acquisition section 81a acquires the musical piece data X0, the performance data Y0, and the pointing data P from the communication device 33 (Sbl).

[0076] The learning data acquisition section 81a extracts a portion within an interval (hereinafter, referred to as "specific interval") of a time point designated by the time point data τ of the pointing data P among the musical piece data X0 as the musical piece data Xt (Sb2). The specific interval is, for example, an interval of a prescribed length with the time point designated by the time point data τ as a midpoint. In addition, the learning data acquisition section 81a extracts a portion within the specific interval of the time point designated by the time point data τ of the pointing data P among the performance data Y0 as the performance data Yt (Sb3). That is, the specific interval including the time point at which the performance tendency pointed out by the instructor U2 is extracted for each of the musical piece data X0 and the performance data Y0.

[0077] The learning data acquisition section 81a generates control data Ct for learning including the musical piece data Xt and the performance data Yt generated by the above flow (Sb4). Then, the learning data acquisition section 81a generates the learning data Ta by making the control data Ct for learning and the tendency data Dt included in the pointing data P correspond to each other (Sb5).

[0078] By repeating the above illustrated preparation process Sb, the learning data Ta including the musical piece data Xt and the performance data Yt corresponding to the specific interval and the tendency data Dt of the performance tendency pointed out by the instructor U2 with respect to the specific interval is generated with respect to the performance of a plurality of musical pieces by a plurality of trainees U1.

[0079] Figure 11 is a flowchart illustrating a concrete flow of a learning process Sc in which the control device 31 of the machine learning system 30 creates the trained model Ma. The learning process Sc also represents a method (trained model generation method) of generating the trained model Ma by machine learning.

[0080] If the learning process Sc is started, the learning processing section 82a selects any of the plurality of learning data Ta stored in the storage device 32 (hereinafter, referred to as "selected learning data Ta") (Sc1). The learning processing section 82a extracts a portion within a specific interval of a time point designated by the time point data τ of the pointing data P included in the selected learning data Ta as the musical piece data Xt (Sc2). The specific interval is, for example, an interval of a prescribed length with the time point designated by the time point data τ as a midpoint. In addition, the learning processing section 82a extracts a portion within the specific interval of the time point designated by the time point data τ of the pointing data P among the performance data Yt as the performance data Yt (Sc3). That is, the specific interval including the time point at which the performance tendency pointed out by the instructor U2 is extracted for each of the musical piece data Xt and the performance data Yt. Figure 7As exemplified, the control data Ct of the selection learning data Ta is input to an initial or temporary model (hereinafter, referred to as "temporary model Ma0") (Sc2), and the tendency data D output by the temporary model Ma0 for the input is acquired (Sc3).

[0081] The learning processing section 82a calculates a loss function representing an error between the tendency data D generated by the temporary model Ma0 and the tendency data Dt of the selection learning data Ta (Sc4). The learning processing section 82a updates a plurality of variables of the temporary model Ma0 so that the loss function decreases (ideally, is minimized) (Sc5). For the update of the plurality of variables corresponding to the loss function, for example, an error backpropagation method is used.

[0082] The learning processing section 82a determines whether a prescribed end condition is satisfied (Sc6). The end condition is, for example, that the loss function is lower than a prescribed threshold value, or that the amount of change in the loss function is lower than a prescribed threshold value. In the case where the end condition is not satisfied (Sc6: NO), the learning processing section 82a selects the unselected learning data Ta as new selection learning data Ta (Sc1). That is, the processing of updating the plurality of variables of the temporary model Ma0 (Sc2-Sc5) is repeated until the end condition is satisfied (Sc6: YES). In the case where the end condition is satisfied (Sc6: YES), the learning processing section 82a ends the update of the prescribed plurality of variables of the temporary model Ma0 (Sc2-Sc5). The temporary model Ma0 at the point in time at which the end condition is satisfied is determined as the trained model Ma. That is, the plurality of variables of the trained model Ma is determined as the values at the point in time at which the learning processing Sc ends.

[0083] As understood from the above description, the trained model Ma outputs statistically reasonable tendency data D for unknown control data C based on the potential relationship between the control data Ct and the tendency data Dt of the plurality of learning data Ta. That is, the trained model Ma is a statistical learning model that has learned (trained) the relationship between the performance (control data C) of the piece of music by the performer and the performance tendency (tendency data D) of the performer as described above.

[0084] The learning processing section 82a transmits the trained model Ma created through the above flow from the communication device 33 to the information processing system 20 (Sc7). Specifically, the learning processing section 82a transmits the plurality of variables of the trained model Ma from the communication device 33 to the information processing system 20. The control device 21 of the information processing system 20 saves the trained model Ma received from the machine learning system 30 in the storage device 22. Specifically, the plurality of variables of the trained model Ma are stored in the storage device 22.

[0085] B: 2nd Embodiment

[0086] The 2nd Embodiment will be described. Furthermore, in each of the modes exemplified below, for elements common in function and structure to the 1st Embodiment, the same reference numerals as in the 1st Embodiment are used and the detailed description of each will be appropriately omitted.

[0087] Figure 12 is a block diagram exemplifying the functional structure of the information processing system 20 of the 2nd Embodiment. In the 1st Embodiment, a plurality of practice phrases Z are stored in the storage device 22. In the 2nd Embodiment, instead of the plurality of practice phrases Z of the 1st Embodiment, one reference phrase Zref is stored in the storage device 22.

[0088] The reference phrase Zref, like the practice phrase Z of the 1st Embodiment, is time series data representing a musical piece composed of a plurality of notes. Specifically, the reference phrase Zref is a melody (e.g., a part or all of a practice piece) suitable for practice of the electronic musical instrument 10. The practice phrase determination section 73 of the 2nd Embodiment generates a practice phrase Z by editing the reference phrase Zref in correspondence with the tendency data D generated by the tendency determination section 72. Specifically, the practice phrase determination section 73 edits the reference phrase Zref in a manner that reduces the difficulty of performance with respect to a portion of the reference phrase Zref associated with the performance tendency designated by the tendency data D.

[0089] Figure 13 is a flowchart exemplifying the detailed flow of the determination processing Sa of the 2nd Embodiment. The determination processing Sa of the 2nd Embodiment is processing that replaces the step Sa3 of the determination processing Sa of the 1st Embodiment with the step Sa13.

[0090] The acquisition of the musical piece data X and the performance data Y by the performance data acquisition section 71 (Sa1) and the generation of the tendency data D by the tendency determination section 72 (Sa2) are the same as in the 1st Embodiment. The practice phrase determination section 73 of the 2nd Embodiment generates a practice phrase Z by editing the reference phrase Zref stored in the storage device 22 in correspondence with the tendency data D (Sa13). The sending of the practice phrase Z to the electronic musical instrument 10 by the practice phrase determination section 73 (Sa4) is the same as in the 1st Embodiment. The detailed example of the editing of the reference phrase Zref (Sa13) will be described below.

[0091] For example, in a case where the tendency data D indicates a playing tendency of "not good at playing a chord", the practice piece determining section 73 generates the practice piece Z by changing one or more chords included in the reference piece Zref. For example, the practice piece determining section 73 omits one or more constituent notes other than the root note, for a chord including a number of constituent notes exceeding a prescribed number. In addition, the practice piece determining section 73 omits a prescribed number of constituent notes including the highest note, for a chord in which the pitch difference between the lowest note and the highest note exceeds a prescribed value. The difficulty of playing the chord is reduced by the omission of the constituent notes. As exemplified above, the editing of the reference piece Zref by the practice piece determining section 73 includes the change of the chord.

[0092] In addition, in a case where the tendency data D indicates a playing tendency of "not good at jumping", the practice piece determining section 73 generates the practice piece Z by omitting or changing a jump included in the reference piece Zref. For example, the practice piece determining section 73 omits the latter note of two notes involved in the jump. In addition, the practice piece determining section 73 changes the latter note of two notes involved in the jump to another note on the lower side. As exemplified above, the editing of the reference piece Zref by the practice piece determining section 73 includes the omission or change of the jump.

[0093] The reference piece Zref includes, for example, a designation of a playing method such as fingering. Specifically, the practice piece Z includes a designation of the number of the finger that should play each of the notes. In a case where the tendency data D indicates a playing tendency of "not good at finger dexterity", the practice piece determining section 73 generates the practice piece Z by changing the fingering related to the reference piece Zref. For example, if it is assumed that playing a key with the little finger is more difficult for a beginner at playing, the practice piece determining section 73 changes the number of the little finger designated for a note in the reference piece Zref to the number of another finger other than the little finger. In the electronic musical instrument 10 that receives the edited practice piece Z, the fingering (the number of the finger for each note) changed by the practice piece determining section 73 is displayed together with the musical score of the practice piece Z on the display device 15. As exemplified above, the editing of the reference piece Zref by the practice piece determining section 73 includes the change of the playing method of the musical instrument.

[0094] In the second embodiment, the practice piece Z is generated by the editing of the reference piece Zref, and thus it is possible to provide the user U with an appropriate practice piece Z corresponding to the level of the playing skill of the user U.

[0095] C: Third Embodiment

[0096] Figure 14 is a block diagram illustrating a functional configuration of the information processing system 20 of the third embodiment. In the first embodiment, the exercise piece determination section 73 determines the exercise piece Z corresponding to the tendency data D of the user U among the plurality of exercise pieces Z stored in the storage device 22. The exercise piece determination section 73 of the third embodiment determines the exercise piece Z corresponding to the tendency data D using the trained model Mb. The trained model Mb is an example of the "second trained model".

[0097] As understood from the explanation according to the first embodiment, there is a correlation between the performance tendency of the player (the tendency data D) and the exercise piece Z suitable for the performance tendency. For example, the exercise piece Z corresponding to each tendency data D is a piece of music appropriate to improve the performance tendency designated by the tendency data D. The trained model Mb is a statistical estimation model that learns (trains) the relationship between the tendency data D and the exercise piece Z. The exercise piece determination section 73 of the third embodiment determines the exercise piece Z corresponding to the performance tendency indicated by the tendency data D generated by the tendency determination section 72 by inputting the tendency data D to the trained model Mb. For example, the trained model Mb outputs an index of the appropriateness of the tendency data D (i.e., the degree to which each exercise piece Z is appropriate for the performance tendency of the user U) for each of a plurality of different exercise pieces Z. The exercise piece determination section 73 determines the exercise piece Z having the largest index among the plurality of exercise pieces Z stored in the storage device 22.

[0098] The trained model Mb is constituted, for example, by a deep neural network. For example, any form of neural network such as a recurrent neural network or a convolutional neural network is used as the trained model Mb. The trained model Mb can be constituted by a combination of a plurality of deep neural networks. In addition, an additional element such as a long short-term memory (LSTM) can be mounted on the trained model Mb.

[0099] The trained model Mb is realized by causing the control device 21 to execute a program that estimates the exercise piece Z according to the tendency data D and a combination of a plurality of variables (specifically, weighting values and biases) applied to the operation. The program and the plurality of variables that realize the trained model Mb are stored in the storage device 22. The values of the respective plurality of variables of the trained model Mb are set in advance by machine learning.

[0100] Figure 15 is a flowchart illustrating a specific flow of the determination processing Sa of the third embodiment. The determination processing Sa of the third embodiment is processing in which Sa3 of the determination processing Sa of the first embodiment is replaced with step Sa23.

[0101] The acquisition of the piece data X and the performance data Y by the performance data acquisition section 71 (Sa1) and the generation of the tendency data D by the tendency determination section 72 (Sa2) are the same as in the first embodiment. The practice passage determination section 73 of the third embodiment determines the practice passage Z by inputting the tendency data D to the trained model Mb (Sa23). The practice passage determination section 73 transmits the practice passage Z to the electronic musical instrument 10 (Sa4) is the same as in the first embodiment.

[0102] The trained model Mb exemplified above is generated by the machine learning system 30. Figure 16 is a block diagram exemplifying the functional structure of the machine learning system 30 in relation to the generation of the trained model Mb. The control device 31 functions as a plurality of elements (a learning data acquisition section 81b and a learning processing section 82b) for creating the trained model Mb by machine learning by executing a program stored in the storage device 32.

[0103] The learning processing section 82b creates the trained model Mb by teacher-aided machine learning (learning processing Sd described later) using a plurality of learning data Tb. The learning data acquisition section 81b acquires the plurality of learning data Tb. Specifically, the learning data acquisition section 81b acquires the plurality of learning data Tb saved in the storage device 32 from the storage device 32. The learning data acquisition section 81b is an example of a "second learning data acquisition section", and the learning processing section 82b is an example of a "second learning processing section". In addition, the learning data Tb is an example of "second learning data".

[0104] The plurality of learning data Tb is each constituted by a combination of tendency data Dt for learning and practice passage Zt for learning. The practice passage Zt of each learning data Tb is a piece suitable for the performance tendency indicated by the tendency data Dt of the learning data Tb. The combination of the tendency data Dt and the practice passage Zt is selected by the creator of the learning data T, for example. The tendency data Dt is an example of "tendency data for learning", and the practice passage Zt is an example of "practice passage for learning".

[0105] Figure 17 is a flowchart exemplifying the detailed flow of the learning processing Sd by which the control device 31 creates the trained model Mb. The learning processing Sd also represents a method (a trained model generation method) of generating the trained model Mb by machine learning.

[0106] If the learning processing Sd is started, the learning data acquisition section 81b selects any of the plurality of learning data Tb stored in the storage device 32 (hereinafter referred to as "selected learning data Tb") (Sd1). The learning processing section 82b determines the practice passage Zt for learning by inputting the tendency data Dt of the selected learning data Tb to the trained model Mb (Sd2). The learning processing section 82b determines the practice passage Zt for learning by inputting the tendency data Dt of the selected learning data Tb to the trained model Mb (Sd2). The learning processing section 82b determines the practice passage Zt for learning by inputting the tendency data Dt of the selected learning data Tb to the trained model Mb (Sd2). The learning processing section 82b determines the practice passage Zt for learning by inputting the tendency data Dt of the selected learning data Tb to the trained model Mb (Sd2). Figure 16As exemplified, the tendency data Dt of the selection learning data Tb is input to an initial or provisional model (hereinafter, referred to as "provisional model Mb0") (Sd2), and the exercise phrase Z estimated by the provisional model Mb0 for the input is acquired (Sd3).

[0107] The learning processing section 82b calculates a loss function representing an error between the exercise phrase Z estimated by the provisional model Mb0 and the exercise phrase Zt of the selection learning data Tb (Sd4). The learning processing section 82b updates a plurality of variables of the provisional model Mb0 so that the loss function is reduced (ideally, minimized) (Sd5). For the update of the plurality of variables corresponding to the loss function, for example, an error backpropagation method is used.

[0108] The learning processing section 82b determines whether a prescribed end condition is satisfied (Sd6). In the case where the end condition is not satisfied (Sd6: NO), the learning processing section 82b selects the unselected learning data Tb as new selection learning data Tb (Sd1). That is, the processing of updating the plurality of variables of the provisional model Mb0 (Sd2-Sd5) is repeated until the end condition is satisfied (Sd6: YES). The provisional model Mb0 at the point of time when the end condition is satisfied (Sd6: YES) is determined as the trained model Mb.

[0109] As understood from the above description, the trained model Mb estimates a statistically reasonable exercise phrase Z for unknown tendency data D based on a potential relationship between the tendency data Dt and the exercise phrase Zt of a plurality of learning data Tb. That is, the trained model Mb is a statistical estimation model that learns a relationship between the tendency data D and the exercise phrase Z. The exercise phrase determination section 73 of the third embodiment determines the exercise phrase Z by inputting the tendency data D to the trained model Mb that learns a relationship between the tendency data Dt and the exercise phrase Zt.

[0110] The learning processing section 82b transmits the trained model Mb created through the above procedure from the communication device 33 to the information processing system 20 (Sd7). The control device 21 of the information processing system 20 saves the trained model Mb received from the machine learning system 30 in the storage device 22.

[0111] In the third embodiment, the same effects as those of the first embodiment are also achieved. In the third embodiment, the exercise phrase Z is determined by inputting the tendency data D output from the tendency determination section 72 to the trained model Mb. Therefore, a statistically reasonable exercise phrase Z can be determined based on a potential relationship between the tendency data Dt for learning and the exercise phrase Zt for learning.

[0112] D: Fourth Embodiment

[0113] Figure 18 is a block diagram illustrating a functional configuration of the electronic musical instrument 10 according to the fourth embodiment. In each of the foregoing embodiments, the information processing system 20 is illustrated as having the structure of the performance data acquisition section 71, the tendency determination section 72, and the practice passage determination section 73. In the fourth embodiment, the electronic musical instrument 10 has the performance data acquisition section 71, the tendency determination section 72, and the practice passage determination section 73. The above elements are realized by the control device 11 executing the program stored in the storage device 12. In addition, the control device 11 also functions as the prompting processing section 74.

[0114] In the storage device 12 of the electronic musical instrument 10, in addition to the plurality of musical piece data X identical to the first embodiment, the trained model Ma and the plurality of practice passages Z are stored. The trained model Ma created by the machine learning system 30 is transferred to the electronic musical instrument 10, and the trained model Ma is saved in the storage device 12. In addition, the plurality of practice passages Z each correspond to different tendency data D.

[0115] The performance data acquisition section 71 acquires the performance data Y representing the performance of the musical piece by the user U and the musical piece data X of the musical piece, like the first embodiment. Specifically, the performance data acquisition section 71 generates the performance data Y corresponding to the operation of the performance device 14 by the user U. In addition, the performance data acquisition section 71 acquires the musical piece data X of the musical piece performed by the user U from the storage device 12. The performance data acquisition section 71 generates the control data C including the musical piece data X and the performance data Y.

[0116] The tendency determination section 72 generates the tendency data D representing the performance tendency of the user U corresponding to the control data C, like the first embodiment. Specifically, the tendency determination section 72 determines the tendency data D by inputting the control data C including the musical piece data X and the performance data Y to the trained model Ma.

[0117] The practice passage determination section 73 determines the practice passage Z corresponding to the performance tendency of the user U using the tendency data D determined by the tendency determination section 72, like the first embodiment. Specifically, the practice passage determination section 73 searches for the practice passage Z corresponding to the tendency data D determined by the tendency determination section 72 from among the plurality of practice passages Z stored in the storage device 12.

[0118] The prompting processing section 74 prompts the user U of the practice passage Z determined by the practice passage determination section 73. Specifically, the prompting processing section 74 causes the score of the practice passage Z to be displayed on the display device 15. In addition, the prompting processing section 74 can also cause the playback system 18 to play the performance sound of the practice passage Z.

[0119] As understood from the above description, in the fourth embodiment, the same effects as the first embodiment are also achieved. Furthermore, the structure of the second embodiment in which the practice piece determination section 73 generates the practice piece Z by editing the reference piece Zref, and the structure in which the practice piece determination section 73 determines the practice piece Z using the trained model Mb can also be applied to the fourth embodiment in which the practice piece determination section 73 is mounted on the electronic musical instrument 10.

[0120] E: Fifth Embodiment

[0121] Figure 19 is a block diagram illustrating the structure of the performance system 100 related to the fifth embodiment. The performance system 100 has the electronic musical instrument 10 and the information device 50. The information device 50 is, for example, a device such as a smartphone or a tablet terminal. The information device 50 is connected to the electronic musical instrument 10 in a wired or wireless manner, for example.

[0122] The information device 50 is realized by a computer system having a control device 51 and a storage device 52. The control device 51 is constituted by a single or a plurality of processors that control each element of the information device 50. For example, the control device 51 is constituted by one or more processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC. The storage device 52 is a single or a plurality of memories that store programs executed by the control device 51 and various data used by the control device 51. The storage device 52 is constituted by, for example, a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of a plurality of recording media. Furthermore, a removable type recording medium that is detachable with respect to the information device 50, or a recording medium (for example, cloud storage) that can be written or read by the control device 51 via, for example, the communication network 200 can also be utilized as the storage device 52.

[0123] The control device 51 realizes the performance data acquisition section 71, the tendency determination section 72, and the practice piece determination section 73 by executing the programs stored in the storage device 52. The structures and actions of the performance data acquisition section 71, the tendency determination section 72, and the practice piece determination section 73 are the same as the examples illustrated in the first to fourth embodiments. The practice piece Z determined by the practice piece determination section 73 is transmitted to the electronic musical instrument 10. The control device 11 of the electronic musical instrument 10 causes the score of the practice piece Z to be displayed on the display device 15.

[0124] As understood from the above description, in the fifth embodiment, the same effects as the first to fourth embodiments are also achieved. The information processing system 20 of the first to third embodiments, the electronic musical instrument 10 of the fourth embodiment, and the information device 50 of the fifth embodiment are one example of the “information processing system 20”.

[0125] F: Modified example

[0126] Hereinafter, a mode in which a specific modification is added to each of the above-described modes will be exemplified. A plurality of modes selected arbitrarily from the following examples can be appropriately combined within a range where they do not contradict each other.

[0127] (1) In each of the above-described modes, the tendency data D is generated using one trained model Ma, but the tendency data D can also be generated selectively using a plurality of trained models Ma. For example, a plurality of trained models Ma corresponding to different musical instruments are prepared. The tendency determination section 72 selects a trained model Ma corresponding to the musical instrument played by the user U from among the plurality of trained models Ma, and generates the tendency data D by inputting the control data C to the trained model Ma. The relationship between the content of the performance (the performance data Y) of the user U and the performance tendency (the tendency data D) of the user U differs for each musical instrument. If it is a structure in which a plurality of trained models Ma corresponding to different musical instruments are selectively used, it is possible to generate tendency data D that appropriately represents the performance tendency of the musical instrument actually played by the user U.

[0128] (2) In the third embodiment, the practice passage Z is generated using one trained model Mb, but the practice passage Z can also be generated selectively using a plurality of trained models Mb. For example, a plurality of trained models Mb corresponding to different musical instruments are prepared. The practice passage determination section 73 selects a trained model Mb corresponding to the musical instrument played by the user U from among the plurality of trained models Mb, and generates the practice passage Z by inputting the tendency data D to the trained model Mb.

[0129] (3) Any of the plurality of trained models Ma created by the machine learning system 30 can be selectively transferred to the electronic musical instrument 10 of the fourth embodiment. For example, a trained model Ma corresponding to the musical instrument designated by the user U of the electronic musical instrument 10 from among the plurality of trained models Ma corresponding to different musical instruments is transferred from the machine learning system 30 to the electronic musical instrument 10. Similarly, any of the plurality of trained models Ma created by the machine learning system 30 can be selectively transferred to the information device 50 of the fifth embodiment. In the third embodiment, any of the plurality of trained models Mb created by the machine learning system 30 can be selectively transferred to the information processing system 20.

[0130] (4) In each of the above-described modes, the pointing data P is generated in correspondence with the instruction from the instructor U2, but the pointing data P can also be generated by the control device 11 of the electronic musical instrument 10 in correspondence with the instruction from the practitioner U1. For example, the practitioner U1 instructs the performance tendency (e.g., a performance method that is not good at) and the time point at which the performance tendency can be observed with respect to the own performance. The control device 11 generates the pointing data P in correspondence with the instruction from the user U, and transmits the pointing data P from the communication device 13 to the machine learning system 30.

[0131] (5) In each of the above-described modes, the structure in which the control data C contains the piece data X and the performance data Y is exemplified, but the content of the control data C is not limited to the above exemplification. For example, image data of an image in which the user U playing the electronic musical instrument 10 is captured can be contained in the control data C. For example, image data of both hands of the user U at the time of performance is contained in the control data C. The image data of the image in which the performer is captured is also contained in the control data Ct for learning. According to the above structure, it is possible to determine a suitable practice phrase Z that also reflects the situation of the performance of the user U. In addition, a mode in which the control data C does not contain the piece data X is also envisaged. As understood from the above description, the control data C containing at least the performance data Y is input to the trained model Ma. That is, the tendency determination section 72 generates the tendency data D by inputting the performance data Y to the trained model Ma.

[0132] (6) In the first embodiment, the suitable piece for improving the performance tendency of the user U is exemplified as the practice phrase Z, but as in the second embodiment, the practice phrase determination section 73 can determine a practice phrase Z whose degree of difficulty of performance is low with respect to the portion associated with the performance tendency of the user U.

[0133] (7) The structure of the first embodiment in which an arbitrary one of a plurality of practice phrases Z is selected in correspondence with the tendency data D and the structure of the second embodiment in which the reference phrase Zref is edited in correspondence with the tendency data D can be combined. For example, the practice phrase determination section 73 selects one practice phrase Z corresponding to the tendency data D among a plurality of practice phrases Z stored in the storage device 22 as the reference phrase Zref (Sa3), and generates the practice phrase Z by editing the reference phrase Zref in correspondence with the tendency data D (Sa13). That is, the tendency data D is common to the selection of the practice phrase Z (Sa3) and the editing of the reference phrase Zref (Sa13).

[0134] (8) In the second embodiment, the practice piece Z is generated by the practice piece determination section 73 by editing one reference piece Zref stored in the storage device 22, but it is also possible to generate the practice piece Z using a plurality of reference pieces Zref stored in the storage device 22 selectively. For example, the practice piece Z can be generated by the practice piece generation section using the reference piece Zref of a musical piece selected by the user U of the electronic musical instrument 10 among the plurality of reference pieces Zref stored in the storage device 22.

[0135] (9) In each of the above-described embodiments, the electronic keyboard instrument is exemplified as the electronic musical instrument 10, but the kind of instrument played by the user U is arbitrary. For example, the user can play an electronic string instrument such as an electric guitar. The sound signal (voice data) representing the vibration of the string of the electronic string instrument, or the data in MIDI form generated by the analysis of the sound of the electronic string instrument is utilized as the performance data Y. As the performance tendencies related to the electronic string instrument, tendencies such as "insufficient muting at a position where muting should be performed" "a string other than the string corresponding to the target note is sounded" and the like are exemplified. If a case where the user U plays a wind instrument such as a trumpet or a saxophone is assumed, tendencies such as "the volume of the sound is unstable" "the pitch is inaccurate" and the like are assumed as the performance tendencies represented by the tendency data D. If a case where the user U plays a percussion instrument such as a drum is assumed, tendencies such as "the time point of the strike is shifted" "is not good at continuously striking at a short interval" and the like are assumed as the performance tendencies represented by the tendency data D.

[0136] (10) In each of the above-described embodiments, the deep neural network is exemplified as the trained model Ma, but the trained model Ma is not limited to the deep neural network. For example, a statistical estimation model such as an HMM (Hidden Markov Model) or an SVM (Support Vector Machine) can be utilized as the trained model Ma. The trained model Ma utilizing the SVM is described in detail below.

[0137] For example, an SVM is prepared for each combination of all combinations of 2 performance tendencies selected from a plurality of performance tendencies. For the SVM corresponding to the combination of 2 performance tendencies, a hyperplane in a multi-dimensional space is created by machine learning (learning processing Sc). The hyperplane is a boundary surface that separates the space in which the control data C corresponding to one of the 2 performance tendencies is distributed and the space in which the control data C corresponding to the other performance tendency is distributed. The trained model Ma is constituted by a plurality of SVMs (multi-class SVM) corresponding to combinations of different performance tendencies.

[0138] The tendency determination section 72 inputs the control data C to each of the plurality of SVMs of the trained model Ma. The SVM corresponding to each combination selects either of the two kinds of performance tendencies involved in the combination, depending on whether or not the control data C exists in either of the two spaces separated by the hyperplane. The selection of the performance tendency is performed in each of the plurality of SVMs corresponding to different combinations. The tendency determination section 72 generates tendency data D indicating the performance tendency that has been selected by the plurality of SVMs the greatest number of times among the plurality of performance tendencies.

[0139] As understood from the above examples, regardless of the kind of the trained model Ma, the tendency determination section 72 functions as an element that generates tendency data D indicating the performance tendency of the user U by inputting the control data C to the trained model Ma. Further, in the above description, the trained model Ma is focused on, but the same applies to the trained model Mb of the third embodiment, using, for example, a statistical estimation model such as an HMM or an SVM.

[0140] (11) In each of the above-described embodiments, supervised machine learning using a plurality of learning data T is exemplified as the learning processing Sc, but the trained model Ma can also be created by unsupervised machine learning that does not require learning data T, or reinforcement learning that maximizes a reward. As unsupervised machine learning, for example, machine learning using a publicly known clustering is exemplified. The same applies to the trained model Mb of the third embodiment, which can also be created by unsupervised machine learning or reinforcement learning.

[0141] (12) In each of the above-described embodiments, the machine learning system 30 creates the trained model Ma. However, the function of the machine learning system 30 to create the trained model Ma (the learning data acquisition section 81a and the learning processing section 82a) can be mounted on the information processing system 20 of the first to third embodiments, the electronic musical instrument 10 of the fourth embodiment, or the information device 50 of the fifth embodiment. The same applies to the trained model Mb of the third embodiment. That is, the function of the machine learning system 30 to create the trained model Mb (the learning data acquisition section 81b and the learning processing section 82b) can be mounted on the information processing system 20 of the third embodiment, the electronic musical instrument 10 of the fourth embodiment, or the information device 50 of the fifth embodiment.

[0142] (13) In each of the foregoing modes, for the generation of the tendency data D corresponding to the control data C, the trained model Ma is utilized, but the utilization of the trained model Ma can be omitted. For example, a table in which a plurality of control data C each and a plurality of tendency data D each are associated with each other can be utilized for the generation of the tendency data D. The table in which the correspondence between the control data C and the tendency data D is registered is stored in the storage 22 of the first embodiment, the storage 12 of the fourth embodiment, or the storage 52 of the fifth embodiment, for example. The tendency determination section 72 searches the table for the tendency data D corresponding to the control data C generated by the performance data acquisition section 71.

[0143] (14) In each of the foregoing modes, the trained model Ma that has learned the relationship between the control data C including the piece data X and the performance data Y and the tendency data D is utilized, but the structure and the method for generating the tendency data D from the control data C are not limited to the above example. For example, a reference table in which the tendency data D is associated with different plurality of control data C each can be utilized for the generation of the tendency data D by the tendency determination section 72. The reference table is a data table in which the correspondence between the control data C and the tendency data D is registered, and is stored in the storage 22 (the storage 12 in the fourth embodiment) for example. The tendency determination section 72 searches the reference table for the control data C corresponding to the combination of the piece data X and the performance data Y, and acquires the tendency data D associated with the control data C from among the plurality of tendency data D from the reference table.

[0144] (15) In the third embodiment, the trained model Mb that has learned the relationship between the tendency data D and the practice phrase Z is utilized, but the structure and the method for generating the practice phrase Z from the tendency data D are not limited to the above example. For example, a reference table in which the practice phrase Z is associated with different plurality of tendency data D each can be utilized for the generation of the practice phrase Z by the practice phrase determination section 73. The reference table is a data table in which the correspondence between the tendency data D and the practice phrase Z is registered, and is stored in the storage 22 (the storage 12 in the fourth embodiment) for example. The practice phrase determination section 73 searches the reference table for the practice phrase Z corresponding to the tendency data D, and acquires the practice phrase Z associated with the tendency data D from among the plurality of practice phrase Z from the reference table.

[0145] (16) In each of the above-described modes, the performance data Y indicating the performance by the user U is acquired from the electronic musical instrument 10 by the performance data acquisition section 71, but the method of acquiring the performance data Y by the performance data acquisition section 71 is not limited to the above example. For example, the performance data Y does not need to be acquired by the performance data acquisition section 71 in real time in parallel with the performance to the performance device 14. For example, the performance data acquisition section 71 can acquire the performance data Y from the electronic musical instrument 10, which records the past performance by the user U. That is, whether the performance data Y is acquired by the performance data acquisition section 71 in real time with respect to the performance by the user U is arbitrary in the present application.

[0146] Further, for example, the performance data Y indicating the note string played by the user U does not need to be received by the performance data acquisition section 71 from the electronic musical instrument 10. For example, the performance data acquisition section 71 can receive animation data, which captures the situation of the performance by the user U, by the communication device 23, and generate the performance data Y by analyzing the animation data. That is, with respect to the "acquisition" of the performance data Y by the performance data acquisition section 71, in addition to the process of receiving the performance data Y from the external device such as the electronic musical instrument 10, the process of generating the performance data Y from information such as the animation data is also included.

[0147] (17) In each of the above-described modes, the performance data Y0 indicating the performance of the musical piece by the trainee U1 and the criticism data P indicating the criticism by the instructor U2 are acquired by the learning data acquisition section 81a, but the method of acquiring the learning data Ta by the learning data acquisition section 81a is not limited to the above example. For example, the performance data Y0 and the criticism data P (further, the learning data Ta) do not need to be acquired by the learning data acquisition section 81a in parallel with the performance by the trainee U1 and the guidance by the instructor U2. For example, the performance data Y0, which records the past performance by the trainee U1, and the criticism data P, which records the past guidance by the instructor U2, can be acquired by the learning data acquisition section 81a. That is, whether the performance data Y0 and the criticism data P are acquired by the learning data acquisition section 81a in real time with respect to the performance by the trainee U1 and the guidance by the instructor U2 is arbitrary in the present application.

[0148] Further, for example, the performance data Y0 indicating the note string played by the trainee U1 does not need to be received by the learning data acquisition section 81a from the electronic musical instrument 10. For example, the learning data acquisition section 81a can receive animation data, which captures the situation of the performance by the trainee U1, by the communication device 23, and generate the performance data Y0 by analyzing the animation data. That is, with respect to the "acquisition" of the performance data Y0 by the learning data acquisition section 81a, in addition to the process of receiving the performance data Y0 from the external device such as the electronic musical instrument 10, the process of generating the performance data Y0 from information such as the animation data is also included.

[0149] Likewise, it is not necessary for the learning data acquisition section 81a to receive the pointing-out data P indicating the pointing-out by the instructor U2 from the information device 40. For example, the learning data acquisition section 81a can receive animation data that captures a situation of the instruction by the instructor U2 through the communication device 23, and generate the pointing-out data P by analyzing the animation data. That is, for the "acquisition" of the pointing-out data P by the learning data acquisition section 81a, in addition to the process of receiving the pointing-out data P from the external device such as the information device 40, the process of generating the pointing-out data P from information such as the animation data is also included.

[0150] (18) In the above-described each aspect, the portion within the specific interval of the time point designated by the learning data acquisition section 81a from the time-series data Y0 of the performance transmitted from the electronic musical instrument 10 that contains the time point data τ of the pointing-out data P is extracted as the performance data Yt, but the performance data Yt for learning can also be transmitted from the electronic musical instrument 10 to the machine learning system 30. For example, the control device 11 of the electronic musical instrument 10 receives the pointing-out data P from the information device 40, and transmits the portion within the specific interval corresponding to the time point data τ of the pointing-out data P from the performance data Y0 as the performance data Yt from the communication device 13 to the machine learning system 30. The learning data acquisition section 81a receives the performance data Yt transmitted from the electronic musical instrument 10 through the communication device 33. According to the above structure, the machine learning system 30 does not need to acquire the time point data τ from the information device 40. That is, the time point data τ can be omitted with respect to the pointing-out data P transmitted from the information device 40 to the machine learning system 30.

[0151] Further, in the above description, the performance data Yt is focused on, but with respect to the musical piece data Xt for learning, likewise, it can be transmitted from the electronic musical instrument 10 to the machine learning system 30. For example, the control device 11 of the electronic musical instrument 10 transmits the portion within the specific interval corresponding to the time point data τ of the pointing-out data P from the musical piece data X0 as the musical piece data Xt from the communication device 13 to the machine learning system 30. The learning data acquisition section 81a receives the musical piece data Xt transmitted from the electronic musical instrument 10 through the communication device 33.

[0152] (19) The functions (the performance data acquisition unit 71, the tendency determination unit 72, and the practice passage determination unit 73) illustrated in the above-described embodiments are realized by the cooperative operation of a single or a plurality of processors constituting a control device and a program stored in a storage device, as described above. The above program can be provided in a manner of being stored in a computer-readable recording medium and installed in a computer. The recording medium is, for example, a non-transitory recording medium, and is preferably an optical recording medium (optical disc) such as a CD-ROM, and also includes a semiconductor recording medium or a magnetic recording medium, or any known recording medium of arbitrary form. In addition, as the non-transitory recording medium, any recording medium other than a transitory, propagating signal is included, and a volatile recording medium can also be excluded. In addition, in a configuration in which a transmission device transmits a program via a communication network 200, a recording medium in which the program is stored in the transmission device corresponds to the above-described non-transitory recording medium.

[0153] G: Appendix

[0154] According to the above-described embodiments, for example, the following configuration is grasped.

[0155] One embodiment (Embodiment 1) relates to an information processing system including: a performance data acquisition unit that acquires performance data representing performance of a musical piece by a user; a tendency determination unit that generates tendency data representing a tendency of performance by the user by inputting the performance data acquired by the performance data acquisition unit to a first trained model that has learned (trained) a relationship between learning performance data representing performance of a reference musical piece and learning tendency data representing a tendency of performance represented by the learning performance data; and a practice passage determination unit that determines a practice passage corresponding to the tendency data generated by the tendency determination unit. According to the above-described embodiment, by inputting performance data representing performance of a musical piece by a user to a first trained model, tendency data representing a tendency of performance by the user is generated, and a practice passage corresponding to the tendency of performance by the user is determined in correspondence with the tendency data. Therefore, by performance of the practice passage, effective practice corresponding to the tendency of performance by the user is realized.

[0156] The "performance data" is arbitrary data representing performance by a user. For example, music data (e.g., MIDI data) representing a time series of notes played by a user, sound data representing a performance sound emitted from a musical instrument by performance by a user, or the like is exemplified as performance data. In addition, animation data in which a situation of performance by a user is captured can be included in the performance data.

[0157] "tendency data" is arbitrary data indicating a tendency of the performance of the user. The "tendency of the performance" is, for example, a tendency of the performance error of the user or a tendency of the unskilled performance method. The tendency data designates, for example, any of a plurality of tendencies related to the performance error or the performance method.

[0158] "exercise phrase" is a note string (melody) for practicing the performance by the user. The "exercise phrase corresponding to the tendency of the performance of the user" is, for example, a note string appropriate for overcoming the performance error or the unskilled performance method of the user that tends to occur when the user performs. The exercise phrase can be the entire piece of music or a part of the piece of music.

[0159] In the specific example of Mode 1 (Mode 2), the first trained model is a model that has learned a relationship between learning control data including learning piece data indicating a musical score of the reference piece of music and the learning performance data, and the learning tendency data, and the tendency determination section generates the tendency data by inputting control data including the performance data and the piece data indicating the musical score of the piece of music to the first trained model. According to the above mode, the piece data is included in the control data in addition to the performance data, and thus appropriate tendency data reflecting a relationship (for example, difference) between the performance data and the piece data can be generated.

[0160] In the specific example of Mode 1 or 2 (Mode 3), the exercise phrase determination section selects an exercise phrase corresponding to the tendency indicated by the tendency data from among a plurality of exercise phrases corresponding to different tendencies of the performance. According to the above mode, the exercise phrase corresponding to the tendency of the performance of the user is selected from among a plurality of exercise phrases, and thus the load of the exercise phrase determination section on the process of determining the exercise phrase can be reduced.

[0161] In the specific example of Mode 1 or 2 (Mode 4), the exercise phrase determination section generates the exercise phrase by editing a reference phrase corresponding to the tendency indicated by the tendency data. According to the above mode, the exercise phrase is generated by editing the reference phrase, and thus an appropriate exercise phrase corresponding to the level of the performance skill of the user can be provided to the user.

[0162] "editing of the reference phrase" refers to a process of changing the reference phrase so that the difficulty of the performance corresponds to the tendency indicated by the tendency data. For example, as the "editing", simplification of the chord within the reference phrase (for example, omission of the constituent notes of the chord), omission of the skip run (a part in which two notes with a large pitch difference are performed in succession), or simplification of the finger movement during the performance, and the like are exemplified.

[0163] In a specific example (way 5) of the way 4, the reference phrase includes a time series of chords, and the editing of the reference phrase includes a change of the chords. In another specific example (way 6) of the way 4, the reference phrase includes a jump progression in which a pitch difference exceeds a prescribed value, and the editing of the reference phrase includes omission or change of the jump progression. In addition, in another specific example (way 7) of the way 4, the reference phrase includes a specification of a performance method of an instrument, and the editing of the reference phrase includes a change of the performance method. The "performance method" refers to a method of performance of an instrument. For example, as the "performance method", fingering of a keyboard instrument or a string instrument, special playing such as hammering, pulling, or cutting of a string instrument such as a guitar or a bass, and the like are exemplified.

[0164] In a specific example (way 8) of any of the ways 1 to 4, the practice phrase determination section determines the practice phrase by inputting the tendency data output by the tendency determination section to a second trained model that has learned a relationship between learning use tendency data indicating a tendency of performance and a learning use practice phrase corresponding to the tendency indicated by the learning use tendency data. According to the above way, the practice phrase determination section determines the practice phrase by inputting the tendency data output by the tendency determination section to the second trained model. Therefore, it is possible to determine a statistically reasonable practice phrase based on a latent relationship between the learning use tendency data and the learning use practice phrase.

[0165] In a specific example (way 9) of the way 8, the practice phrase determination section determines the practice phrase using selectively any of a plurality of second trained models corresponding to different instruments. According to the above way, compared to a structure in which only one second trained model is used, it is possible to determine a practice phrase appropriate for an instrument actually played by the user.

[0166] In a specific example (way 10) of any of the ways 1 to 9, the tendency determination section generates the tendency data using selectively any of a plurality of first trained models corresponding to different instruments. According to the above way, the plurality of first trained models corresponding to different instruments are used selectively in the generation of the tendency data, and therefore, compared to a structure in which only one first trained model is used, it is possible to generate tendency data that appropriately indicates a tendency of performance of an instrument actually played by the user.

[0167] One embodiment (Mode 11) of the present application relates to an electronic musical instrument including a performance accepting section that accepts performance of a musical piece by a user, a performance data obtaining section that obtains performance data representing the performance accepted by the performance accepting section, a tendency determining section that outputs tendency data representing a tendency of performance by the user from a first trained model by inputting the performance data obtained by the performance data obtaining section to the first trained model that has learned a relationship between learning performance data representing performance of a musical piece and learning tendency data representing a tendency of performance represented by the learning performance data, a practice phrase determining section that determines a practice phrase corresponding to the tendency of performance by the user using the tendency data output by the tendency determining section, and a prompting processing section that prompts the practice phrase to the user.

[0168] The prompting processing section prompts the practice phrase to the user in a manner that the user can visually or aurally perceive. For example, as the prompting processing section, an element that causes a score of the practice phrase to be displayed on a display device, or an element that causes a sound output device to output a performance sound of the practice phrase is exemplified.

[0169] One embodiment (Mode 12) of the present application relates to an information processing method that obtains performance data representing performance of a musical piece by a user, generates tendency data representing a tendency of performance by the user by inputting the obtained performance data to a first trained model that has learned a relationship between learning performance data representing performance of a musical piece and learning tendency data representing a tendency of performance represented by the learning performance data, and determines a practice phrase corresponding to the tendency data.

[0170] In a specific example (Mode 13) of Mode 12, in the determination of the practice phrase, a practice phrase corresponding to the tendency represented by the tendency data is selected from among a plurality of practice phrases corresponding to different tendencies of performance. In another specific example (Mode 14) of Mode 12, in the determination of the practice phrase, the practice phrase is generated by editing a reference phrase in correspondence with the tendency represented by the tendency data. In another specific example (Mode 15) of Mode 12, in the determination of the practice phrase, the practice phrase is determined by inputting the tendency data to a second trained model that has learned a relationship between learning tendency data representing a tendency of performance and learning practice phrase data corresponding to the tendency represented by the learning tendency data.

[0171] One embodiment (Embodiment 16) of the present application relates to a machine learning system including a first learning data acquisition unit configured to acquire first learning data including learning performance data representing performance of a musical piece by a user and learning tendency data representing tendencies of the performance represented by the critique data; and a first learning processing unit configured to create a first trained model that has learned a relationship between the learning performance data and the learning tendency data by machine learning using the first learning data. According to the above embodiment, statistically reasonable tendency data can be generated for performance data by the first trained model based on a potential relationship between the learning performance data and the learning tendency data.

[0172] In a specific example (Embodiment 17) of Embodiment 16, the first learning data acquisition unit acquires performance data representing performance of the musical piece by the user and critique data representing a time point in the musical piece and tendencies of the performance at the time point, generates the first learning data including the learning performance data representing performance in an interval including the time point represented by the critique data among the performance data and the learning tendency data representing tendencies of the performance represented by the critique data. According to the above embodiment, an interval corresponding to the time point represented by the critique data in the performance by the user does not need to be extracted in a supply source of the performance data (e.g., the first device).

[0173] In a specific example (Embodiment 18) of Embodiment 17, the first learning data acquisition unit acquires the performance data from a first device and acquires the critique data from a second device different from the first device. According to the above embodiment, data (performance data and critique data) acquired from the first device and the second device, which are remote from each other, for example, can be used to prepare data for machine learning. The first device is a terminal device used by a practice player who practices performance of a musical instrument, for example, and the second device is a terminal device used by an instructor who evaluates and instructs performance of the practice player, for example.

[0174] In a specific example (Embodiment 19) of any of Embodiments 16 to 18, the first trained model is a model that has learned a relationship between learning control data including learning musical piece data representing a musical score of the reference musical piece and the learning performance data and the learning tendency data. In the above embodiment, the learning musical piece data is included in the learning control data in addition to the learning performance data, and thus the first trained model that can generate appropriate tendency data reflecting a relationship (e.g., similarities and differences) between the learning performance data and the learning musical piece data can be created.

[0175] In a specific example of any of modes 16 to 19 (mode 20), there are further provided a second learning data acquisition section that acquires a plurality of second learning data including learning tendency data that indicates a tendency of performance and a learning exercise phrase corresponding to the tendency indicated by the learning tendency data, and a second learning processing section that creates a second trained model that has learned a relationship between the learning tendency data and the learning exercise phrase of the second learning data by machine learning using the plurality of second learning data.

[0176] One embodiment (mode 21) of the present application relates to a machine learning method that acquires performance data that indicates performance of a musical piece by a user, and critique data that indicates a time point within the musical piece and a tendency of performance at the time point, and creates a first trained model that has learned a relationship between learning performance data that indicates performance within an interval including the time point indicated by the critique data among the performance data, and learning tendency data that indicates a tendency of performance indicated by the critique data, by machine learning using first learning data including the learning performance data and the learning tendency data.

[0177] Explanation of reference numerals

[0178] 100…performance system, 10…electronic musical instrument, 11, 21, 31, 41, 51…control device, 12, 22, 32, 42, 52…storage device, 13, 23, 33, 43…communication device, 14…performance device, 15, 45…display device, 16…sound source device, 17…sound playback device, 18, 46…playback system, 20…information processing system, 30…machine learning system, 40…information device, 44…operation device, 50…information device, 71…performance data acquisition section, 72…tendency determination section, 73…exercise phrase determination section, 74…prompt processing section, 81a, 81b…learning data acquisition section, 82a, 82b…learning processing section.

Claims

1. An information processing system having: a performance data acquisition section that acquires performance data that indicates performance of a musical piece by a user; a tendency determination section that generates tendency data that indicates a tendency of performance by the user by inputting the performance data acquired by the performance data acquisition section to a first trained model that has learned a relationship between learning performance data that indicates performance of a reference musical piece and learning tendency data that indicates a tendency of performance indicated by the learning performance data; and an exercise phrase determination section that generates an exercise phrase by editing a reference phrase in correspondence with the tendency indicated by the tendency data generated by the tendency determination section.

2. The information processing system according to claim 1, wherein the first trained model is a model that has learned a relationship between learning control data that includes learning musical piece data that indicates a musical score of the reference musical piece and the learning performance data, and the learning tendency data, and the tendency determination section generates the tendency data by inputting control data that includes the performance data and musical piece data that indicates a musical score of the musical piece to the first trained model.

3. The information processing system according to claim 1 or 2, wherein the reference phrase includes a time series of chords, and the editing of the reference phrase includes a change of the chords.

4. The information processing system according to claim 1 or 2, wherein the reference phrase includes a jump progression in which a pitch difference exceeds a prescribed value, and the editing of the reference phrase includes omission or a change of the jump progression.

5. The information processing system according to claim 1 or 2, wherein the reference phrase includes a designation of a performance method of an instrument, and the editing of the reference phrase includes a change of the performance method.

6. The information processing system according to claim 1 or 2, wherein the tendency determination section selectively uses any of a plurality of first trained models corresponding to different instruments to generate the tendency data.

7. An information processing system having: a performance data acquisition section that acquires performance data that indicates performance of a musical piece by a user; a tendency determination section that generates tendency data that indicates a tendency of performance by the user by inputting the performance data acquired by the performance data acquisition section to a first trained model that has learned a relationship between learning performance data that indicates performance of a reference musical piece and learning tendency data that indicates a tendency of performance indicated by the learning performance data; and an exercise phrase determination section that determines the exercise phrase by inputting the tendency data generated by the tendency determination section to a second trained model that has learned a relationship between learning tendency data that indicates a tendency of performance and learning exercise phrase that corresponds to the tendency indicated by the learning tendency data.

8. The information processing system according to claim 7, wherein ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The exercise phrase determination section determines the exercise phrase using any of a plurality of second trained models corresponding to different musical instruments.

9. The information processing system according to claim 7 or 8, wherein The tendency determination section generates the tendency data using any of a plurality of first trained models corresponding to different musical instruments.

10. An electronic musical instrument, comprising: a performance accepting section that accepts performance of a musical piece by a user; a performance data obtaining section that obtains performance data representing the performance accepted by the performance accepting section; a tendency determination section that outputs tendency data representing a tendency of performance by the user from a first trained model by inputting the performance data obtained by the performance data obtaining section to the first trained model, the first trained model having learned a relationship between learning performance data representing performance of a musical piece and learning tendency data representing a tendency of performance represented by the learning performance data; an exercise phrase determination section that generates an exercise phrase corresponding to the tendency of performance by the user by editing a reference phrase corresponding to the tendency represented by the tendency data output by the tendency determination section; and a prompting processing section that prompts the exercise phrase to the user.

11. An electronic musical instrument, comprising: a performance accepting section that accepts performance of a musical piece by a user; a performance data obtaining section that obtains performance data representing the performance accepted by the performance accepting section; a tendency determination section that outputs tendency data representing a tendency of performance by the user from a first trained model by inputting the performance data obtained by the performance data obtaining section to the first trained model, the first trained model having learned a relationship between learning performance data representing performance of a musical piece and learning tendency data representing a tendency of performance represented by the learning performance data; an exercise phrase determination section that determines an exercise phrase corresponding to the tendency of performance by the user by inputting the tendency data output by the tendency determination section to a second trained model, the second trained model having learned a relationship between learning tendency data representing a tendency of performance and learning exercise phrase corresponding to the tendency represented by the learning tendency data; and a prompting processing section that prompts the exercise phrase to the user.

12. An information processing method implemented by a computer system, obtaining performance data representing performance of a musical piece by a user, generating tendency data representing a tendency of performance by the user by inputting the obtained performance data to a first trained model, the first trained model having learned a relationship between learning performance data representing performance of a musical piece and learning tendency data representing a tendency of performance represented by the learning performance data, editing a reference phrase corresponding to the tendency represented by the tendency data to generate an exercise phrase.

13. An information processing method implemented by a computer system, play data representing performance of a musical piece by a user, generating tendency data representing a tendency of performance by the user by inputting the obtained play data to a first trained model that has learned a relationship between learning play data representing performance of a musical piece and learning tendency data representing a tendency of performance represented by the learning play data, determining an exercise phrase by inputting the tendency data to a second trained model that has learned a relationship between learning tendency data representing a tendency of performance and learning exercise phrase data corresponding to the tendency represented by the learning tendency data.

14. A machine learning system having: a first learning data obtaining section that obtains first learning data including learning play data representing performance of a musical piece by a user and learning tendency data representing a tendency of the performance, a first learning processing section that creates a first trained model that has learned a relationship between the learning play data and the learning tendency data by machine learning using the first learning data, a second learning data obtaining section that obtains a plurality of second learning data including learning tendency data representing a tendency of performance and learning exercise phrase data corresponding to the tendency represented by the learning tendency data, and a second learning processing section that creates a second trained model that has learned a relationship between the learning tendency data and the learning exercise phrase data of the second learning data by machine learning using the plurality of second learning data.

15. The machine learning system according to claim 14, wherein the first learning data obtaining section obtains play data representing performance of the musical piece by the user and pointing data representing a time point within the musical piece and a tendency of the performance at the time point, the first learning data including the learning play data representing performance within an interval including the time point represented by the pointing data from among the play data and the learning tendency data representing a tendency of performance represented by the pointing data is generated.

16. The machine learning system according to claim 15, wherein the first learning data obtaining section obtains the play data from a first device, the pointing data is obtained from a second device different from the first device.

17. The machine learning system according to any one of claims 14 to 16, wherein the first trained model is a model that has learned a relationship between learning control data including learning musical piece data representing a musical score of the musical piece and the learning play data and the learning tendency data.

Images