Information processing system, electronic musical instrument, information processing method, and method for generating trained model

Abstract

This electronic musical instrument comprises: an acquisition unit that acquires input data including tendency data in which the tendency of playing of a user is reflected; a generation unit that, by inputting the input data acquired by the acquisition unit to a learned model, which has learned a relation between input data for learning and correction data for learning, outputs, from the learned model, correction data for correcting a strength characteristic, which is a relation between playing strength and sound production strength; and a correction unit that sets the strength characteristic by correcting an initial characteristic using the correction data.

Description

[Technical field]

[0001] The present disclosure relates to a technique for controlling the relationship between the intensity with which a user plays an electronic musical instrument and the intensity of musical tones produced by the electronic musical instrument in response to the playing. [Background technology]

[0002] Various techniques have been proposed for controlling the relationship between the strength with which a user plays an instrument and the strength of the musical tones produced by the playing (hereinafter referred to as "strength characteristics"). For example, Patent Document 1 discloses a technique for generating a touch curve from the maximum and average values ​​of key pressing strength. Patent Document 2 discloses a technique for controlling a touch curve according to detection characteristic information specific to a keyboard unit attached to an electronic musical instrument. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-122360 A [Patent Document 2] JP 2008-64983 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, it is actually not easy to determine suitable intensity characteristics for various tendencies of users regarding their playing. In consideration of the above circumstances, an object of one aspect of the present disclosure is to obtain intensity characteristics that appropriately reflect the tendencies of a user's playing. [Means for solving the problem]

[0005] In order to solve the above problems, an information processing system according to one embodiment of the present disclosure includes an acquisition unit that acquires input data including tendency data reflecting a user's performance tendency, a generation unit that generates correction data by inputting the input data acquired by the acquisition unit to a trained model that has learned the relationship between learning input data and learning correction data, and a correction unit that uses the generated correction data to correct a first intensity characteristic, which is the relationship between the performance intensity of the user's performance and the pronunciation intensity of a sound produced in response to the performance.

[0006] An electronic musical instrument according to one embodiment of the present disclosure includes an acquisition unit that acquires input data including tendency data reflecting a user's performance tendency, a generation unit that generates correction data by inputting the input data acquired by the acquisition unit into a trained model that has learned the relationship between learning input data and learning correction data, a correction unit that sets a second intensity characteristic by correcting a first intensity characteristic, which is the relationship between the performance intensity of the user's performance and the pronunciation intensity of a sound produced in response to the performance, using the generated correction data, a performance acceptance unit that accepts a performance by the user, and a playback control unit that causes a playback system to reproduce musical sounds corresponding to the performance accepted by the performance acceptance unit under the second intensity characteristic.

[0007] An information processing method according to one aspect of the present disclosure acquires input data including tendency data reflecting a user's performance tendency, generates correction data by inputting the input data acquired by the acquisition unit into a trained model that has learned the relationship between learning input data and learning correction data, and sets a second intensity characteristic by correcting a first intensity characteristic, which is the relationship between the performance intensity of the user's performance and the pronunciation intensity of a sound produced in response to the performance, using the generated correction data.

[0008] A method for generating a trained model according to one aspect of the present disclosure acquires a plurality of training data representing a combination of learning input data including tendency data reflecting a performer's performance tendency and correction data for correcting intensity characteristics, which are the relationship between the performance intensity of the user's performance and the pronunciation intensity of sounds produced in response to the performance, and establishes a trained model that has learned the relationship between the input data and the correction data in the plurality of training data through machine learning using the plurality of training data. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating an example of the configuration of an electronic musical instrument according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a functional configuration of an electronic musical instrument. [Diagram 3] FIG. [Figure 4] 10 is a flowchart illustrating a specific procedure of a performance process. [Diagram 5] FIG. 4 is a schematic diagram of input data. [Figure 6] 10 is a flowchart illustrating a specific procedure of a setting process. [Figure 7] FIG. 1 is a block diagram illustrating a configuration of a machine learning system. [Figure 8] FIG. 1 is a block diagram illustrating an example of the functional configuration of a machine learning system. [Figure 9] 11 is a flowchart illustrating a specific procedure of a learning process. [Figure 10] FIG. 13 is an explanatory diagram of a correction unit in the third embodiment. [Figure 11] FIG. 13 is an explanatory diagram of a generation unit in the fourth embodiment. [Figure 12] FIG. 13 is a block diagram illustrating the configuration of a performance system in a fifth embodiment. [Figure 13] 13 is a flowchart illustrating a specific procedure of a learning process in the fifth embodiment. [Figure 14]FIG. 13 is a block diagram illustrating the configuration of a performance system in a sixth embodiment. [Figure 15] FIG. 13 is a block diagram illustrating the configuration of a performance system in a seventh embodiment. [Figure 16] FIG. 13 is a side view illustrating the configuration of a performance device in a sixth embodiment. [Figure 17] FIG. 2 is a circuit diagram illustrating the configuration of a detection unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] A: First embodiment 1 is a block diagram illustrating the configuration of an electronic musical instrument 10 according to a first embodiment. The electronic musical instrument 10 is an audio processing system that reproduces musical tones in response to a performance by a user. The electronic musical instrument 10 includes a control device 11, a storage device 12, a communication device 13, an operation device 14, a performance device 15, a sound source device 16, and a sound emission device 17. The electronic musical instrument 10 may be realized as a single device, or may be realized as a plurality of devices configured separately from each other.

[0011] The control device 11 is composed of one or more processors that control each element of the electronic musical instrument 10. For example, the control device 11 is composed of one or more types of processors, such as a CPU (Central Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

[0012] The storage device 12 is one or more memories that store the programs executed by the control device 11 and various data used by the control device 11. The storage device 12 is configured with a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of multiple types of recording media. Note that the storage device 12 may be a portable recording medium that is detachably attached to the electronic musical instrument 10, or a recording medium (e.g., cloud storage) to which the control device 11 can write or read via a communication network 90 such as the Internet.

[0013] The communication device 13 communicates with other devices via a communication network 90. ​​For example, the communication device 13 communicates with a machine learning system 40 described below. It does not matter whether there is a wireless section in the communication line between the communication device 13 and the communication network 90. ​​In addition, a communication device 13 separate from the electronic musical instrument 10 may be connected to the electronic musical instrument 10 by wire or wirelessly. An example of the communication device 13 separate from the electronic musical instrument 10 is an information terminal such as a smartphone or a tablet terminal.

[0014] The operation device 14 is an input device that receives instructions from a user, and is, for example, a plurality of operators operated by the user, or a touch panel that detects contact by the user.

[0015] The performance device 15 is an input device that accepts a performance by a user. Specifically, the performance device 15 is a keyboard on which a plurality of keys 151 corresponding to different pitches are arranged. A user plays a piece of music by sequentially operating desired keys 151 of the performance device 15. The performance device 15 can detect the intensity of the performance by the user (hereinafter referred to as "performance intensity") X. The performance intensity X is the intensity with which the user operates each key 151 (i.e., the key pressing intensity). For example, the performance device 15 is provided with an optical sensor for each key 151 that detects the displacement of the key 151. The displacement speed calculated from the time change of the displacement detected by each optical sensor for each key 151 is used as the performance intensity X for the key 151. Note that the specific form of the performance device 15 is arbitrary and is not limited to a keyboard. The performance device 15 is an example of a "performance acceptance unit" that accepts a performance by a user, and the key 151 is an example of a "performance operator" that the user operates when playing.

[0016] The sound source device 16 generates an audio signal V in response to a performance on the performance device 15. The audio signal V is a sample sequence representing a waveform of a musical tone instructed by a performance on the performance device 15. Specifically, the sound source device 16 generates an audio signal V representing a musical tone of a pitch corresponding to a key 151 pressed by a user among a plurality of keys 151 on the performance device 15. Note that the control device 11 may realize the function of the sound source device 16 by executing a program stored in the storage device 12. In other words, the sound source device 16 dedicated to generating the audio signal V is omitted.

[0017] The sound emitting device 17 emits the musical sound represented by the audio signal V. For example, a speaker or a headphone is used as the sound emitting device 17. As can be understood from the above description, the sound source device 16 and the sound emitting device 17 in the first embodiment function as a reproduction system 18 that reproduces musical sounds according to the performance by the user.

[0018] 2 is a block diagram illustrating an example of the functional configuration of the electronic musical instrument 10. The control device 11 of the electronic musical instrument 10 executes a program stored in the storage device 12 to function as a property setting section 30 and a playback control section 34.

[0019] The characteristic setting unit 30 sets the intensity characteristic Q. As illustrated in Fig. 3, the intensity characteristic Q is the relationship between the performance intensity X for the performance device 15 and the intensity Y of the musical tone (hereinafter referred to as "sound intensity") that the reproduction system 18 generates in response to the performance. In other words, the intensity characteristic Q is a touch curve (or a velocity curve) that represents the relationship between the performance intensity X and the sound intensity Y. Roughly speaking, the intensity characteristic Q defines the relationship between the performance intensity X and the sound intensity Y such that the sound intensity Y becomes a larger value as the performance intensity X increases. The intensity characteristic Q set by the characteristic setting unit 30 is stored in the storage device 12.

[0020] The playback control unit 34 in FIG. 2 controls the playback of musical tones by the playback system 18. Specifically, the playback control unit 34 outputs a sound generation instruction E to the playback system 18 (specifically, the sound source device 16) in response to an operation on the performance device 15. The sound generation instruction E is event data including a designation of a pitch N and a sound generation intensity Y. The pitch N is a pitch number corresponding to the key 151 operated by the user. The sound generation intensity Y is set to a value corresponding to the performance intensity X under the relationship defined by the intensity characteristic Q. As can be understood from the above explanation, the playback control unit 34 controls the playback of musical tones by the playback system 18 by utilizing the intensity characteristic Q set by the characteristic setting unit 30. That is, the playback control unit 34 causes the playback system 18 to play musical tones corresponding to the performance on the performance device 15 under the intensity characteristic Q.

[0021] 4 is a flowchart illustrating a specific procedure of a process (hereinafter referred to as a "performance process") Sa in which the control device 11 (the reproduction control unit 34) controls the reproduction of musical tones by the reproduction system 18. For example, the performance process Sa is started in response to an instruction from the user via the operation device 14.

[0022] When the performance process Sa is started, the playback control unit 34 waits for the user to operate the performance device 15 (Sa1: NO). When the user operates the performance device 15 (Sa1: YES), the playback control unit 34 acquires the performance intensity X of the key 151 by the user from the performance device 15 (Sa2). The playback control unit 34 specifies the sound intensity Y corresponding to the current performance intensity X under the intensity characteristic Q (Sa3). Then, the playback control unit 34 transmits a sound intensity instruction E including the pitch N corresponding to the key 151 operated by the user and the sound intensity Y specified from the performance intensity X to the sound source device 16 (Sa4). The above process is repeated until the user operates the operation device 14 to instruct the end of the performance (Sa5: NO). When the user instructs the end of the performance (Sa5: YES), the performance process Sa ends.

[0023] The characteristic setting unit 30 of the first embodiment sets the intensity characteristic Q according to the performance tendency of the user. That is, the intensity characteristic Q is set individually for each user according to the performance tendency unique to each user. The user plays a predetermined piece of music (hereinafter referred to as a "reference music piece") using the performance device 15 before the intensity characteristic Q is set. The performance tendency of the user is identified by analyzing the performance of the reference music piece by the user. That is, the reference music piece is a known piece of music for observing the performance tendency of the user. The characteristic setting unit 30 generates correction data C from input data D according to the performance tendency, and corrects an initial intensity characteristic (hereinafter referred to as an "initial characteristic") R prepared in advance according to the correction data C to set the intensity characteristic Q. As illustrated in FIG. 3, the initial characteristic R is, for example, a linear relationship (Y=X) between the performance intensity X and the sound intensity Y, and is stored in advance in the storage device 12. As illustrated in FIG. 2, the characteristic setting unit 30 of the first embodiment includes an acquisition unit 31, a generation unit 32, and a correction unit 33.

[0024] The acquisition unit 31 acquires the input data D. FIG. 5 is a schematic diagram of the input data D. The input data D in the first embodiment includes music data d1, performance data d2, and tendency data d3. The music data d1 is data representing a part or all of the musical score of the reference music. Specifically, the music data d1 specifies the pitch, sounding period, sounding intensity, and finger number of each of the multiple notes constituting the reference music. The finger number is the number of the finger that the user should use to press the key when playing each note. For example, for each of the user's right and left hands, the finger number of any of the multiple fingers including the thumb, index finger, middle finger, ring finger, and little finger is specified for each note. The performance data d2 represents the performance of the reference music by the user. Specifically, the performance data d2 specifies the pitch and sounding period of each of the multiple notes that the user plays in sequence by operating the performance device 15. The music data d1 and the performance data d2 are time-series data in a format conforming to the MIDI (Musical Instrument Digital Interface) standard, for example. The sounding periods in the music data d1 and the performance data d2 are expressed, for example, by a combination of the start and end points of the sounding periods, or a combination of the start point and duration. As described above, the music data d1 is data representing an exemplary performance of a reference music piece, and the performance data d2 is data representing the actual performance of each user.

[0025] The tendency data d3 is data that represents the tendency of the user's performance regarding the performance of the reference music piece. Specifically, the tendency data d3 represents the performance characteristics (hereinafter referred to as "performance characteristics") of the user for each of the multiple keys 151 operated when the reference music piece is played. Specifically, the tendency data d3 includes unit data U for each combination of each key 151 of the performance device 15 and a finger (finger number) used by the user to operate the key 151. One unit data U corresponding to a combination of a specific key 151 and a specific finger represents the performance characteristics when the user operates the key 151 with the finger. As described above, the music piece data d1 specifies a finger number for each note of the reference music piece. Unit data U representing the performance characteristics is generated for each combination of the key 151 corresponding to each note of the reference music piece and the finger number specified by the music piece data d1 for the note.

[0026] The performance characteristics represented by each unit data U include, for example, the performance intensity X when the user plays each note of the reference music piece, and the sounding period error τ when the user plays the note. The sounding period error τ is the difference between the exemplary sounding period specified for each note in the music piece data d1 and the sounding period actually played by the user, and is, for example, an error in the start or end point of the sounding period, or an error in the duration.

[0027] If a user is not good at playing a specific key 151 with a specific finger, the performance intensity X of the unit data U corresponding to the combination of the key 151 and the finger is likely to be a small value relative to the tone intensity specified by the music piece data d1. Also, if a user is not good at keeping a specific key 151 pressed with a specific finger (i.e., if the user cannot keep the key pressed for a sufficient tone generation period), the error τ of the unit data U corresponding to the combination of the key 151 and the finger is likely to be a large value. As can be understood from the above explanation, the tendency data d3 is data that reflects the performance tendency of the user. The tendency data d3 can also be said as data that indicates the level of the user's performance technique (i.e., the degree of skill in performance).

[0028] 2 generates correction data C according to input data D acquired by acquisition unit 31. Specifically, the generation unit 32 generates correction data C that corrects initial characteristics R so that intensity characteristics Q that assist the performance tendency indicated by the input data D are set.

[0029] For example, if the user tends to have a lack of performance intensity X when playing the key 151 of the performance device 15 softly, the correction data C is generated so that the sound intensity Y corresponding to the small numerical range of the numerical range of the performance intensity X in the initial characteristic R increases, as shown in Example 2 of FIG. 3. If the user tends to have a lack of performance intensity X when playing the key 151 of the performance device 15 hard, the correction data C is generated so that the sound intensity Y corresponding to the large numerical range of the numerical range of the performance intensity X in the initial characteristic R increases, as shown in Example 3 of FIG. 3. On the other hand, if the user tends to have a tendency not to have a lack of performance intensity X, the correction data C is generated so that the intensity characteristic Q after correction is maintained equal to the initial characteristic R, as shown in Example 1 of FIG. 3. As can be understood from the above examples, the correction data C is data that specifies the correction amount of the sound intensity Y for each numerical value of the performance intensity X (for example, an addition value or a subtraction value to the numerical value of the sound intensity Y in the initial characteristic R).

[0030] The learned model M is used for the generation of the correction data C by the generation unit 32. There is a mutual correlation between the performance of the reference music piece by the user (performance data d2) and the tendency of the performance (tendency data d3), and the intensity characteristic Q (or the correction data C for setting the intensity characteristic Q) that assists the performance by the user. The learned model M is a statistical estimation model that has learned the above correlation. In other words, the learned model M is a model that has learned the relationship between the content of the reference music piece and the tendency of the performance, and the content of the correction to be performed on the initial characteristic R (the relationship between the input data D and the correction data C) by machine learning. The generation unit 32 inputs the input data D to the learned model M, and outputs the correction data C from the learned model M.

[0031] The trained model M is composed of, for example, a deep neural network (DNN). For example, any type of neural network, such as a recurrent neural network (RNN) or a convolutional neural network (CNN), is used as the trained model M. The trained model M may be composed of a combination of multiple types of deep neural networks. In addition, additional elements, such as a long short-term memory (LSTM), may be installed in the trained model M.

[0032] The trained model M is realized by a combination of a program that causes the control device 11 to execute a calculation to generate correction data C from input data D, and a plurality of variables (specifically, weights and biases) that are applied to the calculation. The program and the plurality of variables that realize the trained model M are stored in the storage device 12. The numerical values ​​of each of the plurality of variables that define the trained model M are set in advance by machine learning.

[0033] The correction unit 33 in Fig. 2 sets the intensity characteristic Q by correcting the initial characteristic R using the correction data C generated by the generation unit 32. Specifically, the correction unit 33 generates the intensity characteristic Q by combining the correction data C with the initial characteristic R stored in the storage device 12. For example, the correction unit 33 generates the intensity characteristic Q by adding or multiplying the correction amount specified by the correction data C for each numerical value of the performance intensity X to the numerical value of the sound intensity Y corresponding to the numerical value of the performance intensity X in the initial characteristic R. Note that the initial characteristic R is an example of a "first intensity characteristic," and the intensity characteristic Q is an example of a "second intensity characteristic."

[0034] 6 is a flowchart illustrating a specific procedure of a process (hereinafter, referred to as a "setting process") Sb in which the control device 11 sets the intensity characteristic Q. For example, the setting process Sb is started in response to an instruction from a user via the operation device 14. When the setting process Sb is started, the user plays a reference piece of music using the performance device 15.

[0035] The acquisition unit 31 generates input data D by analyzing the performance of the reference music piece by the user (Sb11). The generation unit 32 generates correction data C by inputting the input data D to the trained model M (Sb12). The correction unit 33 sets the intensity characteristic Q by correcting the initial characteristic R stored in the storage device 12 with the correction data C (Sb13). The correction unit 33 saves the intensity characteristic Q in the storage device 12 (Sb14).

[0036] As described above, in the first embodiment, the intensity characteristic Q that reflects the performance tendency of the user can be set by correcting the initial characteristic R using the correction data C output by the trained model M. For example, the intensity characteristic Q that assists the performance that the user is not good at can be set.

[0037] In particular, in the first embodiment, the tendency data d3 representing the user's performance characteristics for each key 151 (performance intensity X and sounding period error τ) is used, so that it is possible to set the intensity characteristic Q reflecting the user's performance tendency for each key 151. Specifically, the tendency data d3 represents the user's performance characteristics for each combination of key 151 and finger, so that it is possible to generate a suitable intensity characteristic Q reflecting the performance tendency of each finger for each key 151 (for example, a tendency that it is difficult to play a specific key 151 with a specific finger). Also, in the first embodiment, the input data D includes the performance data d2, so that it is possible to set the intensity characteristic Q reflecting the user's performance tendency.

[0038] The machine learning system 40 in Fig. 1 generates the trained model M exemplified above. Fig. 7 is a block diagram illustrating the configuration of the machine learning system 40. The machine learning system 40 includes a control device 41, a storage device 42, and a communication device 43. Note that the machine learning system 40 may be realized as a single device, or may be realized as a plurality of devices configured separately from each other.

[0039] The control device 41 is composed of one or more processors that control each element of the machine learning system 40. The control device 41 is composed of one or more types of processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC. The communication device 43 communicates with the electronic musical instrument 10 via the communication network 90.

[0040] The storage device 42 is a single or multiple memories that store the programs executed by the control device 41 and various data used by the control device 41. The storage device 42 is configured, for example, of a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of multiple types of recording media. In addition, a portable recording medium that is detachable from the machine learning system 40, or a recording medium (for example, cloud storage) to which the control device 41 can write or read via the communication network 90 may be used as the storage device 42.

[0041] 8 is a block diagram illustrating an example of a functional configuration of the machine learning system 40. The control device 41 executes a program stored in the storage device 42 to function as multiple elements (a training data acquisition unit 51 and a learning processing unit 52) ​​for establishing a trained model M by machine learning.

[0042] The learning processing unit 52 establishes a learned model M by supervised machine learning using a plurality of training data T. The training data acquisition unit 51 acquires a plurality of training data T. Specifically, the training data acquisition unit 51 acquires, from the storage device 42, a plurality of training data T stored in the storage device 42.

[0043] Each of the multiple training data T is composed of a combination of training input data Dt and training correction data Ct. The input data Dt is generated from the results of a number of performers playing a reference piece of music. The correction data Ct of each training data T is created in advance by the creator of the training data T as data representing the difference between an intensity characteristic Q suitable for the performance tendency represented by the input data Dt of the training data T and an initial characteristic R prepared in advance. In other words, the correction data Ct of each training data T is data for setting an intensity characteristic Q suitable for the performance tendency represented by the input data Dt of the training data T, and corresponds to a correct value (label) that the trained model M should output for the input data Dt.

[0044] 9 is a flowchart illustrating a specific procedure of a process (hereinafter referred to as a "learning process") Sc in which the control device 41 establishes a trained model M by machine learning. The learning process Sc is also expressed as a method for generating a trained model M (a trained model generation method).

[0045] When the learning process Sc is started, the training data acquisition unit 51 acquires any one of the multiple training data T (hereinafter referred to as "selected training data T") stored in the storage device 42 (Sc1). As illustrated in Fig. 8, the learning processing unit 52 inputs input data Dt of the selected training data T to an initial or provisional model (hereinafter referred to as "provisional model") M0 (Sc2), and acquires correction data C output by the provisional model M0 in response to the input (Sc3).

[0046] The learning processing unit 52 calculates a loss function that represents an error between the corrected data C generated by the provisional model M0 and the corrected data Ct of the selected training data T (Sc4). The learning processing unit 52 updates multiple variables of the provisional model M0 so that the loss function is reduced (ideally minimized) (Sc5). For example, the backpropagation method is used to update the multiple variables according to the loss function.

[0047] The learning processing unit 52 determines whether a predetermined termination condition is satisfied (Sc6). The termination condition is, for example, that the loss function falls below a predetermined threshold, or that the change amount of the loss function falls below a predetermined threshold. If the termination condition is not satisfied (Sc6: NO), the training data acquisition unit 51 selects the unselected training data T as new selected training data T (Sc1). That is, the process (Sc1-Sc5) of updating multiple variables of the provisional model M0 is repeated until the termination condition is satisfied (Sc6: YES). If the termination condition is satisfied (Sc6: YES), the learning processing unit 52 terminates the learning process Sc. The provisional model M0 at the time when the termination condition is satisfied is determined as the trained model M. That is, the multiple variables of the trained model M are determined to be the numerical values ​​at the time when the learning process Sc is terminated.

[0048] As can be understood from the above explanation, the trained model M outputs statistically appropriate correction data C for unknown input data D, based on the underlying relationship between the input data Dt and correction data Ct in multiple training data T. In other words, as described above, the trained model M is a model that has learned, by machine learning, the relationship between the content and performance tendency of the reference music piece, and the content of the correction to be performed on the initial characteristics R (the relationship between the input data D and the correction data C).

[0049] The learning processing unit 52 transmits the trained model M established in the learning process Sc to the electronic musical instrument 10 from the communication device 43. Specifically, the learning processing unit 52 transmits multiple variables of the trained model M from the communication device 43 to the electronic musical instrument 10. The control device 11 of the electronic musical instrument 10 saves the trained model M received from the machine learning system 40 in the storage device 12. Specifically, multiple variables that define the trained model M are stored in the storage device 12.

[0050] B: Second embodiment A second embodiment will be described. Note that, for elements having the same functions as those in the first embodiment in each of the following exemplary aspects, the same reference numerals as those in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.

[0051] In the first embodiment, the control device 41 of the machine learning system 40 executes the learning process Sc. In the second embodiment, the control device 11 of the electronic musical instrument 10 executes the learning process Sc to establish a learned model M. That is, the control device 11 functions as the training data acquisition unit 51 and the learning processing unit 52 illustrated in FIG. 8. The training data acquisition unit 51 acquires multiple pieces of training data T stored in the storage device 12. The learning processing unit 52 establishes a learned model M by supervised machine learning using the multiple pieces of training data T acquired by the training data acquisition unit 51.

[0052] The control device 11 of the electronic musical instrument 10 establishes a learned model M using multiple pieces of training data T pre-stored in the storage device 12 through a learning process Sc similar to that of the first embodiment. In addition to establishing a new learned model M as described above, the control device 11 of the second embodiment updates the learned model M using new training data T (hereinafter referred to as "new training data T") that reflects the user's performance of the electronic musical instrument 10. In other words, the learned model M is re-learned through a learning process Sc that uses multiple pieces of new training data T accumulated through the user's continued use of the electronic musical instrument 10.

[0053] For example, the training data acquisition unit 51 generates input data Dt of new training data T by analyzing the performance of the user on the performance device 15. The training data acquisition unit 51 also generates correction data Ct of the new training data T by analyzing the performance of the user on the performance device 15. For example, the training data acquisition unit 51 selects correction data C corresponding to the actual performance tendency of the user from among a plurality of correction data C prepared in advance for each performance tendency of the reference music piece as correction data Ct (i.e., a correct value) of the new training data T. The training data acquisition unit 51 stores the new training data T including the input data Dt and correction data Ct prepared by the above procedure in the storage device 12. Note that the difference between the strength characteristic Q designated by the user by operating the operation device 14 and the initial characteristic R prepared in advance may be used as the correction data Ct.

[0054] When a sufficient number of new training data T exceeding a predetermined value is accumulated in the storage device 12, the control device 11 of the electronic musical instrument 10 executes a learning process Sc using the multiple new training data T. That is, the control device 11 executes the same learning process Sc as in the first embodiment with the current trained model M as the provisional model M0 of Fig. 8, thereby updating multiple variables of the provisional model M0. Specifically, a process (Sc1) in which the training data acquisition unit 51 selects new training data T and a process (Sc2-Sc5) in which the learning processing unit 52 updates multiple variables of the provisional model M0 are repeated.

[0055] The second embodiment also achieves the same effects as the first embodiment. Moreover, in the second embodiment, multiple variables of the trained model M are updated using new training data T reflecting the performance tendencies of the user (i.e., the trained model M is retrained), which has the advantage that a suitable trained model M reflecting the performance tendencies of each individual user can be established.

[0056] C: Third embodiment 10 is an explanatory diagram of the correction section 33 in the third embodiment. The storage device 12 in the third embodiment stores a plurality of initial characteristics R. The relationship between the performance intensity X and the sound generation intensity Y differs for each initial characteristic R.

[0057] The sound source device 16 of the third embodiment generates an audio signal V representing a performance sound of any one of a plurality of timbres. For example, the sound source device 16 generates an audio signal V representing a performance sound of a timbre (hereinafter referred to as a "selected timbre") selected by a user by operating the operation device 14. The storage device 12 stores a plurality of initial characteristics R corresponding to different timbres that the sound source device 16 can generate. The initial characteristics R corresponding to each timbre are intensity characteristics (relationship between performance intensity X and sound intensity Y) suitable for that timbre.

[0058] The correction unit 33 of the third embodiment sets the intensity characteristic Q by correcting any one of a plurality of initial characteristics R with the correction data C. Specifically, in the setting process Sb, the correction unit 33 selects an initial characteristic R corresponding to a selected timbre from the plurality of initial characteristics R, and generates the intensity characteristic Q by correcting the initial characteristic R with the correction data C (Sb13). The intensity characteristic Q set by the above procedure is applied to the determination of the sound intensity Y in the performance process Sa (Sa3).

[0059] The third embodiment also achieves the same effects as the first embodiment. In the third embodiment, the intensity characteristic Q is set by correcting the initial characteristic R corresponding to the selected tone color among a plurality of initial characteristics R, so that the intensity characteristic Q suitable for the tone color selected by the user can be set. The third embodiment is also applicable to the second embodiment.

[0060] In the above description, multiple initial characteristics R corresponding to different timbres are selectively used, but the musical elements that distinguish each initial characteristic R are not limited to timbre. For example, there is a tendency that the preferred intensity characteristic Q differs depending on the musical genre of the piece of music played by the user. In consideration of the above tendency, a configuration is envisioned in which multiple initial characteristics R corresponding to different musical genres are stored in the storage device 12.

[0061] The initial characteristics R corresponding to each music genre are intensity characteristics (relationship between performance intensity X and sound intensity Y) suitable for playing a piece of music of that music genre. The correction unit 33 selects an initial characteristic R corresponding to the music genre selected by the user from among the multiple initial characteristics R, and sets the intensity characteristic Q by correcting the initial characteristic R with the correction data C. According to the above embodiment, it is possible to set an intensity characteristic Q suitable for the music genre of the piece of music played by the user.

[0062] D: Fourth embodiment FIG. 11 is an explanatory diagram of the generating unit 32 in the fourth embodiment. As in the third embodiment, the sound source device 16 in the fourth embodiment generates an audio signal V representing a performance sound of a selected tone color from among a plurality of types of tone colors. The storage device 12 stores a plurality of trained models M corresponding to different tones that the sound source device 16 can generate. The training process Sc of the trained model M corresponding to each tone color uses training data T including input data Dt indicating the results of a number of performers playing a reference piece of music with that tone color. Therefore, the trained model M corresponding to each tone color is a model for correcting the initial characteristic R to an intensity characteristic Q suitable for that tone color.

[0063] The generation unit 32 of the fourth embodiment generates the correction data C by selectively using one of a plurality of trained models M. Specifically, in the setting process Sb, the generation unit 32 inputs the input data D to a trained model M corresponding to a selected timbre among the plurality of trained models M, and outputs the correction data C from the trained model M (Sb12). The correction data C set by the above procedure is applied to the setting of the intensity characteristic Q (Sb13).

[0064] The fourth embodiment also achieves the same effect as the first embodiment. In addition, in the fourth embodiment, a trained model M corresponding to a selected tone color among a plurality of trained models M is used to generate correction data C, so that an intensity characteristic Q suitable for a tone color selected by a user can be set, as in the third embodiment. The fourth embodiment is also applicable to the second or third embodiment.

[0065] In the above description, multiple trained models M corresponding to different timbres are selectively used, but the musical elements that distinguish each trained model M are not limited to timbre. For example, as described above, the preferred intensity characteristic Q tends to differ for each musical genre of the music played by the user. In consideration of the above tendency, a configuration is assumed in which multiple trained models M corresponding to different musical genres are stored in the storage device 12.

[0066] The trained model M corresponding to each music genre is a model for generating correction data C suitable for playing a piece of music of that music genre. The generation unit 32 selects a trained model M corresponding to a music genre selected by a user from among the multiple trained models M, and generates correction data C by inputting input data D to the trained model M. According to the above embodiment, it is possible to set an intensity characteristic Q suitable for the music genre of a piece of music played by a user.

[0067] E: Fifth embodiment 12 is a block diagram illustrating the configuration of a performance system 100 in the fifth embodiment. The performance system 100 includes an electronic musical instrument 10 and a control system 60. The control system 60 is realized by a computer system including a control device 61, a storage device 62, and a communication device 63.

[0068] The control device 61 is composed of one or more processors that control each element of the control system 60. For example, the control device 61 is composed of one or more types of processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC. The communication device 63 communicates with the electronic musical instrument 10 via the communication network 90.

[0069] The storage device 62 is a single or multiple memories that store the programs executed by the control device 61 and various data used by the control device 61. The storage device 62 is configured with a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of multiple types of recording media. Note that a portable recording medium that is detachable from the control system 60, or a recording medium (e.g., cloud storage) to which the control device 61 can write or read via, for example, the communication network 90 may be used as the storage device 62.

[0070] The control device 61 executes a program stored in the storage device 62 to realize the characteristic setting unit 30 similar to that of the first embodiment. The characteristic setting unit 30 includes an acquisition unit 31, a generation unit 32, and a correction unit 33. The acquisition unit 31 receives input data D generated by the electronic musical instrument 10 via the communication device 63. The generation unit 32 outputs correction data C by inputting the input data D to the trained model M, similar to the first embodiment. The correction unit 33 sets the intensity characteristic Q by correcting the initial characteristic R with the correction data C. The intensity characteristic Q is transmitted from the communication device 63 to the electronic musical instrument 10. The trained model M and the initial characteristic R are stored in the storage device 62. Specifically, the trained model M established by the machine learning system 40 is transferred to the control system 60, and the trained model M is stored in the storage device 62.

[0071] 13 is a flowchart of the setting process Sb executed by the control device 61 of the control system 60. When the setting process Sb is started, the acquisition unit 31 receives the input data D transmitted from the electronic musical instrument 10 via the communication device 63 (Sb21). Note that the generation of the input data D by the acquisition unit 31 in the first embodiment (Sb11) and the reception of the input data D by the acquisition unit 31 in the fifth embodiment (Sb21) are both collectively referred to as "acquisition" of the input data D.

[0072] As in the first embodiment, the generation unit 32 generates correction data C by inputting the input data D to the trained model M (Sb22). The correction unit 33 sets the intensity characteristic Q by correcting the initial characteristic R stored in the storage device 62 with the correction data C (Sb23). The correction unit 33 transmits the intensity characteristic Q from the communication device 63 to the electronic musical instrument 10 (Sb24). The communication device 13 of the electronic musical instrument 10 receives the intensity characteristic Q from the control system 60. The control device 11 of the electronic musical instrument 10 functions as a playback control unit 34 that controls the playback of musical tones by the playback system 18 using the intensity characteristic Q received from the control system 60.

[0073] As can be understood from the above description, the fifth embodiment also achieves the same effects as the first embodiment. The configurations of the second to fourth embodiments are also applied to the fifth embodiment in which the control system 60 executes the setting process Sb. For example, the correction unit 33 of the control system 60 may correct any one of a plurality of initial characteristics R using the correction data C, similar to the third embodiment. Furthermore, the generation unit 32 of the control system 60 may selectively use any one of a plurality of trained models M to generate the correction data C, similar to the fourth embodiment.

[0074] F: Sixth embodiment 14 is a block diagram illustrating the configuration of a performance system 100 according to the sixth embodiment. The performance system 100 according to the sixth embodiment includes an electronic musical instrument 10 and an information device 65. The information device 65 is a terminal device such as a smartphone or a tablet terminal. The information device 65 is connected to the electronic musical instrument 10, for example, by wire or wirelessly.

[0075] The information device 65 is realized by a computer system including a control device 66 and a storage device 67. The control device 66 is composed of one or more processors that control each element of the information device 65. For example, the control device 66 is composed of one or more types of processors such as a CPU, an SPU, a DSP, an FPGA, or an ASIC. The storage device 67 is one or more memories that store programs executed by the control device 66 and various data used by the control device 66. The storage device 67 is composed of a known recording medium such as a magnetic recording medium or a semiconductor recording medium, or a combination of multiple types of recording media. Note that a portable recording medium that is detachable from the information device 65, or a recording medium (e.g., cloud storage) that the control device 66 can write or read via, for example, a communication network 90 may be used as the storage device 67.

[0076] The control device 66 executes a program stored in the storage device 67 to realize a characteristic setting unit 30 similar to that of the first embodiment. The characteristic setting unit 30 is an element that executes a setting process Sb similar to that of the first embodiment, and includes an acquisition unit 31, a generation unit 32, and a correction unit 33. The acquisition unit 31 receives input data D generated by the electronic musical instrument 10. As in the first embodiment, the generation unit 32 outputs correction data C by inputting the input data D to the trained model M. The correction unit 33 sets the intensity characteristic Q by correcting the initial characteristic R with the correction data C. The intensity characteristic Q is transmitted to the electronic musical instrument 10. The trained model M and the initial characteristic R are stored in the storage device 67. Specifically, the trained model M established by the machine learning system 40 is transferred to the information device 65, and the trained model M is stored in the storage device 67.

[0077] The sixth embodiment achieves the same effects as the first embodiment. The configurations of the second to fourth embodiments are also applicable to the sixth embodiment in which the information device 65 executes the setting process Sb.

[0078] G: Seventh embodiment 15 is a block diagram illustrating the configuration of a performance system 100 according to the seventh embodiment. As in the sixth embodiment, the performance system 100 includes an electronic musical instrument 10 and an information device 65. The configurations of the electronic musical instrument 10 and the information device 65 are the same as in the sixth embodiment.

[0079] The machine learning system 40 stores multiple trained models M corresponding to different models of electronic musical instruments 10. The relationship between the input data D and the correction data C learned by the trained models M through the learning process Sc differs for each trained model M (i.e., for each model of electronic musical instrument 10). For example, a set of multiple training data T is individually prepared for each model of electronic musical instrument 10, and the trained model M for each model is established through a learning process Sc using the multiple training data T in the set corresponding to that model. Therefore, even if the input data D input to each trained model M is the same, the correction data C output by the trained model M differs for each trained model M. In other words, the intensity characteristic Q after correction by the correction unit 33 differs for each model of electronic musical instrument 10.

[0080] The information device 65 selectively acquires one of the multiple trained models M held by the machine learning system 40 via the communication network 90. ​​Specifically, the information device 65 acquires one trained model M from the machine learning system 40, among the multiple trained models M, that corresponds to the model of the electronic musical instrument 10 connected to the information device 65. The trained model M acquired from the machine learning system 40 is stored in the storage device 67 and is used in the setting process Sb by the characteristic setting unit 30. The specific procedure of the setting process Sb is the same as in each of the above-mentioned embodiments.

[0081] As can be understood from the above description, the seventh embodiment also achieves the same effects as the sixth embodiment. In addition, in the seventh embodiment, a trained model M is established for each model of electronic musical instrument 10, and therefore, compared to a configuration in which a common trained model M is used regardless of the model of electronic musical instrument 10, there is an advantage in that correction data C (and therefore intensity characteristic Q) appropriate for each model of electronic musical instrument 10 can be generated with high accuracy.

[0082] The configurations of the second to fourth embodiments are similarly applied to the seventh embodiment in which the information device 65 executes the setting process Sb. The electronic musical instruments 10 of the first to fourth embodiments, the control system 60 of the fifth embodiment, and the information device 65 of the sixth and seventh embodiments are each an example of an "information processing system."

[0083] Also, in the first to fourth embodiments, as in the seventh embodiment, a configuration is also assumed in which, of the multiple trained models M, a trained model M corresponding to the model of the electronic musical instrument 10 is transmitted from the machine learning system 40 to the electronic musical instrument 10. Also, in the fifth embodiment, as in the seventh embodiment, a configuration is also assumed in which, of the multiple trained models M, a trained model M corresponding to the model of the electronic musical instrument 10 is transmitted from the machine learning system 40 to the control system 60. Note that, although the above description has focused on the trained model M for each model, the initial characteristics R applied to the setting process Sb may also be prepared individually for each model of the electronic musical instrument 10.

[0084] H: Eighth embodiment FIG. 16 is a side view focusing on one key 151 that constitutes the playing device 15 (keyboard). Each key 151 is supported on a support 153 via a fulcrum 152. The end of each key 151 is displaced vertically when the user presses and releases the key. The playing device 15 is equipped with a plurality of detection units 70 corresponding to different keys 151. The detection unit 70 corresponding to each key 151 is a sensor that generates a detection signal according to the displacement of the key 151. Each detection unit 70 is equipped with a signal generation section 71 and a detected section 72. The signal generation section 71 is attached to the support 153, and the detected section 72 is attached to the key 151.

[0085] 17 is a circuit diagram illustrating the configuration of one detection unit 70. The signal generating section 71 of each detection unit 70 is a resonant circuit including an input terminal 711, an output terminal 712, a resistive element 713, a coil 714, a capacitive element 715, and a capacitive element 716. One end of the resistive element 713 is connected to the input terminal 711, and the other end of the resistive element 713 is connected to one end of the capacitive element 715 and one end of the coil 714. The other end of the coil 714 is connected to the output terminal 712 and one end of the capacitive element 716. The other end of the capacitive element 715 and the other end of the capacitive element 716 are grounded (Gnd).

[0086] The detected portion 72 of each detection unit 70 is a resonant circuit including a coil 721 and a capacitive element 722. Both ends of the coil 721 and both ends of the capacitive element 722 are connected to each other. The resonant frequency of the signal generating portion 71 and the resonant frequency of the detected portion 72 are set to, for example, approximately the same frequency.

[0087] A reference signal G1 is supplied to the input terminal 711 of each signal generating unit 71 in a time-division manner. The reference signal G1 is a periodic signal that fluctuates at a frequency equal to the resonant frequency of the signal generating unit 71 and the detected unit 72. The reference signal G1 is supplied to the coil 714 via the input terminal 711 and the resistive element 713. A magnetic field is generated in the coil 714 by the supply of the reference signal G1. An induced current is generated in the coil 721 of the detected unit 72 by electromagnetic induction caused by the magnetic field generated in the coil 714. That is, a magnetic field is generated in the coil 721 in a direction that offsets the change in the magnetic field of the coil 714. The magnetic field generated in the coil 721 changes depending on the distance between the coil 714 and the coil 721. Therefore, a detection signal G2 having a level according to the distance between the coil 714 and the coil 721 is output from the output terminal 712 of the signal generating unit 71.

[0088] The acquisition unit 31 analyzes the detection signal G2 supplied from the signal generation unit 71 of each key 151 to sequentially calculate the displacement of that key 151. The control device 11 or 61 calculates the performance intensity X from the change over time in the displacement calculated from the detection signal G2. Moreover, the performance data d2 in the eighth embodiment is data representing a time series of the displacement calculated for each key 151 from the detection signal G2. In other words, the performance data d2 represents the continuous or stepwise change over time in the displacement of each key 151.

[0089] The eighth embodiment described above achieves the same effects as the first embodiment. The configurations of the second to seventh embodiments are also applicable to the eighth embodiment.

[0090] I: Variations Specific modified embodiments that can be added to each of the above-mentioned embodiments are illustrated below. A plurality of embodiments arbitrarily selected from the following examples may be appropriately combined as long as they are not mutually contradictory.

[0091] (1) In the above-mentioned embodiments, the performance intensity X and the sounding period error τ are exemplified as the performance characteristics represented by the tendency data d3, but the performance characteristics represented by the tendency data d3 are not limited to the above examples. For example, the performance intensity X at the time when the user touches the key 151 when pressing the key (i.e., initial touch), or the intensity of the operation of the user pressing the key 151 after pressing the key (i.e., aftertouch) may be expressed as the performance characteristics by the tendency data d3. In addition, for example, the tempo at which the user plays a standard piece of music, or the strength of the performance (e.g., piano or forte) are also exemplified as the performance characteristics represented by the tendency data d3.

[0092] (2) In each of the above-described embodiments, the input data D includes the music data d1, the performance data d2, and the tendency data d3, but the elements constituting the input data D are not limited to the above examples. For example, the music data d1 or the performance data d2 may be omitted from the input data D. In addition, the input data D may include video data capturing an image of a user's performance (e.g., the state of hands operating a keyboard).

[0093] (3) Any method may be used to correct the initial characteristic R using the correction data C. For example, the correction unit 33 may generate the intensity characteristic Q by adding or multiplying the numerical value of the sound intensity Y in the initial characteristic R, the numerical value being obtained by multiplying the correction amount specified by the correction data C for each numerical value of the performance intensity X by a weighting value. The weighting value is a variable value that is set, for example, in response to an instruction from the user via the operation device 14. With the above configuration, the degree of correction by the correction data C can be variably controlled.

[0094] (4) In each of the above-described embodiments, a deep neural network is exemplified as the trained model M, but the trained model M is not limited to a deep neural network. For example, a statistical estimation model such as a hidden Markov model (HMM) or a support vector machine (SVM) may be used as the trained model M. A specific example of the trained model M using an SVM will be described in detail below.

[0095] For example, an SVM is prepared for each of all combinations of two types of correction data C selected from a plurality of different correction data C. For the SVM corresponding to the combination of two types of correction data C, a hyperplane in a multidimensional space is established by machine learning (learning process Sc). The hyperplane is a boundary surface that separates a space in which input data D corresponding to one of the two types of correction data C is distributed, from a space in which input data D corresponding to the other correction data C is distributed. The trained model M is composed of multiple SVMs corresponding to different combinations of correction data C (multi-class SVM).

[0096] The generation unit 32 inputs input data D to each of the multiple SVMs of the trained model M. The SVM corresponding to each combination selects one of the two types of correction data C related to the combination depending on which of the two spaces separated by the hyperplane the input data D is in. Selection of correction data C is performed in the same manner in each of the multiple SVMs corresponding to different combinations. The generation unit 32 selects the correction data C that is selected the maximum number of times by the multiple SVMs from among the multiple types of correction data C as the final correction data C. As can be understood from the above example, regardless of the type of trained model M, the generation unit 32 functions as an element that inputs the input data D to the trained model M and outputs the correction data C.

[0097] (5) In each of the above-described embodiments, supervised machine learning using multiple training data T is exemplified as the learning process Sc, but the learned model M may be established by unsupervised machine learning that does not require training data T, or reinforcement learning that maximizes rewards. An example of unsupervised machine learning is machine learning that uses known clustering.

[0098] (6) In each of the above-mentioned embodiments, the machine learning system 40 establishes the trained model M, but the functions of the machine learning system 40 (the training data acquisition unit 51 and the learning processing unit 52) ​​may be incorporated in the electronic musical instrument 10 in the first to fourth embodiments, the control system 60 in the fifth embodiment, or the information device 65 in the sixth and seventh embodiments.

[0099] (7) In each of the above-described embodiments, a trained model M that has learned the relationship between the input data D and the correction data C is used, but the configuration and method for generating the correction data C from the input data D are not limited to the above examples. For example, a reference table in which the correction data C is associated with each of a plurality of different input data D may be used for the generation of the correction data C by the generation unit 32. The reference table is a data table in which the correspondence between the input data D and the correction data C is registered, and is stored in, for example, the storage device 12. The generation unit 32 searches the reference table for the correction data C that corresponds to the input data D acquired by the acquisition unit 31.

[0100] (8) The functions exemplified in each of the above-mentioned embodiments (for example, the acquisition unit 31, the generation unit 32, and the correction unit 33) are realized by the cooperation of one or more processors constituting the control device (11, 61, 65) and the programs stored in the storage device (12, 62, 67), as described above. The above-mentioned programs can be provided in a form stored in a computer-readable recording medium and installed in a computer. The recording medium is, for example, a non-transitory recording medium, and a good example is an optical recording medium (optical disk) such as a CD-ROM, but also includes any known type of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that the non-transitory recording medium includes any recording medium except for a transient, propagating signal, and does not exclude volatile recording media. In addition, in a configuration in which a distribution device distributes a program via a communication network, the recording medium that stores the program in the distribution device corresponds to the above-mentioned non-transitory recording medium.

[0101] J:Additional Notes From the above-described exemplary embodiments, the following configurations can be understood, for example.

[0102] An information processing system according to one aspect (aspect 1) includes an acquisition unit that acquires input data including tendency data reflecting a user's performance tendency, a generation unit that generates correction data by inputting the input data acquired by the acquisition unit to a trained model that has learned the relationship between input data for learning and correction data for learning, and a correction unit that uses the generated correction data to correct a first intensity characteristic, which is a relationship between the performance intensity of the user's performance and the sound intensity of a sound generated in response to the performance. According to the above aspect, it is possible to set an intensity characteristic that reflects the user's performance tendency by correcting the first intensity characteristic using the correction data output by the trained model. For example, it is possible to set an intensity characteristic that assists a performance that the user is not good at.

[0103] "Performance intensity" refers to the intensity of a performance action by a user. For example, the intensity with which a user operates a performance operator for playing a musical instrument (e.g., key pressing intensity) is a typical example of performance intensity. Moreover, "sound intensity" refers to the intensity (e.g., volume) of a musical tone that is generated in response to a performance by a user. Therefore, the intensity characteristic is, for example, a touch curve (velocity curve) that is the relationship of sound intensity to performance intensity.

[0104] "Tendency data" is data in any format that reflects the performance tendencies of a user. For example, the tendency data is data that indicates the tendency of the performance strength of a user. Specifically, data that indicates the tendency of key pressing strength for each finger is exemplified as the tendency data. "Fingers" include the thumb, index finger, middle finger, ring finger, and little finger.

[0105] The "trained model" is a statistical estimation model that has learned the relationship between the learning trend data and the learning correction data by machine learning. In other words, the trained model outputs statistically appropriate correction data for the unknown trend data acquired by the acquisition unit, based on the underlying relationship between the learning trend data and the learning correction data (correct value).

[0106] In a specific example (Aspect 2) of Aspect 1, the correction unit sets the second intensity characteristic by correcting the first intensity characteristic using the correction data. In a specific example (Aspect 3) of Aspect 2, the first intensity characteristic is an initial characteristic prepared in advance, and the second intensity characteristic is a characteristic reflecting the playing tendency of the user.

[0107] In a specific example (Aspect 4) of any one of Aspects 1 to 3, the tendency data represents the performance characteristics of the user with respect to each of a plurality of performance operators operated during performance. According to the above aspects, a second intensity characteristic can be set that reflects the performance characteristics of the user with respect to each performance operator. Note that the performance characteristics represented by the tendency data are, for example, the intensity (performance intensity) with which the user operates each performance operator, or the error in the timing at which each performance operator is operated.

[0108] In a specific example of aspect 4 (aspect 5), the tendency data represents the performance characteristics of the user for each of a plurality of combinations of any of the plurality of performance operators and any of a plurality of fingers of the user. In the above aspect, the tendency data represents the performance characteristics of the user for each combination of a performance operator and the user's fingers. Therefore, it is possible to generate a suitable strength characteristic that reflects the performance tendency of each finger for each performance operator (for example, a tendency that a particular performance operator is not good at being played with a particular finger).

[0109] In a specific example (aspect 6) of aspects 1 to 5, the input data includes performance data specifying a time sequence of notes played by the user. In the above aspects, a second intensity characteristic can be set that reflects the performance tendency of the user.

[0110] In a specific example (Aspect 7) of any one of Aspects 1 to 6, the correction unit corrects the first intensity characteristic corresponding to the tone color selected by the user among a plurality of first intensity characteristics corresponding to different tone colors, using the correction data. In the above aspects, the first intensity characteristic corresponding to the tone color selected by the user is corrected by the correction data. Therefore, a suitable second intensity characteristic can be set for each tone color.

[0111] In a specific example (Aspect 8) of any one of Aspects 1 to 6, the correction unit corrects the first intensity characteristic corresponding to the music genre selected by the user among a plurality of first intensity characteristics corresponding to different music genres, using the correction data. In the above aspects, the first intensity characteristic corresponding to the music genre selected by the user is corrected by the correction data. Therefore, a suitable second intensity characteristic can be set for each music genre.

[0112] In a specific example (aspect 9) of any one of aspects 1 to 8, the generation unit generates the correction data by using the trained model corresponding to the tone color selected by the user among a plurality of trained models corresponding to different tones. In the above aspects, the trained model corresponding to the tone color selected by the user is used to generate the correction data. Therefore, a suitable second intensity characteristic can be set for each tone color.

[0113] In a specific example (Aspect 10) of any one of Aspects 1 to 8, the generation unit generates the correction data by using the trained model corresponding to the music genre selected by the user among a plurality of trained models corresponding to different music genres. In the above aspects, the trained model corresponding to the music genre selected by the user is used to generate the correction data. Therefore, a suitable second intensity characteristic can be set for each music genre.

[0114] An electronic musical instrument according to one embodiment (embodiment 11) of the present disclosure includes an acquisition unit that acquires input data including tendency data reflecting a user's performance tendency, a generation unit that generates correction data by inputting the input data acquired by the acquisition unit to a trained model that has learned the relationship between input data for learning and correction data for learning, a correction unit that sets a second intensity characteristic by correcting a first intensity characteristic, which is a relationship between the performance intensity of the performance by the user and the sound intensity of the sound generated in response to the performance, using the generated correction data, a performance acceptance unit that accepts the performance by the user, and a playback control unit that causes a playback system to play musical sounds corresponding to the performance accepted by the performance acceptance unit under the second intensity characteristic. Note that the "performance acceptance unit" is any element that a user operates for playing, such as a keyboard in a keyboard instrument or an operator (key) in a wind instrument.

[0115] In a specific example (Aspect 12) of Aspect 11, the performance receiving unit includes a performance operator that displaces in response to a performance by the user, a signal generating unit including a first coil to which a periodically varying reference signal is supplied, and a detectable unit installed on the performance operator, the detectable unit including a second coil in which an induced current is generated by electromagnetic induction due to a magnetic field generated in the first coil when the reference signal is supplied to the first coil, and the signal generating unit outputs a detection signal at a level according to the distance between the first coil and the second coil. According to the above aspect, the displacement of the performance operator can be detected with high accuracy using a simple configuration.

[0116] An information processing method according to one aspect (aspect 13) of the present disclosure acquires input data including tendency data reflecting a user's performance tendency, generates correction data by inputting the acquired input data into a trained model that has learned the relationship between learning input data and learning correction data, and sets a second intensity characteristic by correcting a first intensity characteristic, which is the relationship between the performance intensity of the user's performance and the pronunciation intensity of a sound produced in response to the performance, using the generated correction data.

[0117] A method for generating a trained model according to one embodiment (embodiment 14) of the present disclosure includes obtaining a plurality of training data representing a combination of learning input data including tendency data reflecting a performer's performance tendency and correction data for correcting intensity characteristics, which are the relationship between the performance intensity of the user's performance and the pronunciation intensity of the sound produced in response to the performance, and establishing a trained model that has learned the relationship between the input data and the correction data in the plurality of training data through machine learning using the plurality of training data. [Explanation of symbols]

[0118] 10...electronic musical instrument, 11,41,61,66...control device, 12,42,62,67...storage device, 13,43,63...communication device, 14...operation device, 15...performance device, 151...key, 152...fulcrum, 153...support, 16...sound source device, 17...sound emission device, 18...playback system, 30...characteristic setting unit, 31...acquisition unit, 32...generation unit, 33...correction unit, 34...playback control unit, 40...machine learning system, 51...training data acquisition unit, 52...learning processing unit, 60...control system, 65...information device, 70...detection unit, 71...signal generation unit, 72...detected unit.

Claims

1. an acquisition unit that acquires input data including tendency data reflecting a user's playing tendency; A generation unit that generates correction data by inputting the input data acquired by the acquisition unit into a trained model that has learned the relationship between learning input data and learning correction data; a correction unit that corrects a first intensity characteristic, which is a relationship between a performance intensity of the performance by the user and a sound intensity of a sound generated in response to the performance, by using the generated correction data; An information processing system comprising:

2. The correction unit sets a second intensity characteristic by correcting the first intensity characteristic using the correction data.

2. The information processing system of claim 1.

3. the first strength characteristic is an initial characteristic prepared in advance, The second strength characteristic is a characteristic that reflects the playing tendency of the user.

3. The information processing system of claim 2.

4. The tendency data represents the performance characteristics of the user with respect to each of a plurality of performance operators operated during a performance.

4. The information processing system according to claim 1.

5. The tendency data represents the performance characteristics of the user for each of a plurality of combinations of any of the plurality of performance operators and any of a plurality of fingers of the user.

5. The information processing system of claim 4.

6. The input data includes performance data that specifies a time sequence of notes played by the user.

6. An information processing system according to claim 1.

7. The correction unit corrects the first intensity characteristic corresponding to a tone color selected by the user among a plurality of first intensity characteristics corresponding to different tone colors, using the correction data.

7. An information processing system according to claim 1.

8. The correction unit corrects the first intensity characteristic corresponding to the music genre selected by the user among a plurality of first intensity characteristics corresponding to different music genres, using the correction data.

7. An information processing system according to claim 1.

9. The generation unit generates the correction data by using a trained model corresponding to a tone color selected by the user from among a plurality of trained models corresponding to different tone colors.

9. The information processing system according to claim 1.

10. The generation unit generates the correction data by using a trained model corresponding to a music genre selected by the user from among a plurality of trained models corresponding to different music genres.

9. The information processing system according to claim 1.

11. an acquisition unit that acquires input data including tendency data reflecting a user's playing tendency; A generation unit that generates correction data by inputting the input data acquired by the acquisition unit into a trained model that has learned the relationship between learning input data and learning correction data; a correction unit that corrects a first intensity characteristic, which is a relationship between a performance intensity of the performance by the user and a sound intensity of a sound generated in response to the performance, by using the generated correction data to set a second intensity characteristic; a performance reception unit that receives a performance by the user; a reproduction control unit that causes a reproduction system to reproduce, under the second intensity characteristic, the musical sound corresponding to the performance received by the performance receiving unit; An electronic musical instrument comprising:

12. The performance reception unit is a performance operator that is displaced in response to a performance by the user; a signal generating unit including a first coil to which a periodically varying reference signal is supplied; a detection target portion provided on the performance operator, the detected portion includes a second coil in which an induced current is generated by electromagnetic induction caused by a magnetic field generated in the first coil when the reference signal is supplied to the first coil, The signal generating unit outputs a detection signal having a level corresponding to the distance between the first coil and the second coil.

12. The electronic musical instrument of claim 11.

13. Acquire input data including trend data reflecting the user's playing tendencies; The acquired input data is input to a trained model that has learned the relationship between the learning input data and the learning correction data, thereby generating correction data; A first intensity characteristic, which is a relationship between the performance intensity of the performance by the user and the sound intensity of the sound generated in response to the performance, is corrected using the correction data to set a second intensity characteristic. An information processing method implemented by a computer system.

14. The tendency data represents the performance characteristics of the user with respect to each of a plurality of performance operators operated during a performance.

14. The information processing method according to claim 13.

15. The tendency data represents a performance characteristic of the user for each combination of the plurality of performance operators and each of the plurality of fingers of the user.

15. The information processing method according to claim 14.

16. The input data includes performance data that specifies a time sequence of notes played by the user.

16. The information processing method according to any one of claims 13 to 15.

17. In the correction, the first intensity characteristic corresponding to the tone color selected by the user among a plurality of first intensity characteristics corresponding to different tone colors is corrected using the correction data.

17. The information processing method according to claim 13.

18. A method for acquiring a plurality of training data representing a combination of learning input data including tendency data reflecting a user's playing tendency and correction data for correcting an intensity characteristic which is a relationship between a playing intensity of the user's playing and a sound intensity of a sound generated in response to the playing, A trained model is established by learning a relationship between the input data and the correction data in the plurality of training data through machine learning using the plurality of training data. A method for generating a trained model implemented by a computer system.

Images