Information processing method and information processing system

Abstract

To allow a user to easily and appropriately understand evaluation on performance of a musical instrument.SOLUTION: An information processing system 100 includes: an information acquisition unit 41 for acquiring performance information regarding performance of an electronic musical instrument 20 by a user U; a response acquisition unit 42 for acquiring response information R of natural language corresponding to performance information; and an operation control unit 43 for performing notification operation for notifying response information R by a guide character displayed on a display device 15.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a technique for notifying a user of information related to playing a musical instrument. [Background technology]

[0002] Techniques for notifying users of information related to musical instrument performance have been proposed in the past. For example, Patent Document 1 discloses a configuration for generating performance information by analyzing video data generated by shooting a user, and generating an evaluation score by comparing the performance information with reference information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-053852 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is a problem in that a user cannot easily and appropriately understand information related to his / her own performance by simply displaying an evaluation score as in the technology of Patent Document 1. In consideration of the above circumstances, one aspect of the present disclosure has an object to enable a user to easily and appropriately understand information related to musical instrument performance. [Means for solving the problem]

[0005] In order to solve the above problems, an information processing method according to one aspect of the present disclosure acquires performance information regarding a user playing an instrument, acquires response information in natural language corresponding to the performance information, and performs a notification action of notifying the response information by a guide character displayed on a display device.

[0006] Another aspect of the present disclosure relates to an information processing method that acquires performance information regarding a user playing an instrument, and generates a prompt for generating natural language response information using a trained generative model, the prompt including the performance information.

[0007] An information processing system according to one aspect of the present disclosure includes an information acquisition unit that acquires performance information regarding a user playing an instrument, a response acquisition unit that acquires response information in natural language corresponding to the performance information, and an operation control unit that causes a guide character displayed on a display device to perform an announcement operation to announce the response information.

[0008] A program according to one aspect of the present disclosure causes a computer system to function as an information acquisition unit that acquires performance information regarding a user's playing of an instrument, a response acquisition unit that acquires response information in natural language corresponding to the performance information, and an operation control unit that causes a guide character displayed on a display device to perform an announcement action to announce the response information. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating a configuration of an information system in a first embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating a configuration of an information processing system. [Diagram 3] FIG. 4 is a schematic diagram of a setting screen. [Figure 4] FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing system. [Diagram 5] FIG. 1 is a schematic diagram of a basic character string. [Figure 6] FIG. 2 is a schematic diagram of prompt and response information. [Figure 7] FIG. 11 is a schematic diagram of response information. [Figure 8] FIG. 11 is a schematic diagram of response information. [Figure 9] FIG. 13 is a schematic diagram of a guide character. [Figure 10] 13 is a flowchart of an evaluation informing process. [Figure 11]FIG. 11 is a schematic diagram of a prompt and response information in the second embodiment. [Figure 12] FIG. 11 is a block diagram illustrating a functional configuration of an information processing system according to a second embodiment. [Figure 13] 13 is a flowchart of an evaluation notification process in the second embodiment. [Figure 14] FIG. 13 is a schematic diagram of prompt and response information in the third embodiment. [Figure 15] FIG. 13 is a schematic diagram of prompt and response information in a modified example. [Figure 16] FIG. 13 is a schematic diagram of a prompt in a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] A: First embodiment 1 is a block diagram illustrating the configuration of an information system 100 in the first embodiment. The information system 100 is a computer system that guides a user U in playing an electronic musical instrument 20. The information system 100 includes an information processing system 10, the electronic musical instrument 20, and a response generation system 30. The information processing system 10 is capable of communicating with the response generation system 30 via a communication network 200 such as the Internet.

[0011] The response generation system 30 is a server system that generates response information R corresponding to a prompt P. The prompt P in the first embodiment is an action instruction expressed in a natural language. The response information R is text data that expresses a response to the prompt P in a natural language.

[0012] The response generation system 30 generates response information R by a trained generative model M. The generative model M is a generative probabilistic model that generates response information R in response to a prompt P. The generative model M has learned the tendency of response information R to a prompt P by prior machine learning (pre-training). Specifically, the generative model M is an interactive large-scale language model (LLM) that is trained specifically for a natural language processing task such as response generation. For example, a natural language processing model realized by a transformer model using a self-attention mechanism is exemplified as the generative model M.

[0013] The electronic musical instrument 20 is an input device that accepts musical pieces played by the user U. The electronic musical instrument 20 of the first embodiment is, for example, a keyboard instrument that complies with the MIDI (Musical Instrument Digital Interface) standard, and has a plurality of keys 21 that correspond to different pitches. The electronic musical instrument 20 may be mounted on the information processing system 10.

[0014] The user U plays a piece of music by operating each key 21 in sequence. The electronic musical instrument 20 not only emits musical tones according to the performance by the user U, but also outputs a performance data string D representing the performance to the information processing system 10. The performance data string D is, for example, a time series of event data conforming to the MIDI standard. Specifically, the performance data string D specifies, in time series, pitches corresponding to the keys 21 operated by the user U.

[0015] The information processing system 10 transmits a prompt P corresponding to the result of the evaluation of the performance of the electronic musical instrument 20 by the user U to the response generation system 30, and notifies the user U of the response information R received from the response generation system 30. In other words, the response information R in natural language corresponding to the evaluation of the performance by the user U is notified to the user U.

[0016] 2 is a block diagram of the information processing system 10. The information processing system 10 is realized by an information device such as a smartphone, a tablet terminal, or a personal computer. The information processing system 10 includes a control device 11, a storage device 12, a communication device 13, a sound emitting device 14, a display device 15, and an operation device 16. The information processing system 10 may be realized as a single device, or may be realized as a plurality of devices configured separately from each other.

[0017] The control device 11 is composed of one or more processors that control each element of the information processing system 10. For example, the control device 11 is composed of one or more types of processors such as a CPU (Central Processing Unit), an SPU (Sound Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

[0018] The storage device 12 is one or more memories that store the programs executed by the control device 11 and various data used by the control device 11. The storage device 12 is configured with a known recording medium such as a magnetic recording medium or a semiconductor recording medium. The storage device 12 may be configured with a combination of multiple types of recording media. In addition, a portable recording medium that is detachable from the information processing system 10, or a recording medium (e.g., cloud storage) to which the control device 11 can write or read via the communication network 200 may be used as the storage device 12.

[0019] The storage device 12 stores music data C representing a piece of music to be performed by the user U. The music data C is data representing the musical score of the piece of music. Specifically, the music data C specifies the pitch and sound duration of each of the multiple notes that make up the piece of music. For example, the music data C is data that complies with the MIDI standard. Note that the music data C may specify information such as performance symbols that express musical expression.

[0020] The communication device 13 communicates with the response generation system 30 under the control of the control device 11. Specifically, the communication device 13 transmits a prompt P to the response generation system 30 and receives response information R transmitted from the response generation system 30.

[0021] The sound emitting device 14 emits sound waves under the control of the control device 11. The sound emitting device 14 is, for example, a speaker or a headphone. Specifically, a voice signal V representing a voice corresponding to the response information R (hereinafter referred to as "guidance voice") is supplied to the sound emitting device 14. The sound emitting device 14 reproduces the guidance voice for the user U. For convenience, a D / A converter that converts the voice signal V from digital to analog and an amplifier that amplifies the voice signal V are omitted from the illustration. The sound emitting device 14, which is separate from the information processing system 10, may be connected to the information processing system 10 by wire or wirelessly.

[0022] The display device 15 displays an image under the control of the control device 11. The display device 15 is configured with a display panel such as a liquid crystal panel or an organic EL (Electroluminescence) panel. The operation device 16 is an input device that receives instructions from the user U. The operation device 16 is, for example, an operator operated by the user U, or a touch panel that detects contact by the user U. Note that the display device 15 or the operation device 16, which are separate from the information processing system 10, may be connected to the information processing system 10 by wire or wirelessly.

[0023] The user U can specify a condition related to the response information R (hereinafter referred to as "response condition X") by operating the operation device 16. FIG. 3 is a schematic diagram of a setting screen 151 for the user U to specify the response condition X (X1, X2, X3). The setting screen 151 is displayed on the display device 15. The response condition X includes an attribute X1 of a virtual respondent (e.g., a guide character 152 described later) who executes the response represented by the response information R, an attribute X2 of the user U, and a tone X3 of the response according to the response information R. The setting screen 151 is an image in which a plurality of input fields F (F1, F2, F3) are arranged.

[0024] The input field F1 is an input box in which the user U specifies the attribute X1 of a virtual respondent. The user U inputs a character string of the respondent's attribute X1 into the input field F1 by operating the operation device 16. For example, the respondent's occupation (piano teacher, etc.) is input as the attribute X1. Note that the user U may select any one of a plurality of options prepared in advance as the attribute X1.

[0025] The input field F2 is an input box in which the user U specifies the attribute X2 of the user U. The user U inputs a character string of his / her own attribute X2 into the input field F2 by operating the operation device 16. For example, the age of the user U (elementary school student / junior high school student / high school student / university student / adult, etc.) is input as the attribute X2. Note that the user U may select one of a plurality of options prepared in advance as the attribute X2.

[0026] The input field F3 is an input box in which the user U specifies the tone X3 of the response based on the response information R. The user U inputs a character string of the desired tone X3 into the input field F3 by operating the operation device 16. For example, the tone of the remark, such as "kindly," "strictly," or "frankly," is specified as the tone X3. Note that the user U may specify any one of a number of options prepared in advance as the tone X3.

[0027] 4 is a block diagram illustrating an example of a functional configuration of the information processing system 10. The control device 11 executes a program stored in the storage device 12 to realize a plurality of functions (an information acquisition unit 41, a response acquisition unit 42, and an operation control unit 43).

[0028] The information acquisition unit 41 evaluates the performance of the electronic musical instrument 20 by the user U. Specifically, the information acquisition unit 41 compares the performance data string D supplied from the electronic musical instrument 20 with the music data C stored in the storage device 12, and identifies differences between the two as performance errors by the user U. In other words, a performance error is a difference between the performance by the user U and a standard performance specified by the music data C. The information acquisition unit 41 generates evaluation information Y for each performance error that occurs in the performance by the user U. The evaluation information Y is information that represents an evaluation of the performance of the electronic musical instrument 20 by the user U. Specifically, the evaluation information Y includes the importance Y1 of the performance error, the position Y2 of the performance error, the type Y3 of the performance error, and detailed content Y4 of the performance error.

[0029] The importance Y1 of a performance error is the degree of importance of the performance error. In the first embodiment, the importance Y1 is binary information representing the level of importance. Note that the importance Y1 may be information representing the level of importance in multiple stages.

[0030] The position Y2 of the performance error is a point in the music piece where the performance error occurred. For example, a point in the music piece where the performance data string D and the music piece data C differ is identified as the position Y2 of the performance error. For example, it is specified by a bar number in the music piece or an elapsed time from the start point of the music piece. Note that, in addition to a specific point in the music piece, a specific section in the music piece may be specified as the position Y2 of the performance error.

[0031] The type Y3 of a performance error means a classification of a performance error according to its type. For example, "pitch error", "rhythm error", "hesitation in playing", etc. are designated as the type Y3 of a performance error. The importance Y1 described above is set according to the type Y3 of a performance error. For example, the importance Y1 is stored in the storage device 12 in advance for each type Y3 of a performance error.

[0032] The detailed content Y4 of the performance error is the specific content of the performance error or the criticism of the user U regarding the performance error. For example, the detailed content Y4 of the performance error may be, for example, a performance state such as "playing 'B' where 'C' should be played," "playing earlier than the correct timing," or "having to play without stopping." The detailed content Y4 of the performance error may be expressed in, for example, a natural language.

[0033] 4 acquires response information R in a natural language according to evaluation information Y. The response acquisition unit 42 of the first embodiment includes an instruction generation unit 421 and an information receiving unit 422. The instruction generation unit 421 generates a prompt P for the response generation system 30. Specifically, the instruction generation unit 421 generates the prompt P including a response condition X and evaluation information Y.

[0034] The instruction generating unit 421 generates the prompt P using the basic string B in FIG. 5. The basic string B is a template representing a typical string of the prompt P, and is stored in advance in the storage device 12. The basic string B includes a plurality of strings b1 to b6. A blank is set in each of the plurality of strings b1 to b6. Each blank is a portion of the basic string B into which variable information (response condition X, evaluation information Y) is inserted. The instruction generating unit 421 generates the prompt P by inserting the response condition X or the evaluation information Y into each blank of the basic string B. FIG. 6 illustrates a prompt P generated by the instruction generating unit 421 using the basic string B in FIG. 5.

[0035] Character string b1 of basic character string B is a portion expressing a condition related to a respondent. The instruction generating unit 421 generates an instruction related to a respondent in the prompt P by inserting attribute X1 specified by the user U on the setting screen 151 into a blank space of character string b1. As described above, the prompt P in the first embodiment includes attribute X1 of a virtual respondent.

[0036] Character string b2 of basic character string B is a portion expressing the condition of user U to be notified of response information R. The instruction generating unit 421 generates an instruction related to user U from the prompt P by inserting attribute X2 specified by user U on setting screen 151 into the blank space of character string b2. As described above, the prompt P in the first embodiment includes attribute X2 of user U.

[0037] The character string b3 of the basic character string B is a portion expressing a condition regarding the expression of a response by the response information R. The instruction generating unit 421 generates an instruction regarding the expression of the response information R in the prompt P by inserting the tone X3 specified by the user U on the setting screen 151 into the blank space of the character string b3. As described above, the prompt P in the first embodiment includes the tone X3 related to the response information R.

[0038] The character strings b4 to b6 of the basic character string B are portions expressing conditions related to the response content of the response information R. The instruction generating unit 421 generates an instruction related to the content of the response information R in the prompt P by inserting the evaluation information Y generated by the information acquiring unit 41 into each blank of the character strings b4 to b6.

[0039] The character string b4 is a portion for notifying an important performance mistake. The instruction generating unit 421 generates an instruction regarding an important performance mistake in the prompt P by inserting the position Y2, type Y3, and detailed content Y4 designated by each evaluation information Y having a high importance Y1 into each blank of the character string b4.

[0040] Character strings b5 and b6 are portions notifying minor performance errors. The instruction generating unit 421 generates an instruction regarding minor performance errors in the prompt P by inserting the position Y2, type Y3, and detailed content Y4 specified by each piece of evaluation information Y with low importance Y1 into each blank space in the character strings b5 and b6. Note that the number of performance errors specified in the prompt P is variable.

[0041] The instruction generation unit 421 in Fig. 4 transmits the prompt P generated by the above procedure from the communication device 13 to the response generation system 30. The response generation system 30 generates response information R in natural language by processing the prompt P received from the information processing system 10 using the generation model M, and transmits the response information R to the information processing system 10. The information receiving unit 422 in Fig. 4 receives the response information R transmitted from the response generation system 30 using the communication device 13. In other words, the information receiving unit 422 acquires the response information R generated in response to the prompt P by the trained generation model M.

[0042] 6 shows response information R generated from the above-mentioned prompt P. The response information R represents a natural language response that reflects the response condition X and evaluation information Y included in the prompt P. Specifically, the response information R represents a natural language response sentence that a respondent with attribute X1 should utter in tone X3 to a user U with attribute X2. The response information R also represents a natural language response sentence that explains to the user U the performance error represented by the evaluation information Y. Specifically, the response information R distinguishes between important and minor performance errors (importance Y1), and points out the position Y2, type Y3, and detailed content Y4 of the performance error.

[0043] In FIG. 6, the attribute X2 of the user U is "adult", whereas FIG. 7 illustrates an example of response information R when the attribute X2 is "elementary school student". When the attribute X2 of the user U is "adult" (FIG. 6), response information R is generated in a polite tone like a conversation between adults. On the other hand, when the attribute X2 is "elementary school student" (FIG. 7), response information R is generated in a friendly and approachable tone like an adult instructing a child. As described above, in the first embodiment, the attribute X2 of the user U is included in the prompt P, so that a variety of response information R can be generated according to the attribute X2 of the user U. Similarly, in the first embodiment, the attribute X1 of a virtual respondent is included in the prompt P, so that a variety of response information R can be generated according to the attribute X1 of the respondent.

[0044] Also, while in Fig. 6 tone X3 is "gentle", Fig. 8 illustrates an example of response information R when tone X3 is "strict". When tone X3 is "gentle" (Fig. 6), response information R is generated in a gentle tone so as not to hurt the self-esteem of user U. On the other hand, when tone X3 is "strict" (Fig. 8), response information R is generated in a strict tone. As described above, in the first embodiment, tone X3 related to response information R is included in prompt P, so that response information R in a variety of tones can be generated.

[0045] The operation control unit 43 in FIG. 3 executes an operation of notifying the user U of the response information R described above (hereinafter referred to as the "notification operation"). The notification operation in the first embodiment is an operation of notifying the user U of the response information R by the guide character 152 (FIG. 9) displayed on the display device 15. The operation control unit 43 in the first embodiment includes a voice synthesis unit 431 and a display control unit 432.

[0046] The voice synthesis unit 431 generates a voice signal V of a guidance voice corresponding to the response information R. The guidance voice is a voice that reads out a response sentence represented by the response information R. Specifically, the voice synthesis unit 431 generates the voice signal V by executing a voice synthesis process on the response information R. Examples of the voice synthesis process include a voice synthesis process of a segment concatenation type that connects multiple voice segments, or a voice synthesis process of a statistical model type that uses a statistical model such as a deep neural network or an HMM (Hidden Markov Model). The voice synthesis unit 431 supplies the voice signal V to the sound emission device 14. Therefore, the guidance voice represented by the voice signal V is reproduced from the sound emission device 14.

[0047] The display control unit 432 causes the display device 15 to display the guide character 152 of Fig. 9. The guide character 152 is an object (agent) placed in a virtual space. Specifically, the guide character 152 is a virtual instructor who instructs the user U on how to play the electronic musical instrument 20 in the virtual space.

[0048] The display control unit 432 causes the guide character 152 to perform a speaking action in parallel with the reproduction of the guidance voice by the sound emitting device 14. The speaking action is an action of changing the shape of the mouth of the guide character 152 in response to the audio signal V (so-called lip-syncing). The notification action of the first embodiment includes the speaking action of the guide character 152 and the reproduction of the guidance voice. By executing the speaking action and the reproduction of the guidance voice in parallel, it is possible to make the user U perceive a sensation that the guide character 152 is giving instructions to the user U.

[0049] 10 is a flowchart of a process (hereinafter, referred to as a "rating notification process") executed by the control device 11. For example, an instruction from the user U to the operation device 16 is a trigger for starting the rating notification process.

[0050] When the evaluation notification process is started, the control device 11 (response acquisition unit 42) displays the setting screen 151 of FIG. 3 on the display device 15 (Sa1). The control device 11 (response acquisition unit 42) accepts input of response conditions X (X1, X2, X3) for the setting screen 151 from the user U (Sa2). The response conditions X accepted from the user U are stored in the storage device 12.

[0051] Having input the response condition X, the user U starts playing the electronic musical instrument 20. The control device 11 (information acquisition unit 41) evaluates the user U's performance of the electronic musical instrument 20 (Sa3). Specifically, the control device 11 generates evaluation information Y for each performance mistake made by the user U during the course of the performance.

[0052] The control device 11 (instruction generation unit 421) generates a prompt P including a response condition X and evaluation information Y (Sa4). Specifically, the control device 11 generates the prompt P by inserting the response conditions X (X1, X2, X3) and the evaluation information Y (Y2, Y3, Y4) into each blank of a basic character string B stored in the storage device 12.

[0053] The control device 11 (instruction generation unit 421) transmits a prompt P to the response generation system 30 via the communication device 13 (Sa5). That is, the control device 11 requests the response generation system 30 to generate response information R. The control device 11 (information receiving unit 422) receives the response information R generated and transmitted by the response generation system 30 via the communication device 13 (Sa6).

[0054] The control device 11 (voice synthesis unit 431) generates a voice signal V representing a guidance voice corresponding to the response information R (Sa7), and outputs the voice signal V to the sound emitting device 14 (Sa8). In parallel with the output of the voice signal V to the sound emitting device 14, the control device 11 (display control unit 432) causes the guidance character 152 displayed on the display device 15 to execute a speaking action (Sa9). That is, a notification action including the reproduction of the guidance voice (Sa7, Sa8) and the speaking action (Sa9) by the guidance character 152 is executed.

[0055] As described above, in the first embodiment, response information R in natural language corresponding to evaluation information Y regarding the performance of the electronic musical instrument 20 is acquired, and the guidance character 152 executes a notification action of notifying the response information R. Therefore, compared to a format in which a numerical value (e.g., an evaluation score) that evaluates the performance of the electronic musical instrument 20 by the user U is displayed, the user U can easily and appropriately understand the evaluation regarding the performance of the electronic musical instrument 20. In addition, a unique customer experience in which the user U enjoys the sensation of being instructed on the performance of the electronic musical instrument 20 by the guidance character 152 can be provided.

[0056] Furthermore, in the first embodiment, a prompt P is generated that includes evaluation information Y regarding the performance of the electronic musical instrument 20 by the user U, and response information R in natural language that is generated by the trained generative model M in response to the prompt P is obtained. Therefore, it is possible to notify the user U of response information R that is statistically valid and linguistically natural in response to the evaluation (evaluation information Y) of the performance of the electronic musical instrument 20 by the user U. In particular, in the first embodiment, various information regarding performance errors (importance Y1, position Y2, type Y3, and detailed content Y4) is included in the evaluation information Y, so that response information R can be generated that includes a variety of information regarding the performance errors by the user U.

[0057] B: Second embodiment A second embodiment will be described. Note that, for elements having the same functions as those in the first embodiment in each of the following exemplary aspects, the same reference numerals as those in the first embodiment will be used, and detailed descriptions of each will be omitted as appropriate.

[0058] 11 is a schematic diagram of a prompt P and response information R in the second embodiment. In addition to the same elements as in the first embodiment, the prompt P in the second embodiment includes identification information G for each performance error and an instruction Z1 to include the identification information G for each performance error in the response information R.

[0059] The identification information G is a code string (tag) for identifying each performance error represented by the evaluation information Y. Specifically, the identification information G includes a code g1 representing the performance error, a code g2 corresponding to the importance Y1, and a number g3 assigned to the performance error. The code g2 is set to "i" when the importance Y1 is high, and is set to "n" when the importance Y1 is low. On the other hand, the instruction Z1 is a natural language representing the addition of the identification information G to a portion of the response information R that points out each performance error.

[0060] 11 illustrates an example of response information R generated from the above-described prompt P. As can be seen from FIG. 11, the prompt P including the instruction Z1 is processed by the generation model M, and as a result, response information R including identification information G is generated. That is, the response information R is generated in accordance with the instruction Z1. Specifically, the identification information G of each performance error is set immediately after the portion of the response information R that points out the performance error.

[0061] 12 is a block diagram illustrating a functional configuration of an information processing system 10 in the second embodiment. The control device 11 in the second embodiment executes a program stored in the storage device 12, thereby functioning as a judgment processing unit 44 in addition to the same functions as those in the first embodiment (an information acquisition unit 41, a response acquisition unit 42, and an operation control unit 43). The judgment processing unit 44 judges whether the response information R is appropriate. The operation of each unit other than the judgment processing unit 44 is the same as in the first embodiment.

[0062] 13 is a flowchart of the evaluation notification process in the second embodiment. The process from the start of the evaluation notification process to the acquisition of response information R (Sb6) is the same as in the first embodiment. When the response information R is acquired, the control device 11 (determination processing unit 44) executes a determination process Sb to determine whether the response information R is appropriate. The determination process Sb in the second embodiment includes a first determination process Sb1 and a second determination process Sb2. Note that the order of the first determination process Sb1 and the second determination process Sb2 may be reversed.

[0063] The first determination process Sb1 is a process for determining whether or not the response information R is appropriate for the prompt P. Specifically, the determination process unit 44 determines whether or not all of the performance errors specified in the prompt P are mentioned in the response information R.

[0064] For example, the determination processing unit 44 compares the prompt P transmitted to the response generation system 30 with the response information R generated from the prompt P, and determines whether or not all of the identification information G of the performance error included in the prompt P is also included in the response information R. If all of the identification information G is included in the response information R, the result of the first determination process Sb1 is positive. On the other hand, if some of the identification information G included in the prompt P is not included in the response information R, the result of the first determination process Sb1 is negative.

[0065] The second determination process Sb2 is a process for determining whether or not a prohibited word is included in the response information R. A prohibited word is a word that is inappropriate from an educational or social point of view. A plurality of prohibited words are stored in advance in the storage device 12. The prohibited words may be set in response to an operation by the user U on the operation device 16.

[0066] Specifically, the judgment processing unit 44 judges whether or not any of the multiple prohibited words stored in the storage device 12 is included in the response information R. If the response information R does not contain a prohibited word, the result of the second judgment process Sb2 is positive (the response information R is appropriate). On the other hand, if the response information R contains a prohibited word, the result of the second judgment process Sb2 is negative (the response information R is inappropriate).

[0067] If the results of both the first determination process Sb1 and the second determination process Sb2 are positive, the response information R is appropriate. Therefore, the control device 11 (operation control unit 43) executes an announcing operation including the reproduction of a guidance voice (Sa7, Sa8) and the speech operation (Sa9) by the guidance character 152, as in the first embodiment. That is, the control device 11 executes the announcing operation when it is determined that the response information R is appropriate. Note that, in generating the voice signal V (Sa7), the identification information G of the response information R is not included in the guidance voice. That is, the identification information G is excluded from the target of voice synthesis by the voice synthesis unit 431.

[0068] On the other hand, if the result of the first determination process Sb1 or the second determination process Sb2 is negative, the response information R is inappropriate. Therefore, regeneration of the response information R is executed. Specifically, the control device 11 (instruction generation unit 421) retransmits the prompt P generated in the immediately preceding step Sa4 to the response generation system 30 from the communication device 13 (Sa5).

[0069] The response generation system 30 generates response information R by processing the prompt P received from the information processing system 10 using a generation model M. Even if the prompt P is the same, the response information R generated by the generation model M changes for each generation. In other words, the response information R generated is different from the response information R determined by the control device 11 to be inappropriate.

[0070] The control device 11 (information receiving unit 422) receives the response information R generated and transmitted by the response generation system 30 through the communication device 13 (Sa6). As can be understood from the above explanation, the transmission of the prompt P (Sb5) and the reception of the response information R (Sb6) are repeated until the response information R is determined to be appropriate in the determination process Sb.

[0071] The second embodiment also achieves the same effects as the first embodiment. In addition, in the second embodiment, the notification operation is executed when the response information R is determined to be appropriate. Therefore, compared with the embodiment in which the notification operation is executed unconditionally, the possibility that inappropriate response information R is notified to the user U can be reduced.

[0072] In the second embodiment, the response information R including the identification information G of the performance error is generated by the generative model M. Therefore, it is possible to easily check whether all the performance errors specified in the prompt P are also included in the response information R by using the identification information G.

[0073] C: Third embodiment FIG. 14 is a schematic diagram of a prompt P and response information R in the third embodiment. In addition to the same elements as in the first embodiment, the prompt P in the third embodiment includes an instruction Z2 for including action information Q in the response information R. The action information Q is information (tag) for identifying an action to be performed in the process of notifying the response information R. Specifically, the instruction Z2 includes a phrase in a natural language representing an action to be performed in the process of notifying the response information R, and action information Q representing the action. The phrase representing an action is, for example, a phrase such as "when staring at a student," "when staring at a piano," "when smiling," and "when clapping." The action information Q is identification information for identifying each action.

[0074] Fig. 14 illustrates an example of response information R generated from the above-described prompt P. As can be seen from Fig. 14, the prompt P including the instruction Z2 is processed by the generation model M, and as a result, response information R including action information Q is generated. That is, the response information R according to the instruction Z2 is generated.

[0075] Specifically, the motion information Q of each motion is set in the portion of the response information R where each motion should be executed. For example, the motion information Q of the motion of staring at the user U is set at the beginning of the response information R. <gaze-student>is set, and immediately after the part of the response information R that points out the performance mistake, the action information Q of staring at the user U is set. <gaze-student>Or, the motion information Q of staring at the piano <gaze-piano>At the end of the response information R, the action information Q of the smiling action is set as follows: <smile>and the motion information of the clapping action Q= <applause>As can be understood from the above description, the response information R in the third embodiment specifies an operation to be performed in the process of notifying the response information R.

[0076] The procedure of the evaluation notification process is the same as that of the first embodiment, except for the control (Sa9) of the guide character 152. In step Sa9 of the evaluation notification process, the control device 11 (display control unit 432) causes the guide character 152 to perform the action specified in the response information R. Specifically, at the time when the guidance voice for the part of the response information R near the action information Q is reproduced, the control device 11 causes the guide character 152 to perform the action represented by the action information Q.

[0077] For example, while the beginning of the guidance voice is being played back, the guidance character 152 stares at the user U (Q= <gaze-student>Immediately after the part of the guidance voice pointing out the performance mistake, the guidance character 152 performs an action of staring at the user U (Q= <gaze-student>) or staring at the piano (Q= <gaze-piano>) is executed. Also, when the guidance voice is played to the end, the guidance character 152 performs a smiling action (Q= <smile>) and clapping (Q= <applause>) is executed. The action information Q of the response information R is not included in the guidance voice. That is, the action information Q is excluded from the target of voice synthesis by the voice synthesis unit 431.

[0078] The third embodiment also achieves the same effects as the first embodiment. Moreover, in the third embodiment, the guide character 152 performs various actions in the notification operation. Therefore, it is possible to diversify the actions of the guide character 152. Note that, in the above explanation, the third embodiment has been explained based on the first embodiment, but the configuration of the second embodiment for determining the appropriateness of the response information R is also applicable to the third embodiment.

[0079] D: Variation Specific modified embodiments added to each of the above-mentioned embodiments are exemplified below. Two or more embodiments selected from the following examples may be appropriately combined as long as they are not mutually contradictory.

[0080] (1) In the above-described embodiments, the attribute X1 of the respondent, the attribute X2 of the user U, and the tone X3 of the response are exemplified as the response condition X. However, the response condition X is not limited to the above examples. For example, the language of the response may be specified as the response condition X. The response information R is expressed in the language specified as the response condition X.

[0081] Also, a catchphrase X4 of a response in response information R may be specified as response condition X. FIG. 15 shows a prompt P that specifies catchphrase X4, "meow," as response condition X, and response information R generated in response to the prompt P. Response information R is generated in which catchphrase X4 specified as response condition X is added to the end of each sentence. The tone of speech X3 and catchphrase X4 may be interpreted as the tone of speech (tone of speech) of the response. Also, as illustrated in FIG. 16, an overall evaluation X5 resulting from evaluation of the performance of the electronic musical instrument 20 by the user U may be included in the response condition X. The response sentence represented by the response information R changes in response to the overall evaluation X5.

[0082] (2) In each of the above-described embodiments, evaluation information Y is generated for each performance mistake made by the user U, but the evaluation information Y is not limited to information representing a performance mistake. For example, evaluation information Y may represent good points (high evaluation points) of the performance made by the user U. According to a prompt P including evaluation information Y with high evaluation points, response information R is generated that praises the performance of the user U. Also, in the evaluation information Y, each of the importance Y1, position Y2, type Y3, and detailed content Y4 may be omitted.

[0083] (3) In the above-described embodiments, the user U selects the response condition X (X1, X2, X3), but the method of setting the response condition X is not limited to the above-described embodiments. For example, the response condition X may be stored in advance in the storage device 12 for each guide character 152. The user U selects one of the multiple guide characters 152 by operating the operation device 16, for example. The response acquisition unit 42 (the instruction generation unit 421) generates a prompt P including the response condition X corresponding to the guide character 152 selected by the user U from the multiple response conditions X stored in the storage device 12. According to the above-described embodiments, the burden on the user U in setting the response condition X can be reduced.

[0084] (4) In the second embodiment, an example was given in which the already-sent prompt P is retransmitted when the response information R is inappropriate, but the operation when the response information R is inappropriate is not limited to the above example. For example, the control device 11 (instruction generating unit 421) may edit the already-sent prompt P according to a predetermined rule and transmit the edited prompt P. For example, when the response information R is determined to be inappropriate because it contains a prohibited word, the instruction generating unit 421 generates a prompt P to which the additional condition of not containing the prohibited word is added. In addition, the control device 11 (response acquiring unit 42) may display a message to the effect that the response information R is inappropriate on the display device 15.

[0085] (5) In the second embodiment, the method of the judgment process Sb for judging the appropriateness of the response information R is not limited to the above-mentioned example. For example, the first judgment process Sb1 using the identification information G may be omitted. That is, in the embodiment in which the judgment processing unit 44 judges the appropriateness of the response information R, the prompt P and the identification information G of the response information R are not essential.

[0086] (6) In the above-mentioned embodiments, the importance Y1 is set according to the type Y3 of the performance error, but the method of setting the importance Y1 is not limited to the above-mentioned examples. Specifically, the information acquisition unit 41 may set the importance Y1 according to the history of the user U's past performances. For example, there is a tendency that performance errors that frequently occur in the user U's performances have a high priority for improvement and are of high importance. Taking the above tendency into consideration, the information acquisition unit 41 sets a high value to the importance Y1 of a performance error that occurred frequently in the user U's past performances.

[0087] (7) In the above-described embodiments, the prompt P includes the response condition X and the evaluation information Y, but the contents of the prompt P are not limited to the above examples. For example, the prompt P may include a form in which one of the response condition X and the evaluation information Y is omitted, or a form in which information other than the response condition X and the evaluation information Y is included in the prompt P.

[0088] (8) In each of the above-described embodiments, a form has been exemplified in which processing such as obtaining response information R (Sa6) is executed after the user U has finished playing a piece of music, but the evaluation of the performance (Sa3), the obtaining of response information R (Sa4 to Sa6), and the notification operation (Sa7 to Sa9) may be executed in parallel with the user U's playing of a piece of music.

[0089] For example, the above process may be performed for each of a plurality of unit periods into which a song is divided on a time axis. The unit periods are, for example, structural periods into which a song is divided according to musical meaning. The structural periods are, for example, periods such as an intro, a verse, a bridge, a chorus, and an outro.

[0090] In addition, an evaluation of the performance may be performed in parallel with the performance of a piece of music by the user U, and each time a performance error occurs (i.e., each time evaluation information Y is generated), response information R may be obtained (Sa4 to Sa6) and an alarm operation (Sa7 to Sa9) may be performed.

[0091] (9) In the above-described embodiments, the guidance character 152 notifies the user U of the response information R. However, the display of the guidance character 152 may be omitted. For example, a configuration may be envisaged in which the response information R is notified to the user U by only reproducing the guidance voice represented by the audio signal V. In other words, the display control unit 432 may be omitted from the operation control unit 43.

[0092] In addition, in each of the above-mentioned embodiments, a form in which a guidance voice represented by the response information R is reproduced has been exemplified, but the method (notification operation) of notifying the user U of the response information R is not limited to the above examples. For example, the notification operation may be an operation of displaying the response sentence represented by the response information R on the display device 15, or an operation of printing the response sentence represented by the response information R by a printing device. An operation of transmitting the response information R to a terminal device owned by the user U is also an example of the notification operation.

[0093] The operation control unit 43 may also display the musical score represented by the music data C on the display device 15, and highlight a portion of the musical score that corresponds to the performance error represented by the response information R. The operation control unit 43 may contrast the notes played by the user U due to the performance error with the correct notes represented by the music data C on the display device 15. The operation control unit 43 may also cause the guide character 152 to perform an operation to indicate the portion of the musical score that corresponds to the performance error. The operation control unit 43 may also play back, by the sound emitting device 14, the performance of the portion of the music represented by the music data C that corresponds to the performance error represented by the response information R. As illustrated above, any operation that notifies the user U of the response information R is included in the "notification operation".

[0094] (10) In each of the above-described embodiments, the information acquisition unit 41 evaluates the performance of the electronic musical instrument 20 by the user U, but the information acquisition unit 41 may receive the evaluation information Y generated by an external device via the communication device 13. For example, in a configuration in which the electronic musical instrument 20 generates the evaluation information Y, the information acquisition unit 41 receives the evaluation information Y transmitted from the electronic musical instrument 20 via the communication device 13. As can be understood from the above explanation, the information acquisition unit 41 is comprehensively expressed as an element that acquires the evaluation information Y regarding the performance of the electronic musical instrument 20 by the user U. The "acquisition" of the evaluation information Y includes not only the "generation" of the evaluation information Y (i.e., the evaluation of the performance) but also the "reception" of the evaluation information Y.

[0095] (11) In each of the above-described embodiments, the response information R is generated by a response generation system 30 separate from the information processing system 10. However, the information processing system 10 may generate the response information R. For example, the information processing system 10 may be equipped with a generative model M. The response acquisition unit 42 generates the response information R by processing the prompt P generated by the instruction generation unit 421 using the generative model M. As can be understood from the above explanation, the response acquisition unit 42 is comprehensively expressed as an element that acquires the response information R. The "acquisition" of the response information R includes not only the "reception" of the response information R but also the "generation" of the response information R.

[0096] (12) In each of the above-described embodiments, evaluation information Y indicating an evaluation of a performance by the user U has been exemplified, but the information (performance information) acquired by the information acquisition unit 41 is not limited to evaluation information Y. For example, text data regarding the status of a performance by the user U is also exemplified as performance information. An example of text data as performance information is, for example, the type of instrument played by the user U or the name of the note played by the user U. In other words, an evaluation of a performance (evaluation information Y) is not essential for generating performance information.

[0097] Further, for example, the performance information may be video data generated by an imaging device by imaging a performance by the user U, or audio data generated by a sound collection device by collecting musical tones generated by the performance by the user U. The information acquisition unit 41 may acquire, as the performance information, a combination of two or more of the above-mentioned types of information.

[0098] As can be understood from the above examples, "performance information" is comprehensively expressed as information related to the performance by user U, and evaluation information Y, text data, video data, and audio data are examples of "performance information."

[0099] (13) In the above-mentioned embodiments, a keyboard instrument is exemplified as the electronic musical instrument 20, but the musical instruments to be evaluated for performance are not limited to keyboard instruments. For example, the above-mentioned embodiments can be applied to any type of musical instrument, such as a string instrument, a wind instrument, or a percussion instrument.

[0100] In each of the above-mentioned embodiments, an electronic musical instrument 20 capable of generating a performance data string D has been exemplified, but the musical instrument to be evaluated for performance is not limited to the electronic musical instrument 20. For example, each of the above-mentioned embodiments can be similarly applied to the evaluation of the performance of a natural musical instrument. In the embodiment in which the user U plays a natural musical instrument, the information acquisition unit 41 generates evaluation information Y by analyzing an acoustic signal generated by picking up radiated sound from the natural musical instrument. Any known technology can be adopted for the performance evaluation.

[0101] Furthermore, the subject of evaluation is not limited to playing an instrument. For example, the above-mentioned embodiments are similarly applicable to a configuration for evaluating the singing of a user U. For example, the information acquisition unit 41 generates evaluation information Y by analyzing an acoustic signal generated by picking up a singing voice. Any known technology may be adopted for the singing evaluation.

[0102] (14) The functions of the information processing system 10 exemplified above are realized by the cooperation of one or more processors constituting the control device 11 and the program stored in the storage device 12, as described above. The program according to the present disclosure can be provided in a form stored in a computer-readable recording medium and installed in a computer. The recording medium is, for example, a non-transitory recording medium, and a good example is an optical recording medium (optical disk) such as a CD-ROM, but also includes any known type of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that the non-transitory recording medium includes any recording medium except for a transient propagating signal, and does not exclude volatile recording media. In addition, in a configuration in which a distribution device distributes a program via a communication network, the storage medium that stores the program in the distribution device corresponds to the non-transitory recording medium described above.

[0103] E: Notes From the above-described exemplary embodiments, the following configurations can be understood, for example.

[0104] An information processing method according to one aspect (aspect 1) of the present disclosure obtains performance information related to a musical instrument performance by a user, obtains response information in natural language corresponding to the performance information, and executes a notification action of notifying the response information by a guide character displayed on a display device. In the above aspect, response information in natural language corresponding to performance information related to a musical instrument performance is obtained, and a notification action of notifying the response information by a guide character is executed. Therefore, the user can easily and appropriately understand information related to the musical instrument performance, as compared to a form in which, for example, a numerical value (e.g., an evaluation score) related to the musical instrument performance by the user is displayed. In addition, the user can be given the sense of being guided (e.g., instructed) on the performance by the guide character.

[0105] "Performance information" is any information related to a user's performance of an instrument. For example, evaluation information showing the results of evaluating a user's performance of an instrument is exemplified as performance information. Evaluation information is, for example, information showing a performance mistake made by a user. Information showing a performance mistake is, for example, information such as the importance of the performance mistake, the position of the performance mistake in the music piece, the type of the performance mistake, and the specific content of the performance mistake. Furthermore, video data generated by filming a performance made by a user, or audio data generated by collecting the sound of the user playing an instrument, are also exemplified as "performance information". "Acquisition" of performance information encompasses both the operation of receiving performance information generated by an external device and the operation of generating performance information by oneself.

[0106] A "guide character" is a virtual object (agent) displayed on a display device. Specific examples of guide characters are virtual living beings such as humans or animals, but inanimate objects such as robots can also be included in the "guide character." "Control of the guide character's behavior" is a control process for making the guide character perform an action corresponding to response information. For example, a process for making the guide character perform an action of uttering response information is exemplified.

[0107] The "notification action" is an output action for notifying the user of the response information. Examples of the "notification action" include a process of displaying the response information on a display device, a process of playing a voice represented by the response information, a process of moving a virtual guide character according to the response information, a process of displaying the score of a section of a piece of music indicated by the response information, a process of playing a performance of the section, and the like.

[0108] In a specific example (aspect 2) of aspect 1, the acquisition of the response information involves generating a prompt including the performance information, and acquiring the response information generated by a trained generative model in response to the prompt. In the above aspect, a prompt including performance information related to a user's performance of an instrument is generated, and natural language response information generated by a trained generative model in response to the prompt is acquired. Therefore, it is possible to notify the user of response information that is statistically valid and linguistically natural in response to information (e.g., an evaluation) related to the user's performance of an instrument.

[0109] A "prompt" is input information that serves as the basis for a trained generative model to generate response information. A "prompt" can also be expressed as an instruction for a trained generative model to generate response information, or an instruction regarding the response information that the generative model should generate. A "prompt" includes both a single prompt and a set of multiple prompts.

[0110] "Natural language response information" is information that expresses a response to a prompt in natural language. Specifically, a natural language response sentence corresponding to the performance information is generated as "response information". For example, a response sentence that instructs or guides the user based on the performance information included in the prompt is exemplified as the response information. "Acquisition" of response information encompasses both the operation of receiving performance information generated by an external device including a generative model, and the operation of generating performance information by oneself using the generative model.

[0111] In a specific example of aspect 2 (aspect 3), the performance information includes evaluation information that indicates an evaluation of the performance. In a specific example of aspect 3 (aspect 4), the performance information includes at least one of the following: a type of performance error that occurred in the performance, the importance of the performance error, the position of the performance error in the music piece, and the content of the performance error. According to the above aspects, various information related to the performance error is included in the performance information, so that response information including various information related to the performance error can be generated.

[0112] In a specific example (Aspect 5) of any one of Aspects 2 to 4, the prompt includes an instruction to include in the response information identification information of performance errors that occurred in the performance. According to the above aspects, the response information including the identification information of the performance errors is generated by the generative model. Therefore, it is possible to easily check whether all the performance errors specified in the prompt are also included in the response information by using the identification information.

[0113] In a specific example (aspect 6) of any one of aspects 2 to 5, the prompt includes an attribute of the user. In the above aspects, since the attribute of the user is included in the prompt, a variety of response information can be generated according to the attribute of the user.

[0114] "User attributes" include, for example, age, sex, generation, occupation, personality (kind personality, assertive personality, etc.), emotions (angry, sad, etc.), etc. Also, the level of skill in playing a musical instrument (beginner, intermediate, advanced) is included in "user attributes."

[0115] In a specific example (Aspect 7) of any one of Aspects 2 to 6, the prompt includes attributes of a virtual respondent who responds with the response information. In the above aspects, since the attributes of the virtual respondent are included in the prompt, a variety of response information can be generated according to the attributes.

[0116] The "respondent's attributes" include, for example, age, sex, generation, occupation, personality (kind personality, assertive personality, etc.), emotions (angry, sad, etc.), etc. In addition, the level of skill in musical instruction (beginner, intermediate, advanced) is also included in the "respondent's attribute information."

[0117] In a specific example (Aspect 8) of any one of Aspects 2 to 7, the prompt includes a tone of voice related to the response information. In the above aspects, since the tone of voice related to the response information is included in the prompt, response information with a variety of tones can be generated.

[0118] "Tone" refers to the tone of the words expressed by the response information. For example, the mood (tone) or emotion of the words is included in "tone". Examples of "atmosphere" include a gentle tone, a stern tone, a stiff tone, a formal tone, and a frank tone. Examples of "emotion" include an angry tone, a sad tone, and a happy tone. "Tone" includes not only the moods and emotions exemplified above, but also "catchphrases" and "dialects". "Catchphrases" are words that are frequently spoken. For example, adding a specific word (such as "It's meow" or "It's woof") to the end of a sentence is an example of a "catchphrase". "Dialects" are regional differences regarding a specific language.

[0119] In a specific example (Aspect 9) of any one of Aspects 1 to 8, the response information specifies an action to be performed in the process of notifying the response information, and in the notifying action, the guide character is caused to perform the action specified in the response information. In the above aspects, the guide character performs various actions in the notifying action. Therefore, it is possible to diversify the actions of the guide character.

[0120] The "motion to be performed in the process of notifying the response information" is, for example, an auxiliary motion of the guide character that is not directly related to the performance information. For example, various motions that a real instructor may perform in the process of teaching a performance, such as an action of staring at the user, an action of staring at the instrument in the virtual space, or an action of smiling, are exemplified as the "motion to be performed in the process of notifying the response information."

[0121] In a specific example (Aspect 10) of any one of Aspects 1 to 9, the appropriateness of the response information is determined, and the notifying action is executed if the response information is determined to be appropriate. In the above aspects, the notifying action is executed if the response information is determined to be appropriate. Therefore, compared to a form in which the notifying action is executed unconditionally, it is possible to reduce the possibility that inappropriate response information is notified to the user.

[0122] "Determining the appropriateness of the response information" includes determining whether the response information is appropriate for the prompt, as well as determining whether the response information is appropriate from an educational or social perspective. The former determination is, for example, determining whether all of the performance information included in the prompt is reflected in the response information. The latter determination is, for example, determining whether the response information contains words that are educationally or socially inappropriate.

[0123] An information processing method according to one aspect (aspect 11) of the present disclosure acquires performance information on a musical instrument played by a user, and generates a prompt including the performance information, where the trained generative model is a prompt for generating response information in natural language. In the above aspect, a prompt including performance information on a musical instrument played by a user is generated. Therefore, response information that is statistically valid and linguistically natural can be generated in response to the performance information on the musical instrument played by the user. In addition, a unique customer experience of acquiring response information that is statistically valid and linguistically natural in response to the performance information can be provided to the user.

[0124] An information processing system according to one embodiment (embodiment 12) of the present disclosure includes an information acquisition unit that acquires performance information regarding a user playing an instrument, a response acquisition unit that acquires response information in natural language corresponding to the performance information, and an operation control unit that causes a guidance character displayed on a display device to perform an announcement action to announce the response information.

[0125] A program according to one embodiment (embodiment 13) of the present disclosure causes a computer system to function as an information acquisition unit that acquires performance information regarding a user's performance of an instrument, a response acquisition unit that acquires response information in natural language corresponding to the performance information, and an operation control unit that causes a guidance character displayed on a display device to perform an announcement action to notify the response information. [Explanation of symbols]

[0126] 100...information system, 200...communication network, 10...information processing system, 11...control device, 12...storage device, 13...communication device, 14...sound emission device, 15...display device, 151...setting screen, 152...guidance character, 16...operation device, 20...electronic musical instrument, 21...key, 30...response generation system, 41...information acquisition unit, 42...response acquisition unit, 421...instruction generation unit, 422...information receiving unit, 43...operation control unit, 431...voice synthesis unit, 432...display control unit, 44...determination processing unit.< / applause> < / smile> < / applause> < / smile>

Claims

1. Acquire performance information regarding a user's performance of a musical instrument; acquiring response information in natural language corresponding to the performance information; A notification operation is performed to notify the response information by a guide character displayed on a display device. An information processing method implemented by a computer system.

2. In obtaining the response information, generating a prompt including the performance information; Obtain the response information generated by the trained generative model in response to the prompt.

2. The information processing method according to claim 1.

3. The performance information includes evaluation information indicating an evaluation regarding the performance. The information processing method according to claim 2.

4. The evaluation information includes at least one of a type of a performance error that occurred during the performance, a level of importance of the performance error, a position of the performance error in the piece of music, and a content of the performance error. The information processing method according to claim 3.

5. The prompt includes an instruction to include in the response information an identification of a performance error that occurred in the performance. The information processing method according to claim 2.

6. The prompt includes an attribute of the user. The information processing method according to claim 2.

7. The prompt includes attributes of a virtual respondent who responds with the response information. The information processing method according to claim 2.

8. The prompt includes a tone for the response information. The information processing method according to claim 2.

9. The response information specifies an operation to be performed in the process of notifying the response information, In the notification action, the guide character is caused to execute an action designated in the response information.

2. The information processing method according to claim 1.

10. Determining whether the response information is appropriate; When the response information is determined to be appropriate, the notification operation is executed.

2. The information processing method according to claim 1.

11. Acquire performance information regarding a user's performance of a musical instrument; The trained generative model generates a prompt for generating response information in natural language, the prompt including the performance information. An information processing method implemented by a computer system.

12. an information acquisition unit that acquires performance information regarding a musical instrument performance by a user; a response acquisition unit that acquires response information in a natural language corresponding to the performance information; an action control unit that causes a guide character displayed on a display device to perform an informing action of informing the response information; An information processing system comprising:

Images