Information processing method and information processing system

Abstract

This information processing system 100 comprises: an information acquisition unit 41 that acquires performance information pertaining to a performance on an electronic musical instrument 20 by a user U; a response acquisition unit 42 that acquires response information R of a natural language according to the performance information; and an operation control unit 43 that executes a notification operation for notifying the response information R by means of a guide character displayed on the display device 15.

Description

Information processing method and information processing system

[0001] The present disclosure relates to a technique for notifying a user of information related to playing a musical instrument.

[0002] Techniques for notifying users of information about musical instrument performances have been proposed. For example, Patent Literature 1 discloses a configuration in which performance information is generated by analyzing video data captured by a user, and an evaluation score is generated by comparing the performance information with reference information.

[0003] Japanese Patent Application Laid-Open No. 2022-053852

[0004] However, there is a problem in that the user cannot easily and appropriately understand information related to his or her own performance simply by displaying an evaluation score as in the technology of Patent Document 1. In consideration of the above circumstances, one aspect of the present disclosure aims to enable the user to easily and appropriately understand information related to the performance of a musical instrument.

[0005] In order to solve the above problems, an information processing method according to one aspect of the present disclosure acquires performance information regarding a user's performance of an instrument, acquires response information in natural language corresponding to the performance information, and performs an announcement operation to announce the response information using a guide character displayed on a display device.

[0006] Another aspect of the present disclosure relates to an information processing method that acquires performance information regarding a user's performance of an instrument, and generates a prompt for a trained generative model to generate natural language response information, the prompt including the performance information.

[0007] An information processing system according to one aspect of the present disclosure includes an information acquisition unit that acquires performance information regarding a user's performance of an instrument, a response acquisition unit that acquires response information in natural language corresponding to the performance information, and an operation control unit that causes a guide character displayed on a display device to perform an announcement operation to announce the response information.

[0008] A program according to one aspect of the present disclosure causes a computer system to function as an information acquisition unit that acquires performance information regarding a user's performance of an instrument, a response acquisition unit that acquires response information in natural language corresponding to the performance information, and an operation control unit that causes a guide character displayed on a display device to perform an announcement action to announce the response information.

[0009] 1 is a block diagram illustrating an example of the configuration of an information system in a first embodiment. FIG. 1 is a block diagram illustrating an example of the configuration of an information processing system. FIG. 2 is a schematic diagram of a setting screen. FIG. 2 is a block diagram illustrating an example of the functional configuration of an information processing system. FIG. 3 is a schematic diagram of a basic character string. FIG. 4 is a schematic diagram of a prompt and response information. FIG. 5 is a schematic diagram of the response information. FIG. 6 is a schematic diagram of a guide character. FIG. 7 is a flowchart of an evaluation notification process. FIG. 8 is a schematic diagram of a prompt and response information in a second embodiment. FIG. 9 is a block diagram illustrating an example of the functional configuration of an information processing system in a second embodiment. FIG. 10 is a flowchart of an evaluation notification process in a second embodiment. FIG. 11 is a schematic diagram of a prompt and response information in a third embodiment. FIG. 12 is a schematic diagram of a prompt and response information in a modified example. FIG. 13 is a schematic diagram of a prompt in a modified example.

[0010] A: First Embodiment Fig. 1 is a block diagram illustrating the configuration of an information system 100 in the first embodiment. The information system 100 is a computer system that guides a user U in playing an electronic musical instrument 20. The information system 100 includes an information processing system 10, the electronic musical instrument 20, and a response generation system 30. The information processing system 10 can communicate with the response generation system 30 via a communication network 200 such as the Internet.

[0011] The response generation system 30 is a server system that generates response information R corresponding to a prompt P. In the first embodiment, the prompt P is an action instruction expressed in natural language. The response information R is text data that expresses a response to the prompt P in natural language.

[0012] The response generation system 30 generates response information R using a trained generative model M. The generative model M is a generative probabilistic model that generates response information R in response to a prompt P. The generative model M has learned the tendency of response information R in response to a prompt P through pre-training. Specifically, the generative model M is an interactive large language model (LLM) that has been trained specifically for natural language processing tasks such as response generation. For example, a natural language processing model realized by a transformer model that utilizes a self-attention mechanism is an example of the generative model M.

[0013] The electronic musical instrument 20 is an input device that accepts musical pieces played by the user U. The electronic musical instrument 20 of the first embodiment is, for example, a keyboard instrument that conforms to the MIDI (Musical Instrument Digital Interface) standard and has a plurality of keys 21 that correspond to different pitches. The electronic musical instrument 20 may also be installed in the information processing system 10.

[0014] The user U plays a piece of music by operating each key 21 in sequence. The electronic musical instrument 20 not only emits musical tones corresponding to the performance by the user U, but also outputs a performance data string D representing the performance to the information processing system 10. The performance data string D is, for example, a time series of event data conforming to the MIDI standard. Specifically, the performance data string D specifies, in time series, pitches corresponding to the keys 21 operated by the user U.

[0015] The information processing system 10 transmits a prompt P corresponding to the result of the evaluation of the performance of the electronic musical instrument 20 by the user U to the response generation system 30, and notifies the user U of the response information R received from the response generation system 30. In other words, the response information R in natural language corresponding to the evaluation of the performance by the user U is notified to the user U.

[0016] 2 is a block diagram of an information processing system 10. The information processing system 10 is realized by an information device such as a smartphone, a tablet terminal, or a personal computer. The information processing system 10 includes a control device 11, a storage device 12, a communication device 13, a sound emitting device 14, a display device 15, and an operation device 16. The information processing system 10 may be realized as a single device, or may be realized as a plurality of devices configured separately from each other.

[0017] The control device 11 is composed of one or more processors that control each element of the information processing system 10. For example, the control device 11 is composed of one or more types of processors such as a central processing unit (CPU), a sound processing unit (SPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0018] The storage device 12 is one or more memories that store programs executed by the control device 11 and various data used by the control device 11. The storage device 12 is configured with a known storage medium such as a magnetic storage medium or a semiconductor storage medium. The storage device 12 may be configured with a combination of multiple types of storage media. Furthermore, a portable storage medium that is detachable from the information processing system 10, or a storage medium (e.g., cloud storage) that the control device 11 can write to or read from via the communication network 200 may be used as the storage device 12.

[0019] The storage device 12 stores music data C representing the music to be performed by the user U. The music data C is data representing the musical score of the music. Specifically, the music data C specifies the pitch and sound duration for each of the multiple notes that make up the music. For example, the music data C is data that conforms to the MIDI standard. Note that the music data C may also specify information such as performance symbols that represent musical expressions.

[0020] The communication device 13 communicates with the response generation system 30 under the control of the control device 11. Specifically, the communication device 13 transmits a prompt P to the response generation system 30 and receives response information R transmitted from the response generation system 30.

[0021] The sound emitting device 14 emits sound waves under the control of the control device 11. The sound emitting device 14 is, for example, a speaker or headphones. Specifically, an audio signal V representing audio corresponding to the response information R (hereinafter referred to as "guidance audio") is supplied to the sound emitting device 14. The sound emitting device 14 reproduces the guidance audio for the user U. For convenience, a D / A converter that converts the audio signal V from digital to analog and an amplifier that amplifies the audio signal V are not shown in the figure. The sound emitting device 14, which is separate from the information processing system 10, may be connected to the information processing system 10 by wire or wirelessly.

[0022] The display device 15 displays an image under the control of the control device 11. The display device 15 is configured with a display panel such as a liquid crystal panel or an organic EL (Electroluminescence) panel. The operation device 16 is an input device that accepts instructions from the user U. The operation device 16 is, for example, an operator operated by the user U, or a touch panel that detects contact by the user U. Note that the display device 15 or the operation device 16, which are separate from the information processing system 10, may be connected to the information processing system 10 by wire or wirelessly.

[0023] The user U can specify a condition related to the response information R (hereinafter referred to as "response condition X") by operating the operation device 16. FIG. 3 is a schematic diagram of a setting screen 151 on which the user U specifies the response conditions X (X1, X2, X3). The setting screen 151 is displayed on the display device 15. The response condition X includes an attribute X1 of a virtual respondent (e.g., a guide character 152 described below) who will execute the response represented by the response information R, an attribute X2 of the user U, and a tone X3 of the response based on the response information R. The setting screen 151 is an image on which a plurality of input fields F (F1, F2, F3) are arranged.

[0024] The input field F1 is an input box in which the user U specifies the attribute X1 of the virtual respondent. The user U operates the operation device 16 to input a character string of the respondent's attribute X1 into the input field F1. For example, the respondent's occupation (piano teacher, etc.) is input as the attribute X1. Note that the user U may select one of a plurality of options prepared in advance as the attribute X1.

[0025] The input field F2 is an input box in which the user U specifies the attribute X2 of the user U. The user U inputs a character string of his / her own attribute X2 into the input field F2 by operating the operation device 16. For example, the age of the user U (elementary school student / junior high school student / high school student / university student / adult, etc.) is input as the attribute X2. Note that the user U may select one of a plurality of options prepared in advance as the attribute X2.

[0026] The input field F3 is an input box in which the user U specifies the tone X3 of the response based on the response information R. The user U operates the operation device 16 to input a character string with the desired tone X3 into the input field F3. For example, the tone of the speech, such as "kindly," "strictly," or "frankly," is specified as the tone X3. Note that the user U may specify one of a plurality of options prepared in advance as the tone X3.

[0027] 4 is a block diagram illustrating an example of the functional configuration of the information processing system 10. The control device 11 executes a program stored in the storage device 12 to realize multiple functions (an information acquisition unit 41, a response acquisition unit 42, and an operation control unit 43).

[0028] The information acquisition unit 41 evaluates the performance of the electronic musical instrument 20 by the user U. Specifically, the information acquisition unit 41 compares the performance data string D supplied from the electronic musical instrument 20 with the music data C stored in the storage device 12, and identifies differences between the two as performance errors by the user U. In other words, a performance error is a difference between the performance by the user U and the standard performance specified by the music data C. The information acquisition unit 41 generates evaluation information Y for each performance error that occurs in the performance by the user U. The evaluation information Y is information that represents an evaluation of the performance of the electronic musical instrument 20 by the user U. Specifically, the evaluation information Y includes the importance Y1 of the performance error, the position Y2 of the performance error, the type Y3 of the performance error, and detailed content Y4 of the performance error.

[0029] The importance Y1 of a performance error is the degree of importance of the performance error. In the first embodiment, the importance Y1 is binary information representing the level of importance. Note that the importance Y1 may also be information representing the level of importance in multiple stages.

[0030] The position Y2 of the performance error is the point in the music piece where the performance error occurred. For example, a point in the music piece where the performance data string D differs from the music piece data C is identified as the position Y2 of the performance error. For example, the position Y2 of the performance error is specified by a bar number in the music piece or the elapsed time from the start of the music piece. Note that in addition to a specific point in the music piece, a specific section in the music piece may also be specified as the position Y2 of the performance error.

[0031] The performance error type Y3 refers to a classification of performance errors by type. For example, "pitch error," "rhythm error," and "hesitation in performance" are designated as the performance error type Y3. The importance Y1 is set, for example, according to the performance error type Y3. For example, the importance Y1 for each performance error type Y3 is stored in advance in the storage device 12.

[0032] The detailed content Y4 of the performance error is the specific content of the performance error made by the user U or the specific content of the criticism of the performance error. For example, the detailed content Y4 of the performance error may be, for example, a performance state such as "playing a 'B' when a 'C' should be played," "earlier than the correct timing," or "must play without stopping." The detailed content Y4 of the performance error may be expressed in natural language, for example.

[0033] 4 acquires response information R in natural language in accordance with evaluation information Y. The response acquisition unit 42 of the first embodiment includes an instruction generation unit 421 and an information receiving unit 422. The instruction generation unit 421 generates a prompt P for the response generation system 30. Specifically, the instruction generation unit 421 generates the prompt P including a response condition X and evaluation information Y.

[0034] The instruction generator 421 generates the prompt P using the basic character string B in FIG. 5 . The basic character string B is a template representing a standard character string for the prompt P and is stored in advance in the storage device 12. The basic character string B includes multiple character strings b1 to b6. A blank is set in each of the multiple character strings b1 to b6. Each blank is a portion of the basic character string B into which variable information (response condition X, evaluation information Y) is inserted. The instruction generator 421 generates the prompt P by inserting the response condition X or the evaluation information Y into each blank in the basic character string B. FIG. 6 illustrates an example of a prompt P generated by the instruction generator 421 using the basic character string B in FIG. 5 .

[0035] The character string b1 of the basic character string B is a portion that represents a condition related to the respondent. The instruction generation unit 421 generates an instruction related to the respondent in the prompt P by inserting the attribute X1 specified by the user U on the setting screen 151 into the blank of the character string b1. As described above, the prompt P in the first embodiment includes the attribute X1 of a virtual respondent.

[0036] The character string b2 of the basic character string B is a portion that represents the conditions of the user U to whom the response information R is to be notified. The instruction generation unit 421 generates an instruction related to the user U in the prompt P by inserting the attribute X2 specified by the user U on the setting screen 151 into the blank of the character string b2. As described above, the prompt P in the first embodiment includes the attribute X2 of the user U.

[0037] The character string b3 of the basic character string B is a part that represents a condition regarding the expression of a response based on the response information R. The instruction generation unit 421 generates an instruction regarding the expression of the response information R in the prompt P by inserting the tone X3 specified by the user U on the setting screen 151 into the blank of the character string b3. As described above, the prompt P in the first embodiment includes the tone X3 related to the response information R.

[0038] Character strings b4 to b6 of basic character string B are portions that represent conditions related to the response content of response information R. The instruction generation unit 421 generates an instruction related to the content of response information R in prompt P by inserting evaluation information Y generated by the information acquisition unit 41 into each blank of character strings b4 to b6.

[0039] The character string b4 is a portion that notifies of a significant performance error. The instruction generating unit 421 generates an instruction regarding a significant performance error in the prompt P by inserting the position Y2, type Y3, and detailed content Y4 specified by each piece of evaluation information Y with a high importance Y1 into each blank in the character string b4.

[0040] Character strings b5 and b6 are portions notifying minor performance errors. The instruction generation unit 421 generates instructions related to minor performance errors in the prompt P by inserting the position Y2, type Y3, and detailed content Y4 specified by each evaluation information Y with a low importance Y1 into the blanks in character strings b5 and b6. Note that the number of performance errors specified in the prompt P is variable.

[0041] The instruction generation unit 421 in Fig. 4 transmits the prompt P generated by the above procedure from the communication device 13 to the response generation system 30. The response generation system 30 processes the prompt P received from the information processing system 10 using the generation model M to generate response information R in natural language, and transmits the response information R to the information processing system 10. The information receiving unit 422 in Fig. 4 receives the response information R transmitted from the response generation system 30 using the communication device 13. That is, the information receiving unit 422 acquires the response information R generated in response to the prompt P by the trained generation model M.

[0042] 6 shows response information R generated from the aforementioned prompt P. Response information R represents a natural language response that reflects response condition X and evaluation information Y included in prompt P. Specifically, response information R represents a natural language response sentence that should be uttered by a respondent with attribute X1 to user U with attribute X2 in tone X3. Response information R also represents a natural language response sentence that explains to user U the performance error represented by evaluation information Y. Specifically, response information R distinguishes between major and minor performance errors (importance Y1), and indicates the location Y2, type Y3, and details Y4 of the performance error.

[0043] In FIG. 6 , the attribute X2 of user U is "adult," whereas FIG. 7 illustrates an example of response information R when the attribute X2 is "elementary school student." When the attribute X2 of user U is "adult" ( FIG. 6 ), response information R is generated in a polite tone, like a conversation between adults. On the other hand, when the attribute X2 is "elementary school student" ( FIG. 7 ), response information R is generated in a friendly and approachable tone, like an adult instructing a child. As described above, in the first embodiment, the attribute X2 of user U is included in the prompt P, so a variety of response information R can be generated according to the attribute X2 of user U. Similarly, in the first embodiment, the attribute X1 of a virtual respondent is included in the prompt P, so a variety of response information R can be generated according to the attribute X1 of the respondent.

[0044] Furthermore, while in Figure 6 the tone of voice X3 is "gentle," Figure 8 illustrates an example of response information R when the tone of voice X3 is "strict." When the tone of voice X3 is "gentle" (Figure 6), response information R is generated in a gentle tone so as not to hurt the self-esteem of user U. On the other hand, when the tone of voice X3 is "strict" (Figure 8), response information R is generated in a strict tone. As described above, in the first embodiment, the tone of voice X3 related to response information R is included in the prompt P, so that response information R in a variety of tones can be generated.

[0045] The operation control unit 43 in Fig. 3 executes an operation (hereinafter referred to as "announcement operation") of notifying the user U of the response information R described above. The announcement operation in the first embodiment is an operation of notifying the user U of the response information R by the guide character 152 (Fig. 9) displayed on the display device 15. The operation control unit 43 in the first embodiment includes a voice synthesis unit 431 and a display control unit 432.

[0046] The voice synthesis unit 431 generates a voice signal V of a guidance voice corresponding to the response information R. The guidance voice is a voice that reads out a response sentence represented by the response information R. Specifically, the voice synthesis unit 431 generates the voice signal V by executing a voice synthesis process on the response information R. Examples of the voice synthesis process include a segment concatenation type voice synthesis process that connects multiple speech segments, or a statistical model type voice synthesis process that uses a statistical model such as a deep neural network or an HMM (Hidden Markov Model). The voice synthesis unit 431 supplies the voice signal V to the sound emission device 14. Therefore, the guidance voice represented by the voice signal V is reproduced from the sound emission device 14.

[0047] The display control unit 432 displays the guide character 152 shown in Fig. 9 on the display device 15. The guide character 152 is an object (agent) placed in the virtual space. Specifically, the guide character 152 is a virtual instructor who instructs the user U on how to play the electronic musical instrument 20 in the virtual space.

[0048] The display control unit 432 causes the guide character 152 to perform a speaking action in parallel with the reproduction of the guidance voice by the sound emitting device 14. The speaking action is an action of changing the shape of the mouth of the guide character 152 in accordance with the audio signal V (so-called lip-syncing). The notification action of the first embodiment includes the speaking action of the guide character 152 and the reproduction of the guidance voice. By performing the speaking action and the reproduction of the guidance voice in parallel, it is possible to make the user U perceive the sensation that the guide character 152 is giving instructions to the user U.

[0049] 10 is a flowchart of a process (hereinafter referred to as a "rating notification process") executed by the control device 11. For example, the rating notification process is started in response to an instruction from the user U via the operation device 16.

[0050] When the evaluation notification process is started, the control device 11 (response acquisition unit 42) displays the setting screen 151 of Figure 3 on the display device 15 (Sa1). The control device 11 (response acquisition unit 42) accepts input of response conditions X (X1, X2, X3) on the setting screen 151 from the user U (Sa2). The response conditions X accepted from the user U are stored in the storage device 12.

[0051] Having input the response condition X, the user U begins playing the electronic musical instrument 20. The control device 11 (information acquisition unit 41) evaluates the user U's performance of the electronic musical instrument 20 (Sa3). Specifically, the control device 11 generates evaluation information Y for each performance error made by the user U during the performance.

[0052] The control device 11 (instruction generator 421) generates a prompt P including a response condition X and evaluation information Y (Sa4). Specifically, the control device 11 generates the prompt P by inserting the response conditions X (X1, X2, X3) and the evaluation information Y (Y2, Y3, Y4) into each blank of the basic character string B stored in the storage device 12.

[0053] The control device 11 (instruction generation unit 421) transmits a prompt P to the response generation system 30 via the communication device 13 (Sa5). That is, the control device 11 requests the response generation system 30 to generate response information R. The control device 11 (information receiving unit 422) receives the response information R generated and transmitted by the response generation system 30 via the communication device 13 (Sa6).

[0054] The control device 11 (voice synthesis unit 431) generates a voice signal V representing a guidance voice corresponding to the response information R (Sa7), and outputs the voice signal V to the sound emission device 14 (Sa8). In parallel with the output of the voice signal V to the sound emission device 14, the control device 11 (display control unit 432) causes the guide character 152 displayed on the display device 15 to perform a speaking action (Sa9). That is, an announcing action including the reproduction of the guidance voice (Sa7, Sa8) and the speaking action (Sa9) of the guide character 152 is executed.

[0055] As described above, in the first embodiment, response information R in natural language corresponding to evaluation information Y regarding the performance of the electronic musical instrument 20 is acquired, and the guide character 152 performs a notification action to notify the response information R. Therefore, compared to a format in which a numerical value (e.g., an evaluation score) that evaluates the performance of the electronic musical instrument 20 by the user U is displayed, the user U can easily and appropriately understand the evaluation regarding the performance of the electronic musical instrument 20. Furthermore, the user U can be provided with a unique customer experience in which he or she feels as if the guide character 152 is giving instructions on how to play the electronic musical instrument 20.

[0056] Furthermore, in the first embodiment, a prompt P is generated that includes evaluation information Y regarding the performance of the electronic musical instrument 20 by the user U, and natural language response information R is obtained that is generated by the trained generative model M in response to the prompt P. Therefore, it is possible to notify the user U of response information R that is statistically valid and linguistically natural in response to the user U's evaluation (evaluation information Y) of the performance of the electronic musical instrument 20. In particular, in the first embodiment, since the evaluation information Y includes various pieces of information regarding the performance error (importance Y1, position Y2, type Y3, and detailed content Y4), it is possible to generate response information R that includes a variety of information regarding the performance error by the user U.

[0057] B: Second Embodiment A second embodiment will be described. Note that, for elements in the following exemplary aspects that have the same functions as those in the first embodiment, the same reference numerals as those in the first embodiment will be used, and detailed descriptions of each element will be omitted as appropriate.

[0058] 11 is a schematic diagram of the prompt P and response information R in the second embodiment. In addition to the same elements as in the first embodiment, the prompt P in the second embodiment includes identification information G for each performance error and an instruction Z1 to include the identification information G for each performance error in the response information R.

[0059] The identification information G is a code string (tag) for identifying each performance error represented by the evaluation information Y. Specifically, the identification information G includes a code g1 representing the performance error, a code g2 corresponding to the importance Y1, and a number g3 assigned to the performance error. The code g2 is set to "i" if the importance Y1 is high, and to "n" if the importance Y1 is low. On the other hand, the instruction Z1 is a natural language that indicates that the identification information G should be added to the part of the response information R that points out each performance error.

[0060] 11 shows an example of response information R generated from the above-described prompt P. As can be seen from FIG. 11, when the prompt P including the instruction Z1 is processed by the generation model M, response information R including identification information G is generated. In other words, response information R is generated in accordance with the instruction Z1. Specifically, the identification information G of each performance error is set immediately after the portion of the response information R that points out the performance error.

[0061] 12 is a block diagram illustrating an example of the functional configuration of an information processing system 10 according to the second embodiment. The control device 11 according to the second embodiment executes a program stored in the storage device 12, thereby functioning as a determination processing unit 44 in addition to the same functions as those of the first embodiment (an information acquisition unit 41, a response acquisition unit 42, and an operation control unit 43). The determination processing unit 44 determines whether the response information R is appropriate. The operation of each unit other than the determination processing unit 44 is the same as that of the first embodiment.

[0062] 13 is a flowchart of the evaluation notification process in the second embodiment. The process from the start of the evaluation notification process to the acquisition of response information R (Sb6) is the same as in the first embodiment. Upon acquiring the response information R, the control device 11 (determination processing unit 44) executes a determination process Sb to determine whether the response information R is appropriate. The determination process Sb in the second embodiment includes a first determination process Sb1 and a second determination process Sb2. Note that the order of the first determination process Sb1 and the second determination process Sb2 may be reversed.

[0063] The first determination process Sb1 is a process for determining whether the response information R is appropriate for the prompt P. Specifically, the determination processing unit 44 determines whether all of the performance errors specified in the prompt P are mentioned in the response information R.

[0064] For example, the determination processing unit 44 compares the prompt P sent to the response generation system 30 with the response information R generated from the prompt P, and determines whether all of the identification information G of the performance error included in the prompt P is also included in the response information R. If all of the identification information G is included in the response information R, the result of the first determination process Sb1 is positive. On the other hand, if some of the identification information G included in the prompt P is not included in the response information R, the result of the first determination process Sb1 is negative.

[0065] The second determination process Sb2 is a process for determining whether or not the response information R contains a prohibited phrase. The prohibited phrase is a phrase that is inappropriate from an educational or social perspective. A plurality of prohibited phrases are stored in advance in the storage device 12. The prohibited phrases may be set in response to an operation by the user U on the operation device 16.

[0066] Specifically, the determination processing unit 44 determines whether any of the multiple prohibited words stored in the storage device 12 is included in the response information R. If the response information R does not include any prohibited words, the result of the second determination process Sb2 is positive (the response information R is appropriate). On the other hand, if the response information R includes a prohibited word, the result of the second determination process Sb2 is negative (the response information R is inappropriate).

[0067] If the results of both the first determination process Sb1 and the second determination process Sb2 are positive, the response information R is appropriate. Therefore, the control device 11 (operation control unit 43) executes a notification operation including the reproduction of a guidance voice (Sa7, Sa8) and the speech action (Sa9) of the guidance character 152, as in the first embodiment. That is, the control device 11 executes the notification operation when it determines that the response information R is appropriate. Note that, in generating the voice signal V (Sa7), the identification information G of the response information R is not included in the guidance voice. That is, the identification information G is excluded from the target of voice synthesis by the voice synthesis unit 431.

[0068] On the other hand, if the result of the first determination process Sb1 or the second determination process Sb2 is negative, the response information R is inappropriate. Therefore, regeneration of the response information R is executed. Specifically, the control device 11 (instruction generation unit 421) retransmits the prompt P generated in the immediately preceding step Sa4 to the response generation system 30 from the communication device 13 (Sa5).

[0069] The response generation system 30 generates response information R by processing the prompt P received from the information processing system 10 using the generation model M. Even if the prompt P is the same, the response information R generated by the generation model M changes for each generation. In other words, the response information R generated is different from the response information R determined to be inappropriate by the control device 11.

[0070] The control device 11 (information receiving unit 422) receives the response information R generated and transmitted by the response generation system 30 via the communication device 13 (Sa6). As can be understood from the above explanation, the transmission of the prompt P (Sb5) and the reception of the response information R (Sb6) are repeated until the response information R is determined to be appropriate in the determination process Sb.

[0071] The second embodiment also achieves the same effects as the first embodiment. Furthermore, in the second embodiment, the notification operation is executed when it is determined that the response information R is appropriate. Therefore, compared to a configuration in which the notification operation is executed unconditionally, the possibility that inappropriate response information R is notified to the user U can be reduced.

[0072] In the second embodiment, the response information R including the identification information G of the performance errors is generated by the generative model M. Therefore, it is possible to easily check whether all the performance errors specified in the prompt P are also included in the response information R by checking the identification information G.

[0073] C: Third Embodiment FIG. 14 is a schematic diagram of a prompt P and response information R in a third embodiment. In addition to the same elements as in the first embodiment, the prompt P in the third embodiment includes an instruction Z2 for including action information Q in the response information R. The action information Q is information (tag) for identifying an action to be performed in the process of notifying the response information R. Specifically, the instruction Z2 includes a natural language phrase representing an action to be performed in the process of notifying the response information R, and action information Q representing the action. Examples of phrases representing actions include "when staring at the student," "when staring at the piano," "when smiling," and "when clapping." The action information Q is identification information for identifying each action.

[0074] 14 shows an example of response information R generated from the above-described prompt P. As can be seen from Fig. 14, when the prompt P including the instruction Z2 is processed by the generation model M, response information R including action information Q is generated. In other words, response information R is generated in accordance with the instruction Z2.

[0075] Specifically, the motion information Q of each motion is set in the portion of the response information R where the motion should be executed. For example, the motion information Q of the motion of staring at the user U is set at the beginning of the response information R. <gaze-student>is set, and immediately after the part of the response information R that points out the performance error, the action information Q of staring at the user U is set. <gaze-student>, or the motion information Q of the motion of staring at the piano = <gaze-piano>At the end of the response information R, the action information Q of the smiling action is set as follows: <smile>and the motion information Q of the clapping motion is <applause>As can be understood from the above description, the response information R of the third embodiment specifies an operation to be performed in the process of notifying the response information R.

[0076] The procedure of the evaluation notification process is the same as that of the first embodiment, except for the control (Sa9) of the guide character 152. In step Sa9 of the evaluation notification process, the control device 11 (display control unit 432) causes the guide character 152 to perform the action specified in the response information R. Specifically, at the time when the guidance voice for the portion of the response information R near the action information Q is played back, the control device 11 causes the guide character 152 to perform the action indicated by the action information Q.

[0077] For example, while the beginning of the guidance voice is being played, the guidance character 152 performs an action of staring at the user U (Q= <gaze-student>Immediately after the part of the guidance voice pointing out the performance error, the guidance character 152 performs an action of staring at the user U (Q= <gaze-student>) or staring at the piano (Q = <gaze-piano>) is performed. When the guidance voice is played to the end, the guidance character 152 performs a smiling action (Q= <smile>) and clapping (Q = <applause>) is executed. The action information Q of the response information R is not included in the guidance voice. That is, the action information Q is excluded from the target of voice synthesis by the voice synthesis unit 431.

[0078] The third embodiment also achieves the same effects as the first embodiment. Furthermore, in the third embodiment, the guide character 152 performs various actions in the notification action. Therefore, it is possible to diversify the actions of the guide character 152. Note that, although the above description of the third embodiment has been based on the first embodiment, the configuration of the second embodiment for determining the appropriateness of the response information R is also applicable to the third embodiment.

[0079] D: Modifications Specific modifications that can be added to the above-mentioned embodiments are exemplified below. Two or more embodiments arbitrarily selected from the following examples may be combined as appropriate within the scope of not mutually contradicting each other.

[0080] (1) In the above embodiments, the response condition X is exemplified by the respondent's attribute X1, the user U's attribute X2, and the tone of the response X3. However, the response condition X is not limited to these examples. For example, the language of the response may be specified as the response condition X. The response information R is expressed in the language specified as the response condition X.

[0081] Furthermore, a catchphrase X4 of a response in response information R may be specified as response condition X. FIG. 15 illustrates a prompt P that specifies catchphrase X4, "meow," as response condition X, and response information R generated in response to the prompt P. Response information R is generated in which catchphrase X4, specified as response condition X, is added to the end of each sentence. The tone of the response (tone of speech) may be interpreted as including tone X3 and catchphrase X4. Furthermore, as illustrated in FIG. 16, an overall rating X5, which is the result of evaluating the performance of the electronic musical instrument 20 by user U, may be included in response condition X. The response sentence represented by response information R changes depending on overall rating X5.

[0082] (2) In the above-described embodiments, evaluation information Y is generated for each performance error made by user U. However, evaluation information Y is not limited to information representing a performance error. For example, evaluation information Y may represent good points (high evaluation points) in user U's performance. A prompt P including evaluation information Y with high evaluation points generates response information R praising user U's performance. Furthermore, the importance Y1, position Y2, type Y3, and detailed content Y4 may be omitted from evaluation information Y.

[0083] (3) In the above-described embodiments, the user U selects the response condition X (X1, X2, X3). However, the method for setting the response condition X is not limited to the above examples. For example, the response condition X for each guide character 152 may be stored in advance in the storage device 12. The user U may select one of the guide characters 152 by operating the operation device 16, for example. The response acquisition unit 42 (instruction generation unit 421) generates a prompt P including the response condition X corresponding to the guide character 152 selected by the user U from the response conditions X stored in the storage device 12. According to the above-described embodiments, the burden on the user U in setting the response condition X can be reduced.

[0084] (4) In the second embodiment, an example was given in which a previously transmitted prompt P was retransmitted when response information R was inappropriate. However, the behavior when response information R was inappropriate is not limited to the above example. For example, the control device 11 (instruction generation unit 421) may edit a previously transmitted prompt P according to a predetermined rule and transmit the edited prompt P. For example, if response information R is determined to be inappropriate because it contains a prohibited phrase, the instruction generation unit 421 generates a prompt P to which the condition that the prohibited phrase not be included is added. Furthermore, the control device 11 (response acquisition unit 42) may display a message indicating that response information R is inappropriate on the display device 15.

[0085] (5) In the second embodiment, the method of the determination process Sb for determining the appropriateness of the response information R is not limited to the above-described example. For example, the first determination process Sb1 using the identification information G may be omitted. That is, in the configuration in which the determination processing unit 44 determines the appropriateness of the response information R, the prompt P and the identification information G of the response information R are not essential.

[0086] (6) In the above-described embodiments, the importance Y1 is set according to the type Y3 of the performance error, but the method for setting the importance Y1 is not limited to the above examples. Specifically, the information acquisition unit 41 may set the importance Y1 according to the user U's past performance history. For example, performance errors that frequently occur in the user U's performances tend to be given a high priority for improvement and are therefore of high importance. Taking the above tendency into consideration, the information acquisition unit 41 sets a high value to the importance Y1 of a performance error that has occurred frequently in the user U's past performances.

[0087] (7) In the above-described embodiments, the prompt P includes a response condition X and evaluation information Y. However, the content of the prompt P is not limited to the above examples. For example, the prompt P may include information other than the response condition X and evaluation information Y, or one of the response condition X and evaluation information Y may be omitted.

[0088] (8) In each of the above-mentioned embodiments, an example has been given in which processing such as obtaining response information R (Sa6) is executed after the user U has finished playing a piece of music. However, the evaluation of the performance (Sa3), the obtaining of response information R (Sa4 to Sa6), and the notification operation (Sa7 to Sa9) may be executed in parallel with the user U's playing of a piece of music.

[0089] For example, the above processing may be performed for each of a plurality of unit periods obtained by dividing a song on a time axis. The unit periods are, for example, structural periods obtained by dividing a song according to musical meaning. The structural periods may be periods such as an intro, verse, bridge, chorus, and outro.

[0090] In addition, an evaluation of the performance may be carried out in parallel with the performance of a piece of music by the user U, and response information R may be obtained (Sa4 to Sa6) and an alert operation (Sa7 to Sa9) may be performed each time a performance error occurs (i.e., each time evaluation information Y is generated).

[0091] (9) In the above-described embodiments, the guide character 152 notifies the user U of the response information R. However, the display of the guide character 152 may be omitted. For example, a configuration is also conceivable in which the response information R is notified to the user U by only reproducing the guidance voice represented by the audio signal V. In other words, the display control unit 432 may be omitted from the operation control unit 43.

[0092] In addition, in each of the above-described embodiments, a guidance voice represented by the response information R is played back, but the method of notifying the user U of the response information R (notification operation) is not limited to the above examples. For example, the notification operation may be an operation of displaying a response sentence represented by the response information R on the display device 15, or an operation of printing the response sentence represented by the response information R using a printing device. An operation of transmitting the response information R to a terminal device owned by the user U is also an example of a notification operation.

[0093] The operation control unit 43 may also display the musical score represented by the music data C on the display device 15 and highlight a portion of the musical score that corresponds to the performance error represented by the response information R. The operation control unit 43 may also display, on the display device 15, the notes played by the user U due to a performance error and the correct notes represented by the music data C in contrast to each other. The operation control unit 43 may also cause the guide character 152 to perform an operation to indicate the portion of the musical score that corresponds to the performance error. The operation control unit 43 may also play, by the sound emitting device 14, a performance of the portion of the music represented by the music data C that corresponds to the performance error represented by the response information R. As illustrated above, any operation that notifies the user U of the response information R is included in the "notification operation."

[0094] (10) In the above-described embodiments, the information acquisition unit 41 evaluates the performance of the electronic musical instrument 20 by the user U. However, the information acquisition unit 41 may receive evaluation information Y generated by an external device via the communication device 13. For example, in a configuration in which the electronic musical instrument 20 generates the evaluation information Y, the information acquisition unit 41 receives the evaluation information Y transmitted from the electronic musical instrument 20 via the communication device 13. As can be understood from the above explanation, the information acquisition unit 41 is comprehensively expressed as an element that acquires the evaluation information Y regarding the performance of the electronic musical instrument 20 by the user U. The "acquisition" of the evaluation information Y includes not only the "generation" of the evaluation information Y (i.e., evaluation of the performance) but also the "reception" of the evaluation information Y.

[0095] (11) In the above-described embodiments, the response information R is generated by a response generation system 30 separate from the information processing system 10. However, the information processing system 10 may generate the response information R. For example, the information processing system 10 may be equipped with a generation model M. The response acquisition unit 42 generates the response information R by processing the prompt P generated by the instruction generation unit 421 using the generation model M. As can be understood from the above explanation, the response acquisition unit 42 is comprehensively expressed as an element that acquires the response information R. The "acquisition" of the response information R includes not only the "reception" of the response information R but also the "generation" of the response information R.

[0096] (12) In the above-described embodiments, evaluation information Y representing an evaluation of the performance by the user U is exemplified. However, the information (performance information) acquired by the information acquisition unit 41 is not limited to evaluation information Y. For example, text data relating to the performance status of the user U is also exemplified as performance information. Examples of text data that are performance information include the type of instrument played by the user U or the name of the note played by the user U. In other words, an evaluation of the performance (evaluation information Y) is not required to generate performance information.

[0097] Further, examples of the performance information include video data generated by an imaging device by imaging a performance by the user U, and audio data generated by a sound collection device by collecting musical sounds produced by the performance by the user U. The information acquisition unit 41 may acquire a combination of two or more of the above-mentioned types of information as performance information.

[0098] As can be understood from the above examples, "performance information" is comprehensively expressed as information regarding the performance by user U, and evaluation information Y, text data, video data, and audio data are examples of "performance information."

[0099] (13) In the above-described embodiments, a keyboard instrument is used as the electronic musical instrument 20, but the instruments that are the subject of performance evaluation are not limited to keyboard instruments. For example, the above-described embodiments can be applied to any type of musical instrument, such as a string instrument, a wind instrument, or a percussion instrument.

[0100] In the above-described embodiments, an electronic musical instrument 20 capable of generating a performance data sequence D has been exemplified, but the musical instrument to be evaluated for performance is not limited to the electronic musical instrument 20. For example, the above-described embodiments can be similarly applied to evaluating the performance of a natural musical instrument. In the embodiment in which the user U plays a natural musical instrument, the information acquisition unit 41 generates evaluation information Y by analyzing an acoustic signal generated by collecting radiated sound from the natural musical instrument. Any known technology may be employed for performance evaluation.

[0101] Furthermore, the subject of evaluation is not limited to playing an instrument. For example, the above-described embodiments can be similarly applied to a configuration for evaluating the singing of a user U. For example, the information acquisition unit 41 generates evaluation information Y by analyzing an acoustic signal generated by picking up the singing voice. Any known technology can be adopted for singing evaluation.

[0102] (14) As described above, the functions of the information processing system 10 illustrated above are realized through cooperation between one or more processors constituting the control device 11 and a program stored in the storage device 12. The program according to the present disclosure may be provided in a form stored on a computer-readable recording medium and installed on a computer. The recording medium may be, for example, a non-transitory recording medium, such as an optical recording medium (optical disk) such as a CD-ROM, but may also include any known type of recording medium, such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium other than a transient, propagating signal, and does not exclude volatile recording media. Furthermore, in a configuration in which a distribution device distributes a program via a communication network, the storage medium that stores the program in the distribution device corresponds to the non-transitory recording medium described above.

[0103] E: Supplementary Note From the above-described exemplary embodiments, the following configurations can be understood, for example.

[0104] An information processing method according to one aspect (aspect 1) of the present disclosure acquires performance information related to a user's performance of a musical instrument, acquires response information in natural language corresponding to the performance information, and executes a notification action to announce the response information by a guide character displayed on a display device. In the above aspect, response information in natural language corresponding to the performance information related to the musical instrument performance is acquired, and a notification action to announce the response information is executed by a guide character. Therefore, compared to a format in which, for example, a numerical value (e.g., an evaluation score) related to the user's performance of the musical instrument is displayed, the user can easily and appropriately understand information related to the musical instrument performance. Furthermore, the user can feel as if the guide character is guiding (e.g., instructing) the performance.

[0105] "Performance information" is any information related to a user's performance of an instrument. For example, evaluation information showing the results of an evaluation of a user's performance of an instrument is an example of performance information. Evaluation information is, for example, information showing a performance error made by a user. Information showing a performance error includes, for example, the importance of the performance error, the position of the performance error in the music, the type of the performance error, and the specific content of the performance error. Furthermore, video data generated by filming a performance made by a user or audio data generated by capturing the sound of a user playing an instrument is also exemplified as "performance information." "Acquisition" of performance information encompasses both the act of receiving performance information generated by an external device and the act of generating performance information by oneself.

[0106] A "guide character" is a virtual object (agent) displayed on a display device. Specific examples of guide characters include virtual living beings such as humans or animals, but the term "guide character" can also include inanimate objects such as robots. "Control of the guide character's actions" is a control process that causes the guide character to perform an action in response to response information. For example, a process that causes the guide character to perform an action of uttering response information is exemplified.

[0107] The "notification action" is an output action for notifying the user of the response information. Examples of the "notification action" include a process of displaying the response information on a display device, a process of playing back the sound represented by the response information, a process of moving a virtual guide character in response to the response information, a process of displaying the score of a section of a song indicated by the response information, and a process of playing back the performance of that section.

[0108] In a specific example (Aspect 2) of Aspect 1, acquiring the response information involves generating a prompt including the performance information, and acquiring the response information generated by a trained generative model in response to the prompt. In the above aspect, a prompt including performance information related to the user's performance of an instrument is generated, and natural language response information generated by a trained generative model in response to the prompt is acquired. Therefore, it is possible to notify the user of statistically valid and linguistically natural response information in response to information (e.g., an evaluation) related to the user's performance of the instrument.

[0109] A "prompt" is input information that serves as the basis for a trained generative model to generate response information. A "prompt" can also be expressed as an instruction for a trained generative model to generate response information, or an instruction regarding the response information that the generative model should generate. A "prompt" includes both a single prompt and a set of multiple prompts.

[0110] "Natural language response information" is information that expresses a response to a prompt in natural language. Specifically, a natural language response sentence corresponding to performance information is generated as "response information." For example, a response sentence that instructs or guides the user based on the performance information included in the prompt is an example of response information. "Acquisition" of response information encompasses both the operation of receiving performance information generated by an external device including a generative model and the operation of generating performance information by oneself using the generative model.

[0111] In a specific example of Aspect 2 (Aspect 3), the performance information includes evaluation information that indicates an evaluation of the performance. Furthermore, in a specific example of Aspect 3 (Aspect 4), the performance information includes at least one of the following: the type of performance error that occurred in the performance; the importance of the performance error; the position of the performance error in the music; and the content of the performance error. According to these aspects, various types of information related to the performance error are included in the performance information, so that response information including a variety of information related to the performance error can be generated.

[0112] In a specific example (Aspect 5) of any of Aspects 2 to 4, the prompt includes an instruction to include identification information of performance errors that occurred during the performance in the response information. According to the above aspect, the response information including the identification information of the performance errors is generated by a generative model. Therefore, it is possible to easily check whether all of the performance errors specified in the prompt are also included in the response information using the identification information.

[0113] In a specific example (Aspect 6) of any one of Aspects 2 to 5, the prompt includes an attribute of the user. In the above aspects, since the attribute of the user is included in the prompt, a variety of response information can be generated according to the attribute of the user.

[0114] "User attributes" include, for example, age, gender, generation, occupation, personality (kind personality, assertive personality, etc.), emotions (angry, sad, etc.), etc. Also included in "user attributes" is the level of skill in playing a musical instrument (beginner, intermediate, advanced).

[0115] In a specific example (Aspect 7) of any one of Aspects 2 to 6, the prompt includes attributes of a virtual respondent who will respond with the response information. In the above aspects, since the attributes of the virtual respondent are included in the prompt, a variety of response information can be generated according to the attributes.

[0116] The "respondent's attributes" include, for example, age, sex, generation, occupation, personality (kind personality, assertive personality, etc.), emotions (angry, sad, etc.), etc. The "respondent's attribute information" also includes the level of skill in musical instruction (beginner, intermediate, advanced).

[0117] In a specific example (Aspect 8) of any one of Aspects 2 to 7, the prompt includes a tone related to the response information. In the above aspects, since the tone related to the response information is included in the prompt, response information with a variety of tone can be generated.

[0118] "Language tone" refers to the tone of the words and phrases expressed in the response information. For example, the mood (tone) or emotion of the words and phrases is included in "language tone." Examples of "mood" include a gentle tone, a stern tone, a formal tone, and a frank tone. Examples of "emotion" include an angry tone, a sad tone, and a happy tone. "Language tone" includes not only the moods and emotions exemplified above, but also "catchphrases" and "dialects." "Catchphrases" are words that are frequently uttered. For example, adding a specific word (such as "It's meow" or "It's woof") to the end of a sentence is an example of a "catchphrase." "Dialects" are regional differences in a specific language.

[0119] In a specific example (Aspect 9) of any one of Aspects 1 to 8, the response information specifies an action to be performed in the process of notifying the response information, and the notification action causes the guide character to perform the action specified in the response information. In the above aspects, the guide character performs various actions in the notification action. Therefore, it is possible to diversify the actions of the guide character.

[0120] The "actions to be performed in the process of notifying the response information" are, for example, auxiliary actions of the guide character that are not directly related to the performance information. For example, various actions that a real instructor might perform in the process of teaching a performance, such as gazing at the user, gazing at the instrument in the virtual space, or smiling, are examples of "actions to be performed in the process of notifying the response information."

[0121] In a specific example (Aspect 10) of any of Aspects 1 to 9, the appropriateness of the response information is determined, and if it is determined that the response information is appropriate, the notifying action is executed. In the above aspects, if it is determined that the response information is appropriate, the notifying action is executed. Therefore, compared to an aspect in which the notifying action is executed unconditionally, the possibility that inappropriate response information is notified to the user can be reduced.

[0122] "Determining the appropriateness of response information" includes determining whether the response information is appropriate for the prompt, as well as whether the response information is appropriate from an educational or social perspective. The former determination is, for example, determining whether all performance information included in the prompt is reflected in the response information. The latter determination is, for example, determining whether the response information contains words or phrases that are educationally or socially inappropriate.

[0123] An information processing method according to one aspect (aspect 11) of the present disclosure acquires performance information related to a musical instrument performance by a user, and generates a prompt for a trained generative model to generate natural language response information, the prompt including the performance information. In the above aspect, a prompt including performance information related to a musical instrument performance by a user is generated. Therefore, response information that is statistically valid and linguistically natural can be generated in response to the performance information related to the musical instrument performance by the user. Furthermore, a unique customer experience can be provided to the user, in which statistically valid and linguistically natural response information is obtained in response to the performance information.

[0124] An information processing system according to one aspect (aspect 12) of the present disclosure includes an information acquisition unit that acquires performance information regarding a user's performance of an instrument, a response acquisition unit that acquires response information in natural language corresponding to the performance information, and an operation control unit that causes a guide character displayed on a display device to perform an announcement action to announce the response information.

[0125] A program according to one aspect (aspect 13) of the present disclosure causes a computer system to function as an information acquisition unit that acquires performance information regarding a user's performance of an instrument, a response acquisition unit that acquires response information in natural language corresponding to the performance information, and an operation control unit that causes a guide character displayed on a display device to perform an announcement action to announce the response information.

[0126] 100...information system, 200...communication network, 10...information processing system, 11...control device, 12...storage device, 13...communication device, 14...sound emission device, 15...display device, 151...setting screen, 152...guide character, 16...operation device, 20...electronic musical instrument, 21...key, 30...response generation system, 41...information acquisition unit, 42...response acquisition unit, 421...instruction generation unit, 422...information receiving unit, 43...operation control unit, 431...voice synthesis unit, 432...display control unit, 44...determination processing unit.< / applause> < / smile> < / applause> < / smile>

Claims

1. An information processing method implemented by a computer system, which acquires performance information relating to a user's performance of a musical instrument, acquires response information in a natural language corresponding to the performance information, and performs a notification operation to notify the response information by a guide character displayed on a display device.

2. The information processing method according to claim 1, wherein acquiring the response information comprises generating a prompt including the performance information, and acquiring the response information generated by a trained generative model in response to the prompt.

3. The information processing method according to claim 2, wherein the performance information includes evaluation information indicating an evaluation regarding the performance.

4. The information processing method according to claim 3, wherein the evaluation information includes at least one of the type of performance error that occurred during the performance, the importance of the performance error, the position of the performance error in the music piece, and the content of the performance error.

5. The information processing method according to claim 2, wherein the prompt includes an instruction to include in the response information an identification information of a performance error that occurred during the performance.

6. The information processing method according to claim 2, wherein the prompt includes an attribute of the user.

7. The information processing method according to claim 2, wherein the prompt includes attributes of a virtual respondent who responds with the response information.

8. The information processing method of claim 2, wherein the prompt includes a tone related to the response information.

9. The information processing method according to claim 1, wherein the response information specifies an action to be performed in the process of notifying the response information, and the notification action includes having the guide character perform the action specified in the response information.

10. The information processing method according to claim 1, further comprising the steps of: judging whether the response information is appropriate; and executing the notification action when the response information is judged to be appropriate.

11. An information processing method implemented by a computer system, which acquires performance information relating to a user's performance of a musical instrument, and generates prompts including the performance information, for generating natural language response information using a trained generative model.

12. An information processing system comprising: an information acquisition unit that acquires performance information regarding a user playing a musical instrument; a response acquisition unit that acquires response information in natural language corresponding to the performance information; and an operation control unit that causes a guide character displayed on a display device to perform an announcement operation to announce the response information.

Images