Electronic musical instrument, electronic musical instrument control method, and program

Abstract

To enable a user to practice music with an electronic musical instrument having no display device, in an easy-to-understand manner without letting the user lose motivation or get bored.SOLUTION: When key pressing timing and a key pressing pitch in performance practice correctly correspond to automatically-reproduced timing information and pitch information, automatically-reproduced lyric information and the pitch information are input to an acoustic model unit 301 at the key pressing timing, and a synthesis filter unit 305 formed based on spectral information 309 from the acoustic model unit 301 is excited by a sound source signal generated based on sound source information 310 output from the acoustic model unit, thereby outputting singing voice output data 217. At this time, a formant interpolation processing unit 306 and a noise superposing unit 307 interpolate the spectral information 309 and a noise mixture ratio 314 among characters of a plurality of people in accordance with a scoring intermediate result of the performance practice, thereby changing voice property of the singing voice output data 217.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an electronic musical instrument that reproduces a singing voice in response to the operation of a controller such as a keyboard, a method for controlling an electronic musical instrument, and a program. [Background technology]

[0002] There is known a music training system for keyboard instruments that allows a user to take lessons on an electronic instrument based on music data that indicates the pitch to be produced and the timing of producing that sound, and then inputs signals from the electronic instrument to evaluate and score the user on various items (for example, the system described in Patent Document 1).

[0003] In recent years, application software for smart devices (hereinafter referred to as "apps") has become known that is connected to an electronic keyboard instrument via MIDI (Musical Instrument Digital Interface) and adds points by pressing the corresponding key on the keyboard of the electronic keyboard instrument when a bar (piano roll) that falls from the top of the screen reaches the keyboard at the bottom of the screen as the song progresses. Some of these apps can also display the progress of the scoring during the song and the score results after the song ends, allowing users to practice playing without getting bored (for example, the app described in Non-Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 10-187021 [Non-patent literature]

[0005] [Non-Patent Document 1] “Chordana Play for Piano:MIDI Player”, Casio Computer Co., Ltd., [Searched on July 20, 2020], Internet<URL https: / / web.casio.com / app / ja / piano / > Summary of the Invention [Problem to be solved by the invention]

[0006] However, the above-mentioned conventional technologies are based on the premise that the progress of a user's musical instrument performance and scoring results will be displayed on a display screen, and such a display is only possible with a display device such as a high-definition display mounted on an electronic musical instrument or a display on a smart device.

[0007] However, in general electronic musical instruments, particularly low-priced electronic musical instruments aimed at beginners that are the target of such performance practice functions, the display devices mounted on them are often small and low-resolution, or in many cases, no display device is provided at all. Therefore, even if such a performance practice function can be implemented on a stand-alone electronic musical instrument, it has been difficult to provide the user with a function that is easy to understand and that will not tire the user.

[0008] Furthermore, even if the performance progress is shown on the display device, it is conceivable that beginners, in particular, may be so distracted by their own performance that they have no time to look at the display device other than the keyboard.

[0009] To provide an electronic musical instrument having a low-quality display device or no display device that can allow a user to practice playing in an easy-to-understand manner without losing motivation or becoming bored, and to provide a method of transmitting information regarding playing practice to a user even without a display device. [Means for solving the problem]

[0010] An electronic musical instrument as one example of an embodiment includes a performance information acquisition means for acquiring performance information of a performer, a performance evaluation means for evaluating the performer's performance multiple times at different timings while a piece of music is being played based on performance guide data and the performance information, which includes at least lyric information, pitch information, and timing information, a singing voice production means for singing the lyrics based on the performance information and the lyric information, and a voice quality change means for, when the evaluation of the performance evaluation means has changed from the previous evaluation, changing the voice quality of the singing voice in response to the changed evaluation. Effect of the Invention

[0011] According to the present invention, even with an electronic musical instrument that has only a low-quality display device or no display device, it is possible to allow a user to practice playing in an easy-to-understand manner without losing motivation or becoming bored, and it is also possible to transmit information regarding playing practice to a user even without a display device. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram showing an example of the appearance of an embodiment of an electronic keyboard instrument; [Diagram 2] 1 is a block diagram showing an example of a hardware configuration of an embodiment of a control system for an electronic keyboard instrument; [Diagram 3] FIG. 2 is a block diagram showing a configuration example of a voice synthesis LSI. [Figure 4] FIG. [Diagram 5] FIG. 2 is a diagram illustrating an example of a data configuration according to an embodiment. [Figure 6] 13 is a flowchart showing an example of the overall processing of a performance lesson. [Figure 7] 13 is a flowchart showing a detailed example of a lesson process. [Figure 8] 13 is a flowchart showing a detailed example of an automatic playback process. [Figure 9] 13 is a flowchart showing a detailed example of a performance guide process. [Figure 10] 13 is a flowchart showing a detailed example of a key press / release process. [Figure 11]13 is a flowchart showing a detailed example of a scoring process. [Figure 12] 13 is a flowchart showing a detailed example of a voice quality updating process. [Figure 13] 13 is a flowchart showing an example of a voice quality change process. [Figure 14] 13 is a flowchart showing a detailed example of a noise mixing ratio interpolation process and a formant interpolation process. [Figure 15] FIG. 13 is a block diagram showing another example of the configuration of the vocalization model unit 303 in the voice synthesis unit 300 in the voice synthesis LSI. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing an example of the appearance of an embodiment 100 of an electronic keyboard instrument. The electronic keyboard instrument 100 includes a keyboard 101 consisting of a plurality of keys as operators, a first switch panel 102 for instructing various settings such as specifying the volume, setting the tempo for automatic lyrics playback, and starting automatic lyrics playback, and a second switch panel 103 for selecting a song and selecting an instrument tone. Each key of the keyboard 101 includes an LED (Light Emitting Diode) 104. This LED 104 lights up at maximum brightness when the key it includes is the key that should be specified next during automatic lyrics playback, and lights up at half the maximum brightness when the key is the key that should be specified after the next key during automatic lyrics playback. Furthermore, the electronic keyboard instrument 100 includes a speaker (not shown) on the back, side, or rear part, etc., that emits musical tones generated by playing.

[0014] Fig. 2 is a diagram showing an example of a hardware configuration of an embodiment of a control system 200 of the electronic keyboard instrument 100 of Fig. 1. In Fig. 2, the control system 200 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a sound source LSI (Large Scale Integrated Circuit) 204, a voice synthesis LSI 205, a key scanner 206 to which the keyboard 101 of Fig. 1, the first switch panel 102 and the second switch panel 103 are connected, an LED controller 207 to which the LEDs 104 of each key on the keyboard 101 of Fig. 1 are connected, and a network interface 208 for exchanging MIDI data with an external network, all of which are connected to a system bus 209. A timer 210 for controlling the sequence of automatic reproduction of vocal data is also connected to the CPU 201. Furthermore, the musical sound output data 218 and the singing voice output data 217 outputted from the sound source LSI 204 and the voice synthesis LSI 205, respectively, are converted into an analog musical sound output signal and an analog singing voice output signal by D / A converters 211 and 212. The analog musical sound output signal and the analog singing voice output signal are mixed in a mixer 213, and the mixed signal is amplified in an amplifier 214 and then outputted from a speaker or an output terminal (not shown).

[0015] The CPU 201 executes a control program stored in the ROM 202 while using the RAM 203 as a work memory, thereby executing the control operation of the electronic keyboard instrument 100 in Fig. 1. The ROM 202 also stores the control program and various control data, as well as performance guide data including lyric data, which will be described later.

[0016] The CPU 201 is equipped with a timer 210 used in this embodiment, which counts the progress of automatic reproduction of performance guide data in the electronic keyboard instrument 100, for example.

[0017] The sound source LSI 204 reads out musical tone waveform data, for example, from a waveform ROM (not shown) in accordance with sound generation control instructions from the CPU 201, and outputs the data to the D / A converter 211. The sound source LSI 204 has the ability to generate sound simultaneously in a maximum of 256 voices.

[0018] When the voice synthesis LSI 205 receives from the CPU 201 lyric information, which is text data of lyrics, and pitch information regarding pitch as singing voice data 215, the voice synthesis LSI 205 synthesizes singing voice output data 217, which is voice data of the corresponding singing voice, and outputs it to the D / A converter 212.

[0019] A key scanner 206 constantly scans the key-on / key-off states of the keyboard 101 in FIG. 1 and the switch operation states of the first switch panel 102 and the second switch panel 103, and issues an interrupt to the CPU 201 to notify it of state changes.

[0020] The LED controller 207 is an IC (integrated circuit) that controls the display state of each of the LEDs 104 provided on each key on the keyboard 101 in FIG.

[0021] 3 is a block diagram showing an example of the configuration of the voice synthesis unit 300 in this embodiment. The voice synthesis unit 300 is built into the electronic keyboard instrument 100 as one function executed by the voice synthesis LSI 205 in FIG.

[0022] The voice synthesis unit 300 inputs singing voice data 215 including lyric information and pitch information instructed by the CPU 201 in Fig. 2, thereby synthesizing and outputting singing voice output data 217. At this time, the processor of the voice synthesis unit 300 executes vocalization processing for outputting singing voice output data 217 obtained by inferring the singing voice of a singer based on spectrum information 309 and sound source information 310 output by the acoustic model unit 301 in response to the singing voice data 215 including lyric information and pitch information input by the CPU 201, for the acoustic model set in the acoustic model unit 301. The voice synthesis unit 300 is implemented based on the technology described in the following patent documents, for example.

[0023] (Patent document): Patent No. 6610714 The details of the operation of the speech synthesis unit 300 are disclosed in the above-mentioned patent documents, but the outline of the operation will be described below.

[0024] The speech synthesis unit 300 includes a text analysis unit 302 , an acoustic model unit 301 , a speech model unit 303 , a formant interpolation unit 306 , and a noise superimposition unit 307 .

[0025] The voice synthesis unit 300 executes a statistical voice synthesis process, which synthesizes singing voice output data 321 corresponding to singing voice data 215 including lyric information, which is the text of lyrics, and sound source information 310, by predicting the data using a statistical model called an acoustic model set in the acoustic model unit 301.

[0026] The text analysis unit 302 receives the singing voice data 215 including lyric information related to the phonemes, pitches, etc. of the lyrics specified by the CPU 201 in Fig. 2 and analyzes the data. As a result, the text analysis unit 302 analyzes and outputs a linguistic feature sequence 308 expressing the phonemes, parts of speech, words, etc. corresponding to the singing voice data 215.

[0027] The acoustic model unit 301 receives the language feature sequence 308 and the pitch information in the singing data 215, and estimates and outputs the corresponding spectral information 309 and voicing source information 310. That is, based on the language feature sequence 308 and the pitch information in the singing data 215 received from the text analysis unit 302, the acoustic model unit 301 uses an acoustic model set as a learning result by machine learning, for example, to output estimates of the spectral information 309 and the voicing source information 310 that maximize the generation probability.

[0028] The vocalization model unit 303 receives the spectrum information 309 and the sound source information 310 to generate singing voice output data 321 corresponding to the singing voice data 215 including the lyric information and the pitch information designated by the CPU 201. The singing voice output data 321 is output from the D / A converter 212 in Fig. 2 via the mixer 213 and the amplifier 214, and is emitted from a speaker (not shown).

[0029] The acoustic feature quantity output by the acoustic model unit 301 includes spectrum information 309 that models the human vocal tract, and sound source information 310 that models the human vocal cords. For example, parameters of the spectrum information 309 can be line spectral pairs (LSP) or line spectral frequencies (LSF), which can efficiently model a plurality of formant frequencies that are human vocal tract characteristics. For the sound source information 310, a fundamental frequency (F0) and a power value that indicate the pitch frequency of human voice can be used. The voicing model unit 303 includes a sound source generation unit 304 and a synthesis filter unit 305. The sound source generating unit 304 is a part that models the human vocal cords, and by sequentially inputting a series of sound source information 310 input from the acoustic model unit 301, generates a sound source signal consisting of, for example, a pulse train (in the case of a voiced phoneme) that is periodically repeated at the fundamental frequency (F0) and power value included in the sound source information 310, or a white noise (in the case of an unvoiced phoneme) having a power value included in the sound source information 310, or a signal that is a mixture of these. The synthesis filter unit 305 is a part that models the human vocal tract, and forms a digital filter that models the vocal tract based on a series of spectrum information 309 that is sequentially input from the acoustic model unit 301. The sound source signal input from the sound source generating unit 304 is used as an excitation source signal to excite the digital filter, and filter output data 313 of a digital signal that is the basis of singing voice output data 321 is output.

[0030] The sampling frequency for the singing voice output data 217 is, for example, 16 KHz (kilohertz). When LSF parameters obtained by LSP analysis are adopted as parameters of the spectrum information 309, the update frame period is, for example, 5 milliseconds. Furthermore, in the case of LSF analysis, the analysis window length is, for example, 25 milliseconds, the window function is, for example, a Blackman window, and the analysis order is, for example, 10th order.

[0031] The operation of this embodiment will be described in outline with the configuration of Fig. 2 and Fig. 3. First, the CPU 201 operates as a performance guide means for guiding the performer in playing a piece of music based on performance guide data including at least lyric information, pitch information, and timing information. Specifically, in Fig. 2, the CPU 201 executes an automatic playback process in which the CPU 201 sequentially reads out a series of sets of performance guide data including at least lyric information, pitch information, and timing information for automatic playback stored in the ROM 202, which is a memory, and automatically plays back the lyric information and pitch information included in the set of performance guide data at a timing corresponding to the timing information included in the set of performance guide data. Details of this automatic playback process will be described later with reference to the flow chart of Fig. 8.

[0032] At this time, the CPU 201 executes a performance guide process that guides the user to perform a performance lesson (performance practice) by pressing a key on the keyboard 101 corresponding to the pitch information to be automatically reproduced in synchronization with the automatic reproduction by specifying the key on the keyboard 101 corresponding to the pitch information to be automatically reproduced. More specifically, in this performance guide process, the CPU 201 causes the LED 104 of the key (operator) corresponding to the pitch information to be automatically reproduced next to be lit with high brightness, for example, maximum brightness, and causes the LED 104 of the key corresponding to the pitch information to be automatically reproduced after that to be lit with low brightness, for example, half the maximum brightness, in synchronization with the timing of the automatic reproduction, as shown as the key with two LEDs 104 lit in FIG. 9. The details of this performance guide process will be described later with reference to the flowchart in FIG. 9.

[0033] Next, the CPU 201 operates as a performance information acquiring means for acquiring performance information of the performer. Specifically, the CPU 201 acquires the performance operations of the performer pressing or releasing the keys on the keyboard 101 in FIG. 1 according to the performance guide.

[0034] Furthermore, the CPU 201 operates as a performance evaluation means for evaluating the performance of the performer from time to time while the music piece is being played, based on the performance guide data and the performance information. Specifically, the CPU 201 executes a scoring process for scoring the performance lesson by comparing the key pressing timing (operation timing) and key pressing pitch (operation pitch) in the performance lesson with the automatically reproduced timing information and pitch information. The details of this scoring process will be described later with reference to the flowchart in FIG. 11.

[0035] The CPU 201 operates as a singing voice producing means for singing lyrics based on the performance information and the lyrics information. Specifically, when the timing (operation timing) of pressing a key on the keyboard 101 in the performance lesson and the key pressing pitch (operation pitch) of the key correctly correspond to the timing information and pitch information for automatic reproduction, the CPU 201 inputs the lyrics information and pitch information to be automatically reproduced at the key pressing timing as singing voice data 215 to the acoustic model unit 301 via the text analysis unit 302 in Fig. 3, and outputs filter output data 313 by exciting a digital filter of the synthesis filter unit 305 formed based on the spectrum information 309 output from the acoustic model unit 301 by a sound source signal output from the sound source generation unit 304 in which sound source information 310 output from the acoustic model unit 301 is set, and causes the filter output data 313 to be output as the singing voice output data 217 in Fig. 2.

[0036] At this time, the CPU 201 operates as a voice quality changing means for changing the voice quality of the singing voice in response to the performance evaluation. Specifically, the CPU 201 in Fig. 2 and the formant interpolation processing unit 306 and the noise superimposition unit 307 in the voice synthesis unit 300 in Fig. 3 execute a voice quality changing process for changing the voice quality of the singing voice output data 217 output in the vocalization process in response to the intermediate scoring results in the scoring process described above.

[0037] At this time, the CPU 201, which operates as a voice quality changing means, interpolates between a plurality of voice qualities corresponding to a plurality of specific performance evaluations at a ratio according to the performance evaluation during the progression of the music piece. Also, the CPU 201 changes the voice quality by, for example, changing the ratio of the formant components of the human voice and the noise components mixed with the human voice.

[0038] More specifically, in the above-mentioned voice quality change process, the CPU 201 first calculates the practice progress data 311 indicating the degree of progress of the performance lesson according to the intermediate result of the scoring process. Then, the formant interpolation processing unit 306 in Fig. 3 executes an interpolation process at a ratio according to the practice progress data 311 given by the CPU 201 between one or more sets of spectrum information 309 output from the acoustic model unit 301 corresponding to the lyric information in the singing voice data 215 in Fig. 3 that has a pleasant voice quality including a female voice and is automatically reproduced, which corresponds to a case where the intermediate result of the scoring is good, and one or more sets of spectrum information 309 output from the acoustic model unit 301 corresponding to the lyric information in the singing voice data 215 that has a harsh voice quality including a male voice and is automatically reproduced, which corresponds to a case where the intermediate result of the scoring is bad, thereby calculating the target spectrum information 312 and inputting it to the synthesis filter unit 305 in the vocalization model unit 303.

[0039] In the above-mentioned voice quality modification process, in addition to the operation of the formant interpolation processing unit 306, the noise mixing ratio interpolation processing unit 316 in FIG. 3 executes an interpolation process in a ratio corresponding to the practice progress data 311 provided by the CPU 201 between one or more sets of noise mixing ratios output from the acoustic model unit 301 corresponding to lyric information in the singing voice data 215 in FIG. 3 which has a pleasant voice quality including a female voice and is automatically reproduced, corresponding to a case where the intermediate scoring result is good, and one or more sets of noise mixing ratios output from the acoustic model unit 301 corresponding to lyric information in the singing voice data 215 which has a harsh voice quality including a male voice and is automatically reproduced, corresponding to a case where the intermediate scoring result is bad, thereby calculating a target noise mixing ratio 317 and inputting it to the noise superimposition unit 307. Then, the noise superimposition unit 307 generates noise data 315 having an amplitude value obtained by multiplying the maximum amplitude value of the singing voice output data 217 by the target noise mixing ratio 317 calculated by the noise mixing ratio interpolation processing unit 316, mixes it with the filter output data 313 output from the synthesis filter unit 305, and outputs it as the singing voice output data 217.

[0040] The above-mentioned two voice quality changing processes realize a function (hereinafter referred to as "morphing function") that gradually changes the singing voice quality from one character (e.g., adult male) to another character (e.g., adult female) when the user makes the electronic keyboard instrument 100 sing in a performance lesson, as shown in Fig. 4(a), for example. In this embodiment, as shown in Fig. 4(a), the voice quality of the singing function at the start of the performance lesson is set to, for example, adult male, and after the performance lesson starts, as the tasks are cleared and the intermediate score improves, the voice quality gradually changes to adult female, and conversely, if the score decreases, the voice quality changes to a slightly harsh voice, so-called hoarse voice.

[0041] As described above, the voice synthesis unit 300 in this embodiment produces a voice by passing an excitation source corresponding to the vibration of the human vocal cords through a filter corresponding to the characteristics of the human vocal tract. As shown in Fig. 4(b), the characteristics of the filter corresponding to the vocal tract characteristics correspond to the formants of the human voice, and the character of the human voice depends heavily on these characteristics. Therefore, in this embodiment, in the formant interpolation processing unit 306 in Fig. 3, the characteristics in the synthesis filter unit 305 are gradually changed by target spectrum information 312 obtained by interpolating the spectrum information 309 of a plurality of characteristics output by the acoustic model unit 301 based on the practice progress data 311 output by the CPU 201 based on the intermediate scoring results, thereby making it possible to smoothly change the character of a certain person's voice to the character of another person's voice.

[0042] In addition to the characteristics of the synthesis filter unit 305, a white noise component is added to the voice to make it more realistic. Therefore, in this embodiment, the noise mixing ratio interpolation processing unit 316 in Fig. 3 calculates a target noise mixing ratio 317 obtained by an interpolation process based on the practice progress data 311 output from the CPU 201 based on the intermediate scoring results, and the noise superimposition unit 307 calculates noise data 315 obtained by increasing or decreasing the amount of white noise added based on the target noise mixing ratio 317, and mixes the noise data 315 with the filter output data 313 output from the synthesis filter unit 305 to generate singing voice output data. This makes it possible to sing in which the intermediate scoring results, having expressive characteristics such as the characteristics of a so-called husky voice, are reflected.

[0043] The operation of the electronic keyboard instrument 100 of this embodiment having the configuration shown in FIGS. 1, 2 and 3 will be described in detail below.

[0044] Fig. 5(a) is a diagram showing an example of the data structure of performance guide data read from ROM 202 in Fig. 2 to RAM 203 in this embodiment. This data structure example complies with the format of a standard MIDI file, which is one of the MIDI file formats. This song data is made up of data blocks called chunks. More specifically, the song data is made up of a header chunk at the beginning of the file, followed by track chunks in which lyric data for the lyric parts is stored. Note that a separate track chunk in which automatic performance data for the accompaniment parts is stored may also be provided.

[0045] The header chunk consists of four values: ChunkID, ChunkSize, FormatType, NumberOfTrack, and TimeDivision. ChunkID is a 4-byte ASCII code "4D 54 68 64" (numbers are hexadecimal) corresponding to the four half-width characters "MThd" which indicates that it is a header chunk. ChunkSize is 4-byte data indicating the data length of the FormatType, NumberOfTrack, and TimeDivision parts excluding ChunkID and ChunkSize in the header chunk, and the data length is fixed to 6 bytes: "00 00 00 06" (numbers are hexadecimal). In this embodiment, FormatType is 2-byte data "00 00" (numbers are hexadecimal) which means format 0 which uses a single track. In this embodiment, NumberOfTrack is 2-byte data "00 01" (numbers are hexadecimal) which indicates that one track corresponding to the lyrics part is used. The TimeDivision is data indicating a time base value that indicates the resolution per quarter note, and in this embodiment, is 2-byte data "01 E0" that indicates 480 in decimal (numbers are in hexadecimal).

[0046] A track chunk consists of a performance data set (0≦i≦L-1) consisting of ChunkID, ChunkSize, and DeltaTime[i] and Event[i]. ChunkID is a 4-byte ASCII code "4D 54 72 6B" (numbers are hexadecimal) that corresponds to the four half-width characters "MTrk" that indicate a track chunk. ChunkSize is 4-byte data that indicates the data length of the part of the track chunk excluding ChunkID and ChunkSize.

[0047] DeltaTime[i] is timing information indicating the waiting time (relative time) from the execution time of the immediately preceding Event[i-1] (the beginning when i=0), and is variable length data of 1 to 4 bytes. Event[i] is a meta event that includes lyric information, which is text data of lyrics, and pitch information that indicates the pitch. In each performance guide data set DeltaTime1[i] and Event[i], Event[i] is executed after waiting DeltaTime[i] from the execution time of the immediately preceding Event[i-1], thereby realizing the progress of automatic playback (vocalization) of lyrics.

[0048] The data structures in FIGS. 5(b) and 5(c) will be described later.

[0049] 6 is a flow chart showing an example of the overall processing of a musical performance lesson. This processing is realized as a process in which the CPU 201 loads the overall processing program of the musical performance lesson stored in the ROM 202 into the RAM 203 and executes it, and is called from a main processing program (not shown) that controls the electronic keyboard instrument 100.

[0050] First, the CPU 201 causes the user to operate the second switch panel 103 in FIG. 1 to select a vocal piece for which the user wishes to have a performance lesson (step S601).

[0051] Next, the CPU 201 executes a lesson process (step S602), the details of which will be described later.

[0052] When the user's performance lesson is completed by the lesson process, the CPU 201 feeds back the evaluation score of the user's performance (step S603). Here, the CPU 201 provides the singing voice data 215 to the voice synthesis LSI 205 in Fig. 2, and vocalizes the evaluation score in a voice with a voice quality corresponding to the evaluation score finally achieved by the user. As will be described later, there are 11 scores ranging from 0 to 10, and a perfect score of 10 will result in the voice saying "jutten" in a female voice, and a score of 0 will result in the voice saying "reiten" in a hoarse voice.

[0053] Fig. 7 is a flow chart showing a detailed example of the lesson process in step S602 in Fig. 6. First, CPU 201 executes initialization processing (step S701). In this processing, processing such as initialization of parameters required for executing the lesson process and turning on LED 104 of the key on keyboard 101 in Fig. 1 that is to be pressed first at half maximum brightness is executed.

[0054] Next, the CPU 201 executes vocal song start processing to start playing back the performance guide data for the vocal song selected by the user in step S601 of FIG. 6 (step S702). In this vocal song start processing, the CPU 201 executes TickTime initialization processing. In this embodiment, the lyrics progress in units of time called TickTime. The time base value specified as the TimeDivision value in the header chunk of the music data in FIG. 5 indicates the resolution of a quarter note, and if this value is 480, for example, a quarter note has a time length of 480 TickTime. The waiting time DeltaTime[i] value in the track chunk of the music data in FIG. 5 is also counted in units of TickTime. Here, how many seconds one TickTime actually occupies depends on the tempo specified for the music data. Now, assuming that the tempo value is Tempo [beats / minute] and the above time base value is TimeDivision, the CPU 201 calculates TickTime [seconds] by arithmetic processing corresponding to the following formula (1).

[0055]

number

[0056] Next, in vocal music start processing in step S702, the CPU 201 sets a timer interruption based on the calculated TickTime [seconds] for the timer 210 in Fig. 2. As a result, an interruption for vocal music progression (hereinafter referred to as "automatic playback interruption") occurs for the CPU 201 every time the TickTime [seconds] elapses in the timer 210. Therefore, in the automatic playback processing (Fig. 8 described later) executed by the CPU 201 based on this automatic playback interruption, a control processing for progressing a set of performance guide data every tick is executed.

[0057] In the initial state, the tempo value Tempo is a predetermined value, for example, 60 [beats / second], stored in the ROM 202 in FIG. 2. If the tempo value of the music piece is included in the performance guide data as an event in the performance guide data, that tempo value may be used. Alternatively, the tempo value at the time of the previous end may be stored in the non-volatile memory. Also, the user can change the tempo value by operating the first switch panel 102 in FIG. 1, and each time the user changes the tempo value, the TickTime [seconds] is calculated and a timer interrupt is set to the timer 210, although this is not shown in the figure.

[0058] Next, in the vocal song start process of step S702, the CPU 201 initializes the value of a variable DeltaT in the RAM 203 for counting the relative time from the occurrence time of the immediately preceding event in units of TickTime during the progress of the automatic playback process to 0. Next, the CPU 201 initializes the value of a variable SongIndex in the RAM 203 for specifying the value of i of each of the performance data sets DeltaTime[i] and Event[i] (1≦i≦L-1) in the track chunk of the song data exemplified in Fig. 5 to 0. As a result, in the example of Fig. 5, the first performance data sets DeltaTime[0] and Event[0] in the track chunk are first referenced as the initial state. Furthermore, the CPU 201 initializes the value of a variable SongStart in the RAM 203 for indicating whether lyrics are to progress (=1) or not (=0) to 1 (progress).

[0059] Furthermore, in the vocal start process of step S702, the CPU 201 sets the value of the variable Margin in the RAM 203 to an integer value that is closest to 30% of the value of DeltaTime[0]. The Margin variable value will be described later. In the vocal start process of step S702, the CPU 201 also resets the value of a correct flag in the RAM 203, which will be described later, and variables NoteOn_in, NoteOn_out, and NoteOn, which will be described later, to 0.

[0060] After the process of step S702, the CPU 201 repeatedly executes the series of processes from steps S703 to S709, thereby progressing the process of automatic playback of lyrics and the process of performance lessons by the user.

[0061] Fig. 8 is a flow chart showing an example of automatic playback processing executed based on an automatic playback interrupt generated by timer 210 every TickTime [seconds] during the period of repeated processing from steps S703 to S709 in Fig. 7. This processing is realized as a function in which, when an automatic playback interrupt occurs from timer 210, CPU 201 interrupts the processing being executed in the lesson processing in Fig. 7 (any of steps S703 to S709) and executes an automatic playback processing program preloaded from ROM 202 to RAM 203.

[0062] First, the CPU 201 determines whether or not the variable SongStart value of the RAM 203 is 1, that is, whether or not the progress of automatic playback of lyrics has been instructed (step S801).

[0063] If the CPU 201 determines that the automatic playback of lyrics has not been instructed (NO in step S801), the CPU 201 does not play lyrics and ends the automatic playback process illustrated in the flowchart of FIG. 8.

[0064] When the CPU 201 determines that the automatic playback of lyrics is instructed (YES in step S801), it determines whether the DeltaT value, which indicates the relative time from the occurrence time of the previous event related to the track chunk of the performance guide data having the data structure of Fig. 5(a) loaded in the RAM 203, has reached a value obtained by subtracting the Margin value from the waiting time DeltaTime[SongIndex] of the performance guide data set to be executed, which is indicated by the SongIndex value (step S802). If NO in step S802, it further determines whether the DeltaT value has reached a value obtained by adding the Margin value to DeltaTime[SongIndex] (step S803).

[0065] In this embodiment, it is judged whether the user has pressed the key of the correct pitch on the keyboard 101 of FIG. 1, which matches the pitch indicated by the pitch information set for each set of playing guide data, at the timing when the lyrics of the set of playing guide data should be spoken. In this case, in order to give the user some leeway in playing lessons, the timing at which the user's key press is considered correct can be set to have a width before and after the timing that exactly matches the timing information=DeltaTime[SongIndex] set for each set of playing guide data, as shown in FIG. 4(c). This time width is called the key press period. The key press period can be an absolute time such as 100 milliseconds before and after the just timing, but in this embodiment, the key press period before and after the just timing is set to 30% of the note length. That is, for a quarter note at a tempo of 60, the note length is 1 second, so the key press period is a section of 300 milliseconds before and after the just timing, that is, 600 milliseconds. Instead of setting the key press period before and after the just timing to the same time, it can also be set to 20% before and 40% after. In this embodiment, in step S809, a value of 30% of the just timing DeltaTime[SongIndex] is calculated and set to the variable Margin in the RAM 203. Then, in step S802, it is determined whether the variable value DeltaT indicating the relative time from the occurrence time of the previous event has reached "DeltaTime[SongIndex]-Margin" corresponding to the start time of the key-press period, and in the following step S803, it is determined whether DeltaT has reached "DeltaTime[SongIndex]+Margin" corresponding to the end time of the key-press period.

[0066] If the determinations in steps S802 and S803 are both NO, the CPU 201 increments the value of the variable DeltaT in the RAM 203, which indicates the relative time from the occurrence time of the previous event, by +1 to advance the time by one TickTime unit corresponding to the current interrupt (step S804). Thereafter, the CPU 201 ends the automatic playback process shown in the flowchart of Fig. 8, and returns to the execution of the interrupted lesson process in Fig. 7.

[0067] If it is determined in step S802 that the variable value DeltaT indicating the relative time from the occurrence time of the previous event has reached "DeltaTime[SongIndex]-Margin" corresponding to the start time of the key-press period, the CPU 201 sets the value of the variable NoteOn_in in the RAM 203 indicating the start of the key-press period to 1, and further sets the value of the variable NoteOn in the RAM 203 indicating that the key-press period is being entered to 1 (step S805). Next, the CPU 201 proceeds to the process of step S804, increments the value of the variable DeltaT in the RAM 203 indicating the relative time from the occurrence time of the previous event by +1, and advances the time by one TickTime unit corresponding to the current interrupt, and then ends the automatic playback process shown in the flowchart of FIG. 8, and returns to the execution of the process interrupted in the lesson process of FIG. 7.

[0068] After the determination in step S802 is NO, if it is determined in step S803 that the variable value DeltaT, which indicates the relative time from the occurrence time of the previous event, has reached “DeltaTime[SongIndex]+Margin” corresponding to the end time of the key-press period, the CPU 201 sets the value of the variable NoteOn_out in RAM 203, which indicates that the key-press period is just about to end, to 1, and further sets the value of the variable NoteOn in RAM 203, which indicates that the key-press period is still in progress, to the value 0, which indicates that the key-press period is no longer in progress (step S806).

[0069] Next, CPU 501 sets the new value of variable DeltaT to "DeltaT-DeltaTime[SongIndex]+1", which is the point at which variable DeltaT has already advanced from DeltaTime[SongIndex] (step S807).

[0070] Furthermore, the CPU 201 increments the value of the variable SongIndex in the RAM 203 by +1 to advance the set of musical performance guide data used in the reaching determination by one (step S808).

[0071] Finally, the CPU 201 sets the Margin value in Fig. 4(c) used to determine whether the next set of play guide data has been reached to "INT(DeltaTime[SongIndex] x 0.3)" ("INT()" indicates an operation to calculate the integer value of the value in parentheses) obtained by multiplying the new DeltaTime[SongIndex] value referred to by the newly updated SongIndex value by 0.3 (30%) (step S809). After that, the CPU 201 ends the automatic playback process shown in the flowchart of Fig. 8, and returns to the execution of the interrupted lesson process of Fig. 7.

[0072] Returning to the explanation of the flowchart in Fig. 7, in the repeated processing of steps S703 to S709, the CPU 201 first executes a performance guide process (step S703). In this performance guide process, the CPU 201 synchronizes with the timing of automatic playback, and causes the LED 104 of the key (operator) corresponding to the pitch information to be automatically played next to be lit at a high brightness, for example, maximum brightness, as shown by the key with two lit LEDs 104 in Fig. 1, and causes the LED 104 of the key corresponding to the pitch information to be automatically played after that to be lit at a low brightness, for example, half the maximum brightness. The details of this performance guide process will be described later with reference to the flowchart in Fig. 9.

[0073] Next, the CPU 201 executes key pressing / release processing (step S704). In this key pressing / release processing, when the user presses any key on the keyboard 101 in Fig. 1, if the CPU 201 determines that the key pressing timing (operation timing) of the key (operator) on the keyboard 101 in the performance lesson correctly corresponds to the timing information for automatic playback (entering the key pressing period in Fig. 4(c)), and that the key pressing pitch (operation pitch) correctly corresponds to (matches) the pitch information of the set of performance guide data, it generates a vocalization event for outputting the singing voice output data 217 from the voice synthesis LSI 205 in Fig. 2 at the key pressing timing.

[0074] In addition, in the key pressing / release processing of step S704, when the user newly presses any key on the keyboard 101 of FIG. 1, if the CPU 201 determines that the key pressing timing in the performance lesson correctly corresponds to the timing information for automatic playback (the key pressing period of FIG. 4(c) is included) but the key pressing pitch (operation pitch) does not correctly correspond (does not match) to the pitch information of the set of performance guide data, it generates a sound generation event for outputting predetermined musical sound output data 214 (for example, based on the instrument sound and the pitch of the performance guide data previously selected by the user on the second switch panel 103 of FIG. 1) from the sound source LSI 204 of FIG. 2 at the key pressing timing.

[0075] Furthermore, in the key pressing / release processing of step S704, if the CPU 201 determines that when the user presses a new key on the keyboard 101 of FIG. 1, the key pressing timing in the performance lesson does not properly correspond to the timing information for automatic playback (not within the key pressing period of FIG. 4(c)), it does not generate either an event for causing the voice synthesis LSI 205 to produce the singing voice output data 217 or an event for causing the sound source LSI 204 to produce the musical sound output data 214.

[0076] On the other hand, in the key press / release process of step S704, when the user releases any key on the keyboard 101 of FIG. 1, the CPU 201 generates a key release event for ending the production of the corresponding singing voice output data 217 in the voice synthesis LSI 205 or the production of the corresponding musical sound output data 214 in the sound source LSI 204.

[0077] The key-press / key-release process in step S704 will be described in detail later with reference to the flowchart in FIG.

[0078] Next, the CPU 201 executes a scoring process (step S705). In this scoring process, the CPU 201 compares the key pressing timing (operation timing) and key pressing pitch (operation pitch) in the performance lesson with the automatically reproduced timing information and pitch information to score the performance lesson. The details of this scoring process will be described later with reference to the flowchart in FIG. 11.

[0079] Next, the CPU 201 executes a voice quality update process (step S706). In this voice quality update process, the CPU 201 executes a process of setting a value indicating a voice quality of the singing voice output data 217 outputted in a vocalization / pronunciation process in a step S708, which will be described later, according to the intermediate scoring result of the scoring process in the step S705.

[0080] Next, the CPU 201 executes a practice progress calculation process (step S707). In this practice progress calculation process, the CPU 201 in Fig. 2 and the formant interpolation processing unit 306 and the noise superimposition unit 307 in the voice synthesis unit 300 in Fig. 3 calculate practice progress data 311 for changing the voice quality of the singing voice output data 217 output in the vocalization / pronunciation process in step S708, which will be described later, according to the intermediate scoring result in the scoring process in step S705. The details of the voice quality update process and the practice progress calculation process will be described later with reference to the flowchart in Fig. 12.

[0081] Furthermore, the CPU 201 executes vocalization / pronunciation processing (step S708). In this vocalization / pronunciation processing, when the CPU 201 generates a vocalization event of the singing voice output data 217 in the key pressing / release processing in step S704, the CPU 201 issues the vocalization event held in the issued event area on the RAM 203 to the voice synthesis LSI 205 in Fig. 2, thereby causing the voice synthesis LSI 205 to output the singing voice output data 217. Also, in this vocalization / pronunciation processing, when the CPU 201 generates a pronunciation event of the musical sound output data 214 in the key pressing / release processing in step S704, the CPU 201 issues the pronunciation event held in the issued event area on the RAM 203 to the sound source LSI 204 in Fig. 2, thereby causing the sound source LSI 204 to output the musical sound output data 214. Furthermore, in this vocalization / pronunciation process, when the CPU 201 generates a key release event in the key press / key release process of step S704, the CPU 201 issues the key release event stored in the issued event area on the RAM 203 to the voice synthesis LSI 205 or the sound source LSI 204 in Figure 2, thereby stopping the output of the corresponding singing voice output data 217 or musical sound output data 214.

[0082] Finally, the CPU 201 judges whether or not there are no more sets of performance guide data to be read from the RAM 203 and the vocal piece has ended (step S709). If the judgment in step S709 is NO, the process returns to step S703 and repeats the series of steps S703 to S709. If the judgment in step S709 is YES, the CPU 201 ends the lesson process in step S602 in FIG. 6 shown in the flowchart in FIG. 7.

[0083] FIG. 9 is a flow chart showing a detailed example of the performance guide process in step S703 of FIG.

[0084] First, the CPU 201 judges whether the value of the variable NoteOn_in on the RAM 203 is 1, that is, whether the current timing of automatic playback has entered the key-press period (see FIG. 4(c)) (step S901). As described above, the value of this variable NoteOn_in is set to the value "1" indicating that the key-press period has started when it is judged in step S802 of the automatic playback process in FIG. 8 that the variable value DeltaT indicating the relative time from the occurrence time of the previous event has reached "DeltaTime[SongIndex]-Margin" corresponding to the start time of the key-press period.

[0085] When the determination in step S901 is YES, the CPU 201 lights up, at maximum brightness, the LED 104 of the key on the keyboard 101 in Fig. 1 that corresponds to the pitch information contained in the performance guide data set Event[SongIndex] in the RAM 203 referenced by the current variable value SongIndex in the RAM 203 via the LED controller 207 in Fig. 2 (step S902). This allows the user to recognize that the key whose LED 104 is lit at maximum brightness is the key to be pressed next.

[0086] Next, the CPU 201 determines whether or not the performance guide data set Event[SongIndex+1] referenced by the current variable value SongIndex+1 on the RAM 203 exists on the RAM 203 (step S903).

[0087] If the determination in step S903 is YES, the CPU 201 lights up the LED 104 of the key on the keyboard 101 in Fig. 1 corresponding to the pitch information included in the performance guide data set Event[SongIndex+1] on the RAM 203 at half the maximum brightness via the LED controller 207 in Fig. 2 (step S904). This allows the user to recognize that the key whose LED 104 is lit at half the maximum brightness is the key that should be pressed next.

[0088] If the determination in step S903 is NO, the CPU 201 does not execute the process of step S904. As a result, since there is no key whose LED 104 is lit at half the maximum brightness, the user can recognize that the key whose LED 104 is lit at the maximum brightness is the last key of the lesson.

[0089] After the process of step S904 or after the determination of step S903 becomes NO, the CPU 201 sets the value of the variable NoteOn_in in the RAM 203 to 0, thereby ending the state in which the current timing of automatic playback has entered the key-press period. Then, the CPU 201 ends the performance guide process of step S703 in Fig. 7 shown in the flowchart of Fig. 9.

[0090] If the determination in step S901 is NO, the CPU 201 next determines whether the value of the variable NoteOn_out in the RAM 203 is 1, i.e., whether the current timing of automatic playback is the timing of leaving the key-press period (see FIG. 4(c)) (step S906). As described above, the value of this variable NoteOn_out is set to the value "1", which indicates the leaving of the key-press period, when it is determined in step S803 of the automatic playback process in FIG. 8 that the variable value DeltaT, which indicates the relative time from the occurrence time of the previous event, has reached "DeltaTime[SongIndex]+Margin", which corresponds to the end time of the key-press period.

[0091] When the determination in step S906 is YES, the CPU 201 turns off the LED 104 of the key on the keyboard 101 in FIG. 1 corresponding to the pitch information included in the performance guide data set Event[SongIndex-1] in the RAM 203 referenced by the current variable value SongIndex in the RAM 203 minus 1 via the LED controller 207 in FIG. 2 (step S907). This allows the user to recognize that the key-pressing period for this key has ended. Note that the reason SongIndex-1 is referenced instead of SongIndex is that when NoteOn_out=1 in step S806 in FIG. 8, the value of SongIndex is incremented by +1 in the following step S808, so that the LED 104 of the key in the previous key-pressing period can be turned off by referring to Event[SongIndex-1].

[0092] After the process of step S907 or after the determination of step S906 becomes NO, the CPU 201 ends the performance guide process of step S703 in FIG. 7 shown in the flowchart of FIG.

[0093] FIG. 10 is a flow chart showing a detailed example of the key-press / key-release process in step S704 of FIG.

[0094] First, the CPU 201 determines whether or not a new key has been pressed on the keyboard 101 in FIG. 1 by the user via the key scanner 206 in FIG. 2 (step S1001).

[0095] If the determination in step S1001 is NO, the CPU 201 proceeds to step S1007 to perform key-release determination processing.

[0096] If the determination in step S1001 is YES, the CPU 201 determines whether the value of the variable NoteOn in the RAM 203 is 1, that is, whether the current timing of automatic playback is within the key-press period (see FIG. 4(c)) (step S1002).

[0097] If the determination in step S1002 is NO, the CPU 201 proceeds to a key release determination process in step S1007. In this way, when the user newly presses any key on the keyboard 101 in Fig. 1, if it is determined that the key pressing timing in the performance lesson does not correctly correspond to the timing information for automatic playback (not within the key pressing period in Fig. 4(c)), neither an event for making the voice synthesis LSI 205 produce the singing voice output data 217 nor an event for making the sound source LSI 204 produce the musical sound output data 214 is generated, and neither vocalization nor pronunciation is performed.

[0098] In this way, the user can recognize that he or she pressed the key at the wrong time because neither the singing voice output data 217 nor the musical tone output data 214 is produced despite the key being pressed.

[0099] If the determination in step S1002 is YES, the CPU 201 determines whether the pitch of the newly pressed key notified via the key scanner 206 matches the pitch information contained in the Event[SongIndex] in the RAM 203 referenced by the current SongIndex value in the RAM 203 (step S1003).

[0100] If the determination in step S1003 is YES, the CPU 201 generates a vocal event having, as singing voice data 215, lyrics information and pitch information included in Event[SongIndex] in the RAM 203 referenced by the current SongIndex value in the RAM 203, and sets the generated vocal event in the issued event area in the RAM 203 (step S1004). Next, the CPU 201 sets the value of a correct flag variable in the RAM 203 to 1 (step S1005).

[0101] On the other hand, if the determination in step S1003 is NO, the CPU 201 generates a sound production event having tone color information of a predetermined tone color (e.g., a piano tone) as sound production control data 216 (see FIG. 2) together with pitch information contained in Event[SongIndex] on RAM 203 referenced by the current SongIndex value on RAM 203, and sets this in the issued event area of ​​RAM 203 (step S1006).

[0102] 1 and the timing of the key press correctly corresponds to the timing information to be automatically played back, the voice synthesis LSI 205 can output singing voice output data 217 corresponding to the lyrics and pitch during automatic playback as a correct answer if the key press pitch correctly corresponds to the pitch information of the set of performance guide data as in step S1004 or S1005 above, and the sound source LSI 204 can output musical sound output data 214 of a predetermined tone color corresponding to the pitch during automatic playback as an incorrect answer if the key press pitch is incorrect. In this way, the user can easily recognize whether the performance lesson is correct or incorrect by whether the singing voice output data 217 or the musical sound output data 214 is pronounced, without having to look at the display or the like every time.

[0103] If the determination in step S1001 is NO or after the processing in step S1005 or S1006, the CPU 201 determines whether or not a new key has been released by the user on the keyboard 101 in FIG. 1 via the key scanner 206 in FIG. 2 (step S1007).

[0104] If the determination in step S1007 is YES, a key release event is generated to stop the output of the currently sounding singing voice output data 217 or musical sound output data 214 to the voice synthesis LSI 205 or the sound source LSI 204 in FIG. 2, and set in the issued event area of ​​the RAM 203 (step S1008).

[0105] After the process of step S1008 or if the determination in step S1007 is NO, the CPU 201 ends the key-press / key-release process of step S704 in FIG. 8 shown in the flowchart of FIG.

[0106] Fig. 11 is a flowchart showing a detailed example of the scoring process in step S705 in Fig. 7. First, the CPU 201 determines whether the value of the variable NoteOn_out in the RAM 203 is 1, that is, whether the current timing of automatic playback is the timing to exit the key-press period (see Fig. 4(c)) (step S1101).

[0107] If the determination in step S1101 is YES, the following scoring process is executed. That is, the CPU 201 first determines whether the variable value of the correct flag in the RAM 203 is 1 (step S1102). As described above, the variable value of the correct flag is set to the value "1" in step S1005 in Fig. 10 when the user newly presses any key on the keyboard 101 in Fig. 1 and the key pressing timing correctly corresponds to the timing information to be automatically played back, and the key pressing pitch correctly corresponds to the pitch information in the set of performance guide data.

[0108] If the determination in step S1102 is YES, the CPU 201 performs a point increment process (for example, +1) on the variable value indicating the intermediate scoring result in the RAM 203 (step S1103). Next, the CPU 201 resets the variable value of the correct answer flag in the RAM 203 to 0 (step S1104).

[0109] On the other hand, if the determination in step S1102 is NO, the CPU 201 performs a subtraction process (eg, subtraction of 1) on the variable value indicating the intermediate scoring result stored in the RAM 203 (step S1105).

[0110] After the process of step S1104 or S1105, the CPU 201 resets the value of the NoteOn_out variable in the RAM 203 to 0 (step S1106).

[0111] After the process of step S1106 or if the determination in step S1101 is NO, the CPU 201 ends the scoring process of step S705 in FIG. 7 shown in the flowchart of FIG.

[0112] As described above, in this embodiment, the state of the correct answer flag during the key pressing period is checked when the key pressing period has elapsed, so even if the user presses a key outside the key pressing period, it does not affect the score. If a key is not pressed during the key pressing period, points will be deducted.

[0113] Fig. 12 is a flow chart showing a detailed example of the voice quality update process in step S706 in Fig. 7. As described above, in this embodiment, the voice quality is changed so that the initial state is a male voice, and as the score of the intermediate scoring result increases, the voice quality is changed toward a female voice, and as the score decreases, the voice quality is changed toward a male hoarse voice. The voice quality is determined by the frequency characteristic of the vocal tract called formant as described above, as well as the frequency of the sound emitted by the vocal cords (voice pitch). As for formants, as shown in Fig. 4(b), in general, in female voices and children, the first and subsequent formants are located in the high frequency range, while in male voices, the formants are located in the low frequency range. In addition, women and children generally have high voice pitches (frequency of vibration of the vocal cords).

[0114] Considering the above tendency, in this embodiment, as shown in Fig. 5(b), the voice quality is changed in 11 stages, 5 stages upward and 5 stages downward from the initial state of male voice quality 5. The formants are changed to interpolate between female voice, male voice and hoarse voice. In addition, in the third stage of voice quality change (voice quality 8 and voice quality 2), the voice pitch is raised or lowered by one octave.

[0115] In the process shown in the flowchart of FIG. 12, CPU 201 compares the score of the previous intermediate scoring result stored in RAM 203 with the score of the current intermediate scoring result (step S1201), and determines whether the score has increased (step S1202).

[0116] If the determination in step S1202 is YES (if the score has increased), CPU 201 increments the voice quality by one level (step S1205) unless the voice quality has reached the maximum value of 10 (if the determination in step S1204 is NO). If the voice quality has reached the maximum value of 10 (if the determination in step S1204 is YES), CPU 201 does not execute the increment process in step S1205.

[0117] If the voice quality is at 8 or 2, which is the octave switching stage (YES in step S1206), the CPU 201 raises the voice pitch by one octave (step S1207). If the decision in step S1206 is NO, the CPU 201 maintains the octave. Thereafter, the CPU 201 ends the voice quality update process of step S706 in FIG. 7, which is shown in the flowchart in FIG. 12.

[0118] On the other hand, if the determination in step S1202 is NO, the CPU 201 further determines whether or not the score has decreased as a result of the comparison process in step S1201 (step S1203). If the determination in step S1203 is also NO, meaning that the score has remained the same, the CPU 201 does nothing and ends the voice quality update process in step S706 in Fig. 7 shown in the flowchart in Fig. 12.

[0119] If the determination in step S1203 is YES, indicating that the score has decreased, CPU 201 subtracts one level from the voice quality (step S1209) unless the voice quality has reached the minimum value of 0 (NO in step S1208).If the voice quality has reached the minimum value of 0 (YES in step S1208), CPU 201 does not execute the subtraction process in step S1209.

[0120] If the voice quality is at 8 or 2, which is the octave switching stage (YES in step S1210), the CPU 201 lowers the voice pitch by one octave (step S1211). If NO in step S1210, the CPU 201 maintains the octave.

[0121] After calculating the voice quality as described above, CPU 201 stores the voice quality in the variable curNum on RAM 203, and then ends the voice quality update process of step S706 in Fig. 7 shown in the flowchart of Fig. 12. Next, in the practice progress calculation process of step S707 in Fig. 7, CPU 201 uses the voice quality value obtained for the variable curNum in step S706 to calculate the value x of practice progress data 311 by the arithmetic process shown in equation (2).

number

[0122] FIG. 5(c) is a diagram showing an example of the data structure of the voice quality parameters (hereinafter referred to as the "voice quality parameter structure") held in the acoustic model unit 301 in the voice synthesis LSI 205 in this embodiment. The "vowel flag" is a flag indicating whether the phoneme is a vowel or a consonant. If it is a vowel, it is true, and if it is a consonant, it is false. The "noise mixing ratio" is the amplitude ratio of the white noise mixed into the voice. This value is the ratio when the maximum amplitude (32,768 in 16 bits) of the singing voice output data 217 is set to 1. The "LSF1 array pointer" and "LSF2 array pointer", which are formant parameters, are array pointers to LSF values ​​obtained by converting the LSP parameter values ​​obtained as a result of the LSP analysis into frequency parameter values. Since two LSFs make a pair of data, the above data structure holds the pairs of "LSF1 array pointer" and "LSF2 array pointer" for the number of orders of the LSP analysis. The data set of the above voice quality parameter structures, the number of which corresponds to all phonemes x all note numbers, constitutes a set of all voice quality parameters for one human voice character. This voice quality parameter set is stored in a writeable ROM (not shown) in the acoustic model unit 301 for each character, male voice, female voice, and hoarse voice.

[0123] 13 is a flow chart showing an example of voice quality change processing executed as the functions of the formant interpolation processing unit 306 and the noise mixing ratio interpolation processing unit 316 of the voice synthesis unit 300 by the processor of the voice synthesis LSI 205 in FIG. 3 during the vocalization processing in the voice synthesis LSI 205. Here, the voice quality change between male voice and female voice will be explained. The voice quality change between male voice and hoarse voice can also be implemented in the same way.

[0124] The processor of the voice synthesis LSI 205 acquires the phoneme number to be vocalized, which is set via the text analysis unit 302 of Fig. 3 based on the singing voice data 215 issued by the CPU 201 in the vocalization / pronunciation process of step S708 of Fig. 7 (step S1301), and acquires the note number, which is pitch information specified in a form included in the singing voice data 215 (step S1302). The processor of the voice synthesis unit 300 also acquires the value x of the practice progress data 311 calculated by the CPU 201 in step S707 of Fig. 7 through the arithmetic process shown in the above-mentioned equation (2) (step S1303).

[0125] Next, the processor of the speech synthesis LSI 205 acquires voice quality parameter structures for the male voice and the female voice in the data format shown in FIG. 5(c) from the acoustic model unit 301 based on the phoneme number acquired in step S1301 and the note number acquired in step S1302 (steps S1304, S1305).

[0126] Next, the processor of the voice synthesis LSI 205 determines whether or not a vowel flag is set in the voice quality parameter structure acquired in step S1304 or S1305 (step S1306).

[0127] If the phoneme of the current phoneme number acquired in step S1301 is not a vowel and the judgment in step S1306 is NO, that is, if it is a consonant, the phoneme is a noise voice without pitch. In this case, the processor of the voice synthesis LSI 205 extracts the "noise mixture ratio" (see FIG. 5(c)) from the male voice quality parameter structure acquired in step S1304 and sets it as the target noise mixture ratio 317 in the noise superimposition unit 307 of FIG. 3, and extracts the "LSF1 array pointer" and the "LSF2 array pointer" (see FIG. 5(c)) and sets them as the target spectrum information 312 in FIG. 3 in the synthesis filter unit 305 of FIG. 3 (step S1310). It is needless to say that data from a female voice quality parameter structure instead of a male voice may be set as the target noise mixture ratio 317 and the target spectrum information 312.

[0128] If the phoneme of the current phoneme number acquired in step S1301 is a vowel and the determination in step S1306 is YES, the processor of the speech synthesis LSI 205 first executes a noise mixing ratio interpolation process as the function of the noise mixing ratio interpolation processor 316 in Fig. 3 (step S1307), and then executes a formant interpolation process as the function of the formant interpolation processor 306 in Fig. 3 (step S1308). The processor of the speech synthesis LSI 205 then sets the target noise mixing ratio 317 obtained by the noise mixing ratio interpolation process in step S1307 as a target parameter in the noise superimposition unit 307 in Fig. 3, and sets the target spectrum information 312 obtained by the formant interpolation process in step S1308 as a target parameter in the synthesis filter unit 305 in Fig. 3.

[0129] After the process of step S1309 or S1310, the processor of the voice synthesis LSI 205 ends the voice quality modification process shown in the flowchart of FIG.

[0130] FIG. 14A is a flowchart showing a detailed example of the noise mixing ratio interpolation process in step S1307 in FIG. 13, which is executed by the processor of the voice synthesis LSI 205 as a function of the noise superimposing unit 307 in the voice synthesis unit 300 in FIG.

[0131] The processor of the speech synthesis LSI 205 extracts the value of the male noise mixture ratio as maleRate from the male voice quality parameter structure obtained from the acoustic model unit 301 in step S1304 of FIG. 3, and also extracts the value of the female noise mixture ratio as femaleRate from the female voice quality parameter structure obtained from the acoustic model unit 301 in step S1305 of FIG. 3. Then, using the value x of the practice progress data 311 obtained in step S1303 of FIG. 13, the processor calculates the value targetRate of the target noise mixture ratio 317 by performing the arithmetic processing shown in equation (3) below (step S1401).

number

[0132] As described above, the value x of the practice progress data 311 indicates the ratio of the difference value of the current voice quality value curNum corresponding to the intermediate scoring result for the male voice quality value maleNum to the difference value of the female voice quality value femaleNum for the male voice quality value maleNum. Therefore, as shown in the above formula (3), the difference value of the female noise mixture ratio for the male noise mixture ratio (femaleRate-maleRate) is multiplied by the value x of the practice progress data 311 corresponding to the scoring result, and the multiplication result is added to the male noise mixture ratio maleRate, thereby calculating the value targetRate of the target noise mixture ratio 317 corresponding to the voice quality interpolated in response to the scoring result.

[0133] The processor of the voice synthesis LSI 205 in Fig. 3, as a function of the noise mixing ratio interpolation processing unit 316, sets the target noise mixing ratio value targetRate calculated in step S1307 of Fig. 13 shown in the flowchart of Fig. 14(a) as target noise mixing ratio 317, which is a target parameter, in the noise superimposing unit 307 in the voice synthesis LSI 205 in Fig. 13 in step S1309 of Fig. 13. Then, the processor of the voice synthesis LSI 205, as a function of this noise superimposing unit 307, generates noise data 315 having an amplitude value obtained by multiplying the maximum amplitude value of the signal by the value targetRate of the target noise mixing ratio 317, and mixes it with the filter output data 313.

[0134] FIG. 14B is a flowchart showing a detailed example of the formant interpolation process in step S1308 in FIG. 13, which is executed by the processor of the voice synthesis LSI 205 as a function of the formant interpolation processing unit 306 in the voice synthesis unit 300 in FIG.

[0135] The processor of the speech synthesis LSI 205 repeatedly executes the processes of steps S1412 and S1413 for each parameter set of LSF1 and LSF2, while incrementing the value of a variable i held, for example, as a register not specifically shown, from 0 (step S1411) by +1 (step S1414) until it is determined that the value has reached the parameter order N (step S1415).

[0136] 13. First, the processor of the speech synthesis LSI 205 extracts the i-th male LSF1 array pointer from the male voice quality parameter structure obtained from the acoustic model unit 301 in step S1304 of FIG. 3, and obtains the value of the LSF1 parameter as maleLSF1[i] from a memory (not shown) in the acoustic model unit 301 referenced by the pointer. Also, in step S1305 of FIG. 3, the processor extracts the i-th female LSF1 array pointer from the female voice quality parameter structure obtained from the acoustic model unit 301, and obtains the value of the LSF1 parameter as femaleLSF1[i] from the memory in the acoustic model unit 301 referenced by the pointer. Then, using the value x of the practice progress data 311 obtained in step S1303 of FIG. 13, the processor calculates the i-th target LSF1 value targetLSF1[i], which is part of the target spectrum information 312 (step S1412).

number

[0137] As described above, the value x of the practice progress data 311 indicates the ratio of the frequency difference value of the current voice quality value curNum corresponding to the intermediate scoring result for the male voice quality value maleNum to the frequency difference value of the female voice quality value femaleNum for the male voice quality value maleNum. Therefore, as shown in the above formula (4), the frequency difference value (femaleLSF1[i]-maleLSF1[i]) of the female's i-th order LSF1 parameter value for the male's i-th order LSF1 parameter value is multiplied by the value x of the practice progress data 311 corresponding to the scoring result, and the multiplication result is added to the male's i-th order LSF1 parameter value maleLSF1[i], thereby calculating the i-th order target LSF1 parameter value targetLSF1[i] corresponding to the voice quality interpolated in response to the scoring result.

[0138] 13. Next, the processor of the speech synthesis LSI 205 extracts the i-th male LSF2 array pointer from the male voice quality parameter structure acquired from the acoustic model unit 301 in step S1304 of FIG. 3, and acquires the LSF2 parameter value referenced by the pointer as maleLSF2[i] from the memory in the acoustic model unit 301 in step S1305 of FIG. 3. It also extracts the i-th female LSF2 array pointer from the female voice quality parameter structure acquired from the acoustic model unit 301 in step S1305 of FIG. 3, and acquires the LSF2 parameter value referenced by the pointer as femaleLSF2[i] from the memory in the acoustic model unit 301 in step S1305 of FIG. 3. It then calculates the i-th target LSF2 value targetLSF2[i], which is part of the target spectrum information 312, by performing the arithmetic processing shown in equation (5) below using the value x of the practice progress data 311 acquired in step S1303 of FIG. 13 (step S1413).

number

[0139] As in the case of the LSF1 parameter value in equation (4) described above, as shown in equation (5) above, the frequency difference value (femaleLSF2[i]-maleLSF2[i]) of the female's i-th order LSF2 parameter value relative to the male's i-th order LSF2 parameter value is multiplied by the value x of the practice progress data 311 corresponding to the scoring result, and the multiplication result is added to the male's i-th order LSF2 parameter value maleLSF2[i], thereby calculating the i-th order target LSF2 parameter value targetLSF1[i] corresponding to the voice quality interpolated in accordance with the scoring result.

[0140] The processor of the voice synthesis LSI 205 in Fig. 3, as a function of the formant interpolation processing unit 306, sets pairs (0≦i≦N-1) of the target LSF1 parameter value targetLSF1[i] and the target LSF2 parameter value targetLSF2[i] for the LSP analysis order calculated in the formant interpolation processing in step S1308 in Fig. 13 shown in the flowchart in Fig. 14(b) as target spectrum information 312, which is a target parameter, in the synthesis filter unit 305 in the voice synthesis LSI 205 in Fig. 13 in step S1309 described above in Fig. 13. As a result, the synthesis filter unit 305 forms a digital filter using the target spectrum information 312, and outputs filter output data 313 by inputting a sound source signal to this digital filter by the function of the sound source generation unit 304 in the vocalization model unit 303 in Fig. 3. Finally, this filter output data 313 is mixed with noise data 315 and output as singing voice output data 217.

[0141] By the voice quality changing process described with reference to the flowcharts of Fig. 13 and Fig. 14, the voice quality of the singing function at the start of the performance lesson is set to, for example, an adult male voice, and as the intermediate score improves after the start of the performance lesson, the voice quality of the singing voice emitted from the speaker of the electronic keyboard instrument 100 based on the user's key depression operation gradually changes to an adult female voice quality, and conversely, when the score decreases, the voice quality of the singing voice changes from an adult male voice quality to a voice quality that is a little harsh, so-called hoarse voice. Furthermore, it is possible to add a change to the presence or absence of a voice quality such as a husky voice depending on the intermediate score. This makes it possible for the user to easily check the level of his or her performance operation skill as the performance lesson progresses by the voice quality of the uttered singing voice, without having to check the display every time.

[0142] In the embodiment described above, a judgment was made as to whether each note to be pressed was played or not, and the score of the intermediate scoring results was increased or decreased accordingly. However, the score may also be increased or decreased by taking the average score for a group of several notes (phrases) or for several measures, or the score may be increased or decreased if a key is pressed several times in succession or if a key cannot be pressed.

[0143] Furthermore, when scoring is based on groups of notes as described above, a bonus stage may be provided in which more points are added in more difficult parts, or a penalty stage may be provided in which more points are deducted if a mistake is made in an easy part.

[0144] In this embodiment, the formants are shifted, and the mixing level and pitch of the white noise are changed. However, it is also possible to add effects such as reverberation effects like reverb, or effects such as tremolo that cause the pitch to fluctuate, or to add effects such as making the pitch more unstable as the score decreases.

[0145] In this embodiment, the LED corresponding to the next key to be pressed is lit at maximum brightness, and the LED corresponding to the key after that is lit at half the maximum brightness. However, the LED brightness may be constant, and the LED corresponding to the next key to be pressed may be lit at the same brightness, and the LED corresponding to the key after that may be blinking, making it possible to distinguish between the keys.

[0146] In this embodiment, the electronic keyboard instrument 100 is implemented, but the present invention may also be implemented in other musical instrument configurations, such as guitar-type or wind instrument-type electronic musical instruments.

[0147] In this embodiment, the highest score is a female voice and the lowest score is a hoarse voice, but it goes without saying that various other combinations of voice qualities can be considered.

[0148] In this embodiment, the target voice is a typical female voice, but an acoustic model that models the voice of a specific person may be used. For example, if an acoustic model that has been trained on the singing of a famous singer is used, it will be possible to produce a voice that approaches the voice of the singer you admire as your performance improves, making performance lessons even more effective.

[0149] In this embodiment, the LSF is used as the voice quality parameter. However, when the synthesis filter unit 305 in FIG. 3 is realized by a filter bank, it is also possible to regard the amplification factor of each filter constituting the filter bank as a formant shape and perform voice quality interpolation processing on the gain of each filter bank.

[0150] Furthermore, it is also possible to perform an interpolation process in the frequency domain by regarding the frequency amplitude outline generated by applying a moving average filter to the frequency amplitude components of the voice according to the method described in the following patent document as a voice quality parameter. Various other methods for changing the voice quality can be adopted regardless of the description in the embodiments. (Patent Document): JP 2005-084661 A

[0151] In this embodiment, the sound source signal serving as an excitation source signal to the synthesis filter unit 305 in FIG. 3 is generated by the sound source generation unit 304 in the speech synthesis LSI 205 in FIG. 3 based on the pitch information 310 output from the acoustic model unit 301. However, as another embodiment, the sound source signal to be input to the synthesis filter unit 305 may be supplied from the sound source LSI 204 in FIG. 2.

[0152] Fig. 15 is a block diagram showing another example of the configuration of the vocalization model unit 303 in the voice synthesis unit 300 in the voice synthesis LSI for realizing the above configuration. In step S1004 in Fig. 14 in the key pressing / release processing in step S704 in Fig. 7 described above, the CPU 201 generates a vocalization event having lyrics information and pitch information included in Event[SongIndex] in the RAM 203 referenced by the current SongIndex value in the RAM 203 as singing voice data 215, and sets it in the issued event area of ​​the RAM 203. At the same time, the CPU 201 generates a vocalization event for vocalization sound source designation having a predetermined vocalization channel designation as vocalization control data 216 (see Fig. 2) together with the pitch information also included in Event[SongIndex], and sets it in the issued event area of ​​the RAM 203.

[0153] Following the above processing, in the voice production / pronunciation processing of step S708, the CPU 201 issues the voice production event stored in the issued event area on the RAM 203 to the voice synthesis LSI 205 of FIG. 2, and also issues the voice production sound source designation pronunciation event to the sound source LSI 204 of FIG. 2.

[0154] As a result, the sound source LSI 204 generates musical sound output data 1501 for vocal sound source having a pitch corresponding to the pitch information included in the vocal sound source designation pronunciation event by using a specific sound source channel (or multiple channels) designated by the vocal sound source designation pronunciation event, and inputs the data to the voice synthesis LSI 205 in FIG. 2 via a signal path not particularly shown in FIG. 2.

[0155] The vocal sound source musical sound output data 1501 generated and output by the sound source LSI 204 based on the performance lesson by the performer is input to a digital filter formed by the synthesis filter unit 305 based on the target spectrum information 312 input from the acoustic model unit 301 via the formant interpolation processing unit 306 in the voice synthesis LSI 205 of Figure 3, and singing voice output data 217 is output from the synthesis filter unit 305.

[0156] The singing voice output data 217 thus generated and output has as its sound source signal the instrument sound generated by the sound source LSI 204. Therefore, compared to the singer's singing voice, the fidelity is somewhat lost, but the atmosphere of the instrument sound set by the sound source LSI 204 is well retained, and the vocal quality of the singer's singing voice is also well retained, making it possible to output effective singing voice output data 217.

[0157] Furthermore, since the vocal sound source musical sound output data 1501 can also be polyphonic using multiple channels, in that case it is possible to achieve the effect of multiple singing voices harmonizing together.

[0158] The vocal sound source musical sound output data 1501 may be any waveform signal, but due to the nature of the sound source signal, it is preferable to use musical instrument sounds such as brass, strings, and organ sounds that contain many harmonic components and last for a long time. Of course, even if musical instrument sounds that do not follow such standards at all, such as animal cries, are used to achieve a great effect, a very interesting effect can be obtained. As a specific example, the vocal sound source musical sound output data 1501 using waveform data obtained by sampling the barking of a pet dog may be input to the synthesis filter unit 310. This will produce a very interesting effect, as if the pet dog is singing the lyrics.

[0159] Although the disclosed embodiments and their advantages have been described in detail above, those skilled in the art may make various modifications, additions, and omissions without departing from the scope of the present invention as clearly set forth in the claims.

[0160] In addition, the present invention is not limited to the above-mentioned embodiment, and various modifications can be made in the implementation stage without departing from the gist of the invention. In addition, the functions executed in the above-mentioned embodiment may be implemented in appropriate combinations as much as possible. The above-mentioned embodiment includes various steps, and various inventions can be extracted by appropriate combinations of the multiple components disclosed. For example, if an effect can be obtained even if some components are deleted from all the components shown in the embodiment, the configuration from which the components are deleted can be extracted as an invention.

[0161] The following supplementary notes are further disclosed regarding the above-described embodiment. (Appendix 1) A performance information acquisition means for acquiring performance information of a performer; a performance evaluation means for evaluating the performance of a performer multiple times at different timings during the progress of a piece of music based on performance guide data including at least lyric information, pitch information, and timing information and the performance information; a singing voice producing means for producing lyrics by singing voice based on the performance information and the lyrics information; a voice quality changing means for changing the voice quality of the singing voice in response to a change in the evaluation of the performance evaluation means when the evaluation of the performance evaluation means has changed from a previous evaluation; An electronic musical instrument comprising: (Appendix 2) 2. The electronic musical instrument according to claim 1, further comprising a performance guide means for guiding a player in playing a piece of music based on the performance guide data. (Appendix 3) Further comprising a plurality of light emitting elements, the performance guide means causes a light emitting element corresponding to pitch information included in the performance guide data to emit light at a timing corresponding to timing information included in the performance guide data; 2. An electronic musical instrument as described in appended claim 2. (Appendix 4) 4. The electronic musical instrument of claim 1, wherein the voice quality changing means interpolates between a plurality of voice qualities corresponding to a plurality of specific evaluations by the performance evaluation means at a ratio according to the evaluation by the performance evaluation means during the progression of the piece of music. (Appendix 5) 5. The electronic musical instrument according to claim 1, wherein the voice quality changing means changes a ratio of a formant component of a human voice to a noise component mixed with the human voice. (Appendix 6) Electronic musical instrument processors, A performance information acquisition process for acquiring performance information of a performer; a performance evaluation process for evaluating the performance of a performer multiple times at different timings during the progression of a piece of music based on performance guide data including at least lyric information, pitch information, and timing information and the performance information; a singing voice production process for producing lyrics by singing voice based on the performance information and the lyrics information; a voice quality change process for changing the voice quality of the singing voice in response to a change in the evaluation of the performance evaluation means when the evaluation of the performance evaluation means has changed from a previous evaluation; A method for controlling an electronic musical instrument to execute the above. (Appendix 7) Electronic musical instrument processors, A performance information acquisition process for acquiring performance information of a performer; a performance evaluation process for evaluating the performance of a performer multiple times at different timings during the progression of a piece of music based on performance guide data including at least lyric information, pitch information, and timing information and the performance information; a singing voice production process for producing lyrics by singing voice based on the performance information and the lyrics information; a voice quality change process for changing the voice quality of the singing voice in response to a change in the evaluation of the performance evaluation means when the evaluation of the performance evaluation means has changed from a previous evaluation; A program for executing. [Explanation of symbols]

[0162] 100 Electronic Keyboard Instruments 101 Keyboard 102 First Switch Panel 103 Second Switch Panel 104 LED 200 Control System 201 CPU 202 ROM 203 RAM 204 Sound source LSI 205 Voice Synthesis LSI 206 Key Scanner 207 LED Controller 208 Network Interface 209 System Bus 210 Timer 211, 212 D / A converter 213 Mixer 214 Amplifier 215 Vocal Data 216 Sound Control Data 217 Singing voice output data 218 Musical output data 300 Voice synthesis unit 301 Acoustic Model Section 302 Text Analysis Section 303 Vocal Model Section 304 Sound source generation section 305 Synthesis Filter Section 306 Formant Interpolation Processing Unit 307 Noise Superimposition Section 308 Language Feature Sequence 309 Spectral Information 310 Sound Source Information 311 Practice progress data 312 Target Spectral Information 313 Filter Output Data 314 Noise Mix Ratio 315 Noise Data 316 Noise Mixing Ratio Interpolation Processing Unit 317 Target Noise Mixture Ratio 1501 Musical output data for vocal sound source

Claims

1. A performance information acquisition means for acquiring performance information of a performer; a performance evaluation means for evaluating the performance of a performer multiple times at different timings during the progress of a piece of music based on performance guide data including at least lyric information, pitch information, and timing information and the performance information; a singing voice producing means for producing lyrics by singing voice based on the performance information and the lyrics information; a voice quality changing means for changing the voice quality of the singing voice in response to a change in the evaluation of the performance evaluation means when the evaluation of the performance evaluation means has changed from a previous evaluation; An electronic musical instrument comprising:

2. 2. The electronic musical instrument according to claim 1, further comprising a performance guide means for guiding a performer in playing a piece of music on the basis of said performance guide data.

3. Further comprising a plurality of light emitting elements, the performance guide means causes a light emitting element corresponding to pitch information included in the performance guide data to emit light at a timing corresponding to timing information included in the performance guide data; 3. The electronic musical instrument according to claim 2.

4. 4. The electronic musical instrument according to claim 1, wherein said voice quality changing means interpolates between a plurality of voice qualities corresponding to a plurality of specific evaluations by said performance evaluation means at a ratio according to the evaluation by said performance evaluation means during the progression of the piece of music.

5. 5. The electronic musical instrument according to claim 1, wherein said voice quality modification means modifies a ratio of a formant component of a human voice to a noise component mixed with the human voice.

6. Electronic musical instrument processors, A performance information acquisition process for acquiring performance information of a performer; a performance evaluation process for evaluating the performance of a performer multiple times at different timings during the progression of a piece of music based on performance guide data including at least lyrics information, pitch information, and timing information and the performance information; a singing voice production process for producing lyrics by singing voice based on the performance information and the lyrics information; a voice quality change process for changing the voice quality of the singing voice in response to a change in evaluation of the performance evaluation process from a previous evaluation, when the evaluation of the performance evaluation process has changed from a previous evaluation; A method for controlling an electronic musical instrument to execute the above.

7. Electronic musical instrument processors, A performance information acquisition process for acquiring performance information of a performer; a performance evaluation process for evaluating the performance of a performer multiple times at different timings during the progression of a piece of music based on performance guide data including at least lyrics information, pitch information, and timing information and the performance information; a singing voice production process for producing lyrics by singing voice based on the performance information and the lyrics information; a voice quality change process for changing the voice quality of the singing voice in response to a change in evaluation of the performance evaluation process from a previous evaluation, when the evaluation of the performance evaluation process has changed from a previous evaluation; A program for executing.

Images