Audio information playing method and device, audio information generating method and device and program
By converting the singing synthesis music score into time-serialized waveform data and generating segmentation information, the problem of inflexible audio information playback control in the prior art is solved, and real-time and natural playback of audio information is realized.
Patent Information
- Application Number
- CN202080030401.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-04-26
- Filing Date
- 2020-03-19
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2040-03-19
AI Technical Summary
The prior art is difficult to realize real-time and natural playback control of the generated audio information, especially after the music score is converted into audio information by singing synthesis, it is difficult to change pronunciation and silencing according to the user's wishes.
By time-serializing waveform data of multiple vocal units in the determined pronunciation pitch and pronunciation order, segmentation information is generated, and the playback position is moved according to the note on and off information, real-time playback control of audio information is realized.
Real-time and natural playback control of audio information is realized, allowing users to adjust the playback of audio information in real time according to performance operations, and improve the playback flexibility and nature of audio information.
Smart Images

Figure CN113711302B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an audio information playing method and device, an audio information generating method and device, and a program. Background Art
[0002] In the past, there is a known technique for playing data (singing synthesis music score) for associating a plurality of syllables with notes for singing. The device of the following patent document 1 can change the pitch and pronunciation period of the singing voice in real time by performing singing synthesis in accordance with the singing synthesis music score and the user's performance operation. In addition, by performing singing synthesis on the singing synthesis music score and converting it into waveform data, it is also possible to generate audio information in which the waveform data of each of the plurality of syllables is time-series.
[0003] Patent Document 1: Japanese Patent No. 4735544 Summary of the invention
[0004] However, once the music score for singing synthesis is synthesized and converted into audio information, the pronunciation timing and pronunciation length of each syllable of the audio information are determined. Therefore, in the playback of audio information generated by singing synthesis, it is difficult to change the pronunciation and mute in a natural form according to the user's wishes. That is, audio information is usually played in a time series, but it is not suitable for real-time playback control as desired in accordance with performance operations, etc. Therefore, there is room for improvement in achieving real-time and desired playback control of audio information.
[0005] The object of the present invention is to provide an audio information playing method and device, an audio information generating method and device, and a program that can realize real-time and desired playing control of audio information.
[0006] According to one embodiment of the present invention, there is provided an audio information playback method, which reads out audio information obtained by time-serializing waveform data of respective multiple sound units that determine pronunciation pitches and pronunciation orders, reads out segmentation information, which is associated with the audio information and specifies a playback start position, a loop start position, a loop end position, and a playback end position for each of the sound units, obtains note-on information and note-off information, and in response to the fact that the note-on information or the note-off information is obtained, moves the playback position of the audio information based on the segmentation information, and in response to the fact that the note-off information corresponding to the note-on information is obtained, starts playback of the sound unit of the playback object from the loop end position to the playback end position.
[0007] According to another aspect of the present invention, there is provided an audio information generating method, which generates audio information, and the audio information is played in response to the acquisition of note-on information or note-off information, and is obtained by time-serializing waveform data of each of a plurality of vocalization units that determine the pronunciation pitch and the pronunciation order. In the audio information generating method, a music score for singing synthesis is obtained, and the music score for singing synthesis is obtained by time-serializing information that specifies the pitch of a singing voice to be synthesized in a manner that matches the progress of the music, and the audio information is generated by performing singing synthesis on the music score for singing synthesis, and segmentation information is associated with the audio information, wherein the segmentation information specifies a playback start position, a loop start position, a loop end position, and a playback end position that start playback in response to the note-on information for each vocalization unit of the music score for singing synthesis, and the playback end position is the position where playback ends in response to the acquisition of note-off information.
[0008] Effects of the Invention
[0009] According to one aspect of the present invention, the playback of audio information can be controlled in real time and as desired.
[0010] According to another aspect of the present invention, audio information that can be controlled to be played back as desired can be generated in real time. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 It is a block diagram of an audio information playing device.
[0012] Figure 2 This is a conceptual diagram showing the relationship between the music score for singing synthesis and the data for playback.
[0013] Figure 3 It is a functional block diagram of an audio information playing device.
[0014] Figure 4 A conceptual diagram showing a part of the waveform sample data of the audio information and the segmentation information.
[0015] Figure 5 This is a diagram showing an example of segmentation information for one phrase in a musical score for singing synthesis.
[0016] Figure 6 This is a diagram showing an example of segmentation information for one phrase in a musical score for singing synthesis.
[0017] Figure 7 It is a flowchart of real-time playback processing.
[0018] Figure 8 This is a diagram showing a modified example of segmentation information for one phrase in a music score for singing synthesis. DETAILED DESCRIPTION
[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0020] Figure 1 1 is a block diagram of an audio information playback device to which the audio information playback method according to one embodiment of the present invention is applied. The audio information playback device 100 has a function of playing audio information. The audio information playback device 100 can also be used as a device having a function of generating audio information. Therefore, the name of the device to which the present invention is applied is arbitrary. For example, in the case where the present invention is mainly applied to a device having a function of playing audio information, the device can be referred to as an audio information playback device to which the audio information playback method is applied. In addition, in the case where the present invention is mainly applied to a device having a function of generating audio information, the device can be referred to as an audio information generating device to which the audio information generating method is applied.
[0021] The audio information playback device 100 comprises a bus 23, a CPU (Central Processing Unit) 10, a timer 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, and a memory 14.
[0022] 13 and a storage unit 14. The audio information playback device 100 also includes a performance operation element 15, a setting operation element 17, a display unit 18, a sound source 19, an effect circuit 20, a sound system 21, and a communication I / F (Interface) 22.
[0023] The bus 23 transfers data between the various parts of the audio information playback device 100. The CPU 10 is a central processing unit that controls the entire audio information playback device 100. The timer 11 is a module that measures time. The ROM 12 is a non-volatile memory that stores control programs, various data, etc. The RAM 13 is a volatile memory used as a work area of the CPU 10 and various buffers. The display unit 18 is a display module such as a liquid crystal display panel, an organic EL (Electro-Luminescence) panel, etc. The display unit 18 displays the operating status of the audio information playback device 100, various setting screens, messages to the user, etc.
[0024] The performance operating member 15 is a module that mainly receives the performance operation for specifying the pitch and timing. In this embodiment, the audio information (audio data) can be played according to the operation of the performance operating member 15. The audio information playback device 100 is configured as a keyboard instrument type, for example, and the performance operating member 15 includes a plurality of keys (not shown) of the keyboard part. However, the form of the audio information playback device 100 is arbitrary, and the performance operating member 15 can be other forms, such as strings, as long as it is an operating member for specifying the pitch and timing. In addition, the performance operating member 15 is not limited to a physical operating member, and can be a virtual performance operating member displayed on the screen through software.
[0025] The setting operation member 17 is an operation module for performing various settings. The external storage device 3 can be connected to the audio information playback device 100, for example. The storage unit 14 is, for example, a hard disk or a non-volatile memory. The communication I / F 22 is a communication module for communicating with an external device. The communication I / F 22 may also include a MIDI (musical instrument digital interface) interface, a USB (Universal Serial Bus), etc. The program for implementing the present invention may be pre-stored in the ROM 12, or may be obtained via the communication I / F 22 and stored in the storage unit 14.
[0026] In addition, it is not necessary to Figure 1 At least a part of the hardware shown is built into the audio information playback device 100, and may be implemented by an external device connected via an interface such as USB. In addition, the setting operation element 17 and the like may be a virtual operation element displayed on the screen and operated by touch.
[0027] The storage unit 14 can also store one or more music scores 25 for singing synthesis and one or more broadcasting data 28 (see Figure 2). The music score 25 for song synthesis includes information required for song synthesis and lyrics text data. The information required for song synthesis includes the start time and end time of the note, the pitch of the note, the pronunciation mark in the note, and additional parameters for expression (vibrato, consonant length designation, etc.). The lyrics text data is data that describes the lyrics, and the lyrics of each piece of music are recorded by dividing it into syllable units. That is, the lyrics text data has character information that divides the lyrics into syllables, and the character information is also information for display corresponding to the syllables. Here, a syllable refers to a unit that is perceived and pronounced as a complete sound. In this embodiment, one or more speech "groups" associated with one note are called "voice units". "Syllable" is an example of "voice unit". As another example of "voice unit", "mora" is cited. Mora represents a unit of sound with a certain time length. For example, a mora represents a unit of time length equivalent to one character "かな (ka na kana)" in Japanese. As the "voice unit", either "syllable" or "beat" can be used, or "syllable" and "beat" can be mixed and used in a piece of music or a phrase. For example, "syllable" and "beat" can be used separately according to the performance or lyrics.
[0028] In addition, the phoneme information database is stored in the storage unit 14 and is referred to by the sound source 19 during song synthesis. The phoneme information database is a database that stores speech segment data. The speech segment data is data representing the waveform of the speech, for example, spectral data of a sample sequence of the speech segment as waveform data. The speech segment data includes segment pitch data representing the pitch of the waveform of the speech segment. Lyrics text data and speech segment data can be managed separately by databases.
[0029] The sound source 19 converts the performance data and the like into a sound signal. When the singing voice is pronounced based on the singing synthesis sequence data, that is, the singing synthesis score 25, the sound source 19 generates waveform data of the synthesized singing voice, that is, singing voice data, with reference to the phoneme information database read from the storage unit 14. The effect circuit 20 applies a specified sound effect to the singing voice data generated by the sound source 19. The sound system 21 converts the singing voice data processed by the effect circuit 20 into an analog signal through a digital / analog converter. Furthermore, the sound system 21 amplifies the singing voice converted into an analog signal and outputs it from a speaker or the like.
[0030] In this embodiment, in addition to normal playback in which the audio information 26 is played in sequence from the beginning of the music, real-time playback in accordance with the operation of the performance operating member 15 can also be performed. In addition, the audio information 26 can be stored in the storage unit 14 in advance or obtained from the outside afterwards. In addition, the CPU 10 can also generate the audio information 26 by performing singing synthesis on the singing synthesis sheet 25 and converting it into Wave data.
[0031] Figure 2 It is a conceptual diagram showing the relationship between the music score 25 for singing synthesis and the data for playback 28 before singing is synthesized. The data for playback 28 is audio information with segmentation information, and is composed of audio information 26 and segmentation information 27 associated with the audio information 26. The music score 25 for singing synthesis is data that has been time-seriesized so that the information specifying the pitch of the singing voice to be synthesized matches the progress of the music. The music score 25 for singing synthesis is composed of multiple phrases (phrases a to e). Except for the beginning and end of the music, a complete set of syllable groups (sometimes 1 syllable) that are continuously pronounced between rests is equivalent to 1 phrase. Alternatively, a complete set of beat groups (sometimes 1 beat) between rests is equivalent to 1 phrase. Alternatively, a collection of syllables and beats between rests is equivalent to 1 phrase. That is, 1 phrase is composed of 1 or more "voice units".
[0032] The audio information 26 generated by performing singing synthesis on the singing synthesis score 25 has a plurality of phrases (phrases A to E) corresponding to the phrases (phrases a to e) of the singing synthesis score 25. Therefore, the audio information 26 is waveform sample data in which waveform data (a plurality of waveform samples) of a plurality of syllables whose pronunciation pitches and pronunciation orders are determined are time-series-based.
[0033] like Figure 2 As shown, in the playback of the audio information 26, a global play pointer PG and a local play pointer PL are used. The global play pointer PG is global position information for determining which note to play when the note is on. The play pointer PL is position information indicating the playback position within a specific note that becomes the playback object according to the global play pointer PG. In real-time playback, the global play pointer PG moves in note units according to the operation of the performance operating member 15. In addition, the CPU 10 moves the play pointer PL within the note to be played based on the segmentation information 27 associated with the audio information 26. In other words, as Figure 2 As shown in FIG. 1 , the global play pointer PG moves at the division of the syllable, and the play pointer PL moves within the syllable. In other words, the global play pointer PG moves in "sound units", and the play pointer PL moves within "sound units". Figure 4Specific examples of the waveform samples of the audio information 26 and the segmentation information 27 are described in detail.
[0034] In order to create segmentation information 27, the sound source 19 outputs additional information when converting the music score 25 for singing synthesis into audio information 26. This additional information is output for each synthesis frame unit (e.g., 256 samples) of the sound source 19. In audio information, each syllable is composed of a plurality of speech segments. Moreover, each speech segment is composed of a plurality of frames. That is, in audio information, each "voice unit" is composed of a plurality of speech segments. This additional information includes, for example, the segment sample used in the frame (in Figure 5 [Sil-dZ], [i], etc. described later), the position within the fragment sample of the frame (information indicating which side of the Si or dZ in [Sil-dZ] the position is). In addition, the above-mentioned additional information may also include the synthesis pitch and phase information of the frame. The CPU 10 determines the segmentation information 27 that should be played corresponding to each note by matching the additional information with the music score 25 for singing synthesis. In addition, when the above-mentioned additional information cannot be obtained (when natural singing voice is input, etc.), the phoneme recognizer can be used to obtain information equivalent to the additional information.
[0035] Figure 3 1 is a functional block diagram of the audio information playback device 100. The audio information playback device 100 includes a first reading unit 31, a second reading unit 32, a first acquisition unit 33, a point moving unit 34, and a playback unit 35 as main functional blocks related to audio information playback. The audio information playback device 100 includes a second acquisition unit 36 and a generation unit 37 as main functional blocks related to audio information generation.
[0036] Regarding the audio information playback function, the functions of the first reading unit 31 and the second reading unit 32 are mainly realized by the coordinated operation of the CPU 10, RAM 13, ROM 12 and the storage unit 14. The function of the first acquisition unit 33 is mainly realized by the coordinated operation of the performance operating member 15, CPU 10, RAM 13, ROM 12 and the timer 11. The function of the point moving unit 34 is mainly realized by the coordinated operation of the CPU 10, RAM 13, ROM 12, timer 11 and the storage unit 14. The function of the playback unit 35 is mainly realized by the coordinated operation of the CPU 10, RAM 13, ROM 12, timer 11, storage unit 14, effect circuit 20 and sound system 21.
[0037] The first reading unit 31 reads out the audio information 26 from the storage unit 14 or the like. The second reading unit 32 reads out the segmentation information 27 associated with the audio information 26 from the storage unit 14 or the like. The first acquisition unit 33 detects the operation of the performance operating member 15, and acquires the note-on information and the note-off information based on the detection result. In addition, the mechanism for detecting the operation of the performance operating member 15 is arbitrary, for example, it can be a mechanism that detects the operation optically. In addition, the note-on information and the note-off information can also be information acquired from the outside via communication. In accordance with the acquired note-on information or note-off information, the point moving unit 34 moves the global play pointer PG and / or the play pointer PL based on the segmentation information 27.
[0038] exist Figure 4 The detailed operation related to the playback unit 35 is described. Briefly, first, in response to the fact that note-on information is obtained, the playback unit 35 starts playback from the playback start position of the syllable of the playback object indicated by the global playback pointer PG (at this time point, the position indicated by the playback pointer PL). In addition, when the playback pointer PL reaches the loop section, the playback unit 35 enters the loop playback of the loop section. And, in response to the fact that note-off information corresponding to the note-on information is obtained, the playback unit 35 starts playback from the end of the loop section of the syllable of the playback object, that is, the loop end position to the playback end position. The note-off information corresponding to the note-on information is information obtained by, for example, releasing the same key as the key that was pressed among the keys included in the performance operating member 15.
[0039] On the other hand, regarding the audio information generation function, the function of the second acquisition unit 36 is mainly realized by the coordinated action of the CPU 10, RAM 13, ROM 12, and storage unit 14. The function of the generation unit 37 is mainly realized by the coordinated action of the CPU 10, RAM 13, ROM 12, timer 11, and storage unit 14. The second acquisition unit 36 acquires the music score 25 for singing synthesis from the storage unit 14 and the like. The generation unit 37 generates audio information 26 by performing singing synthesis on the acquired music score 25 for singing synthesis, and associates the segmentation information 27 with the generated audio information 26 for each syllable of the music score 25 for singing synthesis. Through this processing, the generation unit 37 generates the broadcast data 28. In addition, the broadcast data 28 used in real time is not limited to the data generated by the generation unit 37.
[0040] Figure 4 2 is a conceptual diagram showing a part of the waveform sample data of the audio information 26 and the segmentation information 27. Figure 4 In FIG. 1 , arrows are used to indicate an example of the playback order of the audio information 26. The audio information 26 is usually played in units of one song, but in Figure 4, showing the waveform of a phrase consisting of 5 syllables. The waveform sample data corresponding to the 5 syllables in the phrase are called samples SP1, SP2, SP3, SP4, and SP5 in sequence. Each sample SP corresponds to each syllable of the music score 25 for singing synthesis before singing synthesis. The playback start position S, loop interval RP, joint C, and playback end position E are specified for each sample SP (corresponding to each syllable) through the segmentation information 27 associated with the audio information 26. The loop interval RP is an interval that starts from the loop start position and ends at the loop end position. The playback start position S indicates the position where playback starts in response to the note-on information. The loop interval RP is a playback interval that becomes the object of loop playback. The playback end position E indicates the position where playback ends in response to the fact that the note-off information is obtained. The boundary between adjacent sample SPs in the phrase is the joint C (C1 to C4).
[0041] For example, for sample SP1, playback start position S1, loop section RP1 and playback end position E1 are defined. Similarly, for samples SP2 to SP5, playback start positions S2 to S5, loop sections RP2 to RP5 and playback end positions E2 to E5 are defined, respectively.
[0042] The joint C1 is the division position between samples SP1 and SP2, and coincides with the playback start position S2 and the playback end position E1. The joint C2 is the division position between samples SP2 and SP3, and coincides with the playback start position S3 and the playback end position E2. The joint C3 is the division position between samples SP3 and SP4, and coincides with the playback start position S4 and the playback end position E3. The joint C4 is the division position between samples SP4 and SP5, and coincides with the playback start position S5 and the playback end position E4.
[0043] In a phrase, for a sample SP having adjacent sample SPs on both sides ( Figure 4 The playback start position S and the playback end position E are the same as the playback end position E of the front sample SP and the playback start position S of the back sample SP, respectively. The sample SP (syllable) at the beginning of the phrase ( Figure 4 The playback start position S of the sample SP1 is the front position of the sample SP. The last sample SP (syllable) in the phrase ( Figure 4 The playback end position E of the sample SP5) is the end position of the sample SP. The loop section RP is a section corresponding to the stationary part (Stationary part vowel part) of the syllable of the music score 25 for singing synthesis.
[0044] Based on the segmentation information 27, the following playback is performed according to the user's operation on the performance operating element 15. The first acquisition unit 33 acquires note-on information when it detects that the performance operating element 15 is pressed, and acquires note-off information when it detects that the pressed performance operating element 15 is released.
[0045] For example, it is assumed that note-on information is obtained in a state where there is no phrase preceding the sample SP1 or the playback of the phrase preceding the sample SP1 has ended. In this way, the point moving unit 34 moves the global playback pointer PG to the playback start position S1, and sets the playback pointer PL at the playback start position S1. In this way, the sample SP1 becomes the playback object, and the playback unit 35 starts to play from the playback start position S1. After playing from the playback start position S1, the playback pointer PL is gradually moved backward by the point moving unit 34 at a prescribed playback speed. The prescribed playback speed is, for example, the same as the playback speed when the audio information 26 is generated by performing singing synthesis on the music score 25 for singing synthesis. If the playback pointer PL reaches the front end of the loop interval RP1, that is, the loop start position, the playback of the loop interval RP1 begins.
[0046] When playing the loop section RP1 of the real-time performance, the playing section 35 may convert the pitch of the loop section RP1 into a pitch based on the note-on information and play it. In this case, the playing pitch is changed according to which key of the performance operating member 15 is pressed.
[0047] For example, the playback unit 35 can play the sample SP1 by performing pitch shifting (Pitch shift) in a manner corresponding to the pitch of the note-on based on the pitch of the singing synthesis score 25 corresponding to the sample SP1 and the input pitch information of the note-on. In addition, the pitch shift can be applied not only to the loop section RP1 but also to the entire sample SP1.
[0048] Finally, if the play pointer PL reaches the end of the loop section RP, i.e., the loop end position, the point moving unit 34 reverses the moving direction of the play pointer PL and moves the play pointer PL to the front end of the loop section RP1, i.e., the loop start position. Then, if the play pointer PL reaches the loop start position, the point moving unit 34 returns the moving direction of the play pointer PL to the forward direction (backward) and moves the play pointer PL toward the loop end position. The reversal of the moving direction of the play pointer PL in the loop section RP1 is repeated until the note-off information corresponding to the note-on information of this time is obtained. Therefore, the loop section RP is played in a loop. Finally, if the note-off information is obtained, the point moving unit 34 jumps the play pointer PL from the playback position at that time to the end of the loop section RP1, i.e., the loop end position. Then, the playback unit 35 starts playing from the loop end position to the playback end position E1. At this time, the playback unit 35 can also play smoothly by performing cross-fade playback. Furthermore, even when note-off information is acquired before the play pointer PL reaches the loop section RP1 , the point moving unit 34 jumps the play pointer PL to the loop end position.
[0049] After starting playback from the end of the loop interval RP1, i.e., the loop end position, and ending playback to the next playback end position E, i.e., the playback end position E1, the playback unit 35 ends playback of the sample SP1. At the same time, the playback unit 35 discards the local playback pointer PL. Then, if the next note-on information is obtained, as a fixed processing of the sequence position, first, the point moving unit 34 determines the moving destination of the global playback pointer PG and moves the global playback pointer PG to the moving destination. For example, if the global playback pointer PG is set to move to the playback start position S2, then the playback unit 35 starts playback of the sample SP2 according to the new playback pointer PL with the playback start position S2 set as the playback start position.
[0050] The playback action of the sample SP2 is the same as that of the sample SP1. In addition, the playback actions of the samples SP3 and SP4 are also the same as that of the sample SP1. For the sample SP5, if the playback of the loop interval RP5 from the loop end position to the playback end position E5 is completed, then Figure 4 The playback of the phrase shown in ends. Figure 4 In the case where there is a subsequent phrase, the point moving unit 34 moves the global play pointer PG to the front end of the sample SP at the beginning of the subsequent phrase. Figure 4 In a case where the phrase shown is the final phrase of the audio information 26, the playback of the audio information 26 ends.
[0051] In addition, the method of loop playback of the loop interval RP is arbitrary. Therefore, it is not necessary to go back and forth in the loop interval RP, but to repeat the forward playback from the loop start position to the loop end position. In addition, the loop playback can also be achieved by using time expansion technology.
[0052] use Figure 5 , Figure 6 , indicating that in the generation unit 37 ( Figure 3 ) When generating the playback data 28 based on the music score 25 for singing synthesis, how to associate the segmentation information 27 with the audio information 26. In addition, if it is limited to the audio information playback method for implementing the present invention, the segmentation information 27 can also be associated afterwards by analyzing the normal audio information. However, in order to associate the segmentation information 27 with higher accuracy, the generation unit 37 generates the segmentation information 27 and associates it at the stage of generating the audio information 26 by performing singing synthesis on the music score 25 for singing synthesis. In addition, the playback start position S1, the loop interval RP1 (loop start position and loop end position), the joint C and the playback end position E1 do not necessarily have to be associated with the audio information 26. Figure 4 The content of the segmentation information 27 varies depending on the rule applied when generating the broadcast data 28. Figure 5 , Figure 6 A representative example of the setting of the segmentation information 27 for enabling natural pronunciation is described below. Figure 8 Modifications will be described.
[0053] Figure 5 , Figure 6 2 is a diagram showing an example of segmentation information for one phrase of the music score 25 for singing synthesis. Figure 5 The segmentation information for the following musical phrase is shown as an example. The musical phrase is composed of three Japanese syllables: "じ (pronounced as [JI] in Japanese characters)", "こ (pronounced as [KO] in Japanese characters)", and "ちゅ() (pronounced as [CYU] in Japanese characters". Figure 6 The segmentation information for a phrase consisting of three syllables of “I”, “test”, and “it” in English is shown as an example. Figure 5 , Figure 6 The playback start position s (s1 to s3) and the playback end position e (e1 to e3) in the singing synthesis score 25 shown in FIG. Figure 4 The playback start position S and the playback end position E of the audio information 26 shown in FIG. Figure 5 , Figure 6 The loop sections loop (loop1 to loop3) and the joints c (c1, c2) of the song synthesis score 25 shown in FIG. Figure 4The loop section RP and the joint section C of the audio information 26 shown correspond to each other.
[0054] exist Figure 5 , Figure 6 In the example, syllables are represented by phoneme symbols based on the form of X-SAMPA (Extended Speech Assessment Methods Phonetic Alphabet). The speech segment database constituting the music score 25 for singing synthesis stores speech segment data of single phonemes such as [a] and [i], or speech segment data of phoneme chains such as [ai] and [ap].
[0055] exist Figure 5 In the example, "じ (Japanese character pronounced as [JI])", "こ (Japanese character pronounced as [KO])", and "ちゅ (Japanese character pronounced as [CYU])" are phonetic characters. If "じ (Japanese character pronounced as [JI])" is represented by phonetic symbols, it is [dZ-i]. If "こ (Japanese character pronounced as [KO])" is represented by phonetic symbols, it is [ko]. If "ちゅ (Japanese character pronounced as [CYU])" is represented by phonetic symbols, it is [ts-M]. In the music score 25 for singing synthesis, the mark of the speech segment of the first syllable of the phrase starts with "Sil-", and the mark of the speech segment of the last syllable ends with "-Sil". In addition, the speech segment of the phoneme chain is configured between the phonemes of the connected pronunciation. Therefore, if "じ (Japanese character pronounced as [JI])", "こ (Japanese character pronounced as [KO])", "ちゅ (Japanese character pronounced as [CYU])" which are pronounced together as one phrase are represented by phonetic symbols, they would be [Sil-dZ][dZ-i][i][ik][ko][o][o-tS][tS-M][M][M-Sil].
[0056] Regarding the playback start position s, the playback start position s1 of the first syllable in the phrase, i.e., "じ (Japanese character [JI])", is the front position of dZ in the speech segment [Sil-dZ]. In addition, the playback start position s of the rear syllable of the two adjacent syllables in the phrase is the rear position of the speech segment composed of the last phoneme of the front syllable and the first phoneme of the rear syllable. For example, for the "こ (Japanese character [KO])" of the adjacent "じ (Japanese character [JI])" and "こ (Japanese character [KO])", the rear position of the speech segment [ik] composed of the last phoneme (i) of "じ (Japanese character [JI])" and the first phoneme (k) of "こ (Japanese character [KO])" becomes the playback start position s2. For "こ(Japanese character [KO])" and "ちゅ(Japanese character [CYU])", the rear end position of the speech segment [o-tS] becomes the playback start position s3.
[0057] Regarding the playback end position e, the playback end position e of the syllable on the front side is the same position as the playback start position s of the syllable on the rear side. For example, the playback end position e1 of "じ(Japanese character [JI])" in the adjacent "じ(Japanese character [JI])" and "こ(Japanese character [KO])" is the same position as the playback start position s2 of "こ(Japanese character [KO])". The playback end position e2 of "こ(Japanese character [KO])" in "ちゅ(Japanese character [CYU])" is the same position as the playback start position s3 of "ちゅ(Japanese character [CYU])". In addition, the playback end position e3 of the last syllable in the phrase, namely "ちゅ(Japanese character [CYU])", is the rear end position of M in the voice segment [M-Sil].
[0058] The speech segments [i], [o], and [M] are fixed parts of each syllable. The intervals of these fixed parts are loop1, loop2, and loop3. In addition, the joints c1 and c2 are the same positions as the playback end positions e1 and e2, respectively. As mentioned above, in Japanese phrases, the joint c is located between consonants.
[0059] The generator 37 generates the segmentation information 27 at the stage of generating the audio information 26 by performing song synthesis on the song synthesis score 25. At this time, the generator 37 generates the segmentation information 27, which associates the playback start position s, the loop section loop (loop start position and loop end position), the joint c, and the playback end position e with the playback start position S, the loop section RP (loop start position and loop end position), the joint C, and the playback end position E, respectively. Then, the generator 37 generates the playback data 28 by associating the generated segmentation information 27 with the audio information 26. Therefore, in the audio information 26, the playback start position s of the first syllable among the adjacent multiple syllables in each phrase becomes the front position of the first syllable. In addition, in the audio information 26, the playback end position e of the last syllable among the adjacent multiple syllables in each phrase becomes the end position of the last syllable.
[0060] Furthermore, when performing song synthesis on the song synthesis score 25, the length of the fixed section interval (loop interval loop) for each syllable of the song synthesis score 25 may be less than the prescribed time. In this case, since the loop interval RP is too short, it may not be properly looped. Therefore, the generator 37 may set the loop interval RP in the segmentation information 27 so that the length of the fixed section interval becomes a length longer than the prescribed time.
[0061] Next, in Figure 6 In the example, if "I", "test", and "it" are represented by phonetic symbols, they become [Sil-aI][aI][aI-t][te][e][es][st][ti][i][it][t-Sil].
[0062] Regarding the playback start position s, the playback start position s1 of the first syllable in the phrase, i.e., "I", is the front position of aI in the speech segment [Sil-aI]. The playback start position s2 of "test" is the back position of the speech segment [aI-t]. The playback start position s3 of "it" is the back position of the speech segment [st].
[0063] Regarding the playback end position e, the playback end position e1 of "I" is the same as the playback start position s2 of "test". The playback end position e2 of "test" is the same as the playback start position s3 of "it". In addition, the playback end position e3, which is the last syllable in the phrase, is the rear end position of t in the speech segment [t-Sil].
[0064] Figure 7This is a flowchart of the real-time broadcast processing. This processing is realized by, for example, the CPU 10 developing a program stored in the ROM 12 into the RAM 13 and executing the program.
[0065] If the power is turned on, the CPU 10 waits until the user receives an operation to select a piece of music to be played (step S101). In addition, if there is no operation to select a piece of music after a certain period of time, the CPU 10 can also determine that the music set by default is selected. If the CPU 10 receives the selection of a piece of music, it performs initial settings (step S102). In this initial setting, the CPU 10 reads the playback data 28 (audio information 26 and segmentation information 27) of the selected piece of music, and sets the sequencing position to the initial position. That is, the CPU 10 makes the global playback pointer PG and the playback pointer PL located at the front end of the beginning syllable of the beginning phrase of the audio information 26.
[0066] Next, the CPU 10 determines whether a note-on is detected (note-on information is obtained) based on the operation of the performance operating member 15 (step S103). If a note-on is not detected, the CPU 10 determines whether a note-off is detected (note-off information is obtained) (step S107). On the other hand, if a note-on is detected, the CPU 10 performs a sequence position determination process (step S104).
[0067] In this determination process, the positions of the global play pointer PG and the local play pointer PL are determined. For example, if the difference between the time when the previous note is on and the time when the current note is on is greater than a specified value, the global play pointer PG moves forward by one position. In addition, the accompaniment of the selected music can also be played in parallel with the real-time playback process. In this case, the global play pointer PG can be moved to follow the playback position of the accompaniment. Alternatively, the accompaniment can be played in accordance with the movement of the global play pointer PG.
[0068] If Figure 4 For example, when the global play pointer PG and the play pointer PL are located at the play start position S1 of the sample SP1, the CPU 10 starts the process of advancing the play pointer PL in the sample SP1. When the play pointer PL is located at the loop section RP1 (in loop playback), the CPU 10 advances the play pointer PL so as to reciprocate in the loop section RP1.
[0069] Furthermore, in the above-mentioned same processing, when multiple notes are detected by pressing multiple keys within a certain period of time, the CPU 10 may not advance the position of the global play pointer PG, but may make the sample SP1 sound in harmony at multiple scales. Alternatively, the CPU 10 may advance the position of the global play pointer PG so that the sample SP1 and the sample SP2 sound at the same time at each scale. Furthermore, when two keys are pressed with a certain time interval, after being determined as YES in step S103, it is determined as YES in step S107, and then it is determined as YES again in step S103.
[0070] In addition, a structure may be set to output only a single tone even when multiple keys are operated simultaneously. In this case, the highest pitch among the keys operated simultaneously may be processed, or the lowest pitch may be processed. In the case of multiple keys being pressed within a certain period of time, the pitch of the last key pressed may also be processed.
[0071] Next, in step S105, the CPU 10 reads out the sample at the sequence position in the audio information 26. In step S106, the CPU 10 starts the sounding process of making the sample read out in step S105 sound. In addition, the CPU 10 shifts the sounding pitch according to the difference between the pitch specified in the audio information 26 and the pitch based on the note-on information of this time. Through this process, the pitch of the sample to be played is converted into the pitch based on the note-on information and played. In addition, in the case of chord sounding, it is sounded at a plurality of pitches based on each note-on information. After step S106, the CPU 10 enters the process into step S107.
[0072] In step S107, if the note-off is not detected, the key state continues, so the CPU 10 determines whether there is a sample being pronounced (step S110). If there is no sample being pronounced, the CPU 10 returns the process to step S103. On the other hand, if there is a sample being pronounced, the CPU 10 performs a pronunciation continuation process (step S111), and returns the process to step S103. Figure 4 In the example shown, for example, if the sample SP1 is being sounded, the playback of the position indicated by the playback pointer PL and thereafter is continued. In particular, when the playback pointer PL is located at the loop section RP1, the loop playback of the loop section RP1 is continued.
[0073] In step S107, when the note-off is detected, it can usually be determined that the pressed key has been released, so the CPU 10 executes the sounding stop process in step S108. Here, the CPU 10 jumps the playback pointer PL to the end of the loop interval RP of the sample SP being sounded, that is, the loop end position, and starts playing from the jumped position to the playback end position E adjacent to the rear. Figure 4 In the example shown, when note-off information is obtained in the sound of sample SP1, the CPU 10 causes the playback pointer PL to jump to the loop end position of loop section RP1. At the same time, the CPU 10 starts playback from the loop end position of loop section RP1 to the playback end position E1 adjacent to the back. Figure 6 In the example of , when "test" is extended and played, "e" as a vowel is extended, and then played until the playback end position E1 in accordance with the note-off information, so that "st" as a consonant is fully pronounced. Therefore, "test" can be played in a natural extended manner.
[0074] Next, in step S109, the CPU 10 determines whether the sequencing has ended, that is, whether the audio information 26 of the selected music has been played to the end. If the audio information 26 of the selected music has not been played to the end, the CPU 10 returns the process to step S103. If the audio information 26 of the selected music has been played to the end, the CPU 10 returns the process to step S104. Figure 7 The real-time playback process shown ends.
[0075] According to the present embodiment, it is possible to realize real-time and desired playback control of audio information. In particular, the CPU 10 starts playback from the playback start position S in response to the fact that the note-on information is obtained, and enters the loop playback when reaching the loop interval RP. In addition, the CPU 10 starts playback from the end of the loop interval RP of the syllable of the playback object, that is, the loop end position to the playback end position e in response to the fact that the note-off information corresponding to the note-on information is obtained. The user can pronounce the syllables in sequence at the desired timing by operating the performance operating member 15. Moreover, by continuing to press the performance operating member 15, the sound of the desired syllable can be extended as desired through the loop playback of the loop interval RP. Moreover, through pitch shifting, the pronunciation pitch of the syllable can be changed in response to the performance operating member 15 operated by the user while playing. Therefore, the playback of audio information can be controlled in real time as desired.
[0076] The CPU 10 generates audio information 26 by performing singing synthesis on the singing synthesis score 25, and associates the segmentation information 27 with the audio information 26 for each syllable of the singing synthesis score 25. Therefore, audio information that can be played back in a desired manner can be generated in real time. In addition, the association accuracy of the segmentation information 27 can be improved.
[0077] The loop section RP is a section corresponding to the fixed portion of each syllable in the song synthesis score 25. Furthermore, when the length of the fixed portion section is less than a predetermined time for each syllable of the song synthesis score 25, the CPU 10 associates a section that makes the length of the fixed portion section equal to or longer than the predetermined time as the loop section RP with the audio information 26. Therefore, the sound during loop playback can be made natural.
[0078] Next, behind Figure 8 A modification example of the setting of the division information 27 will be described. Figure 8 2 is a diagram showing a modified example of segmentation information for one phrase in the music score 25 for singing synthesis. Figure 8 In the example of , segmentation information for a phrase consisting of two syllables, "start" and "start" in English is illustrated. Figure 8 The three modes 1), 2), and 3) have the following characteristics.
[0079] First, in mode 1), all consonant parts are included after the note is on. Therefore, when each note is pronounced slowly and individually, each pronunciation (such as the "サ" line (Japanese [Sa] line) becomes clear. On the other hand, when the pronunciation is matched with the accompaniment, it is necessary to play it quite early depending on the type of consonant.
[0080] In mode 2), the consonants and the connecting parts of the consonants are set to be difficult to detect. In addition, as a variation of this, it is also possible to use the position that is a certain length ahead of the note start as the segmentation position regardless of the type of consonant. In this case, since it is sufficient to perform the performance ahead of time at a certain time regardless of the lyrics, it is relatively easy to perform the performance at a time that matches the accompaniment.
[0081] In mode 3), it is possible to play at the same position as the note opening position of the original singing synthesis score. However, in the case of a separate pronunciation, even if the note of the lyrics of "さ (Japanese character [Sa])" is played, only the [a] part is pronounced.
[0082] Among the three modes 1), 2), and 3), mode 2) is applied Figure 6 The pattern is the same as the rule explained in . If "start" is represented by phonemic symbols,
[0083] It becomes [Sil-s][st][tQ@][Q@][Q@-t][ts][st][tQ@][Q@][Q@-t][t-Sil].
[0084] In any of the modes 1), 2), and 3), the playback end position e of the rear "start" is the rear end position of t in the voice segment [t-Sil]. In addition, in any of the modes 1), 2), and 3), the voice segment [Q@] is a fixed part of each syllable, and these intervals become loop intervals loop.
[0085] In mode 1), regarding the playback start position s, the playback start position s of the beginning "start" in the phrase is the front end position of s in the speech segment [Sil-s]. In addition, the playback start position s of the rear syllable of the two adjacent syllables in the phrase is the same as the joint c. That is, the joint c is the front end position of the rear phoneme in the speech segment composed of the last phoneme of the front syllable and the first phoneme of the rear syllable. For example, the front end position of s in [ts] becomes the joint c. The playback end position e of the front syllable is the same as the playback start position s of the rear syllable and the joint c.
[0086] In mode 3), the playback start position s is the front position of the phoneme (the phoneme corresponding to the fixed part) on the rear side of the speech segment composed of the phoneme (the phoneme corresponding to the fixed part) extended as the loop interval loop and the previous phoneme of the phoneme. For example, the front position of Q@ in the first [tQ@] becomes the playback start position s. In addition, the playback start position s of the rear syllable is the same as the junction c. The junction c is the front position of Q@ in the second [tQ@]. The playback end position e of the front syllable is the same as the playback start position s of the rear syllable and the junction c.
[0087] As described above, the rule applied when generating the broadcast data 28 is not limited to one type. In addition, the rule applied may be different depending on the language.
[0088] In addition, when the length of the fixed section interval (loop interval loop) is less than the prescribed time, it is assumed that the length of the loop interval RP cannot be sufficiently ensured in the audio information 26 without using a process to increase the length of the fixed section interval. In this case, in step S111, for example, the interval [i] of the voice segment [dZ-i] can be used for loop playback.
[0089] Furthermore, even if the music score 25 for song synthesis has expressive parameters such as vibrato, these information may be ignored and the music score 25 for song synthesis may be converted into the audio information 26. On the other hand, the playback data 28 may include expressive parameters such as vibrato as information. Even in this case, the playback of expressive parameters such as vibrato may be invalidated in the real-time playback processing of the audio information 26 in the playback data 28. Alternatively, when playing vibrato, the repetition timing in the loop playback may be made consistent with the amplitude waveform of the vibrato, thereby changing the pronunciation time while maintaining the cycle of the vibrato included in the audio information 26.
[0090] Furthermore, in step S106, formant shift may be used simultaneously. In addition, pitch shift is not necessarily used.
[0091] Furthermore, when the specified sample data is stored in advance and the note-off information is obtained, in step S108, the sample data specified above may be played as an aftertouch process, instead of playing from the end of the loop interval RP, i.e., from the loop end position to the playback end position e. Alternatively, as an aftertouch process, the grouping process shown in the "WO2016 / 152715 Gazette" and the like may be applied. For example, in the case where the syllables "こ(Japanese character [KO])" and "い(Japanese character [I])" are grouped, corresponding to the fact that the note-off information is obtained in the pronunciation of "こ(Japanese character [KO])", after the pronunciation of "こ(Japanese character [KO])" ends, the pronunciation may continue until "い'(Japanese character [I])".
[0092] In addition, the audio information 26 used in the real-time playback process is not limited to audio information having samples SP (waveform data corresponding to syllables) equivalent to the syllables of the song. That is, the audio information playback method of the present invention can also be applied to audio information not based on singing. Therefore, the audio information 26 is not necessarily limited to audio information generated by singing synthesis. In the case of associating the segmentation information with the audio information not based on singing, for example, the interval of loop playback can be associated with S (Sustain) in the envelope waveform, and the end information played when the note is off can be associated with R (Release).
[0093] In addition, in the present embodiment, the performance operating member 15 has a function of specifying the pitch. However, the input operating member for inputting the note-on information and the note-off information may be limited to one or more. In this case, the input operating member may be dedicated, but may also be allocated to a part of the performance operating member 15 (for example, the two white keys of the lowest pitch on the keyboard, etc.). For example, the following structure may be set, that is, each time information is input through the input operating member, the CPU 10 searches for the next split position and moves the global play pointer PG and / or the play pointer PL.
[0094] The number of channels for the sound information 26 is not limited to one. The present invention may be applied to each of a plurality of channels that share the division information 27. In this case, the channel for the accompaniment may be excluded from the process of shifting the sound pitch.
[0095] As mentioned above, although the present invention has been described in detail based on the preferred embodiments of the present invention, the present invention is not limited to these specific embodiments, and various modes within the scope not departing from the gist of the present invention are also included in the present invention.
[0096] In addition, regarding the application of the present invention, when only the audio information playback function is considered, the present device does not necessarily have the audio information generation function. Conversely, when only the audio information generation function is considered, the present device does not necessarily have the audio information playback function.
[0097] In addition, the same effect as the present invention can be achieved by reading a recording medium storing a control program represented by software for implementing the present invention into the present device. In this case, the program code itself read from the recording medium realizes the new function of the present invention, and the non-transitory computer-readable recording medium 5 storing the program code (see Figure 1 ) constitute the present invention. For example, Figure 1 As shown, the CPU 10 can read the program code from the recording medium 5 via the communication I / F 22. In addition, the program code can also be provided via a transmission medium, etc. In this case, the program code itself constitutes the present invention. As a non-temporary computer-readable recording medium 5, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a DVD-ROM, a DVD-R, a magnetic tape, a non-volatile memory card, etc. can be used. In addition, as a non-temporary computer-readable recording medium, it also includes a medium that stores the program for a certain period of time, such as a volatile memory (e.g., DRAM (Dynamic Random Access Memory)) inside a computer system of a server or client when a program is sent via a network such as the Internet or a communication line such as a telephone line.
Claims
1. A method for playing audio information, wherein: The audio information is read out, the audio information being obtained by time-series-forming waveform data of each of a plurality of sound production units whose pronunciation pitches and pronunciation orders are determined, reading out segmentation information, the segmentation information being associated with the audio information and defining a playback start position, a loop start position, a loop end position, and a playback end position for each of the sound emission units, Get note-on information and note-off information, In response to the note-on information or the note-off information being obtained, the playback position of the audio information is moved based on the division information, In response to the fact that the note-off information corresponding to the note-on information is obtained, real-time playback of the sound unit of the playback object from the loop end position to the playback end position is started, In response to the acquisition of the note-on information, playback is started from the playback start position of the sound unit of the playback object indicated by the playback position, and loop playback is started when the playback position reaches the loop start position.
2. The audio information playing method according to claim 1, wherein: The audio information is obtained by performing singing synthesis on a music score for singing synthesis, wherein the music score for singing synthesis is obtained by time-serializing information specifying a pitch of a singing voice to be synthesized in accordance with the progress of the music.
3. The audio information playing method according to claim 2, wherein: The segmentation information is associated with the audio information when singing synthesis is performed on the musical score for singing synthesis.
4. The audio information playing method according to claim 2, wherein: The playback start position of the rear sound unit of the two adjacent sound units of the audio information is equivalent to the rear end position of the voice segment composed of the last phoneme of the front sound unit of the two corresponding sound units in the music score for singing synthesis before singing synthesis and the initial phoneme of the rear sound unit.
5. The audio information playing method according to claim 1 or 2, wherein: The playback end position of the last sound unit among the plurality of sound units in each phrase of the audio information is the end position of the last sound unit.
6. A method for generating audio information, the audio information being played in response to the acquisition of note-on information or note-off information, the audio information being obtained by time-seriesizing waveform data of each of a plurality of sound production units whose pronunciation pitches and pronunciation orders are determined, In the audio information generating method, A music score for singing synthesis is obtained by time-sequencing information specifying the pitch of the singing voice to be synthesized in accordance with the progress of the music. The audio information is generated by performing singing synthesis on the singing synthesis music score, and the segmentation information is associated with the audio information, wherein: The segmentation information specifies, for each vocalization unit of the music score for singing synthesis, a playback start position, a loop start position, a loop end position, and a playback end position, respectively, where playback starts in response to note-on information, and the playback end position is a position where playback ends in response to the acquisition of note-off information. When performing song synthesis on the song synthesis score, the interval of the fixed portion of each sound production unit of the song synthesis score is associated with the audio information as the division information defining the loop start position and the loop end position.
7. The audio information generating method according to claim 6, wherein: When singing is synthesized on the music score for singing synthesis, for each vocalization unit of the music score for singing synthesis, when the length of the interval of the fixed part is less than the specified time, an interval that makes the length of the interval of the fixed part longer than the specified time is associated with the audio information as the segmentation information that specifies the start position of the loop and the end position of the loop.
8. The audio information generating method according to claim 6, wherein: When singing synthesis is performed on the music score for singing synthesis, the rear end position of the voice segment composed of the last phoneme of the front sound unit and the initial phoneme of the rear sound unit of two adjacent sound units of the music score for singing synthesis is associated with the audio information as the segmentation information, wherein the segmentation information specifies the playback start position of the rear sound unit of two adjacent sound units of the audio information.
9. A recording medium storing a program for causing a computer to execute an audio information playback method. The audio information playing method is: The audio information is read out, the audio information being obtained by time-series-forming waveform data of each of a plurality of sound production units whose pronunciation pitches and pronunciation orders are determined, reading out segmentation information, the segmentation information being associated with the audio information and defining a playback start position, a loop start position, a loop end position, and a playback end position for each of the sound emission units, Get note-on information and note-off information, In response to the note-on information or the note-off information being obtained, the playback position of the audio information is moved based on the division information, In response to the fact that the note-off information corresponding to the note-on information is obtained, real-time playback of the sound unit of the playback object from the loop end position to the playback end position is started, In response to the acquisition of the note-on information, playback is started from the playback start position of the sound unit of the playback object indicated by the playback position, and loop playback is started when the playback position reaches the loop start position.
10. A recording medium storing a program that causes a computer to execute an audio information generating method, wherein the audio information generating method generates audio information that is played in response to the acquisition of note-on information or note-off information, and is obtained by time-seriesizing waveform data of each of a plurality of sound production units that determine the pitch and order of production, The audio information generation method is: A music score for singing synthesis is obtained by time-sequencing information specifying the pitch of the singing voice to be synthesized in accordance with the progress of the music. The audio information is generated by performing singing synthesis on the music score for singing synthesis, and segmentation information is associated with the audio information, wherein the segmentation information specifies, for each sounding unit of the music score for singing synthesis, a playback start position, a loop start position, a loop end position, and a playback end position, respectively, where playback starts in response to note-on information, and the playback end position is a position where playback ends in response to the acquisition of note-off information. When performing song synthesis on the song synthesis score, the interval of the fixed portion of each sound production unit of the song synthesis score is associated with the audio information as the division information defining the loop start position and the loop end position.
11. An audio information playing device, comprising: a moving unit that obtains audio information and segmentation information, and moves the playback position of the audio information based on the segmentation information in accordance with the fact that note-on information and note-off information are obtained, wherein: The audio information is obtained by time-serializing waveform data of each of a plurality of sound production units whose pronunciation pitches and pronunciation orders are determined, the segmentation information is associated with the audio information, and a playback start position, a loop start position, a loop end position, and a playback end position are specified for each of the sound production units; and a playing section that, in response to the acquisition of the note-on information, starts playing the sound unit of the playing object from the playing start position indicated by the playing position moved by the moving section, and, in response to the acquisition of the note-off information corresponding to the note-on information, starts playing the sound unit of the playing object in real time from the loop end position to the playing end position, In response to the acquisition of the note-on information, the playback unit starts playback from the playback start position of the sound unit of the playback object indicated by the playback position, and starts loop playback when the playback position reaches the loop start position.
12. The audio information playback device according to claim 11, wherein: The audio information is obtained by performing singing synthesis on a music score for singing synthesis, wherein the music score for singing synthesis is obtained by time-serializing information specifying a pitch of a singing voice to be synthesized in accordance with the progress of the music.
13. The audio information playback device according to claim 12, wherein: The segmentation information is associated with the audio information when singing synthesis is performed on the musical score for singing synthesis.
14. The audio information playback device according to claim 12, wherein: The playback start position of the rear sound unit of the two adjacent sound units of the audio information is equivalent to the rear end position of the voice segment composed of the last phoneme of the front sound unit of the two corresponding sound units in the music score for singing synthesis before singing synthesis and the initial phoneme of the rear sound unit.
15. An audio information generating device, which generates audio information, which is played in response to the acquisition of note-on information or note-off information, and is obtained by time-seriesly converting waveform data of each of a plurality of sound production units whose pronunciation pitches and pronunciation orders are determined, The audio information generating device comprises: an acquisition unit that acquires a music score for singing synthesis, the music score for singing synthesis being obtained by time-serializing information specifying a pitch of a singing voice to be synthesized in accordance with a progression of the music; and a generating unit that generates the audio information by performing singing synthesis on the music score for singing synthesis acquired by the acquiring unit, and associates the segmentation information with the audio information, wherein: The segmentation information specifies, for each vocalization unit of the music score for singing synthesis, a playback start position, a loop start position, a loop end position, and a playback end position, respectively, where playback starts in response to note-on information, and the playback end position is a position where playback ends in response to the acquisition of note-off information. When performing song synthesis on the song synthesis score, the generating unit associates the interval of the fixed part of each sound unit of the song synthesis score with the audio information as the division information defining the loop start position and the loop end position.
16. The audio information generating device according to claim 15, wherein: When the generating unit performs singing synthesis on the music score for singing synthesis, for each vocalization unit of the music score for singing synthesis, if the length of the interval of the fixed part is less than a prescribed time, an interval that makes the length of the interval of the fixed part longer than the prescribed time is associated with the audio information as the segmentation information that specifies the start position of the loop and the end position of the loop.
17. The audio information generating device according to claim 15, wherein: When the generating unit performs singing synthesis on the music score for singing synthesis, the rear end position of the speech segment composed of the last phoneme of the front sound unit and the first phoneme of the rear sound unit of two adjacent sound units of the music score for singing synthesis is associated with the audio information as the segmentation information, wherein the segmentation information specifies the playback start position of the rear sound unit of two adjacent sound units of the audio information.
