Music-piece analysis device, music-piece analysis method, and program

Abstract

Provided is a music-piece analysis device including: a voice analysis portion that extracts, from voice data of a music piece including a first part and a second part that can be separated in voice, voice data of the first part, data of a unit sound of the second part, and data indicating a pronunciation position of the second part; a display portion that causes GUI for rearranging the unit sound of the second part to be displayed; and a mix processing portion that generates mixed voice data in which the unit sound rearranged by an operation via the GUI is mixed with the voice data of the first part, in which the voice analysis portion detects a rhythm pattern of the music piece by synchronously adding the data indicating the pronunciation position of the second part by the unit of bar, and the display portion displays the GUI on the basis of the rhythm pattern.

Description

Music analysis device, music analysis method, and program

[0001] The present invention relates to a music analysis device, a music analysis method, and a program.

[0002] There is known a technology for separating a piece of music into a plurality of parts acoustically and providing a service using the audio data of each of the separated parts. For example, Patent Document 1 describes a technology for creating arranged part data by arranging predetermined part data from the plurality of part data separated by a music separating means, and transmitting the arranged part data and original music part data obtained by excluding the predetermined part data from the plurality of part data.

[0003] JP 2009-186729 A

[0004] As described above, there are known techniques for separating a piece of music into multiple parts and processing or cutting out only the sounds of specific parts. The present invention aims to go beyond these examples and provide a music analysis device, a music analysis method, and a program for providing new expressions that utilize the separation of musical parts.

[0005] [1] A music analysis device comprising: an audio analysis unit that extracts, from audio data of a music piece including a first part and a second part that are phonetically separable, audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part; a display unit that displays a GUI for rearranging the unit sounds of the second part; and a mix processing unit that generates mixed audio data in which the audio data of the first part is mixed with the unit sounds rearranged by operation via the GUI, wherein the audio analysis unit detects a rhythm pattern of the music piece by synchronously adding data indicating the onset positions of the second part on a measure-by-measure basis, and the display unit displays the GUI based on the rhythm pattern. [2] The music analysis device according to [1], wherein the rearrangement of the unit sounds includes moving the unit sounds to a position that is not a onset position of the second part, or deleting the unit sounds from the onset position of the second part. [3] The music analysis device described in [2], wherein the display unit displays the unit notes of the second part separately into first onset positions where the movement or deletion is not recommended and second onset positions where the movement or deletion is possible based on the rhythm pattern. [4] The music analysis device described in any one of [1] to [3], wherein the audio analysis unit weights and synchronously adds data indicating the onset positions of the second part for each section of the music. [5] The music analysis device described in any one of [1] to [4], wherein the audio analysis unit identifies a basic form of the rhythm pattern by comparing the result of the synchronous addition with a known pattern, and the display unit displays the GUI based on the basic form of the rhythm pattern. [6] The music analysis device described in [5], wherein the display unit displays the unit notes of the second part separately into first onset positions included in the basic form and second onset positions not included in the basic form. [7] The music analysis device according to [5] or [6], wherein the display unit displays variations associated with the basic form of the rhythm pattern on the GUI.[8] A method for analyzing a musical piece, comprising the steps of: extracting, from audio data of a musical piece including a first part and a second part that are phonetically separable, audio data of the first part, data of unit sounds of the second part, and data indicating the pronunciation positions of the second part; detecting a rhythm pattern of the musical piece by synchronously adding the data indicating the pronunciation positions of the second part on a measure-by-measure basis; displaying a GUI for rearranging the unit sounds of the second part based on the rhythm pattern; and generating mixed audio data in which the unit sounds rearranged by operation via the GUI are mixed into the audio data of the first part. [9] A program for causing a computer to execute the steps of: extracting audio data of a musical piece including a first part and a second part that are phonetically separable, data of the unit sounds of the second part, and data indicating the pronunciation positions of the second part from the audio data of the musical piece; detecting a rhythm pattern of the musical piece by synchronously adding the data indicating the pronunciation positions of the second part on a measure-by-measure basis; displaying a GUI for rearranging the unit sounds of the second part based on the rhythm pattern; and generating mixed audio data in which the unit sounds rearranged by operation via the GUI are mixed into the audio data of the first part.

[0006] FIG. 1 is a diagram showing the overall configuration of a system according to an embodiment of the present invention. FIG. 2 is a block diagram showing the functional configuration of a music analysis device in the example of FIG. 1. FIG. 3 is a flowchart showing an example of processing performed by a music analysis unit and a display unit in the present embodiment. FIG. 4 is a diagram for conceptually explaining synchronous addition of sound generation position data of kick sounds in the example of FIG. 3. FIG. 5 is a diagram showing a first example of display of kick sounds and rhythm patterns in the present embodiment. FIG. 6 is a diagram showing a second example of display of kick sounds and rhythm patterns in the present embodiment.

[0007] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0008] FIG. 1 is a diagram showing the overall configuration of a system according to an embodiment of the present invention. The system 10 according to this embodiment includes a PC (Personal Computer) 100, a DJ controller 200, and a speaker 300. The PC 100 is a device that stores, processes, and plays audio data. It may be a terminal device such as a tablet or smartphone, not limited to a PC. The PC 100 includes a display 101 that displays information to the user and an input device such as a touch panel or mouse that acquires user input. The DJ controller 200 is connected to the PC 100 via a communication means such as a USB (Universal Serial Bus) and acquires user input related to music playback using a channel fader, crossfader, performance pad, jog dial, and various knobs and buttons. The audio data is played back using, for example, a speaker 300.

[0009] In this embodiment, the PC 100 functions as a music analysis device in the system 10 described above. For example, the PC 100 performs processing on stored audio data in response to user input during playback of the audio data. Alternatively, the PC 100 may perform processing on the audio data prior to playback and store the processed audio data. In this case, the DJ controller 200 and speakers 300 do not need to be connected to the PC 100 when the processing is performed. In this embodiment, the PC 100 functions as a music analysis device. However, in other embodiments, DJ equipment such as a mixer or an all-in-one DJ system (a digital audio player with communication and mixing functions) may function as a music analysis device. Furthermore, a server connected to the PC or DJ equipment via a network may function as a music analysis device.

[0010] 2 is a block diagram showing the functional configuration of the music analysis device in the example of FIG. 1. The PC 100 functioning as the music analysis device includes an audio analysis unit 120, a display unit 140, a mix processing unit 150, and an operation unit 160. These functions are implemented by a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor) operating in accordance with a program. The program is read from the storage or removable recording medium of the PC 100, or downloaded from a server via a network, and then loaded into the memory of the PC 100.

[0011] The audio analysis unit 120 receives musical piece audio data 110 including a first part and a second part that are acoustically separable. In this embodiment, the first part is a vocal and / or instrumental sound part other than the kick sound, and the second part is a kick sound part. Here, the kick sound is a bass drum sound or a synthesized sound that imitates the sound of a bass drum. The audio analysis unit 120 extracts kick sound-removed audio data 131, kick unit sound data 132, and kick sound generation data 133 from the musical piece audio data 110 using, for example, a music separation engine. Here, the kick sound-removed audio data 131 is data of audio from which the kick sound has been removed from the musical piece audio data 110, i.e., audio data of the first part. The kick unit sound data 132 is data of the kick sound included in the musical piece audio data 110, i.e., the unit sound of the second part (hereinafter also referred to as the kick unit sound). The kick sound generation data 133 is data indicating the generation position of the kick sound in the musical piece audio data 110. The sound generation position is the time position at which a kick sound is generated in the music audio data 110, and is recorded, for example, as a time code in the music or a count in units of bars / beats. As will be described later, the audio analysis unit 120 detects the rhythm pattern of the music by synchronously adding the kick sound generation data 133 in units of bars.

[0012] A unit sound is a sound extracted by taking one pronunciation of the sound of the second part as a unit. For example, the audio analysis unit 120 extracts unit sounds by separating a kick sound part from the music audio data 110, further dividing the kick sound part into pronunciations, and classifying the pronunciations according to audio waveform characteristics. Multiple unit sounds with different audio waveform characteristics may be extracted. The kick unit sound data 132 may be, for example, audio data sampled from the kick sound part, or may be temporal position information at which the unit sound is reproduced in the kick sound part, or may be audio data of a sample sound similar to the extracted sound, or an identifier of the sample sound.

[0013] The display unit 140 displays information based on the kick unit sound data 132 or the kick sound generation data 133, for example, on the display 101 of the PC 100. Meanwhile, the operation unit 160 acquires user input via an input device, such as a touch panel or a mouse, of the PC 100. Specifically, for example, the display unit 140 displays a GUI (Graphical User Interface) including a music audio waveform (which may be a waveform based on the music audio data 110 or a waveform based on the kick sound-removed audio data 131) and the kick sound generation positions associated with the waveform, based on the rhythm pattern detected by the audio analysis unit 120, and the operation unit 160 acquires a user's operation via the GUI to change the kick sound generation positions to any position within the music. Alternatively, the display unit 140 may display the arrangement of kick sounds based on a preset rhythm pattern on the GUI, and the operation unit 160 acquires a user's operation to select a rhythm pattern.

[0014] The mix processing unit 150 generates mixed audio data 170 based on the kick-sound-removed audio data 131 and the kick unit sound data 132. The mixed audio data 170 is audio data in which the rearranged kick unit sounds are mixed with the kick-sound-removed audio data 131. The sound generation positions of the kick unit sounds in the mixed audio data 170 are determined in accordance with the user operation acquired by the operation unit 160, as described above. Here, the sound generation positions of the kick unit sounds in the mixed audio data 170 may include positions different from the sound generation positions of the kick sounds in the original music audio data 110.

[0015] In the above configuration, the user can change the rhythm pattern of a song by changing the position of the kick sound to any position within the song or by selecting a rhythm pattern. However, without musical knowledge, it is difficult to change the rhythm pattern without destroying the atmosphere or groove of the song, or to skillfully create a new atmosphere or groove by changing the rhythm pattern.

[0016] Therefore, in this embodiment, the audio analysis unit 120 performs additional analysis of the music, and the display unit 140 displays the results, for example, on the display 101 of the PC 100, thereby helping the user to effectively change the rhythm pattern from the above-mentioned perspective.

[0017] FIG. 3 is a flowchart showing an example of processing performed by the music analysis unit and display unit in this embodiment. Kick sound position detection (step S121) is a process in which kick sound-removed audio data 131, kick unit sound data 132, and kick sound generation data 133 are extracted from music audio data 110 using the music separation engine described above in the processing of the audio analysis unit 120. The kick sound positions are detected, for example, in units of sixteenth notes. The audio analysis unit 120 synchronously adds kick sound generation position data, i.e., kick sound generation data 133 in this embodiment, for the entire music bar by bar (step S122). For example, the kick sound generation position data may be data in which the kick sound generation position is set to 1 and other positions are set to 0, as in the example described below. The synchronous addition may be weighted by music section, such as the verse section and the chorus section. The audio analysis unit 120 detects the rhythmic pattern of the music from the results of the synchronous addition (step S123). While the rhythmic pattern is not necessarily the same throughout the entire music, the detected rhythmic pattern is a prominent rhythmic pattern within the music. Furthermore, the audio analysis unit 120 identifies the basic form of the rhythm pattern by comparing the rhythm pattern resulting from the synchronous addition with a known pattern (step S124). Because rhythm patterns have typical basic forms, the basic form of the rhythm pattern can be identified by comparing the detected rhythm pattern with a pre-stored known pattern. The display unit 140 displays a GUI on the display 101 for rearranging the generation position of the kick sound based on the basic form of the rhythm pattern identified by the audio analysis unit 120 (step S141).

[0018] FIG. 4 is a diagram conceptually illustrating the synchronous addition of kick sound generation position data in the example of FIG. 3 . In the illustrated example, the kick sound generation position data is data in which the kick sound generation position is 1 and other positions are 0, with each digit corresponding to a sixteenth note. Therefore, by dividing this data into 16-digit units starting from the beginning of the song and adding each digit together, the kick sound generation position data for the entire song is synchronously added measure by measure. In the illustrated example, the generation position data for measures Bar1 to Bar4, each of which has 16 digits (each digit is numbered 0 to 15), is synchronously added. Since measures Bar1 to Bar4 all have 1 at the 0, 4, 8, and 12 digits (the beginning of a quarter beat) that indicate the kick sound generation position, the 0, 4, 8, and 12 digits (the beginning of a quarter beat) of the synchronous addition result (SUM) are 4. On the other hand, since the 2nd, 6th, and 10th digits only have kick sound production positions in measures Bar1, Bar3, and Bar4, respectively, the 2nd, 6th, and 10th digits in the result of synchronized addition (SUM) are 1. Since there are no kick sound production positions in any of the other digits in the result of synchronized addition (SUM), all other digits are 0. In the above example, the result of synchronized addition indicates that the kick sound production positions common to many measures are the 0th, 4th, 8th, and 12th digits (the start of quarter beats), that the kick sound production positions in the 2nd, 6th, and 10th digits are not common to many measures, and that there are no kick sound production positions in the other digits.

[0019] In the illustrated example, the result of the synchronous addition (SUM) described above is compared with known patterns, and a quarter note rhythm pattern PTN1 is identified as the basic rhythm pattern. Thus, by synchronously adding kick sound generation position data for the entire song on a measure-by-measure basis, it is possible to identify the basic rhythm pattern that matches the kick sound generation positions common to each measure. If the kick sound generation position data is weighted and synchronously added for each section of the song, the kick sound generation positions in specific sections, such as chorus sections, are more likely to be reflected in the identified rhythm pattern.

[0020] 5 is a diagram showing a first example of the display of kick sounds and rhythm patterns in this embodiment. In the illustrated example, the quarter rhythm pattern PTN1 is identified as the basic form, as in the example of FIG. 4 above, and modified rhythm patterns PTN11 and PTN12 are displayed as candidates for the modified rhythm pattern. In the rhythm patterns PTN11 and PTN12, the onset positions of the kick sounds are displayed as two positions: onset position Pos1, which is included in the basic form of the rhythm pattern and therefore is not recommended for repositioning or deletion, and onset position Pos2, which is not included in the basic form of the rhythm pattern and therefore can be repositioned or deleted.

[0021] The kick sound displayed at the sound production position Pos1 may be controlled so that its position cannot be changed or it cannot be deleted by an operation via the operation unit 160. On the other hand, the kick sound displayed at the sound production position Pos2 may be controlled so that its position can be changed or it can be deleted by an operation via the operation unit 160. In another example, the kick sound displayed at the sound production position Pos1 may also be controlled so that its position can be changed or it can be deleted after a warning message or the like is displayed.

[0022] The modified rhythm patterns PTN11, PTN12 may be stored in advance, for example, in association with the basic rhythm pattern PTN1, or may be generated, for example, by adding the sounding positions of kick sounds other than the basic rhythm pattern in the music piece to the sounding positions of the basic rhythm pattern.

[0023] 6 is a diagram showing a second example of the display of kick sounds and rhythm patterns in this embodiment. In the illustrated example, a rhythm pattern PTN2 different from the example above is identified as the basic form, and modified rhythm patterns PTN21, PTN22, PTN23, and PTN24 are displayed as candidates for the modified rhythm pattern. Similarly, in these examples, the sounding positions of kick sounds in the modified rhythm patterns are displayed as sounding positions Pos1, for which changing or deletion is not recommended, and sounding positions Pos2, for which changing or deletion is possible.

[0024] In one embodiment of the present invention described above, the GUI for rearranging kick unit sounds is displayed based on a rhythm pattern detected by synchronously adding the sounding position data of kick sounds. Specifically, based on the rhythm pattern, the GUI displays separately the sounding positions of kick sounds that are not recommended to be moved or deleted from the sounding positions of kick sounds that can be moved or deleted. This makes it possible to change the rhythm pattern without destroying the atmosphere or groove of the music, or to skillfully create a new atmosphere or groove by changing the rhythm pattern.

[0025] It should be noted that the embodiment of the present invention described above is merely illustrative and various modifications are possible. For example, in the above embodiment, the first part of a song is described as a part other than a kick sound, and the second part is described as a kick sound part. However, there are no limitations on how the vocal and / or instrument sounds are separated and assigned to the first and second parts. The second part may be any part from which unit sounds can be extracted, for example, a hi-hat or snare part, or a percussion sound part such as a drum sound that combines a kick sound with a hi-hat or snare. Because it is possible to extract multiple unit sounds with different audio waveform characteristics as described above, the second part may be a drum sound part, and the kick unit sound, as well as the hi-hat and snare unit sounds, may be rearranged.

[0026] 10...System, 100...PC, 101...Display, 110...Music audio data, 120...Audio analysis unit, 131...Kick sound removed audio data, 132...Kick unit sound data, 133...Kick sound generation data, 140...Display unit, 150...Mix processing unit, 160...Operation unit, 170...Mixed audio data, 200...DJ controller, 300...Speaker

Claims

1. an audio analysis unit that extracts audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part from audio data of a piece of music including a first part and a second part that are phonetically separable; a display unit that displays a GUI for rearranging the unit notes of the second part; a mix processing unit that generates mixed audio data in which the unit sounds rearranged by an operation via the GUI are mixed with the audio data of the first part; Equipped with the audio analysis unit detects a rhythm pattern of the music piece by synchronously adding data indicating the sounding positions of the second part in units of measures; The display unit is a music analysis device that displays the GUI based on the rhythm pattern.

2. The music analysis device according to claim 1 , wherein the rearrangement of the unit sounds includes moving the unit sounds to a position that is not a sounding position of the second part, or deleting the unit sounds from a sounding position of the second part.

3. The music analysis device according to claim 2, wherein the display unit displays the unit notes of the second part based on the rhythm pattern, dividing them into first pronunciation positions where the movement or deletion is not recommended and second pronunciation positions where the movement or deletion is possible.

4. The music analysis device according to claim 1 , wherein the audio analysis unit weights and synchronously adds the data indicating the onset positions of the second part for each section of the music.

5. the audio analysis unit identifies the basic form of the rhythm pattern by comparing the result of the synchronous addition with a known pattern; The music analysis device according to claim 1 , wherein the display unit displays the GUI based on a basic form of the rhythm pattern.

6. The music analysis device according to claim 5 , wherein the display unit displays the unit notes of the second part separately into a first sounding position included in the basic form and a second sounding position not included in the basic form.

7. The music analysis device according to claim 5 , wherein the display unit displays, on the GUI, variations associated with the basic form of the rhythm pattern.

8. extracting, from audio data of a musical piece including a first part and a second part that are phonetically separable, audio data of the first part, data of unit sounds of the second part, and data indicating sounding positions of the second part; detecting a rhythm pattern of the music piece by synchronously adding data indicating the tone generation positions of the second part in units of measures; a step of displaying a GUI for rearranging the unit notes of the second part based on the rhythm pattern; generating mixed audio data in which the unit sounds rearranged by the operation via the GUI are mixed with the audio data of the first part; A music analysis method including:

9. extracting, from audio data of a musical piece including a first part and a second part that are phonetically separable, audio data of the first part, data of unit sounds of the second part, and data indicating sounding positions of the second part; detecting a rhythm pattern of the music piece by synchronously adding data indicating the tone generation positions of the second part in units of measures; a step of displaying a GUI for rearranging the unit notes of the second part based on the rhythm pattern; generating mixed audio data in which the unit sounds rearranged by the operation via the GUI are mixed with the audio data of the first part; A program that causes a computer to execute the following.