Music-piece analysis device, music-piece analysis method, and program

Abstract

There is provided a music piece analysis apparatus including: an audio analysis unit that extracts, from audio data of a music piece including first and second parts acoustically separable, audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part; a display unit that causes a GUI for rearranging the unit sounds of the second part to be displayed; and a mixing unit that generates mixed audio data where the unit sounds rearranged by an operation via the GUI are mixed with the audio data of the first part, in which the audio analysis unit detects a rhythm pattern of the music piece by performing synchronous addition on the data indicating the onset positions of the second part on a bar-by-bar basis, and the display unit causes the GUI to be displayed based on the rhythm pattern.

Description

TECHNICAL FIELD

[0001] The present invention relates to a music piece analysis apparatus, a music piece analysis method, and a program.BACKGROUND ART

[0002] A technique is known for acoustically separating a piece of music into a plurality of parts and providing a service utilizing audio data of the separated parts. For example, Patent Literature 1 describes a technique for arranging predetermined part data among a plurality of part data separated by a music separation unit to create arranged part data, and for transmitting the arranged part data and original part data obtained by excluding the predetermined part data from the plurality of part data.CITATION LISTPATENT LITERATURE(S)

[0003] Patent Literature 1: JP 2009-186729 ASUMMARY OF THE INVENTIONPROBLEM(S) TO BE SOLVED BY THE INVENTION

[0004] As described above, a technique is known for acoustically separating a music piece into a plurality of parts and for processing only sounds of a specific part and / or cutting sounds of a specific part. The invention is not limited to such examples, and an object of the invention is to provide a music piece analysis apparatus, a music piece analysis method, and a program that provide a new form of expression utilizing the separation of a music piece into parts.MEANS FOR SOLVING THE PROBLEM(S)

[0005] [1] A music piece analysis apparatus including: an audio analysis unit configured to extract, from audio data of a music piece including a first part and a second part that are acoustically separable, audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part; a display unit that causes a GUI for rearranging the unit sounds of the second part to be displayed; and a mixing unit that generates mixed audio data where the unit sounds rearranged by an operation via the GUI are mixed with the audio data of the first part, in which the audio analysis unit detects a rhythm pattern of the music piece by performing synchronous addition on the data indicating the onset positions of the second part on a bar-by-bar basis, and the display unit causes the GUI to be displayed based on the rhythm pattern. [2] The music piece analysis apparatus according to [1], in which the rearranging of the unit sounds includes moving the unit sounds to positions other than the onset positions of the second part, or deleting the unit sounds from the onset positions of the second part. [3] The music piece analysis apparatus according to [2], in which the display unit is configured to cause, based on the rhythm pattern, the unit sounds of the second part to be displayed separately at first onset positions where the moving or the deleting of the unit sounds is not recommended and at second onset positions where the moving or the deleting of the unit sounds is possible. [4] The music piece analysis apparatus according to any one of [1] to [3], in which the audio analysis unit is configured to apply weighting to the data indicating the onset positions of the second part for each of sections of the music piece, and to perform synchronous addition on the weighted data. [5] The music piece analysis apparatus according to any one of [1] to [4], in which the audio analysis unit is configured to identify a basic form of the rhythm pattern by comparing a result of the synchronous addition with a known pattern, and the display unit is configured to cause the GUI to be displayed based on the basic form of the rhythm pattern. [6] The music piece analysis apparatus according to [5], in which the display unit is configured to cause the unit sounds of the second part to be displayed separately at first onset positions included in the basic form and at second onset positions not included in the basic form. [7] The music piece analysis apparatus according to [5] or [6], in which the display unit is configured to cause the GUI to be displayed based on a variant rhythm pattern associated with the basic form of the rhythm pattern. [8] A music piece analysis method including: extracting, from audio data of a music piece including a first part and a second part that are acoustically separable, audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part; detecting a rhythm pattern of the music piece by performing synchronous addition on the data indicating the onset positions of the second part on a bar-by-bar basis; causing a GUI for rearranging the unit sounds of the second part to be displayed based on the rhythm pattern; and generating mixed audio data in which the unit sounds rearranged by an operation via the GUI are mixed with the audio data of the first part. [9] A program causing a computer to execute: extracting, from audio data of a music piece including a first part and a second part that are acoustically separable, audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part; detecting a rhythm pattern of the music piece by performing synchronous addition on the data indicating the onset positions of the second part on a bar-by-bar basis; causing a GUI for rearranging the unit sounds of the second part to be displayed based on the rhythm pattern; and generating mixed audio data in which the unit sounds rearranged by an operation via the GUI are mixed with the audio data of the first part. BRIEF DESCRIPTION OF DRAWING(S)

[0006] Fig. 1 is a diagram illustrating an overall configuration of a system according to an exemplary embodiment of the invention. Fig. 2 is a block diagram illustrating a functional configuration of a music piece analysis apparatus in the example of Fig. 1. Fig. 3 is a flowchart illustrating an example of processing performed by the music piece analysis unit and a display unit in the exemplary embodiment. Fig. 4 is a conceptual diagram illustrating synchronous addition of kick sound onset position data in the example of Fig. 3. Fig. 5 is a diagram illustrating a first example of display of kick sounds and rhythm patterns in the exemplary embodiment. Fig. 6 is a diagram illustrating a second example of display of kick sounds and rhythm patterns in the exemplary embodiment. DESCRIPTION OF EMBODIMENT(S)

[0007] Preferred exemplary embodiment(s) of the invention will be described below in detail with reference to the accompanying drawings. It should be noted that components of the same or substantially the same function(s) and configuration(s) will be denoted by the same reference numerals in the specification and drawings, omitting repetition of description thereof.

[0008] Fig. 1 is a diagram illustrating an overall configuration of a system according to an exemplary embodiment of the invention. A system 10 according to the exemplary embodiment includes a personal computer (PC) 100, a DJ controller 200, and a speaker 300. The PC 100 is a device configured to store, process, and play back audio data. Such a device may alternatively be implemented as a terminal device such as a tablet or a smartphone instead of a PC. The PC 100 includes a display 101 configured to display information to a user thereon, and an input device, such as a touch panel and a mouse, configured to acquire an operation input from the user. The DJ controller 200 is connected to the PC 100 via a communication means such as a Universal Serial Bus (USB), and acquires a user operation input relating to playback of a music piece through a channel fader, a crossfader, a performance pad, a jog dial, and various knobs, buttons and the like. Audio data is played back using, for example, the speaker 300.

[0009] In the exemplary embodiment, in the system 10 described above, the PC 100 functions as a music piece analysis apparatus. For example, during playback of the audio data, the PC 100 executes processing corresponding to a user operation input on stored audio data. Alternatively, the PC 100 may execute processing on audio data before playback and store the processed audio data. In this case, at the time of executing the processing, the DJ controller 200 and the speaker 300 do not necessarily need to be connected to the PC 100. In the exemplary embodiment, the PC 100 functions as the music piece analysis apparatus. In another exemplary embodiment, DJ equipment such as a mixer or an all-in-one DJ system (a digital audio player having communication and mixing functions) may function as the music piece analysis apparatus. Furthermore, a server connected to the PC or the DJ equipment via a network may also function as the music piece analysis apparatus.

[0010] Fig. 2 is a block diagram illustrating a functional configuration of the music piece analysis apparatus in the example of Fig. 1. The PC 100 functioning as the music piece analysis apparatus includes an audio analysis unit 120, a display unit 140, a mixing unit 150, and an operation unit 160. These functions are implemented by a processor such as a Central Processing Unit (CPU) or Digital Signal Processor (DSP) operating in accordance with a program. The program is read from storage of the PC 100 or from a removable recording medium, or is downloaded from a server via a network and deployed in a memory of the PC 100.

[0011] Music piece audio data 110 including a first part and a second part that are acoustically separable is input to the audio analysis unit 120. In the exemplary embodiment, the first part is a part of vocals and / or instrument sounds other than kick sounds, and the second part is a part of the kick sounds. Here, the kick sounds are bass drum sounds or synthesized sounds that imitate the bass drum sounds. The audio analysis unit 120 extracts kick-sound-removed audio data 131, kick unit sound data 132, and kick sound onset data 133 from the music piece audio data 110 by using, for example, a music separation engine. The kick-sound-removed audio data 131 is audio data obtained by removing kick sounds from the music piece audio data 110, that is, audio data of the first part. The kick unit sound data 132 is data of unit sounds of kick sounds, that is, the second part (hereinafter also referred to as "kick unit sounds"), included in the music piece audio data 110. The kick sound onset data 133 is data indicating kick sound onset positions in the music piece audio data 110. Each onset position is a temporal position at which a single kick sound is produced in the music piece audio data 110, and is recorded, for example, as a time code or as a count in bars and beats within the music piece. As will be described later, the audio analysis unit 120 detects a rhythm pattern of the music piece by performing synchronous addition on the kick sound onset data 133 on a bar-by-bar basis.

[0012] The unit sound is a sound extracted by using a single sound onset of the second part as a unit. For example, the audio analysis unit 120 extracts the unit sounds by separating a kick sound part from the music piece audio data 110, further dividing the separated kick sound part for each sound onset, and classifying the obtained sounds according to features of audio waveforms. A plurality of unit sounds having different features of audio waveforms may be extracted. The kick unit sound data 132 may be, for example, audio data sampled from the kick sound part, information indicating temporal positions at which the unit sounds are played back in the kick sound part, audio data of sample sounds similar to the extracted sounds, or identifiers of the sample sounds.

[0013] The display unit 140 causes information based on the kick unit sound data 132 or the kick sound onset data 133 to be displayed, for example, on the display 101 of the PC 100. The operation unit 160 acquires a user operation input through an input device such as a touch panel or a mouse of the PC 100. Specifically, for example, based on the rhythm pattern detected by the audio analysis unit 120, the display unit 140 causes a Graphical User Interface (GUI) including audio waveforms of the music piece and kick sound onset positions corresponding to the waveforms to be displayed. The audio waveforms may be based on either the music piece audio data 110 or the kick-sound-removed audio data 131. The operation unit 160 acquires, via the GUI, a user operation for changing the onset positions of the kick sounds to any positions in the music piece. Alternatively, the display unit 140 may cause arrangements of kick sounds according to preset rhythm patterns to be displayed on the GUI, and the operation unit 160 may acquire a user operation to select one of the preset rhythm patterns.

[0014] The mixing unit 150 generates mixed audio data 170 based on the kick-sound-removed audio data 131 and the kick unit sound data 132. The mixed audio data 170 is audio data in which the kick unit sounds rearranged are mixed with the kick-sound-removed audio data 131. The onset positions of the kick unit sounds in the mixed audio data 170 are determined in accordance with the user operation acquired as described above by the operation unit 160. Here, the onset positions of the kick unit sounds in the mixed audio data 170 may include positions different from the onset positions of the kick sounds in the original music piece audio data 110.

[0015] In the configuration described above, the user changes the rhythm pattern of the music piece by changing kick sound onset positions to any positions in the music piece or by selecting a rhythm pattern. However, without musical knowledge, it is difficult to change the rhythm pattern while maintaining the atmosphere or groove of the music piece, or to successfully create a new atmosphere or groove by changing the rhythm pattern.

[0016] Accordingly, in the exemplary embodiment, the audio analysis unit 120 additionally performs analysis of the music piece, and the display unit 140 causes a result of the analysis to be displayed, for example, on the display 101 of the PC 100, thereby assisting the user in successfully changing the rhythm pattern from the above-described viewpoints.

[0017] Fig. 3 is a flowchart illustrating an example of processing performed by the music piece analysis unit and the display unit in the exemplary embodiment. The kick sound position detection (step S121) is a processing of extracting the kick-sound-removed audio data 131, the kick unit sound data 132, and the kick sound onset data 133 from the music piece audio data 110 by using the music separation engine described above in the processing of the audio analysis unit 120. The positions of the kick sounds are detected, for example, on a sixteenth-note basis. The audio analysis unit 120 performs synchronous addition on onset position data of the kick sounds, that is, the kick sound onset data 133 in the exemplary embodiment, on a bar-by-bar basis for the entire music piece (step S122). The kick sound onset data may be data in which the onset positions of the kick sounds are represented by "1" and the other positions are represented by "0," as in the example described later. Weighting may be applied to each section, such as a verse section and a chorus section, of the music piece for the synchronous addition. The audio analysis unit 120 detects a rhythm pattern of the music piece from the result of the synchronous addition (step S123). Although the rhythm pattern is not necessarily identical throughout the entire music piece, the rhythm pattern detected here is a predominant rhythm pattern within the music piece. Furthermore, the audio analysis unit 120 identifies a basic form of the rhythm pattern by comparing the rhythm pattern, which is the result of the synchronous addition, with known patterns (step S124). Since rhythm patterns have typical basic forms, the audio analysis unit 120 can identify a basic form of the rhythm pattern by comparing the detected rhythm pattern with the known patterns stored in advance. The display unit 140 causes, based on the basic form of the rhythm pattern identified by the audio analysis unit 120, the GUI for rearranging the kick sound onset positions to be displayed on the display 101 (step S141).

[0018] Fig. 4 is a conceptual diagram illustrating synchronous addition of the kick sound onset position data in the example of Fig. 3. In the illustrated example, the kick sound onset position data is data in which kick sound onset positions are represented by "1" and other positions are represented by "0," and one digit corresponds to a sixteenth note. Accordingly, if this data is divided every sixteen digits from the beginning of the music piece and values of the corresponding digits are summed, it means that synchronous addition is performed on the kick sound onset position data on a bar-by-bar basis for the entire music piece. In the illustrated example, synchronous addition is performed on onset position data of bars Bar1 to Bar4, each consisting of sixteen digits (each digit being numbered from 0 to 15). In each of the bars Bar1 to Bar4, since a value of "1" indicating kick sound onset positions is given at the digits 0, 4, 8, and 12 (corresponding to the downbeats of quarter notes), the result (SUM) of the synchronous addition shows a value of "4" at each of the digits 0, 4, 8, and 12 (the downbeats of quarter notes). On the other hand, since the kick sound onset positions are present only at the digits 2, 6, and 10 in bars Bar1, Bar3, and Bar4, respectively, the result (SUM) of the synchronous addition shows a value of "1" at each of the digits 2, 6, and 10. Since no kick sound onset position is present at the other digits in any of the bars, the result (SUM) of the synchronous addition shows a value of "0" at all the other digits. In the above example, the result of the synchronous addition indicates that the kick sound onset positions common to many bars are located at the digits 0, 4, 8, and 12 (corresponding to the downbeats of quarter notes), that the kick sound onset positions at the digits 2, 6, and 10 are not common to many bars, and that no kick sound onset position is present at the other digits.

[0019] In the illustrated example, the result (SUM) of the synchronous addition is compared with known patterns, and as a result, a four-on-the-floor rhythm pattern PTN1 is identified as a basic form of the rhythm pattern. As described above, by performing the synchronous addition on the kick sound onset position data on a bar-by-bar basis for the entire music piece, it is possible to identify a basic form of the rhythm pattern that matches the kick sound onset positions common to the respective bars. By applying weighting to the kick sound onset position data for each section of the music piece and performing synchronous addition on the weighted data, kick sound onset positions in a specific section, such as a chorus section, are more likely to be reflected in the identified rhythm pattern.

[0020] Fig. 5 is a diagram illustrating a first example of display of kick sounds and rhythm patterns according to the exemplary embodiment. In the illustrated example, the four-on-the-floor rhythm pattern PTN1 is identified as a basic form, as in the example of Fig. 4, and variant rhythm patterns PTN11 and PTN12 are displayed as candidates for a modified rhythm pattern. In the rhythm patterns PTN11 and PTN12, the kick sound onset positions are displayed separately as onset positions Pos1, which are included in the basic form of the rhythm pattern and for which changing or deleting is not recommended, and onset positions Pos2, which are not included in the basic form of the rhythm pattern and for which changing or deleting is possible.

[0021] For the kick sounds displayed at the onset positions Pos1, control may be performed such that the positions cannot be changed or deleted through operations via the operation unit 160. On the other hand, for kick sounds displayed at the onset positions Pos2, the positions can be changed or deleted through operations via the operation unit 160. In another example, for kick sounds displayed at the onset positions Pos1, the positions may be changed or deleted after, for example, a warning message is displayed.

[0022] The variant rhythm patterns PTN11 and PTN12 may be stored in advance in association with, for example, the basic form of the rhythm pattern PTN1, or may be generated, for example, by adding a kick sound onset position other than those of the basic form of the rhythm pattern within the music piece to the onset positions of the basic form of the rhythm pattern.

[0023] Fig. 6 is a diagram illustrating a second example of display of kick sounds and rhythm patterns in the exemplary embodiment. In the illustrated example, a rhythm pattern PTN2, which is different from that in the above example, is identified as a basic form of the rhythm pattern, and variant rhythm patterns PTN21, PTN22, PTN23, and PTN24 are displayed as candidates for a modified rhythm pattern. Also in the second example, the kick sound onset positions in each of the variant rhythm patterns are displayed separately as the onset positions Pos1 for which changing or deleting is not recommended and as the onset positions Pos2 for which changing or deleting is possible.

[0024] In an exemplary embodiment of the invention described above, the GUI for rearranging kick unit sounds is displayed based on the rhythm pattern detected by performing the synchronous addition on the kick sound onset position data. Specifically, based on the rhythm pattern, the kick sound onset positions for which moving or deleting is not recommended and the onset positions for which moving or deleting is possible are displayed separately. This makes it possible to change the rhythm pattern while maintaining the atmosphere or groove of the music piece, or to successfully create a new atmosphere or groove by changing the rhythm pattern.

[0025] The above-described exemplary embodiment of the invention is merely exemplary and various modifications are possible. For example, it is described in the above-described exemplary embodiment that the first part of the music piece is a part other than the kick sounds, and the second part is a part of the kick sounds. However, how the vocal and / or instrumental sounds are separated and assigned to the first part and the second part is not limited to the above exemplary embodiment. The second part only needs to be any part from which unit sounds can be extracted. The second part may, for example, be a percussion part, such as a hi-hat or snare drum part, or a drum sound part including kick sounds together with hi-hat and snare drum sounds. As described above, since it is possible to extract a plurality of unit sounds having different features of audio waveforms, the second part may be a drum sound part in which the kick unit sounds, as well as unit sounds of the hi-hat and the snare drum, are each rearranged.EXPLANATION OF CODES

[0026] 10...system, 100...PC, 101...display, 110...music piece audio data, 120...audio analysis unit, 131...kick-sound-removed audio data, 132...kick unit sound data, 133...kick sound onset data, 140...display unit, 150...mixing unit, 160...operation unit, 170...mixed audio data, 200...DJ controller, 300...speaker

Claims

1. A music piece analysis apparatus comprising: an audio analysis unit configured to extract, from audio data of a music piece including a first part and a second part that are acoustically separable, audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part; a display unit configured to cause a GUI for rearranging the unit sounds of the second part to be displayed; and a mixing unit configured to generate mixed audio data in which the unit sounds rearranged by an operation via the GUI are mixed with the audio data of the first part, wherein the audio analysis unit is configured to detect a rhythm pattern of the music piece by performing synchronous addition on the data indicating the onset positions of the second part on a bar-by-bar basis, and the display unit is configured to cause the GUI to be displayed based on the rhythm pattern.

2. The music piece analysis apparatus according to claim 1, wherein the rearranging of the unit sounds includes moving the unit sounds to positions other than the onset positions of the second part, or deleting the unit sounds from the onset positions of the second part.

3. The music piece analysis apparatus according to claim 2, wherein the display unit is configured to cause, based on the rhythm pattern, the unit sounds of the second part to be displayed separately at first onset positions where the moving or the deleting of the unit sounds is not recommended and at second onset positions where the moving or the deleting of the unit sounds is possible.

4. The music piece analysis apparatus according to any one of claims 1 to 3, wherein the audio analysis unit is configured to apply weighting to the data indicating the onset positions of the second part for each of sections of the music piece, and to perform synchronous addition on the weighted data.

5. The music piece analysis apparatus according to any one of claims 1 to 4, wherein the audio analysis unit is configured to identify a basic form of the rhythm pattern by comparing a result of the synchronous addition with a known pattern, and the display unit is configured to cause the GUI to be displayed based on the basic form of the rhythm pattern.

6. The music piece analysis apparatus according to claim 5, wherein the display unit is configured to cause the unit sounds of the second part to be displayed separately at first onset positions included in the basic form and at second onset positions not included in the basic form.

7. The music piece analysis apparatus according to claim 5 or 6, wherein the display unit is configured to cause the GUI to be displayed based on a variant rhythm pattern associated with the basic form of the rhythm pattern.

8. A music piece analysis method comprising: extracting, from audio data of a music piece including a first part and a second part that are acoustically separable, audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part; detecting a rhythm pattern of the music piece by performing synchronous addition on the data indicating the onset positions of the second part on a bar-by-bar basis; causing a GUI for rearranging the unit sounds of the second part to be displayed based on the rhythm pattern; and generating mixed audio data in which the unit sounds rearranged by an operation via the GUI are mixed with the audio data of the first part.

9. A program causing a computer to execute: extracting, from audio data of a music piece including a first part and a second part that are acoustically separable, audio data of the first part, data of unit sounds of the second part, and data indicating onset positions of the second part; detecting a rhythm pattern of the music piece by performing synchronous addition on the data indicating the onset positions of the second part on a bar-by-bar basis; causing a GUI for rearranging the unit sounds of the second part to be displayed based on the rhythm pattern; and generating mixed audio data in which the unit sounds rearranged by an operation via the GUI are mixed with the audio data of the first part.

Images