Audio data processing device, audio data processing method, and program

Abstract

Provided is a data processing device comprising: a first audio analysis unit for extracting, from audio data of a musical piece including a first part and a second part that are acoustically separable, the audio data of the first part; a second audio analysis unit for generating data of unit sounds of the second part from the audio data of the musical piece; a third audio analysis unit for generating data indicating sound production positions of the second part from the audio data of the musical piece; a master tempo processing unit for executing master tempo processing of audio data including at least the first part; and a mix processing unit for generating audio data obtained by mixing audio data of the second part, which has been configured by relocating sounds of the second part based on the unit sounds of the second part in accordance with the data indicating the sound production positions of the second part, into the master tempo-processed audio data of the first part.

Description

Audio data processing device, audio data processing method and program

[0001] The present invention relates to an audio data processing device, an audio data processing method, and a program.

[0002] Master tempo processing that changes only the tempo of a song without changing the key is already known in DJ equipment, etc. For example, Patent Document 1 describes a digital player equipped with a master tempo adjustment slider that adjusts the playback speed of a track.

[0003] International Publication No. 2017 / 119115

[0004] However, when the tempo of a song is significantly changed using master tempo processing, for example, the tempo can be changed without causing a noticeable change in the timbre of vocal sounds or pitched instruments, but for percussion instruments, for example, the tempo change can cause a noticeable change in timbre. This phenomenon occurs due to the difference between sounds that are generated as continuous waveforms and sounds that are generated as waveforms that exhibit characteristic time-series changes, such as the attack and drum rumble of a drum. In the latter case, if the waveform length is changed by master tempo processing, the change in timbre is more likely to be noticeable.

[0005] Therefore, an object of the present invention is to provide an audio data processing device, an audio data processing method, and a program that can make it difficult to perceive changes in tone before and after master tempo processing, even for sounds such as percussion instrument sounds.

[0006] [1] An audio data processing device comprising: a first audio analysis unit that extracts audio data of a first part from audio data of a musical piece including a first part and a second part that are phonetically separable, a second audio analysis unit that generates data of unit sounds of the second part from the audio data of the musical piece, a third audio analysis unit that generates data indicating onset positions of the second part from the audio data of the musical piece, a master tempo processing unit that performs master tempo processing on audio data including at least the first part, and a mix processing unit that generates audio data obtained by mixing the master tempo-processed audio data with audio data of the second part configured by rearranging unit sounds of the second part in accordance with the data indicating the onset positions of the second part. [2] The audio data processing device according to [1], wherein the master tempo processing unit performs master tempo processing on the audio data of the first part. [3] The audio data processing device according to [1], wherein the master tempo processing unit performs master tempo processing on the audio data of the musical piece, the first audio analysis unit extracts audio data of the first part from the master tempo-processed audio data of the musical piece, and the second audio analysis unit generates unit sound data of the second part from audio data of the musical piece that has not been master tempo-processed. [4] The audio data processing device according to any one of [1] to [3], wherein the second part is made up of percussion sounds, and the first part is made up of sounds other than the percussion sounds. [5] The audio data processing device according to [4], wherein the percussion sounds include a kick sound. [6] A data processing method comprising the steps of: extracting audio data of a first part from audio data of a musical piece including a first part and a second part that are phonetically separable; generating data of unit sounds of the second part from the audio data of the musical piece; generating data indicating the onset positions of the second part from the audio data of the musical piece; master tempo processing the audio data including at least the first part; and generating audio data by mixing the master tempo processed audio data with audio data of the second part configured by rearranging the unit sounds of the second part in accordance with the data indicating the onset positions of the second part.[7] A program that causes a computer to realize the following functions: extracting audio data of a first part from audio data of a musical piece including a first part and a second part that are phonetically separable; generating data of unit sounds of the second part from the audio data of the musical piece; generating data indicating the onset positions of the second part from the audio data of the musical piece; performing master tempo processing on audio data including at least the first part; and generating audio data that mixes the master tempo-processed audio data with the audio data of the second part, the audio data being configured by rearranging the unit sounds of the second part in accordance with the data indicating the onset positions of the second part.

[0007] In the above configuration, when changing the tempo of the audio data of a song, the audio data of the first part that has been subjected to master tempo processing is mixed with audio data of the second part that has been constructed by rearranging the unit sounds of the second part in accordance with data indicating the sounding positions of the second part. This makes it possible to make the change in timbre less noticeable before and after master tempo processing, even for sounds such as percussion instruments.

[0008] FIG. 5 is a diagram showing the overall configuration of a system according to a first embodiment of the present invention. FIG. 6 is a block diagram showing the schematic functional configuration of an audio data processing device in the example of FIG. 1. FIG. 7 is a diagram conceptually showing master tempo processing in the example of FIG. 1 in comparison with normal master tempo processing. FIG. 8 is a flowchart showing the processing flow of the audio data processing device in the example of FIG. 1. FIG. 9 is a block diagram showing the schematic functional configuration of an audio data processing device according to a second embodiment of the present invention. FIG. 10 is a flowchart showing the processing flow of the audio data processing device in the example of FIG.

[0009] First Embodiment FIG. 1 is a diagram showing the overall configuration of a system according to a first embodiment of the present invention. The system 10 according to this embodiment includes a PC (Personal Computer) 100, a DJ controller 200, and a speaker 300. The PC 100 is a device that stores, processes, and plays audio data. It may be a terminal device such as a tablet or smartphone, and is not limited to a PC. The PC 100 includes a display 101 that displays information to the user and an input device such as a touch panel or mouse that receives user input. The DJ controller 200 is connected to the PC 100 via a communication means such as a USB (Universal Serial Bus) and receives user input related to music playback using a channel fader, crossfader, performance pad, jog dial, and various knobs and buttons. The audio data is played using, for example, a speaker 300.

[0010] In this embodiment, the PC 100 functions as an audio data processing device in the system 10 described above. For example, the PC 100 performs processing of stored audio data in response to user input during playback of the audio data. Alternatively, the PC 100 may perform processing on the audio data prior to playback and store the processed audio data. In this case, the DJ controller 200 and speakers 300 may not be connected to the PC 100 when the processing is performed. In this embodiment, the PC 100 functions as an audio data processing device. However, in other embodiments, DJ equipment such as a mixer or an all-in-one DJ system (a digital audio player with communication and mixing functions) may function as an audio data processing device. Furthermore, a server connected to the PC or DJ equipment via a network may function as an audio data processing device.

[0011] 2 is a block diagram showing a schematic functional configuration of the audio data processing device in the example of FIG. 1. The PC 100 functioning as the audio data processing device includes audio analysis units 121, 122, and 123, a master tempo processing unit 140, and a mix processing unit 150. These functions are implemented by a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor) operating in accordance with a program. The program is read from the storage or removable recording medium of the PC 100, or downloaded from a server via a network, and then loaded into the memory of the PC 100.

[0012] Musical piece audio data 110 including a first part and a second part that can be acoustically separated is input to the audio analysis units 121, 122, and 123. In this embodiment, the first part is a vocal and / or instrumental sound part other than the kick sound, and the second part is a kick sound part. Here, the kick sound is a bass drum sound or a synthesized sound that imitates the sound of a bass drum. The audio analysis unit 121 extracts kick-sound-removed audio data 131 from the music audio data 110 using, for example, a music separation engine. The audio analysis units 122 and 123 respectively generate kick unit sound data 132 and kick sound generation data 133 from the music audio data 110. Here, the kick-sound-removed audio data 131 is audio data from which the kick sound has been removed from the music audio data 110, i.e., the audio data of the first part. The Kick unit sound data 132 is data on the Kick sound, i.e., the unit sound of the second part (hereinafter also referred to as the Kick unit sound), included in the music piece audio data 110. The Kick sound generation data 133 is data indicating the generation position and velocity of the Kick sound in the music piece audio data 110.

[0013] A unit sound is a sound extracted by taking one pronunciation of the sound of the second part as a unit. For example, the audio analysis unit 122 extracts unit sounds by separating the Kick sound part from the music audio data 110, further dividing the Kick sound part into pronunciations, and classifying the pronunciations according to audio waveform characteristics. Multiple unit sounds with different audio waveform characteristics may be extracted. The Kick unit sound data 132 may be, for example, audio data sampled from the Kick sound part, or may be temporal position information at which the unit sound is reproduced in the Kick sound part, or may be audio data of a sample sound similar to the extracted sound, or an identifier of the sample sound.

[0014] The onset position is the time position at which a kick sound is produced in the music audio data 110, and is recorded, for example, as a time code in the music or a count in bar / beat units. Velocity is a parameter that indicates the volume and duration of a sound. For example, in MIDI (registered trademark), velocity is used as a numerical value that indicates the strength of a sound, or more specifically, the speed at which a key is struck when a sound is produced by a keystroke. The higher the velocity, the higher the volume and the longer the duration of the sound. In this embodiment, the audio analysis unit 123 generates kick onset data 133 that records the onset position and velocity for each kick sound separated from the music audio data 110.

[0015] The master tempo processing unit 140 performs master tempo processing on the kick-sound-free audio data 131 extracted by the audio analysis unit 121. Here, the master tempo processing is processing that changes only the tempo without changing the key of the music. The master tempo processing unit 140 may make the tempo of the kick-sound-free audio data 131 faster or slower than the tempo of the original music audio data 110. In this embodiment, since the kick-sound-free audio data 131 on which the master tempo processing is performed does not contain a kick sound, the length of the waveform of the kick sound does not change in the processing by the master tempo processing unit 140.

[0016] The mix processing unit 150 mixes the master tempo-processed Kick sound-removed audio data 131 with Kick sound audio data formed by rearranging Kick unit sounds based on Kick unit sound data 132 in accordance with Kick sound generation data 133, to generate tempo-changed music audio data 160. More specifically, the mix processing unit 150 changes the sound generation positions of the Kick sounds indicated by the Kick sound generation data 133 in accordance with the tempo change rate obtained by the master tempo processing, and sets the velocities that were set for the Kick sounds at the respective sound generation positions in the original music audio data 110 to the rearranged Kick unit sounds. This makes it possible to mix Kick sounds into the tempo-changed music audio data 160 at the same sound generation positions, timbre, and velocity as those of the original music audio data 110.

[0017] 3 is a diagram conceptually comparing the master tempo processing in the example of FIG. 1 with normal master tempo processing. In the illustrated example, master tempo processing is performed to change the music from BPM 120 to BPM 90 (slowing the tempo), resulting in a change in the length of one beat from B1 to B2 (>B1). In the normal master tempo processing shown in the upper part, the length of the kick sound waveform also changes from K1 to K2 (>K1), resulting in a perceptible change in the timbre of the kick sound in the audio data after master tempo processing. In contrast, in the master tempo processing of this embodiment shown in the lower part, even though the length of one beat changes from B1 to B2, the length of the kick sound waveform remains K1. In reality, the length of the waveform does not necessarily exactly match K1 because the kick unit sounds are rearranged, but because the waveform length does not change significantly, the change in the timbre of the kick sound is hardly perceptible.

[0018] Fig. 4 is a flowchart showing the processing flow of the audio data processing device in the example of Fig. 1. In this embodiment, the audio analyzers 121, 122, and 123 extract and generate kick sound-free audio data 131, kick unit sound data 132, and kick pronunciation data 133 from the music piece audio data 110 (steps S101 to S103; in no particular order), the kick sound-free audio data 131 is subjected to master tempo processing (step S104), and the master tempo-processed kick sound-free audio data 131 is mixed with kick sound audio data reconstructed based on the kick unit sound data 132 and the kick pronunciation data 133 (step S105), thereby generating tempo-modified music piece audio data 160.

[0019] In the first embodiment of the present invention described above, when changing the tempo of the original music piece audio data 110, the Kick sound-removed audio data 131 extracted by the audio analysis unit 121 is subjected to master tempo processing, and for the Kick sound, audio data of the Kick sound is mixed, which is configured by rearranging Kick unit sounds based on the Kick unit sound data 132 in accordance with the Kick sound generation data 133. This makes it possible to generate Kick sounds in the tempo-changed music piece audio data 160 at the same generation positions as in the original music piece audio data 110, without giving the impression of a change in tone.

[0020] 5 is a block diagram showing the schematic functional configuration of an audio data processing device according to a second embodiment of the present invention. Note that this embodiment is similar to the first embodiment except for the arrangement of the master tempo processing unit 140 and the processing order, which will be described below, and therefore a detailed description thereof will be omitted.

[0021] In this embodiment, the master tempo processing unit 140 performs master tempo processing on the music audio data 110, and the audio analysis unit 121 extracts kick-sound-free audio data 131 from the master tempo-processed music audio data 110. Here, the music audio data 110 is master tempo-processed while still containing the kick sound, so the length of the waveform of the kick sound has changed, as described above with reference to Fig. 3. By extracting the kick-sound-free audio data 131 from this music audio data 110, the kick sound whose waveform length has changed can be removed, and the kick-sound-free audio data 131 that has been master tempo-processed similarly to the first embodiment can be obtained.

[0022] Meanwhile, similar to the first embodiment, the audio analysis units 122 and 123 generate Kick unit sound data 132 and Kick sound generation data 133 from music piece audio data 110 that has not been subjected to master tempo processing. Similar to the first embodiment, the mix processing unit 150 mixes Kick sound audio data reconstructed based on these data with Kick sound-removed audio data 131 to generate tempo-altered music piece audio data 160.

[0023] Fig. 6 is a flowchart showing the processing flow of the audio data processing device in the example of Fig. 5. In this embodiment, the audio analysis units 122 and 123 generate Kick unit sound data 132 and Kick pronunciation data 133, respectively, from music piece audio data 110 before master tempo processing (steps S201 and S202; in no particular order), the music piece audio data 110 is subjected to master tempo processing (step S203), the audio analysis unit 121 extracts Kick sound-free audio data 131 from the master tempo-processed music piece audio data 110 (step S204), and the Kick sound-free audio data 131 is mixed with audio data of a Kick sound reconstructed based on the Kick unit sound data 132 and the Kick pronunciation data 133 (step S205), thereby generating tempo-altered music piece audio data 160.

[0024] In the second embodiment of the present invention described above, the music piece audio data 110 is subjected to master tempo processing, and then the Kick sound-removed audio data 131 is extracted. In this case, too, by mixing Kick sound audio data constructed by rearranging Kick unit sounds based on Kick unit sound data 132 in accordance with Kick sound generation data 133, it is possible to generate Kick sounds in the tempo-changed music piece audio data 160 at the same generation positions as in the original music piece audio data 110, without any perceived change in tone, as in the first embodiment.

[0025] Note that the above-described embodiments are merely illustrative and may be modified in various ways. For example, in the above embodiments, the first part of a song is a part other than a kick sound, and the second part is a kick sound part. However, there are no limitations on how the vocal and / or instrument sounds are separated and assigned to the first and second parts. The second part may be any part from which unit sounds can be extracted, such as a hi-hat or snare part, or a percussion sound part such as a drum sound formed by adding a hi-hat or snare to a kick sound. Since it is possible to extract multiple unit sounds with different audio waveform characteristics as described above, the second part may be a drum sound part, and the kick unit sound, as well as the hi-hat and snare unit sounds, may be rearranged.

[0026] 10...System, 100...PC, 101...Display, 110...Music audio data, 121...Audio analysis unit, 122...Audio analysis unit, 123...Audio analysis unit, 131...Kick sound removed audio data, 132...Kick unit sound data, 133...Kick pronunciation data, 140...Master tempo processing unit, 150...Mix processing unit, 160...Music audio data, 200...DJ controller, 300...Speaker

Claims

1. a first audio analysis unit that extracts audio data of a musical piece including a first part and a second part that are acoustically separable from audio data of the first part; a second voice analysis unit that generates data of unit sounds of the second part from the voice data of the music piece; a third voice analysis unit that generates data indicating a sounding position of the second part from the voice data of the music piece; a master tempo processing unit that performs master tempo processing on audio data including at least the first part; a mix processing unit that generates audio data by mixing the master tempo processed audio data with audio data of the second part that is configured by rearranging unit sounds of the second part in accordance with data indicating sounding positions of the second part; An audio data processing device comprising:

2. 2. The audio data processing device according to claim 1, wherein the master tempo processing unit performs master tempo processing on the audio data of the first part.

3. the master tempo processing unit performs master tempo processing on the audio data of the music piece; the first audio analysis unit extracts audio data of the first part from the audio data of the music piece that has been subjected to the master tempo process; The audio data processing device according to claim 1 , wherein the second audio analysis section generates data of unit sounds of the second part from audio data of the music piece that has not been subjected to master tempo processing.

4. the second part is composed of percussion sounds; 4. The audio data processing device according to claim 1, wherein the first part is composed of sounds other than the percussion instrument sounds.

5. The audio data processing device according to claim 4 , wherein the percussion sound includes a kick sound.

6. A step of extracting audio data of a musical piece including a first part and a second part which are acoustically separable from audio data of the first part; generating unit sound data of the second part from the voice data of the music piece; generating data indicating a sounding position of the second part from the voice data of the musical piece; a step of performing master tempo processing on audio data including at least the first part; generating audio data by mixing the master tempo processed audio data with audio data of the second part, the audio data being configured by rearranging unit sounds of the second part in accordance with data indicating sounding positions of the second part; A method for processing audio data, comprising:

7. A function of extracting audio data of a musical piece including a first part and a second part that are acoustically separable from audio data of the first part; a function of generating data of unit sounds of the second part from the audio data of the music piece; a function of generating data indicating a sounding position of the second part from the audio data of the musical piece; a function of performing master tempo processing on audio data including at least the first part; a function of generating audio data by mixing the master tempo processed audio data with audio data of the second part, the audio data being configured by rearranging unit sounds of the second part in accordance with data indicating sounding positions of the second part; A program that enables a computer to achieve the above.