Music processing method, music processing program, and music processing device

Abstract

This music processing method for processing music data SD having a plurality of channels comprises estimating the part of each channel for each time block TB of the music data SD, and connecting a channel estimated to be a first part (for example, melody) in each time block TB of the music data SD to generate first part data (for example, melody channel data) corresponding to the first part.

Description

Music processing method, music processing program, and music processing apparatus

[0001] The present invention relates to a music processing method, a music processing program that executes the music processing method, and a music processing apparatus.

[0002] There is music data composed of a plurality of channels. For example, in the case of music composed of a plurality of performance parts, the music data is generated so that each performance part corresponds to each channel. For example, in the case of music played by multiple instruments, the music data is generated so that a plurality of instrument parts correspond to a plurality of channels. As an example, in the case of pop music, a plurality of performance parts such as a guitar part, a bass part, a drum part, a keyboard part, and a vocal part correspond to a plurality of channels of the music data.

[0003] In Patent Document 1 below, it is determined which part of the melody part, the bass part, or the drum part each channel included in the music data (MIDI data) is. And in Document 1, based on the discrimination result of the part, the rhythm pattern for each block of the music is extracted, and the lighting pattern for each block of the music is determined based on the rhythm pattern.

[0004] Japanese Patent No. 3743079

[0005] As described above, the plurality of channels constituting the music data have respective roles, such as corresponding to a plurality of performance parts or a plurality of instruments. If useful information can be generated based on the roles of each channel constituting the music data, it is beneficial to the user.

[0006] An object of the present invention is to generate useful information for the user by estimating the role of each channel for music data having a plurality of channels.

[0007] A music processing method according to one aspect of the present invention is a method for processing music data having multiple channels, wherein the role of each channel is estimated for each time block of the music data, and first role data corresponding to the first role is generated by connecting the channels estimated to have a first role in each time block of the music data.

[0008] Another aspect of the present invention relates to a music processing program that processes music data having multiple channels, and causes a computer to perform the following processes: estimating the role of each channel for each time block of the music data; and generating first role data corresponding to the first role by connecting the channels estimated to have a first role in each time block of the music data.

[0009] Another aspect of the present invention relates to a music processing device for processing music data having multiple channels, comprising: an estimation unit that estimates the role of each channel for each time block of music data; and a generation unit that generates first role data corresponding to the first role by connecting the channels estimated to have a first role in each time block of music data.

[0010] According to the present invention, by estimating the role of each channel in music data having multiple channels, it is possible to generate information that is useful to the user.

[0011] This figure shows how a user utilizes the music processing method according to this embodiment. This is a block diagram of the music processing device according to this embodiment. This is a functional block diagram of the music processing device according to this embodiment. This is a flowchart of the music processing method. This figure shows the processing unit of music data by the music processing method. This figure shows an overview of the processing of the estimation unit. This figure shows the role of the channels estimated for each time block. This figure shows melody channel data generated by extracting only the main melody.

[0012] The following description will refer to the attached drawings and explain the music processing method, music processing program, and music processing apparatus according to embodiments of this invention.

[0013] {1. How to Use the Music Processing Device} Figure 1 shows a user 4 using the music processing method according to this embodiment. User 4 generates a melody channel score based on MIDI (Musical Instrument Digital Interface) (registered trademark) data or XF (registered trademark) data having multiple channels by operating the computer 1. The generated melody channel score is displayed on the computer 1's display 32. User 4 plays the electronic piano 5 while referring to the melody channel score. Note that XF is an extended format of the standard MIDI file.

[0014] {2. Configuration of the Music Processing Device} Figure 2 is a block diagram of a computer 1 equipped with a music processing device 200 according to this embodiment. As shown in Figure 2, the computer 1 includes a control unit 2, a user interface 3, a storage device 11, a sound source 12, a sound system 13, a network interface 14, and an external interface 15. The control unit 2 includes a CPU (Central Processing Unit) 21, RAM (Random Access Memory) 22, and ROM (Read Only Memory) 23. The control unit 2 functions as the music processing device 200 of this embodiment. The user interface 3 includes an operator 31 and a display 32. As the computer 1, for example, a personal computer 1 can be used.

[0015] The storage device 11 includes a storage medium such as a hard disk, optical disk, magnetic disk, or memory card. The storage device 11 stores computer programs such as a program PG, music data SD, melody channel data MCD, and a control program. The program PG is a program for executing music processing according to the present invention.

[0016] The CPU 21, RAM 22, and ROM 23 are connected via the bus 20. The RAM 22 consists of, for example, volatile memory and is used as a work area for the CPU 21, as well as temporarily storing various data. The ROM 23 consists of, for example, non-volatile memory and stores various programs, setting data, etc. The CPU 21 uses the RAM 22 as a work area and executes programs stored in the storage device 11 or ROM 23.

[0017] The sound source 12 is connected to the bus 20 and outputs an acoustic signal (audio data). The acoustic signal is sampled data (for example, PCM (pulse code modulation) data) that represents the waveform of sound. The sound source 12 has pre-stored acoustic signals for all pitches. The sound system 13 includes a digital-to-analog (D / A) conversion circuit, an amplifier, and a speaker. The sound system 13 converts the acoustic signal provided by the sound source 12 into an analog acoustic signal and generates sound based on the analog acoustic signal.

[0018] The control unit 31 includes a keyboard, mouse, etc., and is connected to the bus 20. The display 32 includes, for example, a liquid crystal display and is connected to the bus 20. The display 32 may be a touch panel display. In this case, some or all of the controls 31 may be displayed on the display 32.

[0019] Network interface 14 is connected to a network such as a LAN (Local Area Network). Computer 1 can communicate with other computers connected to the LAN via network interface 14, and can also communicate with servers on the Internet connected via the LAN. External interface 15 is an interface for connecting to external storage media ED. The CPU 21 can access storage media ED such as CD-ROMs, DVDs, MDs, and USB memory via external interface 15.

[0020] {3. Configuration of the Music Processing Device} Next, the functions of the music processing device 200 of this embodiment will be described. Figure 3 is a block diagram showing the functional configuration of the music processing device 200 according to this embodiment. The music processing device 200 is realized by the CPU 21 executing a program PG while utilizing hardware resources such as RAM 22. As shown in Figure 3, the music processing device 200 includes an estimation unit 201, a melody channel generation unit 202, a melody channel score output unit 203, and a melody channel audio output unit 204. Each of the functional units 201 to 204 is a functional unit realized by the CPU 21 executing a program PG while utilizing hardware resources such as RAM 22.

[0021] The estimation unit 201 processes the music data, which consists of multiple channels (or multiple tracks), in time blocks. A time block is a sub-section of music data divided in the time direction. The estimation unit 201 estimates the role of each channel in each time block. Here, "channel role" in this specification refers to the role that each channel constituting the music data has within the music. The role within the music may be the name of the performance part or the name of the instrument. Examples of "channel roles" include "melody," "guitar," "bass," "keyboard," "piano," "accompaniment," "riff," "drums," and "pads." Alternatively, there may be multiple types of "melody," such as "main melody" and "sub-melody." In this embodiment, the estimation unit 201 estimates the role of each channel using a machine learning model.

[0022] The melody channel generation unit 202 generates melody channel data MCD based on the role of each channel estimated by the estimation unit 201. The melody channel data MCD is data generated by concatenating the channels estimated to be "melody" in each time block. Specifically, the melody channel data MCD is generated by concatenating the music data belonging to the channels estimated to be "melody". The melody channel data MCD is an example of the "first role data" of the present invention. The melody channel generation unit 202 generates, for example, melody channel score data or melody channel audio data as melody channel data MCD. The melody channel generation unit 202 is an example of the "generation unit" in the present invention.

[0023] The melody channel score output unit 203 outputs the melody channel score data generated by the melody channel generation unit 202 to the display 32. The melody channel audio output unit 204 plays melody channel audio by providing the melody channel audio data generated by the melody channel generation unit 202 to the sound source 12 and the sound system 13.

[0024] {4. Music Processing Method} Next, the music processing method according to this embodiment will be described. Figure 4 is a flowchart of the music processing method according to this embodiment. The flowchart shown in Figure 4 shows the processing that is realized by having the CPU 21 execute the program PG while utilizing hardware resources such as RAM 22. In other words, the flowchart shown in Figure 4 shows the processing that is executed by the music processing device 200, which includes the estimation unit 201 and the melody channel generation unit 202 shown in Figure 3.

[0025] First, in step S1, the estimation unit 201 acquires the music data SD from the storage device 11. The music data SD is data composed of multiple channels (multiple tracks). In this embodiment, the case where the music data SD is performance description data such as MIDI data or XF data will be explained as an example.

[0026] Next, in step S2, the estimation unit 201 estimates the role of each channel for each time block of the music data. The estimation process in the estimation unit 201 will be explained with reference to Figures 5 and 6.

[0027] Figure 5 shows the data structure of the music data SD. As shown in the figure, the music data SD is composed of multiple channels. In the example shown in the figure, the music data SD is composed of five channels, A to E.

[0028] The estimation unit 201 processes the music data SD, which is composed of multiple channels, for each time block TB. As shown in Figure 5, each time block TB of the music data SD is composed of channels A to E. The unit of a time block may be a block of measures, such as 1 measure or 4 measures, or a block of time, such as 5 seconds or 10 seconds. The figure shows five time blocks TB1 to TB5.

[0029] The estimation unit 201 estimates the role of each channel in each time block TB. In this embodiment, as shown in Figure 6, the estimation unit 201 estimates the role of the channels using a machine learning model. The estimation unit 201 provides the machine learning model with information for each time block of the music data SD. The machine learning model takes the music data SD for each time block as input and outputs the role of each channel for each time block. If the machine learning model outputs the likelihood of each channel's role, the role with the highest likelihood may be assigned.

[0030] A machine learning model is created by taking music data with multiple channels as input and performing a learning process that outputs the role of each channel in each time block TB. The input learning data may be data for each time block, or it may be data for a part of a song or the entire song. In other words, the music data SD in Figure 6 may be data in units of time blocks, or it may be data of any length for which a properly trained machine learning model exists. As a machine learning model, for example, MusicBERT can be used. A machine learning model may also be created by performing additional training on MusicBERT.

[0031] Figure 7 shows the channel roles estimated by the estimation unit 201. As shown in the figure, the roles of channels A to B in each time block TB are estimated. For example, channels A to E in time block TB1 are estimated to be "main melody," "sub-melody," "bass," "riff," and "pad," respectively.

[0032] Refer to Figure 4 again. In step S3, the melody channel generation unit 202 generates melody channel data MCD corresponding to the melody by connecting the channels estimated to be the "main melody" in each time block TB of the music data SD. In this embodiment, as shown in Figure 7, the estimated roles include "main melody" and "sub-melody," so a melody channel is generated by connecting the "main melody." If the estimated role is only "melody," then a melody channel is generated by connecting the "melody."

[0033] Figure 8 shows the melody channel data MCD generated by combining the "main melodies" in each time block TB. The melody channel generation unit 202 generates, for example, musical score data as the melody channel data MCD. Alternatively, the melody channel generation unit 202 generates audio data as the melody channel data MCD.

[0034] In step S4, the melody channel generation unit 202 writes the generated melody channel data MCD to the storage device 11. As described above, the music processing method of this embodiment makes it possible to generate melody channel data MCD by combining channels that have the role of melody from music data SD which is composed of multiple channels.

[0035] {5. How to use melody channel data} If the melody channel data MCD is musical score data, the melody channel musical score output unit 203 displays the melody channel data MCD on the display 32. This allows the user to refer to the melody channel musical score displayed on the display 32. As shown in Figure 1, the user can play an instrument while referring to the melody channel musical score.

[0036] In this way, by using the generated melody channel data MCD, users can eliminate the effort of creating sheet music by extracting only the melody part from music data SD that has multiple performance parts or multiple instrument parts. For example, it is a very troublesome task for a user to create sheet music by extracting only the melody from purchased sheet music, but by using the method of this embodiment, the effort can be greatly reduced. By playing an instrument using the melody channel sheet music, the user can play the melody throughout the entire song using only one instrument. The music processing method of this embodiment can provide the user with an arranged sheet music for enjoying playing an instrument alone, from music data SD that has multiple performance parts.

[0037] If the melody channel data MCD is audio data, the melody channel audio output unit 204 provides the melody channel data MCD to the sound source 12 and the sound system 13. This allows the user to listen to music on the melody channel. By using the generated melody channel data MCD, the user can enjoy music in which only the melody part has been extracted from music data SD which has multiple performance parts or multiple instrument parts.

[0038] {6. Modification} In the above embodiment, the music processing method generated a melody channel by connecting melody (or tune) channels. In other words, channels whose role is melody (main melody in the above example) were connected. The music processing method of this embodiment is not limited to this example, and channels with other roles may be connected to generate a channel with a specific role. In other words, "melody" is just one example of the "first role" in the present invention, and "accompaniment" or the like may also be considered the "first role".

[0039] The music processing method of this embodiment may include an interface that allows the user to specify a "first role". For example, if the user wants to generate a melody channel, they specify "melody" as the "first role" that identifies the channel to be generated. As a result, the music processing method generates a melody channel based on the estimation result in the estimation unit 201. If the user wants to generate an accompaniment channel, they specify "accompaniment" as the "first role". As a result, the music processing method generates an accompaniment channel based on the estimation result in the estimation unit 201.

[0040] Furthermore, if the music data SD supports multiple instruments and has channels that support multiple instruments, the user may specify the name of the instrument to which the "first role" channel will be assigned. The music processing method will then generate channel data corresponding to the specified instrument. In this case, if the specified instrument is a transposing instrument, the music processing method may output transposed musical score data corresponding to that instrument. For example, if the "first role" is melody, transposed melody channel musical score data corresponding to the transposing instrument will be generated. This allows the user to obtain musical score data transposed for the specific instrument.

[0041] In the above-described embodiment, the music data SD has been described by taking the case where it is performance description data such as MIDI data or XF data as an example. As another example, the music data SD may be audio data. The estimation unit 201 gives audio data having a plurality of channels to the machine learning model. The machine learning model estimates the role of each channel for each time block of the audio data.

[0042] In the above-described embodiment, the program PG has been described by taking the case where it is stored in the storage device 11 as an example. As another embodiment, the program PG may be stored and provided in the storage medium ED. The CPU 21 may access the storage medium ED via the external interface 15 and store the program PG stored in the storage medium ED in the storage device 11. Alternatively, the CPU 21 may access the storage medium ED via the external interface 15 and execute the program PG stored in the storage medium ED. Alternatively, the CPU 21 may download the program PG from a server on the network via the network interface 14 and store the downloaded program PG in the storage device 11. Alternatively, the CPU 21 may directly execute the program PG stored in a server on the network via the network interface 14.

[0043] As the computer 1 in the embodiment, in addition to a personal computer, a tablet terminal or a smartphone can also be used. Alternatively, an electronic musical instrument may have the functions of the computer 1.

[0044] {7. Features and Effects of the Embodiment} (1) The music processing method according to the present embodiment is a method for processing music data SD having a plurality of channels, which estimates the role of each channel for each time block TB of the music data SD, and by connecting the channels estimated to have the first role in each time block TB of the music data SD, generates first role data corresponding to the first role.

[0045] By inputting music data having a plurality of channels, a channel with a specific role can be automatically generated. The user can save the trouble of creating a channel with a specific role from music data having a plurality of channels.

[0046] (2) In the above music processing method, the first role includes a melody, and the generation may generate a melody channel by connecting the channels estimated as melodies in each time block of the music data.

[0047] By inputting music data having a plurality of channels, a melody channel can be automatically generated.

[0048] (3) In the above music processing method, the music data may include performance description data.

[0049] Based on the performance description data having a plurality of channels, a channel with a specific role can be automatically generated.

[0050] (4) In the above music processing method, the music data may include audio data.

[0051] Based on the audio data, a channel with a specific role can be automatically generated.

[0052] (5) In the above music processing method, the estimation may estimate the role of each channel in each time block using a machine-learned model.

[0053] Since a machine-learned model is used, high-precision role estimation can be performed by improving the learning accuracy.

[0054] (6) In the above music processing method, the generation may generate score data as the first role data.

[0055] The user can save the trouble of creating a score by extracting a specific role from music having a plurality of channels.

[0056] (7) In the above music processing method, the generation may generate audio data as the first role data.

[0057] Users can listen to audio extracted from songs with multiple channels, focusing on specific roles.

[0058] (8) In the above music processing method, the multiple channels may include channels corresponding to multiple instruments, and further, a specific instrument to which the first role data to be generated is to be assigned may be specified.

[0059] You can specify the instrument you want to play on the first role channel.

[0060] (9) In the above music processing method, if the specified instrument is a transposing instrument, transposed musical score data corresponding to the specified instrument may be generated as the first role data.

[0061] This is highly convenient when users need a first-channel score that is adapted for transposing instruments.

[0062] The music processing program according to this embodiment is a program for processing music data having multiple channels, and causes a computer to perform the following processes: estimating the role of each channel for each time block TB of the music data SD, and generating first role data corresponding to the first role by connecting the channels estimated to have a first role in each time block TB of the music data SD.

[0063] The music processing device according to this embodiment is a music processing device 200 that processes music data SD having multiple channels, and comprises an estimation unit 201 that estimates the role of each channel for each time block TB of the music data SD, and a generation unit 202 that generates first role data corresponding to the first role by connecting the channels estimated to have a first role in each time block TB of the music data SD.

[0064] 1: Computer, 4: User, 5: Electronic piano, 31: Operator, 32: Display, 200: Music processing unit, 201: Estimation unit, 202: Melody channel generation unit, 203: Melody channel score output unit, 204: Melody channel audio output unit, A-E: Channels, MCD: Melody channel data, SD: Music data, TB: Time block, TB1-TB5: Time block

Claims

1. A method for processing music data having multiple channels, comprising: estimating the role of each channel for each time block of the music data; and generating first role data corresponding to the first role by connecting the channels estimated to have a first role in each time block of the music data.

2. The music processing method according to claim 1, wherein the first role includes a melody, and the generation is the generation of a melody channel by connecting the melody and the estimated channel in each time block of the music data.

3. The music processing method according to claim 1, wherein the music data includes performance description data.

4. The music processing method according to claim 1, wherein the music data includes audio data.

5. The music processing method according to claim 1, wherein the estimation involves estimating the role of each channel in each time block using a machine learning model.

6. The method for processing music according to claim 1, wherein the generation described above generates musical score data as the first role data.

7. The music processing method according to claim 1, wherein the generation described above generates audio data as the first role data.

8. The music processing method according to claim 1, wherein the plurality of channels include channels corresponding to a plurality of instruments, and further specifies a particular instrument to which the first role data to be generated is to be assigned.

9. The music processing method according to claim 8, wherein if the specified instrument is a transposing instrument, a transposed musical score data corresponding to the specified instrument is generated as the first role data.

10. A music processing program for processing music data having multiple channels, wherein the program causes a computer to perform the following processes: estimating the role of each channel for each time block of the music data; and generating first role data corresponding to the first role by connecting the channels estimated to have a first role in each time block of the music data.

11. A music processing device for processing music data having multiple channels, comprising: an estimation unit that estimates the role of each channel for each time block of the music data; and a generation unit that generates first role data corresponding to the first role by connecting the channels estimated to have a first role in each time block of the music data.

Images