Acoustic device, control method for acoustic device, and program
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Filing Date
- 2026-03-13
- Publication Date
- 2026-06-15
Abstract
Description
Audio device, audio device control method and program
[0001] The present invention relates to an audio device, a control method for an audio device, and a program.
[0002] In recent years, various technologies have been developed for audio devices and applications related to DAWs (Digital Audio Workstations) and DTMs (Desktop Music). For example, Non-Patent Document 1 discloses an application that can perform audio separation after the fact on a computer.
[0003] Djay Neural Mix (algoriddim GmbH) application introduction homepage (https: / / www.algoriddim.com / neural-mix)
[0004] However, it is difficult to say that the above-mentioned technologies have yet been fully proposed for practical use. Therefore, an object of the present invention is to provide an audio device, a program, and a control method that can improve the convenience and versatility of playback for users by making optimal use of audio data that has been subjected to separation processing.
[0005] [1] An audio device that plays music including a sonically separable first part, the audio device comprising: an extraction unit that extracts a separated music piece by removing the first part from the music piece; and a playback control unit that plays a composite music piece by adding a prepared sample sound to the separated music piece, and that plays the sample sound so that the beat positions of the sample sound match those of the separated music piece. [2] The audio device described in [1] further comprises a reception unit that receives user operations, and when the reception unit receives a predetermined user operation, the extraction unit extracts the separated music piece and the playback control unit plays the composite music piece. [3] The audio device described in [1] or [2], wherein the playback control unit applies a special effect to the sample sound and then adds the sample sound to the separated music piece. [4] The audio device described in any of [1] to [3], wherein the playback control unit applies a special effect to the separated music piece and then adds the sample sound to the separated music piece. [5] The audio device described in any of [1] to [4], wherein the first part is a part that includes a drum sound. [6] A control method for an audio device that plays back music that includes a first part that can be acoustically separable, the control method comprising the steps of: extracting a separated music piece by removing the first part from the music piece; and playing back a composite music piece by adding prepared sample sounds to the separated music piece, while aligning the beat positions of the sample sounds to those of the separated music piece. [7] A program that causes a computer to function as an audio device that plays back music that includes a first part that can be acoustically separable, the program comprising: an extraction unit that extracts a separated music piece by removing the first part from the music piece; and a playback control unit that plays back a composite music piece by adding prepared sample sounds to the separated music piece, while aligning the beat positions of the sample sounds to those of the separated music piece.
[0006] FIG. 1 is a schematic diagram showing an acoustic device according to an embodiment of the present invention. FIG. 2 is another schematic diagram showing an acoustic device according to an embodiment of the present invention. FIG. 3 is a block diagram showing a schematic functional configuration of an acoustic device according to an embodiment of the present invention. FIG. 4 is a schematic diagram illustrating part replacement processing according to an embodiment of the present invention. FIG. 5 is another schematic diagram illustrating part replacement processing according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating a method for controlling an acoustic device according to an embodiment of the present invention.
[0007] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.
[0008] The audio device 1 is a DJ system that integrates two players 2A and 2B that play music, a mixer 3 that controls the players 2A and 2B, and a display unit 4. Specifically, the audio device 1 functions as an audio playback device and an audio playback control device when operated by a user.
[0009] 1, the audio device 1 includes operation units 21A and 21B provided on the players 2A and 2B, respectively, and an operation unit 31 provided on the mixer 3. The operation unit 21A accepts user operations for the player 2A, and the operation unit 21B accepts user operations for the player 2B. The operation unit 31 also accepts user operations for the mixer 3.
[0010] The operation section 21A includes a jog dial 211A, a tempo slider 212A, a cue button 213A, a play / pause button 214A, a performance pad 215A, and a part switch operation section 216A.
[0011] More specifically, the jog dial 211A is a rotary operator that is used to set the playback direction and playback speed of a song being played on the player 2A. The tempo slider 212A is used to adjust the playback speed of a song being played on the player 2A. The cue button 213A is pressed to set a specific position in the song as a cue point. The play / pause button 214A is pressed to start or stop playback of a song on the player 2A. The performance pads 215A are general-purpose operators that can be assigned various functions related to playback control.
[0012] The part switch operation section 216A is an operation section for making settings related to the process of switching parts of a musical piece, and includes four part switch buttons 218A to 221A, as shown in Fig. 2. The part switch buttons 218A to 221A will be described in detail later. The number of part switch buttons may be three or less, or five or more, and may be configured so that one button can be used in multiple ways by combining it with a shift button or the like.
[0013] Similar to the operation unit 21A, the operation unit 21B includes a jog dial 211B, a tempo slider 212B, a cue button 213B, a play / pause button 214B, a performance pad 215B, and a part switch operation unit 216B. Similarly to the part switch operation unit 216A, the part switch operation unit 216B includes part switch buttons 218B to 221B. The layout of each unit is the same as that of the part switch operation unit 216A, and is therefore not shown. For example, when a user operates the operation unit 21A of the player 2A, the sound device 1 plays back the music loaded in the player 2A and applies effects to the music being played back as necessary. The same is true for the player 2B.
[0014] In response to user operations, the mixer 3 switches music playback between the players 2A and 2B, adjusts the volume of each channel, and performs playback with effects applied. The operation unit 31 of the mixer 3 includes effect selection knobs 311A and 311B, effect amount adjustment knobs 312A and 312B, channel faders 313A and 313B, and a crossfader 314.
[0015] More specifically, the effect selection knobs 311A and 311B are used to select the effect to be applied to the music being played on the players 2A and 2B, respectively. The effect amount adjustment knobs 312A and 312B are used to adjust parameters such as the depth and magnitude of the effect to be applied to the music being played on the players 2A and 2B, respectively. The channel faders 313A and 313B are used to adjust the output volume level of the music being played on the players 2A and 2B, respectively.
[0016] The crossfader 314 is an operator for adjusting the volume balance between a first song played on the player 2A, which is the first channel, and a second song played on the player 2B, which is the second channel.
[0017] The display unit 4 includes a display 41 and a touch panel 42 that is provided to cover the display area of the display. The touch panel 42 detects the position of contact by the user and outputs information such as coordinates indicating the contact position, thereby allowing the display unit 4 to also function as an operation unit.
[0018] Effects in players 2A and 2B and mixer 3 include, for example, effects that apply effects such as delay, echo, reverb, or loop to a song to play effect sounds, and effects that process the original sound and play it back.
[0019] Fig. 3 is a block diagram showing a schematic functional configuration of the audio device 1 according to this embodiment. In addition to the above-mentioned components, the audio device 1 also includes a music data storage unit 5, a control unit 6, and an audio output unit 7, as shown in Fig. 3. The functions of the above-mentioned components are realized by a processor operating according to a program in an audio device having the hardware configuration of a computer, for example. The functions of the components will be further described below.
[0020] The music data storage unit 5 is configured to store music data using a hard disk drive (HDD) or flash memory, etc. The music data storage unit 5 stores music data for multiple songs in a predetermined format, such as MP3 format. In addition to audio information, the music data includes tag information such as the song's BPM, artwork, title, artist name, album name, key, number of DJ plays, and genre. The music data stored in the music data storage unit 5 is associated with a timestamp, which is information about the playback position.
[0021] In this embodiment, music data stored in the music data storage unit 5 and playable on the audio device 1 is preliminarily separated into multiple parts. In this embodiment, the multiple parts separated from the music data are determined based on at least one of the presence or absence of instrument sounds and the type of instrument. In other words, unlike multiple parts separated by filter processing, such as multiple parts with different frequency components, the multiple parts reflect the characteristics of the music based on at least one of the presence or absence of instrument sounds and the type of instrument. Here, the parts of a music piece correspond to vocals or the instrument sounds of each instrument, and include, for example, a vocal part, a drum part, a bass part, and a melody part. Note that the melody part may be, for example, a part obtained by excluding the vocal part, drum part, and bass part from the entire music piece. It is also useful to subdivide and separate the parts into individual instrumental parts, such as a bass drum part, a snare drum part, and a hi-hat part for a drum part. Note that various well-known techniques can be used for this separation process into multiple parts, and therefore a detailed description thereof will be omitted.
[0022] 4 is a schematic diagram illustrating a process for replacing some parts of a song executed by the audio device according to this embodiment. In the following description, as an example, it is assumed that song data that has been separated into n parts, part A1 to part An, is stored in the song data storage unit 5. The audio device 1 may externally acquire song data that has already been separated, or may further include a functional unit that executes a process for separating unseparated song data into multiple parts.
[0023] The control unit 6 is implemented in the audio device 1 by, for example, a communication interface, a processor such as a CPU (Central Processing Unit), and a memory that serves as a working area, and controls the operation of the audio device 1. The control unit 6 includes a reception unit 61, an extraction unit 62, and a playback control unit 63, which are realized by the processor operating in accordance with a program stored in the memory or received via the communication interface.
[0024] The reception unit 61 receives user operations on the operation units 21A, 21B, the operation unit 31, and the touch panel 42. The extraction unit 62 extracts a separated piece of music by removing a specific part from the music to be played. The specific part may be a predetermined part, or may be a part determined based on a user operation or the like. The specific part may be one part, or two or more parts. The number of parts included in the specific part may be determined based on a user operation or the like.
[0025] The playback control unit 63 plays back a composite music piece by adding a sample sound prepared in advance to the separated music piece extracted by the extraction unit 62. At this time, the playback control unit 63 aligns the beat positions of the added sample sounds with the beat positions of the separated music piece.
[0026] Here, the sample sounds are pre-stored in the playback control unit 63 and are added to the separated music pieces in place of or in addition to the specific parts described above. The sample sounds are stored in association with each of the part switch buttons 218A to 221A described above.
[0027] The sample sound may be generated by a separation process for any song, including the one currently being played, or may be generated based on a performance by a user, or may be a song that is sold. The process of replacing some parts of a song using sample sounds will be described in detail later.
[0028] The audio device 1 may also be configured to include a communications interface, and acquire music data stored in an external storage device, a computer, or the like via the communications interface (not shown), and store the music data in the music data storage unit 5. In this case, the audio device 1 does not include the music data storage unit 5, and the external storage device functions as the music data storage unit 5. The audio output unit 7 includes an audio output terminal, such as a speaker terminal or a headphone terminal, and outputs the audio signal of the music output by the playback control unit 63 as audio information.
[0029] The process of replacing some parts of a song in the above-described audio device 1 will now be described with reference to Fig. 4. The example of Fig. 4 illustrates a case in which, for example, part A2 is removed as a specific part from a song to be played (hereinafter referred to as the original song) to extract a separated song, and a sample sound Sa is added to the extracted separated song as a prepared sample sound to generate a composite song to be played.
[0030] As described above, part A2 may be any part determined based on at least one of the presence or absence of an instrument sound and the type of instrument, but the following explanation will be given using an example where part A2 is a drum part.
[0031] In general, drum sounds have a significant impact on the genre and atmosphere of a song, and devising drum parts can improve the diversity of musical performances and arrangements. For example, if the drum part of the original song is monotonous and lacks musical development, the beat and rhythm of the song will also be monotonous and lack development, which has the disadvantage of making users bored.
[0032] One possible solution to this problem is to create an arrangement that changes the drum part of the original song. When changing only the drum part of the original song, it is necessary to simultaneously and quickly process the original drum part and the sample sound used to add or replace it. Furthermore, it is necessary to consider the BPM and beat position of both, as well as the timing of the operations. This creates a problem where it is difficult for users with little musical knowledge or experience, or for novice DJs, to execute this process.
[0033] According to the sound device 1 of this embodiment, for example, the extraction unit 62 sets a drum part as a specific part to be removed from the original song, and the playback control unit 63 adds a sample sound containing a predetermined drum sound from sample sounds prepared in advance to the separated song excluding the drum part extracted by the extraction unit 62, thereby replacing only the drum part of the song. Therefore, even a user with little knowledge or experience about music can easily perform and arrange songs with simple operations.
[0034] This process is executed when one of the four part switch buttons 218A to 221A described with reference to Fig. 2 is pressed. A sample sound is stored in advance in association with each of the four part switch buttons 218A to 221A. For example, if multiple drum parts of different genres are stored as sample sounds, the user can easily change the drum parts and arrange the genre of the music piece simply by pressing one of the part switch buttons 218A to 221A.
[0035] Also, for example, if multiple drum parts played by different artists are stored as sample sounds, the user can simply press one of the part switch buttons 218A to 221A to switch between the drum parts and easily arrange the atmosphere of the song, etc.
[0036] To achieve this type of processing, when switching parts, the playback control unit 63 performs a process to match the BPM of the sample sound associated with part switch buttons 218A to 221A with the BPM of the original song being played or the separated song based on the original song, and then adds the sample sound, thereby minimizing the discrepancy between the original song and the sample sound and reducing the sense of discomfort felt by the user.
[0037] Furthermore, the playback control unit 63 aligns the beat position of the added sample sound with the beat position of the original song or the separated music piece based on the original song. The method of aligning the beat positions can be the same as known techniques. For example, the beat position of the sample sound may be aligned with the next beat position in the original song or the separated music piece based on the original song, starting from the time when one of the part switch buttons 218A to 221A is pressed. Alternatively, the beat position of the sample sound may be aligned with the next music development position in the original song or the separated music piece based on the original song, starting from the time when one of the part switch buttons 218A to 221A is pressed. By aligning the beat positions in this way, the continuity of the BPM is maintained, allowing for a more natural performance, thereby reducing the sense of discomfort felt by the user.
[0038] 5 to 7 show modified examples of the processes shown in Fig. 4. Each example is an example of processing that is performed to improve the variety of musical performances and arrangements, or to improve the accuracy of processing.
[0039] For example, as shown in Fig. 5, the playback control unit 63 may be configured to add a special effect to the sample sound and then add the sample sound to the separated music. Here, the special effect includes a process of additionally playing an effect sound or a process of processing and playing the original sound. In this way, by adding a special effect to the sample sound and then adding the sample sound to the separated music, the variety of music performances and arrangements can be improved.
[0040] Note that whether or not to apply special effects to sample sounds may be set by the user, or may be configured so that application is switched depending on predetermined conditions. The type and conditions of special effects to be applied to sample sounds may also be set by the user, or may be configured so that they can be set depending on predetermined conditions.
[0041] Furthermore, the playback control unit 63 may be configured to add sample sounds to the separated music pieces after applying special effects to some or all parts of the separated music pieces, as shown in FIG. 6 . Here, special effects include additional playback of effect sounds, as with the special effects applied to the sample sounds described above, or processing and playback of processed original sounds. In this way, adding sample sounds to the separated music pieces after applying special effects to them can improve the variety of musical performances and arrangements.
[0042] The user may be able to set whether or not to apply special effects to the separated music pieces, or the application may be switched depending on predetermined conditions. The types and conditions of special effects for the separated music pieces may also be set by the user, or may be set depending on predetermined conditions.
[0043] Furthermore, the extraction unit 62 may be configured to add sample sounds to the separated music after performing ducking processing during generation of the separated music, as shown in Figure 7. Here, ducking processing is, for example, a process of periodically adjusting the volume curve, and is a well-known technique. By applying such ducking processing to the music separation process, it is possible to suppress any unnaturalness or roughness that may occur during the separation process itself or the process of removing specific parts in the subsequent extraction process, and to make the sample sounds added by the synthesis process more noticeable.
[0044] Whether or not to apply ducking processing may be set by the user, or application may be switched depending on predetermined conditions. The type and conditions of ducking processing may also be set by the user, or may be set depending on predetermined conditions.
[0045] Furthermore, a configuration may be adopted in which some or all of the processes described with reference to Figures 5 to 7 are executed in combination. Some specific examples of use will be given below.
[0046] [Example of Use 1] When changing the drum part of a currently playing song from a four-on-the-floor beat to a half beat at the time when the chorus begins, the user assigns a half beat drum sound to one of the four part switch buttons 218A to 221A in advance by storing the half beat drum sound in association with the sample sound. Then, the user monitors the development of the currently playing song and presses the assigned button among the part switch buttons 218A to 221A at the time when the chorus begins.
[0047] When one of the part switch buttons 218A to 221A to which a half-beat drum sound is assigned as a sample sound is pressed, the extraction unit 62 removes the drum part as a specific part from the original song and adds the half-beat drum sound as a sample sound. In other words, the user can replace the four-on-the-floor drum part of the original song with the sample sound, which is a half-beat drum sound, simply by pressing the assigned button of the part switch buttons 218A to 221A at a suitable timing.
[0048] [Usage Example 2] To change the genre of the drum part of a song currently being played and apply effect processing to the sample sound to be added, the user assigns a drum sound appropriate for the desired genre to one of the four part switch buttons 218A to 221A by storing the sample sound in association with the drum sound. The type and conditions of the effect processing to be applied to the sample sound are also set in advance. Then, while monitoring the development of the song currently being played, the user presses the assigned button from among the part switch buttons 218A to 221A when the user wishes to change the genre of the drum part.
[0049] When one of the part switch buttons 218A to 221A to which a drum sound appropriate for the genre to be changed is assigned as a sample sound is pressed, the extraction unit 62 removes the drum part as a specific part from the original song, applies effect processing to the sample sound, and then adds the sample sound. In other words, the user can change the genre of the drum part and apply effect processing to the sample sound to be added simply by pressing the assigned button of the part switch buttons 218A to 221A at a suitable timing.
[0050] The type and conditions of the effect processing to be applied to the sample sound may be configured to be set after the assigned button is pressed, instead of being set in advance.
[0051] Next, a control method for processing to switch parts of a piece of music executed by the audio device according to this embodiment will be described with reference to the flowchart of FIG.
[0052] The control unit 6 determines whether any of the part switch buttons 218A to 221A has been pressed (step S201), and if it determines that any of the part switch buttons 218A to 221A has been pressed (step S201 YES), the extraction unit 62 extracts a separated piece of music with the drum part of the original song removed (step S202).
[0053] The playback control unit 63 then adds the sample sound corresponding to the part switch button 218A to 221A pressed in step S201 to the separated music piece (step S203).
[0054] Note that, after one of the part switch buttons 218A to 221A is pressed and the part switching process is initiated, the original song may be restored in response to a predetermined user operation. For example, after one of the part switch buttons 218A to 221A is pressed and the part switching process is initiated, if the same part switch button is pressed again, the part switching may be terminated and the original song may be restored.
[0055] Furthermore, if one of the part switch buttons 218A to 221A is pressed to start the part switching process, and then a different part switch button is pressed, the composite music piece currently being played back becomes the original music piece, and the same process is carried out. With this configuration, each time one of the part switch buttons 218A to 221A is pressed in sequence, the parts are switched in accordance with the part switch button that was pressed.
[0056] As described above, this embodiment includes an extraction unit 62 that extracts a separated piece of music by removing specific parts from the music, and a playback control unit 63 that plays back a composite piece of music by adding prepared sample sounds to the separated piece of music, and that plays back the sample sounds by matching the beat positions of the sample sounds to the beat positions of the separated piece of music. Therefore, by making optimal use of the separated audio data, it is possible to improve the diversity of musical performances and arrangements. This is expected to improve the convenience and diversity of playback for users.
[0057] Furthermore, this embodiment further includes a reception unit 61 that receives user operations. When a predetermined user operation is received by the reception unit 61, the extraction unit 62 extracts the separated music pieces, and the playback control unit 63 plays the combined music piece. Therefore, the user can easily switch parts by simply pressing one of the part switch buttons 218A to 221A, without performing any complicated operations.
[0058] Furthermore, according to this embodiment, the playback control unit 63 applies special effects to the sample sounds and then adds the sample sounds to the separated music pieces. This allows for a variety of different expressions to be realized with the sample sounds that are added.
[0059] Furthermore, according to this embodiment, the playback control unit 63 adds sample sounds to the separated music pieces after applying special effects to the separated music pieces, thereby enabling various expressions to be realized for separated music pieces based on the original music pieces.
[0060] Note that some of the processing described in this embodiment may be omitted. For example, in this embodiment, the extraction unit 62 extracts a separated piece of music by removing a specific part from the music to be played. However, a configuration may be adopted in which the same processing as in this embodiment is performed without removing the specific part.
[0061] For example, a sampled sound may be added so that it is played in parallel with a specific part based on the original song. This configuration opens up the possibility of more detailed and sophisticated performances and arrangements, such as adding a decorative sampled sound like a hi-hat to a drum part based on the original song.
[0062] It is preferable that the extraction unit 62 be configured to be able to switch, based on user operation, whether to remove a specific part from the song to be played, that is, whether to replace the specific part or to maintain the specific part and add a sample sound.
[0063] Furthermore, the audio device with the above-described functions is not limited to the DJ system described in the above embodiment, but may also be, for example, a mixer or a DJ controller with mixer functionality. The present invention may also be applied to DJ applications and music applications executed on computers, smartphones, and tablet devices, or to streaming services using the Internet. While the above example describes a two-channel audio device with two players 2A and 2B, similar functions can also be achieved with, for example, a four-channel audio device. Furthermore, the present invention is not limited to DJ equipment, but may also be applied to audio devices such as general mixers, electronic musical instruments, and even DAWs (Digital Audio Workstations) and DTMs (Desktop Music Players).
[0064] Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art to which the present invention pertains can conceive of various modifications and alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present invention.
[0065] 1...sound device, 2A, 2B...player, 3...mixer, 4...display unit, 5...music data storage unit, 6...control unit, 7...audio output unit, 21A, 21B, 31...operation unit, 41...touch panel, 61...reception unit, 62...extraction unit, 63...playback control unit, 211A, 211B...jog dial, 212A, 212B...tempo slider, 213A, 213B...cue button, 214A, 214B...play / pause button, 215A, 215B...performance pad, 216A, 216B...part switch operation unit, 218A to 221A, 218B to 221B...part switch button, 311A, 311B...effect selection knob, 312A, 312B...effect amount adjustment knob, 313A, 313B...channel fader, 314...crossfader.
Claims
1. In an audio device for playing a musical piece that includes a first part that is audibly separable, An extraction unit that extracts a separated song obtained by removing the first part from the aforementioned song, A playback control unit plays a synthesized song by adding pre-prepared sample sounds to the separated song, and plays the sample sounds in accordance with the beat positions of the separated song. An acoustic device equipped with the following features.
2. It also includes a reception area to handle user inquiries, When the reception unit receives a predetermined user operation, The extraction unit extracts the separated music, The sound device according to claim 1, wherein the playback control unit plays the synthesized music.
3. The sound apparatus according to claim 1, wherein the playback control unit adds a special effect to the sample sound and then adds the sample sound to the separated music.
4. The sound apparatus according to claim 1, wherein the playback control unit adds the sample sound to the separated music after applying special effects to the separated music.
5. The sound device according to any one of claims 1 to 4, wherein the first part is a part that includes drum sounds.
6. A method for controlling an audio device that plays a musical piece including a first part that is audibly separable, A step of extracting a separated song by removing the first part from the aforementioned song, The steps include: playing a synthesized song by adding pre-prepared sample sounds to the separated song, and playing the sample sounds in accordance with the beat positions of the separated song; A method for controlling an acoustic device, including an acoustic device.
7. A program for causing a computer to function as an audio device that plays a musical piece containing a first part that is auditorily separable, An extraction unit that extracts a separated song obtained by removing the first part from the aforementioned song, A program for causing a computer to function as an audio device, comprising a playback control unit that plays a synthesized song by adding pre-prepared sample sounds to the separated song, and plays the sample sounds in accordance with the beat positions of the separated song.