Performance system, terminal device, electronic musical instrument, and method

Abstract

A performance system, a terminal device, an electronic musical instrument, and a method capable of switching the reproduced sound parts through simple operation instructions. The performance system includes an electronic musical instrument and a terminal device. The terminal device includes an output unit. After outputting partial data of only the first track in the music data, and any one of the combined partial data of a first style of multiple track data in the music data, the output unit automatically outputs partial data of only the second track data in the music data, and any one of the combined partial data of a second style of multiple track data, based on the acquisition of instruction data output from the electronic musical instrument. The electronic musical instrument includes an output unit, an acquisition unit, and a sounding unit. The output unit outputs the instruction data according to the user operation. The acquisition unit acquires the partial data output from the terminal device. The sounding unit emits a musical sound corresponding to the acquired partial data while emitting a musical sound corresponding to the user's performance operation.

Description

Technical Field

[0001] The invention relates to a performance system, a terminal device, an electronic musical instrument and a method. Background Art

[0002] Electronic musical instruments, such as numeric keyboards, have processors and memory, and can be said to be embedded computers with keyboards. As long as the model has an interface such as USB (Universal Serial Bus) or BlueTooth (registered trademark), the electronic musical instrument can be connected to a terminal device (computer, smartphone, tablet computer, etc.), and the electronic musical instrument can be played while the terminal device is activated. For example, it is also possible to play the electronic musical instrument while reproducing the audio source stored in the smartphone using the speaker of the electronic musical instrument.

[0003] However, in recent years, audio source separation technology has been developing (for example, refer to Japanese Patent Publication No. 2019-8336). Summary of the invention

[0004] If the sound source separation technology is used, the user can enjoy the performance of the favorite part (e.g., piano 3) on the electronic musical instrument by separating and cutting out a part of the arbitrary part (e.g., vocal 1, guitar 2) instead of reproducing all the parts (e.g., vocal 1, guitar 2, piano 3) included in the sound source, and not reproducing (pronouncing) a part of the arbitrary part (e.g., piano 3). However, especially when the user is playing, the switching operation of which part to reproduce is cumbersome, and it is desired to be able to perform the switching instruction of the reproduced part through a simple operation.

[0005] As a performance system of one embodiment of the present invention,

[0006] An electronic musical instrument (1) and a terminal device (TB) are provided.

[0007] The terminal device (TB) includes an output unit (54), which automatically outputs the second track data among the plurality of track data included in the music data and any one of the first pattern data obtained by arbitrarily combining the track data included in the plurality of track data based on the acquisition of instruction data output from the electronic musical instrument (1), after outputting the first track data among the plurality of track data included in the music data and any one of the first pattern data obtained by arbitrarily combining the track data included in the plurality of track data.

[0008] The electronic musical instrument (1) comprises:

[0009] a communication unit (216) that outputs the instruction data according to a user operation and obtains the data output by the terminal device (TB); and

[0010] The sound producing section (42) produces musical sounds corresponding to the acquired data and musical sounds corresponding to the performance operation of the user.

[0011] According to the present invention, the user can issue an instruction to switch the reproduction part with a simple operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is an external view showing an example of a performance system according to an embodiment.

[0013] Figure 2 This is a block diagram showing an example of the numeric keypad 1 according to the embodiment.

[0014] Figure 3 It is a functional block diagram showing an example of the terminal device TB.

[0015] Figure 4 This is a diagram showing an example of information stored in the ROM 203 and the RAM 202 of the numeric keypad 1 .

[0016] Figure 5 This is a flowchart showing an example of a processing procedure of the terminal device TB and the numeric keypad 1 according to the embodiment.

[0017] Fig. 6A It is a diagram showing an example of a GUI displayed on the display unit 52 of the terminal device TB.

[0018] Figure 6B It is a diagram showing an example of a GUI displayed on the display unit 52 of the terminal device TB.

[0019] Figure 6C It is a diagram showing an example of a GUI displayed on the display unit 52 of the terminal device TB.

[0020] Fig. 7A It is a diagram showing an example of a GUI displayed on the display unit 52 of the terminal device TB.

[0021] Figure 7B It is a diagram showing an example of a GUI displayed on the display unit 52 of the terminal device TB.

[0022] Figure 7C It is a diagram showing an example of a GUI displayed on the display unit 52 of the terminal device TB.

[0023] Figure 8 This is a conceptual diagram showing an example of a processing procedure in the embodiment. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0025] <Structure>

[0026] Figure 1 1 is an external view showing an example of a performance system according to an embodiment. The digital keyboard 1 is an electronic musical instrument such as an electronic piano, a synthesizer, or an electronic organ. The digital keyboard 1 includes a plurality of keys 10 arranged on the keyboard, a display unit 20, an operation unit 30, and a music stand MS. Figure 1 As shown, the terminal device TB connected to the numeric keyboard 1 can be placed on the music stand MS.

[0027] The key 10 is an operation member for the performer to specify a pitch, and the numeric keypad 1 produces and silences a sound corresponding to the specified pitch by the performer pressing and releasing the key 10. Furthermore, the key 10 also functions as a button for providing an instruction message to the terminal.

[0028] The display unit 20 has, for example, a liquid crystal display (LCD) with a touch panel, and displays messages accompanying the performer's operation of the operating unit 30. In the present embodiment, since the display unit 20 has a touch panel function, the display unit 20 can serve as one end of the operating unit 30.

[0029] The operation unit 30 has operation buttons for the performer to perform various settings and the like, and is a portion for performing various setting operations such as volume adjustment and the like.

[0030] The sound generating unit 40 is a part that outputs sound, and includes an output unit such as a speaker 42 and a headphone output.

[0031] Figure 2 1 is a block diagram showing an example of a numeric keypad 1 according to an embodiment. In addition to a display unit 20, an operation unit 30, and a speaker 42, the numeric keypad 1 includes a communication unit 216, a RAM (Random Access Memory) 203, a ROM (Read Only Memory) 202, an LCD controller 208, an LED (Light Emitting Diode) controller 207, a keyboard 101, a key scanner 206, a MIDI interface (I / F) 215, a bus 209, a CPU (Central Processing Unit) 201, a timer 210, a sound source 204, a digital / analog (D / A) converter 211, a mixer 213, a D / A converter 212, a rear panel unit 205, and an amplifier 214.

[0032] The CPU 201 , the sound source 204 , the D / A converter 212 , the rear panel unit 205 , the communication unit 216 , the RAM 202 , the ROM 203 , the LCD controller 208 , the LED controller 207 , the key scanner 206 , and the MIDI interface 215 are connected to the bus 209 .

[0033] The CPU 201 is a processor that controls the numeric keypad 1. That is, the CPU 201 reads out and executes the program stored in the ROM 203 from the RAM 202 as a working memory to realize various functions of the numeric keypad 1. The CPU 201 operates according to the clock supplied from the timer 210. The clock is used to control the sequence of automatic performance and automatic accompaniment, for example.

[0034] RAM 202 stores data generated when the numeric keypad 1 is operated, various setting data, etc. ROM 203 stores programs for controlling the numeric keypad 1, factory preset data, and automatic accompaniment data, etc. The automatic accompaniment data may include pre-set rhythm patterns, chord progressions, bass patterns, or melody data such as obbligato, etc. Melody data may include pitch information of each tone, pronunciation timing information of each tone, etc.

[0035] The timing of each sound can be the interval between each sound or the time elapsed from the start of the automatic performance. The unit of time is often tick. Tick is a unit used in general sequencers based on the speed of the song. For example, if the resolution of the sequencer is 480, 1 / 480 of the time of a quarter note is 1 tick.

[0036] The automatic accompaniment data is not limited to the ROM 203, and may be stored in an information storage device or an information storage medium (not shown). The format of the automatic accompaniment data may also be based on the file format for MIDI.

[0037] The sound source 204 is, for example, a so-called GM sound source based on the GM (General MIDI) standard. Such a sound source can change the timbre by giving a program change as a MIDI message, and can control a predetermined effect by giving a control change.

[0038] The sound source 204 has a maximum sound generation capability of 256 channels at a time. The sound source 204 reads out musical sound waveform data, for example, from a waveform ROM (not shown). The musical sound waveform data is converted into an analog musical sound waveform signal by a D / A converter 211 and input to a mixer 213. On the other hand, digital audio data in the form of mp3, m4a, or wav is input to a D / A converter 212 via a bus 209. The D / A converter 212 converts the audio data into an analog waveform signal and inputs the analog waveform signal to a mixer 213.

[0039] The mixer 213 mixes the analog musical sound waveform signal and the analog waveform signal to generate an output signal. The output signal is amplified by the amplifier 214 and output from the speaker 42 or an output terminal such as a headphone output. The mixer 213, the amplifier 214, and the speaker 42 provide a function as a sound producing unit that synthesizes the digital audio signal etc. received from the terminal device TB with the musical sound and outputs the sound. That is, the sound producing unit produces the musical sound corresponding to the acquired partial data while producing the musical sound corresponding to the user's playing operation.

[0040] The musical sound waveform signal from the sound source 204 and the audio waveform signal from the terminal device TB are mixed in the mixer 213 and output from the speaker 42. Thus, the user can enjoy the performance of the digital keyboard 1 in combination with the audio signal from the terminal device TB.

[0041] The key scanner 206 constantly monitors the key pressing / releasing state of the keyboard 101 and the switch operation state of the operation unit 30. Then, the key scanner 206 transmits the states of the keyboard 101 and the operation unit 30 to the CPU 201.

[0042] The LED controller 207 is, for example, an IC (Integrated Circuit). The LED controller 207 illuminates the keys 10 of the keyboard 101 in accordance with instructions from the CPU 201 to guide the player's performance. The LCD controller 208 controls the display state of the display unit 20 .

[0043] The rear panel portion 205 includes, for example, a socket for inserting a cable extending from the foot pedal FP. In addition, the rear panel portion 205 is also provided with MIDI terminals of MIDI-IN, MIDI-THRU, and MIDI OUT, and a headphone jack in many cases.

[0044] The MIDI interface 215 inputs MIDI messages (performance data, etc.) from an external device such as the MIDI device 4 connected to the MIDI terminal, or outputs MIDI messages to an external device. The received MIDI message is transmitted to the sound source 204 via the CPU 201. The sound source 204 sounds the sound according to the timbre, volume, timing, etc. specified by the MIDI message. In addition, MIDI messages and MIDI data files can also be sent and received with external devices via the USB.

[0045] The communication unit 216 has a wireless communication interface such as Bluetooth (registered trademark), and can transmit and receive digital data with the paired terminal device TB. For example, MIDI data (performance data) generated by playing the numeric keyboard 1 can be sent to the terminal device TB via the communication unit 216 (the communication unit 216 functions as an output unit). In addition, the communication unit 216 also functions as a receiving unit (acquisition unit) that receives digital audio signals sent from the terminal device TB.

[0046] Furthermore, a storage medium not shown in the figure may also be connected to the bus 209 via a slot terminal (not shown in the figure) or the like. Examples of the storage medium include a USB memory, a floppy disk drive (FDD), a hard disk drive (HDD), a CD-ROM drive, and a magneto-optical (MO) drive. In the case where the program is not stored in the ROM 203, by pre-storing the program in the storage medium and reading it into the RAM 202, the CPU 201 can perform the same operation as when the program is stored in the ROM 203.

[0047] Figure 3 1 is a functional block diagram showing an example of a terminal device TB. The terminal device TB of the embodiment is, for example, a tablet-type information terminal, and the application software (application) of the embodiment is installed. In addition, the terminal device TB does not need to be limited to a tablet-type portable terminal, and can also be a laptop computer, a smart phone, etc.

[0048] The terminal device TB mainly includes an operation unit 51, a display unit 52, a communication unit 53, an output unit 54, a memory 55, and a processor 56. Each unit (the operation unit 51, the display unit 52, the communication unit 53, the output unit 54, the memory 55, and the processor 56) is connected to a bus 57 and can transmit and receive data via the bus 57.

[0049] The operation unit 51 includes switches such as a power switch for turning the power on and off. The display unit 52 includes a liquid crystal monitor with a touch panel and displays images. The display unit 52 also has a touch panel function, so it can serve as one end of the operation unit 51.

[0050] The communication unit 53 includes a wireless unit or a wired unit for communicating with other devices, etc. In the embodiment, it is assumed that the terminal device TB is wirelessly connected to the numeric keypad 1 via Bluetooth (registered trademark). That is, the terminal device TB can transmit and receive digital data with the paired numeric keypad 1 via Bluetooth (registered trademark).

[0051] The output unit 54 includes a speaker, an earphone jack, etc., and reproduces and outputs analog sounds or musical sounds. In addition, the output unit 54 outputs a remix signal digitally synthesized by the processor 56. The remix signal can be communicated with the numeric keyboard 1 via the communication unit 53.

[0052] The processor 56 is a computing chip such as a CPU, an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (field-programmable gate array), and is responsible for controlling the terminal device TB. The processor 56 performs various processes according to the program stored in the memory 55. In addition, a DSP (Digital Signal Processor) specialized for processing digital audio signals can also be said to be a processor.

[0053] The memory 55 includes a ROM 60 and a RAM 80. The RAM 80 stores data necessary for operating the program 70 stored in the ROM 60. The RAM 80 also functions as a temporary storage area for data generated by the processor 56, MIDI data sent from the numeric keypad 1, and for developing an application program.

[0054] In the embodiment, RAM 80 stores music data 81 loaded by the user. The music data 81 is in a digital format such as mp3, m4a or wav, and in the embodiment, music containing five or more parts is envisioned. In addition, as a music, it is sufficient to contain at least two parts.

[0055] The ROM 60 stores, for example, a program 70 for causing the terminal device TB as a computer to function as the terminal device of the embodiment. The program 70 includes a sound source separation module 70a, a mixing module 70b, a compression module 70c, and an expansion module 70d.

[0056] The sound source separation module 70a separates the music data 81 into a plurality of sound source parts using a sound source separation engine using a learned model such as a DNN. Figure 3As shown, the music is separated into bass part data 82a, drum part data 82b, piano part data 82c, vocal part data 82d and other part data 82e. That is, the music includes bass part, drum part, piano part, vocal part and other parts (guitar, etc.). The obtained part data is stored in RAM 80 in wav format, for example. In addition, "part" can also be called "stem" or "track", which can be said to be the same concept.

[0057] The mixing module 70b mixes the audio signals (data) of the bass part data 82a, drums part data 82b, piano part data 82c, vocals part data 82d and other part data 82e at a ratio corresponding to the instruction message provided from the numeric keypad 1 to generate a remixed signal.

[0058] That is, after outputting the first track data in the music data or the first style data obtained by combining a plurality of track data based on the acquisition of the first indication data output from the numeric keypad 1, the terminal device TB automatically outputs the second track data in the music data or the second style data obtained by combining a plurality of track data based on the acquisition of the second indication data.

[0059] The terminal device TB acquires the track data whose sound sources have been separated in an arbitrary combination, for example, based on the acquisition of the instruction data, and outputs it to the numeric keypad 1 as a remix signal.

[0060] The compression module 70c compresses at least one of the audio signals (data) of the bass part data 82a, the drum part data 82b, the piano part data 82c, the vocal part data 82d, and the other part data 82e, and stores the data in the RAM 80. This saves the occupied area of ​​the RAM 80, and has the effect of increasing the number of songs and parts that can be stored (pool). The expansion module 70d reads the compressed data from the RAM 80 and expands it when the part data is compressed, and transmits it to the mixing module 70b.

[0061] Figure 4 2 is a diagram showing an example of information stored in the ROM 203 and the RAM 202 of the numeric keypad 1. The RAM 202 stores a plurality of MIX pattern data 22a to 22z in addition to the setting data 21.

[0062] The ROM 203 stores the preset data 22 and the program 23. The program 23 enables the digital keyboard 1 as a computer to function as the electronic musical instrument of the embodiment. The program 23 includes a control module 23a and a mode selection module 23b.

[0063] The control module 23a generates an instruction message for the terminal device TB according to the operation button (operation unit 30) as an operating element or the user's operation on the key 10, and transmits it to the terminal device TB via the bus 209. The instruction message reflects and generates any one of the MIX style data 22a to 22z stored in the RAM 202.

[0064] That is, the MIX pattern data 22a to 22z are data for individually setting the mixing pattern of the bass part data 82a, drum part data 82b, piano part data 82c, vocal part data 82d, and other part data 82e separated from the music. That is, by calling any one of the MIX pattern data 22a to 22z, the mixing ratio of each part data stored in the terminal device TB can be freely changed.

[0065] The terminal device TB can, for example, obtain the individual track data whose sound sources have been separated in any combination according to the acquisition of the instruction data. The combination pattern can include a pattern of selecting all the track data in the music data at the same time, or can be pre-set as the first pattern, the second pattern, and the third pattern. The terminal device TB only needs to be able to switch the selected pattern according to the acquisition of the instruction data.

[0066] The mode selection module 23b provides a function for the user to specify an operation mode of the keyboard 101. That is, the mode selection module 23b exclusively switches and sets a normal first mode and a second mode for controlling the terminal device TB through the keyboard 101. Here, the first mode is a normal performance mode in which musical sounds are generated by performance operations on the keys 10. The second mode generates an instruction message according to a preset operation on the keys 10.

[0067] In addition, as the instruction message, a program change or a control change which is a MIDI message can be used. Of course, it can be other MIDI signals or digital messages with a dedicated format. In addition, the trigger for generating the instruction message can be generated not only by the operation key 10, but also by operating the operation button of the operation unit 30, or stepping on / off the foot pedal FP.

[0068] <Function>

[0069] Next, the operation of the above-mentioned structure will be described.

[0070] Figure 5 1 is a flowchart showing an example of processing steps of the terminal device TB and the numeric keypad 1 according to the embodiment. Figure 5 In the embodiment, when the power of the numeric keypad 1 is turned on (step S21), it waits for BT (Bluetooth (registered trademark)) pairing from the terminal device TB (step S22).

[0071] On the other hand, when the application of the terminal device TB is started by the user's operation, the terminal device TB displays a music selection GUI (Graphical User Interface) on the display unit 52 to prompt the user to select a music. When the user selects a desired music (Open), the terminal device TB loads the music data 81 (step S11). Next, the terminal device TB sets how to switch the MIX style according to the user's operation (step S12). That is, the method of giving the instruction message for switching the MIX style is set.

[0072] The following three cases are assumed regarding the switching setting in step S12.

[0073] (In the case where there is a dedicated button on the numeric keypad 1 side (Case 1))

[0074] If there is a dedicated button in the operation unit 30 of the numeric keypad 1, the number of the mixed sound or the setting of one forward or one backward can be assigned. In this way, the performance can be enjoyed without being affected by the setting of the mixed sound.

[0075] (No dedicated button, three pedals (Case 2))

[0076] If so-called three pedals are attached as the foot pedal FP, a mixing selection function can be assigned to a sostenuto pedal that is used less frequently during performance, thereby enabling a performance with relatively little influence.

[0077] (No dedicated button, one pedal (Case 3))

[0078] In the case of one foot pedal FP, a method of cyclically switching multiple MIX patterns is considered. That is, each time the foot pedal FP is operated, the control module 23a of the numeric keypad 1 sends an instruction message for cyclically switching multiple MIX patterns preset in different settings to the terminal device TB.

[0079] (No dedicated button and no pedal (Case 4))

[0080] It is sufficient to assign the mixing selection function to the lowest and highest notes of the keyboard 101. These are keys that are used less frequently, so the influence on the performance can be minimized.

[0081] Next, the terminal device TB is paired with the numeric keypad 1 using Bluetooth (registered trademark) based on the user operation (step S13). When the pairing is completed, the information of the switching setting given in step S12 is also sent to the numeric keypad 1.

[0082] The numeric keypad 1 determines whether or not there is a need to change the internal settings based on the switch setting information obtained from the terminal device TB (step S23), and if necessary (yes), changes the settings as follows (step S24).

[0083] (Scenario 1)

[0084] The setting changes are None.

[0085] (Scenario 2)

[0086] (When using the sustain pedal during switching)

[0087] Even if it is stepped on, the sustain function is disabled.

[0088] (Scenario 3)

[0089] (When using the damper pedal during switching)

[0090] Even if stepped on, it will shut down as a damper.

[0091] (Case 4)

[0092] Mutes the sound of the assigned key.

[0093] Next, the terminal device TB separates the music data 81 loaded in step S11 into a plurality of music components, that is, each part (step S14). Figure 3 As shown in FIG. 8 , data 82 a to 82 e of the vocal part, piano part, drum part, bass part, and other parts are generated and expanded in the RAM 80 .

[0094] When the user clicks the Play button of the GUI (step S15), the terminal device TB starts audio reproduction (step S16), and mixes the various part data 82a to 82e according to the MIX pattern setting at that time to produce a remixed signal. The remixed signal is sent to the numeric keypad 1 side via BlueTooth (registered trademark) (data transmission) and output from the speaker 42. In addition, when the user starts playing (step S25), the musical sound generated by the performance is also output from the speaker 42. In addition, the Play button may not be located on the terminal device TB side, but on the numeric keypad 1 side.

[0095] During the continuous performance (step S26: No), the numeric keypad 1 waits for a switching operation (step S27). When a switching operation of the MIX pattern is performed (step S27: Yes), the terminal device TB changes the mixing of each part according to the instruction message given by the switching operation (step S17).

[0096] Figure 6A to Figure 6C ,as well as Figure 7A to Figure 7C1 is a diagram showing an example of a GUI displayed on the display unit 52 of the terminal device TB. For example, situations such as practice and ensemble (session) can be considered.

[0097] <An example of practice>

[0098] At the beginning of the performance, for example, Fig. 6A In this setting, a sound source in a mixed state in which all separated parts are simply added is generated and reproduced from the speaker 42 of the numeric keypad 1.

[0099] For example, when the user steps on the pedal FP at the end of the intro, the MIX style is switched and an instruction message is sent to the terminal device TB via Bluetooth (registered trademark). In response, the terminal device TB changes to the next state, for example, Figure 6B The GUI screen also changes in this way. Figure 6B The piano alone is shown. The user can feel the chord playing of the song by playing the chords while listening to the performance.

[0100] Furthermore, for example, when the user steps on the pedal FP during the chorus of the music, the next MIX pattern is switched, and an instruction message is sent to the terminal device TB via Bluetooth (registered trademark). Accordingly, the terminal device TB transitions to the next state, for example, Figure 6C The GUI screen also changes in this way. Figure 6C The user can feel the melody of the song by playing the melody line of the vocal music while listening to the vocal music.

[0101] By pressing the pedal again, the state of the terminal device TB returns to Fig. 6A In addition, the user can freely set the on / off of each sound source, so other states can also be set.

[0102] If the above processing can be carried out to a certain extent, the ensemble step can also be entered.

[0103] <An example of an ensemble>

[0104] At the beginning of the performance, for example Fig. 7A In this setting, a sound source in a mixed state in which all separated parts are simply added is generated and reproduced from the speaker 42 of the numeric keypad 1.

[0105] For example, when the user steps on the pedal FP at the end of the prelude, the MIX style is switched and the instruction message is sent to the terminal device TB via Bluetooth (registered trademark). In response, the terminal device TB is converted to the next state, for example, Figure 7B The GUI screen also changes in this way. Figure 7B The setting is to add the bass, drums, and vocals together and mix them, so that a sound source without chord-based sounds is generated. The user plays the Figure 6B You can enjoy playing the chords you practiced in the past with real sounds.

[0106] Furthermore, for example, when the user steps on the pedal FP during the chorus of the music, the next MIX pattern is switched, and an instruction message is sent to the terminal device TB via Bluetooth (registered trademark). Accordingly, the terminal device TB transitions to the next state, for example, Figure 7C The GUI screen will also change. Figure 7C The setting generates a sound source in which all sounds except vocals are added and mixed. The user plays the Figure 6C You can enjoy playing along with the real sound source by following the melody line of the vocal you practiced.

[0107] By pressing the pedal again, the state of the terminal device TB returns to Fig. 7A In addition, the user can freely set the on / off of each sound source, so other states can also be set.

[0108] Figure 8 : is a conceptual diagram showing an example of a processing step in an embodiment. When a sound source held by a user is selected through the song selection UI of the terminal device TB, the sound source is separated into a plurality of parts by the sound source separation engine. Then, an instruction message (e.g., a MIDI signal) is provided to the terminal device TB, for example, by pedal operation, to switch the mixing ratio of each part. The audio signal produced based on the set mixing is transmitted to the numeric keyboard 1 via Bluetooth (registered trademark), and is outputted from the speaker together with the user's performance.

[0109] As described above, in the embodiment, the music specified by the user is separated into a plurality of parts by the sound source separation engine on the terminal device TB side. On the other hand, according to the instruction message from the numeric keypad 1, the mixing ratio between the separated parts is freely switched, so that the terminal device TB produces a remixed sound source. The remixed sound source is transmitted from the terminal device TB to the numeric keypad 1 via BlueTooth (registered trademark), and is outputted acoustically together with the user's performance. Thus, through a simple operation on the electronic musical instrument side, the mixing of the parts of the sound source output from the terminal device (the terminal device may also be included in the electronic musical instrument) can be freely changed.

[0110] For example, when practicing a song, you can delete the part you don't play from the original song and change the part in the middle of the performance. When playing an ensemble, you can delete the part you play from the original song and change the part in the middle of the performance. In addition, there is no need to systematically prepare two speakers (headphones), and you can use the same speakers (or headphones, etc.) to listen to the sound source mixed after the sound source is separated and the sound source of your own performance at the same time.

[0111] For example, assuming that a person practices a popular lesson song on a keyboard instrument, as described below, different people have different preferences for practice methods, and teachers also recommend different methods.

[0112] ・People who want to practice while listening to the entire original song

[0113] ・People who want to practice while listening to the piano only

[0114] ・People who want to practice while listening to vocal music only

[0115] ・People who want to practice while listening to minus one accompaniment (removing only the piano sound from the original song)

[0116] ・People who want to practice while listening to minus one (minus the vocal sound from the original song)

[0117] In the prior art, it is difficult to switch the mixed sound of the background performance during the reproduction of the song by operating from the instrument side of the practice. According to the present invention, it is also possible to combine the remixed sound source with one's own performance and listen to it with the same speaker (or headphone).

[0118] According to this embodiment, the mixing ratio of the separated sound source can be switched by simple operation, and it is also easy to listen to it by combining it with one's own performance. Therefore, according to these embodiments, it is possible to provide a performance system, terminal device, electronic musical instrument, method and program that can arbitrarily mix and output while playing the separated parts, thereby increasing the user's willingness to practice. Furthermore, playing a musical instrument or practicing a musical instrument becomes more fun.

[0119] In addition, the present invention is not limited to the above-mentioned embodiment.

[0120] <Modification of the button operation member>

[0121] For example, when there are five patterns of mixing, the frequently used mixtures from mixing 1 to 5 are assigned to buttons 1 to 3 of numeric keypad 1 (mixing 4 is assigned to button 1, mixing 2 is assigned to button 2, etc.). Furthermore, the mixing pattern to be reproduced can be switched during the performance by pressing a button on the numeric keypad 1 side. As an example, the following pattern can be considered.

[0122] Mix 1: Except Vocals

[0123] Mix 2: Beyond Piano

[0124] Mix 3: No Drums

[0125] Mix 4: Vocals only

[0126] Mix 5: All MIX

[0127] Depending on which part the user plays (including singing), the mixing of background music can be switched during the performance or at the turning point between songs. That is, in the process of playing one song, the background music can be easily changed, so it is possible to achieve a practice that is fresh and not boring.

[0128] Furthermore, the mixing ratio of each part can be specified not only as 100% or 0%, but also as 20% for vocals or something in between if the vocals are to be slightly left out.

[0129] Furthermore, the instruction message can be generated in any manner, not limited to the foot pedal FP, as long as a predetermined MIDI signal is generated.

[0130] Furthermore, although a trigger for starting the reproduction of the sound source is given by a touch operation on the terminal device TB, the reproduction of the sound source may be started by any operation from the numeric keypad 1 side (such as a foot pedal) instead.

[0131] In addition, in practice applications, you can also set up popular functions such as changing the playback speed, reverse playback, and loop playback.

[0132] Furthermore, the electronic musical instrument is not limited to the numeric keyboard 1, and may be a string instrument or a wind instrument.

[0133] Furthermore, the present invention is not limited to a specific embodiment. For example, in the embodiment, a tablet-type portable terminal separated from the numeric keypad 1 is assumed as the terminal device TB. Not limited to this, a desktop or notebook computer may also be used. Alternatively, the numeric keypad 1 itself may also have the function of an information processing device.

[0134] In addition, the terminal device TB may be connected to the numeric keypad 1 in a wired manner via a USB cable or the like, for example.

[0135] In addition, the technical scope of the present invention includes various modifications, improvements, etc. within the scope of achieving the purpose of the present invention, which are obvious to those skilled in the art from the description of the claimed technical claims.

Claims

1. A performance system comprising an electronic musical instrument (1) and a terminal device (TB), wherein: The terminal device (TB) includes an output unit (54), The output unit (54) outputs to the electronic musical instrument first track data set by the user from a plurality of track data, and any one of the first style data obtained by the user arbitrarily setting a plurality of track data included in the plurality of track data, and in the process of outputting any one of the data to the electronic musical instrument, the output unit (54) automatically outputs to the electronic musical instrument second track data set by the user from a plurality of track data, and any one of the second style data obtained by the user arbitrarily setting a plurality of track data included in the plurality of track data, based on the acquisition of instruction data output from the electronic musical instrument (1), The plurality of track data are music data corresponding to the selected music selected by the user, which is input into a sound source separation engine using a learned model, and the sound source separation engine outputs the plurality of track data based on the input, The electronic musical instrument (1) comprises: a communication unit (216) that outputs the instruction data according to a user operation and obtains the data output by the terminal device (TB); and The sound producing section (42) produces musical sounds corresponding to the acquired data and musical sounds corresponding to the performance operation of the user.

2. The performance system according to claim 1, wherein: The terminal device includes an input unit that allows a user to input an arbitrary combination of the plurality of track data to be switched each time the instruction data is acquired.

3. The performance system according to claim 1, wherein: The electronic musical instrument comprises a pedal operating member (FP), The instruction data is outputted according to the user's operation on the pedal operation member (FP).

4. A terminal device, wherein: The terminal device comprises an output unit (54), and the output unit (54) outputs to the electronic musical instrument first track data set by a user from a plurality of track data, and any one of first style data obtained by the user arbitrarily setting a plurality of track data included in the plurality of track data, and in the process of outputting the any one of the data to the electronic musical instrument, the output unit (54) automatically outputs to the electronic musical instrument second track data set by the user from the plurality of track data, and any one of second style data obtained by the user arbitrarily setting a plurality of track data included in the plurality of track data, based on the acquisition of instruction data output from the electronic musical instrument (1), The plurality of track data are obtained by inputting music data corresponding to the selected music into a sound source separation engine using a learned model based on the music selected by the user, and the sound source separation engine outputs the plurality of track data based on the input.

5. A method for playing an electronic musical instrument, wherein: The terminal device (TB) is configured to output to the electronic musical instrument first track data set by the user from a plurality of track data, and any one of the first style data obtained by the user arbitrarily setting a plurality of track data included in the plurality of track data, and in the process of outputting the any one of the data to the electronic musical instrument, the second track data set by the user from the plurality of track data, and any one of the second style data obtained by the user arbitrarily setting a plurality of track data included in the plurality of track data are automatically output to the electronic musical instrument based on the acquisition of instruction data output from the electronic musical instrument (1), The plurality of track data are obtained by inputting music data corresponding to the selected music into a sound source separation engine using a learned model based on the music selected by the user, and the sound source separation engine outputs the plurality of track data based on the input.

6. An electronic musical instrument, wherein: have: Operation part (FP); and at least one processor (201), The at least one processor (201) is: When the first mode is set, according to the user operation on the operating member (FP), the musical sound corresponding to the user operation is instructed to be emitted, When the second mode is set, based on the user operation of the operating member (FP), an instruction is given to switch the output of style data obtained by arbitrarily combining multiple track data contained in the music data, wherein the multiple track data are track data output by a sound source separation engine using a learned model based on music data corresponding to the selected music input to the sound source separation engine.

7. A method for playing an electronic musical instrument, wherein: At least one processor (201) of the electronic musical instrument is, When the first mode is set, according to the user operation on the operating member (FP), the musical sound corresponding to the user operation is instructed to be emitted. When the second mode is set, based on the user operation of the operating member (FP), an instruction is given to switch the output of style data obtained by arbitrarily combining multiple track data contained in the music data, wherein the multiple track data are track data output by a sound source separation engine using a learned model based on music data corresponding to the selected music input to the sound source separation engine.

Images