Online session server device

Abstract

To provide an online session server device which can perform a session performance online by using karaoke performance data used in a karaoke performance.SOLUTION: An online session server device comprises: a setting unit which sets parts of a performance; a generation unit which generates audio data by parts corresponding to each of the set parts; a transmission unit which transmits minus one audio data, to a terminal device in which one part is set; an acquisition unit which acquires live performance data from the terminal device in which the one part is set; and a storage processing unit which stores live performance data of a certain music piece acquired from each of the plurality of terminal devices.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an online session server device. [Background technology]

[0002] 2. Description of the Related Art Online conference systems using cloud-based server devices have become widespread, and online events are also being held using such online conference systems.

[0003] Patent Document 1 discloses a video conference system including a plurality of terminals, a conference server, and a music server capable of distributing music to the terminals. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2010-074199 A Summary of the Invention [Problem to be solved by the invention]

[0005] As an online event using an online conference system, it is possible for multiple users to play a certain piece of music together (a so-called session performance).

[0006] Here, by applying the technology of Patent Document 1, participants in a video conference can sing karaoke online along with the music being played on their terminal devices. However, the music played using the technology of Patent Document 1 includes the performance sounds of all instrumental parts (e.g., guitar part, bass part, drum part). Therefore, for example, for a user who plays the guitar part in a session performance, the performance sound of the guitar part included in the music performance is a hindrance and makes it difficult for him or her to play.

[0007] An object of the present invention is to provide an online session server device which enables an online session performance to be carried out by utilizing karaoke performance data used in a karaoke performance. [Means for solving the problem]

[0008] One invention for achieving the above object is an online session server device communicatively connected to terminal devices used by each of a plurality of users, the online session server device having a setting unit for setting a performance part to be handled by each of the terminal devices, a generation unit for generating part-specific audio data corresponding to each of the set parts by processing karaoke performance data of a certain song used in a karaoke performance, a transmission unit for transmitting minus-one audio data, which is the part-specific audio data corresponding to another part different from the one part, to the terminal device in which the one part is set, an acquisition unit for acquiring live performance data based on a performance sound obtained by actually playing the one part in accordance with a performance based on the minus-one audio data, from the terminal device in which the one part is set, and a storage processing unit for storing the live performance data of the certain song acquired from each of the plurality of terminal devices. Other features of the present invention will become apparent from the following specification and drawings. Effect of the Invention

[0009] According to the present invention, karaoke performance data used in karaoke performance can be utilized to perform an online session performance. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing an outline of an online session system according to a first embodiment. [Diagram 2] 1 is a diagram illustrating an online conference server device according to a first embodiment. [Diagram 3]5 is a flowchart showing a process of the online conference server device according to the first embodiment. [Figure 4] 13 is a flowchart showing a process of the online conference server device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] First Embodiment The online session server device according to the present embodiment will be described with reference to FIGS.

[0012] ==Online Session System== 1, an online session system 1 according to the present embodiment includes an online conference system S1 and an online karaoke system S2. The online conference system S1 and the online karaoke system S2 are communicatively connected as described in, for example, Patent Document 1.

[0013] [Online conference system] The online conference system S1 is a system for conducting online conferences such as general Web conferences and video conferences. The online conference system S1 includes an online conference server device CS and a plurality of terminal devices. The online conference server device CS and the plurality of terminal devices are connected to each other so as to be able to communicate with each other via a network.

[0014] The online conference server device CS is a computer that manages various types of information and executes various processes. In this embodiment, the online conference server device CS utilizes a general online conference mechanism to enable online session performance (details will be described later). In other words, the online conference server device CS in this embodiment functions as an "online session server device."

[0015] The terminal device is, for example, a computer used by a user, such as a home PC, a tablet terminal, or a mobile terminal. In Fig. 1, terminal device T1 to terminal device T3 are shown. Terminal device T1 to terminal device T3 are used by users U1 to U3, respectively. The number of terminal devices is not limited to the example in Fig. 1, and at least two terminal devices are sufficient.

[0016] Each terminal device has a microphone for collecting the sound of a performance performed by a user, a speaker for emitting sound of a performance based on minus-one audio data (described in detail later), a camera for photographing the user, a display for displaying images and various information, an operating means for performing various operation inputs, etc. Note that performance in this embodiment includes playing an instrument and singing.

[0017] Furthermore, in this embodiment, a program for online sessions is pre-installed in each terminal device. By executing the program on the terminal device, users can use the online conference system to perform an online session.

[0018] [Online Karaoke System] The communication karaoke system S2 distributes songs for singing karaoke and performs karaoke. The communication karaoke system S2 includes a karaoke server KS and a plurality of karaoke devices (not shown). The karaoke server KS and the plurality of karaoke devices are connected to each other via a network so that they can communicate with each other.

[0019] The karaoke server KS is a computer that manages karaoke performance data and various other information, and executes processes such as distribution of music.

[0020] The karaoke performance data is data used in karaoke performance. The karaoke performance data is provided with song identification information for identifying each song. The song identification information is information unique to each song, such as a song ID for identifying the song. In this embodiment, the karaoke performance data includes accompaniment data and guide vocal data.

[0021] The accompaniment data is data for performing karaoke of a song. Specifically, the accompaniment data includes data for each instrument part (e.g., guitar part, bass part, drum part) used in the karaoke performance of the song. The accompaniment data is data in MIDI format.

[0022] The guide vocal data is data corresponding to a singing voice for implementing the guide vocal function. The guide vocal function is a function for assisting karaoke singing, and reproduces a pre-registered guide vocal (sung by a professional singer, etc.) together with the karaoke performance of a song. The guide vocal data is data in audio format.

[0023] ==Online conference server device== 2 is a diagram showing the configuration of the online conference server CS. The online conference server CS includes a storage unit 10, a communication unit 20, and a control unit 30. Each component is connected to a bus B via an interface (not shown).

[0024] [Memory means / Communication means] The storage means 10 is a large-capacity storage device that stores various data. The communication means 20 provides an interface for communicating with each terminal device and the online karaoke system S2.

[0025] [Control Means] The control means 30 performs various controls in the online conference server CS. The control means 30 includes a CPU and a memory (neither of which are shown in the figures). The CPU realizes various functions by executing programs stored in the memory.

[0026] In this embodiment, the control means 30 functions as a setting section 100, a generating section 200, a transmitting section 300, an acquiring section 400, and a storage processing section 500 by the CPU executing a program stored in the memory.

[0027] (Settings section) The setting section 100 sets the part of the performance to be performed for each terminal device.

[0028] A user who wishes to play in an online session inputs the part he or she will play via the operating means of his or her terminal device. The terminal device associates the input part information with its own terminal identification information and transmits it to the online conference server CS. The terminal identification information is information unique to each terminal device, such as a terminal ID for identifying the terminal device.

[0029] The setting unit 100 sets one performance part for the terminal device based on the received information. In this embodiment, the performance part includes an instrument part and a vocal part.

[0030] (Generation part) The generating unit 200 generates part-by-part audio data by processing karaoke performance data of a certain piece of music used in karaoke performance. The part-by-part audio data is data in an audio format corresponding to each of the set parts.

[0031] The audio data for each part can be generated using known rendering techniques such as DAW (for example, JP 2017-097289 A).

[0032] One of the users who wish to play in an online session selects a piece of music that he or she wishes to play in the session through the operation means of his or her terminal device. The terminal device transmits the music identification information of the selected piece of music to the online conference server device CS.

[0033] The generating unit 200 transmits the received music identification information to the karaoke server KS. The karaoke server KS reads out karaoke performance data based on the received music identification information from a storage means (not shown), and transmits the data to the online conference server CS.

[0034] The generating unit 200 generates part-by-part audio data from the received karaoke performance data by rendering the data for each part set by the setting unit 100.

[0035] For example, it is assumed that the selected piece of music X is made up of a vocal part, a guitar part, a bass part, and a drum part, and that the setting unit 100 has set the vocal part, guitar part, bass part, and drum part for each terminal device.

[0036] In this case, the generating unit 200 generates part-specific audio data for the guitar part, bass part, and drum part by rendering the guitar part, bass part, and drum part, respectively, from the accompaniment data included in the karaoke performance data of the song X. Specifically, the generating unit 200 converts the information of the set part into tone color information in the MIDI format data, and renders the accompaniment data of one or more parts corresponding to the tone color information. For example, in rendering the guitar part, musical tone signals for each of the six strings can be generated using the accompaniment data for the six parts.

[0037] As described above, the guide vocal data in this embodiment is in audio format, so the generator 200 uses the guide vocal data included in the karaoke performance data of the song X as part-by-part audio data for the singing parts.

[0038] In addition, by using a publicly known voice synthesis engine or guide melody sound source, data equivalent to guide vocal data can be constructed from accompaniment data. In this case, the generating unit 200 can generate part-specific audio data for the singing parts by rendering the constructed data. In this way, when data equivalent to guide vocal data is constructed from accompaniment data, the karaoke performance data does not need to include the guide vocal data.

[0039] Depending on the number of users participating in the session performance and the types of parts each user will play, all of the parts that make up a piece of music may not necessarily be set.

[0040] When there is a part that has not been set by the setting unit 100 among the multiple parts that make up a piece of music, the generating unit 200 can generate audio data for each part and accompaniment audio data. The accompaniment audio data is data in an audio format that corresponds to all parts other than the part that has been set.

[0041] The accompaniment audio data can be generated using a known rendering technique, as in the case of the part-by-part audio data. The generating unit 200 generates the accompaniment audio data by rendering the data other than the part set by the setting unit 100 from the received karaoke performance data (details will be described later).

[0042] (Transmitter) The transmitting unit 300 transmits the minus-one audio data to the terminal device to which the one part is set. The minus-one audio data is part-by-part audio data corresponding to another part different from the one part.

[0043] For example, in the session performance of the above-mentioned piece of music X, it is assumed that the setting unit 100 has set a singing part for one terminal device.

[0044] In this case, the transmitting unit 300 transmits part-specific audio data corresponding to parts (guitar part, bass part, and drum part) different from the vocal part as minus-one audio data to the one terminal device to which the vocal part is set. The transmitting unit 300 can identify the terminal device to which the minus-one audio data is transmitted by referring to terminal identification information linked to the part information. In addition, when the minus-one audio data includes multiple part-specific audio data as in this example, the transmitting unit 300 mixes each data and transmits it to the one terminal device.

[0045] Similarly, the transmitting unit 300 transmits part-specific audio data corresponding to other parts (vocal part, bass part, and drum part) different from the guitar part as minus-one audio data to one terminal device in which a guitar part is set. Also, the transmitting unit 300 transmits part-specific audio data corresponding to other parts (vocal part, guitar part, and drum part) different from the bass part as minus-one audio data to one terminal device in which a bass part is set. Also, the transmitting unit 300 transmits part-specific audio data corresponding to other parts (vocal part, guitar part, and bass part) different from the drum part as minus-one audio data to one terminal device in which a drum part is set.

[0046] The transmitting unit 300 simultaneously transmits the minus-one audio data to each terminal device.

[0047] As described above, the generating unit 200 may generate audio data for each part and accompaniment audio data. In this case, the transmitting unit 300 transmits minus-one audio data including accompaniment audio data to a terminal device to which one part is set (details will be described later).

[0048] The transmitting unit 300 may transmit count-in sound data together with the minus-one audio data to a terminal device in which one part is set. The count-in sound is a sound that indicates the timing to start an actual performance in accordance with the performance based on the minus-one audio data. The count-in sound data may be generated by the generating unit 200, for example, based on tempo information included in the karaoke performance data. By referring to the count-in sound emitted from the speaker of the terminal device, the user can easily grasp the timing to start the performance even when playing a part without an introduction.

[0049] (Acquisition Department) The acquiring unit 400 acquires live performance data from a terminal device in which one part is set. The live performance data is data based on a performance sound obtained by actually performing one part in accordance with a performance based on the minus-one audio data.

[0050] For example, in the session performance of the above-mentioned song X, the transmitting unit 300 transmits minus-one audio data including part-specific audio data corresponding to the other parts (guitar part, bass part, and drum part) to a terminal device to which the vocal part is set.

[0051] In this case, the one terminal device starts playing the received part-by-part audio data. The user of the one terminal device sings the song X while listening to the performance sound based on the part-by-part audio data. The one terminal device links live performance data based on the user's singing voice with its own terminal identification information and transmits it to the online conference server device CS. The acquisition unit 400 acquires the live singing performance data. Note that in this embodiment, the singing voice is an example of a "performance sound".

[0052] Similarly, the transmitting unit 300 transmits minus-one audio data including part-specific audio data corresponding to other parts (bass part, drum part, and vocal part) to the one terminal device in which the guitar part is set. In this case, the one terminal device starts playing the received part-specific audio data. The user of the one terminal device plays the guitar of the musical piece X while listening to the performance sound based on the part-specific audio data. The one terminal device transmits live performance data based on the guitar performance sound to the online conference server device CS by linking it with its own terminal identification information. The acquiring unit 400 acquires live performance data of the guitar performance. In addition, the transmitting unit 300 transmits minus-one audio data including part-specific audio data corresponding to other parts (guitar part, drum part, and vocal part) to the one terminal device in which the bass part is set. In this case, the one terminal device starts playing the received part-specific audio data. The user of the one terminal device plays the bass of the musical piece X while listening to the performance sound based on the part-specific audio data. The first terminal device transmits live performance data based on the bass performance sound to the online conference server device CS by linking the data with its own terminal identification information. The acquisition unit 400 acquires the live performance data of the bass performance. The transmission unit 300 transmits minus-one audio data including part-specific audio data corresponding to the other parts (guitar part, bass part, and vocal part) to the first terminal device in which the drum part is set. In this case, the first terminal device starts playing the received part-specific audio data. The user of the first terminal device plays the drums of the music piece X while listening to the performance sound based on the part-specific audio data. The first terminal device transmits live performance data based on the drum performance sound to the online conference server device CS by linking the data with its own terminal identification information. The acquisition unit 400 acquires the live performance data of the drum performance.

[0053] As described above, even if the transmitting unit 300 transmits the part-by-part audio data to each terminal device simultaneously, a delay may occur in the timing at which each terminal device receives the part-by-part audio data depending on the communication state. Similarly, a delay may occur in the timing at which the acquiring unit 400 acquires the live performance data transmitted from each terminal device.

[0054] (Memory Processing Unit) The storage processing unit 500 stores live performance data of a certain piece of music acquired from each of the plurality of terminal devices in the storage means 10.

[0055] For example, for a piece of music X, it is assumed that the acquiring unit 400 has acquired live performance data of vocals, live performance data of guitar, live performance data of bass, and live performance data of drums.

[0056] In this case, the storage processing unit 500 stores each live performance data in the storage means 10 every time it acquires the data.

[0057] After each user finishes playing, one user can instruct the distribution of the session performance via his / her own terminal device. Based on the instruction, the online conference server device CS plays back a session performance in which all live performance data stored in the storage means 10 are synchronized, and distributes it to each terminal device. The online conference server device CS can specify the terminal device to distribute the session performance by referring to the terminal identification information linked to the part. Furthermore, the online conference server device CS may automatically distribute the session performance when the transmitting unit 300 receives live performance data from all terminal devices to which the minus-one audio data was transmitted.

[0058] As described above, the generating unit 200 may generate audio data for each part and accompaniment audio data. In this case, the storage processing unit 500 stores live performance data of a certain piece of music and accompaniment audio data in the storage means 10 (details will be described later).

[0059] ==Operation of the online conference server device== Next, a specific example of the operation of the online conference server device CS in this embodiment will be described with reference to Fig. 3. Fig. 3 is a flowchart showing an example of the operation of the online conference server device CS. In this example, it is assumed that users U1 to U3 shown in Fig. 1 will perform an online session performance. It is also assumed that song Y consisting of a vocal part, a guitar part, a bass part, a drum part, and a keyboard part is selected as the song for the session performance.

[0060] Each user inputs the part he or she will play via the operation means of the terminal device. The terminal device associates the input part information with the terminal identification information of the user and transmits the information to the online conference server CS.

[0061] The setting unit 100 of the online conference server CS sets one part for the terminal devices T1 to T3 (one part is set for each terminal device; step 10).

[0062] For example, the user U1 inputs the bass part as the part to be played via the operation means of the terminal device T1. The terminal device T1 associates the input information of the bass part with the terminal identification information ID***T1 of the terminal device T1 and transmits it to the online conference server device CS. The setting unit 100 sets the bass part for the terminal device T1 based on the received information.

[0063] Similarly, user U2 inputs a guitar part as the part he / she is responsible for playing, via the operation means of terminal device T2. Terminal device T2 links the input information of the guitar part with terminal identification information ID***T2 of terminal device T2 and transmits it to online conference server device CS. The setting unit 100 sets the guitar part for terminal device T2 based on the received information. User U3 inputs a singing part as the part he / she is responsible for playing, via the operation means of terminal device T3. Terminal device T3 links the input information of the singing part with terminal identification information ID***T3 of terminal device T3 and transmits it to online conference server device CS. The setting unit 100 sets the singing part for terminal device T3 based on the received information.

[0064] The generation unit 200 renders the karaoke performance data of the selected song Y, and generates part-specific audio data PA1 corresponding to the guitar part, part-specific audio data PA2 corresponding to the bass part, and part-specific audio data PA3 corresponding to the singing part, which were set in step 10 (generating part-specific audio data corresponding to the set parts; step 11).

[0065] In addition, for the drum part and keyboard part that constitute the song Y and that have not been set by the setting unit 100, the generation unit 200 renders the karaoke performance data of the song Y and generates accompaniment audio data AC corresponding to the drum part and keyboard part (generating accompaniment audio data; step 12).

[0066] The transmitting unit 300 transmits minus-one audio data including part-specific audio data corresponding to parts other than the set part, and accompaniment audio data corresponding to all parts other than the set part, to each of the terminal devices T1 to T3 (transmitting minus-one data to each terminal device; step 13).

[0067] For example, the transmitting unit 300 refers to the terminal identification information ID***T1 and transmits the minus-one audio data MA1 including the part audio data PA1 and PA3 and the accompaniment audio data AC to the terminal device T1 in which the bass part is set.

[0068] Similarly, the transmitting unit 300 refers to the terminal identification information ID***T2 and transmits minus-one audio data MA2 including part audio data PA2 and PA3 and accompaniment audio data AC to the terminal device T2 in which a guitar part is set. Also, the transmitting unit 300 refers to the terminal identification information ID***T3 and transmits minus-one audio data MA3 including part audio data PA1 and PA2 and accompaniment audio data AC to the terminal device T3 in which a singing part is set. In this example, the transmitting unit 300 starts transmitting all of the minus-one audio data MA1 to MA3 at the same timing t (t=0).

[0069] The acquiring unit 400 acquires live musical performance data from the terminal devices T1 to T3 (acquires live musical performance data from each terminal device; step 14).

[0070] For example, the terminal device T1 starts playing the minus-one audio data MA1 received from the online conference server device CS at timing Dt1. The user U1 plays the bass of the musical piece Y while listening to the performance sound based on the minus-one audio data MA1. The terminal device T1 transmits the live performance data LM1 based on the bass performance sound to the online conference server device CS, linking it with the terminal identification information ID***T1. The acquisition unit 400 starts acquiring the live performance data LM1 at timing Dt2.

[0071] Similarly, the terminal device T2 starts playing the minus-one audio data MA2 received from the online conference server device CS at timing Dt3. The user U2 plays the music piece Y on the guitar while listening to the performance sound based on the minus-one audio data MA2. The terminal device T2 transmits the live performance data LM2 based on the guitar performance sound to the online conference server device CS by linking it with the terminal identification information ID***T2. The acquisition unit 400 starts acquiring the live performance data LM2 at timing Dt4. The terminal device T3 starts playing the minus-one audio data MA3 received from the online conference server device CS at timing Dt5. The user U3 sings the music piece Y while listening to the performance sound based on the minus-one audio data MA3. The terminal device T3 transmits the live performance data LM3 based on the singing voice to the online conference server device CS by linking it with the terminal identification information ID***T3. The acquisition unit 400 starts acquiring the live performance data LM3 at timing Dt6.

[0072] The timings Dt1 to Dt6 indicate points in time later than the timing t. The timings Dt1 to Dt6 vary depending on the communication environment between the online conference server CS and each terminal device.

[0073] The storage processing unit 500 stores the live performance data of the musical piece Y acquired in step 14 and the accompaniment audio data generated in step 12 in the storage means 10 (store the live performance data and the accompaniment audio data; step 15).

[0074] For example, it is assumed that the acquiring unit 400 acquires live performance data LM1 of a bass performance of a piece of music Y, live performance data LM2 of a guitar performance of a piece of music Y, and live performance data LM3 of a vocal performance of a piece of music Y.

[0075] In this case, the storage processing unit 500 stores the live performance data LM1 to LM3 and the accompaniment audio data AC in the storage means 10.

[0076] After each user has finished playing, the online conference server device CS refers to the terminal identification information associated with the part and distributes the session performance based on the live performance data and accompaniment audio data to the terminal devices T1 to T3 (distributing the session performance to each terminal device; step 16).

[0077] For example, the online conference server CS plays back a session performance in which live performance data LM1 to LM3 and accompaniment audio data AC are synchronized, and distributes the session performance to terminals T1 to T3 based on terminal identification information ID***T1 to ID***T3.

[0078] As is clear from the above, the online conference server CS according to the present embodiment is connected to terminal devices used by each of a plurality of users so as to be able to communicate with them. The online conference server CS includes a setting unit 100 for setting a performance part for each terminal device, a generating unit 200 for generating part-specific audio data corresponding to each set part by processing karaoke performance data of a certain piece of music used in karaoke performance, a transmitting unit 300 for transmitting minus-one audio data, which is part-specific audio data corresponding to a different part from the one part, to the terminal device in which the one part is set, an acquiring unit 400 for acquiring live performance data based on a performance sound obtained by actually playing the one part in accordance with the performance based on the minus-one audio data from the terminal device in which the one part is set, and a storage processing unit 500 for storing the live performance data of the certain piece of music acquired from each of the plurality of terminal devices.

[0079] According to such an online conference server device CS, it is possible to generate part-specific audio data using karaoke performance data and transmit minus-one audio data to a terminal device in which a part is set. The minus-one audio data transmitted to a certain terminal device includes part-specific audio data corresponding to a part other than the part set in the certain terminal device. In other words, the performance sound based on the minus-one audio data transmitted to a certain terminal device does not include the performance sound of the part set in the certain terminal device. Therefore, it becomes easier for a user to perform his / her own part in accordance with the performance sound. In other words, according to the online conference server device CS of this embodiment, it is possible to perform an online session performance by using the karaoke performance data used in the karaoke performance.

[0080] In the online conference server device CS according to the present embodiment, when there is a part that has not been set by the setting unit 100 among the multiple parts that make up a certain piece of music, the generating unit 200 generates part-specific audio data and accompaniment audio data corresponding to all parts other than the set part. The transmitting unit 300 transmits minus-one audio data including accompaniment audio data to a terminal device to which a part has been set. The storage processing unit 500 stores live performance data of a certain piece of music and accompaniment audio data. According to such an online conference server device CS, even if there is a performance part that has not been set among the multiple parts that make up a certain piece of music, it is possible to provide minus-one audio data suitable for the part that the user is playing.

[0081] <Second embodiment> Next, an online session server device according to a second embodiment will be described with reference to Fig. 4. In this embodiment, an example will be described in which some of the part-by-part audio data included in the minus-one audio data is replaced with live performance data. Detailed description of the same configuration as in the first embodiment will be omitted.

[0082] (Transmitter) When the transmitting unit 300 in this embodiment transmits minus-one audio data to a terminal device in which one part is set, if live performance data has been acquired from another terminal device, the transmitting unit 300 transmits minus-one audio data in which the part-specific audio data corresponding to the other part has been replaced with the live performance data acquired from the terminal device in which the other part is set.

[0083] For example, the transmitting unit 300 starts transmitting part-specific audio data corresponding to parts (guitar part, bass part, and drum part) different from the singing part to a certain terminal device in which the singing part of the musical piece X of the first embodiment is set, as minus-one audio data. The certain terminal device starts playing the received part-specific audio data. A user of the certain terminal device sings the musical piece X while listening to the performance sound based on the part-specific audio data. The certain terminal device transmits live performance data based on the singing voice to the online conference server device CS, linking it with its own terminal identification information. The acquiring unit 400 starts acquiring the live performance data of the singing.

[0084] When the acquisition unit 400 starts acquiring live vocal performance data, the transmission unit 300 transmits part-specific audio data corresponding to other parts (vocal part, bass part, and drum part) different from the guitar part as minus-one audio data to another terminal device in which a guitar part is set.

[0085] Here, the online conference server CS has already acquired live singing performance data from a certain terminal device. In this case, the transmitting unit 300 transmits minus-one audio data to another terminal device in which the part-by-part audio data corresponding to the singing part has been replaced with the acquired live singing performance data. In this example, the other terminal device corresponds to a "terminal device with one part set", the certain terminal device corresponds to an "other terminal device", and the singing part corresponds to an "other part".

[0086] ==Operation of the online conference server device== Next, a specific example of the operation of the online conference server device CS in this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of the operation of the online conference server device CS. In this example, it is assumed that users U1 to U3 shown in Fig. 1 will perform an online session performance. It is also assumed that song Y consisting of a vocal part, a guitar part, a bass part, a drum part, and a keyboard part is selected as the song for the session performance.

[0087] The setting unit 100 of the online conference server device CS sets one part for each of the terminal devices T1 to T3 in order (one part for each of the terminal devices in order; step 20). For example, assume that the parts to be performed are input in the order of user U1, user U2, and user U3. In this case, the setting unit 100 sets the parts in the order of user U1 (bass part), user U2 (guitar part), and user U3 (vocal part). The method of setting the parts for each of the terminal devices is the same as step 10 in the first embodiment.

[0088] Steps 21 and 22 are similar to steps 11 and 12 in the first embodiment.

[0089] The transmitting unit 300 starts transmitting minus-one audio data including part-specific audio data corresponding to parts other than the set part, and accompaniment audio data corresponding to all parts other than the set part, to the terminal device T1 used by the user U1 who initially set the part (start transmitting minus-one data to the terminal device T1; step 23).

[0090] Specifically, the transmitting unit 300 refers to the terminal identification information ID***T1 and starts transmitting, at timing t (t=0), the part-specific audio data PA1 and PA3, and the minus-one audio data MA1 including the accompaniment audio data AC to the terminal device T1 in which the bass part is set.

[0091] The acquiring unit 400 starts acquiring live musical performance data from the terminal device T1 (start acquiring live musical performance data from the terminal device T1; step 24).

[0092] The terminal device T1 starts playing the minus-one audio data MA1 received from the online conference server device CS at timing Dt7. The user U1 plays the bass of the musical piece Y while listening to the performance sound based on the minus-one audio data MA1. The terminal device T1 transmits the live performance data LM1 based on the bass performance sound to the online conference server device CS, linking it with the terminal identification information ID***T1. The acquisition unit 400 starts acquiring the live performance data LM1 at timing Dt8.

[0093] The transmitting unit 300 starts transmitting minus-one audio data to the terminal device T2 used by the user U2 who has set the part next to that of the user U1.

[0094] At this point, the online conference server CS acquires live performance data obtained by actually playing the bass part from the terminal T1. Therefore, the transmitting unit 300 starts transmitting minus-one audio data in which the part-by-part audio data corresponding to the bass part has been replaced with the acquired live performance data of the bass part (starts transmitting minus-one data in which some part-by-part audio data has been replaced with live performance data to the terminal T2; step 25).

[0095] Specifically, the transmitting unit 300 refers to the terminal identification information ID***T2 and starts transmitting, at timing Dt8, to the terminal device T2 in which the guitar part is set, the part-specific audio data PA3, the live performance data LM1 that replaces the part-specific audio data PA1, and the minus-one audio data MA2 including the accompaniment audio data AC.

[0096] The acquiring section 400 starts acquiring live musical performance data from the terminal device T2 (start acquiring live musical performance data from the terminal device T2; step 26).

[0097] The terminal device T2 starts playing the minus-one audio data MA2 received from the online conference server device CS at timing Dt9. The user U2 plays the music piece Y on the guitar while listening to the performance sound based on the minus-one audio data MA2. The terminal device T2 transmits the live performance data LM2 based on the guitar performance sound to the online conference server device CS, linking it with the terminal identification information ID***T2. The acquisition unit 400 starts acquiring the live performance data LM2 at timing Dt10.

[0098] The transmitting unit 300 starts transmitting minus-one audio data to the terminal device T3 used by the user U3 who has set the part next to that of the user U2.

[0099] At this point, the online conference server CS acquires live performance data obtained by actually playing the bass part from the terminal device T1, and acquires live performance data obtained by actually playing the guitar part from the terminal device T2. Therefore, the transmitting unit 300 starts transmitting minus-one audio data in which the part-specific audio data corresponding to the bass part and the part-specific audio data corresponding to the guitar part have been replaced with the acquired live performance data of the bass part and the live performance data of the guitar part (starts transmitting minus-one data in which some part-specific audio data have been replaced with live performance data to the terminal device T3; step 27).

[0100] Specifically, the transmitting unit 300 refers to the terminal identification information ID***T3 and starts transmitting, at timing Dt10, to the terminal device T3 in which the singing part is set, the live performance data LM1 replaced with the part-specific audio data PA1, the live performance data LM2 replaced with the part-specific audio data PA2, and the minus-one audio data MA3 including the accompaniment audio data AC.

[0101] The acquiring unit 400 starts acquiring live musical performance data from the terminal device T3 (start acquiring live musical performance data from the terminal device T3; step 28).

[0102] The terminal device T3 starts playing the minus-one audio data MA3 received from the online conference server device CS at timing Dt11. The user U3 sings the song Y while listening to the performance sound based on the minus-one audio data MA3. The terminal device T3 transmits live performance data LM3 based on the singing voice to the online conference server device CS, linking it with the terminal identification information ID***T3. The acquisition unit 400 starts acquiring the live performance data LM3 at timing Dt12.

[0103] Note that timings Dt7 to Dt12 indicate time points later than timing t. Furthermore, timing Dt12 indicates a time point later than timing Dt11, which indicates a time point later than timing Dt10, which indicates a time point later than timing Dt9, which indicates a time point later than timing Dt8, which indicates a time point later than timing Dt7. Furthermore, timings Dt7 to Dt12 change depending on the communication environment between the online conference server CS and each terminal device.

[0104] The storage processing unit 500 stores the live performance data of the musical piece Y, which began to be acquired in steps 24, 26, and 28, and the accompaniment audio data generated in step 22 in the storage means 10 (store the live performance data and accompaniment audio data; step 29).

[0105] After each user has finished playing, the online conference server device CS refers to the terminal identification information associated with the part and distributes the session performance based on the live performance data and accompaniment audio data to the terminal devices T1 to T3 (distributing the session performance to each terminal device; step 30).

[0106] As is clear from the above, when the transmitting unit 300 in the online conference server device CS in this embodiment transmits minus-one audio data to a terminal device in which one part is set, if live performance data has been acquired from another terminal device, the transmitting unit 300 transmits minus-one audio data in which the part-specific audio data corresponding to the other part has been replaced with the live performance data acquired from the terminal device in which the other part is set.

[0107] According to such an online conference server CS, minus-one audio data including live performance data can be transmitted to some terminal devices. Therefore, when a user plays his / her part along with the performance sound based on the minus-one audio data, the user can feel as if he / she is playing an actual session.

[0108] <Modification of the second embodiment> The transmitting unit 300 can transmit minus-one audio data to the terminal device of a user who has selected a certain song, in which all part-by-part audio data corresponding to other parts has been replaced with live performance data acquired from the terminal device in which the other parts are set.

[0109] For example, assume that user U1 selects song Y in the example of the second embodiment.

[0110] In this case, the transmission unit 300 first transmits minus-one audio data to terminal devices T2 and T3 used by users U2 and U3 who are different from the user U1.

[0111] For example, the transmitting unit 300 refers to the terminal identification information ID***T2 and starts transmitting the part audio data PA1 and PA3 and the minus-one audio data MA2 including the accompaniment audio data AC to the terminal device T2 in which the guitar part is set.

[0112] The terminal device T2 starts playing the minus-one audio data MA2 received from the online conference server device CS. The user U2 plays the music piece Y on the guitar while listening to the performance sound based on the minus-one audio data MA2. The terminal device T2 transmits live performance data LM2 based on the guitar performance sound to the online conference server device CS, linking the live performance data LM2 with the terminal identification information ID***T2. The acquisition unit 400 starts acquiring the live performance data LM2.

[0113] The transmitting unit 300 refers to the terminal identification information ID***T3 and begins transmitting the part-specific audio data PA1, the live performance data LM2 replaced with the part-specific audio data PA2, and the minus-one audio data MA3 including the accompaniment audio data AC to the terminal device T3 in which the singing part has been set.

[0114] The terminal device T3 starts playing the minus-one audio data MA3 received from the online conference server device CS. The user U3 sings the song Y while listening to the performance sound based on the minus-one audio data MA3. The terminal device T3 transmits live performance data LM3 based on the singing voice to the online conference server device CS by linking it with the terminal identification information ID***T3. The acquisition unit 400 starts acquiring the live performance data LM3. Note that, in this example, the minus-one audio data is transmitted in the order of the terminal device T2 and the terminal device T3, but this is not limited to this. The order in which the minus-one audio data is transmitted is not particularly limited as long as the terminal device of the user who selected the song is transmitted last.

[0115] The transmitting unit 300 begins transmitting the live performance data LM2 replaced with the part-specific audio data PA2, the live performance data LM3 replaced with the part-specific audio data PA3, and the minus-one audio data MA1 including the accompaniment audio data AC to the terminal device T1 in which the bass part has been set (the terminal device T1 used by the user who selected the song Y).

[0116] The terminal device T1 starts playing the minus-one audio data MA1 received from the online conference server device CS. The user U1 plays the bass of the musical piece Y while listening to the performance sound based on the minus-one audio data MA1. The terminal device T1 transmits the live performance data LM1 based on the bass performance sound to the online conference server device CS, linking it with the terminal identification information ID***T1. The acquisition unit 400 starts acquiring the live performance data LM1.

[0117] It is highly likely that the user who selected the song is keen to play the song in a session. On the other hand, such an online conference server CS can transmit minus-one audio data including all live performance data obtained based on the actual performances of other users to the terminal device used by the user who selected the song for the session performance. Therefore, the user who selected the song can get a stronger sense of being in an actual session performance.

[0118] <Other> The acquiring unit 400 can acquire live performance data of a certain piece of music and video data of a user using the terminal device from the terminal device.

[0119] The video data is obtained, for example, by photographing a user performing with a camera included in the terminal device. The terminal device transmits the video data together with the live performance data to the online conference server device CS. The acquisition unit 400 acquires the live performance data and the video data. The storage processing unit 500 stores the live performance data and the video data in the storage means 10. The online conference server device CS plays back a video based on the stored video data together with a session performance synchronized with the live performance data, and distributes it to each terminal device.

[0120] In the above embodiment, an example has been described in which all users U1 to U3 who use the online conference system S1 perform some kind of performance. On the other hand, there may be a user who does not perform any performance. Therefore, the online conference server device CS can distribute all part-by-part audio data (or all part-by-part audio data and accompaniment audio data) generated by the generation unit 200 to the terminal device used by such a user. By distributing such data, the user who does not perform will not get bored even when other users are in the middle of a session performance.

[0121] Also, the karaoke server KS may have some of the configurations described in the above embodiments. For example, in the example of the first embodiment, the online conference server CS may have the setting unit 100, the transmitting unit 300, the acquiring unit 400, and the storage processing unit 500, and the karaoke server KS may have the generating unit 200. In these cases, the online conference server CS and the karaoke server KS function as the "online session server".

[0122] Furthermore, a server device having the functions of the online conference server device CS and the karaoke server device KS described in the above embodiment may be provided. The terminal devices are configured to be able to directly communicate with the server device. In this case, the server device functions as an "online session server device."

[0123] The above embodiment is presented as an example, and does not limit the scope of the invention. The above configurations can be implemented in appropriate combinations, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. The above embodiment and its modifications are included in the scope of the invention and its equivalents described in the claims, as well as the scope and gist of the invention. [Explanation of symbols]

[0124] 1. Online Session System 100 Setting section 200 Generation part 300 Transmitter 400 Acquisition Department 500 Memory Processing Unit CS Online Conference Server Device KS Karaoke Server Equipment S1 Online Conference System S2 Online Karaoke System T1~T3 terminal equipment

Claims

1. An online conference system including a terminal device used by each of a plurality of users and an online session server device communicably connected to the terminal device, The online session server device: A setting unit that sets one part to be handled by each of the terminal devices; a generating unit for generating part-by-part audio data corresponding to each of the set parts by processing karaoke performance data of a certain piece of music used in a karaoke performance; a transmitting unit for transmitting minus-one audio data, which is the part-by-part audio data corresponding to another part different from the one part, to the terminal device to which the one part is set; an acquisition unit that acquires live performance data based on a performance sound obtained by actually performing the one part in accordance with a performance based on the minus-one audio data from a terminal device to which the one part is set; a storage processing unit that stores live performance data of the certain piece of music acquired from each of the plurality of terminal devices; having The terminal device Transmitting the one part to the online session server device; performing a performance based on the received minus-one audio data; transmitting live performance data based on a performance sound obtained by actually performing the one part in sync with the performance to the online session server device; An online conference system comprising:

2. when there is a part that has not been set by the setting unit among the plurality of parts that constitute the certain piece of music, the generating unit generates the part-specific audio data and accompaniment audio data corresponding to all parts other than the set part, the transmitting unit transmits the minus-one audio data including the accompaniment audio data to the terminal device to which one part is set; 2. The online conference system according to claim 1, wherein the storage processing unit stores live performance data of the certain piece of music and the accompaniment audio data.

3. The online conference system of claim 1 or 2, characterized in that when transmitting the minus-one audio data to the terminal device to which one part is set, if live performance data obtained by actually playing another part has been acquired from another terminal device, the transmitting unit transmits the minus-one audio data in which the part-specific audio data corresponding to the other part has been replaced with the acquired live performance data.

4. The online conference system described in claim 3, characterized in that the transmission unit transmits to the terminal device of the user who selected the certain song the minus-one audio data in which all of the part-specific audio data corresponding to the other parts have been replaced with live performance data obtained from the terminal device in which the other parts are set.

Images