Karaoke program and karaoke system

Abstract

To provide a program for karaoke and a karaoke system that can reduce unplesantness on listener side by suppressing deviation of voice from music being reproduced on listening side when karaoke is performed at a remote location.SOLUTION: A program for karaoke executes: connection processing for connecting to a management server; determination processing for determining whether a terminal is a singer terminal 6a for a song music has been reserved or a listener terminal 6b for which a song has not been reserved; first reception processing for receiving song information from a management server device; second reception processing for receiving singing voice information transmitted from the singer terminal in a streaming manner if the terminal is determined to be a listener terminal in the determination processing; determination processing for determining an amount of delay of the second reception processing; and reproduction processing for delaying the reproduction of music sound with respect to the streaming-reproduced singing voice on the basis of a reproduction timing of the song information determined on the basis of the amount of delay of the second reception processing.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a karaoke program executable on a terminal device and a karaoke system. [Background technology]

[0002] Traditionally, karaoke, where people enjoy singing along to musical accompaniment, has been performed in social settings. Karaoke is usually enjoyed by gathering at karaoke booths or snack bars, etc., and eating and drinking. However, in the event of an infectious disease outbreak, it may be difficult to gather and enjoy karaoke. For this reason, some karaoke users enjoy karaoke by using online web conferences. In karaoke using web conferences, users use their personal computers to connect to a web conference server that hosts a web conference, play downloaded songs, and enjoy singing.

[0003] When enjoying karaoke in a remote location, such as karaoke using a web conference, delays in transmission lines can be a problem. Patent Document 1 discloses a karaoke system that deals with such transmission delays. Specifically, the system measures the transmission delay time of the received singing voice between the karaoke performance device of the remote singing partner, and generates a substitute playback voice to be emitted from a speaker according to the measured transmission delay time of the singing voice. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-206575 A Summary of the Invention [Problem to be solved by the invention]

[0005] Patent Document 1 discloses a method to reduce the discomfort caused by delays when singing at multiple locations. However, no consideration is given to listeners who are not supposed to sing, and so listeners hear substitute audio at terminals with large delays, unable to hear the actual singer's voice, and are unable to enjoy karaoke in the same karaoke space.

[0006] The present invention has been developed in consideration of such circumstances, and provides a mechanism that suppresses the mismatch between the music and the singing voice, even when communication is performed between multiple terminal devices, and does not cause discomfort to listeners. [Means for solving the problem]

[0007] For this reason, the karaoke program according to the present invention employs the following configuration. A karaoke program executable on a plurality of terminal devices connected to each other via a network, the program comprising: A connection process for connecting to a server device; A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal; A first reception process for receiving music information from the server device; The above determination process and it is a listener terminal. discrimination If so, A second receiving process of receiving singing voice information transmitted from the singer terminal in a streaming manner; A determination process for determining an amount of delay in the second reception process; A reproduction process is executed to delay the reproduction of the music information with respect to the streaming-reproduced singing voice, based on the reproduction timing of the music information determined based on the delay amount of the second reception process.

[0008] Furthermore, in the karaoke program according to the present invention, The playback timing used in the playback process is determined based on the delay amounts determined at the plurality of listener terminals.

[0009] Furthermore, in the karaoke program according to the present invention, The playback timing used in the playback process is determined based on the maximum delay amount among the delay amounts determined at the multiple listener terminals.

[0010] Furthermore, in the karaoke program according to the present invention, The determination process is executed before the start of reproduction of the music information or multiple times during the execution of the reproduction process.

[0011] Furthermore, in the karaoke program according to the present invention, In the reproduction process, if the reproduction timing has changed from the previous time, the reproduction speed of the music information is changed.

[0012] Furthermore, in the karaoke program according to the present invention, The reproduction of the music information in the reproduction process is delayed during a non-singing section.

[0013] Further, the karaoke program according to the present invention comprises: A karaoke program executable on a plurality of terminal devices connected via a network, A connection process for connecting to a server device; A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal; A first reception process for receiving music information from the server device; If the discrimination means judges that the terminal is a listener terminal, A second receiving process of receiving singing voice information transmitted from the singer terminal in a streaming manner; a determination process for determining a first delay amount of the first reception process and a second delay amount of the second reception process; A playback process is executed to delay the playback of the music information relative to the streaming-played singing voice, based on a playback timing of the music information determined based on a first delay amount of the first receiving process and a second delay amount of the second receiving process.

[0014] Further, the karaoke system according to the present invention comprises: A karaoke system having a plurality of terminal devices communicatively connected via a network, The terminal device A connection process for connecting to a server device; A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal; A first reception process for receiving music information from the server device; The above determination process and it is a listener terminal. discrimination If so, A second receiving process of receiving singing voice information transmitted from a singer terminal in a streaming manner; A determination process for determining an amount of delay in the second reception process; A reproduction process is executed to delay the reproduction of the music information with respect to the streaming-reproduced singing voice, based on the reproduction timing of the music information determined based on the delay amount of the second reception process. Effect of the Invention

[0015] According to the karaoke program and karaoke system of the present invention, when karaoke is performed between multiple terminal devices, it is possible to reduce the discrepancy between the singing voice and the music on the terminal device of the listener who listens to the singing, thereby reducing the sense of discomfort felt by the listener. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 shows a system configuration according to an embodiment of the present invention. [Diagram 2] FIG. 1 is an external view of a mobile terminal according to an embodiment of the present invention; [Diagram 3] FIG. 1 is a block diagram showing a configuration of a mobile terminal according to an embodiment of the present invention. [Figure 4] FIG. 1 is a flow diagram showing room processing according to the present embodiment. [Diagram 5] 1 is a flow chart showing the karaoke process of the present embodiment; [Figure 6] A flow diagram showing a music playback process A according to the present embodiment. [Figure 7]Flow diagram showing music playback process B of this embodiment. [Figure 8] 1 is a timing chart illustrating the amount of delay in a mobile terminal according to an embodiment of the present invention. [Figure 9] A timing chart illustrating the amount of delay in a mobile terminal according to another embodiment. [Figure 10] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 11] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 12] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 13] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Fig. 1 is a diagram showing the system configuration of this embodiment. In this embodiment, the system allows karaoke to be performed between mobile terminals 6, and therefore communication between the mobile terminals 6 is necessary. Fig. 1 shows various examples of communication formats between the mobile terminals 6. For example, in the case of a private home, a router 12a connected to the Internet is provided, and a wireless router 12b is connected to the router 12a. The mobile terminal 6 can be connected to the Internet via the wireless router 12b.

[0018] In addition to these communication modes, it is also possible to communicate with the Internet using a mobile communication network. The mobile communication network is a communication mode in which the mobile terminal 6 is communicatively connected to the Internet by performing wireless communication between the mobile terminal 6 and a wireless base station 13.

[0019] By using these various communication formats, the mobile terminal 6 can be connected to the Internet. A management server 11a and a distribution server 11b for managing the karaoke system are also connected to the Internet. A karaoke program is installed in the mobile terminal 6, which is connected to the Internet, and the mobile terminal 6 can perform karaoke between the mobile terminals 6 by communicating with the management server 11a and the distribution server 11b.

[0020] In this embodiment, a form in which karaoke is performed using a mobile terminal 6 as a terminal device will be described. Note that, in addition to the mobile terminal 6, a home game machine, a personal computer, or a karaoke device dedicated to karaoke may also be used as the terminal device. Furthermore, a terminal device in which a karaoke program is installed functions as a karaoke device.

[0021] Fig. 2 is a diagram showing the appearance (front view) of a mobile terminal 6 according to an embodiment of the present invention, and Fig. 3 is a block diagram showing the configuration of the mobile terminal 6 according to an embodiment of the present invention. A smartphone, a tablet terminal, a mobile phone, etc. can be used as the mobile terminal 6. By installing a karaoke program in the mobile terminal 6, it becomes possible to perform karaoke between the mobile terminals 6.

[0022] The portable terminal 6 of this embodiment has a control means including a CPU 61, a RAM 62, a storage unit 63, an image processing unit 64, and an audio processing unit 66. A non-volatile storage means such as a NAND type flash memory can be adopted as the storage unit 63. The storage unit 63 stores the karaoke program of this embodiment and various data used in the karaoke program.

[0023] The CPU 61 is a means for controlling the entire mobile terminal 6, and executes various controls based on programs and data stored in the storage unit 63. The RAM 62 temporarily stores data generated under the control of the CPU 61.

[0024] The sound processing unit 66 is a means for inputting and outputting sound related to the portable terminal 6. It is a means for emitting sound from the speaker 67a under the control of the CPU 61. The sound processing unit 66 is also connected to the microphone 67b, and can capture singing voice, telephone call voice, and surrounding sounds from the microphone 67b. An external device such as a headset may be used for the speaker 67a and the microphone 67b.

[0025] The touch panel display unit 68 is configured with a display unit 68m that displays an image and a touch panel 68n disposed on the front or back surface of the display unit 68m. The touch panel 68n can be of various types that can recognize the touch position of the user, such as a capacitance type. The CPU 61 performs display on the display unit 68m and can determine which part of the displayed image has been touched by touch input from the touch panel 68n.

[0026] The image processing unit 64 is a means for inputting and outputting image-related data for the mobile terminal 6. The image processing unit 64 displays and outputs images formed by the CPU 61 on the display unit 68m. The image processing unit 64 has a function of capturing images or, if video, videos from the front camera 65F arranged on the front side (the touch panel display unit 68 side) of the mobile terminal 6 or the rear camera 65R arranged on the rear side of the mobile terminal 6, and storing the captured images or videos in the storage unit 63 as image or video information.

[0027] The mobile terminal 6 of this embodiment includes a first communication unit 69a and a second communication unit 69b as communication means for communicating with the outside. The first communication unit 69a is a communication means for connecting to a mobile communication network, and can perform various communications via the Internet network using a communication line of the mobile communication network by wirelessly connecting to a mobile base station 140. The second communication unit 69b is a communication means for connecting to Wifi. The mobile terminal 6 can wirelessly connect to an access point 110 using the second communication unit 69b, and can communicatively connect to a local area network in which the access point 110 participates. Note that, as communication means of the mobile terminal 6, in addition to the first communication unit 69a and the second communication unit 69b, a short-range communication means such as an infrared communication unit or Bluetooth (registered trademark) may be provided.

[0028] The mobile terminal 6 of this embodiment is configured to include an input switch 70 in addition to the touch panel 68n as an input means. The input switch 70 (input switches 70a, 70b, etc. in FIG. 2) can be provided with various switches such as a power switch, a volume adjustment switch, and a switch whose input purpose is changed depending on the program.

[0029] Furthermore, the mobile terminal 6 of this embodiment has a vibration unit 72 and an LED 73 to provide various notifications to the user who holds the mobile terminal 6. The vibration unit 72 is a member formed by attaching an eccentric weight to the shaft of a motor. By driving the motor to rotate, it is possible to transmit vibrations that can notify the user. The LED 73 is disposed above the front of the mobile terminal 6 as shown in FIG. 2, and can provide various notifications to the user by blinking.

[0030] In this embodiment, karaoke can be performed between the mobile terminals 6 by installing a karaoke program in such mobile terminals 6. Next, various processes performed when performing karaoke using the mobile terminals 6 will be described in detail.

[0031] 4 is a flow diagram showing the room processing of this embodiment. Here, a case where karaoke is performed using the mobile terminals 6a and 6b will be described. In this embodiment, by gathering in a virtually created room, users who gather in the same room can share audio (which may include video) and perform karaoke.

[0032] Here, a case will be described in which a user using the mobile terminal 6a creates a room, and a user using the mobile terminal 6b participates in the room. It is assumed that the users using the mobile terminals 6a and 6b have accounts on the management server 11a in advance. Note that a user who does not have an account can access the management server 11a and create a new account.

[0033] First, the mobile terminal 6a issues a login instruction (S11) to the management server 11a. The login instruction is issued by inputting account information (user ID, password, etc.). Upon receiving the login instruction (S11), the management server 11a executes a login process (S21). In the login process, authentication is performed based on the account information, and if the authentication is correct, the user is allowed to use various functions of the karaoke program.

[0034] FIG. 10(A) shows a room selection screen that is displayed after logging in. On the room selection screen, a room selection field 68a and a new room creation button 68b are displayed. The room selection field 68a shows a room that has already been created, and the status of the room is displayed. The room selection field displays the room name, icons of users participating in the room, the room's holding status ("OPEN", "In preparation"), the date the room is held, and the like. Also, the room selection field 68a with a lock mark displayed in the upper left corner indicates that the room requires a password or that only users who have been invited in advance can participate in the room.

[0035] A user can participate in a room by operating the room selection field 68a. The new room creation button 68b is a button for creating a new room. By operating the new room creation button 68b, the mobile terminal 6a can issue a room creation instruction (S12) to the management server 11a. The management server 11a executes a room creation process (S22) based on the room creation instruction (S12). When the room creation process (S22) is completed, the room selection field 68a shown in FIG. 7(A) is displayed.

[0036] On the other hand, the mobile terminal 6b that is to participate in the room first issues a login instruction (S11) to the management server 11a by inputting account information. The management server 11a performs authentication based on the account information, as in the case of the mobile terminal 6a, and if the authentication is proper, logs in the mobile terminal 6b. The user refers to the room selection screen shown in FIG. 10(A) and issues a room participation instruction by selecting the room selection field 68a that the user wants to enter (S13). In the example of FIG. 4, when the user of the mobile terminal 6a is to participate in a room created by the user, the management server 11a executes a user addition process (S23) that allows the user of the mobile terminal 6b to participate in the room created by the user of the mobile terminal 6a.

[0037] As the users of the mobile terminals 6a and 6b enter the same room, it becomes possible to execute karaoke processing (S12) between the mobile terminals 6a and 6b.

[0038] The karaoke process (S30) mainly comprises reservation management and music playback processing. The reservation management is a process for reserving the music to be sung, and the music playback process is a process for playing the music registered in the reservation list in order based on the reservation processing. In playing the music, the performance sound is played based on the performance information included in the music information, and the lyrics are displayed based on the lyrics information included in the music information.

[0039] Fig. 5 is a flow diagram showing the karaoke process (S30) of this embodiment. Fig. 5 shows a case where the mobile terminal 6a is a singer (a person who has reserved a song) and the mobile terminal 6b is a listener, but the singer and the listener may be interchangeable. Also, in Fig. 5, for ease of explanation, two mobile terminals 6a and 6b are shown as an example participating in the room, but three or more mobile terminals may participate in the room.

[0040] First, the mobile terminals 6a and 6b communicate with the management server 11a and start a session for reservation management (S101, S121). The management server 11a starts a session for reservations, music distribution, etc. for the mobile terminals 6a and 6b participating in the same room (S161).

[0041] Next, the mobile terminals 6a and 6b communicate with the distribution server 11b, respectively, and start a session for the audio stream (S102, S122). After the session for the audio stream starts, audio communication of singing voice information and the like can be performed between the mobile terminals 6a and 6b. The distribution server 11b distributes the singing voice and the like between the mobile terminals 6a and 6b using a streaming method (S151).

[0042] After the session for the audio stream is started, the mobile terminals 6a and 6b each measure the delay amount of the audio upload / download by communicating with the distribution server 11b (S103, S123). The delay amount measured here is used to synchronize the timing between the music piece and the singing voice reproduced on the mobile terminal 6b used by the listener.

[0043] A song reservation process (S104) is executed on the singer's mobile terminal 6a. In the reservation process (S104), a song is searched for and selected to be sung, so that the song can be added to a reservation list managed by the management server 11a. A user who wants to reserve a song searches for a song using the song search screen shown in Fig. 10(B).

[0044] The music search screen has a search category selection field 68c and a music selection field 68d. The search category selection field has search categories such as "Favorites," "Songs," "Artists," and "Albums," and the user can select one of these search categories to perform a music search according to the selected search category. In the state shown in Fig. 10(B), "Songs" is selected as the search category, and the music selection field 68d is displayed in alphabetical order.

[0045] The song selection field 68d displays an icon (such as album art), song title, and artist name for each song. When the user selects a desired song selection field 68d, a song confirmation screen for that song is displayed.

[0046] FIG. 11C is a diagram showing a music confirmation screen of this embodiment. A music confirmation field 68e is displayed on the music confirmation screen. In the music confirmation field, an icon (such as album art), music title, artist name, a reservation button 68g, and a cancel button 68f are displayed for the selected music. The user checks the information displayed on this music confirmation screen, and when making a reservation, the user operates the reservation button 68g to reserve the music. When the reservation is made, reservation information is transmitted to the management server 11a. The reservation information includes a user ID indicating the user who made the reservation, and a music ID indicating the music. The management server 11a registers the received reservation information in a reservation list (S162).

[0047] When the next song to be played in the reservation list arrives, the management server 11a notifies the mobile terminals 6a and 6b belonging to the same room of a reservation response (S163). The reservation response includes information that can identify the person who reserved the song, and the mobile terminals 6a and 6b can determine whether the mobile terminals are singer terminals (mobile terminals 6 that have reserved the song) or listener terminals (mobile terminals 6 that have not reserved the song) by referring to the reservation response.

[0048] In the case of Fig. 5, since the song reservation was made by the mobile terminal 6a, the mobile terminal 6a judges itself to be a singer terminal and sets itself to singer mode (S106). On the other hand, the mobile terminal 6b judges itself to be a listener terminal and sets itself to listener mode (S125).

[0049] After receiving the reservation response, the management server 11a transmits music information to the mobile terminals 6a and 6b participating in the same room (S164). In this embodiment, the music information is distributed using a storage type distribution method rather than a streaming method. Even with a storage type distribution method, it is possible to start playback by storing part of the music information. Upon starting transmission from the management server 11a, the mobile terminals 6a and 6b start receiving the music information (S107, S126) and execute music playback process A (S130) and music playback process B (S140), respectively.

[0050] 6 is a flow diagram showing a music reproduction process A (S130) executed by the mobile terminal 6a set to the singer mode. In the music reproduction process A, the reproduction of music information is started (S132) on the condition that a predetermined amount of music information has been downloaded (S131: Yes). In addition, during the period until the reproduction of the music information is completed (S134: Yes), the amount of delay in uploading the singing voice information from the mobile terminal 6a to the distribution server 11b is measured at an appropriate timing (S133).

[0051] Fig. 12(E) is a music playback screen displayed during music playback on the mobile terminal 6 of this embodiment. As explained in Fig. 11(D), the room name and a music search button 68k are displayed at the top of the music playback screen. By operating the music search button 68k, the music search screen explained in Fig. 10(B) is displayed, and it is possible to reserve a song.

[0052] In addition, the music playback screen is broadly divided into a music-related display field 68h, an upper display field 68i, and a lower display field 68j. The music-related display field 68h includes a lyrics display field 681 and a progress bar 682. The lyrics display field 681 displays lyrics to be sung for the music being played. Note that the lyrics display field 681 may also display background video. The progress bar 682 indicates the progress of music playback.

[0053] The area occupied by the upper display field 68i and the lower display field 68j can be changed. Normally, the state shown in FIG. 11(D) is displayed, that is, both the upper display field 68i and the lower display field 68j are displayed. When the comment tab 686, the reservation confirmation tab 687, and the remote control tab 688 provided in the lower display field 68j are operated, the lower display field 68j is displayed using the area where the upper display field 68i was displayed, as shown in FIG. 13(F). FIG. 13(F) shows the state where the remote control tab 688 is operated. By operating the comment tab 686, the reservation confirmation tab 687, and the remote control tab 688 again, the state returns to that shown in FIG. 12(E).

[0054] In the upper display field 68i, a status display / selection button 683, a microphone selection button 684, and a user icon 685 are displayed. The status display / selection button 683 is a button that indicates the status of the mobile terminal 6, and when the mobile terminal 6 is set to the singer mode, "singing" is marked, and when the mobile terminal 6 is set to the listener mode, "listening" is marked. The state in FIG. 11(D) is a state in which "singing" is marked. The user can manually switch between "singing", "listening", and "away". In this case, when the status is switched to "singing", the setting is changed to the singer mode, and when the status is switched to "listening" or "away", the setting is changed to the listener mode. In this embodiment, three states can be switched, but two statuses, "singing" and "listening", may be used, and various display forms can be adopted.

[0055] The microphone selection button 684 can be manually operated by the user to switch the microphone 67b on and off. In this embodiment, when the mode is set to singer mode, it is automatically set to on, and when the mode is set to listener mode, it is automatically set to off. The user icons 685 show all users participating in the room. In the example of FIG. 11(D), four users have participated. A user whose icon is shown with a thick line ("JOY" in the example of FIG. 11(D)) is singing (set to singer mode). An icon with a speaker and a cross ("Monkichi" in the example of FIG. 11(D)) shows that the user is away. These display forms are merely examples, and statuses such as singing, listening, and away may be shown by other display forms.

[0056] The lower display field 68j has three tabs (a comment tab 686, a reservation confirmation tab 687, and a remote control tab 688). In the state shown in FIG. 11(D), the comment tab 686 is selected, and a comment display field 689 and a comment input field 690 are displayed in the lower display field 68j. In the comment display field 689, in addition to a user comment input by the user using the comment input field 690, a system comment associated with an operation related to a music playback process is displayed. In FIG. 11(D), the comment display field 689 displays a system comment associated with a reservation process, which includes the user name and the reserved music name / artist name. The comment display field 689 sequentially displays the system comment and user comment in chronological order, and the user can communicate smoothly by viewing these system comments and user comments.

[0057] 13(F) shows the state when the remote control tab 688 is operated. As described above, the lower display column 68j is displayed by expanding it to the area where the upper display column 68i is displayed. When the remote control tab 688 is selected, the lower display column 68j displays a singer volume 691, a speaker volume 692, a pitch change button 693, a BGM volume 694, a playback position adjustment button 695, a re-sing button 696, a pause button 697, and a performance stop button 698.

[0058] In this embodiment, the various controls and buttons displayed on the remote control tab 688 can be operated only by the mobile terminal 6 set to the singer mode. Therefore, the mobile terminal 6 set to the listener mode is prohibited from performing operations related to music playback, making it possible to prevent erroneous input by the listener.

[0059] The singer volume 691 is an operator for adjusting the volume of the singing voice input from the microphone 67b of the mobile terminal 6 set to the singer mode. The speaker volume 692 is an operator for adjusting the volume of the singing voice input from the microphone 67b of the mobile terminal 6 set to the listener mode. The singer volume 691 and the speaker volume 692 are set to the default state of each mode when the mobile terminal 6 is set to the singer mode or the listener mode, but can be operated only from the mobile terminal 6 set to the singer mode.

[0060] The pitch change button 693 is a button for changing the pitch of the song being played. The BGM volume 694 is an operator for adjusting the volume of the BGM played between songs. The playback position adjustment button 695 is an operator for changing the playback position. The current playback time and the total playback time are displayed to the left of the operators. The sing again button 696 can be operated to play the song being played from the beginning. The pause button 697 is a button for temporarily stopping the song being played. The performance stop button 698 is a button for stopping the song being played, and when the performance stop button 698 is operated, playback processing moves to the next reserved song.

[0061] Next, a music playback process B executed by the mobile terminal 6a set to the listener mode will be described. Fig. 7 is a flow diagram showing the music playback process B (S140). In the music playback process B, the playback of music information is also started (S144) under the conditions that a predetermined amount of music information has been downloaded (S141: Yes) and the playback start timing has arrived (S143: Yes).

[0062] Here, the playback start timing is a timing calculated (S142) using the amount of delay measured by using the measurements (S103, S123) at the mobile terminals 6a and 6b. The singing voice information input at the mobile terminal 6a is uploaded to the distribution server 11b, downloaded from the distribution server 11b to the mobile terminal 6b, and played at the mobile terminal 6b on the listener's side. Therefore, it is considered that a delay occurs in the singing voice information between the mobile terminal 6a and the mobile terminal 6b. In this embodiment, taking such circumstances into consideration, one of the objectives is to suppress a delay that may occur between the singing voice and the music played at the mobile terminal 6b on the listener's side.

[0063] Fig. 8 is a timing chart for explaining the delay amount in the mobile terminal of this embodiment. Fig. 8 assumes two mobile terminals 6a and 6b, as in Fig. 5. On the singer's side, the singer sings in sync with the playback of the music information, so there is no delay between the music and the singing voice. On the other hand, on the listener A's side, as explained above, a delay amount D1 occurs in the singing voice information. Therefore, if the music is played at the same timing as the singing company's side, the music will be played ahead of the delay amount D1.

[0064] Therefore, in this embodiment, the start of reproduction of the music information is delayed by a delay amount D1 to synchronize the music and the singing voice on the listener side. The delay amount D1 is calculated, for example, by the sum of the delay amount when uploading the singing voice information measured by the mobile terminal 6a (S103) and the delay amount when downloading the singing voice information measured by the mobile terminal 6b (S123). The distribution server 11b receives the delay amount from the mobile terminals 6a and 6b to calculate the delay amount D1, and transmits the reproduction timing St_a received from the mobile terminal 6a on the singer side to the mobile terminal 6b on the listener side. The mobile terminal 6b determines the reproduction timing St_b on the listener A side based on the received delay amount D1 and the reproduction timing St_a (S142).

[0065] In this embodiment, the portable terminal 6b on the listener side corrects the timing of the singing voice and the music during the playback of the music information. The delay of the singing voice information between the portable terminals 6a and 6b may vary depending on the communication conditions. Therefore, the portable terminals 6a and 6b measure the delay of the singing voice information at appropriate timing during the playback of the music information to prevent a gap from occurring between the played singing voice and the music.

[0066] As explained in Fig. 6, in the music reproduction process A executed on the singer's mobile terminal 6a, the delay amount in uploading singing voice information during the music reproduction period is measured (S133) and transmitted to the distribution server 11b. Meanwhile, in the music reproduction process B executed on the listener's side, the delay amount in downloading singing voice information is measured (S145) and transmitted to the distribution server 11b. Then, in the music reproduction process B, the amount of correction for the reproduction of the music information is determined based on the amount of delay received from the distribution server 11b (S146). The amount of correction may be to speed up, delay, or maintain the timing of reproduction of the music information.

[0067] The mobile terminal 6b corrects the playback timing of the music information based on the determined correction amount (S147). One possible correction method would be to change the playback timing to an appropriate timing all at once, but in that case, the continuity of the music playback would be lost, resulting in an unnatural playback sound. For this reason, in this embodiment, a method is adopted in which the playback speed of the music information is corrected until the appropriate playback timing is reached. For example, if the playback timing of the music information is late, the playback speed of the music information is increased, and if the playback timing of the music information is early, the playback speed of the music information is decreased.

[0068] The measurement of the delay amount and the correction of the playback timing are repeated at appropriate timings until the playback of the music information ends (S148: Yes). By performing the correction based on the delay amount during the playback of the music information in this way, it becomes possible to suppress the occurrence of a discrepancy between the singing voice information inputted at the portable terminal 6a of the singer and the music information played at the portable terminal 6b of the listener.

[0069] Fig. 9 is a timing chart for explaining the delay amount in a mobile terminal of another embodiment. In the embodiment of Fig. 8, as in Fig. 5, a state in which two mobile terminals 6a and 6b participate in a room has been explained. On the other hand, it is also possible for three or more mobile terminals 6 to participate in a room. Fig. 9 shows a state in which three mobile terminals 6 participate in the same room, with one mobile terminal 6a being used by the singer and two mobile terminals 6b and 6c being used by the listeners.

[0070] Fig. 9(A) shows an embodiment in which the same process as in Fig. 8 is performed individually in each of the mobile terminals 6b and 6c. The mobile terminals 6b and 6c may have different delay amounts. The mobile terminals 6b and 6c use the respective delay amounts D1 and D2 and playback timings St_b and St_c to achieve consistency between the audio and the music.

[0071] By the way, in addition to enjoying karaoke by listening to singing voices, it is also possible to actively participate and enjoy using the mobile terminals 6b and 6c. For example, it is possible to exchange messages using the comment input field 690 described in Fig. 12. It is possible for listeners to enter messages such as comments to the singer to support him / her, or to communicate with each other by exchanging impressions of the singing.

[0072] As shown in FIG. 9(A), if the timing of playback of music information differs between the listeners' portable terminals 6b, 6c, communication between the listeners may not go well, particularly due to mistiming of comments about the music being played or the singing voice.

[0073] Therefore, in this embodiment, the portable terminals 6a and 6b on the listener side do not use different reproduction timings St_b and St_c, but use the same reproduction timing.

[0074] As shown in Fig. 9(B), the mobile terminal 6b has a delay D1 in the singing voice information, and the mobile terminal 6c has a delay D2. The delay D1 is smaller than the delay D2. In this case, the mobile terminals 6b and 6c unify the playback timing St_c using the larger delay D, thereby synchronizing the playback timing between the mobile terminals 6b and 6c of the listeners. The distribution server 11b decides which of the delays D1 and D2 to use, or receives information necessary for the decision, such as the delays D1 and D2, via the distribution server 11b.

[0075] In this way, by unifying the playback timing on the listeners' portable terminals 6b, 6c, multiple listeners can listen to the music and singing voice at the same timing, which makes it possible to facilitate communication between listeners.

[0076] As described above, in this embodiment, when enjoying karaoke at a distant location, it is possible to suppress the mismatch between the reproduced music information and the singing voice information, and to suppress the discomfort felt by the listener. Note that the present invention is not limited to the above-described embodiment. Various modified examples will be described below.

[0077] [First Modification] In the above-mentioned embodiment, the timing of reproducing the music information is determined by taking into consideration the delay of the singing voice information. In the above-mentioned embodiment, since the storage type music information distribution is used, the possibility of delays occurring in downloading the music information is low. Meanwhile, the music information can also be distributed using a streaming method, as in the case of the singing voice information. In this case, it is considered that delays occur in the music information distributed from the management server 11a, and in some cases, the delays may be greater than those in the singing voice information.

[0078] In consideration of such a situation, the timing of reproduction of music information on the portable terminal 6b on the listener side may take into consideration the delay amount of music information at the time of distribution. In this case, the portable terminal 6b on the listener side measures the delay amount (first delay amount) for downloading from the management server 11a that distributes music information. When the delay amount (first delay amount) of the music information is larger than the delay amount (second delay amount) of the singing voice information transmitted from the portable terminal 6a, the timing of reproduction of the music information is determined based on the delay amount (first delay amount) of the music information. On the other hand, when the delay amount (first delay amount) of the music information is equal to or smaller than the delay amount (second delay amount) of the singing voice information transmitted from the portable terminal 6a, the timing of reproduction of the music information is determined based on the delay amount (second delay amount) of the singing voice information.

[0079] In this way, even if the distribution of music information is delayed due to a streaming method or the like, it is possible to synchronize the reproduction timing of the music information and the singing voice information.

[0080] When there are multiple listeners, the listeners' portable terminals 6b and 6c measure the delay (first delay) for downloading from the management server 11a. The listeners' portable terminals 6b and 6c determine the playback timing using the largest delay between the singing voice information delay (second delay) and the music information delay (first delay) measured by the listeners' portable terminals 6b and 6c, thereby making it possible to unify the playback timing between the listeners' portable terminals 6b and 6c.

[0081] [Second modified example] In the above embodiment, as explained in the music playback process B of FIG. 7, the playback timing is corrected when a deviation occurs during music playback. In the above embodiment, the playback speed is changed to perform the correction (S147) to reproduce a natural playback sound. Instead of this, the correction may be performed at a timing other than the singing section (non-singing section). For example, by performing the correction during an interlude, it is possible to perform the correction without interfering with the listening of the singing. Furthermore, the non-singing section may be not only an interlude, but also a short section between singing phrases.

[0082] Possible methods for detecting non-singing sections include setting information about the singing sections in the music information in advance and detecting them by reading out the information in synchronization with the playback of the music information, or detecting periods when no singing voice is input as non-singing sections.

[0083] [Third Modification] In the above embodiment, the singing voice information is transmitted from the portable terminal 6a of the singer to the portable terminals 6b and 6c of the listeners via the distribution server 11b. Instead of this form, the singing voice information may be transmitted from the portable terminal 6a of the singer to the portable terminals 6b and 6c of the listeners in a peer-to-peer manner without the distribution server 11b. [Explanation of symbols]

[0084] 6 (6a, 6b): Portable terminal 68m: Display unit 11a: Management server 68n: Touch panel 11b: Distribution server 69a: First communication unit 12a: Router 69b: Second communication unit 12b: Wireless router 70 (70a, 70b): Input switch 13: Wireless base station 72: Vibration unit 61: CPU 73: LED 62:RAM 110:Access point 63: Memory unit 140: Mobile base station 64: Image processing section 681: Lyrics display section 65F: Front camera 682: Progress bar 65R: Rear camera 683: Selection button 66: Sound processing unit 684: Microphone selection button 67a: Speaker 685: User icon 67b: Microphone 686: Comments tab 68: Touch panel display 687: Reservation confirmation tab 68a: Room selection section 688: Remote control tab 68b: New room creation button 689: Comment display field 68c: Search category selection field 690: Comment input field 68d: Song selection column 691: Singer volume 68e: Song confirmation column 692: Speaker volume 68f: Cancel button 693: Pitch change button 68g: Reservation button 694: BGM volume 68h: Song-related display area 695: Playback position adjustment button 68i: Top display area 696: Button 68j: Lower display area 697: Pause button 68k: Song search button 698: Play / Stop button

Claims

1. A karaoke program executable on a plurality of terminal devices connected to each other via a network, the program comprising: A connection process for connecting to a server device; A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal; a first receiving process for receiving music information from the server device; If the terminal is determined to be a listener terminal in the determination process, a second receiving process for receiving singing voice information transmitted from the singer terminal in a streaming manner; A determination process for determining a delay amount of the second reception process; and executing a reproduction process of delaying the reproduction of the music information with respect to the singing voice being streamed, based on a reproduction timing of the music information determined based on the delay amount of the second reception process. Karaoke program.

2. The playback timing used in the playback process is determined based on the delay amounts determined at the plurality of listener terminals.

2. The karaoke program according to claim 1.

3. The playback timing used in the playback process is determined based on the maximum delay amount among the delay amounts determined by the multiple listener terminals.

3. The karaoke program according to claim 2.

4. The determination process is executed before the start of reproduction of the music information or multiple times during the execution of the reproduction process.

2. The karaoke program according to claim 1.

5. The playback process changes the playback speed of the music information when the playback timing has changed from the previous time.

5. The karaoke program according to claim 4.

6. The delay in the reproduction of the music information in the reproduction process is performed during a non-singing section.

5. The karaoke program according to claim 4.

7. A karaoke system having a plurality of terminal devices communicatively connected via a network, The terminal device A connection process for connecting to a server device; A discrimination process for discriminating whether the terminal is a singer terminal or a listener terminal; a first receiving process for receiving music information from the server device; If the terminal is determined to be a listener terminal in the determination process, a second receiving process for receiving singing voice information transmitted from a singer terminal in a streaming manner; A determination process for determining a delay amount of the second reception process; and executing a reproduction process of delaying the reproduction of the music information with respect to the singing voice being streamed, based on a reproduction timing of the music information determined based on the delay amount of the second reception process. Karaoke system.

Images