Karaoke program, karaoke device, and karaoke display method

Abstract

This prevents the system from singing along to delayed vocal data received due to communication environment issues. [Solution] The information processing device is made to execute the following: a music playback process that plays music information; a reception process that receives singing audio from another terminal device; an audio playback process that plays the received singing audio; a delay detection process that detects the amount of delay in the singing audio received in the reception process; and a display process that displays information based on the amount of delay detected in the delay detection process.

Description

Technical Field

[0001] The present invention relates to karaoke that performs singing in accordance with the reproduced music, and particularly relates to a karaoke program for realizing the functions of a karaoke device in an information processing device, a karaoke device, and a karaoke display method.

Background Art

[0002] Conventionally, karaoke in which singing is enjoyed in accordance with the accompaniment has been performed in social settings and the like. With the expansion of the use of networks via the Internet, it has become possible for users at remote locations to enjoy karaoke using personal computers, game devices, mobile terminals, etc. installed at home. Patent Document 1 discloses a singing system in which a plurality of users can sing using communication.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 discloses a karaoke system in which, when multiple users sing using different mobile devices, the accompaniment data is played back with a delay on the mobile device used by the listener compared to the mobile device used by the singer. However, when the singer's mobile device plays back the singing voice data of other singers, the singing voice of the other singers will be heard with a delay relative to the accompaniment sound played back on the mobile device based on the accompaniment data. Therefore, when singing while playing back the singing voice data of other singers, it is possible that the singing timing will be delayed due to being influenced by the singing voice of the other singers. Such a delay in singing timing will cause a discrepancy in the singing voice data on the listening mobile device, which will cause discomfort to the listener.

[0005] This invention takes these circumstances into consideration and aims to provide a display that encourages users not to be influenced by the singing voices of other singers in online karaoke, where karaoke rooms are formed online. [Means for solving the problem]

[0006] Therefore, the karaoke program according to the present invention is Music playback processing that plays back song information, Receiving process for receiving singing audio from other terminal devices, Audio playback processing that plays back the received singing audio, The receiving process includes a delay detection process that detects the amount of delay in the singing audio received, Based on the delay amount detected by the delay amount detection process, the information processing device is instructed to execute a display process that displays the information.

[0007] Furthermore, in the karaoke program according to the present invention, The display process changes the display format according to the amount of delay.

[0008] Furthermore, the karaoke program according to the present invention is The information processing device is instructed to perform a transmission process that sends the user's singing voice, input from the microphone, to another terminal device.

[0009] Furthermore, the karaoke program according to the present invention is The information processing device is instructed to perform a lyric playback process that plays lyric information corresponding to the song information to be played during the music playback process. The display process changes the display format of the lyrics information being played based on the delay amount detected by the delay amount detection process.

[0010] Furthermore, in the karaoke program according to the present invention, The display format of the lyrics information that is changed during the display process is the size of the lyrics.

[0011] Furthermore, in the karaoke program according to the present invention, The display format of the lyrics information that is changed during the display process is the timing of the color change of the lyrics.

[0012] Furthermore, in the karaoke program according to the present invention, The audio playback process changes the volume of the singing audio to be played based on the delay amount detected by the delay amount detection process.

[0013] Furthermore, the karaoke program according to the present invention is A singing timing detection process that detects the timing of the singer's voice input from the microphone in relation to the playback of song information played during the music playback process, If the singing timing detected by the singing timing detection process falls within a predetermined range, the display will either not be shown, or it will be displayed in a different format.

[0014] Furthermore, the karaoke program according to the present invention is A singing timing detection process that detects the timing of the singer's voice input from the microphone in relation to the playback of song information played during the music playback process, If the singing timing detected by the singing timing detection process exceeds a predetermined range, the audio playback process changes the volume of the singing audio to be played based on the delay amount detected by the delay amount detection process.

[0015] In addition, the karaoke device according to the present invention executes a music playback process for playing music information, a reception process for receiving singing voices from other terminal devices, a voice playback process for playing back the received singing voices, a delay amount detection process for detecting the delay amount of the singing voices received in the reception process, and a display process for performing display based on the delay amount detected in the delay amount detection process.

[0016] In addition, the karaoke display method according to the present invention performs a music playback process for playing music information, a reception process for receiving singing voices from other terminal devices, a voice playback process for playing back the received singing voices, a delay amount detection process for detecting the delay amount of the singing voices received in the reception process, and a display process for performing display based on the delay amount detected in the delay amount detection process.

Effect of the Invention

[0017] According to the karaoke program, karaoke device, and karaoke display method according to the present invention, when singing using a plurality of terminal devices (karaoke devices), one of the purposes is to prevent being influenced by the singing voices of other singers.

Brief Description of the Drawings

[0018] [Figure 1] Diagram showing the system configuration of this embodiment [Figure 2] External view of the mobile terminal of this embodiment [Figure 3] Block diagram showing the configuration of the mobile terminal of this embodiment [Figure 4] Flowchart showing the room process of this embodiment [Figure 5] Flowchart showing the reservation process of this embodiment [Figure 6] Flowchart showing the playback process of this embodiment [Figure 7]A diagram illustrating the delay amount of audio signals between mobile terminals in this embodiment. [Figure 8] Time chart showing accompaniment and vocal sounds in the mobile terminal of this embodiment [Figure 9] A time chart for explaining the problems of the present invention. [Figure 10] Diagram showing various screens in the mobile terminal of this embodiment. [Figure 11] Diagram showing various screens in the mobile terminal of this embodiment. [Figure 12] Diagram showing various screens in the mobile terminal of this embodiment. [Figure 13] Flowchart illustrating the main regeneration process and secondary regeneration process of this embodiment. [Figure 14] Flowchart illustrating the warning process of this embodiment [Figure 15] Flowchart illustrating the playback process on the listening side in this embodiment. [Figure 16] Figures showing various display forms of other embodiments. [Modes for carrying out the invention]

[0019] Figure 1 shows the system configuration of this embodiment. This embodiment is a system that enables karaoke by communicating between mobile terminals 6. Figure 1 shows various examples of communication configurations between mobile terminals 6. For example, in a private home, a router 12a connected to the internet is provided, and a wireless router 12b is connected to the router 12a. Mobile terminals 6 can connect to the internet via the wireless router 12b.

[0020] In addition to these communication methods, it is also possible to communicate with the Internet using a mobile communication network. A mobile communication network is a communication method that connects the mobile terminal 6 to the Internet by performing wireless communication between the mobile terminal 6 and the wireless base station 13.

[0021] By using these various communication methods, the mobile terminal 6 can connect to the internet. Furthermore, a management server 11 for managing the karaoke system is connected to the internet. Mobile terminals 6, which have the karaoke program installed and are connected to the internet, can communicate with this management server 11, enabling karaoke to be performed between the mobile terminals 6.

[0022] In this embodiment, we will describe a configuration in which karaoke is performed using a mobile terminal 6 as the terminal device. Note that, in addition to the mobile terminal 6, a home game console, a personal computer, or a dedicated karaoke machine may also be used as the terminal device. Furthermore, a terminal device with a karaoke program installed will function as a karaoke machine.

[0023] Figure 2 is a diagram showing the external appearance (front view) of a mobile terminal 6 according to an embodiment of the present invention, and Figure 3 is a block diagram showing the configuration of the mobile terminal 6 according to an embodiment of the present invention. This mobile terminal 6 can be a smartphone, tablet terminal, mobile phone, etc. By installing a karaoke program on this mobile terminal 6, it becomes possible to perform karaoke between the mobile terminals 6.

[0024] The mobile terminal 6 of this embodiment has a control means comprising a CPU 61, RAM 62, storage unit 63, image processing unit 64, and sound processing unit 66. The storage unit 63 can employ a non-volatile storage means such as NAND flash memory. This storage unit 63 stores the program of this embodiment, as well as various data used by the program.

[0025] The CPU 61 is a means of comprehensively controlling the entire mobile terminal 6, and executes various controls based on the programs and data stored in the memory unit 63. The RAM 62 temporarily stores the data generated by the control of the CPU 61.

[0026] The sound processing unit 66 is a means for handling audio input and output of the mobile terminal 6. It emits sound from the speaker 67a under the control of the CPU 61. The sound processing unit 66 is also connected to the microphone 67b, and can pick up call audio and ambient sounds from the microphone 67b. External devices such as a headset may be used for the speaker 67a and microphone 67b.

[0027] The touch panel display unit 68 comprises a display unit 68m that displays an image and a touch panel 68n disposed on its front or back surface. The touch panel 68n can employ various types, such as a capacitive touch panel, that can recognize the user's touch position. The CPU 61 displays an image on the display unit 68m and can also determine which part of the displayed image has been touched based on the touch input from the touch panel 68n.

[0028] The image processing unit 64 is a means for inputting and outputting image-related information from the mobile terminal 6. The image processing unit 64 outputs the image formed by the CPU 61 to the display unit 68m. It also has the function of capturing images, or if it is video, video, from the front camera 65F located on the front of the mobile terminal 6 (on the side of the touch panel display unit 68), or from the rear camera 65R located on the back of the mobile terminal 6, and storing the image or video information in the storage unit 63.

[0029] The mobile terminal 6 of this embodiment includes a first communication unit 69a and a second communication unit 69b as communication means for communicating with the outside world. The first communication unit 69a is a communication means for connecting to a mobile communication network, and by wirelessly connecting to a mobile base station 140, it is possible to perform various communications via the Internet network using the communication lines of the mobile communication network. The second communication unit 69b is a communication means for Wi-Fi connection. The mobile terminal 6 can use the second communication unit 69b to wirelessly connect to an access point 110 and communicate with the local area network to which the access point 110 participates. In addition to the first communication unit 69a and the second communication unit 69b, the mobile terminal 6 may also be provided with other means of communication, such as an infrared communication unit or short-range communication means such as Bluetooth®.

[0030] The portable terminal 6 of this embodiment is configured to include input switches 70 in addition to the touch panel 68n as an input means. Various types of switches can be provided for these input switches 70 (input switches 70a, 70b, etc. in Figure 2), such as a power switch, a volume control switch, and a switch whose input function is changed according to the program.

[0031] Furthermore, the mobile terminal 6 of this embodiment has a vibration unit 72 and an LED 73 to provide various notifications to the user holding the mobile terminal 6. The vibration unit 72 is a component formed by attaching an eccentric weight to the shaft of a motor. By rotating the motor, it is possible to transmit vibrations that can be felt by the user. The LED 73 is positioned on the upper front of the mobile terminal 6 as shown in Figure 2, and can provide various notifications to the user by flashing.

[0032] In this embodiment, by installing a karaoke program on such a mobile terminal 6, it becomes possible to perform karaoke between the mobile terminals 6. Now, let's explain the details of the various processes involved when performing karaoke using the mobile terminal 6.

[0033] Figure 4 is a flowchart illustrating the room processing in this embodiment. Here, we will explain the case where karaoke is performed using mobile terminals 6a and 6b. In this embodiment, users gather in a virtually formed room, and users gathered in the same room can perform karaoke by sharing audio (which may also include video).

[0034] This section describes the case where a user using mobile device 6a creates a room and a user using mobile device 6b joins that room. It is assumed that users using mobile devices 6a and 6b already have accounts with the management server 11. Users without an account can access the management server 11 and create a new account.

[0035] First, the mobile terminal 6a sends a login instruction (S11) to the management server 11. The login instruction is performed by entering account information (user ID, password, etc.). Upon receiving the login instruction (S11), the management server 11 executes the login process (S21). During the login process, authentication is performed based on the account information, and if the authentication is successful, the user can use various functions of the karaoke program.

[0036] Figure 10(A) shows the room selection screen that appears after logging in. The room selection screen displays a room selection field 68a and a new room creation button 68b. Room selection field 68a shows rooms that have already been created, and the status of those rooms is displayed. The room selection field displays the room name, icons of the users participating in the room, the room's status ("OPEN", "Preparing"), and the date the room will be held. Room selection field 68a with a key icon in the upper left corner indicates a room that requires a password or a room that can only be joined by users who have been previously invited.

[0037] Users can join a room by operating the room selection field 68a. The new room creation button 68b is used to create a new room. By operating the new room creation button 68b, the mobile terminal 6a can send a room creation instruction (S12) to the management server 11. Based on the room creation instruction (S12), the management server 11 executes the room creation process (S22). Once the room creation process (S22) is complete, the room selection field 68a shown in Figure 10(A) is displayed.

[0038] Meanwhile, the mobile device 6b joining the room first issues a login instruction (S11) to the management server 11 by entering its account information. The management server 11 performs authentication based on the account information, similar to the case of mobile device 6a, and if the authentication is successful, logs in mobile device 6b. The user refers to the room selection screen shown in Figure 10(A) and issues a room joining instruction by selecting the room in the room selection field 68a (S13). In the example in Figure 4, the user joins a room created by the user of mobile device 6a, and the management server 11 executes a user addition process (S23) to allow the user of mobile device 6b to join the room created by the user of mobile device 6a.

[0039] When users of mobile devices 6a and 6b enter the same room, it becomes possible to perform karaoke processing (S12) between mobile devices 6a and 6b. Karaoke processing (S12) mainly consists of reservation processing and playback processing. Reservation processing is the process of reserving a song to be sung, and song playback processing is the process of playing the songs registered in the reservation list in order based on the reservation processing.

[0040] Figure 5 is a flowchart illustrating the reservation process in this embodiment. In the reservation process, it is possible to add a song to the reservation list managed by the management server 11 by searching for a song and selecting the song to sing. Users who wish to reserve a song use the song search screen shown in Figure 10(B) to search for a song (S101). The song search screen in Figure 10(B) can be displayed by operating the song search button 68k shown in Figure 12(D), which will be explained later.

[0041] The music search screen includes a search category selection field 68c and a song selection field 68d. The search category selection field offers search categories such as "Favorites," "Songs," "Artists," and "Albums," allowing users to perform music searches according to their selected category. Figure 10(B) shows the state when "Songs" is selected as the search category, and the song selection field 68d is displayed in alphabetical order (Japanese alphabetical order).

[0042] The song selection field 68d displays an icon (album art, etc.), song title, and artist name for each song. When the user selects a desired song from the song selection field 68d (S102: Yes), a song confirmation screen for that song is displayed (S103).

[0043] Figure 11(C) shows the music confirmation screen of this embodiment. The music confirmation screen displays a music confirmation field 68e. The music confirmation field displays an icon (album art, etc.), the music title, the artist name, and a reserve button 68g and a cancel button 68f for the selected music. The user confirms the information displayed on this music confirmation screen and, if they wish to make a reservation, operates the reserve button 68g (S104:Yes) to make the reservation. Once a reservation is made, the reservation information is sent to the management server 11. The reservation information consists of a user ID indicating the user who made the reservation and a music ID indicating the music. The management server 11 registers the received reservation information in the reservation list.

[0044] Figure 6 is a flowchart illustrating the playback process of this embodiment. In this embodiment, each mobile terminal 6 participating in the same room is classified as either a singer or a listener. A singer is a user who sings along to the song being played. In this embodiment, the user who reserved the song is identified as the main singer. In this embodiment, it is also possible to designate users other than the main singer as secondary singers among the users participating in the same room.

[0045] For example, in the music playback screen shown in Figure 12(D), it is possible to participate in singing as a secondary singer by turning on the microphone selection button 684. In this way, in this embodiment, multiple users can participate in singing as the main singer or secondary singer. A listener is a user who does not sing but listens to the singing voice of the singer. Users other than the main singer and secondary singer are classified as listeners.

[0046] In the playback process performed on the mobile terminal 6, the availability of the next song to be played is determined based on the reservation list managed by the management server 11 (S201). If there is a song to be played next (S201: Yes) and the mobile terminal 6 is the mobile terminal 6 that reserved the song (S202: Yes), the mobile terminal 6 is set as the main singer (S204) and the main playback process (S210) is executed. On the other hand, if singing together is required (S205: Yes), for example, if the microphone selection button 684 is set to ON, the secondary playback process (S220) is executed on the mobile terminal 6. All other mobile terminals 6 are set as listeners (S206) and the listener-side playback process (S230) is executed.

[0047] Figure 7 is a diagram illustrating the delay amount (delay time) of singing audio data between the mobile terminals 6 in this embodiment. In Figure 7, for the sake of simplicity, a state in which three people are participating in the same room is shown. Note that one or more people can participate in a room, and it is not limited to this configuration with three participants. Here, the mobile terminals are assigned the codes 6a for the main singer A, 6b for the secondary singer B, and 6c for the listener C. These codes will also be used to distinguish them in the following explanation.

[0048] Incidentally, it is conceivable that delays occur in the singing voice data transmitted and received between mobile terminals 6a to 6c due to the delay amounts between each mobile terminal 6a to 6c. Figure 7 shows the delay amounts of the singing voice data occurring between each mobile terminal 6a to 6c. For singing voice data A transmitted from the main singer A's mobile terminal 6a, a delay amount D_ab occurs for the secondary singer B's mobile terminal 6b. Also, for singing voice data A, a delay amount D_ac occurs for the listener C's mobile terminal 6c. On the other hand, for singing voice data B transmitted from the secondary singer B's mobile terminal 6b, a delay amount D_ba occurs for the main singer A's mobile terminal 6a. Also, for singing voice data B, a delay amount D_bc occurs for the listener C's mobile terminal 6c.

[0049] In this embodiment, the delay is taken into account based on the singing voice data A, and the singing voice data B is played back immediately after reception. Figure 8 shows a time chart showing the accompaniment data and singing voice data for the mobile terminals 6a to 6c of this embodiment. In the time chart, the time charts for the accompaniment data, singing voice data A, and singing voice data B are shown for the main singer A, secondary singer B, and listener C, respectively. In Figures 8 and 9, the accompaniment data is shown in a simplified piano roll format, and the singing voice data is shown in waveform format.

[0050] As shown in Figure 8, in the case of the main singer A's mobile terminal 6a, singing is performed in synchronization with the accompaniment, so there is no delay in the singing audio data A relative to the accompaniment. On the other hand, for the singing audio data B transmitted from the secondary singer B's mobile terminal 6b and received and played back on mobile terminal 6a, a delay amount D_ba occurs as explained in Figure 7.

[0051] Meanwhile, on the mobile terminal 6b of the secondary singer B, the accompaniment data is played back in approximately synchronization with the accompaniment data of the main singer A. As in this embodiment, the accompaniment data is played back based on a playback synchronization signal, and the playback synchronization signal has a smaller data size than the singing voice data and can be transmitted and received with less delay. Therefore, the amount of delay required for its communication is considerably smaller than that of the singing voice data. Here, the delay of the accompaniment data is ignored in this description.

[0052] In the accompanying singer's mobile terminal 6b, singing is performed in synchronization with the accompaniment, so there is no delay in the singing audio data B relative to the accompaniment. On the other hand, the singing audio data A transmitted from the main singer A's mobile terminal 6a and received and played back on the mobile terminal 6b experiences a delay of D_ab, as explained in Figure 7. In this embodiment, the accompanying singer B's mobile terminal 6b also plays back the singing audio data A without considering the delay of D_ab, similar to the main singer A's mobile terminal 6a.

[0053] Then, on listener C's mobile terminal 6b, the accompaniment data is played back with a delay of D_ac relative to the accompaniment data of the main singer A, which is the delay amount D_ac that occurs for the singing voice data A. Specifically, using the playback synchronization signal transmitted from mobile terminal 6a, the accompaniment data is played back with a delay of D_ac relative to the accompaniment data of mobile terminal 6a. As a result, the singing voice data A transmitted from mobile terminal 6a is played back in near-synchronization with the accompaniment data. On the other hand, the singing voice data B is played back with a delay of D_bc. Here, the delay amount D_ac for the singing voice data A and the delay amount D_bc for the singing voice data B played back on mobile terminal 6c are unlikely to differ significantly. Thus, in this embodiment, the delay amount D_ac for the singing voice data A is used to achieve synchronized playback of the accompaniment data and the singing voice data A and B.

[0054] Next, the problems of the present invention will be explained. Figure 9 is a time chart illustrating the problems of the present invention. As explained in Figure 8, on the mobile terminal 6b of the accompanying singer B, the singing voice data A is played back with a delay of D_ab. The singer is required to sing in accordance with the playback of the accompaniment data, but some singers may sing along to the delayed playback of the singing voice data A.

[0055] Figure 9 is a time chart illustrating this situation, showing an example where co-singer B sings along with vocal data A. While co-singer B should ideally sing in accordance with the accompaniment data, this figure shows a case where co-singer B sings along with vocal data A, resulting in a delay of D_ab relative to the accompaniment data. In such a case, on listener C's mobile device 6c, vocal data B will be played back with a delay of D_ab relative to the accompaniment data, causing a significant delay in vocal data B. The above explains the problem of co-singer B singing along with lead singer A, but it is also possible for lead singer A to sing along with co-singer B.

[0056] This invention takes such circumstances into consideration and aims to alert singers to the fact that they should not be distracted by the delayed playback of other people's singing voices.

[0057] Figure 13 is a flowchart illustrating the various playback processes of this embodiment. Figure 13(A) is a flowchart illustrating the main playback process executed on the mobile terminal 6a set as the main singer. In the main playback process, first, playback of the music data for the reserved song is started (S211). The music data consists of accompaniment data and lyrics data. The accompaniment data is played back by the sound processing unit 66, which acts as the playback unit, and the lyrics data is played back by the image processing unit 64. The music data is distributed from the management server 11, etc., and may be distributed in either stream format or storage format.

[0058] After starting playback of the music data (S211), a playback synchronization signal is sent to the mobile terminals 6b and 6c participating in the same room (S212). The synchronization playback signal is data used to synchronize the playback of the music data among the mobile terminals 6a to 6c participating in the same room. The other mobile terminals 6b and 6c play the music data based on this playback synchronization signal. At this time, the playback synchronization signal is the timing when playback of the accompaniment data starts on mobile terminal 6a. In this embodiment, the playback synchronization signal is used to control the playback timing of the accompaniment data on the other mobile terminals 6b and 6c. Specifically, the mobile terminal 6b of singer B uses the playback synchronization signal to start playing the accompaniment data in synchronization (approximately simultaneously) with mobile terminal 6a. On the other hand, the mobile terminal 6c of listener C uses the playback synchronization signal to start playing the accompaniment data at a later timing (delayed) than the playback of the accompaniment data on mobile terminal 6a.

[0059] In the main playback process, the singing voice data A input from microphone 67b is transmitted to mobile terminals 6b and 6c participating in the same room (S213). Various methods can be used for transmitting the singing voice data, such as transmitting it via the management server 11, or transmitting it directly between mobile terminals 6a to 6c (peer to peer). Furthermore, during the playback period of the song, the singing voice data B transmitted from the assistant singer's mobile terminal 6b is played back (S214).

[0060] In this embodiment, the tendency to sing along impulsively, as mentioned earlier, is suppressed by performing a warning process (S300, corresponding to "display processing") during the playback period of the music. Figure 14 is a flowchart illustrating the warning process (S300) in this embodiment. The warning process (S300) in this embodiment measures the delay amount D_ba) from another mobile terminal 6 (mobile terminal 6b in the main playback process) (S301). If the measured delay amount D_ba is smaller than the first threshold (S302: No), nothing is done.

[0061] On the other hand, if the measured delay amount D_ba is greater than or equal to the first threshold (S302: Yes) but less than the second threshold (S303: No), the first message is displayed to the main singer (S304). Also, if the measured delay amount D_ba is greater than or equal to the first threshold (S302: Yes) and greater than or equal to the second threshold (S303: Yes), the second message is displayed to the main singer (S305).

[0062] Here, we will describe the music playback screen of the mobile terminal 6, which displays the first and second warning messages. Figure 12(D) is the music playback screen displayed during music playback on the mobile terminal 6 of this embodiment. At the top of the music playback screen, the room name and a music search button 68k are displayed. By operating the music search button 68k, the music search screen described in Figure 10(A) is displayed, and it is possible to reserve music.

[0063] In addition, the music playback screen is broadly divided into three sections: a music-related display section 68h, an upper display section 68i, and a lower display section 68j. The music-related display section 68h consists of a lyrics display section 681a, a warning display section 681b, and a progress bar 682. The lyrics display section 681a displays the lyrics to be sung for the currently playing song. The lyrics display section 681a may also display a background image. The progress bar 682 indicates the progress of music playback.

[0064] In this embodiment, as explained in the warning process (S300) in Figure 14, if the delay amount is greater than or equal to the first threshold (S302: Yes), a message is displayed in the warning display area 681b. The state in Figure 12(D) shows the case when the first message is displayed (S304). As an example, the message displayed as the first message is "The other singer's voice is slightly delayed. Please sing carefully." By referring to the first message displayed in the warning display area 681b, the accompanying singer can sing while paying attention to the delay in the other singer's voice.

[0065] Furthermore, if the delay amount exceeds the second threshold (S303: Yes), a second message is displayed to draw even more attention than the first message (at least one of the following differs from the first message: content, display format, display color, etc.). For example, the second message displayed might read, "The voices of other singers are significantly delayed. Please sing carefully." The message displayed in the warning display area 681b allows the singer to understand the delay status of other singers and sing in a way that avoids being influenced by them.

[0066] Furthermore, in this embodiment, the area occupied by the upper display area 68i and the lower display area 68j can be changed. Normally, the state shown in Figure 12(D) is when both the upper display area 68i and the lower display area 68j are displayed. When the comment tab 686, reservation confirmation tab 687, and remote control tab 688 provided in the lower display area 68j are operated, the display content of the lower display area 68j is changed accordingly. Figure 12(D) shows the state when the comment tab 686 has been operated.

[0067] The upper display area 68i shows a status display / selection button 683, a microphone selection button 684, and a participating user icon 685. The microphone selection button 684 allows the user to manually switch the microphone 67b on or off. In this embodiment, the mobile terminal 6 of the user who reserved the song is automatically set to on, and the mobile terminals 6 of other users are automatically set to off. Furthermore, the mobile terminal 6 of the user who reserved the song is recognized as the main singer, and the mobile terminals 6 of other users are recognized as listeners. Even a listener's mobile terminal 6 can sing as a secondary singer by switching the microphone selection button 684 to on.

[0068] The status display / selection button 683 indicates the status of the mobile device 6. When the microphone selection button 684 is on, "Singing" is marked; when it is off, "Listening" is marked. Figure 12(D) shows the state where "Singing" is marked. The user can also manually switch between "Singing," "Listening," and "Away."

[0069] The participating user icons represent all users currently in the room. In the example in Figure 12(D), four users are participating. Users whose icons are shown with a thick line (e.g., "JOY" in Figure 12(D)) are singing (set to singer mode). Additionally, icons with a speaker and an "X" (e.g., "Monkichi" in Figure 12(D)) indicate that the user is away.

[0070] The lower display area 68j has three tabs (Comment tab 686, Reservation confirmation tab 687, and Remote control tab 688). In Figure 12(D), the Comment tab 686 is selected, and the lower display area 68j displays the Comment display area 689 and the Comment input area 690. The Comment display area 689 displays user comments entered by the user using the Comment input area 690, as well as system comments related to operations during music playback.

[0071] In Figure 12(D), the comment display field 689 shows system comments related to the reservation process, including the user name and the name of the reserved song / artist. System comments and user comments are displayed sequentially in chronological order in the comment display field 689, allowing users to communicate smoothly by viewing these system and user comments.

[0072] The subordinate playback process (S220), which is performed on the mobile terminal 6b set as the subordinate singer, also performs a warning process (S300) similar to the main playback process (S210) explained in Figure 13(A). Figure 13(B) is a flowchart showing the subordinate playback process (S220). In the subordinate playback process, the playback of the music data is started (S222) by first receiving the playback synchronization signal transmitted from the mobile terminal 6a (S221). In the subordinate playback process, the singing voice data B input from the microphone 67b is transmitted to the mobile terminals 6a and 6c participating in the same room (S223).

[0073] Furthermore, during the playback period of the song, the singing voice data A transmitted from the main singer's mobile terminal 6b is played (S224). During the playback period, a warning process (S300) is performed in the same way as the main playback process, thereby suppressing the user from singing along to the singing voice data A.

[0074] Next, we will explain the listener-side playback processing performed on listener C's mobile terminal 6c. Figure 15 is a flowchart showing the listener-side playback processing (S230) performed on listener C's mobile terminal 6c. In the listener-side playback processing, first, the delay amount of the singing voice data A on mobile terminal 6c is determined from mobile terminal 6a (S231). Then, based on the playback synchronization signal, the music data is played back (S233), and at that time, delayed playback of the music data is performed based on the delay amount D_ac.

[0075] During music playback, vocal audio data A from mobile terminal 6a and vocal audio data B from mobile terminal 6b are received and played back (S234, S235). The reception and playback of vocal audio data A (S234) and vocal audio data B (S235) continue until music playback ends (S236: Yes). By delaying the playback of the music data in this way, it becomes possible to synchronize the playback of the accompaniment data and the vocal audio data, as explained in Figure 8.

[0076] Furthermore, in this embodiment, by performing a warning process (S300) in the main playback process (S210) and the secondary playback process (S220), it is possible to suppress delays in singing caused by being influenced by other singers, for example, as explained in Figure 9.

[0077] The above explanation has been given using a karaoke system using a mobile terminal 6 as an example. With the karaoke program of this embodiment, singers (main singer A, secondary singer B) can understand the delay status of other singers by the message displayed in the warning display field 681b and sing without being influenced by other singers.

[0078] [First variation] Next, various modifications of the karaoke system according to the present invention will be described. Figure 16 is a diagram showing various display forms for the first modification. In the above-described embodiment, a message (first and second message) was used to notify the user that other singing voices were delayed. However, the notification display can be not limited to this form, and various other forms can be adopted.

[0079] Figure 16(A) shows a configuration where the delay time of other singers is displayed as the delay amount. Figure 16(B) shows a format that displays large and small delay amounts. The display format for the delay amount may also be multi-level, such as "none," "small," "medium," and "large," depending on the delay amount. Figure 16(C) shows the signal strength in a format similar to that of a mobile device, indicated by the number of antenna bars. The greater the delay of other singers, the fewer antenna bars are displayed. Alternatively, the display could be formatted so that the greater the delay of other singers, the more antenna bars are displayed.

[0080] Figure 16(D) shows a graph displaying the delay amounts of other singers. The bar extends to the right according to the delay amounts of the other singers. Figure 16(E) shows a display format that indicates the delay amount of other singers by signal format (color). The difference in color (e.g., blue, yellow, red) makes it possible to understand the delay amount of other singers. The above describes various display formats based on the amount of delay, but the display format is not limited to those described above, and various other display formats can be adopted. It is also possible to use a combination of the various display formats described above.

[0081] [Second variation] In the aforementioned embodiment, the alert processing (S300) displayed a message based on the amount of delay in the singing voice data of other singers. In this form, it is possible that a message may be displayed even if the singer is singing properly (singing without delay to the accompaniment). Therefore, the alert processing may perform a singing timing detection process to detect the singing timing of the singer's own singing voice input from the microphone in relation to the playback of song information played back in the song playback process. If the singing timing detected by the singing timing detection process is within a predetermined range, that is, if it is determined that the singer is singing without delay, a message may not be displayed. In this modified form, even if there is a delay in the singing voice data of other singers, if the singer is singing properly without delay, a message will not be displayed, and unnecessary alerts will not be issued. In addition, if it is determined that the singer is singing without delay, instead of not displaying a message, another message may be displayed, or the message may be displayed in a different form.

[0082] [Third variation] In the above-described embodiment, two singers were described: main singer A using mobile terminal 6a and secondary singer B using mobile terminal 6b. The number of singers is not limited to two; there may be three or more. If there are three or more singers, one singer will receive singing voice data from multiple other singers. Therefore, one singer may receive multiple singing voice data with different delay amounts.

[0083] In this case, the volume of the singing audio data of other singers (average volume within a predetermined time, etc.) is measured, and a warning process is executed based on the delay amount of the singing audio data with the highest volume among multiple singing audio data with different delay amounts. This modified method makes it possible to suppress being influenced by singing audio data with a high volume. Alternatively, the warning process may be executed based on the average delay amount among the singing audio data of other singers. Or, the warning process may be executed based on the largest or smallest delay amount among the singing audio data of other singers.

[0084] [Fourth variation] As explained in Figure 12, the singer will sing while looking at the lyrics displayed in the lyrics display area 681a. Because the singer will be focusing on the lyrics, there is a possibility that they may miss the display in the warning display area 681b. Therefore, the warning processing (corresponding to "display processing") may be performed on the lyrics based on the amount of delay. For example, the lyrics may be displayed larger as the amount of delay increases. In karaoke, the displayed lyrics are sometimes color-coded according to the progress of the song. By color-coding the lyrics, the singer can grasp the appropriate timing for singing.

[0085] If the delay in the vocal data of other singers becomes large, increasing the size of the lyrics can make the timing of singing clearer, thus preventing singers from being influenced by other singers. Alternatively, if the delay in the vocal data of other singers becomes large, the timing of the color change could be accelerated. By accelerating the color change, singers are expected to sing earlier, potentially eliminating delays caused by being influenced by other singers.

[0086] As an example, the timing of the color change could be advanced by a time corresponding to the delay amount of the singing audio data of other singers. Furthermore, as in the embodiment described above, if the delay amount is greater than or equal to the first threshold, the color change timing could be advanced by a first timing. Also, if the delay amount is greater than or equal to the second threshold, the color change timing could be advanced by a second timing, which is earlier than the first timing.

[0087] [Fifth variation] In the above-described embodiment, a message was displayed via a warning process (S300) to warn users not to sing along with other singers. In addition to such warnings via displays, it is also conceivable to use voice-based singing assistance. For example, the volume of the accompaniment could be increased (or decreased) as the delay amount of the singing voice data of other singers increases.

[0088] Thus, the greater the delay, the greater the volume of the accompaniment relative to the vocal data of other singers, thereby reducing the influence of their vocal data. In addition, karaoke machines have a function to play a guide melody (vocal melody) included in the song data to assist with singing. Using such a singing assistance function, if the delay in the vocal data of other singers becomes large, it is conceivable to play the guide melody, or to increase the volume of the guide melody as the delay in the vocal data of other singers increases.

[0089] [Sixth variation] In the embodiments described above, as well as in the first, fourth, and fifth modifications, various countermeasures based on the delay amount of singing audio data (changes in display format, changes in lyric display format, and singing assistance using voice) were explained. Such countermeasures may be effective for some users but ineffective (or have little effect) for other users.

[0090] Therefore, it is conceivable that the singer's mobile device 6 could be configured to determine which countermeasures are effective. For example, after the start of music playback, the countermeasures could be switched at predetermined intervals, and the singing delay could be measured to obtain the singing delay for each countermeasure. Then, by using the countermeasure with the smallest singing delay, it would be possible to utilize the most effective countermeasure for the singing user.

[0091] As an example, after music playback begins, the display format is changed for a first predetermined time and the first delay time between the singing voice input from microphone 67b and the accompanying data is measured. After the first predetermined time has elapsed, the display format of the lyrics is changed for a second predetermined time and the second delay time between the singing voice input from microphone 67b and the accompanying data is measured. After the second predetermined time has elapsed, vocal assistance is provided for a third predetermined time and the third delay time between the singing voice input from microphone 67b and the accompanying data is measured. The countermeasure corresponding to the smallest delay time among the first to third delay times is applied after the third predetermined time has elapsed.

[0092] In the example above, the steps were performed in the order of changing the display format, changing the lyric display format, and providing vocal singing assistance, but this order is not mandatory. The countermeasure is only applied to the first song sung, and the same countermeasure will be applied to subsequent songs. The sixth variation may be configured to be executed when the user selects "Automatic" on the warning settings screen, where the user can select the warning setting.

[0093] [7th variation] In the sixth modification, the embodiments described above, as well as in the first, fourth, fifth, and sixth modifications, various countermeasures based on the delay amount of the singing voice data (change in display format, change in lyric display format, and voice-based singing assistance) were explained. Multiple countermeasures may be implemented simultaneously. For example, both the change in display format and the change in lyric display format may be implemented simultaneously. Also, both the change in display format and voice-based singing assistance may be implemented simultaneously. Furthermore, both the change in lyric display format and voice-based singing assistance may be implemented simultaneously.

[0094] Furthermore, the display format change, the lyric display format change, and the voice singing assistance may all be implemented simultaneously. Combining multiple measures can more effectively alert users. Note that the change in the timing of color changes and the voice singing assistance do not need to be implemented simultaneously. This is because changing the timing of color changes makes the color change timing earlier than the original timing, and implementing this simultaneously with the voice singing assistance may confuse users.

[0095] [8th variation] In a warning settings screen where users can select warning settings, the system may be configured so that when a user selects their preferred countermeasure from among various options, the selected countermeasure is implemented. For example, the warning settings screen may display multiple options (changing the display format, changing the display format of lyrics, providing voice singing assistance), and the countermeasure implemented according to the user's selection may be implemented.

[0096] Furthermore, the display format of the lyrics may be configured to allow users to select either the size of the lyrics or the timing of the color change of the lyrics. In that case, only one of the two options may be selected: either the timing of the color change of the lyrics or the voice-assisted singing. Additionally, the multiple options may include "Automatic" as shown in the sixth modified example.

[0097] The karaoke program used in the mobile terminal 6 of this embodiment has been described above, but the present invention is not limited to karaoke programs, and also includes karaoke devices equipped with karaoke programs, or karaoke display methods. [Explanation of Symbols]

[0098] 6(6a~6c): Mobile device 68i: Top display area 11: Management Server 68j: Bottom Display Area 12a: Router 68k: Music search button 12b: Wireless router 68m: Display unit 13: Wireless base station 68n: Touch panel 61: CPU 69a: 1st communication section 62: RAM 69b: Second Communications Department 63: Memory unit 70 (70a, 70b): Input switches 64: Image processing unit 72: Vibration unit 65F: Front camera 73: LED 65R: Rear camera 110: Access point 66: Acoustic processing unit 140: Mobile base station 67a: Speaker 681a: Lyrics display area 67b: Microphone 681b: Warning display area 68: Touch panel display 682: Progress bar 68a: Room selection field 683: Select button 68b: Create New Room button 684: Select Microphone button 68c: Search category selection field 685: Participating user icon 68d: Song selection field 686: Comments tab 68e: Track confirmation section 687: Reservation confirmation tab 68f: Cancel button 688: Remote control tab 68g: Reservation button 689: Comment display field 68h: Song-related display area 690: Comment input area

Claims

1. Music playback processing that plays back song information, Receiving process for receiving singing audio from other terminal devices, Audio playback processing that plays back the received singing audio, The receiving process includes a delay detection process that detects the amount of delay in the singing audio received, Based on the delay amount detected by the delay amount detection process, the information processing device is instructed to execute a display process that displays the information. A program for karaoke.

2. The display process changes the display format according to the amount of delay. The karaoke program according to claim 1.

3. The information processing device is used to perform the transmission process of sending the user's singing voice, input from the microphone, to another terminal device. The karaoke program according to claim 1.

4. The information processing device is instructed to perform a lyric playback process that plays lyric information corresponding to the song information to be played during the music playback process. The display process changes the display format of the lyrics information being played based on the delay amount detected by the delay amount detection process. The karaoke program according to claim 1.

5. The display format of the lyrics information that is changed during the display process is the size of the lyrics. The karaoke program according to claim 4.

6. The display format of the lyrics information that is changed during the display process is determined by the timing of the color change of the lyrics. The karaoke program according to claim 4.

7. The audio playback process changes the volume of the singing audio to be played based on the delay amount detected by the delay amount detection process. The karaoke program according to claim 1.

8. A singing timing detection process that detects the timing of the singer's voice input from the microphone in relation to the playback of song information played during the music playback process, If the singing timing detected by the singing timing detection process falls within a predetermined range, the display will either not be shown, or it will be displayed in a different format. The karaoke program according to claim 1.

9. A singing timing detection process that detects the timing of the singer's voice input from the microphone in relation to the playback of song information played during the music playback process, If the singing timing detected by the singing timing detection process exceeds a predetermined range, the audio playback process changes the volume of the singing audio to be played based on the delay amount detected by the delay amount detection process. The karaoke program according to claim 1.

10. Music playback processing that plays back song information, Receiving process for receiving singing audio from other terminal devices, Audio playback processing that plays back the received singing audio, The receiving process includes a delay detection process that detects the amount of delay in the singing audio received, Based on the delay amount detected by the delay amount detection process, a display process is executed to display the result. Karaoke machine.

11. Music playback processing that plays back song information, Receiving process for receiving singing audio from other terminal devices, Audio playback processing that plays back the received singing audio, The receiving process includes a delay detection process that detects the amount of delay in the singing audio received, Based on the delay amount detected by the delay amount detection process, a display process is performed to display the information. Display method for karaoke.

Images