Karaoke program and karaoke device

Abstract

To achieve smooth communication when Karaoke is performed among a plurality of terminal devices.SOLUTION: A Karaoke program can perform: music reproduction processing for reproducing and outputting music information; acoustic processing for outputting reproduced music sound and a voice signal inputted from a voice input part; communication processing for allowing voice communication with another terminal device; and display processing for forming a lyric display region in a display part of the terminal device. Lyrics reproduced in the music reproduction processing are displayed in the lyric display region, and an effective image based on an effect display indication is displayed when effect display indication operation is performed in at least another terminal device.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present invention relates to a karaoke program executable on a terminal device, and a karaoke device. [Background technology]

[0002] Traditionally, karaoke, where people enjoy singing along to musical accompaniment, has been performed in social settings. Currently, with the spread of infectious diseases, it can be difficult to gather in a karaoke booth to sing. For this reason, some karaoke lovers enjoy karaoke through online web conferences. In karaoke through web conferences, users use their personal computers to connect to a web conference server that hosts a web conference, play downloaded songs, and enjoy singing.

[0003] Meanwhile, in karaoke, where multiple users gather, various effects are used to liven up the singing atmosphere. Patent Document 1 discloses a karaoke device that can play back interjections (recorded information) posted by users during non-singing sections. Patent Document 2 discloses a karaoke transmission system that generates karaoke background images using captured images uploaded when using karaoke, thereby enriching the variety of karaoke background images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-191162 A [Patent Document 2] JP 2011-75701 A Summary of the Invention [Problem to be solved by the invention]

[0005] Usually, when family or friends gather to enjoy karaoke at a karaoke booth, people cheer on the singer and liven up the atmosphere by joining in, clapping, playing tambourines or maracas, etc. The singer can feel the listeners' emotions through such reactions and direct conversations, and becomes more immersed in singing.

[0006] On the other hand, in karaoke using web conferences, it is difficult for the singer to know whether the listeners are listening or not, and it is difficult to sense how they are feeling. It is also difficult for the listeners to convey to the singer that they are listening, which can make them feel uneasy.

[0007] The effects described in Patent Documents 1 and 2 are different from the effects of karaoke, which is usually performed in a karaoke booth or the like, in that the reactions from listeners are not real-time, and do not fully solve the problems with karaoke using web conferences described above.

[0008] The present invention has been made in consideration of the above circumstances, and has as its object to enable smooth communication between users who enjoy karaoke via communication. [Means for solving the problem]

[0009] For this reason, the karaoke program according to the present invention employs the following configuration. A karaoke program executable on a terminal device, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction; The display process displays, on a display unit of the terminal device, an icon of a user set in a terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves to the lyrics display area starting from the icon of the user who performed the effect display instruction operation. Furthermore, the karaoke program according to the present invention includes: A karaoke program executable on a terminal device, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, The lyrics display area displays lyrics reproduced in the music reproduction process, and when an effect display instruction operation is performed on at least another terminal device, a first effect image or A design different from the first effect image, which becomes the background of the displayed lyrics Display the second effect image, The first effect image and the second effect image are not prohibited from being displayed simultaneously, and a plurality of second effect images are prohibited from being displayed simultaneously.

[0010] Further, in the karaoke program according to the present invention, In the lyrics display area, the display position of the effect image is determined based on the position of the lyrics being displayed or a preset lyrics display position.

[0012] Further, in the karaoke program according to the present invention, The display process displays an effect image for a certain period of time after displaying an effect based on an effect display instruction.

[0017] The karaoke device according to the present invention further comprises: A karaoke device capable of reproducing and outputting music information, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction; The display process displays, on a display unit of the terminal device, an icon of a user set in a terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves to the lyrics display area starting from the icon of the user who performed the effect display instruction operation. The karaoke device according to the present invention further comprises: A karaoke device capable of reproducing and outputting music information, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction; The display process displays, on a display unit of the terminal device, an icon of a user set in a terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves to the lyrics display area starting from the icon of the user who performed the effect display instruction operation. The karaoke device according to the present invention further comprises: A karaoke device capable of reproducing and outputting music information, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, The lyrics display area displays lyrics reproduced in the music reproduction process, and when an effect display instruction operation is performed on at least another terminal device, a first effect image or A design different from the first effect image, which becomes the background of the displayed lyrics Display the second effect image, The first effect image and the second effect image are not prohibited from being displayed simultaneously, but a plurality of second effect images are prohibited from being displayed simultaneously. Effect of the Invention

[0018] According to the karaoke program and the karaoke device of the present invention, good communication is possible when singing karaoke among a plurality of terminal devices (karaoke devices), and users can fully enjoy karaoke. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 shows a system configuration according to an embodiment of the present invention. [Diagram 2] FIG. 1 is an external view of a mobile terminal according to an embodiment of the present invention; [Diagram 3] FIG. 1 is a block diagram showing a configuration of a mobile terminal according to an embodiment of the present invention. [Figure 4] FIG. 1 is a flow diagram showing room processing according to the present embodiment. [Diagram 5] A flow diagram showing reservation processing according to the present embodiment. [Figure 6] A flow diagram showing processing between mobile terminals according to the present embodiment. [Figure 7]1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 8] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 9] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 10] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 11] FIG. 2 is a flow diagram showing the effect display process of the present embodiment. [Figure 12] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 13] 1 is a diagram showing various screens of a mobile terminal according to an embodiment of the present invention; [Figure 14] FIG. 13 is a diagram showing a result screen based on effect history information of the present embodiment. [Figure 15] FIG. 13 is a diagram showing a display screen of the tally result according to another embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Fig. 1 is a diagram showing the system configuration of this embodiment. In this embodiment, the system allows karaoke to be performed between mobile terminals 6, and therefore communication between the mobile terminals 6 is necessary. Fig. 1 shows various examples of communication formats between the mobile terminals 6. For example, in the case of a private home, a router 12a connected to the Internet is provided, and a wireless router 12b is connected to the router 12a. The mobile terminal 6 can be connected to the Internet via the wireless router 12b.

[0021] In addition to these communication modes, it is also possible to communicate with the Internet using a mobile communication network. The mobile communication network is a communication mode in which the mobile terminal 6 is communicatively connected to the Internet by performing wireless communication between the mobile terminal 6 and a wireless base station 13.

[0022] By using these various communication formats, the mobile terminal 6 can be connected to the Internet. A management server 11 for managing the karaoke system is also connected to the Internet. A karaoke program is installed in the mobile terminal 6, which is connected to the Internet, and the mobile terminal 6 can perform karaoke between the mobile terminals 6 by communicating with the management server 11.

[0023] In this embodiment, a form in which karaoke is performed using a mobile terminal 6 as a terminal device will be described. Note that, in addition to the mobile terminal 6, a home game machine, a personal computer, or a karaoke device dedicated to karaoke may also be used as the terminal device. Furthermore, a terminal device in which a karaoke program is installed functions as a karaoke device.

[0024] Fig. 2 is a diagram showing the appearance (front view) of a mobile terminal 6 according to an embodiment of the present invention, and Fig. 3 is a block diagram showing the configuration of the mobile terminal 6 according to an embodiment of the present invention. A smartphone, a tablet terminal, a mobile phone, etc. can be used as the mobile terminal 6. By installing a karaoke program in the mobile terminal 6, it becomes possible to perform karaoke between the mobile terminals 6.

[0025] The mobile terminal 6 of this embodiment has a control means including a CPU 61, a RAM 62, a storage unit 63, an image processing unit 64, and an audio processing unit 66. A non-volatile storage means such as a NAND type flash memory can be adopted as the storage unit 63. The storage unit 63 stores the program of this embodiment and various data used by the program.

[0026] The CPU 61 is a means for controlling the entire mobile terminal 6, and executes various controls based on programs and data stored in the storage unit 63. The RAM 62 temporarily stores data generated under the control of the CPU 61.

[0027] The audio processing unit 66 is a means for inputting and outputting audio related information of the mobile terminal 6. It is a means for emitting sound from the speaker 67a under the control of the CPU 61. The audio processing unit 66 is also connected to a microphone 67b, and can pick up call voice and surrounding sounds from the microphone 67b. An external device such as a headset may be used for the speaker 67a and the microphone 67b.

[0028] The touch panel display unit 68 is configured with a display unit 68m that displays an image and a touch panel 68n disposed on the front or back surface of the display unit 68m. The touch panel 68n can be of various types that can recognize the touch position of the user, such as a capacitance type. The CPU 61 performs display on the display unit 68m and can determine which part of the displayed image has been touched by touch input from the touch panel 68n.

[0029] The image processing unit 64 is a means for inputting and outputting image-related data for the mobile terminal 6. The image processing unit 64 displays and outputs images formed by the CPU 61 on the display unit 68m. The image processing unit 64 has a function of capturing images or, if video, videos from the front camera 65F arranged on the front side (the touch panel display unit 68 side) of the mobile terminal 6 or the rear camera 65R arranged on the rear side of the mobile terminal 6, and storing the captured images or videos in the storage unit 63 as image or video information.

[0030] The mobile terminal 6 of this embodiment includes a first communication unit 69a and a second communication unit 69b as communication means for communicating with the outside. The first communication unit 69a is a communication means for connecting to a mobile communication network, and can perform various communications via the Internet network using a communication line of the mobile communication network by wirelessly connecting to a mobile base station 140. The second communication unit 69b is a communication means for connecting to Wifi. The mobile terminal 6 can wirelessly connect to an access point 110 using the second communication unit 69b, and can communicatively connect to a local area network in which the access point 110 participates. Note that, as communication means of the mobile terminal 6, in addition to the first communication unit 69a and the second communication unit 69b, a short-range communication means such as an infrared communication unit or Bluetooth (registered trademark) may be provided.

[0031] The mobile terminal 6 of this embodiment is configured to include an input switch 70 in addition to the touch panel 68n as an input means. The input switch 70 (input switches 70a, 70b, etc. in FIG. 2) can be provided with various switches such as a power switch, a volume adjustment switch, and a switch whose input purpose is changed depending on the program.

[0032] Furthermore, the mobile terminal 6 of this embodiment has a vibration unit 72 and an LED 73 to provide various notifications to the user who holds the mobile terminal 6. The vibration unit 72 is a member formed by attaching an eccentric weight to the shaft of a motor. By driving the motor to rotate, it is possible to transmit vibrations that can notify the user. The LED 73 is disposed above the front of the mobile terminal 6 as shown in FIG. 2, and can provide various notifications to the user by blinking.

[0033] In this embodiment, karaoke can be performed between the mobile terminals 6 by installing a karaoke program in such mobile terminals 6. Next, various processes performed when performing karaoke using the mobile terminals 6 will be described in detail.

[0034] 4 is a flow diagram showing the room processing of this embodiment. Here, a case where karaoke is performed using the mobile terminals 6a and 6b will be described. In this embodiment, by gathering in a virtually created room, users who gather in the same room can share audio (which may include video) and perform karaoke.

[0035] Here, a case will be described in which a user using the mobile terminal 6a creates a room, and a user using the mobile terminal 6b participates in the room. It is assumed that the users using the mobile terminals 6a and 6b already have accounts on the management server 11. Note that a user who does not have an account can access the management server 11 and create a new account.

[0036] First, the mobile terminal 6a issues a login instruction (S11) to the management server 11. The login instruction is issued by inputting account information (user ID, password, etc.). Upon receiving the login instruction (S11), the management server 11 executes a login process (S21). In the login process, authentication is performed based on the account information, and if the authentication is correct, the user is allowed to use various functions of the karaoke program.

[0037] FIG. 7(A) shows a room selection screen that is displayed after logging in. On the room selection screen, a room selection field 68a and a new room creation button 68b are displayed. The room selection field 68a shows a room that has already been created, and the status of the room is displayed. The room selection field displays the room name, icons of users participating in the room, the room's holding status ("OPEN", "In preparation"), the date the room is held, and the like. Also, the room selection field 68a with a lock mark displayed in the upper left corner indicates that the room requires a password or that only users who have been invited in advance can participate in the room.

[0038] A user can participate in a room by operating the room selection field 68a. The new room creation button 68b is a button for creating a new room. By operating the new room creation button 68b, the mobile terminal 6a can issue a room creation instruction (S12) to the management server 11. The management server 11 executes a room creation process (S22) based on the room creation instruction (S12). When the room creation process (S22) is completed, the room selection field 68a shown in FIG. 7(A) is displayed.

[0039] On the other hand, the mobile terminal 6b that will participate in the room first issues a login instruction (S11) to the management server 11 by inputting account information. The management server 11 performs authentication based on the account information, as in the case of the mobile terminal 6a, and if the authentication is proper, logs in the mobile terminal 6b. The user refers to the room selection screen shown in FIG. 7(A) and issues a room participation instruction by selecting the room selection field 68a that the user wants to enter (S13). In the example of FIG. 4, in the case of participating in a room created by the user of the mobile terminal 6a, the management server 11 executes a user addition process (S23) that allows the user of the mobile terminal 6b to participate in the room created by the user of the mobile terminal 6a.

[0040] As the users of the mobile terminals 6a and 6b enter the same room, it becomes possible to execute karaoke processing (S12) between the mobile terminals 6a and 6b.

[0041] The karaoke process (S12) mainly includes a reservation process and a song playback process. The reservation process is a process for reserving a song to be sung, and the song playback process is a process for playing songs registered in a reservation list in order based on the reservation process. In playing a song, a performance sound is played based on the performance information included in the song information, and lyrics are displayed based on the lyrics information included in the song information.

[0042] Fig. 5 is a flow diagram showing the reservation process of this embodiment. In the reservation process, a song is searched for and a song to be sung is selected, so that the song can be added to a reservation list managed by the management server 11. A user who wants to reserve a song searches for a song using the song search screen shown in Fig. 7(B) (S101). The song search screen in Fig. 7(B) can be displayed by operating a song search button 68k shown in Figs. 8(C) and 8(D) described later.

[0043] The song search screen has a search category selection field 68c and a song selection field 68d. The search category selection field has search categories such as "Favorites," "Songs," "Artists," and "Albums," and the user can select one of these search categories to perform a song search according to the selected search category. In the state shown in Fig. 7(B), "Songs" is selected as the search category, and the song selection field 68d is displayed in alphabetical order.

[0044] The song selection field 68d displays an icon (such as album art), song title, and artist name for each song. When the user selects the desired song selection field 68d (S102: Yes), a song confirmation screen for the selected song is displayed (S103).

[0045] FIG. 8(C) is a diagram showing a music confirmation screen of this embodiment. A music confirmation field 68e is displayed on the music confirmation screen. In the music confirmation field, an icon (such as album art), music title, artist name, a reservation button 68g, and a cancel button 68f are displayed for the selected music. The user checks the information displayed on this music confirmation screen, and when making a reservation, operates the reservation button 68g (S104: Yes) to reserve the music. When a reservation is made, reservation information is transmitted to the management server 11. The reservation information includes a user ID indicating the user who made the reservation, and a music ID indicating the music. The management server 11 registers the received reservation information in a reservation list.

[0046] 6 is a flow diagram showing the process between mobile terminals in this embodiment. Here, an example will be described in which two mobile terminals 6a and 6b are in the same room. In reality, the number of mobile terminals 6 that can be in the room can be two or more.

[0047] First, the mobile terminal 6a executes a reservation process (S100) to reserve a desired song. The reservation process (S100) is as described in FIG. 5, and the mobile terminal 6a transmits reservation information to the management server 11. The management server 11, which has received the reservation information, registers the reservation information in a reservation list (S201). At that time, in this embodiment, in conjunction with the reservation operation, a system comment is displayed on all the mobile terminals 6a and 6b that have entered the room (S101, S151).

[0048] The management server 11 that receives the reservation information transmits a system comment to all the mobile terminals 6a, 6b in the room in which the user who made the reservation is present (S202), and the mobile terminals 6a, 6b that receive the system comment display the system comment (S101, S151). Based on this configuration, it becomes possible for users who sing karaoke at a distance to be notified of operations related to music playback processing (in this case, reservation processing). Therefore, each user can know who is performing what operation, and smooth communication can be achieved.

[0049] The management server 11 checks the reservation list (S203), and if there is a song to be played next (S204), executes a song playback process for that song, as enclosed by a dashed line in Fig. 6. In the song playback process, first, the user who made the reservation is identified as the singer based on the user ID included in the reservation information (S205). Then, an instruction is given to the mobile terminal 6a used by the user identified as the singer to change to singer mode (S208). The mobile terminal 6a that has received the instruction changes to singer mode (S102).

[0050] On the other hand, the management server 11 identifies users other than the singer who are in the same room as the singer as listeners, and instructs the portable terminal 6b used by the user identified as the singer to change to listener mode (S207). The portable terminal 6a that receives the change instruction changes to the singer mode (S152). In each portable terminal 6, a function change process is executed to change at least one of the image display function and the sound output function executed in the portable terminal 6 based on the change instruction to listener mode or singer mode.

[0051] Here, the singer mode and the listener mode will be explained. In the portable terminal 6 set to the singer mode, the volume of the microphone 67b is sufficiently increased to make it easier to capture the user's singing voice. In addition, in the portable terminal 6 set to the singer mode, the singing voice captured by the microphone 67b is transmitted to the network (such as the management server 11) without adding any sound effects such as echo. In addition, in the singer mode, a communication mode with lower delay than the communication processing in the listener mode is set. By setting such a communication mode, the singing voice in the singer mode is transmitted to other portable terminals 6 with reduced delay.

[0052] On the other hand, in the portable terminal 6 set to the listener mode, the microphone 67b is turned off or the volume of the microphone 67b is lowered so as not to interfere with the singing. Also, the singing voice transmitted via the network is given a sound effect by the sound processor 66 and is output from the speaker 67a. Note that music is played back in both the portable terminal 6 set to the singer mode and the listener mode.

[0053] In this way, by adjusting the volume in the singer mode and the listener mode, it is possible to create an environment that is easy to listen to. In addition, by applying sound effects to the singing voice by the sound processor 66 of the mobile terminal 6 set to the listener mode, it is possible to obtain the following effects.

[0054] In audio transmission via a network, if an acoustic environment problem occurs that causes a delay in the singer's voice, it is possible that communication between the singer and the listener will not go well. In addition, when a singing voice to which sound effects (especially delay-type sound effects such as echo, delay, and reverb) have been applied is transmitted via a network, the singing voice may be significantly degraded. In this embodiment, by applying sound effects to the singing voice by the sound processing unit 66 of the mobile terminal 6 set to the listener mode, it is possible to reproduce a high-quality singing voice and provide a good acoustic environment on the mobile terminal 6 set to the listener mode.

[0055] Returning to the flow diagram of Fig. 6, when the settings of the singer mode and the listener mode are completed, the management server 11 starts transmitting music to each of the mobile terminals 6a, 6b (S208). In this embodiment, music is transmitted by streaming playback, but it is also possible to use a storage type in which music information is stored in advance in the mobile terminals 6a, 6b and played back. In either case, music playback is synchronized in each of the mobile terminals 6a, 6b.

[0056] Each of the mobile terminals 6a, 6b plays back music (S103, S153). Then, in the mobile terminal 6a set to the singer mode, the microphone 67b collects the singing voice (S104). The collected singing voice is transmitted to the management server 11, which then transfers it to another mobile terminal 6b that is in the same room (S209). The mobile terminal 6b that receives the singing voice applies sound effects to the received singing voice in the sound processor 66 (S154), and emits the sound from the speaker 67a together with the music playback sound.

[0057] Fig. 9(E) is a music playback screen displayed during music playback on the mobile terminal 6 of this embodiment. As shown in Fig. 8(D), the room name and a music search button 68k are displayed at the top of the music playback screen. By operating the music search button 68k, the music search screen described in Fig. 7(B) is displayed, and it is possible to reserve a song.

[0058] In addition, the music playback screen is broadly divided into a music-related display field 68h, an upper display field 68i, and a lower display field 68j. The music-related display field 68h includes a lyrics display field 681 and a progress bar 682. The lyrics display field 681 displays lyrics to be sung for the music being played. Note that the lyrics display field 681 may also display background video. The progress bar 682 indicates the progress of music playback.

[0059] The area occupied by the upper display column 68i and the lower display column 68j can be changed. Normally, the state shown in FIG. 8(D) is displayed, that is, both the upper display column 68i and the lower display column 68j are displayed. When the comment tab 686, the reservation confirmation tab 687, and the remote control tab 688 provided in the lower display column 68j are operated, the lower display column 68j is displayed using the area where the upper display column 68i was displayed, as shown in FIG. 10. FIG. 10 shows the state where the remote control tab 688 has been operated. By operating the comment tab 686, the reservation confirmation tab 687, and the remote control tab 688 again, the state returns to that shown in FIG. 9(E).

[0060] In the upper display field 68i, a status display / selection button 683, a microphone selection button 684, and a user icon 685 are displayed. The status display / selection button 683 is a button that indicates the status of the mobile terminal 6, and when the mobile terminal 6 is set to the singer mode, "singing" is marked, and when the mobile terminal 6 is set to the listener mode, "listening" is marked. The state in FIG. 8(D) is a state in which "singing" is marked. In addition, the user can manually switch between "singing", "listening", and "away". In this case, when the status is switched to "singing", the setting is changed to the singer mode, and when the status is switched to "listening" or "away", the setting is changed to the listener mode. In this embodiment, three states can be switched, but two statuses, "singing" and "listening", may be used, and various display forms can be adopted.

[0061] The microphone selection button 684 can be manually operated by the user to switch the microphone 67b on and off. In this embodiment, when the mode is set to singer mode, it is automatically set to on, and when the mode is set to listener mode, it is automatically set to off. The user icons 685 show all users participating in the room. In the example of FIG. 8(D), four users have participated. The user whose icon is shown with a thick line ("JOY" in the example of FIG. 8(D)) is singing (set to singer mode). Also, the icon with a speaker and a cross ("Monkichi" in the example of FIG. 8(D)) shows that the user is away. These display forms are merely examples, and the status such as singing, listening, and away may be shown by other display forms.

[0062] The lower display field 68j has three tabs (a comment tab 686, a reservation confirmation tab 687, and a remote control tab 688). In the state shown in FIG. 8(D), the comment tab 686 is selected, and a comment display field 689 and a comment input field 690 are displayed in the lower display field 68j. In addition to a user comment input by the user using the comment input field 690, a system comment associated with an operation related to a music playback process is displayed in the comment display field 689. In FIG. 8(D), the comment display field 689 displays a system comment associated with a reservation process, which includes the user name and the reserved music name / artist name. The comment display field 689 sequentially displays the system comment and user comment in chronological order, and the user can communicate smoothly by viewing these system comments and user comments.

[0063] Furthermore, the comment input field 690 is provided with a first button 690a and a second button 690b for displaying various effects in an area including the lyrics display field 681. When the first button 690a is operated, a first effect selection field for selecting a first display effect image is displayed on the operated mobile terminal 6. When the second button 690b is operated, a second effect selection field for selecting a second display effect image is displayed on the operated mobile terminal 6. The first display effect image and the second display effect image will be described in detail later.

[0064] 10(F) shows the state when the remote control tab 688 is operated. As described above, the lower display field 68j is displayed by expanding it to the area where the upper display field 68i is displayed. When the remote control tab 688 is selected, the lower display field 68j displays a singer volume 691, a speaker volume 692, a pitch change button 693, a BGM volume 694, a playback position adjustment button 695, a re-sing button 696, a pause button 697, and a performance stop button 698.

[0065] In this embodiment, the various controls and buttons displayed on the remote control tab 688 can be operated only by the mobile terminal 6 set to the singer mode. Therefore, the mobile terminal 6 set to the listener mode is prohibited from performing operations related to music playback, making it possible to prevent erroneous input by the listener.

[0066] The singer volume 691 is an operator for adjusting the volume of the singing voice input from the microphone 67b of the mobile terminal 6 set to the singer mode. The speaker volume 692 is an operator for adjusting the volume of the singing voice input from the microphone 67b of the mobile terminal 6 set to the listener mode. The singer volume 691 and the speaker volume 692 are set to the default state of each mode when the mobile terminal 6 is set to the singer mode or the listener mode, but can be operated only from the mobile terminal 6 set to the singer mode.

[0067] The pitch change button 693 is a button for changing the pitch of the song being played. The BGM volume 694 is an operator for adjusting the volume of the BGM played between songs. The playback position adjustment button 695 is an operator for changing the playback position. The current playback time and the total playback time are displayed to the left of the operators. The sing again button 696 can be operated to play the song being played from the beginning. The pause button 697 is a button for temporarily stopping the song being played. The performance stop button 698 is a button for stopping the song being played, and when the performance stop button 698 is operated, playback processing moves to the next reserved song.

[0068] During music playback, a user who has been set to singer mode can perform various setting operations related to music playback by using various controls and buttons displayed on the remote control tab 688 in Fig. 10(F). In the flow diagram of Fig. 6, if a setting operation is performed during music playback (S105: Yes), the mobile terminal 6a transmits a setting change instruction to the management server 11. The management server 11 executes the setting change according to the received setting change instruction (S210).

[0069] The setting change can be not only performed by the management server 11, such as changing the pitch of a song, but also can be performed on various mobile terminals 6a and 6b. For example, when the mobile terminal 6a set to the singer mode changes the sound settings of the sound effects (echo, delay, etc.) on the other mobile terminal 6b (set to the listener mode), the mobile terminal 6a set to the singer mode transmits sound setting information related to the sound effects to the management server 11 (S106). The management server 11 that has received the sound setting information transfers the sound setting information to the other mobile terminal 6b. The mobile terminal 6b that has received the sound setting information imparts sound effects to the singing voice signal or the sound of the song being played based on the received sound setting information.

[0070] The management server 11 transmits (S211) a system comment based on the setting change (S210) to each of the mobile terminals 6a, 6b. In each of the mobile terminals 6a, 6b, the system comment is displayed in the comment display field 689 described in FIG. 8(D) (S107, S155). Therefore, the users participating in the room can check the operation related to the music playback process from the system comment displayed in the comment display field 689. Therefore, the users participating in the room can communicate smoothly by checking the system comment.

[0071] When the management server 11 completes the music transmission (S212: Yes), it checks the reservation list to play the next music (S203). In addition, with the completion of the music playback transmission, each of the mobile terminals 6a and 6b ends the music playback (S108, S156: Yes).

[0072] FIG. 11 is a flow diagram showing the effect display process of this embodiment. In this embodiment, in order to facilitate communication between users, it is possible to display user comments in the comment display field 689 described in FIG. 9(E). A user can convey a message to other users by inputting characters in the comment input field 690. In this embodiment, an effect image is displayed in the lyrics display field 681, etc., based on the operation of the first button 690a and the second button 690b provided in the comment input field 690, and it is possible to convey emotions to other users in the same way as a message, and to facilitate communication.

[0073] FIG. 12(G) is a display screen when the first button 690a is operated in the mobile terminal 6 of this embodiment. When the first button 690a is operated, the first effect selection field 710 is displayed. In the first effect selection field 710, a plurality of first effect images 711 are displayed, and the user can select the first effect image 711 that matches his / her own feelings when listening to the singing of other users from among them. Since the first effect image 711 also has a text message, the user can convey his / her feelings about the singing to other users by selecting the first effect image 711 that matches his / her own feelings. Here, a case where the first effect image 711 with a dashed circle frame is selected in the first effect selection field 710 will be described.

[0074] When the first effect image 711 is selected, as shown by the dashed arrow in Fig. 12(H), an effect is produced in which the selected first effect image 711 moves from the user icon 685 (in this case, "Pirori") of the selected user toward the lyrics display field 681. Fig. 12(H) shows the screen of the mobile terminal 6 of the selected user, but the effect of the selected first effect image 711 moving from the user icon 685 (in this case, "Pirori") toward the lyrics display field 681 is also produced on the screens of the mobile terminals 6 of other users who are in the same room.

[0075] In this manner, in the present embodiment, by displaying the first effect image 711 moving from the position of the user icon 685 that selected the first effect image 711 into the lyrics display field 681, it becomes possible to convey to other users what emotion each user is expressing, thereby facilitating communication.

[0076] In this embodiment, first effect image 711 is moved and displayed by utilizing the position of lyrics displayed in lyrics display field 681 or a lyrics display position that is set in advance in lyrics display field 681. For example, by displaying first effect image 711 so as to avoid these lyrics display positions, it is possible to prevent the visibility of lyrics from being impaired by first effect image 711. Note that, when moving to avoid the first effect image 711, first effect image 711 and the displayed lyrics may be partially overlapped to an extent that the visibility of the lyrics can be confirmed.

[0077] Moreover, the first effect image 711 is displayed in the lyrics display column 681 for a predetermined period from the start of display, and thereafter disappears from the lyrics display column 681 with a predetermined disappearance effect (for example, gradually fading, etc.). The disappearance effect may be different for each first effect image 711. Moreover, in this embodiment, when the same user selects the first effect image 711 multiple times within a short period of time, or when multiple users select the first effect image 711 multiple times within the same period of time, the selected multiple first effect images 711 are displayed.

[0078] 13(I) is a display screen when the second button 690b is operated on the mobile terminal 6 of this embodiment. When the second button 690b is operated, the second effect selection field 720 is displayed. In the second effect selection field 720, samples of a plurality of second effect images 721 are displayed, and the user can select from these a second effect image 721 that matches his / her own feelings when listening to the singing of other users. Here, a case where a second effect image 721 enclosed in a dashed circular frame is selected in the second effect selection field 720 will be described.

[0079] 13(J), when the second effect image 721 is selected, the second effect image 721 is displayed as a background of the lyrics to be displayed in the lyrics display field 681. The second effect image 721 in this embodiment is in an animated format with movement, and is displayed for a predetermined period of time after the user makes a selection.

[0080] Since the second effect image 721 is an effect image that is displayed using the entire area of ​​the lyrics display field 681, if a plurality of second effect images 721 are displayed simultaneously, the screen may become cluttered. For this reason, in this embodiment, the simultaneous display of a plurality of second effect images 721 is prohibited. Note that, in this embodiment, the first effect image 711 and the second effect image 721 can be displayed simultaneously.

[0081] Returning to the flow diagram of the effect display process in Fig. 11, a description will be given of the display process of the first effect image 711 and the second effect image 721 described in Fig. 12 and Fig. 13. Note that the music playback period from S110 to S120 in the effect display process corresponds to the music playback periods from S103 to S108 and S153 to S156 on the mobile terminals 6a and 6b described in Fig. 6.

[0082] When music playback begins (S110), the mobile terminal 6 confirms the selection of the first effect image 711 using the first effect selection field 710 (S111: Yes), the receipt of an operation instruction for the first effect image 711 from another mobile terminal 6 (S114: Yes), the selection of the second effect image 721 using the second effect selection field 720 (S115: Yes), or the receipt of an operation instruction for the second effect image 721 from the other mobile terminal 6 (S119: Yes).

[0083] When the selection operation of the first effect image 711 is performed (S111: Yes), the mobile terminal 6 transmits an operation instruction to the management server 11 (S112). This operation instruction includes information about which user selected which first effect image 711. The management server 11 that has received the operation instruction transmits an operation instruction for the first effect image 711 to the mobile terminals 6 of the other users who are in the same room.

[0084] Then, in the mobile terminal 6 where the selection operation of the first effect image 711 has been performed, the first effect image 711 is displayed (S113). Also, when an operation instruction for the first effect image 711 is received from another mobile terminal 6 via the management server 11 (S114: Yes), the first effect image 711 is similarly displayed (S113).

[0085] On the other hand, when the selection operation of the second effect image 721 is performed (S115: Yes), the mobile terminal 6 first checks whether or not it is a period during which the second effect image 721 can be operated (S116). As described in FIG. 13, since the second effect image 721 is an effect image that is displayed over the entire lyrics display field 681, in this embodiment, it is prohibited to display it in duplicate. For example, when the second effect image 721 is displayed during the selection operation (S115: Yes), it is determined that it is not a period during which the image can be selected (S116: Yes). On the other hand, when it is a period during which the image can be selected (S116: Yes), an operation instruction is transmitted to the management server 11 in the same manner as in the case of the first effect image (S117). The management server 11 that has received the operation instruction transmits an operation instruction for the second effect image 721 to the mobile terminals 6 of other users who are in the same room.

[0086] Whether or not it is the selection period for the second effect image 721 may be confirmed with the management server 11, rather than being determined only by the mobile terminal 6. If an operation instruction has already been received from another mobile terminal 6, it is determined that it is not the selection period, thereby making it possible to prevent the second effect image 721 from being selected multiple times due to delays in communication lines, etc.

[0087] In the mobile terminal 6 where the selection operation of the second effect image 721 has been performed, the second effect image 721 is displayed (S118). Also, when an operation instruction for the second effect image 721 is received from another mobile terminal 6 via the management server 11 (S119: Yes), the second effect image 721 is similarly displayed (S118).

[0088] In this embodiment, after the first effect image 711 is displayed (S113) and after the second effect image 721 is displayed (S118), information associated with the display of the first effect image 711 and the second effect image 721 is recorded and updated as effect history information (S120). In this embodiment, the effect history information is stored and managed on the management server 11. The effect history information includes information such as the target song, the user who sang (reserved), and which user selected the first effect image 711 or the second effect image 721 and at what timing. By collecting this effect history information after the fact, it is possible to perform evaluation for each song or each user.

[0089] When the playback of the music ends (S121: Yes), a compilation process (S122) is performed on the effect history information recorded and updated during the music playback period. The compilation process (S122) of this embodiment is a process of tallying up the number of times the first effect image 711 was selected for the played music and the user who made the reservation (or the user who sang). In this embodiment, a type is set for the first effect image 711, and by compiling by type, it is possible to form an index of the emotions the user felt about the played music.

[0090] When the tallying process (S122) is completed, the effect history information and a result screen based on the tallying results are displayed on the touch panel display unit 68.

[0091] 14 is a diagram showing a result screen based on the effect history information of this embodiment. In this embodiment, the played music is divided into unit sections in time series, and the number of times the first effect image 711 was selected for each unit section is displayed in a graph. It can be said that a unit section that has been selected many times is a section that has attracted the attention of users. In addition, the unit section displays the first effect image 711 that has been selected the most and the user icon 685 of the user who performed the most selection operation. From these first effect image 711 and user icon 685, it is possible to confirm what emotions users had in the unit section and which user was most interested.

[0092] In the upper right corner of the result screen, a user ranking 731 and a singing ranking 732 based on the results of the tallying process (S122) are displayed. The user ranking 731 indicates a ranking based on the total number of times the selected first effect image 711 was selected in the room formed this time for the user who sang the song (in this case, "Pirori"). The singing ranking 732 indicates a ranking based on the total number of times each type of first effect image 711 was selected for the song (in this case, "Inu no Omawarisan") sung in the room formed this time.

[0093] In this manner, in this embodiment, by displaying the result screen, it is possible to notify the user of various situations and rankings based on the specification of the first effect image 711 performed during playback of the song, and the user can review the state of his / her singing based on the results, or can devote himself / herself to singing in the future in order to compete for the rankings.

[0094] In this embodiment, the result screen described in FIG. 14 is displayed when the song ends, but it may also be displayed in response to a user operation performed while participating in the room or at an appropriate time other than while participating in the room.

[0095] Fig. 15 is a diagram showing a display screen of the tally result of another embodiment. In Fig. 14, the ranking of the sung songs or the users who sang is displayed, but the tally result may be displayed and output in an appropriate format. Fig. 15(a) is a display screen of the tally result showing "Today's Like! Ranking" which is tallied for the first effect image 711 displayed in the upper right. In Fig. 15(a), the number of times each sung song is liked for the type to which the first effect image 711 belongs (in this case, "Like!") is tallied, and a ranking of songs that users felt "Like!" for each song is displayed.

[0096] Fig. 15(b) is a display screen of the tally result showing "Today's Good! Ranking" tallied for the first effect image 711 displayed in the upper right. In Fig. 15(b), for the type to which the two first effect images 711 belong (in this case, "Good!"), the number of times each song was sung is tallied, and a ranking of songs that users felt were "Good!" is displayed.

[0097] Fig. 15(c) shows a ranking of the number of times the first effect image 711 was selected when the user sang, for each user participating in the room. Since all first effect images 711 are the subject of the ranking, the first effect image 711 that was the subject of the ranking is not displayed as in Figs. 15(a) and 15(b). Also, as shown in Fig. 15(c), instead of ranking the users who were cheered, the cheering users, i.e., the users participating in the room, may compete in a ranking format based on the number of times they selected the first effect image 711.

[0098] As described above, in this embodiment, by displaying user comments entered by user operations and system comments formed based on operations related to music playback processing in chronological order, good communication between users is possible, allowing users to fully enjoy karaoke.

[0099] This embodiment can employ various modified examples, which will be described below. The modified examples will be described below.

[0100] [First Modification] The tallying process in the above embodiment is targeted only at the first effect image 711, but it may also be targeted at the second effect image 721. Also, in the above embodiment, the tallying process targets the participants in a certain room and the songs sung in that room, but the tallying process may not be limited to this form and may be targeted at all rooms held in the karaoke system, or may be targeted in any other suitable form.

[0101] [Second modified example] In the above embodiment, the selection operation of the first effect image 711 and the second effect image 721 can be performed regardless of the singer or the listener. However, a restriction may be set so that the selection operation of the first effect image 711 and the second effect image 721 can be performed only by the listener, that is, only by the mobile terminal 6 set in the listener mode. If the singer himself performs the selection operation of the first effect image 711 and the second effect image 721, his / her operation may be included in the tally. By setting the above restriction, it becomes possible to perform a fair evaluation.

[0102] Even if the singer is able to select the first effect image 711 or the second effect image 721, the singer's selection operation is not tallied, so that the singer can enjoy the generation of the effect image by the selection operation and also a fair evaluation can be made.

[0103] [Third Modification] In the effect display process described in the flow chart of FIG. 11, the second effect image 721 is provided with a selectable period (S116) so that the second effect image 721 is not displayed overlappingly. Instead of this form, the second effect image 721 may be displayed overlappingly like the first effect image 711. Also, instead of providing a selectable period, a reservation system may be used. The second effect image 721 selected first is displayed, and after the display of the second effect image 721 is completed, the second effect image 721 selected next is displayed, so that the second effect image 721 is displayed without overlapping.

[0104] [Fourth Modification] Each user may be limited in the number of times of selection operation of the first effect image 711 and the second effect image 721. For example, by limiting the number of times of selection operation of each of the first effect image 711 and the second effect image 721 or the total number of both during the playback period of one song, the user can use the first effect image 711 and the second effect image 721 at an appropriate timing without selecting them indiscriminately. In addition, for various tabulations such as rankings, each user will perform careful selection operation while the number of times of selection is limited, and it is expected that the accuracy of the tabulations will be high. [Explanation of symbols]

[0105] 6 (6a, 6b): Portable terminal 70b: Input switch 11: Management server 72: Vibration unit 12a: Router 73: LED 12b: Wireless router 110: Access point 13: Wireless base station 140: Mobile base station 61:CPU 681:Lyrics display field 62:RAM 682:Progress bar 63: Memory section 683: Selection button 64: Image processing unit 684: Microphone selection button 65F: Front camera 685: User icon 65R: Rear camera 686: Comments tab 66: Sound processing unit 687: Reservation confirmation tab 67a: Speaker 688: Remote control tab 67b: Microphone 689: Comment display column 68: Touch panel display 690: Comment input field 68a: Room selection field 690a: First button 68b: New room creation button 690b: Second button 68c: Search category selection field 691: Singer volume 68d: Song selection column 692: Speaker volume 68e: Song confirmation column 693: Pitch change button 68f: Cancel button 694: BGM volume 68g: Reservation button 695: Playback position adjustment button 68h: Song related display area 696: Button 68i: Top display area 697: Pause button 68j: Lower display area 698: Playback stop button 68k: Song search button 710: First effect selection field 68m: Display unit 711: First effect image 68n: Touch panel 720: Second effect selection field 69a: First communication unit 721: Second effect image 69b: Second Communication Section 731: User Ranking 70: Input Switch 732: Singing Ranking 70a: Input switch

Claims

1. A karaoke program executable on a terminal device, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction; The display process displays, on a display unit of the terminal device, an icon of a user set in the terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves from the icon of the user who performed the effect display instruction operation to the lyrics display area as a starting point. Karaoke program.

2. A karaoke program executable on a terminal device, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed at least on another terminal device, a first effect image or a second effect image, which has a different pattern from the first effect image and serves as a background for the displayed lyrics, is displayed based on the effect display instruction; The first effect image and the second effect image are not prohibited from being displayed simultaneously, and a plurality of second effect images are prohibited from being displayed simultaneously. Karaoke program.

3. In the lyrics display area, the display position of the effect image is determined based on the position of the lyrics being displayed or a preset lyrics display position.

3. The karaoke program according to claim 1 or 2.

4. The display process displays the effect image for a certain period of time after displaying the effect based on the effect display instruction.

4. The karaoke program according to claim 1, wherein the karaoke program is a program for performing karaoke.

5. A karaoke device capable of reproducing and outputting music information, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed on at least another terminal device, an effect image is displayed based on the effect display instruction; The display process displays, on a display unit of the terminal device, an icon of a user set in the terminal device participating in the communication process, and when an effect display instruction operation is performed in at least another terminal device, displays an effect image that moves from the icon of the user who performed the effect display instruction operation to the lyrics display area as a starting point. Karaoke equipment.

6. A karaoke device capable of reproducing and outputting music information, The device is capable of performing music playback processing, audio processing, communication processing, and display processing; The music playback process plays and outputs music information. The audio processing unit outputs the reproduced music sound and the audio signal input from the audio input unit, The communication process enables voice communication with other terminal devices; The display process forms a lyrics display area on a display unit of the terminal device, in the lyrics display area, the lyrics reproduced in the music reproduction process are displayed, and when an effect display instruction operation is performed at least on another terminal device, a first effect image or a second effect image, which has a different pattern from the first effect image and serves as a background for the displayed lyrics, is displayed based on the effect display instruction; The first effect image and the second effect image are not prohibited from being displayed simultaneously, and a plurality of second effect images are prohibited from being displayed simultaneously. Karaoke equipment.

Images