Karaoke program, karaoke equipment, and karaoke scoring method

Abstract

The purpose is to appropriately evaluate voice-changed audio. [Solution] The system is characterized by performing a playback process that plays music based on music information, a first input process that receives voice-changed audio, a second input process that receives audio that is not voice-changed, a sound output process that outputs the played music and voice-changed audio, a scoring process that scores at least a portion of a plurality of scoring items, namely a first scoring item, using the voice-changed audio, and a result output process that outputs the results of the scoring process.

Description

Technical Field

[0001] The present invention relates to a karaoke device that performs singing in accordance with a reproduced music piece, a karaoke program for realizing the functions of the karaoke device in an information processing device, and a karaoke scoring method for evaluating singing.

Background Art

[0002] Conventionally, karaoke in which singing is enjoyed in accordance with an accompaniment has been performed in social places and the like. Some karaoke devices have a function of converting the singing voice into a voice different from the voice quality of the singer.

[0003] Patent Document 1 discloses a karaoke device that converts a singer's singing voice into the singing style of an original singer and outputs it. In the karaoke device of Patent Document 1, an object that models the voice quality and singing method of an original singer is stored as a singing parameter, and at the time of karaoke performance, the singing parameter is read out to convert the singing voice. With such a configuration, the karaoke device of Patent Document 1 can make the singing sound better even if the singer's singing is flat singing, because singing methods such as vibrato and growl are imparted in accordance with the singing style of the original singer.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Some karaoke machines have a scoring function that evaluates singing. In karaoke machines with a voice-changing function, as described in Patent Document 1, conventional methods use the singing voice that has not been voice-changed for scoring. As a result, there may be discrepancies between the subjective evaluation of the audience or the singer who hears the voice-changed singing voice and the calculated scoring result. Such discrepancies in scoring results can undermine the satisfaction of both the listener and the singer.

[0006] This invention takes these circumstances into consideration and aims to reduce the discrepancy between the subjective evaluation of the audience and the actual performance by using voice-changed singing voices for scoring, thereby improving the satisfaction of both listeners and singers with the scoring results. [Means for solving the problem]

[0007] Therefore, the karaoke program according to the present invention is Executable in an information processing device, A playback process that plays the music based on the music information, A first input process receives voice-changed audio, A second input process receives audio that is not subjected to voice change processing, The audio output process outputs the played music sound and the voice-changed audio. A scoring process that scores a first scoring item, which is at least a part of multiple scoring items, using a voice-changed voice, The program then performs a result output process to output the graded results.

[0008] Furthermore, in the karaoke program according to the present invention, The scoring process uses voice-changed audio for all scoring items.

[0009] Furthermore, in the karaoke program according to the present invention, The scoring process uses unprocessed audio for a second scoring item, which is different from the first scoring item.

[0010] Furthermore, in the karaoke program according to the present invention, The second scoring criterion is at least one of the following: stability or long tone.

[0011] Furthermore, in the karaoke program according to the present invention, The scoring process uses voice-changed audio to evaluate at least one of the scoring items: pitch, intonation, technique, stability, and long tone.

[0012] Furthermore, in the karaoke program according to the present invention, The scoring process uses the pitch difference between the voice-changed audio and the un-voice-changed audio, and then corrects the pitch of either the reference data or the un-voice-changed audio to score the result.

[0013] Furthermore, in the karaoke program according to the present invention, The scoring process allows users to configure whether to score each scoring item using either a voice-changed or un-voice-changed audio.

[0014] Furthermore, in the karaoke program according to the present invention, The scoring process can be configured to either use voice-changed audio for all scoring items, or to use un-voice-changed audio for at least some scoring items.

[0015] Furthermore, in the karaoke program according to the present invention, The scoring process determines, depending on the type of voice-changing process used, whether to score all scoring items using voice-changed audio or to score at least some scoring items using unchanged audio.

[0016] Furthermore, the karaoke device according to the present invention is A playback process that plays the music based on the music information, A first input process for inputting voice-processed voice, A second input process for inputting voice that is not voice-processed, A sound output process for outputting the reproduced music sound and the voice-processed voice, A scoring process for scoring at least some of a plurality of scoring items using the voice-processed voice, A result output process for outputting the scored result is executed.

[0017] In addition, the karaoke scoring method according to the present invention, A reproduction process for reproducing music sound based on music information, A first input process for inputting voice-processed voice, A second input process for inputting voice that is not voice-processed, A sound output process for outputting the reproduced music sound and the voice-processed voice, A scoring process for scoring at least some of a plurality of scoring items using the voice-processed voice, A result output process for outputting the scored result is executed.

Effect of the Invention

[0018] According to the karaoke program, karaoke device, and karaoke scoring method according to the present invention, the difference between the scoring result when singing with voice processing and the subjective evaluation of the listener or singer can be reduced, and it is possible to improve the acceptance of the listener or singer with respect to the scoring result.

Brief Description of the Drawings

[0019] [Figure 1] A diagram showing the system configuration of the present embodiment [Figure 2] An external view of the mobile terminal of the present embodiment [Figure 3] A block diagram showing the configuration of the mobile terminal of the present embodiment [Figure 4]Flowchart showing the room processing in this embodiment [Figure 5] Flowchart showing the reservation process in this embodiment [Figure 6] Diagram illustrating the configuration of this embodiment [Figure 7] Diagram showing the voice change table of this embodiment [Figure 8] Flowchart showing the inter-mobile terminal processing in this embodiment [Figure 9] Diagram showing various screens in the mobile terminal of this embodiment. [Figure 10] Diagram showing various screens in the mobile terminal of this embodiment. [Figure 11] Diagram showing various screens in the mobile terminal of this embodiment. [Figure 12] Figure illustrating the configuration of other embodiments. [Modes for carrying out the invention]

[0020] Figure 1 shows the system configuration of this embodiment. In this embodiment, karaoke can be performed between mobile terminals 6, and therefore, communication takes place between the mobile terminals 6. Figure 1 shows various examples of communication configurations between mobile terminals 6. For example, in a private home, a router 12a connected to the internet is provided, and a wireless router 12b is connected to the router 12a. The mobile terminals 6 can connect to the internet via the wireless router 12b.

[0021] In addition to these communication methods, it is also possible to communicate with the Internet using a mobile communication network. A mobile communication network is a communication method that connects the mobile terminal 6 to the Internet by performing wireless communication between the mobile terminal 6 and the wireless base station 13.

[0022] By using these various communication methods, the mobile terminal 6 can connect to the internet. A management server 11 for managing the karaoke system is also connected to the internet. Once the karaoke program is installed and the mobile terminal 6 is connected to the internet, it can communicate with this management server 11, enabling karaoke sessions between the mobile terminals 6.

[0023] In this embodiment, we will describe a configuration in which karaoke is performed using a mobile terminal 6 as the terminal device. Note that, in addition to the mobile terminal 6, a home game console, a personal computer, or a dedicated karaoke machine may also be used as the terminal device. Furthermore, a terminal device with a karaoke program installed will function as a karaoke machine.

[0024] Figure 2 is a diagram showing the external appearance (front view) of a mobile terminal 6 according to an embodiment of the present invention, and Figure 3 is a block diagram showing the configuration of the mobile terminal 6 according to an embodiment of the present invention. This mobile terminal 6 can be a smartphone, tablet terminal, mobile phone, etc. By installing a karaoke program on this mobile terminal 6, it becomes possible to perform karaoke between the mobile terminals 6.

[0025] The mobile terminal 6 of this embodiment has a control means comprising a CPU 61, RAM 62, storage unit 63, image processing unit 64, and sound processing unit 66. The storage unit 63 can employ a non-volatile storage means such as NAND flash memory. This storage unit 63 stores the program of this embodiment, as well as various data used by the program.

[0026] The CPU 61 is a means of comprehensively controlling the entire mobile terminal 6, and executes various controls based on the programs and data stored in the memory unit 63. The RAM 62 temporarily stores data generated by the control of the CPU 61.

[0027] The sound processing unit 66 is a means for handling audio-related input and output of the mobile terminal 6. It is a means for emitting sound from the speaker 67a under the control of the CPU 61. The sound processing unit 66 is also connected to the microphone 67b, and can pick up call audio and ambient sounds from the microphone 67b. External devices such as a headset may be used for the speaker 67a and microphone 67b.

[0028] The touch panel display unit 68 comprises a display unit 68m that displays an image and a touch panel 68n disposed on its front or back surface. The touch panel 68n can employ various types, such as a capacitive touch panel, that can recognize the user's touch position. The CPU 61 displays an image on the display unit 68m and can also determine which part of the displayed image has been touched based on the touch input from the touch panel 68n.

[0029] The image processing unit 64 is a means for inputting and outputting image-related information from the mobile terminal 6. The image processing unit 64 outputs the image formed by the CPU 61 to the display unit 68m. It also has the function of capturing images, or if it is video, video, from the front camera 65F located on the front of the mobile terminal 6 (on the side of the touch panel display unit 68), or from the rear camera 65R located on the back of the mobile terminal 6, and storing the image or video information in the storage unit 63.

[0030] The mobile terminal 6 of this embodiment is equipped with a first communication unit 69a and a second communication unit 69b as communication means for communicating with the outside world. The first communication unit 69a is a communication means for connecting to a mobile communication network, and by wirelessly connecting to the wireless base station 13, it is possible to perform various communications via the Internet network using the communication lines of the mobile communication network. The second communication unit 69b is a communication means for Wi-Fi connection. The mobile terminal 6 can use the second communication unit 69b to wirelessly connect to the wireless router 12b and communicate with the local area network to which the wireless router 12b participates. In addition to the first communication unit 69a and the second communication unit 69b, the mobile terminal 6 may also be equipped with an infrared communication unit, Bluetooth®, or other short-range communication means.

[0031] The portable terminal 6 of this embodiment is configured to include input switches 70 in addition to the touch panel 68n as an input means. Various types of switches can be provided for these input switches 70 (input switches 70a, 70b, etc. in Figure 2), such as a power switch, a volume control switch, and a switch whose input function is changed according to the program.

[0032] Furthermore, the mobile terminal 6 of this embodiment has a vibrating unit 72 and an LED 73 to provide various notifications to the user who possesses the mobile terminal 6. The vibrating unit 72 is a component formed by attaching an eccentric weight to the shaft of a motor. By rotating the motor, it is possible to transmit vibrations that can be notified to the user. As shown in Figure 2, the LED 73 is positioned on the upper front of the mobile terminal 6 and can provide various notifications to the user by flashing.

[0033] In this embodiment, by installing a karaoke program on such a mobile terminal 6, it becomes possible to perform karaoke between the mobile terminals 6. Now, let's explain the details of the various processes involved when performing karaoke using the mobile terminal 6.

[0034] Figure 4 is a flowchart illustrating the room processing in this embodiment. Here, we will explain the case where karaoke is performed using mobile terminals 6a and 6b. In this embodiment, by gathering in a virtually formed room, users in the same room can perform karaoke by sharing audio (which may also include video).

[0035] This section describes the case where a user using mobile device 6a creates a room and a user using mobile device 6b joins that room. It is assumed that users using mobile devices 6a and 6b already have accounts with the management server 11. Users without an account can access the management server 11 and create a new account.

[0036] First, the mobile terminal 6a sends a login instruction (S11) to the management server 11. The login instruction is performed by entering account information (user ID, password, etc.). Upon receiving the login instruction (S11), the management server 11 executes the login process (S21). During the login process, authentication is performed based on the account information, and if the authentication is successful, the user can use various functions of the karaoke program.

[0037] Figure 9(A) shows the room selection screen displayed after logging in. The room selection screen displays a room selection field 68a and a new room creation button 68b. Room selection field 68a shows rooms that have already been created, and the status of those rooms is displayed. The room selection field displays the room name, icons of users participating in the room, the room's status ("OPEN", "Preparing"), and the date the room will be held. Room selection field 68a with a key icon in the upper left corner indicates a room that requires a password or a room that can only be joined by users who have been previously invited.

[0038] Users can join a room by operating the room selection field 68a. The new room creation button 68b is used to create a new room. By operating the new room creation button 68b, the mobile terminal 6a can send a room creation instruction (S12) to the management server 11. Based on the room creation instruction (S12), the management server 11 executes the room creation process (S22). Once the room creation process (S22) is complete, the room selection field 68a shown in Figure 7(A) is displayed.

[0039] Meanwhile, the mobile device 6b joining the room first sends a login instruction (S11) to the management server 11 by entering its account information. The management server 11 performs authentication based on the account information, as in the case of mobile device 6a, and if the authentication is successful, logs in mobile device 6b.

[0040] The user refers to the room selection screen shown in Figure 9(A) and makes a room joining instruction by selecting the room they wish to join in the room selection field 68a (S13). In the example in Figure 4, the user joins a room created by the user of mobile terminal 6a, and the management server 11 performs a user addition process (S23) to add the user of mobile terminal 6b to the room created by the user of mobile terminal 6a.

[0041] When users of mobile devices 6a and 6b enter the same room, it becomes possible to perform karaoke processing (S14) between mobile devices 6a and 6b. Karaoke processing (S12) mainly consists of reservation processing and song playback processing. Reservation processing is the process of reserving a song to be sung, and song playback processing is the process of playing the songs registered in the reservation list in order based on the reservation processing.

[0042] Figure 5 is a flowchart illustrating the reservation process in this embodiment. In the reservation process, it is possible to add a song to the reservation list managed by the management server 11 by searching for a song and selecting the song to sing. Users who wish to reserve a song use the song search screen shown in Figure 9(B) to search for a song (S101). The song search screen in Figure 9(B) can be displayed by operating the song search button 68k shown in Figures 9(D) and 10(E), which will be explained later.

[0043] The music search screen includes a search category selection field 68c and a song selection field 68d. The search category selection field offers search categories such as "Favorites," "Songs," "Artists," and "Albums," allowing users to perform music searches according to their selected category. Figure 9(B) shows the state when "Songs" is selected as the search category, and the song selection fields 68d are displayed in alphabetical order.

[0044] The song selection field 68d displays an icon (album art, etc.), song title, and artist name for each song. When the user selects a desired song from the song selection field 68d (S102: Yes), a song confirmation screen for that song is displayed (S103).

[0045] Figure 10(C) shows the music confirmation screen of this embodiment. The music confirmation screen displays the music confirmation field 68e. The music confirmation field 68e displays the selected music, including an icon (album art, etc.), the music title, the artist name, and a reserve button 68g and a cancel button 68f. The user confirms the information displayed on this music confirmation screen and, if they wish to make a reservation, operates the reserve button 68g (S104:Yes) to make the reservation. Once a reservation is made, the reservation information is sent to the management server 11. The reservation information consists of a user ID indicating the user who made the reservation and a music ID indicating the music. The management server 11 registers the received reservation information in the reservation list.

[0046] Here, the problems of the present invention and the configuration for solving those problems will be described. In karaoke, voice-changing processing is known to allow people to enjoy altering their singing voice. Voice-changing processing includes, for example, converting a male voice to a female voice by raising the pitch of the voice, or vice versa to convert a female voice to a male voice. Alternatively, by performing various filtering, such as distorting the voice in addition to changing the pitch, it is possible to enjoy a variety of voice types.

[0047] On the other hand, some karaoke machines have a scoring function that evaluates singing performance with a score. By using the scoring function, users can numerically recognize whether their singing is good or bad. However, when the voice-changing process mentioned earlier is performed, scoring is usually done using the voice before the voice change. Therefore, there may be discrepancies between the subjective evaluation of the audience or the singer who hears the voice-changed singing voice and the calculated scoring result.

[0048] This invention takes these circumstances into consideration and aims to reduce the discrepancy between the subjective evaluation of the audience and the actual performance by using voice-changed singing voices for scoring, thereby improving the satisfaction of both listeners and singers with the scoring results.

[0049] Furthermore, when voice-changing processing is performed, the pitch of the voice changes, which may cause it to not match the pitch of the played music, resulting in a sense of unease for the listener. Therefore, in this embodiment, it is possible to make the voice-changed voice and the played music sound as harmonious as possible.

[0050] Figure 6 is a diagram illustrating the configuration of this embodiment. The management server 11 of this embodiment includes a music transmission unit 11a. The music transmission unit 11a transmits music to the mobile terminals 6a and 6b that are in the same room via stream playback. Note that the music playback method may not be limited to this stream playback method, but may also use a storage-type playback method that plays music data stored in advance on the mobile terminals 6a and 6b. Furthermore, although this explanation uses the singer's mobile terminal 6a and the listener's mobile terminal 6b, there may be multiple units of each of the singer's mobile terminal 6a and the listener's mobile terminal 6b.

[0051] The music transmission unit 11a transmits the music to the singer's mobile terminal 6a and the listener's mobile terminal 6b. The music playback unit 61a of the singer's mobile terminal 6a plays the music based on the set key. In this embodiment, the karaoke program installed on the mobile terminal 6a includes a voice changer 61b.

[0052] The voice changer 61b processes the input singing voice based on the configured voice change type and outputs it. The singing voice output from the voice changer 61b is transmitted to the listener's mobile terminal 6b via the management server 11. The voice-changed singing voice is played back by the audio playback unit 61c of the listener's mobile terminal 6b, added to the played song by the addition unit 61f, and output to the listener.

[0053] Furthermore, in this embodiment, the set voice change type is transmitted to the listener's mobile terminal 6b via the management server 11. The listener's mobile terminal 6b stores a voice change table 61d and can determine the pitch difference of the received voice change type based on that type.

[0054] Figure 7 shows the voice change table of this embodiment. The voice change table stores the voice change type and pitch difference in association. The pitch difference is the difference between the pitch of the singing voice before voice change and the pitch of the singing voice after voice change. On the listener's mobile terminal 6b, the song is played back with the pitch difference corrected by referring to the voice change table 61d in response to the setting key set on the singer's mobile terminal 6a.

[0055] For example, if "Anime (Female Voice)" is selected and the setting key is set to standard, the song will be played at the standard key on the singer's mobile device 6a. On the other hand, on the listener's mobile device 6b, the song will be played at a key of +4 due to the pitch difference corresponding to the "Anime (Female Voice)" setting. Similarly, if "Anime (Male Voice)" is selected and the setting key is set to +1, the song will be played at a key of +1 on the singer's mobile device 6a. On the other hand, on the listener's mobile device 6b, the song will be played at a key of +1-4=-3 due to the pitch difference corresponding to the "Anime (Male Voice)" setting.

[0056] In the example described above, when "Anime (female voice)" is selected, the voice-changed singing voice is output with a pitch difference of +4 compared to the original singing voice. Therefore, on the listener's mobile device 6b, both the song and the voice-changed singing voice will be +4 key, resulting in their pitches being aligned. Thus, the listener can hear the song and the voice-changed singing voice with their pitches aligned.

[0057] Next, the scoring process performed on the singer's mobile terminal 6a will be described. As shown in Figure 6, the mobile terminal 6a is configured to include a scoring unit 61e. In this embodiment, the scoring unit 61e is configured to include two scoring units (a first scoring unit 611 and a second scoring unit 612). The first scoring unit 611 receives a voice-changed singing voice as input and performs scoring based on that singing voice. The second scoring unit 612 receives an un-voice-changed singing voice as input and performs scoring based on that singing voice. The scoring unit 61e can output an overall scoring result based on the scoring results of the first scoring unit 611 and the scoring results of the second scoring unit 612.

[0058] Here, the first scoring unit 611 refers to the voice change table 61d and the setting key set on the mobile terminal 6a, and performs scoring after key correction in the same way as the correction key corrected on the listening mobile terminal 6b. Specifically, the reference data included in the song data is corrected with the correction key, and then scored by comparing it with the voice-changed singing voice. In this way, scoring can be improved by correcting the reference data with the correction key. Note that the correction by the correction key may be applied to the voice-changed singing voice instead of the reference data.

[0059] The scoring unit 61e performs scoring on various items. In this embodiment, items are provided for each characteristic of singing, such as pitch, intonation, technique, stability, and long tone. The first scoring unit 611 and the second scoring unit 612 are responsible for scoring the items that require high accuracy.

[0060] Specifically, the first scoring unit 611, which uses voice-changed singing audio, is responsible for pitch points, intonation points, and technique points, and calculates scores for each item. On the other hand, the second scoring unit 612, which uses singing audio that has not been voice-changed, is responsible for stability points and long tone points, and calculates scores for each item. Voice-changed singing audio may have distortions and interruptions due to the characteristics of the voice-change processing.

[0061] When scoring stability points and long tone points using voice-changed singing audio, there is a possibility of significant point deductions. Scoring stability points and long tone points using un-voice-changed singing audio may result in less discrepancy with the subjective evaluation of the audience or singer than scoring using voice-changed singing audio. Therefore, in this embodiment, stability points and long tone points are scored using un-voice-changed singing audio.

[0062] On the other hand, scoring pitch, intonation, and technique using voice-changed singing audio may result in less discrepancy with the subjective evaluation of the audience or singer compared to scoring using un-voice-changed singing audio. Therefore, in this embodiment, pitch, intonation, and technique are scored using voice-changed singing audio. In this embodiment, the user is presented with a total score using both the scoring results of the first scoring unit 611 and the scoring results of the second scoring unit 612.

[0063] Thus, in this embodiment, by using a first scoring unit 611 that uses voice-changed singing audio and a second scoring unit 612 that uses un-voice-changed singing audio for each item, it is possible to present scores that are highly accurate and satisfactory to both the listener and the singer.

[0064] Next, we will explain the time-series processing that takes place between mobile terminals 6a and 6b. Figure 8 is a flowchart showing the processing between mobile terminals in this embodiment. Here, for the sake of simplicity, we will use two mobile terminals 6a and 6b that are in the same room as in Figure 6 as an example. In reality, the number of mobile terminals 6 that enter the room can be two or more.

[0065] First, the mobile terminal 6a performs a reservation process (S100) to reserve the desired song. The reservation process (S100) is as described in Figure 5, in which the mobile terminal 6a sends reservation information to the management server 11. Upon receiving the reservation information, the management server 11 registers the reservation information in the reservation list (S201). In this embodiment, a system comment is displayed on all mobile terminals 6a and 6b that are in the room in conjunction with the reservation operation (S101, S151).

[0066] Upon receiving the reservation information, the management server 11 sends a system comment to all mobile terminals 6a and 6b in the room where the user who made the reservation is present (S202). The receiving mobile terminals 6a and 6b then display the system comment (S101, S151). Based on this configuration, it becomes possible to notify each user of operations related to music playback (in this case, reservation processing) when users are performing karaoke remotely. Therefore, each user can know who is performing what operations, enabling smooth communication.

[0067] The management server 11 checks the reservation list (S203), and if there is a song to be played next (S204), it executes the song playback process for that song. In the song playback process, first, the user who made the reservation is identified as the singer based on the user ID included in the reservation information (S205). Then, an instruction is sent to the mobile terminal 6a used by the identified singer to change to singer mode (S206). The mobile terminal 6a that receives the instruction changes to singer mode (S102).

[0068] Meanwhile, the management server 11 identifies users other than the singer who are in the same room as listeners and issues an instruction to the mobile terminal 6b used by the user identified as the singer to change to listener mode (S207). The mobile terminal 6a that receives the instruction changes to singer mode (S152).

[0069] Here, we will explain singer mode and listener mode. In singer mode, the volume of the microphone 67b is increased sufficiently to easily capture the user's singing voice. In singer mode, a communication mode with lower latency is set than in listener mode. By using this communication mode, the singing voice in singer mode is transmitted to other mobile devices 6 with reduced latency.

[0070] On the other hand, in the case of the mobile terminal 6 set to listener mode, the microphone 67b is turned off or its volume is lowered so as not to interfere with singing. By adjusting the volume in this way in singer mode and listener mode, it is possible to create an environment that is easy to listen to.

[0071] On the singer's mobile terminal 6a, which is set to singer mode, the voice change setting is confirmed (S103). The voice change setting can be configured in the voice change setting field 694 of the remote control tab 688 shown in Figure 11. In the example shown in Figure 11, "Anime (female voice)" is selected. The voice change setting confirmed on the singer's mobile terminal 6a is notified (sent) to the management server 11 as part of the singing setting information (S104).

[0072] The singing settings include not only voice change settings but also key (pitch) settings made by the singer. Furthermore, the mobile terminal 6a allows users to adjust the key for easier singing. The key setting can be changed using the key (pitch) setting button 693 on the remote control tab 688 shown in Figure 11. The management server 11 transfers the received singing settings to the listener's mobile terminal 6b (S208).

[0073] Next, the management server 11 starts transmitting music to each mobile terminal 6a and 6b (S209). In this embodiment, music is transmitted via streaming playback, but it is also possible to use a storage type where music information is stored in advance on the mobile terminals 6a and 6b and then played back. In either case, music playback will be synchronized on each mobile terminal 6a and 6b.

[0074] On the singer's mobile terminal 6a, the song is played at the key set by the key (pitch) setting button 693 (S105). Meanwhile, the listener's mobile terminal 6b receives the song from the management server 11 (S153), corrects the key based on the voice change setting and key setting included in the singing setting, and plays the song (S154).

[0075] Regarding the singing voice, the singing voice collected by the singer's mobile terminal 6a (S106) is voice-changed by a voice changer (S107) and transmitted to the management server 11. The management server 11 transfers the voice-changed singing voice to the listener's mobile terminal 6b (S210). The listener's mobile terminal 6b plays the received singing voice (S156) and outputs it added to the played song.

[0076] Furthermore, during the music playback period, the mobile terminal 6a performs a scoring process (S108) based on the singing voice. As explained in Figure 6, the scoring process is performed by the scoring unit 61e. The first scoring unit 611 uses the voice-changed singing voice for scoring, while the second scoring unit 612 uses the un-voice-changed singing voice for scoring.

[0077] Figure 10(D) shows the music playback screen displayed during music playback on the mobile terminal 6 of this embodiment. The room name and a music search button 68k are displayed at the top of the music playback screen. By operating the music search button 68k, the music search screen described in Figure 9(B) is displayed, and it is possible to reserve music.

[0078] In addition, the music playback screen is broadly divided into three sections: a music-related display section 68h, an upper display section 68i, and a lower display section 68j. The music-related display section 68h consists of a lyrics display section 681 and a progress bar 682. The lyrics display section 681 displays the lyrics to be sung for the currently playing song. The lyrics display section 681 may also display a background video. The progress bar 682 indicates the progress of music playback.

[0079] The upper display area 68i and the lower display area 68j can be changed. Normally, the state shown in Figure 10(D) is when both the upper display area 68i and the lower display area 68j are displayed. When the comment tab 686, reservation confirmation tab 687, or remote control tab 688 located in the lower display area 68j are operated, the lower display area 68j will also use the area where the upper display area 68i was displayed, as shown in Figure 11(E). Figure 11(E) shows the state. By operating the comment tab 686, reservation confirmation tab 687, or remote control tab 688 again, the state shown in Figure 10(D) is returned.

[0080] The upper display area 68i shows the status display / selection button 683, the microphone selection button 684, and the participating user icon 685. The status display / selection button 683 is a button that indicates the status of the mobile device 6. When the device is set to the singer mode described above, it is marked "Singing," and when it is set to the listener mode, it is marked "Listening."

[0081] Figure 10(D) shows the state where "Singing" is marked. The user can also manually switch between "Singing," "Listening," and "Away." When switched to "Singing," the setting changes to singer mode, and when switched to "Listening" or "Away," the setting changes to listener mode.

[0082] The microphone selection button 684 allows the user to manually switch the microphone 67b on or off. In this embodiment, it is automatically set to on when the system is set to singer mode, and automatically set to off when the system is set to listener mode. The participating user icons represent all users participating in the room.

[0083] In the example in Figure 10(D), four users are participating. Users whose icons are shown with a thick line (e.g., "JOY" in Figure 10(D)) are singing (set to singer mode). Additionally, icons with a speaker and an "X" (e.g., "Monkichi" in Figure 10(D)) indicate that the user is away.

[0084] The lower display area 68j has three tabs (Comment tab 686, Reservation confirmation tab 687, and Remote control tab 688). In Figure 10(D), the Comment tab 686 is selected, and the lower display area 68j displays the Comment display area 689 and the Comment input area 690. The Comment display area 689 displays user comments entered by the user using the Comment input area 690, as well as system comments related to operations during music playback.

[0085] In Figure 10(D), the comment display field 689 shows system comments related to the reservation process, including the user name and the name of the reserved song / artist. System comments and user comments are displayed sequentially in chronological order in the comment display field 689, allowing users to communicate smoothly by viewing these system and user comments.

[0086] Figure 10(E) shows the state when the remote control tab 688 is operated. As mentioned above, the lower display area 68j is extended to display the area where the upper display area 68i is displayed. When the remote control tab 788 is selected, the lower display area 68j displays the singer volume control 691, speaker volume control 692, key (pitch) setting button 693, voice change setting area 694, playback position adjustment button 695, re-sing button 696, pause button 697, and stop playback button 698.

[0087] In this embodiment, the various controls and buttons displayed on the remote control tab 688 can only be operated by the mobile terminal 6 set to singer mode. Therefore, the mobile terminal 6 set to listener mode is prohibited from performing operations related to music playback, thereby preventing erroneous input by the listener.

[0088] The singer volume control 691 is an operation that adjusts the volume of the singing voice input from the microphone 67b of the mobile terminal 6 set to singer mode. Similarly, the speaker volume control 692 is an operation that adjusts the volume of the singing voice input from the microphone 67b of the mobile terminal 6 set to listener mode. When the device is set to singer mode or listener mode, the singer volume control 691 and speaker volume control 692 are set to the default state for each mode, but they can only be operated from the mobile terminal 6 set to singer mode.

[0089] The key setting button 693 is used to set the key (pitch) of the song being played. When the key setting button 693 displays a sharp, it raises the key of the song being played. When the key setting button 693 displays a flat, it lowers the key of the song being played. When the key setting button 693 displays "Original," it sets the key to the original key of the song. The set key is displayed numerically as "Key" to the left of the key setting button 693 (in the example in Figure 11(E), it is set to "0").

[0090] The voice change setting field 694 is a field where the user can select the type of voice change. By selecting the operator displayed to the right of the voice change setting field 694, the user can choose the desired voice change type from the voice change types shown in the voice change table in Figure 7. It is also possible to set the voice change setting to not perform any voice change using this voice change setting field 694.

[0091] The playback position adjustment button 695 is an operator used to change the playback position. The current playback time and the total playback time are displayed to the left of the operator. The restart button 696 allows you to restart the currently playing song from the beginning. The pause button 697 is a button used to pause the currently playing song. The stop playback button 698 is a button used to stop the currently playing song. When the stop playback button 698 is pressed, the system proceeds to play the next reserved song.

[0092] While a song is playing, a user set to singer mode can perform various settings related to song playback using the various controls and buttons displayed on the remote control tab 688 in Figure 11(E). If a setting operation is performed while a song is playing, the mobile terminal 6a sends a setting change instruction to the management server 11. The management server 11 then executes the setting change according to the received setting change instruction.

[0093] Settings can be changed not only on the management server 11, such as changing the key of a song, but also on the various mobile terminals 6a and 6b. For example, when changing the key (pitch) setting from a mobile terminal 6a set to singer mode, the mobile terminal 6a sends setting information regarding the key (pitch) change to the management server 11. The management server 11, upon receiving the setting information, forwards it to the other mobile terminals 6b. The mobile terminals 6b, upon receiving the setting information, change the key (pitch) of the song being played based on the received setting information. In this embodiment, as mentioned above, not only the set key (pitch) change but also the key (pitch) change based on voice change is performed.

[0094] When the music transmission is complete on the management server 11 (S211: Yes), it checks the reservation list for the next song to be played (S203). Also, upon completion of music playback transmission, each mobile terminal 6a and 6b stops playing the music (S109, S157: Yes).

[0095] After the song finishes playing, the singer's mobile device 6a displays the scoring results. The scoring results are scores calculated based on the scoring process (S108), and in this embodiment, they are displayed as a total score calculated by the first scoring unit 611 and the second scoring unit 612. The scoring results are also transmitted (S111) to the listener's mobile device 6b, which receives (S158) and displays (S159) the scoring results.

[0096] The above describes a system using mobile terminals 6a and 6b. In this embodiment, since scoring is performed using voice-changed audio, it is possible to reduce the difference between the subjective evaluation of the audience and the scoring results, thereby improving the satisfaction of listeners and singers with the scoring results. Furthermore, it is possible to allow listeners to hear the voice-changed audio and the played music without any sense of incongruity. The present invention can be adapted to various modifications in addition to the embodiments described above. Various modifications are described below.

[0097] [First variation] In the above-described embodiment, the scoring results displayed on the mobile terminals 6a and 6b were displayed as a combined score from the scoring results of the first scoring unit 611 and the second scoring unit 612. The scoring results can be displayed in various forms, not just this form. For example, in addition to the combined score, or instead of a combined score, the scoring results may display scores for each item. By knowing the scores for each item, the user can grasp the characteristics of the singing.

[0098] [Second variation] In the embodiment described above, the scoring unit 61e performed scoring using a first scoring unit 611 that scored using voice-changed singing voice and a second scoring unit 612 that scored using singing voice that had not been voice-changed. However, the scoring unit 61e may also perform scoring using only voice-changed singing voice. Depending on the type of voice change, scoring all scoring items using voice-changed singing voice may result in less discrepancy with the subjective evaluation of the audience or singer than scoring some scoring items using singing voice that had not been voice-changed. Therefore, by having the scoring unit 61e perform scoring using only voice-changed singing voice, it is possible to obtain greater satisfaction from the audience or singer.

[0099] [Third variation] In the embodiment described above, the scoring items to be scored by the first scoring unit 611 and the second scoring unit 612 were predetermined. Instead of this configuration, the scoring items to be scored by the first scoring unit 611 and the second scoring unit 612 may be set by the user. For example, it is conceivable that the user could set whether to score all scoring items with voice-changed audio, or to score at least some scoring items with audio that is not voice-changed.

[0100] As a concrete example, the touch panel display 68 of the mobile terminal 6 may display an option on whether or not to score all scoring items using voice-changed voices, and accept the user's selection. Alternatively, one or more combination patterns of scoring items that are scored using voice-changed voices and scoring items that are not scored using voice-changed voices may be displayed, and the user may be given an option to select one of these patterns, and accept the user's selection.

[0101] Furthermore, for each scoring item, the system may display an option to either score using un-voice-changed audio or not, allowing the user to make a selection. Depending on the type of voice change, the combination of scoring items that yields the least discrepancy with the subjective evaluation of the audience or singer may differ. Therefore, allowing the user to select combinations of scoring items that are scored using voice-changed audio and those that are not may lead to greater satisfaction for the audience or singer.

[0102] [Fourth variation] In the above-described embodiment, the scoring process was performed on the singer's mobile terminal 6a. However, the scoring process may be performed not only on the singer's mobile terminal 6a, but also on the listener's mobile terminal 6b, or on the management server 11, or on a server different from the management server 11.

[0103] [Fifth variation] As explained in Figure 7, it is possible to select multiple voice change types in the voice change processing. In the scoring process, it is conceivable to change the content according to the type of voice change. For example, depending on the selected voice change type, it is conceivable to decide whether to score all scoring items with the voice changed voice, or to score at least some scoring items with the voice that has not been changed. By adopting such a form, it becomes possible to perform scoring appropriate to the type of voice change, thereby improving the accuracy of the scoring.

[0104] [Sixth variation] In the embodiment described above, the voice change table 61d was referenced to detect the pitch difference corresponding to the selected voice change type, and this was used for correction during music playback and for correcting the reference data in the second scoring unit 612. The pitch difference may also be detected by comparing the voice-changed singing voice with the un-voice-changed singing voice, without using the voice change table 61d in this way.

[0105] [7th variation] The above-described embodiment describes a system using mobile terminals 6a and 6b connected via a network. However, the present invention can be applied not only to such network-based systems but also to a standalone karaoke device 2.

[0106] [8th variation] In the above-described embodiment, for example, the music playback unit 61a, the voice changer 61b, and the scoring unit 61e were provided as a karaoke program for the mobile terminal 6a. Thus, the music playback unit 61a, the voice changer 61b, and the scoring unit 61e may not only be provided as a single karaoke program, but the voice changer 61b may also be provided as an external component. In that case, the voice changer 61b may be provided not only as a program for the mobile terminal 6a, but also as hardware.

[0107] [9th variation] Figure 12 is a diagram illustrating the configuration of the ninth modified example. In the previously described embodiment and the first modified example, a system using mobile terminals 6a and 6b connected via a network was described. The present invention can be applied not only to such network-based systems but also to a standalone karaoke device 2. The configuration shown in Figure 12 comprises a karaoke device 2 and a voice changer 3 connected thereto. The voice changer 3 may be built into the karaoke device 2.

[0108] The ninth modification uses a voice change table 24, similar to the embodiment described above. The music playback unit 21 plays music for the singer using a set key. In the ninth modification, the singer and listener are in the same location, so for example, the singer will sing while listening to the music using headphones. Meanwhile, the listener will listen to the music emitted from the speaker. The way the singer and listener listen to the music may be reversed as described above.

[0109] In the ninth modified example, the singer and listener are in the same location, but they may be in different locations. In this case, the singer may operate the karaoke machine, and the listener may operate a different karaoke machine or terminal device. Information output from the karaoke machine operated by the singer is received via the network by the karaoke machine or terminal device operated by the listener. The listener may then listen to or view information output from the karaoke machine or terminal device operated by the listener.

[0110] The voice changer 3 outputs the voice-changed singing voice and the voice change type set in the voice changer 3 to the karaoke device 2. The karaoke device 2 refers to the voice change table and outputs the pitch difference corresponding to the input voice change type to the correction unit 22. The correction unit 22 corrects the song sound played by the song playback unit 21 with the pitch difference output from the voice change table 24, and outputs the song sound in the corrected key. The adder 23 adds the voice-changed singing voice input from the voice changer 3 and the key-corrected song sound and outputs it to the listener.

[0111] Furthermore, the karaoke device 2 is equipped with a scoring unit 25. Similar to the embodiment described above, the scoring unit 25 includes a first scoring unit 251 that scores the singing voice after voice change processing, and a second scoring unit 252 that scores the singing voice that has not undergone voice change processing. The first scoring unit 251 refers to a voice change table and performs scoring after key-correcting the reference data for scoring.

[0112] Thus, in the ninth modification, the karaoke device 2 installed in a karaoke box or the like can also be scored using at least a voice-changed singing voice, similar to the embodiment described above, thereby improving accuracy and the listener's satisfaction with the scoring results. Furthermore, it is possible to listen to the voice-changed voice and the played music without any sense of incongruity. In the ninth modification, a voice-change table 24 is used to detect the pitch difference, but as explained in the sixth modification, the pitch difference may also be detected by comparing the voice-changed singing voice with the singing voice that has not been voice-changed.

[0113] The above describes various embodiments using a karaoke program and karaoke device 2 executed on a mobile terminal 6 as examples. However, the present invention is not limited to a karaoke program and karaoke device 2 executed on a mobile terminal 6, but also extends to karaoke programs and karaoke scoring methods executed on various information processing devices such as personal computers and game devices. [Explanation of Symbols]

[0114] 2: Karaoke machine 68g: Reservation button 3: Voice changer 68h: Music-related display area 6(6a, 6b): Mobile device 68i: Top display area 11: Management Server 68j: Bottom Display Area 11a: Music transmission section 68k: Music search button 12a: Router 68m: Display Unit 12b: Wireless router 68n: Touch panel 13: Wireless base station 69a: First communications department 21: Music Playback Unit 69b: Second Communications Unit 22: Correction section 70 (70a, 70b): Input switch 23: Addition section 72: Vibration section 24: Voice changer table 73: LED 25: Grading Department 251: First Grading Department 61: CPU 252: Second scoring department 61a: Music playback section 611: First scoring section 61b: Voice changer 612: Second scoring section 61c: Audio playback section 681: Lyrics display area 61d: Voice Change Table 682: Progress Bar 61e: Scoring section 683: Select button 61f: Addition section 684: Microphone selection button 62: RAM 685: Participating User Icon 63: Memory section 686: Comments tab 64: Image Processing Unit 687: Reservation Confirmation Tab 65F: Front camera 688: Remote control tab 65R: Rear camera 689: Comment display section 66: Acoustic Processing Unit 690: Comment Input Field 67a: Speaker 691: Singer volume control 67b: Microphone 692: Speaker volume control 68: Touch panel display 693: Settings button 68a: Room selection field 694: Voice change settings field 68b: Create New Room button 695: Playback Position Adjustment button 68c: Search category selection field 696: Button 68d: Song selection field 697: Pause button 68e: Track confirmation field 698: Stop playback button 68f: Cancel button 788: Remote control tab

Claims

1. A karaoke program executable on an information processing device, A playback process that plays the music based on the music information, A first input process receives voice-changed audio, A second input process receives audio that is not subjected to voice change processing, The audio output process outputs the played music sound and the voice-changed audio. A scoring process that scores a first scoring item, which is at least a part of multiple scoring items, using a voice-changed voice, The process executes a result output process that outputs the graded results. A program for karaoke.

2. The scoring process uses voice-changed audio for all scoring items. The karaoke program according to claim 1.

3. The scoring process uses unprocessed audio for the second scoring item, which is different from the first scoring item. The karaoke program according to claim 1.

4. The second scoring criterion is at least one of the following: stability or long tone. The karaoke program according to claim 3.

5. The scoring process uses voice-changed audio to score at least one of the following scoring items: pitch, intonation, technique, stability, and long tone. The karaoke program according to claim 1.

6. The scoring process uses the pitch difference between the voice-changed audio and the un-voice-changed audio, and then corrects the pitch of either the reference data or the un-voice-changed audio before scoring. The karaoke program according to claim 1.

7. The scoring process allows users to set whether to score each scoring item using either a voice-changed or un-voice-changed audio. The karaoke program according to claim 1.

8. The scoring process can be configured to either use voice-changed audio for all scoring items, or to use un-voice-changed audio for at least some scoring items. The karaoke program according to claim 1.

9. The scoring process determines, depending on the type of voice-changing process used, whether to score all scoring items using voice-changed audio or to score at least some scoring items using unchanged audio. The karaoke program according to claim 1.

10. A playback process that plays the music based on the music information, A first input process receives voice-changed audio, A second input process receives audio that is not subjected to voice change processing, The audio output process outputs the played music sound and the voice-changed audio. A scoring process that uses voice-changed audio to score at least some of the scoring items among multiple scoring items, The process executes a result output process that outputs the graded results. Karaoke machine.

11. A playback process that plays the music based on the music information, A first input process receives voice-changed audio, A second input process receives audio that is not subjected to voice change processing, The audio output process outputs the played music sound and the voice-changed audio. A scoring process that uses voice-changed audio to score at least some of the scoring items among multiple scoring items, The process executes a result output process that outputs the graded results. Karaoke scoring method.

Images