Karaoke device

Abstract

To provide a Karaoke device which enables a plurality of users to smoothly perform Karaoke singing of medley music.SOLUTION: A Karaoke device includes: an information storage unit for storing user identification information of a user and a face image of the user in association with each other; a reservation reception unit for sequentially receiving medley reservation including the user identification information of the user, music identification information of music which the user selects and section information indicating a prescribed performance section; a determination unit for determining a singing order of each user based on a seating position on which each user seats and the face image of each user when medley reservation is received from all the users; a reservation change unit for changing an order of medley reservation according to the determined singing order; and a generation unit for generating medley music based on the changed order of medley reservation.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a karaoke machine. [Background technology]

[0002] Among the songs that can be played on a karaoke device are medley songs. A medley song is composed of a number of songs, each of which has a specific section (for example, the chorus) of its melody. A medley song may be composed only of songs by a specific artist, or may be composed of songs from a certain genre, such as pop or rock (hereinafter, each song in a medley song may be referred to as a "constituent song").

[0003] Generally, a medley song is created by a karaoke company, and therefore the constituent songs are determined in advance.

[0004] On the other hand, for example, Patent Document 1 discloses a technique for selecting parts of karaoke songs that a singer likes based on a command input via an input unit, and joining the selected parts together to make a medley. By using such a technique, a user can sing a karaoke medley consisting of only the constituent songs that the user wants to sing. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 7-295581 Summary of the Invention [Problem to be solved by the invention]

[0006] When multiple users use a karaoke machine, they may sing karaoke medley songs, taking turns singing each constituent song. In this case, when the karaoke performance of a certain constituent song is finished, the user who was singing the karaoke of that constituent song passes the microphone to the user next to him / her. The user who receives the microphone sings the karaoke of the next constituent song. By repeatedly passing the microphone in this way, multiple users can all enjoy singing karaoke medley songs.

[0007] Here, when singing karaoke to a medley song consisting of constituent songs desired by each user, which was created using the technology of Patent Document 1, depending on where the users are seated in the karaoke room, passing the microphone can become complicated, making it difficult to sing the karaoke medley song.

[0008] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide a karaoke apparatus which enables a plurality of users to smoothly sing karaoke medley pieces of music. [Means for solving the problem]

[0009] One invention for achieving the above object is a karaoke device communicatively connected to an indoor camera installed in a karaoke room, the karaoke device having an information storage unit for storing user identification information of a user of the karaoke room and a facial image of the user in association with each other, a reservation reception unit for accepting medley reservations in order, including the user identification information of the user, song identification information of a song selected by the user, and section information indicating a specified performance section, a determination unit for, when medley reservations have been accepted from all users using the karaoke room, determining the singing order of each user based on the seating positions of each user in the image captured by the indoor camera and the facial image of each user, a reservation change unit for changing the order of the medley reservations in accordance with the determined singing order, and a generation unit for generating a medley song based on the changed order of the medley reservations. Other features of the present invention will become apparent from the following specification and drawings. Effect of the Invention

[0010] According to the present invention, a plurality of users can smoothly sing karaoke medley songs. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing a karaoke room according to an embodiment. [Diagram 2] 1 is a diagram showing a karaoke device according to an embodiment; [Diagram 3] FIG. 2 is a diagram showing a karaoke main unit according to the embodiment. [Figure 4] 1 is a table that associates user identification information with face images according to the embodiment. [Diagram 5] 13 is a tentative reservation list for a medley of music pieces according to the embodiment. [Figure 6] 13 is a tentative reservation list for a medley of music pieces according to the embodiment. [Figure 7] FIG. 2 is a diagram showing a medley song according to an embodiment. [Figure 8] 4 is a flowchart showing a process of the karaoke device according to the embodiment. [Figure 9A] FIG. 4 is a diagram showing a display screen of the remote control device according to the embodiment. [Figure 9B] FIG. 4 is a diagram showing a display screen of the remote control device according to the embodiment. [Figure 10] 4 is a video image of the inside of a karaoke room captured by an indoor camera according to the embodiment. [Figure 11] 13 is a video of the inside of a karaoke room captured by an indoor camera according to the first modified example. [Figure 12] 13 is a tentative reservation list for a medley of music pieces according to the first modification example. [Figure 13] FIG. 13 is a diagram showing a medley music piece according to Modification 1. [Figure 14] 13 is a table associating user identification information with face images according to Modification 3. [Figure 15] 13 is a tentative reservation list for a medley music piece in accordance with Modification Example 3. [Figure 16] 13 is a video of the inside of a karaoke room taken by an indoor camera according to the third modified example. [Figure 17] 13 is a tentative reservation list for a medley music piece in accordance with Modification Example 3. [Figure 18] FIG. 13 is a diagram showing a medley music piece according to Modification 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <Embodiment> A karaoke device according to an embodiment will be described with reference to FIGS. 1 to 10.

[0013] ==Karaoke Equipment== A karaoke machine is a device for playing karaoke songs and for allowing users to sing karaoke. Karaoke machines are installed in karaoke rooms in karaoke stores.

[0014] The karaoke room is provided with a door for entering and exiting. An indoor camera is installed in the karaoke room to capture images of the interior of the room. The karaoke device is connected to the indoor camera via wired or wireless communication so that they can communicate with each other. The karaoke device starts communication with the indoor camera at a predetermined timing (for example, when the power is turned on). The indoor camera transmits captured images to the karaoke device continuously or at regular intervals.

[0015] The karaoke machine is communicably connected to an external server device. The server device is a computer that manages and processes various types of information. The server device is communicably connected to the karaoke machine via a network such as a public telephone line network or the Internet.

[0016] The server device stores user information. The user information relates to a user who uses the karaoke device. Specifically, the user information includes user identification information, name, nickname, a face image of the user, gender, date of birth, singing history, scoring history, etc. The user identification information is information unique to each user, such as a user ID for identifying the user. The user information is input by the user, for example, via the karaoke device (specifically, the remote control device described below). The karaoke device transmits the input user information to the server device. The server device stores the received user information in a storage means.

[0017] As shown in Fig. 1, a karaoke device K according to this embodiment is installed in a karaoke room R. Users U1 to U4 are people who use the karaoke device K (that is, the karaoke room R).

[0018] Karaoke room R is provided with a door D. An indoor camera C is installed in karaoke room R. A table T is also placed in karaoke room R. Note that a plurality of indoor cameras C may be installed in one karaoke room. Also, the karaoke device may have an indoor camera.

[0019] The karaoke device K is communicably connected to a server device SV. The server device SV stores user information on users U1 to U4.

[0020] As shown in FIG. 2, the karaoke device K includes a karaoke main unit 10, a speaker 20, a display device 30, a microphone 40, and a remote control device 50.

[0021] The karaoke main unit 10 performs various controls related to karaoke performance and singing, such as controlling the karaoke performance of songs, controlling the display of lyrics and background images, and processing audio signals input through the microphone 40. The speaker 20 is configured to emit sound based on a signal from the karaoke main unit 10. The display device 30 is configured to display videos and images on a screen based on a signal from the karaoke main unit 10. The microphone 40 is configured to convert the singing voice of the user singing karaoke into an analog audio signal and input it to the karaoke main unit 10. The remote control device 50 is a device for operating the karaoke main unit 10.

[0022] 3, the karaoke machine 10 according to this embodiment includes a storage means 10a, a communication means 10b, an input means 10c, a performance means 10d, and a control means 10e. Each of these components is connected to a bus B via an interface (not shown).

[0023] [Storage means] The storage means 10a is a large-capacity storage device that stores various data. The storage means 10a stores music data. The music data is provided with music identification information. The music identification information is information unique to each music piece, such as a music ID for identifying the music piece. The music data includes karaoke performance data, reference data, section information, and the like. The section information indicates a performance section. A performance section is a section in which karaoke performance is performed. A performance section includes a singing section and a non-singing section. A singing section is a section in which lyrics to be sung in a music piece are set (for example, the A-melody, B-melody, and chorus of the first verse). A non-singing section is a section in which lyrics to be sung in a music piece are not set, such as an introduction, interlude, or postlude.

[0024] In addition, the storage means 10a stores, for each song, background image data corresponding to the background image displayed during karaoke performance, lyric subtitle data for displaying the lyrics of the song, and attribute information of the song (song title, singer name, genre, performance time, etc.).

[0025] In this embodiment, a part of the storage area of ​​the storage means 10 a functions as an information storage unit 100 .

[0026] (Information storage unit) The information storage unit 100 is for storing user identification information of users of the karaoke rooms and facial images of the users in association with each other.

[0027] The facial image of the user can be one contained in the user information received from the server device.

[0028] For example, when a user performs a login operation via remote control device 50 and logs into karaoke device K, karaoke device K can determine that use of the karaoke room has begun.

[0029] A known method can be used to log in to the karaoke device K. A user who wishes to log in operates the remote control device 50 to input his / her own user identification information and a preset password. The karaoke device K transmits a login request including the input user ID and password to the server device SV. The server device SV reads out user information corresponding to the user ID included in the received login request from the storage means and transmits it to the karaoke device K. The karaoke device K that made the login request completes the login to the karaoke device K by storing the received user information in the storage means 10a.

[0030] Karaoke device K extracts the user ID and face image from the received user information and stores them in information storage unit 100. Information storage unit 100 stores face images of users in a certain group from the time the group starts using the karaoke room until the group finishes using the karaoke room. Fig. 4 is a table that associates the user IDs and face images of users U1 to U4 who use karaoke room R. For example, face image FI***U1 is associated with user ID ***U1 of user U1.

[0031] On the other hand, the karaoke device K can determine that the use of the karaoke room has ended when all users have performed a logout operation. A known method can be used to log out from the karaoke device K. For example, the user selects a logout icon displayed on the display screen of the remote control device 50. The karaoke device K completes the logout from the karaoke device K by deleting the user information received from the server device from the storage means 10a. In this case, the karaoke device K erases the user ID and face image stored in the information storage unit 100.

[0032] [Communication means / input means] The communication means 10b provides an interface for communicating with the outside. The input means 10c is a component for allowing a user to input instructions. The input means 10c is a button or the like provided on the karaoke main unit 10. Alternatively, the remote control device 50 may function as the input means 10c.

[0033] [Means of performance] The performance means 10d, under the control of the control means 10e, performs karaoke performance of music pieces and processes signals based on singing voices input through the microphone 40. The performance means 10d includes a sound source, a mixer, an amplifier, etc. (none of which are shown).

[0034] [Control Means] The control means 10e performs various controls in the karaoke device K. The control means 10e includes a CPU and a memory (neither of which are shown in the figures). The CPU realizes various functions by executing programs stored in the memory.

[0035] In this embodiment, the CPU executes a program stored in the memory, causing the control means 10e to function as a reservation receiving section 200, a determination section 300, a reservation change section 400, a generation section 500, and a performance processing section 600 (see FIG. 3).

[0036] (Reservations Department) The reservation receiving section 200 sequentially receives medley reservations, each including the user identification information of a user, music identification information of a music piece selected by the user, and section information indicating a predetermined performance section.

[0037] The predetermined performance section is a singing section within a section in which karaoke is performed. The predetermined performance section may be preset for each song or each karaoke device, or may be arbitrarily selected by the user via the remote control device 50.

[0038] After completing login to karaoke device K, the user operates remote control device 50 to select songs that the user wishes to sing as karaoke medley songs. The user also operates remote control device 50 to select a predetermined performance section. The reservation acceptance unit 200 generates a medley reservation that associates the song ID of the song selected by the user and the predetermined performance section selected by the user with the user ID of the user. The reservation acceptance unit 200 registers the generated medley reservation in a tentative reservation list for medley songs, thereby accepting the medley reservation. Each time a medley reservation is generated, the reservation acceptance unit 200 registers it in the tentative reservation list for medley songs in order.

[0039] 5 is an example of a tentative reservation list for medley songs. In this example, medley reservations MR1, MR2, MR3, and MR4 are registered in this order. For example, the medley reservation MR1 is associated with the user ID ***U1 of user U1, the song ID ***X1 of the song X1 selected by user U1, and the predetermined performance section (i.e., section information) "chorus 1" selected by user U1.

[0040] (Decision Section) When the determination unit 300 receives medley reservations from all users who use the karaoke room, it determines the singing order of each user based on the seating positions of each user and the facial images of each user in the image captured by the indoor camera.

[0041] A known image processing technique can be used to identify the seating positions and facial images of users in the video. The singing order is a list in which facial images of users are registered in order. The singing order is determined based on a predetermined condition. The predetermined condition is a condition that makes it easy to pass the microphone, such as "the user who is the leftmost in the video of the indoor camera is the first, and the users who appear on the rightmost in the video of the indoor camera are ...

[0042] The determination unit 300 refers to the tentative reservation list for medley music pieces at a predetermined timing, and checks whether or not medley reservations have been accepted from all users who use the karaoke room.

[0043] Specifically, the determining unit 300 compares the user ID stored in the information storage unit 100 with the user ID included in the medley reservation registered in the tentative reservation list for the medley music piece.

[0044] When all user IDs included in the medley reservations registered in the tentative reservation list for the medley music piece are stored in the information storage unit 100, the determination unit 300 determines that medley reservations have been accepted from all users.

[0045] In this case, the determination unit 300 analyzes the video captured by the indoor camera at the time (or immediately before or after) when it is determined that medley reservations have been accepted from all users, and identifies the seating positions where the users are sitting and the facial images of each user. The determination unit 300 determines the singing order for each facial image by referring to the identified seating positions and facial images of each user and predetermined conditions (details will be described later). The determination unit 300 outputs the identified facial images and the determined singing order to the reservation change unit 400.

[0046] (Reservation Change Department) The reservation change section 400 changes the order of medley reservations according to the determined singing order.

[0047] For example, the reservation change unit 400 identifies a face image that matches the face image that is first in the determined singing order from among the face images stored in the information storage unit 100, and extracts a user ID associated with the face image. The reservation change unit 400 refers to the tentative reservation list for medley music and identifies a medley reservation that includes a user ID that matches the extracted user ID. The reservation change unit 400 updates the tentative reservation list for medley music by changing the order of the identified medley reservation to be first. Note that if the extracted medley reservation is already registered first, the reservation change unit 400 does not change the order. The reservation change unit 400 changes the order of the medley reservations registered in the tentative reservation list for medley music by performing the above process on all face images included in the determined singing order.

[0048] Fig. 6 shows an example where the tentative reservation list for medley music pieces in Fig. 5 has been changed. In this example, the medley reservations are registered in the order of MR3, MR2, MR4, and MR1. In other words, the order of the medley reservations MR1, MR3, and MR4 has been changed (the process relating to the change will be described later).

[0049] (Generation part) The generating section 500 generates a medley musical piece based on the changed order of the medley reservations.

[0050] The generating unit 500 sequentially reads out the song IDs and section information registered in the tentative reservation list for the medley song, and generates the medley song by assigning the song ID for the medley song. The generating unit 500 registers the song ID of the generated medley song in a reservation queue.

[0051] Fig. 7 shows the structure of a medley song M1 generated based on the tentative reservation list for medley songs in Fig. 6. The medley song M1 is assigned a song ID ***M1. The medley song M1 is made up of songs X1, X2, X3, and X4. The medley song M1 is performed karaoke in the following order: the second B-melody of song X3, the first A-melody of song X2, the second chorus of song X4, and the first chorus of song X1.

[0052] When registering the music IDs of the generated medley music pieces in the reservation queue, the generating unit 500 may register them at the end of the reservation queue, or may make an interrupt reservation at the top of the reservation queue. Also, the medley music pieces may incorporate connecting phrases (for example, short interludes) to make it easier to match the timing and pitch between the music pieces when singing each piece. The connecting phrase data is stored in advance in the storage means 10a.

[0053] (Performance processing unit) The performance processing unit 600 controls the performance means 10d to perform karaoke performance of the reserved music piece.

[0054] The performance control unit 600 acquires karaoke performance data of a song from the storage means 10a based on the song identification information registered in the reservation queue. The performance control unit 600 controls the performance means 10d to play the acquired karaoke performance data and emit karaoke performance sounds from the speaker 20. The user can use the microphone 40 to sing karaoke in time with the karaoke performance sounds emitted from the speaker 20.

[0055] Here, the performance processing unit 600 according to this embodiment can control the performance means 10d to perform karaoke performance of the reserved medley music pieces.

[0056] The performance processing unit 600 reads out the song IDs and section information included in each medley reservation that constitutes the medley song based on the song IDs for the medley songs registered in the reservation queue. The performance processing unit 600 acquires karaoke performance data for a predetermined performance section of each song from the storage unit 10a based on the read song IDs and section information. The performance processing unit 600 controls the performance unit 10d to play back the acquired karaoke performance data in accordance with the performance order of the medley songs and to emit karaoke performance sounds from the speaker 20.

[0057] ==About the operation of the karaoke machine== Next, a specific example of the operation of the karaoke device K in this embodiment will be described with reference to Figs. 8 to 10. Fig. 8 is a flow chart showing the operation of the karaoke device K. Figs. 9A and 9B are display screens of the remote control device 50. Fig. 10 is an example of an image captured inside the karaoke room R by the indoor camera C. In this example, as shown in Fig. 1, four users U1 to U4 are using the karaoke room R (i.e., the karaoke device K). It is also assumed that each user has completed logging in to the karaoke device K, and that the user ID and face image shown in Fig. 3 are stored in the information storage unit 100.

[0058] The user operates remote control device 50 to select songs that the user wishes to sing as karaoke medley songs. The user also operates remote control device 50 to select a predetermined performance section. The reservation acceptance unit 200 accepts medley reservations in order, each of which includes the user identification information of the user, the song ID of the song selected by the user, and section information indicating the predetermined performance section (accepting medley reservations in order; step 10).

[0059] When medley reservations are accepted from all users (users U1 to U4) using karaoke room R (Y in step 11), the determination unit 300 determines the singing order for each user based on the seating positions of each user and the facial images of each user in the image captured by the indoor camera C (determining the singing order for each user; step 12).

[0060] The reservation change section 400 changes the order of the medley reservations in accordance with the singing order determined in step 12 (changing the order of the medley reservations; step 13).

[0061] The generating section 500 generates a medley music piece based on the order of the medley reservations changed in step 13 (generate a medley music piece; step 14).

[0062] The performance processing unit 600 controls the performance means 10d to perform karaoke performance of the medley music piece generated in step 14 (karaoke performance of the medley music piece, step 15).

[0063] Specifically, after completing login to the karaoke device K, the user U1 operates the remote control device 50 to select the song X1 (see FIG. 9A). If the user U1 wishes to sing karaoke using the song X1 as a medley song, the user U1 selects the "medley reservation" icon shown in FIG. 9A. In response to the selection of the "medley reservation" icon, the remote control device 50 displays a screen for selecting section information (see FIG. 9B). Here, it is assumed that the user U1 operates the remote control device 50 to select "first chorus" as the predetermined performance section. In this case, the reservation receiving unit 200 generates a medley reservation MR1 in which the song ID ***X1 of the song X1 selected by the user U1 and the predetermined performance section "first chorus" selected by the user U1 are associated with the user ID ***U1 of the user U1, and registers the medley reservation MR1 in the tentative reservation list for medley songs.

[0064] Next, assume that user U2 selects song X2 by the same operation as above and selects "A melody of verse 1" as the predetermined performance section. In this case, the reservation receiving unit 200 generates a medley reservation MR2 that associates the song ID ***X2 of song X2 selected by user U2 and the predetermined performance section "A melody of verse 1" selected by user U2 with the user ID ***U2 of user U2, and registers it in the tentative reservation list for medley songs next to the medley reservation MR1.

[0065] Next, assume that user U3 selects song X3 by performing the same operation as above and selects "2nd verse B melody" as the predetermined performance section. In this case, the reservation receiving unit 200 generates a medley reservation MR3 that associates the song ID ***X3 of song X3 selected by user U3 and the predetermined performance section "2nd verse B melody" selected by user U3 with the user ID ***U3 of user U3, and registers it in the tentative reservation list for medley songs next to the medley reservation MR2.

[0066] Finally, let us assume that user U4 selects song X4 by the same operation as above, and selects "second chorus" as the predetermined performance section. In this case, the reservation receiving unit 200 generates a medley reservation MR4 that associates the song ID ***X4 of song X4 selected by user U4 and the predetermined performance section "second chorus" selected by user U4 with the user ID ***U4 of user U4, and registers it in the tentative reservation list for medley songs next to the medley reservation MR3 (see FIG. 5).

[0067] Thereafter, the determination unit 300 refers to the tentative reservation list for the medley music piece, and checks whether or not medley reservations have been accepted from all users U1 to U4 who use the karaoke room R.

[0068] The determining unit 300 compares the user IDs stored in the information storage unit 100 shown in Fig. 3 with the user IDs included in the medley reservations registered in the tentative reservation list for medley music shown in Fig. 5. In this example, all user IDs included in the medley reservations registered in the tentative reservation list for medley music are stored in the information storage unit 100. Therefore, the determining unit 300 determines that medley reservations have been accepted from all users.

[0069] When it is determined that medley reservations have been accepted from all users, the determining unit 300 analyzes the video captured by the indoor camera C and identifies the seating positions of the users U1 to U4 and the facial images of each user.

[0070] Here, it is assumed that the image shown in Fig. 10 is obtained as an image captured by indoor camera C. It is also assumed that the predetermined condition is set such that "the user who is on the far left in the image captured by the indoor camera is number 1, and the users thereafter are ranked in order in a clockwise direction."

[0071] The determination unit 300 rearranges the facial images of the users identified from the video of FIG. 10 in order while referring to the predetermined conditions and the seating positions. That is, the determination unit 300 determines the singing order in such a way that the facial image of the user who is seated at the leftmost position in the video (the facial image of user U3) is the first, the facial image of the user who is seated to the right of user U3 (the facial image of user U2) is the second, the facial image of the user who is seated to the right of user U2 (the facial image of user U4) is the third, and the facial image of the user who is seated to the right of user U4 (the facial image of user U1) is the fourth. The determination unit 300 outputs the determined singing order to the reservation change unit 400. Note that the determination unit 300 cannot distinguish whether a certain user in the video is one of users U1 to U4.

[0072] The reservation change unit 400 identifies, from among the facial images stored in the information storage unit 100, a facial image FI***U3 that matches the facial image of user U3 who is first in the determined singing order, and extracts a user ID***U3 associated with the facial image FI***U3 (see FIG. 4). The reservation change unit 400 refers to the tentative reservation list for medley music (see FIG. 5), and identifies a medley reservation MR3 that includes a user ID that matches the extracted user ID***U3. The reservation change unit 400 updates the tentative reservation list for medley music by changing the order of the identified medley reservation MR3 so that it becomes first. In this embodiment, the reservation change unit 400 updates the provisional reservation list for medley songs by swapping the medley reservation MR1, which was first in the provisional reservation list for medley songs before the change, with the medley reservation MR3, which was third in the order. However, by moving the medley reservation MR3, which was third in the provisional reservation list for medley songs before the change, to the first position, the medley reservation MR1, which was first in the order, and the medley reservation MR2, which was second in the order, may be demoted to second and third positions, respectively.

[0073] Next, the reservation change unit 400 identifies, from among the facial images stored in the information storage unit 100, a facial image FI***U2 that matches the facial image of user U2 who is second in the determined singing order, and extracts the user ID***U2 associated with the facial image FI***U2 (see FIG. 4). The reservation change unit 400 refers to the tentative reservation list for medley music (see FIG. 5) and identifies a medley reservation MR2 that includes a user ID that matches the extracted user ID***U2. Here, in the example of FIG. 5, the medley reservation MR2 is second in order. Therefore, the reservation change unit 400 does not change the order of the identified medley reservation MR2.

[0074] Next, the reservation change unit 400 identifies a face image FI***U4 that matches the face image of the user U4 who is third in the determined singing order from among the face images stored in the information storage unit 100, and extracts the user ID***U4 associated with the face image FI***U4 (see FIG. 4). The reservation change unit 400 refers to the tentative reservation list for medley music (see FIG. 5) and identifies a medley reservation MR4 that includes a user ID that matches the extracted user ID***U4. The reservation change unit 400 updates the tentative reservation list for medley music by changing the order of the identified medley reservation MR4 so that it becomes third. In this embodiment, the reservation change unit 400 updates the tentative reservation list for medley music by swapping the medley reservation MR1, which was third in the tentative reservation list for medley music before the change, with the medley reservation MR4, which was fourth in the order.

[0075] Finally, the reservation change unit 400 identifies the face image FI***U1 that matches the face image of the user U1 whose singing order is decided to be fourth among the face images stored in the information storage unit 100, and extracts the user ID***U1 associated with the face image FI***U1 (see FIG. 4). The reservation change unit 400 refers to the tentative reservation list for medley music (see FIG. 5) and identifies the medley reservation MR1 that includes the user ID that matches the extracted user ID***U1. The reservation change unit 400 updates the tentative reservation list for medley music by changing the order of the identified medley reservation MR1 so that it becomes fourth (see FIG. 6). Note that, by performing the above-mentioned process in this embodiment, the order of the medley reservation MR1 is already fourth at the time when the user ID***U1 is extracted. Therefore, the reservation change unit 400 does not change the order of the identified medley reservation MR1.

[0076] The generation unit 500 sequentially reads out the song IDs and section information registered in the updated tentative reservation list for the medley song, and generates the medley song M1 by assigning the song ID ***M1 for the medley song (see FIG. 7). The generation unit 500 registers the song IDs of the generated medley songs in a reservation queue.

[0077] When the performance turn of the medley piece M1 arrives, the performance processing unit 600 reads out the piece ID ***X3 and section information (second B verse) included in the medley reservation MR3 constituting the medley piece M1, the piece ID ***X2 and section information (first A verse) included in the medley reservation MR2, the piece ID ***X4 and section information (second chorus) included in the medley reservation MR4, and the piece ID ***X1 and section information (first chorus) included in the medley reservation MR1, based on the piece ID ***M1 of the medley piece M1 registered in the reservation queue. The performance processing unit 600 acquires karaoke performance data of a predetermined performance section of each piece from the storage unit 10a based on the read piece ID and section information. The performance processing unit 600 controls the performance unit 10d to reproduce the acquired karaoke performance data in the performance order of the medley piece M1 (the order of the medley reservations MR3, MR2, MR4, and MR1) and to emit karaoke performance sounds from the speaker 20.

[0078] In this case, user U3 first sings the second B verse of song X3, which he selected, in sync with the karaoke performance of the medley song M1. After singing the second B verse of song X3, user U3 passes the microphone 40 to user U2, who is sitting next to him. Following user U3's karaoke singing, user U2 sings the first A verse of song X2, which he selected. After singing the first A verse of song X2, user U2 passes the microphone 40 to user U4, who is sitting next to him. Following user U2's karaoke singing, user U4 sings the second chorus of song X4, which he selected. After singing the second chorus of song X4, user U4 passes the microphone 40 to user U1, who is sitting next to him. Following user U4's karaoke singing, user U1 sings the first chorus of song X1 that he / she selected. In this way, each user can simply hand the microphone over to the user next to them after singing their own karaoke, and there is no need to know which user will sing the next song, so the microphone can be easily handed over. Therefore, karaoke singing can be done smoothly.

[0079] As is clear from the above, the karaoke device K according to this embodiment is connected to an indoor camera C installed in a karaoke room R so as to be able to communicate with the karaoke device K. The karaoke device K includes an information storage unit 100 for storing user identification information of users of the karaoke room R and facial images of the users in association with each other, a reservation acceptance unit 200 for accepting medley reservations in order, including the user identification information of the users, music identification information of the music selected by the users, and section information indicating a predetermined performance section, a determination unit 300 for determining the singing order of each user based on the seating positions of the users and the facial images of the users in the video captured by the indoor camera when medley reservations are accepted from all users using the karaoke room, a reservation change unit 400 for changing the order of the medley reservations in accordance with the decided singing order, and a generation unit 500 for generating a medley music piece based on the changed order of the medley reservations.

[0080] According to such a karaoke device K, the order of medley reservations can be changed according to the singing order determined based on the extracted seating positions and the facial images of each user. Therefore, when a plurality of users take turns singing karaoke for each component song of a medley, a user who has finished singing karaoke only needs to hand over the microphone to the user next to him / her, and there is no need to check who will be the next user to sing karaoke or to hand over the microphone to a user sitting far away, so karaoke singing is not interrupted. In other words, according to the karaoke device K of this embodiment, a plurality of users can smoothly sing karaoke for medley songs.

[0081] <Variation 1> After the medley music piece is generated, the seating positions of the users may change before the karaoke performance begins. In this modified example, an example will be described in which a medley music piece can be smoothly sung by multiple users even if the seating positions of the users change after the medley music piece is generated.

[0082] The determination unit 300 in this modified example determines a new singing order for each user based on the new seating positions of each user and facial images of each user in the image captured by the indoor camera when the karaoke performance of a medley song begins.

[0083] For example, suppose that when a karaoke performance of a medley song starts, the image shown in Fig. 11 is captured by indoor camera C. Also, suppose that the predetermined condition is set such that "the user who is on the far left in the image captured by the indoor camera is number 1, and the users thereafter are ranked in order in a clockwise direction."

[0084] The determination unit 300 rearranges the facial images of the users identified from the video of Fig. 11 in order while referring to the predetermined conditions and the seating positions. That is, the determination unit 300 determines a new singing order for each facial image, with the facial image of the user who is seated at the leftmost position in the video (the facial image of user U3) as the first, the facial image of the user who is seated to the right of user U3 (the facial image of user U2) as the second, the facial image of the user who is seated to the right of user U2 (the facial image of user U1) as the third, and the facial image of the user who is seated to the right of user U1 (the facial image of user U4) as the fourth. The determination unit 300 outputs the identified facial images and the determined new singing order to the reservation change unit 400.

[0085] The reservation change section 400 according to this modified example changes the order of medley reservations in accordance with the determined new singing order.

[0086] A reservation change section 400 compares the singing order determined when medley reservations are accepted from all users who use the karaoke room with the new singing order, and changes the order of medley reservations for orders in which the face images do not match.

[0087] For example, in the embodiment, the singing order determined when medley reservations are accepted from all users who use the karaoke room is the facial image of user U3, the facial image of user U2, the facial image of user U4, and the facial image of user U1. On the other hand, the new singing order determined in the above example is the facial image of user U3, the facial image of user U2, the facial image of user U1, and the facial image of user U4. That is, in this example, the third and fourth facial images do not match.

[0088] Therefore, the reservation change section 400 changes the order of the medley reservations for the third and fourth. Specifically, the reservation change section 400 updates the tentative reservation list for medley music by interchanging the third medley reservation MR4 with the fourth medley reservation MR1 (see FIG. 12).

[0089] The generating section 500 according to this modification modifies the generated medley music piece based on the changed order of medley reservations.

[0090] For example, in the embodiment, the medley music piece M1 shown in Fig. 7 is generated. If the order of the medley reservations is changed in accordance with a new singing order, the generating unit 500 reads out the music IDs and section information registered in the updated tentative reservation list for the medley music pieces (see Fig. 12) in order, and generates a new medley music piece M1 (see Fig. 13).

[0091] As is clear from the above, the determination unit 300 according to this modification determines a new singing order for each user based on the new seating positions of each user and the facial images of each user in the image captured by the indoor camera C at the start of karaoke performance of the generated medley music piece, the reservation change unit 400 changes the order of medley reservations according to the determined new singing order, and the generation unit 500 modifies the generated medley music piece based on the changed order of medley reservations. According to such a karaoke device K, if the seating positions of users are changed after generating a medley music piece, the already generated medley music piece can be modified based on the changed order of medley reservations according to the change in seating positions. In other words, according to the karaoke device K according to this modification, even if the seating positions of users are changed, karaoke singing of the medley music piece by multiple users can be performed smoothly.

[0092] <Variation 2> There may be a case where one user is away from the seat when the performance of a medley piece starts. In this modified example, an example will be described in which multiple users can smoothly sing karaoke of a medley piece even if one user is away from the seat when the performance of the medley piece starts.

[0093] In this modified example, when a user is away from the seat when the performance of a medley piece of music begins, the performance processing unit 600 controls the performance means 10d so as not to perform karaoke for a specified performance section of the music included in the medley reservation corresponding to that user.

[0094] When the karaoke performance of the medley music piece starts, the karaoke device K analyzes the image from the indoor camera and extracts the face image of the user.

[0095] The karaoke device K refers to the information storage unit 100 and reads out a face image associated with a user ID included in a medley reservation that constitutes a medley song. The karaoke device K compares the extracted face image with the read face image. If the extracted face image matches the face image read out from the information storage unit 100, the karaoke device K does not perform any special process.

[0096] On the other hand, if the extracted facial image does not match the facial image read from the information storage unit 100 (i.e., if the facial image read from the information storage unit 100 contains a facial image other than the extracted facial image), the karaoke device K outputs the song ID and section information associated with the mismatched facial image to the performance processing unit 600.

[0097] When performing karaoke performance of a medley piece of music, the performance processing unit 600 controls the performance means 10d based on the music ID and section information so as not to perform karaoke performance of a predetermined performance section of the music.

[0098] For example, in the embodiment, assume that user U2 leaves the room just before the karaoke performance of medley piece M1 is to be performed. In this case, user U2 is not seen in the video captured by indoor camera C at the start of the karaoke performance of the medley piece. Therefore, karaoke device K compares the extracted face images (users U1, U3, U4) with the face images (users U1 to U4) associated with the user IDs included in the medley reservation that constitutes the medley piece, and outputs the piece ID ***X2 and section information (first verse A melody) associated with the mismatched face image (user U2) to the performance processing unit 600.

[0099] Thereafter, the performance processing unit 600 starts the karaoke performance of the medley piece M1. Here, based on the piece ID ***M1 of the medley piece M1 registered in the reservation queue, the performance processing unit 600 reads out the piece ID ***X3 and section information (second B verse) included in the medley reservation MR3 constituting the medley piece M1, the piece ID ***X2 and section information (first A verse) included in the medley reservation MR2, the piece ID ***X4 and section information (second chorus) included in the medley reservation MR4, and the piece ID ***X1 and section information (first chorus) included in the medley reservation MR1. Based on the read piece ID and section information, the performance processing unit 600 acquires karaoke performance data of a predetermined performance section of each piece from the storage unit 10a. The performance processing unit 600 controls the performance unit 10d to reproduce the acquired karaoke performance data in the order of the medley reservations (medley reservations MR3, MR2, MR4, MR1) and to emit karaoke performance sounds from the speaker 20. In this modified example, the performance processing unit 600 controls the performance means 10d to skip the karaoke performance of the A-melody of the first verse of the music piece X2 included in the medley reservation MR2.

[0100] The performance processing unit 600 may avoid performing karaoke performance of the A-melody 1 of the song X2 included in the medley reservation MR2 by not reading out the song ID ***X2 and section information (A-melody 1) included in the medley reservation MR2, or by not acquiring karaoke performance data of the A-melody 1 of the song X2 from the storage means 10a.

[0101] As is clear from the above, the karaoke device K according to this modification has a performance processing unit 600 that controls the performance means 10d not to perform karaoke performance of a predetermined performance section of a song included in a medley reservation corresponding to a single user when the user leaves the seat when the performance of the medley song starts. With such a karaoke device K, even if some of the users leave the seat after the medley song is generated, multiple users can smoothly sing the karaoke of the medley song.

[0102] <Variation 3> After a medley song is generated, a new user may enter the karaoke room. In this modified example, when a new user joins the karaoke room after the medley song is generated, the new user can also sing the medley song as karaoke.

[0103] When a new user is present in the karaoke room after a medley song has been generated, the reservation reception unit 200 in this modified example accepts a new medley reservation including the user identification information of the new user, the song identification information of the song selected by the new user, and section information indicating a specified performance section.

[0104] For example, after the medley piece M1 described in the embodiment is generated, a new user U5 enters the karaoke room R before the karaoke performance of the medley piece M1. The user U5 operates the remote control device 50 to input his / her own user ID ***U5 and a preset password. The karaoke device K transmits a login request including the input user ID ***U5 and password to the server device SV. The server device SV reads out the user information corresponding to the user ID ***U5 included in the received login request from the storage means and transmits it to the karaoke device K. The karaoke device K that made the login request stores the received user information in the storage means 10a, thereby completing the login to the karaoke device K.

[0105] The karaoke device K extracts the user ID***U5 and the face image FI***U5 of the user U5 from the received user information, and stores them in the information storage unit 100 (see FIG. 14).

[0106] After completing login to the karaoke device K, the user U5 operates the remote control device 50 to select the song X5 and the "third chorus" as the predetermined performance section. In this case, the reservation receiving unit 200 generates a medley reservation MR5 that associates the song ID ***X5 of the song X5 selected by the user U5 and the predetermined performance section "third chorus" selected by the user U5 with the user ID ***U5 of the user U5, and registers it in the provisional reservation list for medley songs (see FIG. 15). In this example, the provisional reservation list for medley songs already has medley reservations MR1 ​​to MR4 registered therein, and the medley reservation MR5 is registered at the end of the provisional reservation list for medley songs.

[0107] In addition, if the time between the user U5's entry into the room and the start of the karaoke performance of the medley songs is short, it is difficult for the user U5 to select songs, etc. In such a case, the reservation acceptance unit 200 may select one song from the songs in the singing history included in the user information received from the server device (i.e., songs that the user U5 can sing karaoke), and accept a medley reservation based on a predetermined performance section (for example, the first chorus). In this modified example, the song automatically selected in such a case also corresponds to "a song selected by the user."

[0108] The determination unit 300 in this modified example determines a new singing order for each user based on the seating positions of each user, including the new user, and facial images of each user extracted from footage captured by an indoor camera.

[0109] For example, suppose that after user U5 enters karaoke room R, the image shown in Fig. 16 is captured by indoor camera C. Also, suppose that a predetermined condition is set such that "the user who appears on the far left in the image captured by the indoor camera is number 1, and the users thereafter are ranked in order in a clockwise direction."

[0110] The determination unit 300 rearranges the face images of the users identified from the video of FIG. 16 in order while referring to the predetermined conditions and the seating positions. That is, the determination unit 300 determines a new singing order for each face image, with the face image of the user who is seated at the leftmost position in the video (face image of user U3) as the first, the face image of the user who is seated to the right of user U3 (face image of user U2) as the second, the face image of the user who is seated to the right of user U2 (face image of user U4) as the third, the face image of the user who is seated to the right of user U4 (face image of user U5) as the fourth, and the face image of the user who is seated to the right of user U5 (face image of user U1) as the fifth. The determination unit 300 outputs the identified face images and the determined new singing order to the reservation change unit 400.

[0111] The reservation change section 400 according to this modified example changes the order of medley reservations to include a new medley reservation, in accordance with the determined new singing order.

[0112] In this modified example, the medley reservation MR3 is first, the medley reservation MR2 is second, and the medley reservation MR4 is third, as shown in Fig. 15. Therefore, the reservation change section 400 does not change the order of the identified medley reservation MR3, medley reservation MR2, and medley reservation MR4.

[0113] On the other hand, the reservation change unit 400 identifies a face image FI***U5 that matches the face image of the user U5 who is fourth in the decided singing order from among the face images stored in the information storage unit 100, and extracts a user ID***U5 associated with the face image FI***U5 (see FIG. 14). The reservation change unit 400 refers to the tentative reservation list for medley music (see FIG. 15) and identifies a medley reservation MR5 that includes a user ID that matches the extracted user ID***U5. The reservation change unit 400 updates the tentative reservation list for medley music by changing the order of the identified medley reservation MR5 so that it becomes fourth. Specifically, the reservation change unit 400 updates the tentative reservation list for medley music by interchanging the medley reservation MR1, which was fourth in the tentative reservation list for medley music before the change, with the medley reservation MR5, which was fifth in the order (see FIG. 17).

[0114] The generating section 500 according to this modified example generates a new medley musical piece based on the changed order of medley reservations.

[0115] In the above example, the generating section 500 sequentially reads out the music IDs and section information registered in the updated tentative reservation list for the medley music, and generates a new medley music M1 (see FIG. 18).

[0116] As is clear from the above, in the karaoke device K according to this modification, when a new user is present in the karaoke room after the generation of a medley song, the reservation receiving unit 200 accepts a new medley reservation including the user identification information of the new user, the song identification information of the song selected by the new user, and section information indicating a predetermined performance section, the determination unit 300 determines a new singing order for each user based on the seating positions of each user including the new user and the facial images of each user extracted from the video captured by the indoor camera C, the reservation change unit 400 changes the order of the medley reservations to include the new medley reservation according to the determined new singing order, and the generation unit 500 generates a new medley song based on the changed order of the medley reservations. According to such a karaoke device K, when a new user joins after the generation of a medley song, a new medley song can be generated that incorporates the constituent songs (the predetermined performance sections of the constituent songs) that the new user sings karaoke. Therefore, the new user can also enjoy singing the medley song karaoke.

[0117] <Other> Some examples of combinations of the disclosed technical features are set out below. (1) A karaoke device that is communicably connected to an indoor camera installed in a karaoke room, an information storage unit for storing user identification information of a user of the karaoke room and a face image of the user in association with each other; a reservation receiving unit that sequentially receives medley reservations including user identification information of a user, music identification information of a music piece selected by the user, and section information indicating a predetermined performance section; a decision unit that decides the singing order of each user based on the seating positions of each user and facial images of each user in an image captured by an indoor camera when medley reservations are received from all users who use the karaoke room; a reservation change unit that changes the order of the medley reservations in accordance with the determined singing order; a generating unit that generates a medley music piece based on the changed order of the medley reservations; A karaoke device having the following features: (2) the determination unit determines a new singing order for each user based on a new seating position of each user and a face image of each user in the image captured by the indoor camera at the start of karaoke performance of the generated medley song; the reservation change unit changes the order of medley reservations in accordance with the determined new singing order; The karaoke device according to (1), wherein the generating unit modifies the generated medley music piece based on the changed order of the medley reservations. (3) The karaoke device according to (1) or (2), further comprising a performance processing unit that controls the performance means so as not to perform karaoke for a specified performance section of a song included in a medley reservation corresponding to a certain user when the user is away from the seat when the performance of the medley song begins. (4) When a new user is present in the karaoke room after the medley music piece is generated, the reservation accepting unit accepts a new medley reservation including user identification information of the new user, music identification information of a music piece selected by the new user, and section information indicating a predetermined performance section; The determination unit determines a new singing order for each user based on a seating position of each user, including the new user, and a face image of each user in the image captured by the indoor camera, and the reservation change unit changes the order of medley reservations to include the new medley reservation in accordance with the determined new singing order; The karaoke device according to any one of (1) to (3), wherein the generating unit generates a new medley piece based on the changed order of the medley reservations.

[0118] The above-mentioned embodiments are presented as examples and do not limit the scope of the invention. The above-mentioned configurations can be implemented in appropriate combinations, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. The above-mentioned embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention. [Explanation of symbols]

[0119] 100 Information storage section 200 Reservations 300 Decision Section 400 Reservation Change Department 500 generation section 600 Performance processing section C Indoor camera K Karaoke equipment SV server device

Claims

1. A karaoke device that is communicably connected to an indoor camera installed in a karaoke room, an information storage unit for storing user identification information of a user of the karaoke room and a face image of the user in association with each other; a reservation receiving unit that sequentially receives medley reservations including user identification information of a user, music identification information of a music piece selected by the user, and section information indicating a predetermined performance section; a decision unit that decides the singing order of each user based on the seating positions of each user and facial images of each user in an image captured by an indoor camera when medley reservations are received from all users who use the karaoke room; a reservation change unit that changes the order of the medley reservations in accordance with the determined singing order; a generation unit that generates a medley music piece based on the changed order of the medley reservations; A karaoke device having the above configuration.

2. the determination unit determines a new singing order for each user based on a new seating position of each user and a face image of each user in the image captured by the indoor camera at the start of karaoke performance of the generated medley song; the reservation change unit changes the order of medley reservations in accordance with the determined new singing order; 2. The karaoke apparatus according to claim 1, wherein the generating unit modifies the generated medley music piece based on the changed order of the medley reservations.

3. 2. The karaoke apparatus according to claim 1, further comprising a performance processing unit which controls the performance means so that, when a user is away from the seat when the performance of the medley music piece starts, the karaoke performance of a predetermined performance section of the music piece included in the medley reservation corresponding to the user is not performed.

4. When a new user is present in the karaoke room after the generation of the medley music piece, the reservation receiving unit receives a new medley reservation including user identification information of the new user, music identification information of a music piece selected by the new user, and section information indicating a predetermined performance section; The determination unit determines a new singing order for each user based on a seating position of each user, including the new user, and a face image of each user in the image captured by the indoor camera, and the reservation change unit changes the order of medley reservations to include the new medley reservation in accordance with the determined new singing order; 4. The karaoke apparatus according to claim 1, wherein the generating unit generates a new medley piece based on the changed order of the medley reservations.

Images