Karaoke device

Abstract

To provide a Karaoke device capable of displaying background video with less discomfort.SOLUTION: A Karaoke device includes: an extraction part for extracting video identification information, video tempo information and beat start time; a first setting part for setting reproduction magnification when one video is reproduced; a second setting part for setting performance beat time; a third setting part for setting reproduction start time of one video; a generation part for generating a background video reproduction table; a reproduction processing part for reproducing video based on video data; and a display processing part for allowing display means to display one video during reproduction according to display time of video made to correspond to the number of one video included in script data.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a karaoke device. [Background technology]

[0002] The karaoke machine can display background images on the monitor in time with the karaoke performance.

[0003] Some background images have a visually recognizable tempo, such as an image of an audience waving their hands at a constant tempo (such a tempo that can be visually recognized from background images is sometimes referred to as "image tempo"). On the other hand, the tempo of a karaoke performance (hereinafter sometimes referred to as "performance tempo") varies from song to song. Therefore, when background images are displayed on a monitor in time with a karaoke performance, if the performance tempo and the image tempo are different, the singer will feel uncomfortable when viewing the background images.

[0004] Here, Patent Document 1 discloses a technique in which, if the tempo value indicated by the extracted video tempo information is not 0, the tempo value indicated by the performance tempo information of the music piece is divided by the tempo value indicated by the video tempo information, and the resulting value is set as the playback magnification when playing back a video, and the playback speed of the video is adjusted by the playback magnification. In this case, the video is played back at a playback speed faster (or slower) than the original playback speed. By playing back the video with the playback speed adjusted in this way, it is possible to reduce the sense of discomfort felt by the singer due to the discrepancy between the performance tempo of the music piece and the video tempo. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Application No. 2024-012917 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, some background images contain visually recognizable beats, such as images of audience members clapping at a constant tempo. However, the beats of songs performed in karaoke vary from song to song. Therefore, even if the playback speed of the background image is adjusted using conventional technology, if the visually recognizable beats from the image do not match the beats of the songs being performed, the singer will feel uncomfortable when watching the image.

[0007] An object of the present invention is to provide a karaoke device that can display a background image that does not create a sense of incongruity. [Means for solving the problem]

[0008] The invention for achieving the above object includes a video storage unit that stores, in association with one another, video identification information for identifying a plurality of videos that constitute a background video to be displayed on a display means in sync with a karaoke performance, a video keyword that indicates the image of the video, video data for displaying the video corresponding to the video identification information, a playback speed for playing the video, video tempo information that indicates the value of the tempo if the tempo can be recognized from the video and indicates a value of 0 if the tempo cannot be recognized from the video, and a beat start time that indicates the time when the first beat can be recognized in the video if the timing of the beat can be recognized from the video; an extraction unit that extracts from the video storage unit video identification information, video tempo information, and beat start time that are associated with a video keyword that matches a video keyword associated with a video number included in script data of a song to be karaoke performed; and, if the tempo value indicated by the extracted video tempo information is 0, sets a playback magnification for playing the video to 1, and if the tempo value indicated by the extracted video tempo information is not 0, sets the tempo value indicated by the performance tempo information of the song to the tempo value indicated by the video tempo information. a first setting unit that, when the tempo value indicated by the extracted video tempo information is not 0, divides the display time of a video associated with the number of the video included in the script data by the duration of one beat based on performance tempo information of the music, obtains a value, multiplies the value by the duration of one beat, and sets the result as a performance beat time for matching beats recognizable from the video with beats of the music; and a second setting unit that, when the tempo value indicated by the extracted video tempo information is 0, divides the display time of a video associated with the number of the video included in the script data by the duration of one beat based on performance tempo information of the music, obtains a value, and multiplies the result by the duration of one beat, and sets the result as a performance beat time for matching beats recognizable from the video with beats of the music. a third setting unit that sets the display time of a video associated with the number of one of the included videos as the playback start time of the one video, and if the tempo value indicated by the extracted video tempo information is not 0, calculates a value by multiplying the extracted beat start time by the set playback magnification, and sets the time obtained by subtracting this value from the set performance beat time as the playback start time; and a generation unit that generates a background video playback table by associating the extracted video identification information, the set playback magnification, and the set playback start time in numerical order of the multiple videos included in the script data of the music piece.a playback processing unit that refers to the background video playback table in accordance with the karaoke performance of the song, reads out video data and a playback speed from the video storage unit for each of the plurality of video numbers, adjusts the playback speed at a set playback magnification, and plays back a video based on the video data at the adjusted playback speed in accordance with a set playback start time; and a display processing unit that refers to the script data in accordance with the karaoke performance of the song, and displays one video currently being played on a display means in accordance with a display time of a video associated with the number of the video included in the script data. Other features of the present invention will become apparent from the following description and drawings. [Effects of the Invention]

[0009] According to the present invention, a background image that does not create a sense of incongruity can be displayed. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing a karaoke device according to an embodiment; [Figure 2] 1 is a diagram showing a karaoke machine main body according to an embodiment; [Figure 3] FIG. 2 is a diagram showing script data according to the embodiment. [Figure 4] 10 is an example of information stored in a video storage unit according to the embodiment. [Figure 5] 4 is a flowchart showing the process of the karaoke device according to the embodiment. [Figure 6] 10 is an example of a background image playback table according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] <Embodiment> A karaoke device K according to an embodiment will be described with reference to FIGS.

[0012] ==Karaoke Equipment== The karaoke device K is a device for performing karaoke music and for singers to sing karaoke. As shown in Fig. 1, the karaoke device K includes a karaoke main unit 10, a speaker 20, a display device 30, a microphone 40, and a remote control device 50.

[0013] The karaoke machine main unit 10 performs various controls related to karaoke performance and singing, such as controlling the performance of the selected song, controlling the display of lyrics and background images, and processing audio signals input through the microphone 40. The speaker 20 is configured to emit sound based on the sound emission signal from the karaoke machine main unit 10. The display device 30 is configured to display videos and images on a screen based on the signal from the karaoke machine main unit 10. The display device 30 is an example of a "display means." The microphone 40 is configured to convert the singer's singing voice into an analog audio signal and input it to the karaoke machine main unit 10. The remote control device 50 is a device for performing various operations on the karaoke machine main unit 10.

[0014] 2, the karaoke machine 10 according to this embodiment includes a storage unit 10a, a communication unit 10b, an input unit 10c, a performance unit 10d, and a control unit 10e. Each component is connected to a bus B via an interface (not shown).

[0015] [Storage means] The storage means 10a is a large-capacity storage device that stores various types of data. The storage means 10a stores music data. The music data is assigned music identification information. The music identification information is information unique to each music piece, such as a music ID for identifying the music piece. The music data includes accompaniment data, reference data, script data, etc.

[0016] The accompaniment data is data that is the source of the karaoke performance sound. The accompaniment data includes performance tempo information that indicates the tempo of the karaoke performance (i.e., the performance tempo). A predetermined value (e.g., 100 BPM, 120 BPM) is set for the performance tempo for each song. The reference data is data used as a standard when scoring the singer's karaoke singing. The script data is data related to background images to be displayed on the display device 30 or the like during the karaoke performance. In this embodiment, the background images are made up of multiple images.

[0017] Specifically, the script data includes video numbers, video display times, and video keywords. The video numbers indicate the order in which multiple videos are played. The video display times are the times at which the played video is displayed on the display means, and are indicated by the elapsed time from the start of the karaoke performance. The video keywords are words that indicate the image of the video, such as "the sea" or "audience clapping."

[0018] FIG. 3 is an example of script data for song X. The background video for song X consists of four videos. Each video is assigned a number from "1" to "4" according to the order in which it is played. Number "1" is associated with a display time of "0 msec" and a video keyword of "ocean," number "2" is associated with a display time of "30,400 msec" and a video keyword of "sand beach," number "3" is associated with a display time of "155,300 msec" and a video keyword of "audience clapping," and number "4" is associated with a display time of "210,600 msec" and a video keyword of "sunset."

[0019] The storage means 10a also stores lyrics caption data for displaying lyrics corresponding to each piece of music on the display device 30 or the like, and attribute information of the music (information about the music such as the singer's name, lyricist / composer's name, genre, etc.).

[0020] In this embodiment, a part of the storage area of ​​the storage means 10a functions as a video storage section 11a.

[0021] (Video storage section) The video storage unit 11a stores video identification information, video keywords, video data, playback speed, video tempo information, and beat start time in association with each other.

[0022] The video identification information is information for identifying multiple videos that constitute the background video displayed on the display means in sync with the karaoke performance. The video identification information is information unique to each video, such as a video ID for identifying the video. As mentioned above, the video keyword is a word that indicates the image of the video. Note that the same video keyword may be set for different videos. The video data is data for displaying the video corresponding to the video identification information on the display means. The video data stores videos ranging in length from several tens of seconds to several minutes in a predetermined data format. The playback speed indicates the speed value when the video is played. The playback speed is expressed, for example, as the frame rate (fps). The video tempo information is information indicating the tempo that can be visually recognized from the video (i.e., the video tempo). Specifically, if the tempo can be recognized from the video, the video tempo information indicates the tempo value. On the other hand, if the tempo cannot be recognized from the video, the video tempo information indicates a value of 0. The video tempo value can be expressed in BPM, similar to the performance tempo. The beat start time indicates the time when the first beat can be recognized in the video. Specifically, if the beat can be recognized from the video, the beat start time is indicated by the elapsed time from the start of playback of the video. For example, in the case of a video of an audience clapping 0.2 seconds after playback begins, the beat start time is "200 msec." On the other hand, if the beat cannot be recognized from the video, there is no beat start time. In other words, no beat start time is associated with the video. Note that in video where the beat can be recognized, the beat can be recognized at a regular interval. For example, in the video in the above example, starting from the beat start time "200 msec," the beat can be recognized for each beat based on the number of beats per minute indicated by the video tempo information.

[0023] 4 shows an example of information stored in the video storage unit 11a. For example, video ID ***m1 is associated with a video keyword "ocean," video data m1, a playback speed of "30 fps," and video tempo information of "0 BPM." No beat start time is associated with video ID ***m1. On the other hand, video ID ***m3 is associated with a video keyword "audience clapping," video data m3, a playback speed of "30 fps," video tempo information of "100 BPM," and beat start time of "200 msec."

[0024] The video keyword for video data m1 is "ocean." In other words, video based on video data m1 is video of the ocean. Such video is generally video with little movement. In other words, it is video in which the tempo and beat are not visually discernible. On the other hand, the video keyword for video data m3 is "audience clapping." In other words, video based on video data m3 is video of an audience clapping. Such video is generally video with movement, in which the same action is repeated at a regular interval. In other words, it is video in which the tempo and beat are visually discernible. Specifically, from the video displayed based on video data m3, starting from the beat start time of "200 msec," beats can be discerned every "approximately 600 msec," which is the time for one beat based on the beats per minute of "100 BPM" indicated in the video tempo information.

[0025] The video storage unit 11a may be configured as multiple storage units, such as a first storage unit that stores video identification information, video keywords, video tempo information, and beat start times in association with each other, and a second storage unit that stores video identification information, video data, and playback speed in association with each other.

[0026] [Communication means / input means] The communication means 10b provides an interface for communicating with the remote control device 50. The input means 10c is configured to allow the singer to input instructions. The input means 10c is a button or the like provided on the karaoke main unit 10. Alternatively, the remote control device 50 may function as the input means 10c.

[0027] [Means of performance] Based on the control of the control means 10e, the performance means 10d performs karaoke performance of music pieces and processes signals based on singing voices input through the microphone 40. The performance means 10d includes a sound source, a mixer, an amplifier, etc. (none of which are shown).

[0028] [Control means] The control means 10e performs various controls in the karaoke device K. The control means 10e includes a CPU and a memory (neither of which is shown). The CPU executes programs stored in the memory to realize various functions.

[0029] In this embodiment, the CPU executes a program stored in the memory, causing the control means 10e to function as an extraction unit 100, a first setting unit 200, a second setting unit 300, a third setting unit 400, a generation unit 500, a playback processing unit 600, and a display processing unit 700 (see FIG. 2).

[0030] (Extraction part) The extraction unit 100 extracts from the video storage unit 11a video identification information, video tempo information, and beat start time associated with a video keyword that matches a video keyword associated with the number of a video included in the script data of the song to be karaoke performed.

[0031] A singer who uses the karaoke device K to sing karaoke selects a song he or she wishes to sing by operating the remote control device 50. The remote control device 50 registers the song identification information of the selected song in a reservation queue.

[0032] The extraction unit 100 checks the script data included in the music data of the selected music piece and reads out video keywords in the order of video numbers. The extraction unit 100 refers to the video storage unit 11a and searches for video keywords that match the read video keywords. The extraction unit 100 extracts the video ID, video tempo information, and beat start time (if a beat start time is associated) associated with the matching video keyword. The extraction unit 100 performs the same process for all numbers included in the script data. For example, if the video numbers included in the script data are "1" to "4," the extraction unit 100 extracts four video IDs, video tempo information, and beat start times. As mentioned above, the same video keyword may be assigned to different videos in the video storage unit 11a. In this case, the extraction unit 100 randomly selects one video keyword from multiple identical video keywords and extracts the video ID, video tempo information, and beat start time associated with the one video keyword.

[0033] (First setting section) If the tempo value indicated by the extracted video tempo information is 0, the first setting unit 200 sets the playback magnification when playing back a video to 1, and if the tempo value indicated by the extracted video tempo information is not 0, the first setting unit 200 sets the tempo value indicated by the performance tempo information of the music piece divided by the tempo value indicated by the video tempo information as the playback magnification when playing back a video.

[0034] When the extraction unit 100 extracts the video ID and video tempo information (or the beat start time if associated) for one video number, the first setting unit 200 checks the tempo value indicated by the extracted video tempo information.

[0035] If the tempo value is 0, the first setting unit 200 sets the playback magnification when playing back the video to 1. If the tempo value is 0, the tempo cannot be recognized from the video, so the singer will not feel any discrepancy with the tempo of the music being played. Therefore, even if the playback speed is not changed, the singer who watches the video will not feel uncomfortable due to a discrepancy in tempo.

[0036] On the other hand, if the tempo value is not 0, the first setting unit 200 sets the tempo value indicated by the performance tempo information of the music piece divided by the tempo value indicated by the extracted video tempo information as the playback magnification. If the tempo value is not 0, the tempo can be recognized from the video, so the singer may feel a discrepancy with the performance tempo of the music piece. Therefore, if the playback speed is not changed, the singer watching the video may feel uncomfortable due to the discrepancy in tempo. For example, suppose the tempo value indicated by the performance tempo information of the music piece is "100 BPM" and the tempo value indicated by the extracted video tempo information is "50 BPM." In this case, the first setting unit 200 sets the playback magnification to "2" (= 100 BPM ÷ 50 BPM).

[0037] The first setting unit 200 may perform the above process each time the extraction unit 100 extracts the video ID and video tempo information (or the beat start time if the beat start time is associated), or may perform the above process all at once after the extraction unit 100 extracts the video ID and video tempo information (or the beat start time if the beat start time is associated) for all numbers included in the script data.

[0038] (Second setting section) If the tempo value indicated by the extracted video tempo information is not 0, the second setting unit 300 obtains a value by dividing the display time of the video associated with the number of one video included in the script data by the time of one beat based on the performance tempo information of the music, and sets the value obtained by multiplying this value by the time of one beat as the performance beat time.

[0039] The performance beat time is a time for matching the beat recognizable from one image with the beat of the music. Specifically, the performance beat time is a time corresponding to the beat that is closest to the start time of the beat of the music and does not exceed the display time of one image. The performance beat time is indicated by the elapsed time from the start of the karaoke performance of the music.

[0040] When the extraction unit 100 extracts the video ID and video tempo information (or the beat start time if associated) for one video number, the second setting unit 300 checks the tempo value indicated by the extracted video tempo information.

[0041] If the tempo value is not 0, the second setting unit 300 obtains a value by dividing the display time of a video associated with a video number included in the script data by the duration of one beat based on the performance tempo information of the music, and sets the value obtained by multiplying this value by the duration of one beat as the performance beat time. For example, assume that the display time of a video associated with a video number is "155,300 msec" and the duration of one beat based on the performance tempo information of the music is "500 msec." In this case, the second setting unit 300 divides the display time of the video, "155,300 msec," by the duration of one beat, "500 msec," to obtain a value of "310" (≒155,300 ÷ 500). The second setting unit 300 multiplies the value "310" by the duration of one beat, "500 msec," to obtain a value of "155,000 msec." The second setting section 300 sets the obtained value "155000 msec" as the performance beat time.

[0042] On the other hand, if the tempo value is 0, the second setting unit 300 does not execute the above process. Furthermore, the second setting unit 300 may execute the above process each time the extracting unit 100 extracts a video ID and video tempo information (or a beat start time if a beat start time is associated with the video ID and the tempo information), or may execute the above process all at once after the extracting unit 100 extracts a video ID and video tempo information (or a beat start time if a beat start time is associated with the video ID and the tempo information) for all numbers included in the script data.

[0043] (Third setting section) If the tempo value indicated by the extracted video tempo information is 0, the third setting unit 400 sets the display time of the video associated with the number of a video included in the script data as the playback start time of that video, and if the tempo value indicated by the extracted video tempo information is not 0, it calculates a value by multiplying the extracted beat start time by the set playback multiplier, and sets the time obtained by subtracting this value from the set performance beat time as the playback start time.

[0044] The video playback start time is the time when the video playback starts, and is indicated by the elapsed time from the start of the karaoke performance.

[0045] When the extraction unit 100 extracts the video ID and video tempo information (or the beat start time if associated) for one video number, the third setting unit 400 checks the tempo value indicated by the extracted video tempo information.

[0046] When the tempo value is 0, the third setting unit 400 sets the display time of a video associated with the number of one video included in the script data as the playback start time of that one video. When the tempo value is 0, no beat start time is associated. When no beat start time is associated, the beat cannot be recognized from the video, so the singer does not feel a discrepancy with the beat of the music. Therefore, even if the playback start time is not changed, the singer who watches the video will not feel uncomfortable due to a discrepancy with the beat.

[0047] On the other hand, if the tempo value is not 0, the third setting unit 400 multiplies the extracted beat start time by the set playback magnification, calculates a value, and subtracts the value from the set performance beat time to set the result as the playback start time. If the tempo value is not 0, a beat start time is associated. If a beat start time is associated, the beat can be recognized from the video, so the singer may feel a discrepancy with the beat of the music. Therefore, it is necessary to change the playback start time using the set performance beat time so that the singer watching the video does not feel uncomfortable due to the discrepancy with the beat. For example, suppose the extracted beat start time is "200 msec," the playback magnification set by the first setting unit 200 is "1.2," and the performance beat time set by the second setting unit 300 is "155,000 msec." In this case, the third setting unit 400 calculates "240 msec" by multiplying the beat start time "200 msec" by the set playback magnification "1.2." The third setting section 400 sets the time "154760 msec" obtained by subtracting the value "240 msec" from the performance beat time "155000 msec" as the playback start time.

[0048] (Generation part) The generation unit 500 generates a background video playback table by associating the extracted video identification information, the set playback magnification, and the set playback start time in numerical order of the multiple videos included in the script data of the music.

[0049] After the second setting unit 300 and the third setting unit 400 set the playback magnification and playback start time for all video numbers included in the script data, the generation unit 500 generates a background video playback table that associates video IDs, set playback magnifications, and set playback start times in the order of video numbers. There is a one-to-one correspondence between the video numbers included in the script data and the video numbers included in the background video playback table.

[0050] (Playback processing unit) The playback processing unit 600 refers to the background video playback table in accordance with the karaoke performance of the song, reads out video data and playback speeds from the video storage unit 11a in the order of the video numbers, adjusts the playback speeds at the set playback magnification, and plays back the video based on the video data at the adjusted playback speed in accordance with the set playback start time.

[0051] The karaoke device K acquires accompaniment data for a song from the storage means 10a based on the song identification information registered in the reservation queue. The karaoke device K controls the performance means 10d to play the acquired accompaniment data and emit karaoke performance sounds from the speaker 20. The singer can sing karaoke using the microphone 40 in time with the karaoke performance sounds emitted from the speaker 20.

[0052] In synchronization with the karaoke performance of the song, the playback processing unit 600 refers to the background video playback table and acquires the video ID associated with the video number. The playback processing unit 600 reads out the video data and playback speed corresponding to the acquired video ID from the video storage unit 11a. The playback processing unit 600 refers to the background video playback table and acquires the playback magnification associated with the video ID. The playback processing unit 600 adjusts the read playback speed by the acquired playback magnification. The playback processing unit 600 also refers to the background video playback table and acquires the playback start time associated with the video ID. The playback processing unit 600 plays back the video based on the read video data at the adjusted playback speed in synchronization with the arrival of the acquired playback start time.

[0053] For example, if the playback magnification is "1", the video played by the playback processing unit 600 is played at the original playback speed (the playback speed stored in the video storage unit 11a). On the other hand, if the playback magnification is not "1", the video played by the playback processing unit 600 is played at a playback speed faster (or slower) than the original playback speed.

[0054] The playback processing unit 600 according to this embodiment includes multiple decoders as software. Therefore, when a certain video is being played back and the playback start time of another video arrives, the playback processing unit 600 can start playing the other video. Note that the decoders may be hardware included in the karaoke device K. In this case, the playback processing unit 600 controls each decoder to play back the video according to the acquired playback start time.

[0055] (Display processing unit) The display processing unit 700 refers to the script data in accordance with the karaoke performance of the song, and displays the video being played on the display means in accordance with the display time of the video associated with the number of the video included in the script data.

[0056] When the display time of a video associated with a video number arrives, the display processing unit 700 displays the video being played back by the playback processing unit 600 on a display screen of the display device 30 or the like. Note that in this embodiment, the display time and playback start time for a video may differ. Even in this case, the playback and display of the video are executed separately, so some video is displayed on the display means.

[0057] ==About the processing of Karaoke device K== Next, a specific example of the processing of the karaoke device K in this embodiment will be described with reference to Figures 5 and 6. Figure 5 is a flowchart showing the processing of the karaoke device K. Figure 6 is a diagram showing the background video playback table generated by the generation unit 500. In this example, the script data for song X is that shown in Figure 3. The video storage unit 11a stores the information shown in Figure 4. It is assumed that the performance tempo value indicated by the performance tempo information for song X is "120 BPM."

[0058] Singer S operates remote control device 50 to select song X that he or she wishes to sing as karaoke. Remote control device 50 registers the song ID of the selected song X in the reservation queue (reserving karaoke performance of song X; step 10).

[0059] The extraction unit 100 extracts from the video storage unit 11a the video ID, video tempo information, and beat start time associated with a video keyword that matches the video keyword associated with the number of one video included in the script data of the song X (extracting the video ID, video tempo information, and beat start time; step 11).

[0060] If the tempo value indicated by the video tempo information extracted in step 11 is 0, the first setting unit 200 sets the playback magnification when playing back a video to 1, and if the tempo value indicated by the extracted video tempo information is not 0, the first setting unit 200 sets the tempo value indicated by the performance tempo information of the music piece divided by the tempo value indicated by the video tempo information as the playback magnification when playing back a video (setting the video playback magnification; step 12).

[0061] Furthermore, if the tempo value indicated by the video tempo information extracted in step 11 is not 0 (N in step 13), the second setting unit 300 obtains a value by dividing the display time of a video associated with the number of one video included in the script data of musical piece X by the time of one beat based on the performance tempo information of musical piece X, and sets the value obtained by multiplying this value by the time of one beat as the performance beat time (setting the performance beat time; step 14).

[0062] If the tempo value indicated by the video tempo information extracted in step 11 is 0, the third setting unit 400 sets the display time of a video associated with the number of one video included in the script data of musical piece X as the playback start time of that one video, and if the tempo value indicated by the extracted video tempo information is not 0, it calculates a value by multiplying the extracted beat start time by the set playback magnification, and sets the time obtained by subtracting this value from the set performance beat time as the playback start time (setting the playback start time; step 15).

[0063] The generation unit 500 generates a background video playback table by associating the extracted video IDs, the playback magnification set in step 13, and the playback start times set in step 15 in numerical order of the multiple videos included in the script data of song X (generating background video playback table; step 16).

[0064] The karaoke device K acquires accompaniment data for the song X from the storage means 10a based on the song ID registered in the reservation queue. The karaoke device K plays back the acquired accompaniment data and controls the performance means 10d to emit karaoke performance sounds from the speaker 20 (karaoke performance of the song X starts; step 17). The singer S sings karaoke using the microphone 40 in time with the karaoke performance sounds emitted from the speaker 20.

[0065] The playback processing unit 600 refers to the background video playback table generated in step 16 in accordance with the karaoke performance of song X, reads out video data and playback speeds from the video storage unit 11a in the order of the video numbers, adjusts the playback speeds at the set playback magnification, and plays back the video based on the video data at the adjusted playback speed in accordance with the playback start time of the video set in step 15 (play back the video at the adjusted playback speed; step 18).

[0066] The display processing unit 700 refers to the script data in accordance with the karaoke performance of the song X, and causes the display device 30 to display the video currently being played in accordance with the display time of the video associated with the number of the video included in the script data (displaying the video currently being played in accordance with the display time; step 19). The karaoke device K repeatedly performs the processes of steps 18 and 19 in accordance with the playback start time and display time of each video until the karaoke performance ends (if Y in step 20).

[0067] Specifically, the extraction unit 100 checks the script data (see FIG. 3) included in the music data of the song X, and reads out the video keyword "sea" associated with video number "1." The extraction unit 100 references the video storage unit 11a and searches for a video keyword that matches the read-out video keyword "sea." The extraction unit 100 extracts the video ID ***m1 and video tempo information "0 BPM" that are associated with the matching video keyword "sea." Next, the extraction unit 100 reads out the video keyword "sand beach" associated with video number "2." The extraction unit 100 references the video storage unit 11a and searches for a video keyword that matches the read-out video keyword "sand beach." The extraction unit 100 extracts the video ID ***m2 and video tempo information "0 BPM" that are associated with the matching video keyword "sand beach." Next, the extraction unit 100 reads out the video keyword "audience clapping" associated with video number "3." The extraction unit 100 refers to the video storage unit 11a and searches for a video keyword that matches the read-out video keyword "audience clapping." The extraction unit 100 extracts the video ID ***m3, video tempo information "100 BPM," and beat start time "200 msec" that are associated with the matching video keyword "audience clapping." Finally, the extraction unit 100 reads out the video keyword "sunset" that is associated with video number "4." The extraction unit 100 refers to the video storage unit 11a and searches for a video keyword that matches the read-out video keyword "sunset." The extraction unit 100 extracts the video ID ***m4 and video tempo information "0 BPM" that are associated with the matching video keyword "sunset."

[0068] For video numbers "1," "2," and "4" with a tempo value of "0 BPM," the first setting unit 200 sets the playback magnification to 1 when playing the videos corresponding to video ID ***m1, video ID ***m2, and video ID ***m4.

[0069] On the other hand, for video number "3" whose tempo value is "100 BPM", the first setting unit 200 sets the playback multiplier to "1.2", which is the value obtained by dividing the performance tempo value "120 BPM" indicated by the performance tempo information of song X by the tempo value "100 BPM" indicated by the extracted video tempo information.

[0070] Furthermore, for video number "3" with a tempo value of "100 BPM," the second setting unit 300 checks the script data (see FIG. 3) included in the music data of song X and reads out the display time "155,300 msec" associated with video number "3." Here, based on the performance tempo information of song X (i.e., the performance tempo value "120 BPM"), the duration of one beat is "500 msec." The second setting unit 300 divides the read-out display time "155,300 msec" by the duration of one beat, "500 msec," to obtain a value of "310" (≒155,300 ÷ 500). The second setting unit 300 multiplies the value "310" by the duration of one beat, "500 msec," to obtain a value of "155,000 msec." The second setting unit 300 sets the obtained value "155,000 msec" as the performance beat time.

[0071] Next, for video number "1" having a tempo value of "0 BPM", the third setting unit 400 sets the display time "0 msec" of the video associated with video number "1" included in the script data of song X as the playback start time of that video. Similarly, for video number "2" having a tempo value of "0 BPM", the third setting unit 400 sets the display time "30400 msec" of the video associated with video number "2" included in the script data of song X as the playback start time of that video, and sets the display time "210600 msec" of the video associated with video number "4" as the playback start time of that video.

[0072] On the other hand, for video number "3" with a tempo value of "100 BPM", the third setting unit 400 calculates the value "240 msec" by multiplying the extracted beat start time "200 msec" by the set playback multiplier "1.2", and sets the time "154760 msec" obtained by subtracting the value "240 msec" from the set performance beat time "155000 msec" as the playback start time.

[0073] The generation unit 500 generates a background video playback table by associating the extracted video ID, the set playback magnification, and the set playback start time with the video numbers "1" to "4" included in the script data of song X, in that order. As shown in Fig. 6, in the background video playback table in this example, video number "1" is associated with video ID ***m1, playback magnification "1," and playback start time "0 msec," video number "2" is associated with video ID ***m2, playback magnification "1," and playback start time "30400 msec," video number "3" is associated with video ID ***m3, playback magnification "1.2," and playback start time "154760 msec," and video number "4" is associated with video ID ***m4, playback magnification "1," and playback start time "210600 msec."

[0074] In synchronization with the karaoke performance of song X, the playback processing unit 600 refers to the background video playback table and acquires video ID ***m1 associated with video number "1." The playback processing unit 600 refers to the video storage unit 11a and reads out video data MD1 and a playback speed of "30 fps" corresponding to the acquired video ID ***m1. The playback processing unit 600 refers to the background video playback table and acquires the playback magnification "1" associated with video ID ***m1. The playback processing unit 600 multiplies the read playback speed "30 fps" by the acquired playback magnification "1" (in this case, the playback speed remains "30 fps"). The playback processing unit 600 refers to the background video playback table and acquires the playback start time "0 msec" associated with video number "1." In synchronization with the arrival of the acquired playback start time (i.e., simultaneously with the start of the karaoke performance), the playback processing unit 600 plays back video M1 based on the read video data MD1 at the adjusted playback speed of "30 fps."

[0075] Next, the playback processing unit 600 refers to the background video playback table and obtains the video ID ***m2 associated with video number "2." The playback processing unit 600 refers to the video storage unit 11a and reads out the video data MD2 and playback speed "30 fps" corresponding to the obtained video ID ***m2. The playback processing unit 600 refers to the background video playback table and obtains the playback magnification "1" associated with the video ID ***m2. The playback processing unit 600 multiplies the read playback speed "30 fps" by the obtained playback magnification "1" (in this case, the playback speed remains "30 fps"). The playback processing unit 600 refers to the background video playback table and obtains the playback start time "30400 msec" associated with video number "2." The playback processing unit 600 plays back the video M2 based on the read video data MD2 at the adjusted playback speed of "30 fps" when the acquired playback start time arrives (i.e., when 30,400 msec has elapsed since the start of the karaoke performance).

[0076] Next, the playback processing unit 600 refers to the background video playback table and obtains the video ID ***m3 associated with video number "3." The playback processing unit 600 refers to the video storage unit 11a and reads out the video data MD3 and playback speed "30 fps" corresponding to the obtained video ID ***m3. The playback processing unit 600 refers to the background video playback table and obtains the playback magnification "1.2" associated with the video ID ***m3. The playback processing unit 600 multiplies the read playback speed "30 fps" by the obtained playback magnification "1.2" (in this case, the playback speed becomes "36 fps"). The playback processing unit 600 refers to the background video playback table and obtains the playback start time "154760 msec" associated with video number "3." The playback processing unit 600 plays back the video M3 based on the read video data MD3 at the adjusted playback speed of "36 fps" when the acquired playback start time arrives (i.e., when 154760 msec has elapsed since the start of the karaoke performance).

[0077] Finally, the playback processing unit 600 refers to the background video playback table and acquires the video ID ***m4 associated with video number "4." The playback processing unit 600 refers to the video storage unit 11a and reads out the video data MD4 and playback speed "30 fps" corresponding to the acquired video ID ***m4. The playback processing unit 600 refers to the background video playback table and acquires the playback magnification "1" associated with the video ID ***m4. The playback processing unit 600 multiplies the read playback speed "30 fps" by the acquired playback magnification "1" (in this case, the playback speed remains "30 fps"). The playback processing unit 600 refers to the background video playback table and acquires the playback start time "210600 msec" associated with video number "4." The playback processing unit 600 plays back the video M4 based on the read video data MD4 at the adjusted playback speed of "30 fps" when the acquired playback start time arrives (i.e., when 210600 msec has elapsed since the start of the karaoke performance).

[0078] The display processing unit 700 refers to the script data in accordance with the karaoke performance of the song X, and displays the video corresponding to video number "1" that has been started to be played by the playback processing unit 600 on the display device 30 in accordance with the display time "0 msec" of the video associated with video number "1" included in the script data. Thereafter, when the display time "30400 msec" of the video associated with video number "2" arrives, the display processing unit 700 switches from the video with video number "1" to the video corresponding to video number "2" that has been started to be played by the playback processing unit 600, and displays it on the display device 30.

[0079] Thereafter, when the display time "155300 msec" of the video associated with video number "3" arrives, the display processing unit 700 switches from video number "2" to the video corresponding to video number "3" being played by the playback processing unit 600, and displays it on the display device 30. Here, the video corresponding to video number "3" began playing 154760 msec after the start of the karaoke performance, and beats can be visually recognized every 500 msec from the performance beat time of 155000 msec. When the video associated with video number "3" is displayed in this state, the visually recognizable beats from the video match the beats of the song being performed karaoke.

[0080] Thereafter, when the display time "210600 msec" of the video corresponding to video number "4" arrives, the display processing unit 700 switches from the video with video number "3" to the video corresponding to video number "4" that has started to be played by the playback processing unit 600, and displays it on the display device 30.

[0081] As is clear from the above, the karaoke device K according to this embodiment comprises a video storage unit 11a that stores, in association with one another, video identification information for identifying a plurality of videos constituting a background video to be displayed on a display means in sync with a karaoke performance, a video keyword that indicates the image of the video, video data for displaying a video corresponding to the video identification information, a playback speed for playing the video, video tempo information that indicates the tempo value if the tempo can be recognized from the video and that indicates a value of 0 if the tempo cannot be recognized from the video, and a beat start time that indicates the time at which the first beat can be recognized in the video if the timing of the beat can be recognized from the video; an extraction unit 100 that extracts from the video storage unit 11a the video identification information, video tempo information, and beat start time that are associated with a video keyword that matches a video keyword associated with the number of one video included in script data of a song to be karaoke performed; and an extraction unit 100 that sets a playback magnification for playing one video to 1 if the tempo value indicated by the extracted video tempo information is 0, and sets a tempo value indicated by the performance tempo information of the song to 1 if the tempo value indicated by the extracted video tempo information is not 0. a first setting unit (200) that sets a value obtained by dividing the value by the tempo value indicated by the video tempo information as a playback magnification when playing back one video; a second setting unit (300) that, when the tempo value indicated by the extracted video tempo information is not 0, obtains a value by dividing the display time of a video associated with the number of one video included in the script data by the time of one beat based on performance tempo information of the music, and sets the value obtained by multiplying this value by the time of one beat as a performance beat time for matching beats recognizable from the one video with beats of the music; and a third setting unit (400) that sets the display time of a video associated with the number of one video included in the data as the playback start time of that one video, and if the tempo value indicated by the extracted video tempo information is not 0, calculates a value by multiplying the extracted beat start time by a set playback magnification, and sets the time obtained by subtracting this value from the set performance beat time as the playback start time; and a generation unit (500) that generates a background video playback table by associating the extracted video identification information, the set playback magnification, and the set playback start time in numerical order of multiple videos included in the script data of the music piece.The apparatus includes a playback processing unit 600 that refers to the background video playback table in accordance with the karaoke performance of a song, reads out video data and playback speeds from the video storage unit 11a for each of a plurality of video numbers, adjusts the playback speed at a set playback magnification, and plays back video based on the video data at the adjusted playback speed in accordance with a set playback start time, and a display processing unit 700 that refers to script data in accordance with the karaoke performance of a song, and displays one video being played on a display means in accordance with the display time of the video associated with the number of the video included in the script data.

[0082] As described above, when a beat start time is associated with video tempo information, the singer may perceive the beat from the video, potentially causing the singer to feel a discrepancy with the beat of the music. Therefore, the karaoke device K of this embodiment sets a performance beat time when the tempo value indicated by the extracted video tempo information is not 0, and sets a playback start time based on the performance beat time. The karaoke device K starts playback of the video when the set playback start time arrives, and displays the video being played when the display time of the video arrives. This reduces the sense of discomfort felt by the singer due to a discrepancy between the beat visually recognizable from the video and the beat of the music being karaoke-played. In other words, the karaoke device K of this embodiment can display a background video that is less uncomfortable.

[0083] <Other> The program can also be supplied to a computer using a non-transitory computer-readable medium with an executable program stored thereon. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), CD-ROMs (Read Only Memory), etc.

[0084] The above-described embodiments are presented as examples and do not limit the scope of the invention. The above configurations can be implemented in appropriate combinations, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The above-described embodiments and their modifications are included in the scope and spirit of the invention, as well as in the inventions described in the claims and their equivalents. [Explanation of symbols]

[0085] K Karaoke equipment 10 Karaoke machine 11a Video storage section 100 Extraction part 200 First Setting Section 300 Second Setting Section 400 Third Setting Section 500 generation section 600 Reproduction processing unit 700 Display processing unit

Claims

[Claim 1] a video storage unit that stores, in association with one another, video identification information for identifying a plurality of videos constituting a background video to be displayed on a display means in sync with a karaoke performance, video keywords that indicate the image of the video, video data for displaying the video corresponding to the video identification information, a playback speed for playing the video, video tempo information that indicates the value of the tempo if the tempo can be recognized from the video and indicates a value of 0 if the tempo cannot be recognized from the video, and a beat start time that indicates the time when the first beat can be recognized in the video if the timing of the beat can be recognized from the video; an extracting unit that extracts from the video storage unit video identification information, video tempo information, and beat start time associated with a video keyword that matches a video keyword associated with a video number included in script data of a song to be karaoke performed; a first setting unit that sets a playback magnification ratio for playing back the one video to 1 when the tempo value indicated by the extracted video tempo information is 0, and sets a value obtained by dividing the tempo value indicated by performance tempo information of the music piece by the tempo value indicated by the video tempo information when the extracted tempo value indicated by the extracted video tempo information is not 0 as a playback magnification ratio for playing back the one video; a second setting unit that, when the tempo value indicated by the extracted video tempo information is not 0, obtains a value by dividing the display time of a video associated with the number of a video included in the script data by the duration of one beat based on the performance tempo information of the music piece, and sets the value obtained by multiplying this value by the duration of one beat as a performance beat time for matching the beat recognizable from the video with the beat of the music piece; a third setting unit that, when the tempo value indicated by the extracted video tempo information is 0, sets the display time of a video associated with the number of one video included in the script data as the playback start time of that one video, and, when the tempo value indicated by the extracted video tempo information is not 0, calculates a value by multiplying the extracted beat start time by the set playback magnification, and sets the time obtained by subtracting this value from the set performance beat time as the playback start time; a generation unit that generates a background video playback table by associating extracted video identification information, a set playback magnification, and a set playback start time in numerical order of the plurality of videos included in the script data of the music piece; a playback processing unit that refers to the background video playback table in accordance with the karaoke performance of the song, reads out video data and a playback speed from the video storage unit for each of the plurality of video numbers, adjusts the playback speed at a set playback magnification, and plays back a video based on the video data at the adjusted playback speed in accordance with a set playback start time; a display processing unit that refers to the script data in accordance with the karaoke performance of the song, and displays one video being played on a display means in accordance with a display time of the video associated with the number of the video included in the script data; A karaoke device having:

Images