Information processing apparatus, moving image distribution method, and moving image distribution program

Abstract

To provide an information processing apparatus, a moving image distribution method and a moving image distribution program capable of giving a boost to a viewer and a moving image itself during distribution of the moving image of a user who performs singing or the like.SOLUTION: A server receives a tip input as a display instruction of a tip effect image 60 for decorating a song moving image from a portable terminal of a viewing user, superimposes the tip effect image 60 on a song moving image at a superimposition timing T after receiving the tip input, and distributes the song moving image on which the tip effect image 60 is superimposed, to the portable terminal.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a video distribution method, and a video distribution program. [Background technology]

[0002] BACKGROUND ART In recent years, online services such as websites (web pages, web services), online games, and application software (hereinafter referred to as "apps") that are distributed to information processing devices such as mobile terminals via computer networks have become widespread.

[0003] One type of online service is a video posting site where users can post videos they have filmed and other users can view those videos. Users also post performance videos, such as singing videos, on these video posting sites. Some video posting sites also have a function that allows users who have viewed the posted videos to rate (judge) the videos.

[0004] Patent Document 1 discloses a karaoke performance device that changes an image of a singer based on the singer's singing ability. This karaoke performance device allows a viewer who listens to the singer's singing to input the judgment result of the singer's singing ability, and adds a specific image to the singer's image based on the judgment result. Patent Document 1 also discloses that, as an image to be added to the singer's image based on the judgment result, a crown is placed on the singer's image, i.e., the singer's face, if the singer's singing ability is judged to be good, and a check mark is displayed on the image of the singer's face, or the image of the singer's face explodes or is severely distorted, etc., if the singer's singing ability is judged to be poor. This modulates the image signal from the camera to add a change to the image of the singer, i.e., the singer's face, based on the judgment result. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 6-250683 Summary of the Invention [Problem to be solved by the invention]

[0006] The karaoke device disclosed in Patent Document 1 counts a judgment signal indicating the judgment result using a counter when the singer finishes singing, and changes the image of the singer according to the counting result, so it does not excite the audience while the singer is singing.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide an information processing device, a video distribution method, and a video distribution program that can liven up viewers and the video itself while a user is distributing a video of the user singing, etc. [Means for solving the problem]

[0008] In order to solve the above problems, the information processing device, the video distribution method, and the video distribution program of the present invention employ the following means.

[0009] In order to solve the above problem, one aspect of the present invention, an "information processing device," is an information processing device that distributes a video to a terminal device so that it can be viewed by a viewing user, and includes: a reception means that receives a display instruction from the terminal device of the viewing user to display a decorative image that decorates the video being distributed; a superimposition means that superimposes the decorative image on the video after the reception means receives the display instruction; and a distribution means that distributes the video with the decorative image superimposed to the terminal device.

[0010] In order to solve the above problem, a "video distribution method" according to one aspect of the present invention includes a first step of distributing a video to a terminal device so that the video can be viewed by a viewing user; a second step of receiving a display instruction from the terminal device of the viewing user to display a decorative image that decorates the video being distributed; a third step of superimposing the decorative image on the video after receiving the display instruction; and a fourth step of distributing the video with the decorative image superimposed to the terminal device.

[0011] In order to solve the above problem, one aspect of the present invention, a "video distribution program," causes a computer provided with an information processing device that distributes videos to a terminal device so that the viewing user can view them, to function as: a reception means that receives a display instruction from the terminal device of the viewing user to display a decorative image that decorates the video being distributed; a superimposition means that, after the reception means receives the display instruction, superimposes the decorative image on the video; and a distribution means that distributes the video with the decorative image superimposed to the terminal device.

[0012] Various technical limitations may be imposed on the above-mentioned "information processing device," as exemplified below. Furthermore, technical limitations of the same nature may be imposed on the processing steps executed by the "video distribution method" or the functions of the "video distribution program."

[0013] The superimposing means superimposes the decorative image on the moving image at a predetermined timing of the moving image.

[0014] A plurality of timings are provided for the moving image, and the superimposing means superimposes the decorative image on the moving image in accordance with the display instruction at the timing from the next time onward after the display instruction is received.

[0015] The video is a video that is recorded while music is being played back, and a plurality of timings are provided according to the music.

[0016] The maximum number of decorative images that can be superimposed on the moving image at the same time is set in advance, and the superimposing means superimposes the decorative images on the moving image in the order in which the display instructions are received so as not to exceed the maximum number.

[0017] If, at a predetermined timing immediately before the end of the video, there are decorative images for which the display instruction has been received but which have not been superimposed on the video, the superimposition means superimposes all of the decorative images on the video at the predetermined timing.

[0018] The decorative image has a set point, and the design varies depending on the set point.

[0019] The video camera includes an acquisition unit for acquiring, from the video, images of scenes in which the decorative images are relatively numerous. [Effects of the Invention]

[0020] According to the present invention, it is possible to provide an effect of making it possible to liven up the viewers and the video itself while a video of a user singing or the like is being distributed. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a configuration diagram of a karaoke system according to a first embodiment of the present invention. [Figure 2] 1 is a block diagram showing the electrical configuration of a server according to a first embodiment of the present invention. [Figure 3] 1 is a block diagram showing the electrical configuration of a mobile terminal according to a first embodiment of the present invention. [Figure 4] FIG. 3 is a diagram showing a screen display of a mobile terminal when shooting a singing video according to the first embodiment of the present invention. [Figure 5] FIG. 2 is a diagram showing a screen display of a mobile terminal when viewing a singing video according to the first embodiment of the present invention. [Figure 6] FIG. 10 is a diagram showing a screen display when selecting a coin tip effect image according to the first embodiment of the present invention. [Figure 7] FIG. 1 is a diagram showing a singing video on which a coin tip effect image is superimposed according to the first embodiment of the present invention. [Figure 8] FIG. 4 is a schematic diagram showing the timing of superimposing a coin throwing effect image according to the first embodiment of the present invention. [Figure 9] FIG. 1 is a functional block diagram relating to a coin tip effect function according to the first embodiment of the present invention. [Figure 10] 10 is a flowchart showing the flow of a coin throwing effect process according to the first embodiment of the present invention. [Figure 11]FIG. 10 is a schematic diagram showing the timing of superimposing a coin throwing effect image according to the second embodiment of the present invention. [Figure 12] 10 is a flowchart showing the flow of a coin throwing effect process according to a second embodiment of the present invention. [Figure 13] FIG. 10 is a diagram showing a coin throwing effect image according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0022] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing device, a video distribution method, and a video distribution program according to an embodiment of the present invention will be described below with reference to the accompanying drawings.

[0023] [1. First embodiment] A first embodiment of the present invention will be described below.

[0024] In this embodiment, a performer records their own performance as a video using a mobile device and sends it to a server, thereby uploading it to a video sharing site. The video uploaded to the video sharing site (posted video) can be viewed via a mobile device. In this embodiment, a performance is referred to as singing, a performer is referred to as a singing user, and a video uploaded to the video sharing site is referred to as a singing video. Furthermore, a user who views a singing video uploaded to the video sharing site is referred to as a viewing user.

[0025] [1-1. Karaoke system configuration] 1 is a schematic diagram of a karaoke system 1 according to this embodiment. The karaoke system 1 includes a communication line 2, a plurality of mobile terminals 3 (mobile terminals 3A and 3B), and a server 4.

[0026] The communication line 2 forms a computer network, and is, for example, a wide area communication line provided by an electric utility company.

[0027] The mobile terminal 3 is an information processing terminal such as a smartphone, tablet, or laptop computer, and is used by users to access online services. The mobile terminal 3 includes a touch panel display 3a for displaying images, a speaker 3b for outputting sound, a microphone 3c for inputting sound, a camera 3d for capturing images of a subject, and an earphone jack 3e for connecting earphones (not shown). "Capturing" here refers to a state in which the camera 3d is functioning and the subject is displayed on the touch panel display 3a, regardless of whether recording is occurring. The touch panel display 3a includes, for example, an LCD (Liquid Crystal Display) and a touch sensor. The LCD displays various images, and the touch sensor accepts various input operations performed using a pointing device such as a finger, a stylus, or a pen. In the following description, the touch panel display 3a is also referred to as the screen 3a.

[0028] The server 4 is an information processing device that provides online services to the mobile terminal 3 via the communication line 2. In the example of FIG. 1, a singing user uploads his / her singing video (singing video data) to a video posting site by transmitting the singing video from the mobile terminal 3A to the server 4. A viewing user then accesses the video posting site using the mobile terminal 3B and views the singing video. The singing user can also view the singing video that he / she has uploaded by accessing the video posting site using the mobile terminal 3A. The user of the mobile terminal 3B can also become a singing user and upload a singing video to the video posting site.

[0029] [1-2. Server configuration] FIG. 2 is a block diagram showing the electrical configuration of the server 4 according to this embodiment.

[0030] The server 4 according to this embodiment includes a CPU (Central Processing Unit) 20, which is a main control unit that controls the overall operation of the server 4, a ROM in which various programs and various data are stored in advance, (Read Only Memory) 22, RAM (Random Access Memory) 24 used as a work area when the CPU 20 executes various programs, and HDD (Hard Disk Drive) 26 as storage means for storing various programs and various data.

[0031] The HDD 26 stores singing video data transmitted from the mobile terminal 3A, i.e., singing video data uploaded to a video posting site, song data indicating songs that the singing user can sing, etc. The storage means is not limited to the HDD 26, and may be, for example, another storage medium such as a semiconductor memory, such as a flash memory.

[0032] Furthermore, the server 4 is equipped with an operation input unit 28 consisting of a keyboard, mouse, etc., which accepts input of various operations, a monitor 30 such as a liquid crystal display device which displays various images, and an external interface 32 which is connected to other information processing devices such as a mobile terminal 3 via the communication line 2 and sends and receives various data to and from the other information processing devices.

[0033] The CPU 20, ROM 22, RAM 24, HDD 26, operation input unit 28, monitor 30, and external interface 32 are electrically connected to one another via a system bus 34. Therefore, the CPU 20 can access the ROM 22, RAM 24, and HDD 26, grasp the operation status of the operation input unit 28, display images on the monitor 30, and send and receive various data to and from other information processing devices, etc. via the external interface 32.

[0034] [1-3. Electrical configuration of mobile terminal] FIG. 3 is a functional block diagram showing the electrical configuration of the mobile terminal 3. As shown in FIG.

[0035] The mobile terminal 3 includes a main control unit 40, a main memory unit 42, an auxiliary memory unit 44, a communication unit 46, and an operation button 48 in addition to the components shown in FIG.

[0036] The main control unit 40 is, for example, a CPU, a microprocessor, a DSP (Digital Signal Processor), or the like, and controls the overall operation of the mobile terminal 3.

[0037] The main storage unit 42 is configured with, for example, RAM or DRAM (Dynamic Random Access Memory), and is used as a work area when the main control unit 40 executes processes based on various programs.

[0038] The auxiliary storage unit 44 is, for example, a non-volatile memory such as a flash memory, and stores various data such as images, and programs used for processing by the main control unit 40. The programs stored in the auxiliary storage unit 44 include, for example, an operating system (OS) for implementing the basic functions of the mobile terminal 3, drivers for controlling various hardware, and programs for implementing email, web browsing, and various other functions. The auxiliary storage unit 44 also stores applications for shooting and posting singing videos and for viewing videos on video posting sites. (hereinafter referred to as the "video posting and viewing app") is pre-stored.

[0039] The communication unit 46 is, for example, a NIC (Network Interface Controller), and has a function of connecting to the communication line 2. Note that the communication unit 46 may have, instead of or together with the NIC, a function of connecting to a wireless LAN (Local Area Network), a function of connecting to a wireless WAN (Wide Area Network), a function of enabling short-range wireless communication such as Bluetooth (registered trademark), infrared communication, etc.

[0040] The operation buttons 48 are provided on the side of the mobile terminal 3, and include a power button for starting or stopping the mobile terminal 3, a volume adjustment button for adjusting the volume of the sound output by the speaker 3b, and the like.

[0041] The main control unit 40, main memory unit 42, auxiliary memory unit 44, communication unit 46, operation buttons 48, touch panel display 3a, speaker 3b, microphone 3c, camera 3d, and earphone jack 3e are electrically connected to one another via a system bus 49. Therefore, the main control unit 40 can access the main memory unit 42 and auxiliary memory unit 44, display images on the touch panel display 3a, grasp the operation status of the touch panel display 3a or operation buttons 48 by the user, input sound to the microphone 3c, output sound from the speaker 3b or earphones connected to the earphone jack 3e, control the camera 3d, and access various communication networks or other information processing devices via the communication unit 46.

[0042] [1-4. Filming of singing videos by singing users] A case where a singing user shoots a singing video using the mobile terminal 3A will be described.

[0043] When a singing user shoots a singing video, the singing user launches a video posting and viewing app on the mobile terminal 3A. When the video posting and viewing app is launched, the mobile terminal 3A accesses the server 4, which stores multiple pieces of music data. The singing user then arbitrarily selects a piece of music to sing from the video posting and viewing app and downloads the music data from the server 4 to the mobile terminal 3A. The singing user then plays the music at a timing of their choice using the video posting and viewing app and sings. The video posting and viewing app starts playing the music and starts shooting a video with the camera 3d. In other words, the singing video is a video shot by the mobile terminal 3A while the music is being played from the mobile terminal 3A.

[0044] Note that song data is also associated with lyric data, and when song data is downloaded from the server 4 to the mobile terminal 3A, the associated lyric data is also downloaded to the mobile terminal 3A. Note that in the following explanation, song data is assumed to include lyric data.

[0045] FIG. 4 shows an example of a display state (hereinafter referred to as "screen display") on the screen 3a of the mobile terminal 3A when shooting a singing video.

[0046] 4, the screen 3a is divided into a lyrics display area 50A and a captured image display area 50B. The lyrics display area 50A includes a lyrics image 52 that displays the lyrics of the song being sung by the user, a pitch image 54 that displays the pitch of the song, and a progress bar 56 that displays the progress of the capture.

[0047] The lyric image 52 and the pitch image 54 are updated as the song progresses. In the present embodiment, as an example, the lyric image 52 and the pitch image 54 are updated every few phrases and displayed in the lyrics display area 50A. Note that the lyric image 52 and the pitch image 54 may be updated at the same time or at different times.

[0048] As an example, the lyrics image 52 displays lyrics in multiple rows (two rows in the example of FIG. 4), and the color of the lyrics in the upper row changes from the left end to the right end as the song progresses so that the singing user can understand the lyrics that should currently be sung. When the color change of the lyrics in the upper row reaches the right end, the lyrics in the lower row rises and are displayed in the upper row, and new lyrics are displayed in the lower row, and the color of the lyrics in the upper row changes again from the left end to the right end as the song progresses.

[0049] As an example, the pitch image 54 displays multiple pitch bars 54A in a stepped pattern in the left-right direction according to the pitch strength. Then, to enable the singing user to grasp the pitch of the lyrics that should currently be sung, the color of the pitch bars 54A changes from the left end to the right end as the song progresses, and the pointer 54B moves from the left end to the right end. When the color of the pitch bar 54A changes and the pointer 54B reaches the right end, the pitch image 54 indicating the next pitch is updated and displayed.

[0050] As an example, the length of the progress bar 56 from the left end to the right end indicates the length of the entire song. When playback of the song begins, a pointer 56A indicating the playback position of the song moves from the left end to the right end, and when the pointer 56A reaches the right end, the song ends. Note that the part of the progress bar 56 that the pointer 56A has passed through is displayed thicker than its position before the progress bar 56 began.

[0051] Recording of a singing video starts a predetermined time (e.g., 10 seconds) after the singing user selects a song and clicks a recording start button (not shown) displayed on screen 3a. The start and end of video recording may coincide with the start and end of the song, but this is not limiting. Video recording may start a predetermined time (e.g., 5 seconds) before the start of the song, or may end a predetermined time (e.g., 5 seconds) after the song ends.

[0052] The singing user connects earphones to the earphone terminal 3e, listens to the music being played using the earphones, and sings along with the music. The mobile terminal 3A captures a photo of the singing user using the camera 3d and records the singing user's singing using the microphone 3c. In other words, the microphone 3c does not capture the sound of the music being played. The mobile terminal 3A then records the singing user's singing voice captured by the microphone 3c and uses the recorded data as singing data.

[0053] The singing data may be filtered to extract the frequency band of the human voice, which removes noise from the singing user's surrounding environment, making the recorded singing voice clearer.

[0054] The video posting and viewing app then combines the music data and singing data with the recording data to generate singing video data that can be sent to the server 4. As an example, the user can select one of the following two options for the timing of sending the singing video data to the server 4, i.e., the timing of uploading the data to the video posting site.

[0055] One is live streaming, in which a singing user uploads singing video data to a video sharing site in real time while singing. In live streaming, viewing users watch the singing user sing in real time. The other is non-live streaming, in which a singing user uploads singing video data to a video sharing site at a timing of their choice after completing singing of a song.

[0056] When performing live streaming, the singing user sets up the live streaming before recording the singing video, and the singing video data is uploaded to the video sharing site at the same time as the video recording starts. Note that in the case of live streaming, the singing video data may be uploaded to the video sharing site without being stored in the mobile terminal 3A.

[0057] In addition, when performing live streaming, a singing user can set either a first live streaming setting that allows viewing users to view the singing video during live streaming, or a second live streaming setting that allows viewing users to view the singing video even after the live streaming. That is, with the first live streaming setting, the singing video data is deleted from the server 4 when the live streaming ends, and viewing users cannot view the live-streamed singing video after the live streaming ends. On the other hand, with the second live streaming setting, the server 4 continues to store the singing video data even after the live streaming ends, so viewing users can view the singing video as a non-live stream even after the live streaming ends.

[0058] In the case of non-live distribution, the singing video data is temporarily stored in the mobile terminal 3A, and the singing user operates the video posting and viewing application to upload the singing video to the video posting site at any time.

[0059] [1-5. Viewing of singing videos by viewers] A case where a viewing user views a singing video using the mobile terminal 3B will be described.

[0060] When a viewing user wishes to view a singing video, the viewing user launches a video posting and viewing app on the mobile terminal 3B. When the video posting and viewing app is launched, the mobile terminal 3B accesses a server 4 that stores multiple singing video data, i.e., a video posting site. The viewing user then selects the singing video they wish to view via the video posting and viewing app and displays it on the screen 3a. The server 4 distributes the singing video to the mobile terminal 3B by streaming, for example.

[0061] FIG. 5 is an example of a screen display on the mobile terminal 3B when watching a singing video, showing a screen display when live streaming is being performed.

[0062] The screen 3a displays a singing video and also has a singing user display area 50C, a lyrics display area 50D, and a comment input display area 50E. The singing user display area 50C, the lyrics display area 50D, and the comment input display area 50E may be displayed superimposed on the singing video.

[0063] The singing user display area 50C displays the username of the singing user who posted the singing video, an indication of whether or not it is a live broadcast, and the name of the song being sung.

[0064] The lyrics of the singing video are displayed in the lyrics display area 50D. For example, the lyrics displayed are multiple phrases at a time, and the color of the lyrics changes from the left end to the right end as the music progresses. For example, the lyrics display area 50D may display lyrics in multiple rows. In this case, when the color change of the lyrics in the upper row reaches the right end, the lyrics in the lower row rise and are displayed in the upper row, and new lyrics are displayed in the lower row. The color of the lyrics in the upper row changes again from the left end to the right end as the music progresses.

[0065] In the comment input display area 50E, a comment input field is displayed, and comments from viewing users who are viewing the singing video are displayed together with the user names. As an example, each time a comment is input from a viewing user, the comment is additionally displayed at the top of the comment input display area 50E, and comments that were previously displayed are moved down. When there are more comments than can be displayed in the comment input display area 50E, a scroll bar (not shown) is displayed on the right side of the comment display area, and the viewing user can operate the scroll bar to display comments that were not previously displayed on the screen 3a.

[0066] Furthermore, operation icons 58A to 58D are displayed on the screen 3a to allow the viewing user to perform various operations.

[0067] The operation icon 58A is an icon that is clicked when the viewing user empathizes with the singing video that the viewing user is viewing, and the total number of clicks on the operation icon 58A for the singing video is displayed above the operation icon 58A.

[0068] Operation icon 58B is an icon that is clicked when a viewing user challenges a singing user who is live streaming a singing video displayed on screen 3a to a battle (hereinafter referred to as a "singing battle"). A singing battle involves simultaneously displaying multiple singing videos (first singing video, second singing video) by different singing users on screen 3a of mobile terminal 3B used by the viewing user, and the singing videos alternate singing the same song. In other words, the viewing user who clicks operation icon 58B becomes a singing user who will participate in the singing battle.

[0069] The operation icon 58C is an icon that is clicked by the viewing user when making various settings for the video posting and viewing application.

[0070] The operation icon 58D is an icon that the viewing user clicks when superimposing an ornamental image on the singing video displayed on the screen 3a. The ornamental images according to this embodiment have a price determined according to their type, and can be purchased by the viewing user for a fee. The viewing user then superimposes an ornamental image on the singing video that they are viewing by clicking the operation icon 58D. The singing user of the singing video on which the ornamental image is superimposed receives money corresponding to the superimposed ornamental image from the operator of the video posting site. In other words, the viewing user's superimposition (display instruction) of an ornamental image on a singing video corresponds to a so-called tipping payment to the singing user.

[0071] [1-6. Tipping effect function] The video posting / viewing app according to this embodiment has a coin tip effect function that superimposes a coin tip effect image on a singing video being streamed after receiving a display instruction from the viewing user's mobile terminal 3B for a decorative image (hereinafter referred to as a "coin tip effect image") that decorates the singing video being streamed. The coin tip effect function performs a process of superimposing a coin tip effect image on the singing video displayed on the screen 3a after the operation icon 58D displayed on the screen 3a of the mobile terminal 3B used by the viewing user is clicked. That is, the viewing user decorates the singing video by superimposing a coin tip effect image on a singing video that he or she identifies with. Therefore, the coin tip effect function can liven up the viewers and the singing video itself during the streaming of the singing video. The coin tip effect function is executed by the server 4 in response to a display instruction from the viewing user via the video posting / viewing app.

[0072] [1-6-1. Coin tip effect image] FIG. 6 shows the screen display of the mobile terminal 3B to which the viewing user transitions when he or she clicks the operation icon 58D, and displays a plurality of different tip effect images 60 that the viewing user can select. Points are assigned to each tip effect image 60, and the design differs depending on the points. The viewing user selects one of the multiple tip effect images 60 displayed on the screen that can be obtained with the points they own. Note that different designs refer to, for example, different shapes, colors, orientations, etc.

[0073] As an example, a charge amount is set for each point, and viewing users purchase points, and the types of tip effect images 60 that can be selected increase or decrease depending on the points they own. When a tip effect image 60 is selected by a viewing user, the points owned by the viewing user decrease according to the points of the selected tip effect image 60. Note that points may be acquired not only by charging, but also by using other online services, etc.

[0074] The coin tip effect image 60 illustrated in Fig. 6 includes four types of spotlight images that mimic spotlights and have different fan-shaped directions, and four types of combined spotlight images that combine multiple spotlight images and have different colors. By using the coin tip effect image 60 as an image that mimics a spotlight, the viewing user can feel as if they are participating in a concert or the like. Note that the coin tip effect image 60 is not limited to the example in Fig. 6, and other images may also be used.

[0075] The time for which the coin tip effect image 60 is superimposed on the singing video, in other words, the time for which it is displayed on the screen 3a (hereinafter referred to as the "display time"), is predetermined, and the display time may vary depending on the coin tip effect image 60. For example, the display time of the 5p coin tip effect image 60 shown in FIG. 6 is 0.3 seconds, and the display time of the 20p coin tip effect image 60 is 0.5 seconds. Note that the coin tip effect image 60 shown in FIG. 6 imitates a spotlight, and therefore may dynamically change within the display time, for example, so that the light extends from the top to the bottom of the screen 3a.

[0076] Furthermore, the selection of the coin throwing effect image 60 itself corresponds to an instruction to display the coin throwing effect image 60 on the singing video. In the following description, the instruction (selection) by the viewing user to display the coin throwing effect image 60 via the mobile terminal 3B is referred to as a coin throwing input, as described above.

[0077] Tipping is possible using a video posting and viewing app for both live and non-live streaming singing videos. Tipping is done while the singing video is being streamed, so viewers can tip when they feel like it, and they can watch the effects-overlaid singing video in real time based on their tipping.

[0078] 7 is an example of a singing video on which a coin-throwing effect image 60 is superimposed when a viewing user inputs a coin-throwing effect by selecting a combined spotlight image as the coin-throwing effect image 60 (hereinafter referred to as an "effect-superimposed singing video"). The effect-superimposed singing video is stored in the server 4 and distributed via the video posting site so that it can be viewed not only by the viewing user who inputs a coin-throwing effect, but also by other viewing users.

[0079] As an example, the effect-superimposed singing video has a layer structure, and the tip effect image 60 is a layer (hereinafter referred to as the "effect layer") that is different from the singing video. The effect layer is synchronized with the singing video, and a new tip effect image 60 is added in response to a tip input by the viewing user. Because the effect-superimposed singing video has this layer structure, the viewing user can remove the superimposition of the tip effect image 60 by changing the settings of the video posting and viewing app, even if the singing video has effects superimposed, and watch the singing video.

[0080] [1-6-2. Timing of coin tip effect image overlay] The tip effect function according to this embodiment superimposes a tip effect image 60 onto the singing video at a predetermined timing (hereinafter referred to as "superimposition timing") of the singing video after receiving a tip input from the viewing user's mobile terminal 3B. As described above, the singing video is a video filmed while a song is being played, and multiple superimposition timings are provided discretely depending on the song. That is, the singing video is associated with music data in which the superimposition timing is set, and the tip effect function superimposes the tip effect image 60 onto the singing video based on the music data in which the superimposition timing is set.

[0081] Fig. 8 is a schematic diagram showing the superimposition timing of the coin throwing effect image 60. In Fig. 8, T indicates the superimposition timing, and P indicates the period between the superimposition timings T (hereinafter referred to as "non-superimposition period").

[0082] The superimposition timing T is a timing that is considered appropriate for livening up a singing video and differs for each song, and is determined, for example, by the tempo, pitch, beat, guide melody, measures, melody, tune, rhythm, etc. of the song. The superimposition timing T may be determined manually, for example, or automatically by analyzing the song using a predetermined algorithm. Furthermore, the superimposition timing T may be set near the end of the song, regardless of the tempo, etc. In the example of FIG. 8, the superimposition timing T is determined based on the pitch of the song, and the superimposition timing T is set at a position where the pitch is relatively high.

[0083] The non-overlapping period P is a period during which the coin throwing effect image 60 is not superimposed on the singing video, and is determined in conjunction with the determination of the superimposing timing T. In the example of Fig. 8, the non-overlapping period P1 is before the superimposing timing T1, the non-overlapping period P2 is between the superimposing timings T1 and T2, and the non-overlapping period P3 is after the superimposing timing T2.

[0084] The coin tip effect function according to this embodiment superimposes a coin tip effect image 60 corresponding to a coin tip input onto the singing video at superimposition timing T from the next time onward after the coin tip input is received. Specific explanations are given with reference to Fig. 8 , and the coin tip effect image 60 corresponding to a coin tip input made in non-overlapping period P1 is superimposed on the singing video at superimposition timing T1, the coin tip effect image 60 corresponding to a coin tip input made in non-overlapping period P2 is superimposed on the singing video at superimposition timing T2, and the coin tip effect image 60 corresponding to a coin tip input made in non-overlapping period P3 is superimposed on the singing video at the next superimposition timing T.

[0085] More specifically, the server 4 stores the tip input information indicating the content of the tip input received during the non-overlapping period P in a tip input storage area provided in the RAM 24 or the HDD 26, and sets the tip effect image 60 indicated by the tip input information to be superimposed on the singing video at the next superimposition timing T. In other words, the more tip inputs received during the non-overlapping period P, the more colorful the tip effect image 60 superimposed at the next superimposition timing T will be. The tip input information that has been superimposed on the singing video is deleted from the tip input storage area. The tip input information includes the name of the user who made the tip input, the tip effect image 60, points, the timing of the tip input (time from the start of the singing video), etc.

[0086] Note that, if multiple coin tip inputs are accepted during one non-overlapping period P, the multiple coin tip effect images 60 corresponding to the accepted coin tip inputs are all superimposed and displayed at the next superimposition timing T. Therefore, if multiple identical coin tip effect images 60 are present and are superimposed on each other, it becomes difficult for the viewing user to distinguish them from each other. Therefore, if multiple identical coin tip effect images 60 are superimposed on the singing video at the superimposition timing T, the coin tip effect function may display one of the coin tip effect images 60 by shifting the display angle. For example, if two coin tip effect images 60 of a single spotlight pointing directly downward from above are accepted as coin tip inputs, the coin tip effect function displays one of the coin tip effect images 60 on the screen 3a as a coin tip effect image 60 pointing diagonally right from above or diagonally left from above.

[0087] In addition, without being limited to this, when there are a plurality of coin throwing effect images 60 to be superimposed at the superimposition timing T, the timing to superimpose the coin throwing effect image 60 on the singing video may be set to a minute time. In this case, the superimposition timing T has a certain time span (for example, 5 seconds).

[0088] In addition, the maximum number of coin-throwing effect images 60 that can be superimposed at the superimposition timing T is determined in advance, and if the number of coin-throwing effect images 60 superimposed at the next superimposition timing T exceeds the maximum number, the coin-throwing effect images 60 may be superimposed on the singing video at the superimposition timing T after the next.

[0089] In addition, the tip effect function of this embodiment does not place any restrictions on the number of tip inputs that a single viewing user can make, and as long as the viewing user has points, multiple tip inputs are possible for one singing video, and multiple tip inputs are also possible during one non-overlapping period P.

[0090] In this way, the coin tip effect function of this embodiment superimposes the coin tip effect image 60 onto the singing video at the superimposition timing T that is deemed appropriate for livening up the singing video, thereby livening up the viewers and the singing video itself while the singing video is being distributed.

[0091] [1-6-3. Image cropping function] The video posting and viewing application has a function of acquiring an image of a scene by cutting (copying) a scene in which the coin tip effect image 60 is relatively large in the singing video from the singing video. (hereinafter referred to as "image cropping function"), which allows the viewing user to easily post the most exciting scenes from the singing video to other services.

[0092] The image cutting function cuts out, for example, a singing video of a predetermined period including the superimposition timing T at which the number of superimposed coin tip effect images 60 is the greatest, or cuts out as a still image an image corresponding to the superimposition timing T. The image (video or still image) cut out by the image cutting function is, for example, transmitted to the mobile terminal 3B and stored in the auxiliary storage unit 44, but is not limited to this, and may also be posted to an SNS (Social Networking Service) or the like to which the viewing user is registered.

[0093] [1-7. Function block of coin tip effect] 9 is a functional block diagram of the tip effect function according to this embodiment. The CPU 20 of the server 4 includes a tip input receiving unit 70, a tip effect superimposing unit 72, a video distribution unit 74, and an image cropping unit 76. The processes executed by each function of the server 4 are realized by programs stored in the HDD 26.

[0094] The tip input receiving unit 70 receives a display instruction (tip input) for a tip effect image 60 that decorates the singing video from the mobile terminal 3B of the viewing user to whom the singing video is distributed.

[0095] The coin tip effect superimposing unit 72 superimposes the coin tip effect image 60 on the singing video after the coin tip input accepting unit 70 accepts the coin tip input. Note that the coin tip effect superimposing unit 72 according to this embodiment superimposes the coin tip effect image 60 corresponding to the coin tip input on the singing video at the superimposing timing T from the next time onwards after the coin tip input is accepted.

[0096] The video distribution unit 74 distributes the singing video uploaded to the video posting site to the mobile terminal 3B used by the viewing user. Note that if a tip effect image 60 is superimposed on the singing video, the video distribution unit 74 distributes the effect-superimposed singing video to the mobile terminal 3B.

[0097] The image cutting unit 76 acquires a scene in which the coin tip effect image 60 is relatively numerous from the singing video, and acquires a still image of the scene or a moving image including the scene.

[0098] [1-8. Video distribution processing flowchart] 10 is a flowchart showing the flow of the tip effect process according to this embodiment, which is executed by the CPU 20 of the server 4. A program for executing the tip effect process according to this embodiment is stored in advance in a predetermined area of ​​the HDD 26.

[0099] First, in step S100, the video distribution unit 74 distributes the singing video to the mobile terminal 3B used by the viewing user.

[0100] In the next step S102, the tip input accepting unit 70 determines whether or not a tip input has been accepted from the mobile terminal 3B, and if the answer is yes, the process proceeds to step S104, and if the answer is no, the process proceeds to step S106.

[0101] In step S104, the tip input receiving unit 70 stores tip input information indicating the content of the received tip input in a tip input storage area of ​​the HDD 26 (or RAM 24).

[0102] In step S106, the coin tip effect superimposition unit 72 determines whether the progress of the singing video has reached the next superimposition timing T, and if the determination is affirmative, the process proceeds to step S108, and if the determination is negative, the process proceeds to step S114.

[0103] In step S108, the coin-donating effect superimposition unit 72 determines whether or not coin-donating input information is stored (memorized) in the coin-donating input storage area, and if the determination is affirmative, the process proceeds to step S110, and if the determination is negative, the process proceeds to step S114.

[0104] In step S110, the coin-throwing effect superimposing unit 72 superimposes the coin-throwing effect image 60 indicated by the coin-throwing input information stored in the coin-throwing input storage area onto the singing video.

[0105] In the next step S112, the tip effect superimposing unit 72 deletes the superimposed tip input information from the tip input storage area.

[0106] In the next step S114, the video distribution unit 74 determines whether or not the distribution of the singing video has ended, and if the determination is negative, the process returns to step S102, whereas if the determination is positive, the tip effect process ends.

[0107] As described above, the server 4 accepts a coin tip input, which is an instruction to display a coin tip effect image 60 that decorates the singing video being distributed from the viewing user's mobile terminal 3B, superimposes the coin tip effect image 60 on the singing video at superimposition timing T after accepting the coin tip input, and distributes the singing video with the coin tip effect image 60 superimposed to the mobile terminal 3B.

[0108] As a result, a singing video that has resonated with viewing users is decorated with the tip effect image 60, which will excite viewing users and the singing video itself during distribution of the singing video.

[0109] [2. Second Embodiment] A second embodiment of the present invention will now be described.

[0110] The configurations of the karaoke system 1, the portable terminal 3, and the server 4 according to this embodiment are the same as those of the karaoke system 1, the portable terminal 3, and the server 4 according to the first embodiment, and therefore description thereof will be omitted.

[0111] [2-1. Timing of coin tip effect image overlay] The coin tip effect function according to this embodiment superimposes a coin tip effect image 60 corresponding to the coin tip input onto the singing video every time a coin tip input is received, in other words, without much time delay from the reception of the coin tip input. Note that the coin tip effect function according to this embodiment has a preset maximum number of coin tip effect images 60 (hereinafter referred to as the "maximum number of superimpositions") that can be superimposed on the singing video at the same time. The coin tip effect function (coin tip effect superimposition unit 72) then superimposes the coin tip effect images 60 onto the singing video in the order in which the coin tip input is received so as not to exceed the maximum number of superimpositions.

[0112] Fig. 11 is a schematic diagram showing the timing of superimposing the tip effect image 60 according to this embodiment. Fig. 11(A) is a schematic diagram of the tip input storage area 80 according to this embodiment, and the tip input storage area 80 stores tip input information in the chronological order in which the tip inputs were received. Note that the numbers at the end of the "tip input information" in Fig. 11(A) indicate the order in which the tip inputs were received.

[0113] In Fig. 11(B), the horizontal axis indicates the passage of time, and the vertical axis schematically shows the superimposed tip effect image 60, and the numbers in Fig. 11(B) correspond to the order in which the tip inputs shown in Fig. 11(A) were accepted. Also, as an example, in Fig. 11, the maximum number of superimpositions is 4, and the display time of tip input information 3, 4, 5, and 6 is approximately half the time of tip input information 1, 2, and 7.

[0114] As illustrated in FIG. 11(B), the coin tip effect images 60 are superimposed on the singing video in the order in which the coin tip input was received until the maximum number of superimpositions is reached (coin tip effect images 60-1 to 60-4). Then, when a certain coin tip effect image 60 ends and the number of coin tip effect images 60 superimposed on the singing video falls below the maximum number of superimpositions, the next coin tip effect image 60 (coin tip effect images 60-5 to 60-7) is superimposed in the order in which the coin tip input was received. In other words, the coin tip effect function according to this embodiment superimposes coin tip effect images 60 on the singing video in a first-in first-out (FIFO) order so as not to exceed the maximum number of superimpositions, similar to so-called queuing.

[0115] 11(B), the display time of the coin throwing effect image 60-3 is shorter than the display time of the coin throwing effect image 60-2, so the superimposition of the coin throwing effect image 60-3 finishes before the superimposition of the coin throwing effect image 60-2. Therefore, the superimposition of the coin throwing effect image 60-5 starts before the superimposition of the coin throwing effect image 60-2 finishes.

[0116] In this way, the coin tip effect function according to this embodiment superimposes coin tip effect images 60 on the singing video so as not to exceed the maximum number of superimpositions. This prevents the singing video from being filled with a large number of coin tip effect images 60, preventing the viewing user from being unable to view the singing video itself, while also boosting the excitement of the viewing user and the singing video itself during distribution of the singing video.

[0117] Furthermore, if there are any coin throwing effect images 60 (hereinafter referred to as "unsuperimposed coin throwing effect images") that have received a coin throwing input but have not been superimposed on the singing video at a predetermined timing just before the end of the singing video, the coin throwing effect superimposing unit 72 according to this embodiment superimposes all of the unsuperimposed coin throwing effect images on the singing video at the predetermined timing. The predetermined timing is, for example, 10 seconds before the end of the singing video.

[0118] As a result, the tip effect function can superimpose all tip effect images 60 corresponding to the accepted tip inputs on the singing video, and can liven up the singing video at the end of the singing video.

[0119] However, this is not limited to this, and even if an unsuperimposed coin throwing effect image is present immediately before the end of the singing video, the singing video may be ended without superimposing the unsuperimposed coin throwing effect image on the singing video at a predetermined timing as described above.

[0120] [2-2. Video distribution processing flowchart] 12 is a flowchart showing the flow of the tip effect process according to this embodiment, which is executed by the CPU 20 of the server 4. A program for executing the tip effect process according to this embodiment is stored in advance in a predetermined area of ​​the HDD 26.

[0121] First, in step S200, the moving image distribution unit 74 distributes the singing moving image to the mobile terminal 3B used by the viewing user.

[0122] In the next step S202, the tip input accepting unit 70 determines whether or not a tip input from the mobile terminal 3B has been accepted, and if the answer is affirmative, the process proceeds to step S204, and if the answer is negative, the process proceeds to step S206.

[0123] In step S204, the tip input receiving unit 70 stores the tip input information indicating the content of the received tip input in the tip input storage area 80 of the HDD 26 (or RAM 24).

[0124] In step S206, the coin-donating effect superimposition unit 72 determines whether or not coin-donating input information is stored (memorized) in the coin-donating input storage area 80, and if the determination is affirmative, the process proceeds to step S208, and if the determination is negative, the process proceeds to step S214.

[0125] In step S208, the coin tip effect superimposition unit 72 determines whether the number of coin tip effect images 60 superimposed on the singing video at the current timing is less than the maximum number of superimpositions, and if the determination is positive, proceeds to step S210, and if the determination is negative, return to step S202.

[0126] In step S210, the coin-donating effect superimposing unit 72 superimposes the coin-donating effect image 60 on the singing video in the order in which the coin-donating inputs are received.

[0127] In the next step S212, the tip effect superimposing unit 72 deletes the superimposed tip input information from the tip input storage area 80.

[0128] In the next step S214, the video distribution unit 74 determines whether or not the distribution of the singing video has ended, and if the determination is negative, the process returns to step S202, whereas if the determination is positive, the tip effect process ends.

[0129] 3. Other Embodiments Although the present invention has been described above using the above-mentioned embodiments, the technical scope of the present invention is not limited to the scope described in the above-mentioned embodiments. Various modifications or improvements can be made to the above-mentioned embodiments without departing from the gist of the invention, and such modifications or improvements are also included in the technical scope of the present invention. Furthermore, the above-mentioned embodiments may be combined as appropriate.

[0130] For example, in the first embodiment, a configuration in which a viewing user selects a coin tip effect image 60 to be superimposed on a singing video has been described, but the present invention is not limited to this. For example, when inputting a coin tip, the viewing user does not select a coin tip effect image 60 by himself / herself but inputs points, and the coin tip effect function may superimpose a coin tip effect image 60 on the singing video according to the total points received up to the superimposition timing T. Then, for example, the coin tip effect image 60 superimposed at the next superimposition timing T1 changes according to the total points input during the non-superimposition period P1, and the coin tip effect image 60 superimposed at the next superimposition timing T2 changes according to the total points input during the non-superimposition period P2. Note that the total points may include points input by multiple viewing users, not just one viewing user.

[0131] With reference to FIG. 13 , the change in the coin throwing effect image 60 according to the points will be described. For example, if five points are input during a certain non-overlapping period P, the coin throwing effect image 60 shown in FIG. 13(A) is superimposed on the singing video at the next superimposition timing T. Furthermore, if ten points are input during a certain non-overlapping period P, the coin throwing effect image 60 shown in FIG. 13(B) is superimposed on the singing video at the next superimposition timing T. Similarly, if the points are 15, the coin throwing effect image 60 shown in FIG. 13(C) is superimposed; if the points are 20, the coin throwing effect image 60 shown in FIG. 13(D) is superimposed; and if the points are 25, the coin throwing effect image 60 shown in FIG. 13(E) is superimposed. In the example of FIG. 13 , the shape of the coin throwing effect image 60 varies depending on the points so that the greater the total number of points, the more colorful the image. However, this is merely an example, and the color of the coin throwing effect image 60 may also change depending on the points.

[0132] In addition, in the above-described embodiments, the performance is singing and the video uploaded to the video sharing site is a singing video, but the present invention is not limited to this. For example, the performance may be something other than singing, such as dancing and the video uploaded to the video sharing site is a dance video.

[0133] Furthermore, the flow of the coin tip effect processing described in each of the above embodiments is also an example, and unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope of the spirit of the present invention. [Explanation of symbols]

[0134] 3B Mobile terminal (terminal device) 4 Server (information processing device) 60 coin tip effect images (decorative images) 70 Tipping input reception unit (reception means) 72 coin throwing effect superimposition unit (superimposition means) 74 Video Distribution Department (Distribution Method) 76 Image cropping unit (acquisition means) T Superimposition timing (predetermined timing)

Claims

1. An information processing device that distributes a video to a terminal device so that a viewing user can view the video, a receiving means for receiving an instruction to display a decorative image that decorates the video being distributed from the terminal device of the viewing user; a superimposing means for superimposing the decorative image on the video after the accepting means accepts the display instruction; and a distribution means for distributing the video on which the decorative image is superimposed to the terminal device.

2. The information processing apparatus according to claim 1 , wherein the superimposing means superimposes the decorative image on the moving image at a predetermined timing of the moving image.

3. a plurality of timings are provided in the video; The information processing apparatus according to claim 2 , wherein the superimposing unit superimposes the decorative image on the moving image in accordance with the display instruction at the timing from the next time the display instruction is received onward.

4. The video is a video recorded while a song is being played, 4. The information processing apparatus according to claim 2, wherein a plurality of timings are provided according to the music piece.

5. a maximum number of the decorative images that can be superimposed on the video at the same time is set in advance; 5. The information processing apparatus according to claim 1, wherein the superimposing unit superimposes the decorative images on the moving image in the order in which the display instructions are received so as not to exceed the maximum number.

6. The information processing device according to claim 5, wherein, if there are decorative images for which the display instruction has been received but which have not been superimposed on the video at a predetermined timing immediately before the end of the video, the superimposition means superimposes all of the decorative images on the video at the predetermined timing.

7. 7. The information processing device according to claim 1, wherein points are set for the decorative image, and the design varies depending on the points.

8. 8. The information processing apparatus according to claim 1, further comprising: an acquisition unit that acquires, from the moving image, an image of a scene in which the decorative image is relatively numerous.

9. a first step of delivering the video to a terminal device so that the video can be viewed by a viewing user; a second step of receiving, from the terminal device of the viewing user, an instruction to display a decorative image that decorates the video being distributed; a third step of superimposing the decorative image on the video after receiving the display instruction; and a fourth step of distributing the video on which the decorative image is superimposed to the terminal device.

10. A computer included in an information processing device that distributes video to a terminal device so that a viewing user can view the video, a receiving means for receiving an instruction to display a decorative image that decorates the video being distributed from the terminal device of the viewing user; a superimposing means for superimposing the decorative image on the video after the accepting means accepts the display instruction; a video distribution program for causing the video to function as a distribution means for distributing the video on which the decorative image is superimposed to the terminal device;

Images