Mobile devices

Abstract

To create a singing video and save memories of a ride in the singing video while switching between a selfie image and an outside image during the ride.SOLUTION: A portable terminal (10) is equipped with a sound collector (14) capable of collecting a singing voice of a singer, an outside camera (11) capable of photographing an outside image of an outside scene of a vehicle, and an inside camera (12) capable of photographing a selfie image of a singer. The portable terminal is further equipped with a performance control unit (21) for controlling a karaoke performance of a musical piece, a first acquisition unit (22) for selectively acquiring the outside image and the selfie image during the karaoke performance, and a generation unit (23) for generating a singing video based on the karaoke performance sound of the musical piece, the singing voice of the singer, the outside image and the selfie image.SELECTED DRAWING: Figure 2

Description

【Technical Field】 【0001】 The present invention relates to a mobile terminal. 【Background Art】 【0002】 There is known a technique of photographing a singer in accordance with the movement of the singer performing karaoke singing and the progress of the music and displaying the photograph on a monitor (see, for example, Patent Document 1). A video camera is connected to the karaoke device described in Patent Document 1, and camera control data is downloaded to the karaoke device together with music data from a distribution center. When karaoke performance is started based on the music data by the karaoke device, the orientation and focal length of the video camera are controlled based on the camera control data. Even when the singer sings while moving in accordance with choreography, it is possible to photograph the singer without losing sight of the singer's figure. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Patent Application Laid-Open No. 2002-318588 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 By the way, it is possible to use a karaoke app on a mobile terminal while driving and take a self-portrait of the singer singing karaoke to create a singing video. However, if the in-camera of the mobile terminal is directed outside the vehicle to leave a memory of the drive, the poor image quality of the distant view is prominent and the lyrics displayed on the display of the mobile terminal cannot be seen. In addition, in order to photograph the scenery outside the vehicle, it is troublesome to operate the mobile terminal during karaoke singing to switch to the out-camera. 【0005】 An object of the present invention is to provide a mobile terminal capable of generating a singing video while switching between a self-shot video and an outside-vehicle video during driving and leaving a memory of the drive in the singing video. [Means for solving the problem] 【0006】 The main invention for achieving the above objective is a mobile terminal comprising: a sound collector capable of collecting the singing voice of a singer; an out-camera capable of capturing an external image of the scenery outside the vehicle; and an in-camera capable of capturing a selfie video of the singer, comprising: a performance control unit for controlling the karaoke performance of a song; a first acquisition unit for selectively acquiring external images and selfie videos during karaoke performance; and a generation unit for generating a singing video based on the karaoke performance sound of the song, the singing voice of the singer, the external images, and the selfie video. During karaoke performance, the rear camera and the front camera automatically switch to selectively acquire images of the outside of the car and selfie footage. This is a mobile device characterized by the following features. [Effects of the Invention] 【0007】 According to this invention, during karaoke performance, the rear and front cameras automatically switch to selectively capture external footage of the scenery outside the car and a selfie of the singer. A singing video is then generated in which the external footage and the selfie are arranged chronologically in accordance with the progress of the karaoke performance. Therefore, the scenery outside the car can be filmed without pointing the front camera out of the car, and the singer can sing while viewing the lyrics on the mobile device's display. A singing video is generated while switching between external footage and selfie footage during a drive, allowing users to preserve memories of their drive in a singing video. [Brief explanation of the drawing] 【0008】 [Figure 1] This is a diagram showing the configuration of the karaoke system according to the first embodiment. [Figure 2] This is a functional block diagram of a mobile terminal according to the first embodiment. [Figure 3] This figure shows an example of a singing video according to the first embodiment. [Figure 4] This is a flowchart showing an example of the processing operation of a mobile terminal according to the first embodiment. [Figure 5] This figure shows an example of a singing video for variation 1. [Figure 6] This flowchart shows an example of the processing operation of a mobile device in Modification Example 1. [Figure 7]This figure shows an example of a singing video for variation 2. [Figure 8] This is a functional block diagram of a mobile terminal according to the second embodiment. [Figure 9] This figure shows an example of a singing video according to the second embodiment. [Figure 10] This is a flowchart showing an example of the processing operation of a mobile terminal in the second embodiment. [Modes for carrying out the invention] 【0009】 <First Embodiment> The mobile terminal of the first embodiment will be described with reference to Figures 1 to 3. Figure 1 is a configuration diagram of the karaoke system of the first embodiment. Figure 2 is a functional block diagram of the mobile terminal of the first embodiment. Figure 3 is a diagram showing an example of a singing video of the first embodiment. Note that, for the sake of explanation, the functional block diagram in Figure 2 shows functional blocks for realizing specific processing, but it is assumed that the configurations that a mobile terminal normally has are also included. 【0010】 As shown in Figure 1, singer U1 is sitting in the passenger seat of vehicle 1 driven by driver U2, and a karaoke application is installed on singer U1's mobile device 10. Singer U1's mobile device 10 is connected to the karaoke server 2 via a mobile communication network 3. The karaoke server 2 manages song data, consisting of audio data, etc., through a song database. When the karaoke application on the mobile device 10 is launched and the mobile device 10 reserves a karaoke performance of a song, the song data is delivered from the karaoke server 2 to the mobile device 10. 【0011】 The mobile device 10 has an out-camera 11 on its back and an in-camera 12 and a display 13 on its front. The out-camera 11 captures the direction the back of the mobile device 10 is facing, and the in-camera 12 captures the direction the front of the mobile device 10 is facing, but the resolution of the out-camera 11 is set higher than that of the in-camera 12. The display 13 shows the images captured by the out-camera 11 and the in-camera 12, as well as lyrics and other text overlays. The display 13 also shows the menu screen of the karaoke application, and the display 13 accepts operations such as song reservation on the mobile device 10. 【0012】 An earphone 15 with a microphone (sound collector) 14 is connected to the mobile device 10. The microphone 14 collects the singer's voice and converts it into a singing voice signal. When karaoke playback starts on the mobile device 10, the karaoke playback sound signal and the singing voice signal output from the microphone 14 are mixed and emitted from the earphone 15. In this way, the karaoke application on the mobile device 10 provides an in-car karaoke service during driving, using external video of the scenery outside the car or a selfie video of the singer U1 as the background video. 【0013】 The karaoke application includes a function to automatically generate singing videos while driving. With this function, when the karaoke application is instructed to play karaoke, the singer U1 points the rear camera 11 out of the car and the front camera 12 towards the singer U1, and the recording of the singing video begins. The rear camera 11 captures the scenery outside the car, and the front camera 12 captures a selfie of the singer U1. Then, the singing video of the singer U1 is generated, with the footage of the scenery outside the car during the drive and the selfie footage of the singer singing karaoke automatically switching between each other. 【0014】 As shown in FIG. 2, in addition to cameras 11 and 12 and microphone 14, etc., the mobile terminal 10 is provided with a performance control unit 21, a first acquisition unit 22, and a generation unit 23. The performance control unit 21 controls the karaoke performance of music. In this case, a distribution request for a reserved music is transmitted from the performance control unit 21 to the karaoke server 2, and music data of the reserved music is received from the karaoke server 2 by the performance control unit 21 as a distribution response. The karaoke performance means (not shown) is controlled by the performance control unit 21, and the karaoke performance data included in the music data is reproduced by the karaoke performance means. 【0015】 The first acquisition unit 22 selectively acquires the outside-vehicle video and the self-shot video during the karaoke performance. In this case, the process of acquiring the outside-vehicle video from the out-camera 11 by the first acquisition unit 22 during the first period (for example, 10 seconds) and the process of acquiring the self-shot video from the in-camera 12 by the first acquisition unit 22 during the second period (for example, 30 seconds) are alternately repeated. As a result, the outside-vehicle video and the self-shot video are alternately acquired as time passes. The outside-vehicle video and the self-shot video acquired by the first acquisition unit 22 serve as the background video, and the lyrics telop included in the music data is superimposed on the background video and displayed on the display 13 (see FIG. 1) of the mobile terminal 10 during the karaoke performance. 【0016】 Specifically, assuming that the time from the start to the end of the karaoke performance of music X is 3 minutes and 25 seconds (205 sec), and the elapsed time from the start of the karaoke performance is T, then the outside-vehicle video is acquired when 0 sec ≦ T < 10 sec, the self-shot video is acquired when 10 sec ≦ T < 40 sec, the outside-vehicle video is acquired when 40 sec ≦ T < 50 sec, the self-shot video is acquired when 50 sec ≦ T < 80 sec, the outside-vehicle video is acquired when 80 sec ≦ T < 90 sec, the self-shot video is acquired when 90 sec ≦ T < 120 sec, the outside-vehicle video is acquired when 120 sec ≦ T < 130 sec, the self-shot video is acquired when 130 sec ≦ T < 160 sec, the outside-vehicle video is acquired when 160 sec ≦ T < 170 sec, the self-shot video is acquired when 170 sec ≦ T < 200 sec, and the outside-vehicle video is acquired when 200 sec ≦ T < 205 sec. 【0017】 The generation unit 23 generates a singing video based on the karaoke performance sound of the music, the singing voice of the singer, the outside-vehicle video, and the self-shot video. In this case, the karaoke performance sound signal output from the performance control unit 21 and the singing voice signal output from the microphone 14 are appropriately mixed, and the outside-vehicle video or the self-shot video is stored by the generation unit 23 while being synchronized with the karaoke performance sound signal. In this way, while driving, a singing video in which the outside-vehicle video of the outside scenery and the self-shot video of the singer U1 are arranged in time series is automatically generated in accordance with the progress of the karaoke performance of the music. Note that a lyric caption may be included in the singing video. 【0018】 For example, in the singing video shown in FIG. 3, the outside-vehicle video of the outside scenery is displayed for 10 seconds from the start of the karaoke performance, and the self-shot video of the singer U1 is displayed for the next 30 seconds. Then, until the karaoke performance ends, the 10-second outside-vehicle video and the 30-second self-shot video are alternately switched and displayed. The change of the outside scenery during driving and the singer U1 during karaoke singing are included in the singing video. At this time, since the outside scenery is photographed by the out-camera 11 with a high resolution, the poor image quality of the long-shot when the outside scenery is photographed by the in-camera 12 is not conspicuous. 【0019】 Note that the processing of each part of the mobile terminal 10 may be realized by software using a processor, or may be realized by a logic circuit (hardware) formed on an integrated circuit or the like. When using a processor, various processes are performed by the processor reading and executing a program stored in a memory. As the processor, for example, a CPU (Central Processing Unit) is used. The memory is composed of one or more storage media such as a ROM (Read Only Memory) and a RAM (Random Access Memory) according to the application. 【0020】 The processing operation of the mobile terminal of the first embodiment will be described with reference to Figure 4. Figure 4 is a flowchart showing an example of the processing operation of the mobile terminal of the first embodiment. Note that the reference numerals from Figure 2 will be used as appropriate in this explanation. Also, the following flowchart is merely an example and can be modified as needed. 【0021】 As shown in Figure 4, when the performance control unit 21 starts karaoke performance (step S01), a timer (not shown) starts counting for a first period (for example, 10 seconds) (step S02). The first acquisition unit 22 acquires an external image of the scenery outside the vehicle from the rear camera 11 (step S03), and the generation unit 23 generates a singing video based on the karaoke performance sound, singing voice, and external image (step S04). If the karaoke performance does not end (No in step S05), the process from steps S03 to S05 is repeated while the timer is counting for the first period (No in step S06). 【0022】 When the timer finishes counting for the first period (Yes in step S06), the timer starts counting for the second period (for example, 30 seconds) (step S07). The first acquisition unit 22 acquires a selfie video of the singer U1 from the in-camera 12 (step S08), and the generation unit 23 generates the rest of the singing video based on the karaoke performance sound, singing voice, and selfie video (step S09). If the karaoke performance does not finish (No in step S10), the process from steps S08 to S10 is repeated while the timer is counting for the second period (No in step S11). 【0023】 When the timer finishes counting for the second period (Yes in step S11), the process from step S02 to step S11 is repeated while the karaoke performance continues (No in step S12). When the karaoke performance ends (Yes in S05, S10, or step S12), the singing video is completed and the processing operation of the mobile terminal 10 ends. In this way, a singing video is automatically generated in which the external video footage from the first period and the selfie video footage from the second period are arranged chronologically from the start to the end of the karaoke performance. Note that after the singing video for the first song is generated, the singing video for the second song may be generated immediately afterward. 【0024】 As described above, according to the mobile terminal 10 of the first embodiment, the rear camera 11 and the front camera 12 automatically switch during karaoke performance, selectively acquiring images of the scenery outside the car and a selfie of the singer. A singing video is generated in which the images of the scenery outside the car and the selfie are arranged chronologically in accordance with the progress of the karaoke performance. Therefore, the scenery outside the car can be filmed without pointing the front camera 12 outside the car, and the singer can sing while viewing the lyrics on the display 13 of the mobile terminal 10. A singing video is generated while switching between images of the scenery outside the car and a selfie during a drive, allowing users to preserve memories of their drive in a singing video. 【0025】 <Example 1> In the first embodiment, the external video and selfie video were switched according to the elapsed time from the start of the karaoke performance, but the external video and selfie video may be switched according to the performance section of the song. In this case, the karaoke performance data is accompanied by performance section information indicating the performance section during the karaoke performance, and the performance section information is output from the performance control unit 21 to the first acquisition unit 22. The performance section information includes, for example, the start and end positions of each performance section such as the intro section, the first A section, the first B section, the first chorus section, the interlude section, the second A section, the second B section, the second chorus section, and the outro section. 【0026】 The first acquisition unit 22 selectively acquires selfie video and external video based on performance section information. In this case, the first acquisition unit 22 alternately repeats the process of acquiring external video from the rear camera 11 during non-singing sections and acquiring selfie video from the front camera 12 during singing sections. As a result, external video and selfie video are acquired alternately as the non-singing and singing sections switch. Specifically, external video is acquired during the intro section, selfie video from the first A section to the first chorus section, external video during the interlude section, selfie video from the second A section to the second chorus section, and external video during the outro section. 【0027】 The generation unit 23 automatically generates a singing video in which, in sync with the progress of the karaoke performance of the song while driving, the video switches between footage of the scenery outside the car and a self-shot video of the singer U1 according to the performance section. For example, in the singing video shown in Figure 5, the intro section displays footage of the scenery outside the car, and the section from the first A-melody to the first chorus displays a self-shot video of the singer U1. Furthermore, the interlude section displays footage of the scenery outside the car, and the section from the second A-melody to the second chorus displays a self-shot video of the singer U1. Finally, the outro section displays footage of the scenery outside the car again. 【0028】 Referring to Figure 6, the processing operation of the mobile terminal in Modification Example 1 will be explained. Figure 6 is a flowchart showing an example of the processing operation of the mobile terminal in Modification Example 1. Note that the reference numerals from Figure 2 will be used as appropriate in this explanation. Also, the following flowchart is merely an example and can be modified as needed. 【0029】 As shown in Figure 6, when the performance control unit 21 starts karaoke performance (step S21), the performance control unit 21 outputs performance section information to the first acquisition unit 22 (step S22). If the performance section is a non-singing section (Yes in step S23), the first acquisition unit 22 acquires an external video of the scenery outside the vehicle from the rear camera 11 (step S24), and the generation unit 23 generates a singing video based on the karaoke performance sound, singing voice, and external video (step S25). If the performance section is not a non-singing section (No in step S23), the process proceeds to step S26. 【0030】 If the performance section is a singing section (Yes in step S26), the first acquisition unit 22 acquires a selfie video of the singer U1 from the front camera 12 (step S27), and the generation unit 23 generates a singing video based on the karaoke performance sound, singing voice, and selfie video (step S28). If the performance section is not a singing section (No in step S26), the process moves to step S29. While the karaoke performance continues, the processes from step S22 to step S28 are repeated (No in step S29). When the karaoke performance ends (Yes in step S29), the singing video is completed and the processing operation of the mobile terminal 10 ends. 【0031】 As described above, with the mobile device 10 of Modified Example 1, a singing video linked to the karaoke performance of a song is generated, allowing users to preserve memories of their drive in a singing video. 【0032】 <Modification 2> Furthermore, the external video feed and the selfie video feed may be switched between each section of the song. In this case, the first acquisition unit 22 alternately acquires external video feeds from the rear camera 11 during odd-numbered sections of the song, and the first acquisition unit 22 alternately acquires selfie video feeds from the front camera 12 during even-numbered sections of the song. This ensures that external video feeds and selfie video feeds are acquired alternately each time the song section changes. Specifically, external video feeds are acquired during the intro, first B section, interlude, second B section, and outro, while selfie video feeds are acquired during the first A section, first chorus, second A section, and second chorus. 【0033】 The generation unit 23 automatically generates a singing video in which, in sync with the progress of the karaoke performance of the song while driving, the video switches between footage of the scenery outside the car and a self-shot video of the singer U1 for each section of the performance. For example, in the singing video shown in Figure 7, the footage of the scenery outside the car is displayed during the intro section immediately after the start of the karaoke performance, and the self-shot video of the singer U1 is displayed during the first A section. The footage then switches alternately between the scenery outside the car and the self-shot video each time the performance section changes, until the karaoke performance ends. 【0034】 As described above, even with the mobile device 10 of the modified example 2, a singing video linked to the karaoke performance of the song is generated, allowing users to preserve memories of their drive in a singing video. 【0035】 <Second Embodiment> Next, the mobile terminal of the second embodiment will be described. Figure 8 is a functional block diagram of the mobile terminal of the second embodiment. Figure 9 is a diagram showing an example of a singing video of the second embodiment. The mobile terminal of the second embodiment differs from the mobile terminal of the first embodiment in that it displays a composite video that combines a singing video and a navigation image. Therefore, for the mobile terminal of the second embodiment, the same configuration as the mobile terminal of the first embodiment will not be described. 【0036】 As shown in Figure 8, the mobile terminal 31 is equipped with a performance control unit 41, a first acquisition unit 42, a navigation means 43, a second acquisition unit 44, and a generation unit 45, in addition to cameras 32, 33 and a microphone 34. In the second embodiment, the mobile terminal 31 has a car navigation application installed in addition to the karaoke application. When the karaoke application and the car navigation application are launched, the karaoke application and the car navigation application work together to provide the mobile terminal 31 with a function to generate singing videos associated with navigation images. 【0037】 Similar to the first embodiment, the performance control unit 41 controls the karaoke performance of a song, and the first acquisition unit 42 selectively acquires external video and selfie video during the karaoke performance. The navigation means 43 guides the vehicle's route from the starting point to the destination. In this case, the navigation means 43 is implemented by an application such as Drive Supporter (registered trademark). The navigation means 43 generates a navigation image, and the vehicle's route is guided by the navigation image. The navigation image can be any image on which the driving route and vehicle position information are overlaid on a map image or aerial photograph. 【0038】 The second acquisition unit 44 acquires navigation images from the navigation means 43. In this case, the second acquisition unit 44 continuously acquires navigation images from the start to the end of the karaoke performance of the song. The generation unit 45 generates a composite video that can display the singing video and navigation images on one screen. In this case, the composite video is generated by synthesizing the navigation images acquired by the second acquisition unit 44 with the singing video generated by the generation unit 45 in the same way as in the first embodiment. The generation unit 45 also applies various processing to the navigation images. 【0039】 As shown in Figure 9, the navigation image of the composite video includes a first icon 51 indicating the starting point of the karaoke performance of the song, a second icon 52 indicating the ending point of the karaoke performance of the song, a driving route 53 from the first icon 51 to the second icon 52, and a vehicle icon 54 indicating the vehicle's position as it moves along the driving route during the karaoke performance. Therefore, the time it takes for the vehicle icon 54 to travel from the first icon 51 to the second icon 52 is approximately the same as the karaoke performance time. Until the karaoke performance ends, the external video and selfie video are displayed alternately, and the vehicle's driving position is displayed on the map. 【0040】 The processing operation of the mobile terminal of the second embodiment will be described with reference to Figure 10. Figure 10 is a flowchart showing an example of the processing operation of the mobile terminal of the second embodiment. Note that the reference numerals from Figure 8 will be used as appropriate in this explanation. Also, the following flowchart is merely an example and can be modified as needed. 【0041】 As shown in Figure 10, when the performance control unit 41 starts karaoke performance (step S31), the process of generating a singing video and the process of acquiring navigation images are performed in parallel. The process from step S32 to step S42 of the singing video generation process is the same as the process from step S02 to step S12 in Figure 4. While the karaoke performance continues, the second acquisition unit 44 continues to acquire navigation images (No in steps S43 and S44). When the karaoke performance ends (Yes in step S44), the generation unit 45 combines the singing video and navigation images to generate a composite video (step S45). Here, the acquired navigation image includes a driving route 53 from the point where the karaoke performance started to the point where the karaoke performance ended, and a vehicle icon 54 that shows the vehicle's position moving along the driving route 53 during the karaoke performance. On the other hand, the first icon 51 that shows the point where the karaoke performance started and the second icon 52 that shows the point where the karaoke performance ended are generated by the generation unit 45. 【0042】 As described above, according to the mobile terminal 31 of the second embodiment, driving memories can be recorded in a singing video associated with the driving position in the navigation image. 【0043】 In each embodiment and each modified example, the generation unit may obtain the song title from the music data and overlay the text data indicating the song title as the title onto the singing video. 【0044】 In the second embodiment, the generation unit may obtain the name of the destination from the navigation means and overlay the text data indicating the destination as a title onto the singing video. 【0045】 In the second embodiment, the generation unit may obtain the song title X from the music data and the name of the destination Y from the navigation means, and overlay text data such as "I tried singing X while driving to Y" as the title onto the singing video. 【0046】 In addition, in each embodiment and each modified example, a mobile device may be used in which the resolution of the rear camera and the front camera are approximately the same. 【0047】 Furthermore, in the above embodiment, a function for generating singing videos may be added by installing a program on the mobile terminal. This program is stored in a storage medium. The storage medium is not particularly limited, but may be a non-transient storage medium such as an optical disc, magneto-optical disc, or flash memory. 【0048】 Furthermore, although this embodiment has been described, other embodiments may be combinations of the above embodiments and modifications, either entirely or partially. 【0049】 Furthermore, the technology of the present invention is not limited to the embodiments described above, and may be modified, substituted, or transformed in various ways without departing from the spirit of the technical idea. Moreover, if the technical idea can be realized in a different way by advances in the technology or by other derived technologies, it may be implemented by that method. Accordingly, the claims cover all embodiments that may fall within the scope of the technical idea. [Explanation of symbols] 【0050】 10, 31: Mobile devices 11, 32: Rear camera 12, 33: Front camera 14, 34: Microphone (sound collector) 21, 41: Performance Control Unit 22, 42: First acquisition section 23, 45: Generation part 43: Navigation methods 44: Second acquisition section 51: The first icon 52: The second icon 53: Driving route 54: Vehicle icon

Claims

[Claim 1] A mobile device equipped with a sound collector capable of collecting the singer's voice, an out-camera capable of capturing images of the scenery outside the vehicle, and an in-camera capable of capturing a selfie of the singer, A performance control unit that controls the karaoke performance of a song, A first acquisition unit selectively acquires external video and selfie video while karaoke is being played, It comprises a generation unit that generates a singing video based on the karaoke performance sound of the song, the singer's singing voice, the video footage from outside the car, and the selfie video, A mobile device characterized in that, during karaoke performance, the rear camera and the front camera automatically switch to selectively acquire images of the outside of the vehicle and selfie footage. [Claim 2] The performance control unit outputs performance section information indicating the performance section during karaoke performance. The mobile terminal according to claim 1, characterized in that the first acquisition unit selectively acquires selfie video and external video based on performance section information. [Claim 3] A navigation system that guides the vehicle's route from its starting point to its destination, The system includes a second acquisition unit that acquires a navigation image from the aforementioned navigation means, The mobile terminal according to claim 1 or 2, characterized in that the generation unit generates a composite video capable of displaying a singing video and a navigation image on a single screen. [Claim 4] The mobile terminal according to claim 3, characterized in that the navigation image includes the starting point of the karaoke performance of the song, the ending point of the karaoke performance of the song, the driving route from the starting point of the karaoke performance of the song to the ending point of the karaoke performance of the song, and the position of the vehicle moving along the driving route during the karaoke performance.

Images