Video generation system

Abstract

To facilitate creation of a road trip movie.SOLUTION: A moving image generation server 20 acquires imaging data during a road trip by a drive recorder 11 mounted in a vehicle 10 and analyzes the video and sound of the imaging data. Then the moving image generation server 20 sets a segmentation range based on an analytic result of the video and sound and automatically generates a moving image obtained by segmenting an image of the segmentation range from the imaging data, as a road trip movie.SELECTED DRAWING: Figure 1

Description

[[Technical Field]]

[0001] The present invention relates to a moving image generation system. [[Background Art]]

[0002] Patent Document 1 describes a technique for automatically generating a road trip album by automatically recording video and audio inside and outside a vehicle based on analysis results of the facial expressions and utterances of an occupant. [[Prior Art Documents]] [[Patent Documents]]

[0003] [[Patent Document 1]] Japanese Unexamined Patent Application Publication No. 2017-194950 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] In the above-mentioned conventional technology, it is necessary to analyze the occupant's facial expressions and utterances in real time in a moving vehicle. Therefore, it is necessary to mount an analysis system with high processing capacity on the vehicle. [[Means for Solving the Problem]]

[0005] A moving image generation system that solves the above problem is configured to set a clipping range based on an analysis result of image data captured by a drive recorder, and automatically generate a moving image by clipping video of the clipping range from the captured image data. [[Effects of the Invention]]

[0006] The above-described moving image generation system has the effect of facilitating creation of a road trip movie. [[Brief Description of the Drawings]]

[0007] [Figure 1] It is a diagram schematically showing the configuration of an embodiment of the moving image generation system. [Figure 2] This is a sequence diagram showing the video generation procedure in the video generation system described above. [Modes for carrying out the invention]

[0008] Below, one embodiment of the video generation system will be described in detail with reference to Figures 1 and 2. <About the video generation system configuration> First, the configuration of the video generation system of this embodiment will be described with reference to Figure 1. The video generation system of this embodiment automatically edits videos taken by the drive recorder 11 during a road trip and provides them to the user of the vehicle 10.

[0009] As shown in Figure 1, a drive recorder 11 is installed in the vehicle 10. The drive recorder 11 includes an in-vehicle camera 12 for recording the inside of the vehicle, an out-of-vehicle camera 13 for recording the outside of the vehicle, and a microphone 14 for recording the sound inside the vehicle. The vehicle 10 is also equipped with a data storage 15 that records the video footage captured by the in-vehicle camera 12 and the out-of-vehicle camera 13 of the drive recorder 11, as well as the audio data recorded by the microphone 14. The data storage 15 may be built into the drive recorder 11 or may be installed outside the drive recorder 11.

[0010] Furthermore, the vehicle 10 is equipped with a human-machine interface (HMI) 16 and a communication module 17. The HMI 16 is an interface that connects the vehicle 10's systems to the occupants. An example of the HMI 16 is a touch panel display. The communication module 17 is an electronic component for wireless communication with the outside of the vehicle. The drive recorder 11, data storage 15, HMI 16, and communication module 17 are connected to each other via an in-vehicle network 18 so that they can communicate with one another.

[0011] The video generation system of this embodiment is configured to automatically generate a road trip movie as a video based on footage captured by the drive recorder 11 of the vehicle 10. The video generation system includes a video generation server 20, which is a server device for video generation. The video generation server 20 includes a processor 21 that performs various processing for video generation, and a storage device 22 that stores the video generation program and various data used for video generation. The video generation server 20 also includes a communication module 23 for data communication with the vehicle 10.

[0012] <About creating road trip movies> Next, with reference to Figure 2, the automatic generation of road trip movies in this embodiment will be described. The video generation system of this embodiment creates a road trip movie based on the shooting data of the drive recorder 11 during the road trip. The shooting data of the drive recorder 11 includes video footage shot by the in-vehicle camera 12 and the exterior camera 13, and audio captured by the microphone 14. The shooting data also includes information on the shooting location and time of the video acquired from the in-vehicle GPS device.

[0013] When creating a road trip movie, the user of vehicle 10 enters video settings on the HMI 16 (S1). Video settings are the specifications for the video to be generated. Items in the video settings include the length of the video, the title of the song to be used as background music, and the title of the video. In addition, the user specifies the people, time period, and location to be included in the generated video. Furthermore, the user enters the date and time of the road trip in the video settings. After that, when the user of vehicle 10 performs a send command operation on the HMI 16 (S2), the video setting information entered by the user in S1 is sent to the video generation server 20 along with the shooting data during the road trip stored in the data storage 15 (S3).

[0014] The video generation server 20 performs video and audio analysis of the received shooting data (S4), and sets the video extraction range based on the analysis results (S5). Next, the video generation server 20 edits the video from the shooting data during the road trip by extracting and stitching together the video from the selected range, and generates road trip movie video data (S6). After that, the video generation server 20 transmits the generated video data to the vehicle 10 (S7). The vehicle 10 saves the received video data to the data storage 15 (S8).

[0015] <Regarding setting the cropping area> The setting of the cropping range performed by the video generation server 20 in S5 of Figure 2 will be explained. Based on the video and audio analysis results, the video generation server 20 extracts the period during which the occupants of the vehicle 10 are in an excited mood and sets that period as the cropping range. Whether or not the occupants are in an excited mood is determined by analyzing their facial expressions and voices. The music being played in the vehicle may be used as a criterion for determining whether or not the occupants are in an excited mood. Specifically, the video generation server 20 analyzes the volume, style, or title of the music being played in the vehicle and uses the analysis results to determine whether or not the occupants are in an excited mood.

[0016] In addition to the above, the video generation server 20 sets the following video durations as the extraction range: The video generation server 20, upon confirming from the video analysis results that a crew member is performing a predetermined gesture, sets the period starting from the time the gesture was performed as the extraction range.

[0017] The video generation server 20 sets the video clipping range to include the time and location specified by the user in the video settings. Examples of such times include the departure of a road trip and the arrival at the destination. The video generation server 20 sets these clipping ranges based on the shooting location and shooting time information included in the video data.

[0018] ·The video generation server 20 sets, as the clipping range, the range in which the person specified in the video settings is included in the video. The video generation server identifies the person in the video through face recognition. Furthermore, the video generation server 20 sets the clipping range such that the duration of the generated video matches the time set in the video settings. Furthermore, when generating a video, the video generation server 20 inserts the music specified by the user in the video settings as background music.

[0019] <Operational Effects of Embodiment> The operation of the video generation system of the present embodiment will be described. In the present embodiment, the data captured by the drive recorder 11 during a road trip is transmitted to the video generation server 20 outside the vehicle. The video generation server 20 sets the clipping range based on the analysis results of the video and audio in the captured data, and generates a road trip movie by clipping the video of the clipping range from the captured data.

[0020] If such video and audio analysis for video generation is performed in real time during capturing by the drive recorder 11, it is necessary to mount an analysis device with high processing capacity on the vehicle 10. In the case of the present embodiment, the video and audio analysis for video generation is performed by the video generation server 20 outside the vehicle after capturing. Automatic generation of road trip movies is possible without mounting an analysis device with high processing capacity on the vehicle 10.

[0021] As described above, in the present embodiment, automatic generation of a road trip movie using the data captured by the drive recorder 11 is enabled without mounting an analysis device with high processing capacity on the vehicle 10. Therefore, the video generation system of the present embodiment has the effect of facilitating creation of a road trip movie.

[0022] <Other Embodiments> The present embodiment can be implemented with the following modifications. The present embodiment and the following modified examples can be implemented in combination with each other within a range where there is no technical contradiction.

[0023] The music playing in the car while the footage was being shot may be used as background music for the generated video. In this case, the video generation server 20 identifies the music playing in the car by analyzing the sound of the footage. The video generation server 20 then inserts the identified music as background music into the generated video. At this time, the climactic parts of the image may be synchronized with the chorus of the music.

[0024] The video settings and transmission instructions in S1 and S2 of Figure 2 may be performed on a user terminal (PC, etc.) outside the vehicle. In this case, the recorded data from the drive recorder 11 is moved or copied to the user terminal using a memory card, etc., and then transmitted from the user terminal to the video generation server 20. Furthermore, the processing performed by the video generation server 20 may be performed on the user terminal. In this case, the user terminal constitutes the video generation system.

[0025] In the above embodiment, the video generation process performed by the video generation server 20 may be performed in the vehicle 10. In this case as well, it is not necessary to perform real-time analysis of video and audio for video generation. Therefore, the processing power required for the processing unit installed in the vehicle 10 for video generation can be kept low. [Explanation of Symbols]

[0026] 10 vehicles 11. Dashcam 12 In-car cameras 13. Exterior car camera 14 Microphones 15 Data Storage 16. Human-Machine Interface (HMI) 17. Communication Module 18 In-vehicle network 20 Video Generation Servers 21 processors 22 Data Storage 23 Communication Module

Claims

[Claim 1] A video generation system that sets a cropping range based on the analysis results of dashcam recording data and automatically generates a video by cropping the video within the cropping range from the recording data, wherein the cropping range is set based on the analysis results of the volume of music played in the car during the recording of the recording data, and the music played in the car during the recording of the recording data is used as background music for the video.

Images