Method for producing a video film of a journey of a motor vehicle

Abstract

A method for producing a video film of a journey of a motor vehicle is proposed, wherein a face of a driver of the motor vehicle is monitored with a first camera, wherein position point clouds of the face are recorded on the basis of first camera data of the first camera and facial expressions and / or changes in facial expressions are inferred from this and / or from the camera data (3), wherein the facial expressions and / or changes in facial expressions are stored as a first file with a first time stamp, wherein a video of the journey is recorded by at least one second camera of the motor vehicle, provided with a second time stamp and stored in a second file (4), wherein the first file and the second file are transferred to a computer,wherein the computer selects excerpts from the video (6) with the aid of the first file and a comparison of the time stamps of the first file and the second file and makes them available for the production of the video film (7).,

Description

[0001] The present invention relates to a method for producing a video film of a journey of a motor vehicle.

[0002] Driving a motor vehicle goes far beyond transportation from A to B. Especially with high-performance or classic cars, the driving experience is paramount. The sensation of acceleration, the vehicle's response to the driver's inputs, and the interaction of the engine and other components are often accompanied by fantastic visual impressions, be it a beautiful landscape or the course of a road.

[0003] It's human nature to want to capture such experiences. It's common to create videos of journeys along scenic roads. When the driver watches the video afterward, they often notice unpleasant long stretches that they didn't notice during the journey, but which make the video seem boring. One way to deal with this is to adapt the video through clever editing so that the content is reduced to the most exciting scenes. However, this is time-consuming and—if good results are to be achieved—not easy to accomplish.

[0004] Especially given that driving scenes are perceived as interesting and exciting by different people, the task of editing the video film cannot simply be left to others or to a dull automatic process.

[0005] Methods for recording a film while driving a vehicle are known from the publications DE 10 2017 200 970 B3 and DE 10 2015 008 448 A1. For example, it is known to capture and take into account the driver's emotional states.

[0006] It is therefore an object of the present invention to provide a method for producing a video film of a journey of a motor vehicle, which method does not have the disadvantages of the prior art mentioned, but instead offers as a result a highly individualized video film that personally appeals to the driver.

[0007] This object is achieved by a method for producing a video film of a journey of a motor vehicle according to claim 1

[0008] In the method according to the invention for producing a video film of a journey of a motor vehicle, the face of a driver of the motor vehicle is monitored with a first camera. Based on first camera data from the first camera, position point clouds of the face are captured. From these position point clouds and / or from the first camera data, inferences are made regarding facial expressions and / or changes in facial expressions. In particular, it is provided that facial expressions are inferred from the position point clouds, and that changes in facial expressions are inferred from the first camera data.

[0009] The facial expressions and changes in facial expressions are assigned a first timestamp and saved as the first file. For example, it is conceivable that the facial expressions and changes in facial expressions are saved with the first timestamp as a CSV file. It is also conceivable that the first camera data is saved separately as a video file.

[0010] Furthermore, it is provided that a video of the journey is recorded with at least one second camera of the motor vehicle, which is provided with a second time stamp and saved in a second file. Preferably, several videos of the journey are recorded by several second cameras of the motor vehicle, provided with the second time stamp, and saved in further second files. The video(s) of the journey include, for example, videos of the surroundings of the motor vehicle or videos of the interior of the motor vehicle.

[0011] The first and second files are transferred to a computer, which uses the first file and a comparison of the timestamps of the first and second files to select excerpts from the video and prepares them for production of the video. This advantageously makes it possible to assign the video(s) to the driver's various emotions. Based on the created link between emotion and excerpt, the excerpts can be compiled, for example, according to a narrative arc, thus creating an engaging video in a highly automated manner, without the need for tedious manual editing, while still allowing for a highly personalized selection of the excerpts.

[0012] Advantageous embodiments and further developments of the invention can be found in the dependent claims and the description with reference to the drawings.

[0013] According to a preferred embodiment of the present invention, physiological data of the driver are preferably collected using a smartwatch. The physiological data are provided with a third timestamp and saved in a third file, for example, a CSV file. The third file is then transferred to the computer, which selects the sections with the aid of the third file. Physiological data can include, for example, heart rate, body temperature, blood pressure, or skin conductivity. This significantly improves the recognition of emotions, for example, relaxation, joy, or stress. Alternatively or in addition to collecting physiological data using a smartwatch, a sensor for recording physiological data can, for example, be arranged in a steering wheel of the motor vehicle, which collects the physiological data.

[0014] According to a further preferred embodiment of the present invention, vehicle data is collected, the vehicle data being provided with a fourth time stamp and stored in a fourth file. The fourth file is transferred to the computer, and the selection of the sections is carried out with the aid of the fourth file. The vehicle data includes, for example, acceleration, speed, location, etc. By taking the vehicle data into account and, in particular, linking the vehicle data with the other collected and processed data, an even better selection of suitable sections is achieved.

[0015] A particularly preferred embodiment of the present invention is one in which the first camera is a camera of a personal terminal device. A personal terminal device can be, for example, a mobile phone, a tablet, a laptop, or the like. The position point clouds are captured by the personal terminal device. Furthermore, the personal terminal device infers facial expressions and / or changes in facial expressions. A neural network on the personal terminal device is preferably used to capture the point clouds and / or determine facial expressions. This advantageously enables the processing of highly personal data locally on a device used exclusively by the user.Furthermore, this advantageously makes it possible to carry out the method not only in a motor vehicle that has been specifically prepared for the process; the method for producing the video film is thus possible with any motor vehicle. For example, it is conceivable that the vehicle data is recorded by the personal device, assigned a fourth timestamp, stored in the fourth file, and forwarded to the computer.

[0016] According to a further preferred embodiment of the present invention, it is provided that a storyboard is selected from a plurality of storyboards. The selection of excerpts is made to match the selected storyboard. This advantageously makes it possible to provide a highly appealing video film with an emotionally charged arrangement of excerpts. For example, suspense arcs can be personalized. The storyboard is either selected by the user or driver themselves or automatically selected by a computer. It is conceivable, for example, that the plurality of storyboards are made available online, for example by a community of like-minded people. It is also conceivable that one or more storyboards are created independently.

[0017] For this purpose, it is preferably provided that a piece of music is transferred to the computer, whereby the storyboard is selected by the computer based on the piece of music, and / or whereby the storyboard is adapted based on the piece of music. For example, the storyboard can be selected based on the tempo or beat of the piece of music, or adapted to the tempo or beat of the piece of music. It is also conceivable that the storyboard is selected based on a sequence of musical passages, such as an intro, verses, choruses, etc., or adapted to these. In particular, it is provided that the excerpts are sorted based on the storyboard and preferably synchronized with the piece of music.

[0018] According to a further exemplary embodiment of the present invention, the excerpts are transferred to a video editing program. This allows for a seamless transition to further processing, such as adding video effects. In particular, this advantageously provides that the method is not carried out in a closed system, but rather that the user can further edit the selected excerpts entirely as desired.

[0019] It is particularly preferred that the computer be a cloud-based computer. This advantageously makes it possible to provide computing power externally and to use only the personal device and the smartwatch for the on-site procedure itself.

[0020] Further details, features, and advantages of the invention will become apparent from the drawings and the following description of preferred embodiments with reference to the drawings. The drawings merely illustrate an exemplary embodiment of the invention, which does not limit the inventive concept. Fig. 1 schematically illustrates a flow of a method according to an exemplary embodiment of the present invention.

[0021] Fig. Figure 1 schematically illustrates a flow of a method according to an exemplary embodiment of the present invention. The method shown is a method for easily producing a highly personalized video of a journey in a motor vehicle.

[0022] While driving, the face of a driver of the motor vehicle is monitored with a first camera. In the exemplary embodiment shown here, the first camera is a camera of a personal end device, for example a front-facing camera of a mobile phone. Based on first camera data recorded by the first camera, position point clouds of the face are recorded. Using the position point clouds, a neural network on the personal end device is used to record the driver's facial expressions while driving. Furthermore, a change in facial expression is inferred from the first camera data. This also takes place on the personal end device. Both the recognized facial expressions and the changes in facial expressions are saved together with a first timestamp as a first file and transferred to a cloud-based computer, for example via a web app.

[0023] During the journey, videos of the journey are also recorded using second cameras on the vehicle. 4 These videos can include videos of the vehicle's surroundings as well as videos of the vehicle's interior. The videos from the second cameras are given a second time stamp and also transferred to the computer as second files.

[0024] Furthermore, the driver's physiological data is collected during the journey using a smartwatch. 1 This physiological data includes, for example, heart rate, skin conductivity, and body temperature. After collection, the physiological data is sent from the smartwatch to the personal device, where it is timestamped and saved as a third file. This third file is also transferred to the computer.

[0025] Finally, vehicle data is collected, assigned a fourth timestamp, and sent to the computer as a fourth file. The vehicle data includes, for example, location, acceleration, or speed and can be collected from the vehicle itself or from a personal device.

[0026] To create the video, the user selects a piece of music. A storyboard is then generated based on the piece of music, 5 which takes into account, for example, musical passages within the piece of music and the tempo of the piece of music. It is conceivable that the user selects both the piece of music and the storyboard itself. It is particularly conceivable that storyboards, preferably with a piece of music, are created and shared in a community, for example, via an internet platform.

[0027] The computer compares the timestamps and assigns the driver's emotional states to excerpts from the videos in the second file. Based on facial expressions, changes in facial expressions, and physiological data, it is possible to determine, against the background of the vehicle data, emotional states that stand out and mark a particular moment during the journey. The computer then selects excerpts 6 that match the storyboard and provides them for the production of the video 7.

[0028] The excerpts are then sorted and synchronized to match the music piece 8, and—if necessary—adjusted in size and presentation, such as exposure. Depending on whether further editing, such as adding video effects, is required, the excerpts and the music piece are transferred to an external video editing program 9. Alternatively or additionally, the sorted and synchronized excerpts can be output with the music piece as a video film 10. List of reference symbols 1 Collecting physiological data 2 Collecting vehicle data 3 Understanding facial expressions and changing facial expressions 4 Recording the video of the trip 5 Generating the storyboard based on a piece of music 6 Selection of excerpts 7 Provision of the excerpts 8 Sorting and synchronizing the excerpts with the music piece 9 Transferring the excerpts to a video editing program 10 Creating the video film from the excerpts and the piece of music 100 motor vehicles

Claims

[1] Method for producing a video film of a journey of a motor vehicle, wherein a face of a driver of the motor vehicle is monitored with a first camera, wherein position point clouds of the face are recorded on the basis of first camera data of the first camera and facial expressions and / or changes in facial expressions are inferred from this and / or from the camera data (3), wherein the facial expressions and / or changes in facial expressions are stored as a first file with a first time stamp, wherein a video of the journey is recorded by at least one second camera of the motor vehicle, provided with a second time stamp and stored in a second file (4), wherein the first file and the second file are transferred to a computer,wherein the computer selects excerpts from the video (6) with the aid of the first file and a comparison of the time stamps of the first file and the second file and makes them available for the production of the video film (7)., [2] Method according to claim 1, characterized by that physiological data of the driver are preferably collected with the aid of a smartwatch (1), the physiological data being provided with a third time stamp and stored in a third file, the third file being transferred to the computer and the selection of the sections being carried out with the aid of the third file. [3] Method according to one of the preceding claims, characterized bythat vehicle data are collected (2), the vehicle data being provided with a fourth time stamp and stored in a fourth file, the fourth file being transferred to the computer and the selection of the sections being carried out with the aid of the fourth file. [4] Method according to one of the preceding claims, characterized by that the first camera is a camera of a personal terminal, wherein the position point clouds are captured by the personal terminal, wherein facial expressions and / or changes in facial expressions are inferred by the personal terminal, wherein a neural network on the personal terminal is preferably used to capture the point clouds and / or to determine the facial expressions. [5] Method according to claim 4 referring back to claim 2, characterized bythat the physiological data are transmitted from the smartwatch to the personal device, whereby the physiological data are provided with the third time stamp by the personal device and stored in the third file, whereby the third file is transferred from the personal device to the computer. [6] Method according to one of the preceding claims, characterized by that a storyboard is selected from a variety of storyboards, with the selection of excerpts being made to match the selected storyboard. [7] Method according to claim 6, characterized by that a piece of music is passed to the computer, the storyboard being selected by the computer based on the piece of music, and / or the storyboard being adapted and / or generated based on the piece of music (5). [8] Method according to one of claims 6 to 7, characterized bythat the excerpts are sorted according to the storyboard and preferably synchronized with the piece of music (8). [9] Method according to one of the preceding claims, characterized by that the excerpts are passed on to a video editing program (9). [10] Method according to one of the preceding claims, characterized by that the computer is a cloud-based computer.

Images