A method for generating an image of a motor vehicle with a person by a support system, a computer program product, a non transitory computer-readable storage

Abstract

A method for generating a composite image 14 of a motor vehicle 12 with a person (e.g. vehicle driver) 16 comprises the steps of capturing a first image 30 of the person inside the vehicle 12 using fi

Description

[0001] The present invention relates to the field of automobiles. More specifically, the present invention relates to a method for generating an image of a motor vehicle with a person by a support system of the motor vehicle. Furthermore, the present invention relates to a corresponding computer program product, a corresponding non-transitory computer-readable storage medium, as well as to a corresponding support system. BACKGROUND INFORMATION

[0002] In case someone wants to take a picture of him / herself driving a motor vehicle, with the motor vehicle / motorcycle in the picture as well this picture usually is either taken by someone outside of the motor vehicle or by traffic cameras. It is hard for the driver to take a picture for him / herself. The best one can do is to take from inside the motor vehicle or on the motorcycle, but the exterior of the motor vehicle will be missing in the picture. SUMMARY OF THE INVENTION

[0003] It is an object of the present invention to provide a method, a corresponding computer program product, a corresponding non-transitory computer-readable storage medium, as well as a corresponding support system, by which a picture of a person inside the motor vehicle with the motor vehicle is generated in an improved manner.

[0004] This object is solved by a method, a corresponding computer program product, a corresponding non-transitory computer-readable storage medium, as well as a corresponding support system according to the independent claims. Advantageous embodiments are presented in the dependent claims.

[0005] One aspect of the invention relates to a method for generating an image of a motor vehicle with a person by a support system of the motor vehicle. A first image of the person inside of the motor vehicle is captured by a first camera of the support system inside of the motor vehicle. At least a second image of the motor vehicle is captured by a second camera of the support system outside of the motor vehicle. The first image and the second image are fused to generate the image by an electronic computing device of the support system.

[0006] Therefore, a method is provided for taking an image of the person with the motor vehicle in an improved manner. In particular, a method for generating / creating selfies of driving with the motor vehicle in the picture is provided.

[0007] The proposed solution takes the picture of a person, for example a driver of the motor vehicle, using interior driving monitoring camera (DMC), and the potential image segmentation step can be added to further extract human from the image. Rendering the high-fidelity exterior of the motor vehicle is provided. The viewing angle and distance may be derived from the extrinsic parameters of the interior camera, so that it is consistent with the picture of the driver. The image, the difference in contrast, brightness, view, noise level and furthermore is synthesized and needs to be taken into consideration.

[0008] According to an embodiment in the first image a segmentation algorithm for segmentizing the person is performed.

[0009] In another embodiment a viewing angle of the second camera is taken into consideration for generating the image.

[0010] In another embodiment a distance of the motor vehicle in the second image is taken into consideration for generating the image.

[0011] In another embodiment at least a corresponding brightness in the first and second image and / or a corresponding contrast in the first and second image is taken into consideration for generating the image.

[0012] In another embodiment a 360 degree image is generated by at least two second cameras and the 360 degree image is fused with the first image.

[0013] In particular, the present invention is a computer-implemented method. Therefore, another aspect of the invention relates to a computer program product comprising program code means for performing a method according to the preceding aspect.

[0014] A still further aspect of the invention relates to a non-transitory computer-readable storage medium comprising at least the computer program product according to the preceding aspect.

[0015] Furthermore, the present invention relates to a support system for generating an image of a motor vehicle with a person, comprising at least a first camera, a second camera and an electronic computing device, wherein the support system is configured for performing a method according to the preceding aspect. In particular, the method is performed by the support system.

[0016] A still further aspect of the invention relates to a motor vehicle comprising at least the support system according to the preceding aspect.

[0017] Advantageous embodiments of the method are to be regarded as advantageous embodiments of the computer program product, the non-transitory computer-readable storage medium, the support system, as well as the motor vehicle. The support system as well as the motor vehicle therefore comprises means for performing the method.

[0018] A computing unit may in particular be understood as a data processing device, which comprises processing circuitry. The computing unit can therefore in particular process data to perform computing operations. This may also include operations to perform indexed accesses to a data structure, for example a look-up table, LUT.

[0019] In particular, the computing unit may include one or more computers, one or more microcontrollers, and / or one or more integrated circuits, for example, one or more application-specific integrated circuits, ASIC, one or more field-programmable gate arrays, FPGA, and / or one or more systems on a chip, SoC. The computing unit may also include one or more processors, for example one or more microprocessors, one or more central processing units, CPU, one or more graphics processing units, GPU, and / or one or more signal processors, in particular one or more digital signal processors, DSP. The computing unit may also include a physical or a virtual cluster of computers or other of said units.

[0020] In various embodiments, the computing unit includes one or more hardware and / or software interfaces and / or one or more memory units.

[0021] A memory unit may be implemented as a volatile data memory, for example a dynamic random access memory, DRAM, or a static random access memory, SRAM, or as a non-volatile data memory, for example a read-only memory, ROM, a programmable read-only memory, PROM, an erasable programmable read-only memory, EPROM, an electrically erasable programmable read-only memory, EEPROM, a flash memory or flash EEPROM, a ferroelectric random access memory, FRAM, a magnetoresistive random access memory, MRAM, or a phase-change random access memory, PCRAM.

[0022] Further advantages, features, and details of the invention derive from the following description of a preferred embodiment as well as from the drawing. The features and feature combinations previously mentioned in the description as well as the features and feature combinations mentioned in the following description of the figure and / or shown in the figure alone can be employed not only in the respectively indicated combination but also in any other combination or taken alone without leaving the scope of the invention. BRIEF DESCRIPTION OF THE DRAWING

[0023] The novel features and characteristic of the disclosure are set forth in the appended claims. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate exemplary embodiments and together with the description, serve to explain the disclosed principles. The same numbers are used throughout the figures to reference like features and components. Some embodiments of system and / or methods in accordance with embodiments of the present subject matter are now described below, by way of example only, and with reference to the accompanying figures.

[0024] The drawing show in:

[0025] Fig. 1 a schematic block diagram according to an embodiment of the method.

[0026] In the figure the same elements or elements having the same function are indicated by the same reference signs. DETAILED DESCRIPTION

[0027] In the present document, the word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or implementation of the present subject matter described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

[0028] While the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof have been shown by way of example in the drawing and will be described in detail below. It should be understood, however, that it is not intended to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the scope of the disclosure.

[0029] The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion so that a setup, device or method that comprises a list of components or steps does not include only those components or steps but may include other components or steps not expressly listed or inherent to such setup or device or method. In other words, one or more elements in a system or apparatus preceded by “comprises” or “comprise” does not or do not, without more constraints, preclude the existence of other elements or additional elements in the system or method.

[0030] In the following detailed description of the embodiment of the disclosure, reference is made to the accompanying drawing that forms part hereof, and in which is shown by way of illustration a specific embodiment in which the disclosure may be practiced. This embodiment is described in sufficient detail to enable those skilled in the art to practice the disclosure, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present disclosure. The following description is, therefore, not to be taken in a limiting sense.

[0031] Fig. 1 shows a schematic block diagram according to an embodiment of a support system 10 for a motor vehicle 12. The support system 10 is configured for providing an image 14 with the motor vehicle 12 and a person 16. The support system 10 is configured for generating the image 14. The support system 10 therefore comprises at least a first camera 18, a second camera 20, 22, 24, 26, and at least one electronic computing device 28.

[0032] According to an embodiment a first image 30 of the person 16 inside of the motor vehicle 12 is captured by the first camera 18. At least a second image 32 of the motor vehicle 12 is captured by the second camera 20, 22, 24, 26 of the support system 10. The first image 30 and the second image 32 is fused in order to generate the image 14 by the electronic computing device 28.

[0033] In particular, as shown in Fig. 1, the motor vehicle 12 may comprise a capturing system 34. The capturing system 34 may comprise at least the second cameras 20, 22, 24, 26. In particular, the second cameras 20, 22, 24, 26 may be configured as fisheye cameras. For example, a first second camera 20 may be configured as a front fisheye camera. A second second camera 22 may be configured as a right fisheye camera. A third second camera 24 may be configured as a rear fisheye camera and a fourth second camera 26 may be configured as a left fisheye camera. Therefore, a typical surround view system is provided.

[0034] The first camera 18 may be for example a driver monitor, or any interior camera. The first image 30 may be provided with a camera position and a camera angle for the electronic computing device 28. In particular, based on camera pose of the first image 30, the electronic computing device 28 may generate a view that is looking into the windshield of the motor vehicle 10.

[0035] Furthermore, some post-processing, such as segmentation and cropping may be provided by the electronic computing device 28.

[0036] In particular, in the first image 30 a segmentation algorithm for segmenting the person 16 is performed. Furthermore, a viewing angle of the second camera 20, 22, 24, 26 is taken into consideration for generating the image 14. In particular, this is shown with the second image 32. In the second image 32 the motor vehicle 12 is shown in different viewing angles, for example by top view or different side views.

[0037] Furthermore, a distance of the motor vehicle 12 in the second image 32 is taken into consideration for generating the image 14.

[0038] In another embodiment at least a corresponding brightness in the first and second image 16, 32 and / or a corresponding contrast in the first and second image 16, 32 is taken into consideration for generating the image.

[0039] Furthermore, it is shown in Fig. 1, a 360 degree image is generated by at least two second cameras 20, 22, 24, 26 and the 360 degree image is fused with the first image 30.

[0040] In particular Fig. 1 shows that the first image 30 for example from the driver is taken by the driver monitoring camera (DMC) where a potential image segmentation step can be added to further extract the human from the first image 30. Rendering the high-fidelity exterior of the motor vehicle 12 is provided in the electronic computing device 28. The viewing angle and distance may be derived from the extrinsic parameters of the interior camera, in particular the first camera 18, so that it coincides with the picture of the driver. The images 30, 32 are synthesized, wherein the difference contrast, brightness, view, noise level etc. is taken into consideration. signs: Support system Motor vehicle Image Person First camera First second camera Second second camera Third second camera Fourth second camera Electronic computing device First image Second image Capturing system

Claims

1. A method for generating an image (14) of a motor vehicle (12) with a person (16) by a support system (10) of the motor vehicle (12), comprising the steps of:- capturing a first image (30) of the person (16) inside of the motor vehicle (12) by a first camera (18) of the support system (10) inside of the motor vehicle (12);- capturing at least a second image (32) of the motor vehicle (12) by a second camera (20, 22, 24, 26) of the support system (10) outside of the motor vehicle (12); and- fusing the first image (30) and the second image (32) to generate the image (14) by an electronic computing device (28) of the support system (10).

2. The method according to claim 1, characterized in thatin the first image (30) a segmentation algorithm for segmenting the person (16) is performed.

3. The method according to claim 1 or 2, characterized in thata viewing angle of the second camera (20, 22, 24, 26) is taken into consideration for generating the image (14).

4. The method according to any one of claims 1 to 3, characterized in thata distance of the motor vehicle (12) in the second image (32) is taken into consideration for generating the image (14).

5. The method according to any one of claims 1 to 4, characterized in thatat least a corresponding brightness in the first and second image (30, 32) and / or a corresponding contrast in the first and second image (30, 32) is taken into consideration for generating the image (14).

6. The method according to any one of claims 1 to 5, characterized in thata 360° image is generated by at least two second cameras (20, 22, 24, 26) and the 360° image is fused with the first image (30).

7. A computer program product comprising program code means for performing the method according to any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium comprising at least the computer program product according to claim 7.

9. A support system (10) for generating an image (14) of a motor vehicle (12) with a person (16), comprising at least a first camera (18), a second camera (20, 22, 24, 26) and an electronic computing device (28), wherein the support system (10) is configured for performing a method according to any one of claims 1 to 6.

Images